转换GeoLite2数据以与xtables geoip一起使用

Question

如果此处或其他地方已经涵盖了这一点，我表示歉意。我读了回到2016年的帖子。

我的debian系统停止更新xtables geoip数据库。经过调查，它发现这是因为Maxmind放弃了对传统GeoIP数据库的支持。我已经为GeoLite2数据库安装和配置Maxmind的geoipupdate程序，并在crontab中每周安排它。

此时我很难过。 geoipupdate返回.mmdb数据库。 debian提供的脚本无法使用它，它将.CSV文件转换为/ usr / share / xt_geoip / LE和/ usr / share / xt_geoip / BE中的国家/地区代码文件。

debian软件包xtables-addons尚未更新以处理这种情况。

将非常感谢您提供帮助或指向解决方案的指针。目前我仍然使用最后一个有效的数据库，现在已经超过六个月了。

Answer 1

看看GeoLite2xtables： - https://github.com/mschmitt/GeoLite2xtables

你可以下载一个zip（或git克隆）。它具有遗留GeoLite CSV的示例工作流程（shell命令）（可能是您在2019年1月初停止工作的那个）和GeoLite2 CSV（您可以使用它）。

Answer 2

您也可以从xtable-addon的项目（直接或从xtables-addons-common软件包的sid版本）下载源代码并获取脚本的更新版本。

https://sourceforge.net/projects/xtables-addons/files/Xtables-addons/

请参阅以下askubuntu答案：https://askubuntu.com/questions/1117669/xtables-addons-issues-with-maxmind-geolite2

Answer 3

我最终写完了这个脚本，现在每周运行一次。到目前为止（三个月）它似乎令人满意。

cat update-geoip.sh

#!/bin/bash -e

GEOLITE_URL="https://geolite.maxmind.com/download/geoip/database/GeoLite2-Country-CSV.zip"
GEOLITE_ZIP="GeoLite2-Country-CSV.zip"
COUNTRY_URL="http://download.geonames.org/export/dump/countryInfo.txt"

#
# Switch to the GeoIP directory if not already there
#
echo "--> cd /usr/share/xt_geoip"
cd /usr/share/xt_geoip

#
# Remove anything remaining from previous failed runs
#
# Note:  DO NOT delete the existing BE and LE subfolders at this
#        time.  If the download fails the result would be no
#        database at all.
#
echo "--> rm -r GeoLite2*"
rm -r -f GeoLite2*
echo "--> rm countryInfo.txt"
rm -f countryInfo.txt
echo "--> rm GeoIP-legacy.csv"
rm -f GeoIP-legacy.csv

#
# Get the GeoIP ZIP file
#
echo "--> wget --no-check-certificate $GEOLITE_URL"
wget --no-check-certificate $GEOLITE_URL

#
# See if the ZIP file now exists
#
if [ ! -e $GEOLITE_ZIP ]; then
  echo "--> GeoIP ZIP file did not download"
  echo "--> Send email to root and stop here"
  /usr/sbin/sendmail root << EOM
From: Update_GeoIP
To: root
Subject: GeoIP update failed

GeoIP update failed.
Unable to download GeoIP ZIP file
$GEOLITE_ZIP
EOM
  exit
fi

#
# Unzip the ZIP file
#
echo "--> unzip $GEOLITE_ZIP"
unzip $GEOLITE_ZIP

#
# Delete the ZIP file
#
#echo "--> rm $GEOLITE_ZIP"
rm $GEOLITE_ZIP

#
# Move the received data directory to a standard name
#
echo "--> mv GeoLite2-Country-CSV_* GeoLite2"
mv GeoLite2-Country-CSV_* GeoLite2

#
# See if the critical GeoIP data files now exist
#
if [ ! -e "GeoLite2/GeoLite2-Country-Blocks-IPv4.csv" ] ||
   [ ! -e "GeoLite2/GeoLite2-Country-Blocks-IPv6.csv" ]; then
  echo "--> GeoIP data files are missing"
  echo "--> Send email to root and stop here"
  /usr/sbin/sendmail root << EOM
From: Update_GeoIP
To: root
Subject: GeoIP update failed

GeoIP update failed.
GeoIP data file(s) are missing
GeoLite2/GeoLite2-Country-Blocks-IPv4.csv
GeoLite2/GeoLite2-Country-Blocks-IPv6.csv
EOM
  exit
fi

#
# Get the country info data file
#
echo "--> wget --no-check-certificate $COUNTRY_URL"
wget --no-check-certificate $COUNTRY_URL

#
# See if the country info data file now exists
#
if [ ! -e "countryInfo.txt" ]; then
  echo "--> Country info data file did not download"
  echo "--> Send email to root and stop here"
  /usr/sbin/sendmail root << EOM
From: Update_GeoIP
To: root
Subject: GeoIP update failed

GeoIP update failed.
Unable to download country info data file
$COUNTRY_URL
EOM
  exit
fi

#
# Build an old format data file from the new format data files
#
echo "--> cat ./GeoLite2/GeoLite2-Country-Blocks-IPv{4,6}.csv | ./convert_GeoLite2.pl ./countryInfo.txt > /usr/share/xt_geoip/GeoIP-legacy.csv"
cat ./GeoLite2/GeoLite2-Country-Blocks-IPv{4,6}.csv | ./convert_GeoLite2.pl ./countryInfo.txt > /usr/share/xt_geoip/GeoIP-legacy.csv

#
# Delete the downloaded data files
#
echo "--> rm -r GeoLite2"
rm -r GeoLite2
echo "--> rm countryInfo.txt"
rm country_Info.txt

#
# Preserve the old BE and LE directories just in case
#
echo "--> rm -r -f LastBE LastLE"
rm -r -f LastBE LastLE
echo "--> mv BE LastBE"
mv BE LastBE
echo "--> mv LE LastLE"
mv LE LastLE

#
# Convert the generated database to the xtables GeoIP format
#
echo "--> /usr/lib/xtables-addons/xt_geoip_build -D /usr/share/xt_geoip ./GeoIP-legacy.csv"
/usr/lib/xtables-addons/xt_geoip_build -D /usr/share/xt_geoip ./GeoIP-legacy.csv

#
# Delete the remaining data files
#
echo "--> rm countryInfo.txt"
rm countryInfo.txt
echo "--> rm GeoIP-legacy.csv"
rm GeoIP-legacy.csv

#
# Notify root that the update succeeded
#
echo "--> Send notification email to root"
/usr/sbin/sendmail root << EOM
From: Update_GeoIP
To: root
Subject: Weekly update of xtables GeoIP completed

Weekly update of xtables GeoIP database successful.
EOM
echo "xtables GeoIP database update completed"