Want to subscribe to topics you're interested in?
Become a Member

Nginx GeoLite2-City.tar.gz file is empty

Discussion in 'Bug Reports' started by deltahf, Sep 22, 2023.

Tags:
  1. deltahf

    deltahf Premium Member Premium Member

    576
    259
    63
    Jun 8, 2014
    Ratings:
    +475
    Local Time:
    10:58 AM
    • CentOS Version: CentOS 7 64bit
    • Centmin Mod Version Installed: 130.00beta01
    • Nginx Version Installed: 1.25.2
    • PHP Version Installed: 8.2.10
    • MariaDB MySQL Version Installed: 10.6.15
    • When was last time updated Centmin Mod code base ? : just now
    • Persistent Config:
    Code (Text):
    PHP_PGO='y'
    ENABLE_MARIADBTENFIVEUPGRADE='y'
    ENABLE_MARIADBTENSIXUPGRADE='y'
    MM_LICENSE_KEY='redacted'
    NGINX_GEOIPTWOLITE='y'
    NGXDYNAMIC_GEOIPTWOLITE='y'
    


    I was following the guide to Enable GeoIP 2 Lite Nginx Module Support. When I added the reference to the new geoip2.conf file in my site's vhost.conf file, nginx -t returned the following error:


    Code (Text):
    # nginx -t
    nginx: [emerg] MMDB_open("/usr/share/GeoIP/GeoLite2-City.mmdb") failed - Error opening the specified MaxMind DB file in /usr/local/nginx/conf/geoip2.conf:9
    nginx: configuration file /usr/local/nginx/conf/nginx.conf test failed
    


    When I checked that directory, sure enough, that file did not exist. The GeoLite2-City.tar.gz file was there, but it is empty and the corresponding mmbd file does not exist:

    Code (Text):
    # ll /usr/share/GeoIP/
    total 66416
    -rw-r--r-- 1 root root 20539238 Jan  8  2019 GeoIPCity.dat
    -rw-r--r-- 1 root root  1242574 Jan  8  2019 GeoIP.dat
    -rw-r--r-- 1 root root  1242574 Aug  8  2019 GeoIP-initial.dat
    lrwxrwxrwx 1 root root       19 Jul  6 01:35 GeoIPv6.dat -> GeoIPv6-initial.dat
    -rw-r--r-- 1 root root  2322773 Aug  8  2019 GeoIPv6-initial.dat
    -rw-r--r-- 1 root root  8263188 Sep 21 14:27 GeoLite2-ASN.mmdb
    -rw-r--r-- 1 root root  4589217 Sep 21 14:30 GeoLite2-ASN.tar.gz
    -rw-r--r-- 1 root root        0 Sep 22 02:36 GeoLite2-City.tar.gz
    -rw-r--r-- 1 root root  6122406 Sep 21 16:43 GeoLite2-Country.mmdb
    -rw-r--r-- 1 root root  3126894 Sep 21 16:52 GeoLite2-Country.tar.gz
    -rw-r--r-- 1 root root 20539238 Jan  8  2019 GeoLiteCity.dat
    


    Is this a bug?
     
  2. eva2000

    eva2000 Administrator Staff Member

    51,743
    11,946
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,438
    Local Time:
    12:58 AM
    Nginx 1.25.x
    MariaDB 10.x
    after you set in persistent config file /etc/centminmod/custom_config.inc the variables
    Code (Text):
    NGINX_GEOIPTWOLITE='y'
    NGXDYNAMIC_GEOIPTWOLITE='y'
    

    did you run centmin.sh menu option 4 to recompile Nginx with GeoIP2 Lite Nginx module support? It's at this stage that the GeoLite2 databases are downloaded.

    If so, then you should of logged a nginx upgrade log that you can inspect and troubleshoot for https://community.centminmod.com/threads/how-to-troubleshoot-nginx-installs-upgrades.17778/ to see what happened at GeoLite2 download stage

    example of latest nginx_upgrade.log at /root/centminlogs/centminmod_130.00beta01.b384_140923-113253_nginx_upgrade.log
    Code (Text):
    ls -lahrt /root/centminlogs/| grep nginx_upgrade | tail -1
    -rw-r--r--   1 root root 1.8M Sep 14 11:34 centminmod_130.00beta01.b384_140923-113253_nginx_upgrade.log
    

    check GeoLite2 downloads in log using case insensitive grep with 3 line context
    Code (Text):
    grep -C3 -in 'geolite' /root/centminlogs/centminmod_130.00beta01.b384_140923-113253_nginx_upgrade.log
    

    Gives output below, i masked my MAXMIND API key with = MY_MAXMIND_KEY
    Code (Text):
    grep -C3 -in 'geolite' /root/centminlogs/centminmod_130.00beta01.b384_140923-113253_nginx_upgrade.log
    
    4203-
    4204-/usr/share/GeoIP /svr-setup /svr-setup/nginx-1.25.2 ~
    4205-------------------------------------------------------
    4206:GeoLite2 City database download ...
    4207-------------------------------------------------------
    4208-HTTP/2 200
    4209:--2023-09-14 11:33:41--  https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-City&license_key=MY_MAXMIND_KEY&suffix=tar.gz
    4210-Resolving download.maxmind.com (download.maxmind.com)... 104.16.37.47, 104.16.38.47
    4211-Connecting to download.maxmind.com (download.maxmind.com)|104.16.37.47|:443... connected.
    4212-HTTP request sent, awaiting response... 200 OK
    4213-Length: 35372350 (34M) [application/gzip]
    4214:Saving to: ‘/usr/share/GeoIP/GeoLite2-City.tar.gz’
    4215-
    4216-     0K .......... .......... .......... .......... ..........  0% 45.0M 1s
    4217-    50K .......... .......... .......... .......... ..........  0% 28.6M 1s
    --
    4905- 34450K .......... .......... .......... .......... .......... 99%  288M 0s
    4906- 34500K .......... .......... .......... .......... ...       100%  263M=0.6s
    4907-
    4908:2023-09-14 11:33:42 (54.6 MB/s) - ‘/usr/share/GeoIP/GeoLite2-City.tar.gz’ saved [35372350/35372350]
    4909-
    4910:GeoLite2-City_20230912/
    4911:GeoLite2-City_20230912/LICENSE.txt
    4912:GeoLite2-City_20230912/COPYRIGHT.txt
    4913:GeoLite2-City_20230912/GeoLite2-City.mmdb
    4914:GeoLite2-City_20230912/README.txt
    4915-------------------------------------------------------
    4916:GeoLite2 Country database download ...
    4917-------------------------------------------------------
    4918-HTTP/2 200
    4919:--2023-09-14 11:33:43--  https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-Country&license_key=MY_MAXMIND_KEY&suffix=tar.gz
    4920-Resolving download.maxmind.com (download.maxmind.com)... 104.16.37.47, 104.16.38.47
    4921-Connecting to download.maxmind.com (download.maxmind.com)|104.16.37.47|:443... connected.
    4922-HTTP request sent, awaiting response... 200 OK
    4923-Length: 3186469 (3.0M) [application/gzip]
    4924:Saving to: ‘/usr/share/GeoIP/GeoLite2-Country.tar.gz’
    4925-
    4926-     0K .......... .......... .......... .......... ..........  1% 47.8M 0s
    4927-    50K .......... .......... .......... .......... ..........  3%  142M 0s
    --
    4987-  3050K .......... .......... .......... .......... .......... 99%  208M 0s
    4988-  3100K .......... .                                          100%  137M=0.07s
    4989-
    4990:2023-09-14 11:33:43 (41.6 MB/s) - ‘/usr/share/GeoIP/GeoLite2-Country.tar.gz’ saved [3186469/3186469]
    4991-
    4992:GeoLite2-Country_20230912/
    4993:GeoLite2-Country_20230912/LICENSE.txt
    4994:GeoLite2-Country_20230912/COPYRIGHT.txt
    4995:GeoLite2-Country_20230912/GeoLite2-Country.mmdb
    4996-------------------------------------------------------
    4997:GeoLite2 ASN database download ...
    4998-------------------------------------------------------
    4999-HTTP/2 200
    5000:--2023-09-14 11:33:44--  https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-ASN&license_key=MY_MAXMIND_KEY&suffix=tar.gz
    5001-Resolving download.maxmind.com (download.maxmind.com)... 104.16.37.47, 104.16.38.47
    5002-Connecting to download.maxmind.com (download.maxmind.com)|104.16.37.47|:443... connected.
    5003-HTTP request sent, awaiting response... 200 OK
    5004-Length: 4600544 (4.4M) [application/gzip]
    5005:Saving to: ‘/usr/share/GeoIP/GeoLite2-ASN.tar.gz’
    5006-
    5007-     0K .......... .......... .......... .......... ..........  1% 10.6M 0s
    5008-    50K .......... .......... .......... .......... ..........  2% 5.73M 1s
    --
    5095-  4400K .......... .......... .......... .......... .......... 99%  246M 0s
    5096-  4450K .......... .......... .......... .......... ..        100% 32.8M=0.1s
    5097-
    5098:2023-09-14 11:33:44 (35.1 MB/s) - ‘/usr/share/GeoIP/GeoLite2-ASN.tar.gz’ saved [4600544/4600544]
    5099-
    5100:GeoLite2-ASN_20230915/
    5101:GeoLite2-ASN_20230915/GeoLite2-ASN.mmdb
    5102:GeoLite2-ASN_20230915/LICENSE.txt
    5103:GeoLite2-ASN_20230915/COPYRIGHT.txt
    5104-------------------------------------------------------
    5105-Check GeoIP2 Lite Databases
    5106-------------------------------------------------------
    5107-
    5108:-rw-r--r-- 1 root root 7.9M Sep 14 09:50 /usr/share/GeoIP/GeoLite2-ASN.mmdb
    5109:-rw-r--r-- 1 root root  69M Sep 11 18:24 /usr/share/GeoIP/GeoLite2-City.mmdb
    5110:-rw-r--r-- 1 root root 6.0M Sep 11 17:26 /usr/share/GeoIP/GeoLite2-Country.mmdb
    5111-
    5112-
    5113-  mmdblookup --file /path/to/file.mmdb --ip 1.2.3.4 [path to lookup]
    --
    5152-  will be shown.
    5153-
    5154-
    5155:/usr/local/nginx-dep/bin/mmdblookup --file /usr/share/GeoIP/GeoLite2-Country.mmdb --ip 8.8.8.8 country names en
    5156-
    5157-  "United States" <utf8_string>
    5158-
    5159-
    5160:/usr/local/nginx-dep/bin/mmdblookup --file /usr/share/GeoIP/GeoLite2-ASN.mmdb --ip 8.8.8.8 autonomous_system_number
    5161-
    5162-  15169 <uint32>
    5163-
    5164-
    5165:/usr/local/nginx-dep/bin/mmdblookup --file /usr/share/GeoIP/GeoLite2-ASN.mmdb --ip 8.8.8.8 autonomous_system_organization
    5166-
    5167-  "GOOGLE" <utf8_string>
    5168-
    

    looks like maxmind.com is behind Cloudflare, so maybe your server got blocked?
    Code (Text):
    dig maxmind.com NS +short
    josh.ns.cloudflare.com.
    kim.ns.cloudflare.com.
    

    You can try running the GeoLite2 database update cronjob manually to see if it redownloads it
    Code (Text):
    /usr/local/src/centminmod/tools/geoip2db-update.sh
    
     
  3. deltahf

    deltahf Premium Member Premium Member

    576
    259
    63
    Jun 8, 2014
    Ratings:
    +475
    Local Time:
    10:58 AM
    Yes, I recompiled Nginx after making those changes. I also just recompiled it again, but that file was still empty.

    Code (Text):
    # grep -C3 -in 'geolite' centminmod_130.00beta01.b388_230923-033316_nginx_upgrade.log
    


    The line above returned no results.

    This is an OVH server, that dig command returned the same Cloudflare name servers.

    Yes, running this script manually worked. It downloaded and unzipped the files, no problem.

    Strange!
     
  4. eva2000

    eva2000 Administrator Staff Member

    51,743
    11,946
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,438
    Local Time:
    12:58 AM
    Nginx 1.25.x
    MariaDB 10.x
    strange, the nginx routine for this has a curl HTTP status check for download URLs before wget downloading. So if there's no wget downloads, then the curl HTTP status check must of failed. If you search nginx upgrade log for text 'GeoLite2 City database download ...' that would be around the time the download should run

    Could be at Cloudflare blocking the curl HTTP status check?

    But /usr/local/src/centminmod/tools/geoip2db-update.sh also has same check/routine.

    edit: ok slightly different in Nginx version check
    Code (Text):
      curl -${ipv_forceopt}Is --connect-timeout 30 --max-time 30 $maxmind_city_url | grep 'HTTP\/' | grep '200'
      GEOIPTWOCITYDATA_CURLCHECK=$?
    

    versus geoip2db-update.sh check
    Code (Text):
    GEOIPTWOCITYDATA_CURLCHECK=$(curl -${ipv_forceopt}Is --connect-timeout 30 --max-time 30 "$maxmind_city_url" | grep 'HTTP\/' | grep '200' >/dev/null 2>&1; echo $?)  
    
     
  5. deltahf

    deltahf Premium Member Premium Member

    576
    259
    63
    Jun 8, 2014
    Ratings:
    +475
    Local Time:
    10:58 AM
    I can't find any reference to GeoLite2 in the Nginx upgrade log. I can share the whole log if you think that might be interesting.

    Code (Text):
    # grep -C3 -in 'GeoLite2' centminmod_130.00beta01.b388_230923-033316_nginx_upgrade.log
    #
    


    On a whim I asked Phind (GPT-4) to analyze the different commands. I'm sure you're aware of this, but thought the response was interesting. :) It definitely looks like a network/blocking issue, but I've got no idea what it could be.

     
  6. eva2000

    eva2000 Administrator Staff Member

    51,743
    11,946
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,438
    Local Time:
    12:58 AM
    Nginx 1.25.x
    MariaDB 10.x
    Use keyword = 'geolite' :)

    haha nice :D So I could change my check. But even then it should work as I'm grepping for HTTP status = 200. So if curl failed, the HTTP status !=200 so grep would fail like intended anyway.

    But that still means network issue on your end, probably Maxmind's Cloudflare Firewall rules blocking your server?

    Even then, in Nginx there's a check to fallback to locally hosted version of the file too if GEOIPTWOCITYDATA_CURLCHECK != 0
    Code (Text):
      if [[ "$GEOIPTWOCITYDATA_CURLCHECK" = '0' ]]; then
        wget -${ipv_forceopt} $maxmind_city_url -O /usr/share/GeoIP/GeoLite2-City.tar.gz
      else
        maxmind_city_url='${LOCALCENTMINMOD_MIRROR}/centminmodparts/geoip2-lite/GeoLite2-City.tar.gz'
        wget -${ipv_forceopt} $maxmind_city_url -O /usr/share/GeoIP/GeoLite2-City.tar.gz
      fi