Nginx GeoLite2-City.tar.gz file is empty

deltahf · Sep 22, 2023

CentOS Version: CentOS 7 64bit

Centmin Mod Version Installed: 130.00beta01

Nginx Version Installed: 1.25.2

PHP Version Installed: 8.2.10

MariaDB MySQL Version Installed: 10.6.15

When was last time updated Centmin Mod code base ? : just now

Persistent Config:
Code (Text):
PHP_PGO='y'
ENABLE_MARIADBTENFIVEUPGRADE='y'
ENABLE_MARIADBTENSIXUPGRADE='y'
MM_LICENSE_KEY='redacted'
NGINX_GEOIPTWOLITE='y'
NGXDYNAMIC_GEOIPTWOLITE='y'
I was following the guide to Enable GeoIP 2 Lite Nginx Module Support. When I added the reference to the new geoip2.conf file in my site's vhost.conf file, nginx -t returned the following error:
Code (Text):
# nginx -t
nginx: [emerg] MMDB_open("/usr/share/GeoIP/GeoLite2-City.mmdb") failed - Error opening the specified MaxMind DB file in /usr/local/nginx/conf/geoip2.conf:9
nginx: configuration file /usr/local/nginx/conf/nginx.conf test failed
When I checked that directory, sure enough, that file did not exist. The GeoLite2-City.tar.gz file was there, but it is empty and the corresponding mmbd file does not exist:
Code (Text):
# ll /usr/share/GeoIP/
total 66416
-rw-r--r-- 1 root root 20539238 Jan  8  2019 GeoIPCity.dat
-rw-r--r-- 1 root root  1242574 Jan  8  2019 GeoIP.dat
-rw-r--r-- 1 root root  1242574 Aug  8  2019 GeoIP-initial.dat
lrwxrwxrwx 1 root root       19 Jul  6 01:35 GeoIPv6.dat -> GeoIPv6-initial.dat
-rw-r--r-- 1 root root  2322773 Aug  8  2019 GeoIPv6-initial.dat
-rw-r--r-- 1 root root  8263188 Sep 21 14:27 GeoLite2-ASN.mmdb
-rw-r--r-- 1 root root  4589217 Sep 21 14:30 GeoLite2-ASN.tar.gz
-rw-r--r-- 1 root root        0 Sep 22 02:36 GeoLite2-City.tar.gz
-rw-r--r-- 1 root root  6122406 Sep 21 16:43 GeoLite2-Country.mmdb
-rw-r--r-- 1 root root  3126894 Sep 21 16:52 GeoLite2-Country.tar.gz
-rw-r--r-- 1 root root 20539238 Jan  8  2019 GeoLiteCity.dat
Is this a bug?

eva2000 · Sep 22, 2023

after you set in persistent config file /etc/centminmod/custom_config.inc the variables

Code (Text):

NGINX_GEOIPTWOLITE='y'
NGXDYNAMIC_GEOIPTWOLITE='y'

did you run centmin.sh menu option 4 to recompile Nginx with GeoIP2 Lite Nginx module support? It's at this stage that the GeoLite2 databases are downloaded.

If so, then you should of logged a nginx upgrade log that you can inspect and troubleshoot for https://community.centminmod.com/threads/how-to-troubleshoot-nginx-installs-upgrades.17778/ to see what happened at GeoLite2 download stage

example of latest nginx_upgrade.log at /root/centminlogs/centminmod_130.00beta01.b384_140923-113253_nginx_upgrade.log

Code (Text):

ls -lahrt /root/centminlogs/| grep nginx_upgrade | tail -1
-rw-r--r--   1 root root 1.8M Sep 14 11:34 centminmod_130.00beta01.b384_140923-113253_nginx_upgrade.log

check GeoLite2 downloads in log using case insensitive grep with 3 line context

Code (Text):

grep -C3 -in 'geolite' /root/centminlogs/centminmod_130.00beta01.b384_140923-113253_nginx_upgrade.log

Gives output below, i masked my MAXMIND API key with = MY_MAXMIND_KEY

Code (Text):

grep -C3 -in 'geolite' /root/centminlogs/centminmod_130.00beta01.b384_140923-113253_nginx_upgrade.log

4203-
4204-/usr/share/GeoIP /svr-setup /svr-setup/nginx-1.25.2 ~
4205-------------------------------------------------------
4206:GeoLite2 City database download ...
4207-------------------------------------------------------
4208-HTTP/2 200
4209:--2023-09-14 11:33:41--  https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-City&license_key=MY_MAXMIND_KEY&suffix=tar.gz
4210-Resolving download.maxmind.com (download.maxmind.com)... 104.16.37.47, 104.16.38.47
4211-Connecting to download.maxmind.com (download.maxmind.com)|104.16.37.47|:443... connected.
4212-HTTP request sent, awaiting response... 200 OK
4213-Length: 35372350 (34M) [application/gzip]
4214:Saving to: ‘/usr/share/GeoIP/GeoLite2-City.tar.gz’
4215-
4216-     0K .......... .......... .......... .......... ..........  0% 45.0M 1s
4217-    50K .......... .......... .......... .......... ..........  0% 28.6M 1s
--
4905- 34450K .......... .......... .......... .......... .......... 99%  288M 0s
4906- 34500K .......... .......... .......... .......... ...       100%  263M=0.6s
4907-
4908:2023-09-14 11:33:42 (54.6 MB/s) - ‘/usr/share/GeoIP/GeoLite2-City.tar.gz’ saved [35372350/35372350]
4909-
4910:GeoLite2-City_20230912/
4911:GeoLite2-City_20230912/LICENSE.txt
4912:GeoLite2-City_20230912/COPYRIGHT.txt
4913:GeoLite2-City_20230912/GeoLite2-City.mmdb
4914:GeoLite2-City_20230912/README.txt
4915-------------------------------------------------------
4916:GeoLite2 Country database download ...
4917-------------------------------------------------------
4918-HTTP/2 200
4919:--2023-09-14 11:33:43--  https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-Country&license_key=MY_MAXMIND_KEY&suffix=tar.gz
4920-Resolving download.maxmind.com (download.maxmind.com)... 104.16.37.47, 104.16.38.47
4921-Connecting to download.maxmind.com (download.maxmind.com)|104.16.37.47|:443... connected.
4922-HTTP request sent, awaiting response... 200 OK
4923-Length: 3186469 (3.0M) [application/gzip]
4924:Saving to: ‘/usr/share/GeoIP/GeoLite2-Country.tar.gz’
4925-
4926-     0K .......... .......... .......... .......... ..........  1% 47.8M 0s
4927-    50K .......... .......... .......... .......... ..........  3%  142M 0s
--
4987-  3050K .......... .......... .......... .......... .......... 99%  208M 0s
4988-  3100K .......... .                                          100%  137M=0.07s
4989-
4990:2023-09-14 11:33:43 (41.6 MB/s) - ‘/usr/share/GeoIP/GeoLite2-Country.tar.gz’ saved [3186469/3186469]
4991-
4992:GeoLite2-Country_20230912/
4993:GeoLite2-Country_20230912/LICENSE.txt
4994:GeoLite2-Country_20230912/COPYRIGHT.txt
4995:GeoLite2-Country_20230912/GeoLite2-Country.mmdb
4996-------------------------------------------------------
4997:GeoLite2 ASN database download ...
4998-------------------------------------------------------
4999-HTTP/2 200
5000:--2023-09-14 11:33:44--  https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-ASN&license_key=MY_MAXMIND_KEY&suffix=tar.gz
5001-Resolving download.maxmind.com (download.maxmind.com)... 104.16.37.47, 104.16.38.47
5002-Connecting to download.maxmind.com (download.maxmind.com)|104.16.37.47|:443... connected.
5003-HTTP request sent, awaiting response... 200 OK
5004-Length: 4600544 (4.4M) [application/gzip]
5005:Saving to: ‘/usr/share/GeoIP/GeoLite2-ASN.tar.gz’
5006-
5007-     0K .......... .......... .......... .......... ..........  1% 10.6M 0s
5008-    50K .......... .......... .......... .......... ..........  2% 5.73M 1s
--
5095-  4400K .......... .......... .......... .......... .......... 99%  246M 0s
5096-  4450K .......... .......... .......... .......... ..        100% 32.8M=0.1s
5097-
5098:2023-09-14 11:33:44 (35.1 MB/s) - ‘/usr/share/GeoIP/GeoLite2-ASN.tar.gz’ saved [4600544/4600544]
5099-
5100:GeoLite2-ASN_20230915/
5101:GeoLite2-ASN_20230915/GeoLite2-ASN.mmdb
5102:GeoLite2-ASN_20230915/LICENSE.txt
5103:GeoLite2-ASN_20230915/COPYRIGHT.txt
5104-------------------------------------------------------
5105-Check GeoIP2 Lite Databases
5106-------------------------------------------------------
5107-
5108:-rw-r--r-- 1 root root 7.9M Sep 14 09:50 /usr/share/GeoIP/GeoLite2-ASN.mmdb
5109:-rw-r--r-- 1 root root  69M Sep 11 18:24 /usr/share/GeoIP/GeoLite2-City.mmdb
5110:-rw-r--r-- 1 root root 6.0M Sep 11 17:26 /usr/share/GeoIP/GeoLite2-Country.mmdb
5111-
5112-
5113-  mmdblookup --file /path/to/file.mmdb --ip 1.2.3.4 [path to lookup]
--
5152-  will be shown.
5153-
5154-
5155:/usr/local/nginx-dep/bin/mmdblookup --file /usr/share/GeoIP/GeoLite2-Country.mmdb --ip 8.8.8.8 country names en
5156-
5157-  "United States" <utf8_string>
5158-
5159-
5160:/usr/local/nginx-dep/bin/mmdblookup --file /usr/share/GeoIP/GeoLite2-ASN.mmdb --ip 8.8.8.8 autonomous_system_number
5161-
5162-  15169 <uint32>
5163-
5164-
5165:/usr/local/nginx-dep/bin/mmdblookup --file /usr/share/GeoIP/GeoLite2-ASN.mmdb --ip 8.8.8.8 autonomous_system_organization
5166-
5167-  "GOOGLE" <utf8_string>
5168-

looks like maxmind.com is behind Cloudflare, so maybe your server got blocked?

Code (Text):

dig maxmind.com NS +short
josh.ns.cloudflare.com.
kim.ns.cloudflare.com.

You can try running the GeoLite2 database update cronjob manually to see if it redownloads it

Code (Text):

/usr/local/src/centminmod/tools/geoip2db-update.sh

deltahf · Sep 23, 2023

Yes, I recompiled Nginx after making those changes. I also just recompiled it again, but that file was still empty.
Code (Text):
# grep -C3 -in 'geolite' centminmod_130.00beta01.b388_230923-033316_nginx_upgrade.log
The line above returned no results.

This is an OVH server, that dig command returned the same Cloudflare name servers.

eva2000 said: ↑

You can try running the GeoLite2 database update cronjob manually to see if it redownloads it
Click to expand...

Yes, running this script manually worked. It downloaded and unzipped the files, no problem.

Strange!

eva2000 · Sep 23, 2023

deltahf said: ↑

Yes, I recompiled Nginx after making those changes. I also just recompiled it again, but that file was still empty.
Click to expand...

strange, the nginx routine for this has a curl HTTP status check for download URLs before wget downloading. So if there's no wget downloads, then the curl HTTP status check must of failed. If you search nginx upgrade log for text 'GeoLite2 City database download ...' that would be around the time the download should run

Could be at Cloudflare blocking the curl HTTP status check?

But /usr/local/src/centminmod/tools/geoip2db-update.sh also has same check/routine.

edit: ok slightly different in Nginx version check
Code (Text):
  curl -${ipv_forceopt}Is --connect-timeout 30 --max-time 30 $maxmind_city_url | grep 'HTTP\/' | grep '200'
  GEOIPTWOCITYDATA_CURLCHECK=$?
versus geoip2db-update.sh check
Code (Text):
GEOIPTWOCITYDATA_CURLCHECK=$(curl -${ipv_forceopt}Is --connect-timeout 30 --max-time 30 "$maxmind_city_url" | grep 'HTTP\/' | grep '200' >/dev/null 2>&1; echo $?)  

deltahf · Sep 25, 2023

I can't find any reference to GeoLite2 in the Nginx upgrade log. I can share the whole log if you think that might be interesting.
Code (Text):
# grep -C3 -in 'GeoLite2' centminmod_130.00beta01.b388_230923-033316_nginx_upgrade.log
#
On a whim I asked Phind (GPT-4) to analyze the different commands. I'm sure you're aware of this, but thought the response was interesting. It definitely looks like a network/blocking issue, but I've got no idea what it could be.
The second version of the command could succeed when the first one doesn't due to how the shell handles the exit status of pipelines.

In the first version, the shell will only consider the exit status of the last command in the pipeline (the second `grep '200'`). If the `curl` command fails (for example, due to a network issue or an invalid URL), the shell won't take this into account when setting the value of `$?`, and instead only consider the exit status of `grep '200'`. This means that even if `curl` fails, `GEOIPTWOCITYDATA_CURLCHECK` might still be set to 0 (indicating success), because `grep '200'` successfully executed (even though it didn't receive any input from `curl`) [Source 14].

In contrast, in the second version, the entire pipeline is enclosed in a command substitution, and the exit status of the whole pipeline is explicitly printed with `echo $?`. This means that if any command in the pipeline fails, the exit status of the whole pipeline will be non-zero, and this value will be captured by the command substitution and assigned to `GEOIPTWOCITYDATA_CURLCHECK`. This makes the second version more robust in handling errors in the `curl` command [Source 14].

Here's an example to illustrate this point:
Code (Text):
# Suppose the curl command fails (e.g., due to a network issue)
$ curl -Is --connect-timeout 30 --max-time 30 "http://invalid-url" | grep 'HTTP\/' | grep '200'
$ echo $?
0

# In this case, even though curl fails, the exit status is still 0, because grep '200' successfully executed (even though it didn't receive any input)

# Now, if we use the second version of the command:
$ echo $(curl -Is --connect-timeout 30 --max-time 30 "http://invalid-url" | grep 'HTTP\/' | grep '200'; echo $?)
7

# In this case, the exit status is 7, indicating that the curl command failed
Therefore, the second version of the command can successfully detect errors in the `curl` command, while the first version cannot [Source 14].
Click to expand...

eva2000 · Sep 25, 2023

deltahf said: ↑

I can't find any reference to GeoLite2 in the Nginx upgrade log. I can share the whole log if you think that might be interesting.
Click to expand...

Use keyword = 'geolite'

deltahf said: ↑

On a whim I asked Phind (GPT-4) to analyze the different commands. I'm sure you're aware of this, but thought the response was interesting. It definitely looks like a network/blocking issue, but I've got no idea what it could be.
Click to expand...

haha nice So I could change my check. But even then it should work as I'm grepping for HTTP status = 200. So if curl failed, the HTTP status !=200 so grep would fail like intended anyway.

But that still means network issue on your end, probably Maxmind's Cloudflare Firewall rules blocking your server?

Even then, in Nginx there's a check to fallback to locally hosted version of the file too if GEOIPTWOCITYDATA_CURLCHECK != 0
Code (Text):
  if [[ "$GEOIPTWOCITYDATA_CURLCHECK" = '0' ]]; then
    wget -${ipv_forceopt} $maxmind_city_url -O /usr/share/GeoIP/GeoLite2-City.tar.gz
  else
    maxmind_city_url='${LOCALCENTMINMOD_MIRROR}/centminmodparts/geoip2-lite/GeoLite2-City.tar.gz'
    wget -${ipv_forceopt} $maxmind_city_url -O /usr/share/GeoIP/GeoLite2-City.tar.gz
  fi

Log in / Register

Forums

Discover Centmin Mod today

Nginx GeoLite2-City.tar.gz file is empty

deltahf Premium Member Premium Member

eva2000 Administrator Staff Member

deltahf Premium Member Premium Member

eva2000 Administrator Staff Member

deltahf Premium Member Premium Member

eva2000 Administrator Staff Member

.

Log in / Register

Useful Searches

Discover Centmin Mod today

Nginx GeoLite2-City.tar.gz file is empty

deltahf Premium Member Premium Member

eva2000 Administrator Staff Member

deltahf Premium Member Premium Member

eva2000 Administrator Staff Member

deltahf Premium Member Premium Member

eva2000 Administrator Staff Member

.