Beta Branch prep work for GeoIP2 Lite nginx module support 123.09beta01

eva2000 · May 9, 2018

prep work for GeoIP2 Lite nginx module support 123.09beta01

- prep work for MaxMind GeoIP2 Lite database support via nginx GeoIP2 nginx module leev/ngx_http_geoip2_module due to eventual deprecation and shutdown of legacy GeoIP database updates and downloads after January 19th, 2019 GeoLite Legacy Downloadable Databases « MaxMind Developer Site
- update adds support for downloading MaxMind GeoIP2 Lite databases for country and city downloads GeoLite2 Free Downloadable Databases « MaxMind Developer Site while still also downloading the legacy GeoIP databases from GeoLite Legacy Downloadable Databases « MaxMind Developer Site. Updated routine will only overwrite existing legacy GeoIP databases if a precheck of the download urls are considered working = HTTP 200 status code. When MaxMind eventually discontinues the download ability of legacy GeoIP database, the existing downloaded database will remain intact just there won't be any more updates from MaxMind and need to use the GeoIP2 Lite database instead
- fresh Centmin Mod 123.09beta01 installs will have updated entries in /usr/local/nginx/conf/geoip.conf which are commented out by default for GeoIP2 Lite configuration. One will define the geoip2 /usr/share/GeoIP/GeoLite2-City.mmdb context and other will define the geoip2 /usr/share/GeoIP/GeoLite2-Country.mmdb context wrapped in {}. For existing Centmin Mod installs, you would need to add the missing configuration settings into /usr/local/nginx/conf/geoip.conf as per contents shown at centminmod/centminmod as end users may have customised their /usr/local/nginx/conf/geoip.conf file and overwriting the contents of end users customised file would not be a good idea.
- to enable nginx GeoIP2 module support in Centmin Mod 123.09beta01 Nginx builds, need to add 2 variables into persistent config file at /etc/centminmod/custom_config.inc before recompiling nginx via centmin.sh menu option 4
Code (Text):
NGINX_GEOIPTWOLITE='y'
NGXDYNAMIC_GEOIPTWOLITE='y'
Continue reading...

123.09beta01 branch

Branch: centminmod/centminmod

Commit History: centminmod/centminmod

eva2000 · May 9, 2018

Example

nginx -V
nginx version: nginx/1.13.12
built by gcc 7.2.1 20170829 (Red Hat 7.2.1-1) (GCC)
built with OpenSSL 1.1.0h 27 Mar 2018
TLS SNI support enabled
configure arguments: --with-ld-opt='-L/usr/local/lib -ljemalloc -Wl,-z,relro -Wl,-rpath,/usr/local/lib' --with-cc-opt='-I/usr/local/include -m64 -march=native -g -O3 -fstack-protector-strong -flto --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wimplicit-fallthrough=0 -fcode-hoisting -Wp,-D_FORTIFY_SOURCE=2 -Wno-deprecated-declarations -gsplit-dwarf' --sbin-path=/usr/local/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --with-compat --with-http_stub_status_module --with-http_secure_link_module --with-http_flv_module --with-http_mp4_module --add-module=../nginx-rtmp-module --add-dynamic-module=../nginx-module-vts --with-libatomic --with-http_gzip_static_module --add-dynamic-module=../ngx_brotli --add-dynamic-module=../ngx_http_geoip2_module --add-dynamic-module=../incubator-pagespeed-ngx-1.13.35.2-stable --with-http_sub_module --with-http_addition_module --with-http_image_filter_module=dynamic --with-http_geoip_module --with-stream_geoip_module --with-stream_realip_module --with-stream_ssl_preread_module --with-threads --with-stream=dynamic --with-stream_ssl_module --with-http_realip_module --add-dynamic-module=../ngx-fancyindex-0.4.2 --add-module=../ngx_cache_purge-2.4.2 --add-module=../ngx_devel_kit-0.3.0 --add-dynamic-module=../set-misc-nginx-module-0.32 --add-dynamic-module=../echo-nginx-module-0.61 --add-module=../redis2-nginx-module-0.15 --add-module=../ngx_http_redis-0.3.7 --add-module=../memc-nginx-module-0.18 --add-module=../srcache-nginx-module-0.31 --add-dynamic-module=../headers-more-nginx-module-0.33 --with-pcre=../pcre-8.42 --with-pcre-jit --with-zlib=../zlib-cloudflare-1.3.0 --with-http_ssl_module --with-http_v2_module --with-openssl=../openssl-1.1.0h --with-openssl-opt='enable-ec_nistp_64_gcc_128'
Click to expand...

dynamic modules include file /usr/local/nginx/conf/dynamic-modules.conf
Code (Text):
# place custom load_module lines in this dynamic-modules-includes.conf
# file so that they persistent i.e. for manually dropped in dynamic modules
include /usr/local/nginx/conf/dynamic-modules-includes.conf;
load_module "modules/ngx_http_image_filter_module.so";
load_module "modules/ngx_http_headers_more_filter_module.so";
load_module "modules/ngx_http_set_misc_module.so";
load_module "modules/ngx_http_echo_module.so";
load_module "modules/ngx_http_fancyindex_module.so";
load_module "modules/ngx_http_vhost_traffic_status_module.so";
load_module "modules/ngx_pagespeed.so";
load_module "modules/ngx_http_brotli_filter_module.so";
load_module "modules/ngx_http_brotli_static_module.so";
load_module "modules/ngx_stream_module.so";
load_module "modules/ngx_http_geoip2_module.so";
dynamic module directory at /usr/local/nginx/modules
Code (Text):
-rwxr-xr-x  1 root root  20K May  9 12:56 ngx_http_brotli_filter_module.so
-rwxr-xr-x  1 root root  14K May  9 12:56 ngx_http_brotli_static_module.so
-rwxr-xr-x  1 root root  63K May  9 12:56 ngx_http_echo_module.so
-rwxr-xr-x  1 root root 150K May  9 12:49 ngx_http_echo_module.so.old
-rwxr-xr-x  1 root root  29K May  9 12:56 ngx_http_fancyindex_module.so
-rwxr-xr-x  1 root root  47K May  9 12:49 ngx_http_fancyindex_module.so.old
-rwxr-xr-x  1 root root  19K May  9 12:56 ngx_http_geoip2_module.so
-rwxr-xr-x  1 root root  32K May  9 12:49 ngx_http_geoip2_module.so.old
-rwxr-xr-x  1 root root  42K May  9 12:56 ngx_http_headers_more_filter_module.so
-rwxr-xr-x  1 root root  76K May  9 12:49 ngx_http_headers_more_filter_module.so.old
-rwxr-xr-x  1 root root  28K May  9 12:56 ngx_http_image_filter_module.so
-rwxr-xr-x  1 root root  49K May  9 12:49 ngx_http_image_filter_module.so.old
-rwxr-xr-x  1 root root  40K May  9 12:56 ngx_http_set_misc_module.so
-rwxr-xr-x  1 root root 155K May  9 12:49 ngx_http_set_misc_module.so.old
-rwxr-xr-x  1 root root 234K May  9 12:56 ngx_http_vhost_traffic_status_module.so
-rwxr-xr-x  1 root root  15M May  9 12:56 ngx_pagespeed.so
-rwxr-xr-x  1 root root  18K May  9 12:56 ngx_stream_geoip2_module.so
-rwxr-xr-x  1 root root  24K May  9 12:49 ngx_stream_geoip2_module.so.old
-rwxr-xr-x  1 root root 198K May  9 12:56 ngx_stream_module.so
-rwxr-xr-x  1 root root 382K May  9 12:49 ngx_stream_module.so.old
contents of /usr/local/nginx/conf/geoip.conf with geoip2 configs uncommented without hash in front
Code (Text):
# SET the path to the .dat file used for determining the visitors country from the IP-address ###
geoip_country /usr/share/GeoIP/GeoIP.dat;

# SET the path to the .dat file used for determining the visitors country from the IP-address ###
geoip_city /usr/share/GeoIP/GeoIPCity.dat;

# GeoIP2 Lite databases
  geoip2 /usr/share/GeoIP/GeoLite2-Country.mmdb {
    $geoip2_data_country_code country iso_code;
    $geoip2_data_country_name country names en;
  }

  geoip2 /usr/share/GeoIP/GeoLite2-City.mmdb {
    $geoip2_data_city_name city names en;
    $geoip2_data_postal_code postal code;
    $geoip2_data_latitude location latitude;
    $geoip2_data_longitude location longitude;
    $geoip2_data_state_name subdivisions 0 names en;
    $geoip2_data_state_code subdivisions 0 iso_code;
  }
GeoIP2 nginx module compiled and loaded as a nginx dynamic module
Code (Text):
lsof | grep geoip2
nginx       381            root  mem       REG         182,160625      18480   2243997 /usr/local/nginx/modules/ngx_http_geoip2_module.so
nginx       382           nginx  mem       REG         182,160625      18480   2243997 /usr/local/nginx/modules/ngx_http_geoip2_module.so
scheduler   382   396     nginx  mem       REG         182,160625      18480   2243997 /usr/local/nginx/modules/ngx_http_geoip2_module.so
nginx       383           nginx  mem       REG         182,160625      18480   2243997 /usr/local/nginx/modules/ngx_http_geoip2_module.so
scheduler   383   389     nginx  mem       REG         182,160625      18480   2243997 /usr/local/nginx/modules/ngx_http_geoip2_module.so

eva2000 · May 9, 2018

Using libmaxmindb tool test against IP 8.8.8.8

Code (Text):

/usr/local/bin/mmdblookup --file /usr/share/GeoIP/GeoLite2-Country.mmdb --ip 8.8.8.8 country names en

  "United States" <utf8_string>

Code (Text):

/usr/local/bin/mmdblookup --file /usr/share/GeoIP/GeoLite2-Country.mmdb --ip 8.8.8.8 country iso_code   

  "US" <utf8_string>

though GeoIP2 Lite City database doesn't have field for city name ?

Code (Text):

mmdblookup --file /usr/share/GeoIP/GeoLite2-City.mmdb --ip 8.8.8.8 

  {
    "continent": 
      {
        "code": 
          "NA" <utf8_string>
        "geoname_id": 
          6255149 <uint32>
        "names": 
          {
            "de": 
              "Nordamerika" <utf8_string>
            "en": 
              "North America" <utf8_string>
            "es": 
              "Norteamérica" <utf8_string>
            "fr": 
              "Amérique du Nord" <utf8_string>
            "ja": 
              "北アメリカ" <utf8_string>
            "pt-BR": 
              "América do Norte" <utf8_string>
            "ru": 
              "Северная Америка" <utf8_string>
            "zh-CN": 
              "北美洲" <utf8_string>
          }
      }
    "country": 
      {
        "geoname_id": 
          6252001 <uint32>
        "iso_code": 
          "US" <utf8_string>
        "names": 
          {
            "de": 
              "USA" <utf8_string>
            "en": 
              "United States" <utf8_string>
            "es": 
              "Estados Unidos" <utf8_string>
            "fr": 
              "États-Unis" <utf8_string>
            "ja": 
              "アメリカ合衆国" <utf8_string>
            "pt-BR": 
              "Estados Unidos" <utf8_string>
            "ru": 
              "США" <utf8_string>
            "zh-CN": 
              "美国" <utf8_string>
          }
      }
    "location": 
      {
        "accuracy_radius": 
          1000 <uint16>
        "latitude": 
          37.751000 <double>
        "longitude": 
          -97.822000 <double>
      }
    "registered_country": 
      {
        "geoname_id": 
          6252001 <uint32>
        "iso_code": 
          "US" <utf8_string>
        "names": 
          {
            "de": 
              "USA" <utf8_string>
            "en": 
              "United States" <utf8_string>
            "es": 
              "Estados Unidos" <utf8_string>
            "fr": 
              "États-Unis" <utf8_string>
            "ja": 
              "アメリカ合衆国" <utf8_string>
            "pt-BR": 
              "Estados Unidos" <utf8_string>
            "ru": 
              "США" <utf8_string>
            "zh-CN": 
              "美国" <utf8_string>
          }
      }
  }

eva2000 · May 9, 2018

Specifics GeoLite Legacy Discontinuation Notice | MaxMind Support Center

We have decided to discontinue our GeoLite Legacy databases effective January 2, 2019. We will stop updating the databases before that date (see “General Information” below), and the databases will no longer be available for download on January 2, 2019. This was a difficult decision for us and we have created this page to provide more information and answer any immediate questions you may have.
Click to expand...

GeoLite Legacy Customers
If you are using the GeoLite Legacy databases, you will need to switch to the GeoLite2 databases and update your integrations before January 1, 2019 to ensure your continued access to geolocation data. Like the GeoLite Legacy databases, the GeoLite2 databases are free. However, you will need to update your GeoLite Legacy integrations to work with GeoLite2 databases.

Another option is to switch to the paid GeoIP Legacy equivalent of the database you are using. In that way, you would be able to continue to use your existing integration(s). Please note, however, that since these are also legacy databases, we will discontinue these databases at some point in the future.

Update (4/11/2018)
As part of our announcement to discontinue the GeoLite Legacy databases, we originally mentioned that we would be removing latitude and longitude coordinates from the GeoLite2 databases in 2019. After careful consideration, taking into account customer feedback, we have decided against removing latitude and longitude coordinates from the GeoLite2 databases. We are in the process of reviewing coordinates used in all of our GeoLite2 and GeoIP databases to ensure there is no risk of misuse. (Please remember to use the accuracy radius if displaying coordinates on a map.)
Click to expand...

Why should I switch?
GeoLite2 databases are free, more contemporary versions of the GeoLite Legacy geolocation databases. They are comparable to, but less accurate than, MaxMind’s GeoIP2 databases. They are also IPv6 compatible and offer commercial redistribution licensing through the GeoLite2 Developer’s Package.The GeoLite2 Country and City databases are updated on the first Tuesday of each month. The GeoLite2 ASN database is updated weekly on Tuesdays.

You can get more information about the GeoLite2 Developer’s Package here.

Additional Resources

For additional information about the GeoLite2 database, please visit the GeoLite2 web page.

If you have questions about the GeoLite2 databases or GeoIP APIs, please see stackoverflow’s GeoIP questions and answers. (MaxMind does not provide official support for the free GeoLite2 databases.)

If you are a GeoLite2 Developer’s Package customer and need assistance or are not sure what to do next, please let us know at support@maxmind.com.

Click to expand...

eva2000 · May 10, 2018

Latest result working with GeoIP2 Lite and PHP-FPM fastcgi_param passed variables for PHP. Added variable to detect if IP address is in European Union (EU)

eva2000 · Jan 9, 2019

True to their word, maxmind removed GeoIP legacy database downloads after January 2, 2019 so GeoIP2 lite database support needs renewed attention

We have decided to discontinue our GeoLite Legacy databases effective January 2, 2019. We will stop updating the databases before that date (see “General Information” below), and the databases will no longer be available for download on January 2, 2019. This was a difficult decision for us and we have created this page to provide more information and answer any immediate questions you may have.
Click to expand...

For Centmin Mod use at least I have uploaded a copy of the old maxmind GeoIP legacy databases here. Though they won't have up to date data entries since maxmind no longer updates them.

wmtech · Jan 10, 2019

eva2000 said: ↑

For Centmin Mod use at least I have uploaded a copy of the old maxmind GeoIP legacy databases here. Though they won't have up to date data entries since maxmind no longer updates them.
Click to expand...

Those databases have not been updated for a very long time (1 year). Everybody should have switched to the new databases by now.

But thanks for providing a download option for very late adopters.

rdan · Jan 10, 2019

wmtech said: ↑

Everybody should have switched to the new databases by now.
Click to expand...

Which database?

wmtech · Jan 10, 2019

rdan said: ↑

Which database?
Click to expand...

GeoIP2 Lite

eva2000 · Jan 10, 2019

wmtech said: ↑

Those databases have not been updated for a very long time (1 year). Everybody should have switched to the new databases by now.

But thanks for providing a download option for very late adopters.
Click to expand...

Yeah tricky part is for nginx / php-fpm and geoip/geoip2 usage is the PHP and nginx variables aren't 100% match between them - so switching between the 2 can break web app/php and nginx's usage of GeoIP. I'll have to do more testing to make sure such a switch closely matches the variables that web app scripts/php expect. While some web app/php scripts don't even support GeoIP2 databases - only GeoIP legacy.

eva2000 · Apr 2, 2019

Checking to see if geoip2 nginx module works with latest Nginx 1.15.10 - built as a dynamic nginx module

nginx -V
nginx version: nginx/1.15.10 (020419-123438-centos7-kvm)
built by gcc 8.2.1 20180905 (Red Hat 8.2.1-3) (GCC)
built with OpenSSL 1.1.1b 26 Feb 2019
TLS SNI support enabled
configure arguments: --with-ld-opt='-Wl,-E -L/usr/local/zlib-cf/lib -L/usr/local/lib -ljemalloc -Wl,-z,relro -Wl,-rpath,/usr/local/zlib-cf/lib:/usr/local/lib' --with-cc-opt='-I/usr/local/zlib-cf/include -I/usr/local/include -m64 -march=x86-64 -mavx -mavx2 -mpclmul -msse4 -msse4.1 -msse4.2 -DTCP_FASTOPEN=23 -g -O3 -fstack-protector-strong -flto --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wimplicit-fallthrough=0 -fcode-hoisting -Wp,-D_FORTIFY_SOURCE=2' --sbin-path=/usr/local/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --build=020419-123438-centos7-kvm --with-compat --with-http_stub_status_module --with-http_secure_link_module --with-libatomic --with-http_gzip_static_module --add-dynamic-module=../ngx_http_geoip2_module --with-http_sub_module --with-http_addition_module --with-http_image_filter_module=dynamic --with-http_geoip_module --with-stream_geoip_module --with-stream_realip_module --with-stream_ssl_preread_module --with-threads --with-stream --with-stream_ssl_module --with-http_realip_module --add-dynamic-module=../ngx-fancyindex-0.4.2 --add-module=../ngx_cache_purge-2.5 --add-dynamic-module=../ngx_devel_kit-0.3.0 --add-dynamic-module=../set-misc-nginx-module-0.32 --add-dynamic-module=../echo-nginx-module-0.61 --add-module=../redis2-nginx-module-0.15 --add-module=../ngx_http_redis-0.3.7 --add-module=../memc-nginx-module-0.18 --add-module=../srcache-nginx-module-0.31 --add-dynamic-module=../headers-more-nginx-module-0.33 --with-pcre-jit --with-zlib=../zlib-cloudflare-1.3.0 --with-http_ssl_module --with-http_v2_module --with-openssl=../openssl-1.1.1b --with-openssl-opt='enable-ec_nistp_64_gcc_128 enable-tls1_3'
Click to expand...

include file /usr/local/nginx/conf/geoip2.conf
Code (Text):
# GeoIP2 Lite databases
  #  geoip2 /usr/share/GeoIP/GeoLite2-Country.mmdb {
  #    $geoip2_data_country_code country iso_code;
  #    $geoip2_data_country_name country names en;
  #  }

  geoip2 /usr/share/GeoIP/GeoLite2-City.mmdb {
    $geoip2_data_city_name city names en;
    $geoip2_data_city_geonameid city geoname_id;
    $geoip2_data_continent_code continent code;
    $geoip2_data_continent_geonameid continent geoname_id;
    $geoip2_data_continent_name continent names en;
    $geoip2_data_country_geonameid country geoname_id;
    $geoip2_data_country_iso country iso_code;
    $geoip2_data_country_name country names en;
    $geoip2_data_country_is_eu country is_in_european_union;
    $geoip2_data_location_accuracyradius location accuracy_radius;
    $geoip2_data_location_latitude location latitude;
    $geoip2_data_location_longitude location longitude;
    $geoip2_data_location_metrocode location metro_code;
    $geoip2_data_location_timezone location time_zone;
    $geoip2_data_postal_code postal code;
    $geoip2_data_rcountry_geonameid registered_country geoname_id;
    $geoip2_data_rcountry_iso registered_country iso_code;
    $geoip2_data_rcountry_name registered_country names en;
    $geoip2_data_rcountry_is_eu registered_country is_in_european_union;
    $geoip2_data_region_geonameid subdivisions 0 geoname_id;
    $geoip2_data_region_iso subdivisions 0 iso_code;
    $geoip2_data_region_name subdivisions 0 names en;
  }
manual lookup
Code (Text):
mmdblookup --file /usr/share/GeoIP/GeoLite2-City.mmdb --ip 8.8.8.8 

  {
    "continent": 
      {
        "code": 
          "NA" <utf8_string>
        "geoname_id": 
          6255149 <uint32>
        "names": 
          {
            "de": 
              "Nordamerika" <utf8_string>
            "en": 
              "North America" <utf8_string>
            "es": 
              "Norteamérica" <utf8_string>
            "fr": 
              "Amérique du Nord" <utf8_string>
            "ja": 
              "北アメリカ" <utf8_string>
            "pt-BR": 
              "América do Norte" <utf8_string>
            "ru": 
              "Северная Америка" <utf8_string>
            "zh-CN": 
              "北美洲" <utf8_string>
          }
      }
    "country": 
      {
        "geoname_id": 
          6252001 <uint32>
        "iso_code": 
          "US" <utf8_string>
        "names": 
          {
            "de": 
              "USA" <utf8_string>
            "en": 
              "United States" <utf8_string>
            "es": 
              "Estados Unidos" <utf8_string>
            "fr": 
              "États-Unis" <utf8_string>
            "ja": 
              "アメリカ合衆国" <utf8_string>
            "pt-BR": 
              "Estados Unidos" <utf8_string>
            "ru": 
              "США" <utf8_string>
            "zh-CN": 
              "美国" <utf8_string>
          }
      }
    "location": 
      {
        "accuracy_radius": 
          1000 <uint16>
        "latitude": 
          37.751000 <double>
        "longitude": 
          -97.822000 <double>
      }
    "registered_country": 
      {
        "geoname_id": 
          6252001 <uint32>
        "iso_code": 
          "US" <utf8_string>
        "names": 
          {
            "de": 
              "USA" <utf8_string>
            "en": 
              "United States" <utf8_string>
            "es": 
              "Estados Unidos" <utf8_string>
            "fr": 
              "États-Unis" <utf8_string>
            "ja": 
              "アメリカ合衆国" <utf8_string>
            "pt-BR": 
              "Estados Unidos" <utf8_string>
            "ru": 
              "США" <utf8_string>
            "zh-CN": 
              "美国" <utf8_string>
          }
      }
  }

eva2000 · Apr 8, 2019

Guide up now for How to enable GeoIP 2 Lite Nginx Module Support ?

Log in / Register

Forums

Welcome to Centmin Mod Community

Beta Branch prep work for GeoIP2 Lite nginx module support 123.09beta01

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

wmtech Active Member

rdan Well-Known Member

wmtech Active Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

.

Log in / Register

Useful Searches

Welcome to Centmin Mod Community

Beta Branch prep work for GeoIP2 Lite nginx module support 123.09beta01

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

wmtech Active Member

rdan Well-Known Member

wmtech Active Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

.