Get the most out of your Centmin Mod LEMP stack
Become a Member

Nginx IPv6 not answering on IP

Discussion in 'Nginx, PHP-FPM & MariaDB MySQL' started by Tracy Perry, Nov 7, 2016.

  1. Tracy Perry

    Tracy Perry Active Member

    280
    118
    43
    Aug 24, 2014
    Texas
    Ratings:
    +210
    Local Time:
    12:50 PM
    1.21.6
    MariaDB 10.3.36
    nginx is compiled with IPv6 enabled (this is a test of 1.9.11 to see if it's an nginx issue, using 1.11.5 normally)
    Code:
    [root@whiskey conf.d]# nginx -V
    nginx version: nginx/1.9.11
    built by clang 3.4.2 (tags/RELEASE_34/dot2-final)
    built with LibreSSL 2.4.3
    TLS SNI support enabled
    configure arguments: --with-ld-opt='-lrt -ljemalloc -Wl,-z,relro -Wl,-rpath,/usr/local/lib' --with-cc-opt='-m64 -mtune=native -mfpmath=sse -g -O3 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wno-sign-compare -Wno-string-plus-int -Wno-deprecated-declarations -Wno-unused-parameter -Wno-unused-const-variable -Wno-conditional-uninitialized -Wno-mismatched-tags -Wno-sometimes-uninitialized -Wno-parentheses-equality -Wno-tautological-compare -Wno-self-assign -Wno-deprecated-register -Wno-deprecated -Wno-invalid-source-encoding -Wno-pointer-sign -Wno-parentheses -Wno-enum-conversion -Wno-c++11-compat-deprecated-writable-strings -Wno-write-strings' --sbin-path=/usr/local/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --with-ipv6 --with-http_stub_status_module --with-http_secure_link_module --add-module=../nginx-module-vts --with-libatomic --with-http_gzip_static_module --with-http_sub_module --with-http_addition_module --with-http_image_filter_module=dynamic --with-http_geoip_module --with-threads --with-stream=dynamic --with-stream_ssl_module --with-http_realip_module --add-dynamic-module=../ngx-fancyindex-0.4.0 --add-module=../ngx_cache_purge-2.3 --add-module=../ngx_devel_kit-0.3.0 --add-module=../set-misc-nginx-module-0.31 --add-module=../echo-nginx-module-0.60 --add-module=../redis2-nginx-module-0.13 --add-module=../ngx_http_redis-0.3.7 --add-module=../memc-nginx-module-0.17 --add-module=../srcache-nginx-module-0.31 --add-module=../headers-more-nginx-module-0.31 --with-pcre=../pcre-8.39 --with-pcre-jit --with-http_ssl_module --with-http_v2_module --with-openssl=../libressl-2.4.3
    
    I have IPv6 configured.
    Code:
    eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
            inet 192.99.17.77  netmask 255.255.255.0  broadcast 192.99.17.255
            inet6 2607:5300:60:424d::120  prefixlen 64  scopeid 0x0<global>
            inet6 2607:5300:60:424d::20  prefixlen 64  scopeid 0x0<global>
            inet6 2607:5300:60:424d::30  prefixlen 64  scopeid 0x0<global>
            inet6 2607:5300:60:424d::40  prefixlen 64  scopeid 0x0<global>
            inet6 fe80::ec4:7aff:fea8:a1a6  prefixlen 64  scopeid 0x20<link>
            inet6 2607:5300:60:424d::50  prefixlen 64  scopeid 0x0<global>
            inet6 2607:5300:60:424d::60  prefixlen 64  scopeid 0x0<global>
            inet6 2607:5300:60:424d::4  prefixlen 64  scopeid 0x0<global>
            inet6 2607:5300:60:424d::10  prefixlen 64  scopeid 0x0<global>
            ether 0c:c4:7a:a8:a1:a6  txqueuelen 1000  (Ethernet)
            RX packets 38822557  bytes 2855688457 (2.6 GiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 13443907  bytes 12094292034 (11.2 GiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
            device memory 0xfb120000-fb13ffff
    
    
    I can ping a remote site with no issues via IPv6.
    Code:
    [root@whiskey conf.d]# ping6 -c3 ipv6.google.com
    PING ipv6.google.com(lga15s43-in-x0e.1e100.net) 56 data bytes
    64 bytes from lga15s43-in-x0e.1e100.net: icmp_seq=1 ttl=57 time=23.8 ms
    64 bytes from lga15s43-in-x0e.1e100.net: icmp_seq=2 ttl=57 time=23.7 ms
    64 bytes from lga15s43-in-x0e.1e100.net: icmp_seq=3 ttl=57 time=23.8 ms
    
    --- ipv6.google.com ping statistics ---
    3 packets transmitted, 3 received, 0% packet loss, time 2002ms
    rtt min/avg/max/mdev = 23.787/23.798/23.806/0.008 ms
    

    I an use a remote VPS I have access to and ping my domain name via IPv6 with no issues
    Code:
    root@cyprinidae webmail]# ping6 -c3 thepipestand.com
    PING thepipestand.com(thepipestand.com) 56 data bytes
    64 bytes from thepipestand.com: icmp_seq=1 ttl=54 time=65.8 ms
    64 bytes from thepipestand.com: icmp_seq=2 ttl=54 time=65.8 ms
    64 bytes from thepipestand.com: icmp_seq=3 ttl=54 time=65.7 ms
    I have (as is apparent) IPv6 DNS entry configured.
    I have set the listen parameter in my nginx vhost
    Code:
    server {
        listen  158.69.107.150:443 ssl http2 ;
    listen [2607:5300:60:424d:0:0:0:120]:80 ipv6only=on;
    listen [2607:5300:60:424d:0:0:0:120]:443 ipv6only=on;
    server_name thepipestand.com;
    
    netstat -tulpna|grep nginx returns this
    Code:
    [root@whiskey conf.d]# netstat -tulpna | grep nginx
    tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      24153/nginx: master
    tcp        0      0 149.56.33.157:443       0.0.0.0:*               LISTEN      24153/nginx: master
    tcp        0      0 158.69.107.150:443      0.0.0.0:*               LISTEN      24153/nginx: master
    tcp        0      0 149.56.33.158:443       0.0.0.0:*               LISTEN      24153/nginx: master
    tcp        0      0 149.56.35.97:443        0.0.0.0:*               LISTEN      24153/nginx: master
    tcp        0      0 149.56.33.159:443       0.0.0.0:*               LISTEN      24153/nginx: master
    tcp        0      0 149.56.33.158:443       64.188.254.247:65105    ESTABLISHED 24154/nginx: worker
    tcp        0      0 158.69.107.150:443      64.188.254.247:65106    ESTABLISHED 24154/nginx: worker
    tcp6       0      0 2607:5300:60:424d::3:80 :::*                    LISTEN      24153/nginx: master
    tcp6       0      0 2607:5300:60:424d::1:80 :::*                    LISTEN      24153/nginx: master
    tcp6       0      0 2607:5300:60:424d:::443 :::*                    LISTEN      24153/nginx: master
    tcp6       0      0 2607:5300:60:424d:::443 :::*                    LISTEN      24153/nginx: master
    udp        0      0 192.99.17.77:54915      8.8.8.8:53              ESTABLISHED 24154/nginx: worker
    udp        0      0 192.99.17.77:57278      8.8.8.8:53              ESTABLISHED 24154/nginx: worker
    notice that it's not listing the 120 address but does list a 1 and a 3 . Those are not defined anywhere in any vhost but there is a 10 and a 30 defined.

    Any suggestions? It used to work on an older version of nginx, so I'm wondering if that may be an issue. I'm using 1.11.5 mainline.
    @eva2000, you got any ideas?

     
    Last edited: Nov 7, 2016
  2. eva2000

    eva2000 Administrator Staff Member

    55,239
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,833
    Local Time:
    3:50 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    strange not sure if related but try changing
    Code (Text):
    listen [2607:5300:60:424d:0:0:0:120]:443 ipv6only=on;
    

    to
    Code (Text):
    listen [2607:5300:60:424d:0:0:0:120]:443 ssl http2 ipv6only=on;
    

    FYI, nginx 1.11.5 now enables ipv6 by default and centmin mod nginx routine accounts for this by removing --with-ipv6 - well that's what it should do if nginx 1.11.5 or higher detected.
     
  3. Tracy Perry

    Tracy Perry Active Member

    280
    118
    43
    Aug 24, 2014
    Texas
    Ratings:
    +210
    Local Time:
    12:50 PM
    1.21.6
    MariaDB 10.3.36
    I've tried just about every different setting I can.. even using the above, I get this when I go to validate the site

    Screen Shot 2016-11-06 at 10.09.54 AM.png

    It's like nginx is not responding to the IPv6 address it is told to use. I've even turned off CSF just to see if that may have been it (and 443 is allowed for both IPv4 and IPv6).

    EDIT:
    I'm wondering if it's an ip6tables issue. In CSF it lists https for the ruleset to allow, but in ip6tables -L I get this

    Code:
    Chain INPUT (policy ACCEPT)
    target     prot opt source               destination        
    ACCEPT     all      anywhere             anywhere             state RELATED,ESTABLISHED
    ACCEPT     ipv6-icmp    anywhere             anywhere            
    ACCEPT     all      anywhere             anywhere            
    ACCEPT     tcp      anywhere             anywhere             state NEW tcp dpt:ssh
    ACCEPT     udp      anywhere             fe80::/64            udp dpt:dhcpv6-client state NEW
    REJECT     all      anywhere             anywhere             reject-with icmp6-adm-prohibited
    
    Chain FORWARD (policy ACCEPT)
    target     prot opt source               destination        
    REJECT     all      anywhere             anywhere             reject-with icmp6-adm-prohibited
    
    Chain OUTPUT (policy ACCEPT)
    target     prot opt source               destination
     
  4. eva2000

    eva2000 Administrator Staff Member

    55,239
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,833
    Local Time:
    3:50 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    ah did you enable IPv6 after centmin mod install ? make sure in /etc/csf/csf.conf you have enabled IPV6 variable with IPV6 = "1"
    Code (Text):
    IPV6 = "1"
    

    restart CSF Firewall
    Code (Text):
    csf -r

    CSF firewall on initial install will detect if IPv6 is enabled and auto enable IPV6 = "1" but if you had IPv6 disabled on centmin mod initial install, CSF Firewall would of disbaled IPV6 variable.
     
  5. Tracy Perry

    Tracy Perry Active Member

    280
    118
    43
    Aug 24, 2014
    Texas
    Ratings:
    +210
    Local Time:
    12:50 PM
    1.21.6
    MariaDB 10.3.36
    I hate the use of 0/1 for on off! :oops:

    Screen Shot 2016-11-06 at 10.18.52 AM.png

    Been fighting this thing for about 1.5 hours.. and felt like it was an firewall rule set.
     
  6. eva2000

    eva2000 Administrator Staff Member

    55,239
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,833
    Local Time:
    3:50 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    hehe.. might need to add to 123.09beta01 a IPv6 enable or not check on nginx recompiles via centmin.sh menu option 4 and auto enable IPV6 in /etc/csf/csf.conf if IPv6 is detected as enabled :)
     
  7. Tracy Perry

    Tracy Perry Active Member

    280
    118
    43
    Aug 24, 2014
    Texas
    Ratings:
    +210
    Local Time:
    12:50 PM
    1.21.6
    MariaDB 10.3.36
    Most likely that was the issue.. when I first set the new OVH server up, I didn't set up any of the IPv6 stuff as I was busy having my 20 IPv4's transferred over and creating individual eth0:xx configurations for them.
    Had the IPv6 stuff commented out as I always hate figuring out the format for the new addresses.
    Was a case of forest vs trees... and I looked at that setting several times and it didn't "seem" right but I ignored it.
    Fixed now.. see @CarpCharacin how useful @eva2000 is to have around - even I call upon his all-seeing all-knowing knowledge frequently. :smuggrin:
     
  8. eva2000

    eva2000 Administrator Staff Member

    55,239
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,833
    Local Time:
    3:50 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Yeah most likely what happened
    @Tracy Perry for my ipv6 check, can you tell me what you get for output for these 2 commands
    Code (Text):
    ping6 -c1 $(hostname) >/dev/null 2>&1 ; echo $?
    

    Code (Text):
    awk -F "=" '/NETWORKING_IPV6/ {print $2}' /etc/sysconfig/network | grep 'yes' >/dev/null 2>&1; echo $?
    
     
  9. Tracy Perry

    Tracy Perry Active Member

    280
    118
    43
    Aug 24, 2014
    Texas
    Ratings:
    +210
    Local Time:
    12:50 PM
    1.21.6
    MariaDB 10.3.36
    Code:
    [root@whiskey csf]# ping6 -c1 $(hostname) >/dev/null 2>&1 ; echo $?
    1
    Code:
    [root@whiskey csf]# awk -F "=" '/NETWORKING_IPV6/ {print $2}' /etc/sysconfig/network | grep 'yes' >/dev/null 2>&1; echo $?
    0
    are what are returning currently.
    I don't have an entry for IPv6 for my main server as I strictly use IPv4 for it.
     
  10. eva2000

    eva2000 Administrator Staff Member

    55,239
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,833
    Local Time:
    3:50 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    thanks.. so will use just the 2nd check for ipv6 instead of both :)
     
  11. eva2000

    eva2000 Administrator Staff Member

    55,239
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,833
    Local Time:
    3:50 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
  12. Tracy Perry

    Tracy Perry Active Member

    280
    118
    43
    Aug 24, 2014
    Texas
    Ratings:
    +210
    Local Time:
    12:50 PM
    1.21.6
    MariaDB 10.3.36
    Yeah, would probably be the safest in case others do like I do. I don't want mail processed by anything other than IPv4 and minimize my server (actual iron) to IPv4 only, so I don't want the main domain answering on anything else.
     
  13. dorobo

    dorobo Active Member

    420
    104
    43
    Jun 6, 2014
    Ratings:
    +162
    Local Time:
    1:50 AM
    latest
    latest
    holy moly, I've been having the same problem with permission denied for ipv6 on my OpenVZ vps but if I run it from Linode it works. I thought it was a configuration problem with my host but after setting ipv6 = 1 in /etc/csf/csf.conf it now works!

    thanks guys.

    P.S. but why does it work in Linode though even if I haven't change ipv6 = 1 setting in csf.conf
     
  14. dorobo

    dorobo Active Member

    420
    104
    43
    Jun 6, 2014
    Ratings:
    +162
    Local Time:
    1:50 AM
    latest
    latest
    ah nevermind. I got the answer. CSF detects ipv6 properly in Linode that's why it sets ipv6 = 1 automatically whereas with my current OpenVZ host, it does not. Something like that? :D
     
  15. Tracy Perry

    Tracy Perry Active Member

    280
    118
    43
    Aug 24, 2014
    Texas
    Ratings:
    +210
    Local Time:
    12:50 PM
    1.21.6
    MariaDB 10.3.36
    Did you set up IPv6 after you installed CentMin? If so, that's the issue. CentMin detects if IPv6 is enabled on the VPS/dedi and if not, disables it in CSF's config file
     
  16. dorobo

    dorobo Active Member

    420
    104
    43
    Jun 6, 2014
    Ratings:
    +162
    Local Time:
    1:50 AM
    latest
    latest
    No. The VPS has IPv6 working from the get go.
     
  17. Tracy Perry

    Tracy Perry Active Member

    280
    118
    43
    Aug 24, 2014
    Texas
    Ratings:
    +210
    Local Time:
    12:50 PM
    1.21.6
    MariaDB 10.3.36
    So, you configured a specific valid IPv6 address to use in /etc/sysconfig/network or /etc/sysconfig/network-scripts/ifcfg-eth0?
    I'd love to know how you got it to get a valid IPv6 address upon install. Did you assign it during the CentOS setup (been so long since I've done one I dont' remember if that's an option or not - think it is under the advanced options though now that I think of it some more).
    I know by default I've always had to manually input the IPv6 gateway - I normally just do it via a text editor after the initial install.

    I know when I was helping an individual with their ProxMox install, CentOS would ping outward from hit on IPv6 with no issues, but getting traffic in was the issue. We had to configure it (and jump through some hoops) on the main server (which was Debian) for each venet.
     
    Last edited: Nov 7, 2016
  18. dorobo

    dorobo Active Member

    420
    104
    43
    Jun 6, 2014
    Ratings:
    +162
    Local Time:
    1:50 AM
    latest
    latest
    I didn't have to do any of that. The assigned IPv6 addresses just works out of the box. Just like how IPv4 works right away from the start.
     
  19. dorobo

    dorobo Active Member

    420
    104
    43
    Jun 6, 2014
    Ratings:
    +162
    Local Time:
    1:50 AM
    latest
    latest
  20. Tracy Perry

    Tracy Perry Active Member

    280
    118
    43
    Aug 24, 2014
    Texas
    Ratings:
    +210
    Local Time:
    12:50 PM
    1.21.6
    MariaDB 10.3.36
    IPv4 didn't work for me "out of the box". I had to configure the IP address and related info. IPv6 may be able to use AutoConf to do that for it, but does it keep the same IPv6 address?