Welcome to Centmin Mod Community
Register Now

Nginx Is this normal? Missing content-encoding headers

Discussion in 'Nginx, PHP-FPM & MariaDB MySQL' started by BobbyWibowo, Mar 8, 2017.

  1. BobbyWibowo

    BobbyWibowo Active Member

    166
    36
    28
    Jul 30, 2015
    Medan, Indonesia
    Ratings:
    +57
    Local Time:
    8:38 AM
    Please fill in any relevant information that applies to you:
    • CentOS Version: CentOS 7 64bit
    • Centmin Mod Version Installed: 123.09beta01
    • Nginx Version Installed: 1.11.10
    • PHP Version Installed: 7.1.2
    • MariaDB MySQL Version Installed: 10.1.21
    • When was last time updated Centmin Mod code base ? : probably not too long ago (cronjob)
    • Persistent Config: Ayy, sir!
      Code:
      LIBRESSL_SWITCH=y
      NGINX_LIBBROTLI=n
      NGINX_PAGESPEED=y
      NGXDYNAMIC_NGXPAGESPEED=y
      NGINX_ZERODT=y
      PHP_PGO=y
    1488936959829screensave.png
    Is it normal that all my JS files that went through PageSpeed, then CloudFlare and then KeyCDN doesn't have any encoding listed in the Content-Encoding header? I didn't enable brotli since my site will go through CloudFlare anyway, so I thought it'd be pointless since CloudFlare doesn't support brotli (I don't know if I'll benefit anything if I had brotli from my server to CloudFlare though).
     
  2. eva2000

    eva2000 Administrator Staff Member

    30,903
    6,908
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,404
    Local Time:
    11:38 AM
    Nginx 1.13.x
    MariaDB 5.5
    Doesn't look normal though I don't recall as I no longer have a working KeyCDN zone as I let my account expire as couldn't keep topping up to have it expiry every 12 months heh.

    what if you disable KeyCDN and just serve through Cloudflare ?
     
  3. BobbyWibowo

    BobbyWibowo Active Member

    166
    36
    28
    Jul 30, 2015
    Medan, Indonesia
    Ratings:
    +57
    Local Time:
    8:38 AM
    Tried going to CloudFlare version by directly accessing one of the CSS/JS files through the origin domain, and apparently it didn't have any Content-Encoding header either.
     
  4. BobbyWibowo

    BobbyWibowo Active Member

    166
    36
    28
    Jul 30, 2015
    Medan, Indonesia
    Ratings:
    +57
    Local Time:
    8:38 AM
    Where do I change the encoding settings? I'm pretty sure I had seen it before, but I can't remember :(
     
  5. BobbyWibowo

    BobbyWibowo Active Member

    166
    36
    28
    Jul 30, 2015
    Medan, Indonesia
    Ratings:
    +57
    Local Time:
    8:38 AM
    So.... apparently I already have these in nginx.conf:
    Code:
      gzip on;
      gzip_vary on;
      gzip_disable "MSIE [1-6]\.";
      gzip_static on;
      gzip_min_length 1400;
      gzip_buffers 32 8k;
      gzip_http_version 1.0;
      gzip_comp_level 5;
      gzip_proxied any;
      gzip_types text/plain text/css text/xml application/javascript application/x-javascript application/xml application/xml+rss application/ecmascript application/json image/svg+xml;
    which probably means I should at least have gzip encoding on?
     
  6. eva2000

    eva2000 Administrator Staff Member

    30,903
    6,908
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,404
    Local Time:
    11:38 AM
    Nginx 1.13.x
    MariaDB 5.5
  7. BobbyWibowo

    BobbyWibowo Active Member

    166
    36
    28
    Jul 30, 2015
    Medan, Indonesia
    Ratings:
    +57
    Local Time:
    8:38 AM
    Tried using ?PageSpeed=off and I can see all the non-cached version of the CSS/JS files, but unfortunately they don't have Content-Encoding either. I can try accessing the files' header directly from SSH (with curl -I) since the website has a localhost line, but I don't see any Content-Encoding header for one of the CSS/JS files either (unless curl -I command isn't enough or something?).
     
  8. eva2000

    eva2000 Administrator Staff Member

    30,903
    6,908
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,404
    Local Time:
    11:38 AM
    Nginx 1.13.x
    MariaDB 5.5
    what about turning pagespeed off
    Code (Text):
    pscontrol off
    
     
  9. eva2000

    eva2000 Administrator Staff Member

    30,903
    6,908
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,404
    Local Time:
    11:38 AM
    Nginx 1.13.x
    MariaDB 5.5
    seems similar to this reported bug for mod_pagespeed which is the base code for ngx_pagespeed content-encoding -- suspected intermittent double-gzip encoding on css/js · Issue #1362 · pagespeed/mod_pagespeed · GitHub

    but if you turn off ngx_pagespeed you shouldn't have this issue
     
  10. eva2000

    eva2000 Administrator Staff Member

    30,903
    6,908
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,404
    Local Time:
    11:38 AM
    Nginx 1.13.x
    MariaDB 5.5
  11. BobbyWibowo

    BobbyWibowo Active Member

    166
    36
    28
    Jul 30, 2015
    Medan, Indonesia
    Ratings:
    +57
    Local Time:
    8:38 AM
    So I tried various things and after adding -H "Accept-Encoding: gzip, deflate, sdch" to the curl command, I can see Content-Encoding kicks in (the value was gzip), but only when trying to access the file from the localhost line/domain. Accessing the HTTPS domain kept saying that the peer certificate couldn't be verified or something even though I added --cacert /etc/ssl/certs/cacert.pem (from cacertinfo.ini file), so I can't test it further (from SSH, at the very least). With HTTP Headers add-on for Chrome, I can see that the request header had the Accept-Encoding part but there was no Content-Encoding for the response. So, I'm guessing this have something to do with CloudFlare itself?

    EDIT: I had PageSpeed turned on, so I'm guessing it wasn't PageSpeed.
    Also tried accessing the PageSpeed-cached version of the JS file from the localhost line/domain and Content-Encoding existed.
     
  12. eva2000

    eva2000 Administrator Staff Member

    30,903
    6,908
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,404
    Local Time:
    11:38 AM
    Nginx 1.13.x
    MariaDB 5.5
    yes that is needed if you use curl from command like

    you can temporarily rule it out using curl and local linux /etc/hosts file edit with entry
    Code (Text):
    yourdomainrealip domain.com
    

    replace with your domain.com domain name and it's real IP for server not cloudflare

    then any curl requests to domain.com for server with /etc/hosts edit will go to domain.com's real IP bypassing cloudflare
     
  13. BobbyWibowo

    BobbyWibowo Active Member

    166
    36
    28
    Jul 30, 2015
    Medan, Indonesia
    Ratings:
    +57
    Local Time:
    8:38 AM
    Ohhh.... what the hell!? Is it my browser after all!??
    1488941521757screensave.png
    That was the result from http://web-sniffer.net/ and we can clearly see that Content-Encoding was returned and it was even from CloudFlare's cache. But when I access the exact same URL from my Chrome browser, there's no Content-Encoding even though the request headers had Accept-Encoding: gzip, deflate, sdch, br.
     
    • Informative Informative x 1
  14. eva2000

    eva2000 Administrator Staff Member

    30,903
    6,908
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,404
    Local Time:
    11:38 AM
    Nginx 1.13.x
    MariaDB 5.5
    tried opera or firefox browsers to confirm ?
     
  15. BobbyWibowo

    BobbyWibowo Active Member

    166
    36
    28
    Jul 30, 2015
    Medan, Indonesia
    Ratings:
    +57
    Local Time:
    8:38 AM
    Weird.. tried Chrome, Firefox and even Opera developer, but all of them didn't get any Content-Encoding in the response headers. But using web-sniffer or curl-ing the localhost domain, Content-Encoding will always be there.