Learn about Centmin Mod LEMP Stack today
Register Now

[Solved] static files expire

Discussion in 'Bug Reports' started by raciasolvo, Oct 20, 2016.

  1. raciasolvo

    raciasolvo Member

    98
    11
    8
    Oct 7, 2016
    Ratings:
    +28
    Local Time:
    10:12 AM
    Nginx 1.11.6
    MariaDB 10.0.27
    ${vhostname}.ssl.conf:
    Code (Text):
     include /usr/local/nginx/conf/staticfiles.conf;


    staticfiles.conf:
    Code (Text):
        location ~* \.(js)$ {
            #add_header Pragma public;
            add_header Cache-Control "public, must-revalidate, proxy-revalidate";
            access_log off;
            expires 30d;
            break;
            }
    
        location ~* \.(css)$ {
            #add_header Pragma public;
            add_header Cache-Control "public, must-revalidate, proxy-revalidate";
            access_log off;
            expires 30d;
            break;
            }


    curl:
    Code (Text):
    no expire and Cache-Control


    It seems that the static files go through another location.

     
  2. raciasolvo

    raciasolvo Member

    98
    11
    8
    Oct 7, 2016
    Ratings:
    +28
    Local Time:
    10:12 AM
    Nginx 1.11.6
    MariaDB 10.0.27
    autoprotect for autoptimize before staticfiles?
    Code (Text):
    location ~ ^/wp-content/cache/autoptimize/(.+/)?(.+)\.(js)$ { allow all; }
    location ~ ^/wp-content/cache/autoptimize/(.+/)?(.+)\.(css)$ { allow all; }
     
  3. eva2000

    eva2000 Administrator Staff Member

    55,163
    12,249
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,825
    Local Time:
    5:12 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    yeah that would be correct autoprotect include file comes before staticfiles.conf include file and if you curl against those it autoptimized js and css files it would be missing expires headers.

    Going to add them to autoprotect.sh rule for it. Thanks for bug report (y)

    123.09beta01 updated for fix https://community.centminmod.com/th...-jss-css-and-image-file-expire-headers….9218/

    next time tools/autoprotect.sh cronjob runs or if you manually run it, it will update all nginx vhosts autoprotect include files with the expires headers :)
     
    Last edited: Oct 20, 2016
  4. raciasolvo

    raciasolvo Member

    98
    11
    8
    Oct 7, 2016
    Ratings:
    +28
    Local Time:
    10:12 AM
    Nginx 1.11.6
    MariaDB 10.0.27
    We have autoprotect include before staticfiles include now. I think that is right. What do you think about this fast patch for autoprotect?
    Code (Text):
    error_page 418 = @static_file;
    location ~ ^/wp-content/cache/autoptimize/(.+/)?(.+)\.(js)$ { allow all; return 418; }


    staticfiles.conf
    Code (Text):
    location @static_file {
            #add_header Pragma public;
            add_header Cache-Control "public, must-revalidate, proxy-revalidate";
            access_log off;
            expires 30d;
            break;
    }

    need tests
     
  5. raciasolvo

    raciasolvo Member

    98
    11
    8
    Oct 7, 2016
    Ratings:
    +28
    Local Time:
    10:12 AM
    Nginx 1.11.6
    MariaDB 10.0.27
    better for different redirections
    Code (Text):
    location ~ ^/wp-content/cache/autoptimize/(.+/)?(.+)\.(js)$ { allow all; error_page 418 = @static_js; return 418; }


    or more flexible and stable:
    Code (Text):
    location ~ ^/wp-content/cache/autoptimize/(.+/)?(.+)\.(js)$ { allow all; include ../../js/*.conf; }

    and we do not need to delete the staticfiles.conf include (if the autoprotect include is commented out staticfiles.conf include will work).
     
    Last edited: Oct 20, 2016
  6. eva2000

    eva2000 Administrator Staff Member

    55,163
    12,249
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,825
    Local Time:
    5:12 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    probably making it more complicated then it needs to be for now

    but yes needs testing ;)
     
  7. raciasolvo

    raciasolvo Member

    98
    11
    8
    Oct 7, 2016
    Ratings:
    +28
    Local Time:
    10:12 AM
    Nginx 1.11.6
    MariaDB 10.0.27
    Autoptimize is a cache plugin and I added new include before autoprotect:
    Code (Text):
    include /usr/local/nginx/conf/wpincludes/${vhostname}/wpautoptimize_${vhostname}.conf;

    and created new conf - wpincludes/${vhostname}/wpautoptimize_${vhostname}.conf
    Code (Text):
    location ~ /wp-content/cache/autoptimize/ {
        access_log  off;
        error_log   /var/log/nginx/error.log crit;
        deny        all;
        location ~ [^/]+\.(js|css)$ {
            allow all;
            add_header Cache-Control "public, must-revalidate, proxy-revalidate";
            expires max;
            #break;
        }
        location ~ [^/]+\.(php|cgi|pl|php3|php4|php5|php6|phtml|shtml|html)$ {
            allow 127.0.0.1;
            deny all;
        }
    }

    no bugs now :D
     
  8. eva2000

    eva2000 Administrator Staff Member

    55,163
    12,249
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,825
    Local Time:
    5:12 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    so the problem with old autoprotect.sh locations for autoptimize wasn't just with missing cache control headers ?
     
  9. raciasolvo

    raciasolvo Member

    98
    11
    8
    Oct 7, 2016
    Ratings:
    +28
    Local Time:
    10:12 AM
    Nginx 1.11.6
    MariaDB 10.0.27
    Problem with nginx behavior. You cannot use 2 locations for the file without a redirection. We can use either the autoprotect:
    Code (Text):
    location ~ ^/wp-content/cache/autoptimize/(.+/)?(.+)\.(js)$
    or the staticfiles:
    Code (Text):
    location ~* \.(js)$

    Worked always first location.

    I merged autoprotect with staticfiles.
     
    Last edited: Oct 21, 2016
  10. raciasolvo

    raciasolvo Member

    98
    11
    8
    Oct 7, 2016
    Ratings:
    +28
    Local Time:
    10:12 AM
    Nginx 1.11.6
    MariaDB 10.0.27
    I think that you should not touch the autoprotect now. Simply move the autoprotect down after including all caches, and add protection in cache confs if needed.
     
  11. eva2000

    eva2000 Administrator Staff Member

    55,163
    12,249
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,825
    Local Time:
    5:12 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    you don't need to use both locations, should be able separately match for those location matches

    maybe it's an incorrect match

    should instead of
    Code (Text):
    location ~ ^/wp-content/cache/autoptimize/(.+/)?(.+)\.(js)$

    be
    Code (Text):
    location ~ ^/wp-content/cache/autoptimize/(.+)\.(js)$
     
  12. raciasolvo

    raciasolvo Member

    98
    11
    8
    Oct 7, 2016
    Ratings:
    +28
    Local Time:
    10:12 AM
    Nginx 1.11.6
    MariaDB 10.0.27
    However only one and first regexp will always to work without redirection. Either the autoprotect or staticfiles, that is the question. :)
     
  13. raciasolvo

    raciasolvo Member

    98
    11
    8
    Oct 7, 2016
    Ratings:
    +28
    Local Time:
    10:12 AM
    Nginx 1.11.6
    MariaDB 10.0.27
    /wp-content/cache/autoptimize/js/file.js is the correct path
     
  14. eva2000

    eva2000 Administrator Staff Member

    55,163
    12,249
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,825
    Local Time:
    5:12 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    i see :)

    strangely tests show it working, i set autoptimize css regex match to expire 1d
    Code (Text):
    location ~ ^/wp-content/cache/autoptimize/(.+/)?(.+)\.(css)$ { allow all; expires 1d; }

    Code (Text):
    curl -I http://localhost/wp-content/cache/autoptimize/css/test.css       
    HTTP/1.1 200 OK
    Date: Thu, 20 Oct 2016 19:09:30 GMT
    Content-Type: text/css
    Content-Length: 0
    Last-Modified: Thu, 20 Oct 2016 19:03:21 GMT
    Connection: keep-alive
    ETag: "580914f9-0"
    Server: nginx centminmod
    X-Powered-By: centminmod
    Expires: Fri, 21 Oct 2016 19:09:30 GMT
    Cache-Control: max-age=86400
    Accept-Ranges: bytes


    and left staticfiles.conf with expire 30d
    Code (Text):
    curl -I http://localhost/css/test2.css       
    HTTP/1.1 200 OK
    Date: Thu, 20 Oct 2016 19:09:14 GMT
    Content-Type: text/css
    Content-Length: 0
    Last-Modified: Thu, 20 Oct 2016 19:06:19 GMT
    Connection: keep-alive
    ETag: "580915ab-0"
    Server: nginx centminmod
    X-Powered-By: centminmod
    Expires: Sat, 19 Nov 2016 19:09:14 GMT
    Cache-Control: max-age=2592000
    Access-Control-Allow-Origin: *
    Cache-Control: public, must-revalidate, proxy-revalidate
    Accept-Ranges: bytes
    
     
    Last edited: Oct 21, 2016
  15. raciasolvo

    raciasolvo Member

    98
    11
    8
    Oct 7, 2016
    Ratings:
    +28
    Local Time:
    10:12 AM
    Nginx 1.11.6
    MariaDB 10.0.27
    Good nginx behavior for autoprotect and included (or not) modules:
    Code (Text):
    # autoprotect
    location ~ /module/file\.js { allow all; }
    location ~ /module/file\.css { allow all; }
    location ~ /module/bad\.file { deny all; }
    location ~ /module/good\.file { allow all; }
    
    # module
    location /module/ { # I use prefix and inner locations checks first!
        location ~ /module/.+\.(js|css) { it_is_the_static; } # I allow it if include the module
        # other.files are not interesting for me, autoprotect them.
    }

    need tests :)
     
  16. eva2000

    eva2000 Administrator Staff Member

    55,163
    12,249
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,825
    Local Time:
    5:12 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    i see what you mean prefix match before regex match :)
     
  17. raciasolvo

    raciasolvo Member

    98
    11
    8
    Oct 7, 2016
    Ratings:
    +28
    Local Time:
    10:12 AM
    Nginx 1.11.6
    MariaDB 10.0.27
    Not strange. Try now. Staticfiles are not works.
     
  18. raciasolvo

    raciasolvo Member

    98
    11
    8
    Oct 7, 2016
    Ratings:
    +28
    Local Time:
    10:12 AM
    Nginx 1.11.6
    MariaDB 10.0.27
    Yes. It is the best way for autoprotect. Like iptables - deny all now and allow after. :D
     
  19. eva2000

    eva2000 Administrator Staff Member

    55,163
    12,249
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,825
    Local Time:
    5:12 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    it does work for me staticfiles.conf include matched this with autoprotect in place above it
    Code (Text):
    curl -I http://localhost/css/test2.css      
    HTTP/1.1 200 OK
    Date: Thu, 20 Oct 2016 19:09:14 GMT
    Content-Type: text/css
    Content-Length: 0
    Last-Modified: Thu, 20 Oct 2016 19:06:19 GMT
    Connection: keep-alive
    ETag: "580915ab-0"
    Server: nginx centminmod
    X-Powered-By: centminmod
    Expires: Sat, 19 Nov 2016 19:09:14 GMT
    Cache-Control: max-age=2592000
    Access-Control-Allow-Origin: *
    Cache-Control: public, must-revalidate, proxy-revalidate
    Accept-Ranges: bytes
     
  20. raciasolvo

    raciasolvo Member

    98
    11
    8
    Oct 7, 2016
    Ratings:
    +28
    Local Time:
    10:12 AM
    Nginx 1.11.6
    MariaDB 10.0.27
    Code (Text):
    curl -I http://localhost/wp-content/cache/autoptimize/css/test.css
    :)