Join the community today
Register Now

Beta Branch update staticfiles.conf include file cache-control headers

Discussion in 'Centmin Mod Github Commits' started by eva2000, May 19, 2017.

  1. eva2000

    eva2000 Administrator Staff Member

    31,001
    6,919
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,424
    Local Time:
    11:50 PM
    Nginx 1.13.x
    MariaDB 5.5
  2. Jimmy

    Jimmy Premium Member Premium Member

    1,165
    256
    83
    Oct 24, 2015
    East Coast USA
    Ratings:
    +626
    Local Time:
    8:50 AM
    1.13.x
    MariaDB 10.1.x
    Just curious, when do these changes apply? When we rebuild nginx? Or do I have to manually enter them into my usr/nginx/conf/staticfiles.conf?
     
    Last edited: May 19, 2017
    • Like Like x 1
  3. eva2000

    eva2000 Administrator Staff Member

    31,001
    6,919
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,424
    Local Time:
    11:50 PM
    Nginx 1.13.x
    MariaDB 5.5
    you have to manually edit your local server copy of /usr/local/nginx/conf/staticfiles.conf as centmin mod tries not to modifying existing vhosts as folks may have their own customisations :)
     
    • Informative Informative x 1
  4. Jimmy

    Jimmy Premium Member Premium Member

    1,165
    256
    83
    Oct 24, 2015
    East Coast USA
    Ratings:
    +626
    Local Time:
    8:50 AM
    1.13.x
    MariaDB 10.1.x
    Thanks. Wasn't sure. I'll have to keep my eye out for changes like this.

    I wonder if there's an ability to tag items like this on the forum here which need to be manually changed by users? I'm subscribed to most of the forums, but I'd hate to have something pass by me.

    Thinking a little further, maybe a notice system is something you could add to the premium membership? A notice system for changes to CMM which need to be manually done to files, like this one. Just a thought.
     
    • Winner Winner x 1
  5. eva2000

    eva2000 Administrator Staff Member

    31,001
    6,919
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,424
    Local Time:
    11:50 PM
    Nginx 1.13.x
    MariaDB 5.5
    • Informative Informative x 1
  6. tjk

    tjk Member

    74
    16
    8
    Jun 27, 2015
    Ratings:
    +25
    Local Time:
    8:50 AM
    Which, imo, is a huge PITA trying to keep up with all the changes and having to go back and make config file changes. Why I mentioned earlier a flag to overwrite all config files that you update and keep user changes in include files or some other method.
     
    • Like Like x 1
  7. pamamolf

    pamamolf Well-Known Member

    2,825
    253
    83
    May 31, 2014
    Ratings:
    +449
    Local Time:
    3:50 PM
    Nginx-1.13.x
    MariaDB 10.1.x
    That's an issue to track such manual changes......

    Maybe we should have for them (as are not many) a post and note there the changes of each file so we will be able to easily check what it should like the latest version of a file and adjust ours :)
     
    • Like Like x 1
  8. Sunka

    Sunka Active Member

    932
    243
    43
    Oct 31, 2015
    Rijeka, Croatia
    Ratings:
    +394
    Local Time:
    2:50 PM
    Nginx 1.13.3
    MariaDB 10.1.24
    Or maybe just prefix "Manual" and "Automatic" so we should know if change is be added automaticall or need our working.

    I agree that is confusing and problematic as is now
     
  9. eva2000

    eva2000 Administrator Staff Member

    31,001
    6,919
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,424
    Local Time:
    11:50 PM
    Nginx 1.13.x
    MariaDB 5.5
    Yeah I hear you about it.. so will need to come up with a system.
     
  10. pamamolf

    pamamolf Well-Known Member

    2,825
    253
    83
    May 31, 2014
    Ratings:
    +449
    Local Time:
    3:50 PM
    Nginx-1.13.x
    MariaDB 10.1.x
    Github maybe with config files that will have the latest changes?

    or

    Official centminmod page ?

    Don't know ....
     
  11. eva2000

    eva2000 Administrator Staff Member

    31,001
    6,919
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,424
    Local Time:
    11:50 PM
    Nginx 1.13.x
    MariaDB 5.5
    these are some of the files which may get changes which may need manual updates or have had dynamic updates made on the defaults based on your server specs
    • /usr/local/nginx/conf/nginx.conf
    • /usr/local/nginx/conf/conf.d/virtual.conf
    • /usr/local/nginx/conf/staticfiles.conf
    • /usr/local/etc/php-fpm.conf
    • /usr/local/nginx/conf/php.conf
    • /usr/local/nginx/conf/drop.conf
    • /usr/local/nginx/conf/errorpage.conf
    • /usr/local/nginx/conf/pagespeed.conf
    • /usr/local/nginx/conf/pagespeedhandler.conf
    • /usr/local/nginx/conf/pagespeedstatslog.conf
    you can use sdiff to do diff side by side compare within ssh provided your ssh client is configured with wider screen width buffer

    i.e. sdiff with width of 200 to compare the latest copies locally available in /usr/local/src/centminmod with ones live on server

    Code (Text):
    sdiff -w200 /usr/local/src/centminmod/config/nginx/nginx.conf /usr/local/nginx/conf/nginx.conf
    sdiff -w200 /usr/local/src/centminmod/config/nginx/conf.d/virtual.conf /usr/local/nginx/conf/conf.d/virtual.conf
    sdiff -w200 /usr/local/src/centminmod/config/nginx/staticfiles.conf /usr/local/nginx/conf/staticfiles.conf
    sdiff -w200 /usr/local/src/centminmod/config/nginx/php.conf /usr/local/nginx/conf/php.conf
    sdiff -w200 /usr/local/src/centminmod/config/nginx/drop.conf /usr/local/nginx/conf/drop.conf
    sdiff -w200 /usr/local/src/centminmod/config/nginx/errorpage.conf /usr/local/nginx/conf/errorpage.conf
    sdiff -w200 /usr/local/src/centminmod/config/nginx/pagespeed.conf /usr/local/nginx/conf/pagespeed.conf
    sdiff -w200 /usr/local/src/centminmod/config/nginx/pagespeedhandler.conf /usr/local/nginx/conf/pagespeedhandler.conf
    sdiff -w200 /usr/local/src/centminmod/config/nginx/pagespeedstatslog.conf /usr/local/nginx/conf/pagespeedstatslog.conf
    sdiff -w200 /usr/local/src/centminmod/config/php-fpm/php-fpm.conf /usr/local/etc/php-fpm.conf
    sdiff -w200 /usr/local/src/centminmod/config/php-fpm/php-fpm-min.conf /usr/local/etc/php-fpm.conf
    

    if you only want to show differences add -s flag
    Code (Text):
    sdiff -w200 -s /usr/local/src/centminmod/config/nginx/nginx.conf /usr/local/nginx/conf/nginx.conf
    sdiff -w200 -s /usr/local/src/centminmod/config/nginx/conf.d/virtual.conf /usr/local/nginx/conf/conf.d/virtual.conf
    sdiff -w200 -s /usr/local/src/centminmod/config/nginx/staticfiles.conf /usr/local/nginx/conf/staticfiles.conf
    sdiff -w200 -s /usr/local/src/centminmod/config/nginx/php.conf /usr/local/nginx/conf/php.conf
    sdiff -w200 -s /usr/local/src/centminmod/config/nginx/drop.conf /usr/local/nginx/conf/drop.conf
    sdiff -w200 -s /usr/local/src/centminmod/config/nginx/errorpage.conf /usr/local/nginx/conf/errorpage.conf
    sdiff -w200 -s /usr/local/src/centminmod/config/nginx/pagespeed.conf /usr/local/nginx/conf/pagespeed.conf
    sdiff -w200 -s /usr/local/src/centminmod/config/nginx/pagespeedhandler.conf /usr/local/nginx/conf/pagespeedhandler.conf
    sdiff -w200 -s /usr/local/src/centminmod/config/nginx/pagespeedstatslog.conf /usr/local/nginx/conf/pagespeedstatslog.conf
    sdiff -w200 -s /usr/local/src/centminmod/config/php-fpm/php-fpm.conf /usr/local/etc/php-fpm.conf
    sdiff -w200 -s /usr/local/src/centminmod/config/php-fpm/php-fpm-min.conf /usr/local/etc/php-fpm.conf
    

    example sdiff differences only -s with width -w300

    upload_2017-5-19_6-48-40.png

    or use diff command
    Code (Text):
    diff -u /usr/local/src/centminmod/config/nginx/nginx.conf /usr/local/nginx/conf/nginx.conf
    diff -u /usr/local/src/centminmod/config/nginx/conf.d/virtual.conf /usr/local/nginx/conf/conf.d/virtual.conf
    diff -u /usr/local/src/centminmod/config/nginx/staticfiles.conf /usr/local/nginx/conf/staticfiles.conf
    diff -u /usr/local/src/centminmod/config/nginx/php.conf /usr/local/nginx/conf/php.conf
    diff -u /usr/local/src/centminmod/config/nginx/drop.conf /usr/local/nginx/conf/drop.conf
    diff -u /usr/local/src/centminmod/config/nginx/errorpage.conf /usr/local/nginx/conf/errorpage.conf
    diff -u /usr/local/src/centminmod/config/nginx/pagespeed.conf /usr/local/nginx/conf/pagespeed.conf
    diff -u /usr/local/src/centminmod/config/nginx/pagespeedhandler.conf /usr/local/nginx/conf/pagespeedhandler.conf
    diff -u /usr/local/src/centminmod/config/nginx/pagespeedstatslog.conf /usr/local/nginx/conf/pagespeedstatslog.conf
    diff -u /usr/local/src/centminmod/config/php-fpm/php-fpm.conf /usr/local/etc/php-fpm.conf
    diff -u /usr/local/src/centminmod/config/php-fpm/php-fpm-min.conf /usr/local/etc/php-fpm.conf
    

    Code (Text):
    diff -u /usr/local/src/centminmod/config/nginx/staticfiles.conf /usr/local/nginx/conf/staticfiles.conf
    

    output
    Code (Text):
    diff -u /usr/local/src/centminmod/config/nginx/staticfiles.conf /usr/local/nginx/conf/staticfiles.conf            
    --- /usr/local/src/centminmod/config/nginx/staticfiles.conf     2017-05-18 14:38:01.877605699 +0000
    +++ /usr/local/nginx/conf/staticfiles.conf      2017-04-14 13:28:00.648971866 +0000
    @@ -9,7 +9,7 @@
       #add_header X-Xss-Protection "1; mode=block" always;
       #add_header X-Content-Type-Options "nosniff" always;
       add_header Access-Control-Allow-Origin *;
    -  add_header Cache-Control "public, must-revalidate, proxy-revalidate, immutable, stale-while-revalidate=86400, stale-if-error=604800";
    +  add_header Cache-Control "public, must-revalidate, proxy-revalidate";
            access_log off;
            expires 30d;
            break;
    @@ -21,7 +21,7 @@
       #add_header X-Xss-Protection "1; mode=block" always;
       #add_header X-Content-Type-Options "nosniff" always;
       add_header Access-Control-Allow-Origin *;
    -  add_header Cache-Control "public, must-revalidate, proxy-revalidate, immutable, stale-while-revalidate=86400, stale-if-error=604800";
    +  add_header Cache-Control "public, must-revalidate, proxy-revalidate";
            access_log off;
            expires 30d;
            break;
    @@ -33,7 +33,7 @@
       #add_header X-Xss-Protection "1; mode=block" always;
       #add_header X-Content-Type-Options "nosniff" always;
       add_header Access-Control-Allow-Origin *;
    -  add_header Cache-Control "public, must-revalidate, proxy-revalidate, immutable, stale-while-revalidate=86400, stale-if-error=604800";
    +  add_header Cache-Control "public, must-revalidate, proxy-revalidate";
            access_log off;
            expires 30d;
            break;
    
     
    Last edited: May 19, 2017
    • Like Like x 1
  12. eva2000

    eva2000 Administrator Staff Member

    31,001
    6,919
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,424
    Local Time:
    11:50 PM
    Nginx 1.13.x
    MariaDB 5.5
    actually should reverse the files so you have local server copy first and centmin mod copy 2nd

    sdiff could cut off characters in display so diff -u command is more accurate
    Code (Text):
    sdiff -w200 /usr/local/nginx/conf/nginx.conf /usr/local/src/centminmod/config/nginx/nginx.conf
    sdiff -w200 /usr/local/nginx/conf/conf.d/virtual.conf /usr/local/src/centminmod/config/nginx/conf.d/virtual.conf
    sdiff -w200 /usr/local/nginx/conf/staticfiles.conf /usr/local/src/centminmod/config/nginx/staticfiles.conf
    sdiff -w200 /usr/local/nginx/conf/php.conf /usr/local/src/centminmod/config/nginx/php.conf
    sdiff -w200 /usr/local/nginx/conf/drop.conf /usr/local/src/centminmod/config/nginx/drop.conf
    sdiff -w200 /usr/local/nginx/conf/errorpage.conf /usr/local/src/centminmod/config/nginx/errorpage.conf
    sdiff -w200 /usr/local/nginx/conf/pagespeed.conf /usr/local/src/centminmod/config/nginx/pagespeed.conf
    sdiff -w200 /usr/local/nginx/conf/pagespeedhandler.conf /usr/local/src/centminmod/config/nginx/pagespeedhandler.conf
    sdiff -w200 /usr/local/nginx/conf/pagespeedstatslog.conf /usr/local/src/centminmod/config/nginx/pagespeedstatslog.conf
    sdiff -w200 /usr/local/etc/php-fpm.conf /usr/local/src/centminmod/config/php-fpm/php-fpm.conf
    sdiff -w200 /usr/local/etc/php-fpm.conf /usr/local/src/centminmod/config/php-fpm/php-fpm-min.conf
    

    Code (Text):
    sdiff -w200 -s /usr/local/nginx/conf/nginx.conf /usr/local/src/centminmod/config/nginx/nginx.conf
    sdiff -w200 -s /usr/local/nginx/conf/conf.d/virtual.conf /usr/local/src/centminmod/config/nginx/conf.d/virtual.conf
    sdiff -w200 -s /usr/local/nginx/conf/staticfiles.conf /usr/local/src/centminmod/config/nginx/staticfiles.conf
    sdiff -w200 -s /usr/local/nginx/conf/php.conf /usr/local/src/centminmod/config/nginx/php.conf
    sdiff -w200 -s /usr/local/nginx/conf/drop.conf /usr/local/src/centminmod/config/nginx/drop.conf
    sdiff -w200 -s /usr/local/nginx/conf/errorpage.conf /usr/local/src/centminmod/config/nginx/errorpage.conf
    sdiff -w200 -s /usr/local/nginx/conf/pagespeed.conf /usr/local/src/centminmod/config/nginx/pagespeed.conf
    sdiff -w200 -s /usr/local/nginx/conf/pagespeedhandler.conf /usr/local/src/centminmod/config/nginx/pagespeedhandler.conf
    sdiff -w200 -s /usr/local/nginx/conf/pagespeedstatslog.conf /usr/local/src/centminmod/config/nginx/pagespeedstatslog.conf
    sdiff -w200 -s /usr/local/etc/php-fpm.conf /usr/local/src/centminmod/config/php-fpm/php-fpm.conf
    sdiff -w200 -s /usr/local/etc/php-fpm.conf /usr/local/src/centminmod/config/php-fpm/php-fpm-min.conf
    

    Code (Text):
    diff -u /usr/local/nginx/conf/nginx.conf /usr/local/src/centminmod/config/nginx/nginx.conf
    diff -u /usr/local/nginx/conf/conf.d/virtual.conf /usr/local/src/centminmod/config/nginx/conf.d/virtual.conf
    diff -u /usr/local/nginx/conf/staticfiles.conf /usr/local/src/centminmod/config/nginx/staticfiles.conf
    diff -u /usr/local/nginx/conf/php.conf /usr/local/src/centminmod/config/nginx/php.conf
    diff -u /usr/local/nginx/conf/drop.conf /usr/local/src/centminmod/config/nginx/drop.conf
    diff -u /usr/local/nginx/conf/errorpage.conf /usr/local/src/centminmod/config/nginx/errorpage.conf
    diff -u /usr/local/nginx/conf/pagespeed.conf /usr/local/src/centminmod/config/nginx/pagespeed.conf
    diff -u /usr/local/nginx/conf/pagespeedhandler.conf /usr/local/src/centminmod/config/nginx/pagespeedhandler.conf
    diff -u /usr/local/nginx/conf/pagespeedstatslog.conf /usr/local/src/centminmod/config/nginx/pagespeedstatslog.conf
    diff -u /usr/local/etc/php-fpm.conf /usr/local/src/centminmod/config/php-fpm/php-fpm.conf
    diff -u /usr/local/etc/php-fpm.conf /usr/local/src/centminmod/config/php-fpm/php-fpm-min.conf
    

    example
    Code (Text):
    diff -u /usr/local/nginx/conf/staticfiles.conf /usr/local/src/centminmod/config/nginx/staticfiles.conf
    --- /usr/local/nginx/conf/staticfiles.conf      2017-04-14 13:28:00.648971866 +0000
    +++ /usr/local/src/centminmod/config/nginx/staticfiles.conf     2017-05-18 14:38:01.877605699 +0000
    @@ -9,7 +9,7 @@
       #add_header X-Xss-Protection "1; mode=block" always;
       #add_header X-Content-Type-Options "nosniff" always;
       add_header Access-Control-Allow-Origin *;
    -  add_header Cache-Control "public, must-revalidate, proxy-revalidate";
    +  add_header Cache-Control "public, must-revalidate, proxy-revalidate, immutable, stale-while-revalidate=86400, stale-if-error=604800";
            access_log off;
            expires 30d;
            break;
    @@ -21,7 +21,7 @@
       #add_header X-Xss-Protection "1; mode=block" always;
       #add_header X-Content-Type-Options "nosniff" always;
       add_header Access-Control-Allow-Origin *;
    -  add_header Cache-Control "public, must-revalidate, proxy-revalidate";
    +  add_header Cache-Control "public, must-revalidate, proxy-revalidate, immutable, stale-while-revalidate=86400, stale-if-error=604800";
            access_log off;
            expires 30d;
            break;
    @@ -33,7 +33,7 @@
       #add_header X-Xss-Protection "1; mode=block" always;
       #add_header X-Content-Type-Options "nosniff" always;
       add_header Access-Control-Allow-Origin *;
    -  add_header Cache-Control "public, must-revalidate, proxy-revalidate";
    +  add_header Cache-Control "public, must-revalidate, proxy-revalidate, immutable, stale-while-revalidate=86400, stale-if-error=604800";
            access_log off;
            expires 30d;
            break;
    

    means
    Code (Text):
    -  add_header Cache-Control "public, must-revalidate, proxy-revalidate";
    +  add_header Cache-Control "public, must-revalidate, proxy-revalidate, immutable, stale-while-revalidate=86400, stale-if-error=604800";
    

    replace all 3 instances of
    Code (Text):
    add_header Cache-Control "public, must-revalidate, proxy-revalidate";
    

    with
    Code (Text):
    add_header Cache-Control "public, must-revalidate, proxy-revalidate, immutable, stale-while-revalidate=86400, stale-if-error=604800";
    
     
  13. eva2000

    eva2000 Administrator Staff Member

    31,001
    6,919
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,424
    Local Time:
    11:50 PM
    Nginx 1.13.x
    MariaDB 5.5
  14. tjk

    tjk Member

    74
    16
    8
    Jun 27, 2015
    Ratings:
    +25
    Local Time:
    8:50 AM
    @eva2000 I think at one point you said you manage 100+ live centminmod servers, what method do you use to update those conf files when you make changes?

    I still think there should be a master conf file that is always updated with your changes and then an include that has all the admin changes, where possible.

    I've been playing around with runcloud, and that is how they sort of do it. They overwrite all their conf files when/if there are changes, and put in includes for user changes.
     
  15. eva2000

    eva2000 Administrator Staff Member

    31,001
    6,919
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,424
    Local Time:
    11:50 PM
    Nginx 1.13.x
    MariaDB 5.5
    some don't get all the minor cosmetic updates while some i setup private git repository for config files and manage the changes via git which auto sync the changes to dozens of servers without my intervention every 8hrs. I can only do that as I know my before and after custom changes/settings to my live server config files and they're tracked by git commit history records similar to one example update staticfiles.conf include file cache-control headers · centminmod/centminmod@470bb72 · GitHub and commit history for that specific file History for config/nginx/staticfiles.conf - centminmod/centminmod · GitHub so that every server config change I ever make has a commit history log

    example is this staticfiles.con include file change was committed to my private git repo for my live server config for 40+ servers and it automatically gets synced to my 40+ servers every 8 hrs or i can manually do a simultaneous sync to them when i need to.
    problem is user changes and include files can all go out the door as some users totally change their configs from what centmin mod nginx uses by wiping centmin mod set and using their own. That latter environment is hard to migrate user changes over to.
     
    Last edited: May 19, 2017
  16. Jimmy

    Jimmy Premium Member Premium Member

    1,165
    256
    83
    Oct 24, 2015
    East Coast USA
    Ratings:
    +626
    Local Time:
    8:50 AM
    1.13.x
    MariaDB 10.1.x
    Sorry @eva2000 looks like I opened a can of worms with my first response.
     
  17. eva2000

    eva2000 Administrator Staff Member

    31,001
    6,919
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,424
    Local Time:
    11:50 PM
    Nginx 1.13.x
    MariaDB 5.5
    Unfortunately, it's tricky as the other side of the argument would be folks not happy with their own customised config files being overwritten or broken by Centmin Mod auto syncing updates post-initial install. I would be one of those folks, if auto updated config files broke my custom setups :)

    Above outlined diff/sdiff would be easiest way to check for yourself changes. If you familiar with git/svn versioning frameworks, you can also setup your own git repo privately to track and update changes to your server configs.
     
  18. Jimmy

    Jimmy Premium Member Premium Member

    1,165
    256
    83
    Oct 24, 2015
    East Coast USA
    Ratings:
    +626
    Local Time:
    8:50 AM
    1.13.x
    MariaDB 10.1.x
    Oh, I'm with you, I wouldn't want CMM to just go writing over files I modified. That would be bad. I think if you have a stock text which you could throw up in thread which a user might need to make changes, I know for myself, I would be alerted to that fact. There might be a file in my local setup which I never touched and don't even realize that needs to be updated... that's my biggest fear. I've made modifications to my staticfiles.conf file in the past and I'm familiar with the file, so I knew this one needed to be updated when I saw your notice. But other ones could fly by me.
     
  19. eva2000

    eva2000 Administrator Staff Member

    31,001
    6,919
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,424
    Local Time:
    11:50 PM
    Nginx 1.13.x
    MariaDB 5.5
  20. Jimmy

    Jimmy Premium Member Premium Member

    1,165
    256
    83
    Oct 24, 2015
    East Coast USA
    Ratings:
    +626
    Local Time:
    8:50 AM
    1.13.x
    MariaDB 10.1.x
    I'm on 09 and have only just deployed a production server. So, I'm pretty up to date. Plus I use Phpstorm to manage my files which has version control. I'll just have to pay closer attention.
     
    • Like Like x 1