/ Register

Want to subscribe to topics you're interested in?
Become a Member

Error when execute script

Discussion in 'Nginx, PHP-FPM & MariaDB MySQL' started by Sunka, Dec 1, 2015.

Tags:
Previous Thread Next Thread
Loading...
Page 1 of 3
  1. Dec 1, 2015 #1
    Sunka

    Sunka Well-Known Member

    1,150
    325
    83
    Oct 31, 2015
    Pula, Croatia
    Ratings:
    +525
    Local Time:
    7:50 AM
    Nginx 1.17.9
    MariaDB 10.3.22
    During backup of all files under public (about 10 gb of files) it shows me an error in xenforo ACP.
    Script gzip all files under /..public and then transfer them to backup server.
    I can see that files backup exist, and they are transfered to backup server but sizes are not the same (9,75GB vs 9,82GB)
    Also, settings in custom php.ini is max_execution_time = 240 so I do not now why error here show Maximum execution time of 120 seconds exceeded

    Edit: I try to set time to 400 and run cron manually and after exactly 180 sec, it shows me 504 Gateway Time. So somewhere is another variable for maximum execution time to 180 second which I need to change. But where?

    Code:
    Server Error Log
Error Info
ErrorException: Fatal Error: Maximum execution time of 120 seconds exceeded - library/SolidMean/ForumBackup/PHPSSH2/SFTP.php:128
Generated By: Unknown Account, 55 minutes ago
Stack Trace

#0 [internal function]: XenForo_Application::handleFatalError()
#1 {main}

Request State

array(3) {
  ["url"] => string(39) "https://www.pijanitvor.com/deferred.php"
  ["_GET"] => array(0) {
  }
  ["_POST"] => array(3) {
    ["_xfRequestUri"] => string(20) "/threads/mravi.1346/"
    ["_xfNoRedirect"] => string(1) "1"
    ["_xfResponseType"] => string(4) "json"
  }
}

     
    Last edited: Dec 1, 2015
  2. Dec 1, 2015 #2
    eva2000

    eva2000 Administrator Staff Member

    55,237
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,833
    Local Time:
    3:50 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    For custom php settings read centminmod.com/phpfpm.html#customphpini. You might want to create /etc/centminmod/php.d/b_customphp.ini to override centmin mod defaults at /etc/centminmod/php.d/a_customphp.ini and restart PHP-FPM.

    That's the inherent problem of using php scripts to backup data it's slower and uses more resources and/or blocks resources i.e. php time than using pure shell based scripts. As data set size increases, the less reliable php based backup scripts get. It's why i don't use phpmyadmin to backup databases as you have same issues outlined at Addon phpmyadmin.sh - CentminMod.com LEMP Nginx web stack for CentOS under Troubleshooting section.

    For very very large data sets, do you really want to risk backup process timing out or not completing and thus resulting in incomplete backups ?

    It's reason why shell based scripts like dbbackup.sh were written MariaDB - dbbackup.sh - quick mysql database backups for Centmin Mod stack | Centmin Mod Community :)
     
    Last edited: Dec 1, 2015
  3. Dec 1, 2015 #3
    Sunka

    Sunka Well-Known Member

    1,150
    325
    83
    Oct 31, 2015
    Pula, Croatia
    Ratings:
    +525
    Local Time:
    7:50 AM
    Nginx 1.17.9
    MariaDB 10.3.22
    It is files backup, not database. Look here

    And solution two post below linked one.
    I have raised in /etc/centminmod/php.d/zzz_customphp.ini max_execution_time = 400
     
  4. Dec 1, 2015 #4
    eva2000

    eva2000 Administrator Staff Member

    55,237
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,833
    Local Time:
    3:50 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    yeah dbbackup.sh will later be evolved to include site files ;)
     
  5. Dec 1, 2015 #5
    Sunka

    Sunka Well-Known Member

    1,150
    325
    83
    Oct 31, 2015
    Pula, Croatia
    Ratings:
    +525
    Local Time:
    7:50 AM
    Nginx 1.17.9
    MariaDB 10.3.22
    What about this solution posted here
     
  6. Dec 1, 2015 #6
    eva2000

    eva2000 Administrator Staff Member

    55,237
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,833
    Local Time:
    3:50 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    centmin mod timeout outlined at [SolidMean] ForumBackup SFTP Transfer [Paid] | Page 2 | XenForo Community are already higher than that and not related your error is 'Fatal Error: Maximum execution time'

    you can edit them in /usr/local/nginx/conf/php.conf for fastcgi not proxy

    Code:
    fastcgi_connect_timeout 60;
fastcgi_send_timeout 180;
fastcgi_read_timeout 180;
    also preview from private Premium members forum of what it may look like when combining db and files backup :)

     
    Last edited: Dec 1, 2015
  7. Dec 1, 2015 #7
    Sunka

    Sunka Well-Known Member

    1,150
    325
    83
    Oct 31, 2015
    Pula, Croatia
    Ratings:
    +525
    Local Time:
    7:50 AM
    Nginx 1.17.9
    MariaDB 10.3.22
    So where is problem exactly?
    What variable to change?
    Database backup is working Ok, it is files backup that failed, and if I understood that errors, it is failed after files backup is created, in process of moving to remote server
    Yep, but sadly I do not have enough money to buy premium membership :cry:
    SneakDave's addon give me chance to make database and files backup, rotate them (keep only x number of backups and delete old one) and more important give me solution to automatically transfer backups to Dropbox, GoogleDrive and remote server. Because of google drive, I bought his addon
     
  8. Dec 1, 2015 #8
    Sunka

    Sunka Well-Known Member

    1,150
    325
    83
    Oct 31, 2015
    Pula, Croatia
    Ratings:
    +525
    Local Time:
    7:50 AM
    Nginx 1.17.9
    MariaDB 10.3.22
    I see you edit post.
    So, Should I change that 3 variables to 400?
     
  9. Dec 1, 2015 #9
    eva2000

    eva2000 Administrator Staff Member

    55,237
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,833
    Local Time:
    3:50 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Problem is due to using php to run backups, php and nginx have timeouts and restrictions to prevent rogue or long running scripts or backends from queuing and overloading resources. Large data sets will take longer to backup and thus hog php resources so negatively effects all sites on server that run php. Raising timeouts only makes the negative effects last longer for all sites on server.

    /usr/local/nginx/conf/php.conf is server wide so ideally you shouldn't as it applies to all nginx vhosts which include the file /usr/local/nginx/conf/php.conf

    you should instead make a custom copy of your /usr/local/nginx/conf/php.conf i.e. /usr/local/nginx/conf/phpforbackups.conf

    Code:
    cp -a /usr/local/nginx/conf/php.conf /usr/local/nginx/conf/phpforbackups.conf
    and only in a specific vhost for xenforo include it in a specified location context for the php code that runs your backup and edit that file's /usr/local/nginx/conf/phpforbackups.conf values for

    Code:
    fastcgi_connect_timeout 60;
fastcgi_send_timeout 180;
fastcgi_read_timeout 180;
    i.e. any php files located in /library/SolidMean/ForumBackup/ will use /usr/local/nginx/conf/phpforbackups.conf based settings and timeouts
    Code:
    location ~ ^/library/SolidMean/ForumBackup/ {
    include /usr/local/nginx/conf/phpforbackups.conf;
}
    restart nginx and php-fpm
    Code:
    nprestart
     
    Last edited: Dec 1, 2015
  10. Dec 1, 2015 #10
    Sunka

    Sunka Well-Known Member

    1,150
    325
    83
    Oct 31, 2015
    Pula, Croatia
    Ratings:
    +525
    Local Time:
    7:50 AM
    Nginx 1.17.9
    MariaDB 10.3.22
    That will be /usr/local/nginx/conf/conf.d/pijanitvor.com.ssl.conf file if I understand your guide?

    Code:
    [root@tvor-ocean conf.d]# cat pijanitvor.com.ssl.conf
# Centmin Mod Getting Started Guide
# must read http://centminmod.com/getstarted.html
# For SPDY SSL Setup
# read http://centminmod.com/nginx_configure_https_ssl_spdy.html

# redirect from www to non-www  forced SSL
# uncomment, save file and restart Nginx to enable
# if unsure use return 302 before using return 301
server {
   server_name pijanitvor.com www.pijanitvor.com;
    return 301 https://www.$server_name$request_uri;
}

server {
  listen 443 ssl http2;
  server_name pijanitvor.com www.pijanitvor.com;

  ssl_dhparam /usr/local/nginx/conf/ssl/pijanitvor.com/dhparam.pem;
  ssl_certificate      /usr/local/nginx/conf/ssl/pijanitvor.com/ssl-unified.crt;
  ssl_certificate_key  /usr/local/nginx/conf/ssl/pijanitvor.com/pijanitvor.com.key;
  include /usr/local/nginx/conf/ssl_include.conf;

  # mozilla recommended
  ssl_ciphers ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA:!CAMELLIA:!DES-CBC3-SHA;
  ssl_prefer_server_ciphers   on;
  ##add_header Alternate-Protocol  443:npn-spdy/3;
  #add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;";
  #add_header  X-Content-Type-Options "nosniff";
  #add_header X-Frame-Options DENY;
  ##spdy_headers_comp 5;
  ssl_buffer_size 1400;
  ssl_session_tickets on;
 
  #enable ocsp stapling
  resolver 8.8.8.8 8.8.4.4 valid=10m;
  resolver_timeout 10s;
  ssl_stapling on;
  ssl_stapling_verify on;
  ssl_trusted_certificate /usr/local/nginx/conf/ssl/pijanitvor.com/ssl-trusted.crt; 

# ngx_pagespeed & ngx_pagespeed handler
#include /usr/local/nginx/conf/pagespeed.conf;
#include /usr/local/nginx/conf/pagespeedhandler.conf;
#include /usr/local/nginx/conf/pagespeedstatslog.conf;

  # limit_conn limit_per_ip 16;
  # ssi  on;

  access_log /home/nginx/domains/pijanitvor.com/log/access.log combined buffer=256k flush=60m;
  error_log /home/nginx/domains/pijanitvor.com/log/error.log;

  root /home/nginx/domains/pijanitvor.com/public;

  location / {
        index index.php index.html index.htm;
        try_files $uri $uri/ /index.php?$uri&$args;
    }

    location /internal_data/ {
        internal;
        allow 127.0.0.1;
        allow 46.101.130.248;
        deny all;
    }

    location /library/ {
        internal;
        allow 127.0.0.1;
        allow 46.101.130.248;
        deny all;
    }
 
  # prevent access to ./directories and files
        location ~ (?:^|/)\. {
   deny all;
        } 

  #location / {

# block common exploits, sql injections etc
#include /usr/local/nginx/conf/block.conf;

  # Enables directory listings when index file not found
  #autoindex  on;

  # Shows file listing times as local time
  #autoindex_localtime on;

  # Enable for vBulletin usage WITHOUT vbSEO installed
  # More example Nginx vhost configurations at
  # http://centminmod.com/nginx_configure.html
  #try_files    $uri $uri/ /index.php;

  #}

  include /usr/local/nginx/conf/staticfiles.conf;
  include /usr/local/nginx/conf/php.conf;
  include /usr/local/nginx/conf/drop.conf;
  #include /usr/local/nginx/conf/errorpage.conf;
  include /usr/local/nginx/conf/vts_server.conf;
}
    Where to put that:
    Code:
    location ~ ^/library/SolidMean/ForumBackup/ {
    include /usr/local/nginx/conf/phpforbackups.conf;
}
     
  11. Dec 1, 2015 #11
    eva2000

    eva2000 Administrator Staff Member

    55,237
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,833
    Local Time:
    3:50 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    yup

    maybe after the library location {} context
     
  12. Dec 1, 2015 #12
    Sunka

    Sunka Well-Known Member

    1,150
    325
    83
    Oct 31, 2015
    Pula, Croatia
    Ratings:
    +525
    Local Time:
    7:50 AM
    Nginx 1.17.9
    MariaDB 10.3.22
    So it will be:

    Code:
    # Centmin Mod Getting Started Guide
# must read http://centminmod.com/getstarted.html
# For SPDY SSL Setup
# read http://centminmod.com/nginx_configure_https_ssl_spdy.html

# redirect from www to non-www  forced SSL
# uncomment, save file and restart Nginx to enable
# if unsure use return 302 before using return 301
server {
   server_name pijanitvor.com www.pijanitvor.com;
    return 301 https://www.$server_name$request_uri;
}

server {
  listen 443 ssl http2;
  server_name pijanitvor.com www.pijanitvor.com;

  ssl_dhparam /usr/local/nginx/conf/ssl/pijanitvor.com/dhparam.pem;
  ssl_certificate      /usr/local/nginx/conf/ssl/pijanitvor.com/ssl-unified.crt;
  ssl_certificate_key  /usr/local/nginx/conf/ssl/pijanitvor.com/pijanitvor.com.key;
  include /usr/local/nginx/conf/ssl_include.conf;

  # mozilla recommended
  ssl_ciphers ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA:!CAMELLIA:!DES-CBC3-SHA;
  ssl_prefer_server_ciphers   on;
  ##add_header Alternate-Protocol  443:npn-spdy/3;
  #add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;";
  #add_header  X-Content-Type-Options "nosniff";
  #add_header X-Frame-Options DENY;
  ##spdy_headers_comp 5;
  ssl_buffer_size 1400;
  ssl_session_tickets on;
  #enable ocsp stapling
  resolver 8.8.8.8 8.8.4.4 valid=10m;
  resolver_timeout 10s;
  ssl_stapling on;
  ssl_stapling_verify on;
  ssl_trusted_certificate /usr/local/nginx/conf/ssl/pijanitvor.com/ssl-trusted.crt;

# ngx_pagespeed & ngx_pagespeed handler
#include /usr/local/nginx/conf/pagespeed.conf;
#include /usr/local/nginx/conf/pagespeedhandler.conf;
#include /usr/local/nginx/conf/pagespeedstatslog.conf;

  # limit_conn limit_per_ip 16;
  # ssi  on;

  access_log /home/nginx/domains/pijanitvor.com/log/access.log combined buffer=256k flush=60m;
  error_log /home/nginx/domains/pijanitvor.com/log/error.log;

  root /home/nginx/domains/pijanitvor.com/public;

  location / {
        index index.php index.html index.htm;
        try_files $uri $uri/ /index.php?$uri&$args;
    }

    location /internal_data/ {
        internal;
        allow 127.0.0.1;
        allow 46.101.130.248;
        deny all;
    }

    location /library/ {
        internal;
        allow 127.0.0.1;
        allow 46.101.130.248;
        deny all;
    }

   location ~ ^/library/SolidMean/ForumBackup/ {
    include /usr/local/nginx/conf/phpforbackups.conf;
   }

  # prevent access to ./directories and files
        location ~ (?:^|/)\. {
   deny all;
        }

  include /usr/local/nginx/conf/staticfiles.conf;
  include /usr/local/nginx/conf/php.conf;
  include /usr/local/nginx/conf/drop.conf;
  #include /usr/local/nginx/conf/errorpage.conf;
  include /usr/local/nginx/conf/vts_server.conf;
}
     
  13. Dec 1, 2015 #13
    Sunka

    Sunka Well-Known Member

    1,150
    325
    83
    Oct 31, 2015
    Pula, Croatia
    Ratings:
    +525
    Local Time:
    7:50 AM
    Nginx 1.17.9
    MariaDB 10.3.22
    So steps will be:

    Code:
    cp -a /usr/local/nginx/conf/php.conf /usr/local/nginx/conf/phpforbackups.conf
    Code:
    nano /usr/local/nginx/conf/phpforbackups.conf
    edit that three variables to 400 or maybe less or more?
    Code:
    fastcgi_connect_timeout 60;
fastcgi_send_timeout 180;
fastcgi_read_timeout 180;
    then edit pijanitvor.com.ssl.conf to be like in post above
    Code:
    nano /usr/local/nginx/conf/conf.d/pijanitvor.com.ssl.conf
    And then restart nginx and php-fpm
    Code:
    nprestart
     
  14. Dec 1, 2015 #14
    eva2000

    eva2000 Administrator Staff Member

    55,237
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,833
    Local Time:
    3:50 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    yeah you might want to adjust it to
    Code:
       location ~ ^/library/SolidMean/ForumBackup/ {
        include /usr/local/nginx/conf/phpforbackups.conf;
        internal;
        allow 127.0.0.1;
        allow 46.101.130.248;
        deny all;
   }
     
  15. Dec 2, 2015 #15
    rdan

    rdan Well-Known Member

    5,450
    1,410
    113
    May 25, 2014
    Ratings:
    +2,204
    Local Time:
    1:50 PM
    Mainline
    10.2
  16. Dec 2, 2015 #16
    eva2000

    eva2000 Administrator Staff Member

    55,237
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,833
    Local Time:
    3:50 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
  17. Dec 2, 2015 #17
    rdan

    rdan Well-Known Member

    5,450
    1,410
    113
    May 25, 2014
    Ratings:
    +2,204
    Local Time:
    1:50 PM
    Mainline
    10.2
    max_execution_time for php.ini
    request_terminate_timeout for php-fpm.conf
    fastcgi_read_timeout for php.conf

    Can be applied to Centmin based also.
     
  18. Dec 2, 2015 #18
    eva2000

    eva2000 Administrator Staff Member

    55,237
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,833
    Local Time:
    3:50 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Yes just file locations differ. For custom php settings read centminmod.com/phpfpm.html#customphpini. You might want to create /etc/centminmod/php.d/b_customphp.ini to override centmin mod defaults at /etc/centminmod/php.d/a_customphp.ini and restart PHP-FPM. Otherwise changes to php.ini will be lost of updates or override by a_customphp.ini
     
  19. Dec 2, 2015 #19
    Sunka

    Sunka Well-Known Member

    1,150
    325
    83
    Oct 31, 2015
    Pula, Croatia
    Ratings:
    +525
    Local Time:
    7:50 AM
    Nginx 1.17.9
    MariaDB 10.3.22
    Yep, I have custom php.ini and in there I setup /etc/centminmod/php.d/zzz_customphp.ini
    max_execution_time = 600
    So settings in there will not be overrided when php will be upgraded
    -------------------------------------------------------

    If bellow settings I add to /usr/local/nginx/conf/phpforbackups.conf and connected like @eva2000 said
    Code:
    fastcgi_connect_timeout 600;
fastcgi_send_timeout 600;
fastcgi_read_timeout 600;
    What then about @RoldanLT suggestion?
    It is disabled by default, so I think it is not necessary to change (raise) that variable, or I am wrong?
     
  20. Dec 2, 2015 #20
    eva2000

    eva2000 Administrator Staff Member

    55,237
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,833
    Local Time:
    3:50 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    no need to set it
     
Previous Thread Next Thread
Loading...
Page 1 of 3

.