Want more timely Centmin Mod News Updates?
Become a Member

504 Gateway Timeout during installation using Duplicator

Discussion in 'Blogs & CMS usage' started by KlueMaster, Aug 21, 2020.

  1. KlueMaster

    KlueMaster Member

    53
    10
    8
    Aug 5, 2017
    Ratings:
    +20
    Local Time:
    5:39 PM
    MariaDB 10
    Hi,

    I'm trying to migrate my site from a vestacp server to CMM server on a different hosting provider. I'm using Duplicator for this purpose.

    Now, during the site installation on CMM, on step 2 - database installation, I'm getting 504 gateway timeout during database table copy, probably due to huge size of database (1.6 GB compressed).

    Any suggestion as to how can I get rid of the issue and do a clean migration?

    Thanks!
     
  2. eva2000

    eva2000 Administrator Staff Member

    45,201
    10,280
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +15,934
    Local Time:
    10:09 PM
    Nginx 1.19.x
    MariaDB 5.5/10.x
    PHP backup/restore tools are notorious for timing out if you're working with large databases. You may not even have a proper backup to begin with if you backup timed out and had an incomplete backup. So do not delete your original server too soon as you might end up finding out the imported data isn't fully complete in the first place!

    PHP Limits



    Check your upload_max_filesize, post_max_size, memory_limit and max_execution_time in php via
    Code (Text):
    php -i | egrep 'upload_max_filesize|post_max_size|memory_limit|max_execution_time'
    

    Centmin Mod will dynamically configure these values at initial install time based on detected server resources i.e. cpu, memory but you may need to adjust based on your web app's usage requirements.

    Note max_execution_time = 0 means unlimited but your web app (PHP scripts) may have some hardcoded values limiting them from within their scripts too.
    Code (Text):
    php -i | egrep 'upload_max_filesize|post_max_size|memory_limit|max_execution_time'
    max_execution_time => 0 => 0
    memory_limit => 1024M => 1024M
    post_max_size => 1024M => 1024M
    upload_max_filesize => 1024M => 1024M
    

    PHP: Description of core php.ini directives - Manual

    For custom php settings read centminmod.com/phpfpm.html#customphpini. You might want to create /etc/centminmod/php.d/b_customphp.ini to override centmin mod defaults at /etc/centminmod/php.d/a_customphp.ini and restart PHP-FPM. Such changes will persist when doing centmin mod upgrades.

    Also outlined in Getting Started Guide Step 17

    Nginx/PHP-FPM Limits



    For Nginx, there's client_max_body_size setting contained in main /usr/local/nginx/conf/nginx.conf outlined at Centmin Mod Configuration Files - CentminMod.com LEMP Nginx web stack for CentOS
    Code (Text):
    grep 'client_max_body_size' /usr/local/nginx/conf/nginx.conf
    client_max_body_size 1024m;
    


    PHP-FPM config file at /usr/local/nginx/conf/php.conf has default limits for
    Code (Text):
    fastcgi_send_timeout 180;
    fastcgi_read_timeout 180;
    

    which may need raising. For changes to take affect, need to restart both nginx and php-fpm

    command shortcut to restart both nginx + php-fpm at same time
    Code (Text):
    nprestart


    Cloudflare HTTP POST Upload Limits



    If you're using Cloudflare in front of your site, then they have their own HTTP POST upload max limits of 100MB for free and pro plan, 200MB for business plan and 500MB for enterprise plans see Understanding Cloudflare's CDN. 2 workarounds are offered by Cloudflare in that article and below where one is disable orange cloud proxy for upload url/paths to directly allow visitors to your origin Nginx server for uploads.
    Or instead of disabling Cloudflare orange cloud proxy, you can just edit your local DNS hosts files on your local PC/Linux computer to bypass Cloudflare DNS and tell your PC to connect your domain to real origin Nginx server IP address similar to post outlined at Upgrade - Nginx - Insight Guide - Centmin Mod Site Data Migration Guide. Be sure to reverse the changes after you've done the upload.
     
    Last edited: Aug 22, 2020
  3. KlueMaster

    KlueMaster Member

    53
    10
    8
    Aug 5, 2017
    Ratings:
    +20
    Local Time:
    5:39 PM
    MariaDB 10
    Thanks for such a detailed informative post.

    Although, I solved it a little differently. I excluded the large non core tables during migration, and later dumped them manually. This exclusion resulted in smaller package size which allowed installer to work without any problems.

    EDIT: Just in case someone else tries to do it this way, a word of caution. You cannot do it with core wordpress tables (highlighted in dark red color in Duplicator interface). And you need to dump the excluded tables manually into the database before running Step 3 of installer. No guarantees!
     
    Last edited: Aug 22, 2020
  4. eva2000

    eva2000 Administrator Staff Member

    45,201
    10,280
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +15,934
    Local Time:
    10:09 PM
    Nginx 1.19.x
    MariaDB 5.5/10.x
    Yup you can do it that way too :)
     
  5. Salmon Skin

    Salmon Skin New Member

    15
    1
    3
    Oct 15, 2019
    Ratings:
    +7
    Local Time:
    2:09 PM
    1.17.4
    10.4
    but what if you have a different hardcoded path in your new server? (or even, different domain).
    manually exporting and importing the database without proper search&replace is just as bad. can be even worse if you do it improperly, e.g by changing GUID's .

    best backup is file backup or full image backup, making sure that this backup includes mysqldump files, because u can't just copy over mysql folders and call it a day.
    use a cron to run mysqlbackup, or install automysqlbackup (old but still working).

    not a fan of duplicator, or any other wp plugins. but had better experience with all-in-one wp migration, and migration guru
     
  6. grouthalved

    grouthalved New Member

    4
    0
    1
    Jul 30, 2020
    Ratings:
    +0
    Local Time:
    5:09 AM
    MariaDB 10
    Umm, why not do a two part installation?