Welcome to Centmin Mod Community
Become a Member

Wordpress HTTP Error 524 / Cloudflare & Wordpress All Import

Discussion in 'Blogs & CMS usage' started by tininho, Jun 2, 2019.

  1. tininho

    tininho Premium Member Premium Member

    17
    1
    3
    May 22, 2019
    Ratings:
    +7
    Local Time:
    3:17 PM
    Nginx Ver: 1.17
    MariaDB Ver: 10.4
    My goal is to get most out of Hetzner PX62-NVME for a single Woocommerce site.

    More so, to get the WP All Import plugin to handle the 23.000 SKU from different providers (but one CSV/XML file) efficiently, without choking the server.

    After months of testing different providers and stacks, I have estabilished that Hetzner + Centminmod leads to best WPPerformanceTester score.

    Problem now:


    When I run WP All Import with "High Speed Small File Processing" or Iterative, Piece-by-Piece Processing with iteration count >10 I get error HTTP Error 524

    HTTP Error 524

    The Cloudflare HTTP timeout is a network wide setting that can't be altered, if you need to have scripts that run for longer than around 1m40 without returning any data we will present Error 524: A timeout occurred.
    I have paid package from Cloudflare, so I asked their support. Their answer:

    1. Run the scripts via a grey-clouded subdomain

    2. Change the script so that it kicks off a long-running background process, and quickly returns a status which the browser can poll until the background process has completed, at which point the full response can be returned. This is the way most people do this type of activity these days - as keeping HTTP connections open for a long time is unreliable and can be very taxing also.

    3. If you really need this feature, we can increase the value for our Enterprise customers as part of a custom set up
    1 and 3 are out of question, obviously. Solution 2 is the way to go, but I have not succeeded yet. I have tried:

    add_action('wp_loaded', 'wpai_wp_loaded', 10);
    function wpai_wp_loaded() {
    if (isset($_GET['import_key']) || isset($_GET['id']) && isset($_GET['action']) && $_GET['action'] == 'process') {
    echo str_pad('', 4096, "\n");
    }
    }​

    - marcialpaulg/Fixing-Cloudflare-Error-524

    Settings changed:

    /etc/centminmod/php.d/a_customphp.ini
    max_execution_time = 600
    upload_max_filesize = 1280M
    memory_limit = 2048M
    post_max_size = 1280M
    max_input_vars = 10000

    /usr/local/etc/php-fpm.conf
    pm = static
    pm.max_children = 300
    pm.max_requests = 0

    None of this helped. With Cloudflare enabled I can get only <10 per iteration, which is weak and slow. Any ideas how to tackle this?

    Edit: no errors in any of the log files.


    -----
    Persistent Config: https://pastebin.com/2iAuuHMv
    CentOS Version: CentOS 7.6
    Centmin Mod Version Installed: 123.09beta01
    Nginx Version Installed: 1.17
    PHP Version Installed: 7.3.5
    MariaDB MySQL Version Installed: 10.4.x
    When was last time updated Centmin Mod code base ?: Today 2.6.2019
     
    Last edited: Jun 2, 2019
  2. eva2000

    eva2000 Administrator Staff Member

    40,624
    9,016
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +13,883
    Local Time:
    10:17 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    Yup as per Cloudflare HTTP Error 524 info at https://support.cloudflare.com/hc/en-us/articles/200171926-Error-524-A-timeout-occurred.