Want to subscribe to topics you're interested in?
Become a Member

IP.Board 500 Error on CSV Import

Discussion in 'Forum software usage' started by Jimmy, Jun 17, 2016.

  1. Jimmy

    Jimmy Well-Known Member

    1,788
    390
    83
    Oct 24, 2015
    East Coast USA
    Ratings:
    +990
    Local Time:
    1:16 AM
    I'm trying to upload a file via the members import. I keep getting a 500 error during the process.

    I removed a bunch of entries from the CSV (total of about 5 remaining) and did an import and it worked fine. The org file had about 20K entires.

    Trying to figure out what I have to adjust in the php.conf because it appears to be a fastcgi error.


    Code:
    2016/06/16 18:06:53 [error] 2847#2847: *2 open() "/home/nginx/domains/10.0.0.121/public/500.html" failed (2: No such file or directory), client: 10.0.0.245, server: 10.0.0.121, request: "POST /admin/?adsess=kn34qisbbq7b498btge08rcpg7&app=core&module=members&controller=members&do=import HTTP/2.0", upstream: "fastcgi://127.0.0.1:9000", host: "10.0.0.121", referrer: "https://10.0.0.121/admin/?adsess=kn34qisbbq7b498btge08rcpg7&app=core&module=members&controller=members&do=import"
    Sends me to a page which states "10.0.0.121 unexpectedly closed the connection."

    Thinking I have to adjust something in the php.conf here:

    Code:
    fastcgi_connect_timeout 60;
    fastcgi_send_timeout 180;
    fastcgi_read_timeout 180;
    fastcgi_buffer_size 512k;
    fastcgi_buffers 512 16k;
    fastcgi_busy_buffers_size 1m;
    fastcgi_temp_file_write_size 4m;
    fastcgi_max_temp_file_size 4m;
    fastcgi_intercept_errors on;
    Not exactly sure what I have to adjust. Error happens after 30ish seconds (timed it) and the file is 1.8MB in size.

    Also, do I have to rebuilt PHP after I make these adjustments?

    Thanks!
     
  2. eva2000

    eva2000 Administrator Staff Member

    55,237
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,833
    Local Time:
    3:16 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
  3. Jimmy

    Jimmy Well-Known Member

    1,788
    390
    83
    Oct 24, 2015
    East Coast USA
    Ratings:
    +990
    Local Time:
    1:16 AM
    I'm not having an error with phpmyadmin - I'm uploading it in the admin control panel of the script. I tried adjusting the settings in the php.ini with no luck - I maxed out just about every setting. From researching the issue of uploading CSV file on stackexchange there were a lot of posts about fastcgi. I haven't tried to adjust those yet. Figured I'd throw up a thread here and see if anyone had any suggestions.

    I know it does work with the smaller file - about 5 rows. It's the larger file with 20K rows that give the 500.
     
  4. eva2000

    eva2000 Administrator Staff Member

    55,237
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,833
    Local Time:
    3:16 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    same thing phpmyadmin uploads/imports via php-fpm as does any forum or web script running via php-fpm
     
  5. eva2000

    eva2000 Administrator Staff Member

    55,237
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,833
    Local Time:
    3:16 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Might need to debug nginx and php-fpm

    If you have a lot of free disk space, you can enable debug nginx version and compile a nginx debug build of nginx via centmin.sh option NGINX_DEBUG=y centminmod/centmin.sh at 123.08stable · centminmod/centminmod · GitHub

    You can place the option in persistent config /etc/centminmod/custom_config.inc outlined at centminmod.com/upgrade.html#persistent so place in file /etc/centminmod/custom_config.inc
    Code:
    NGINX_DEBUG=y
    then recompile nginx via centmin.sh menu option 4 and then in your nginx error_log directive add debug option A debugging log and restart nginx and check logs

    They will be very very very verbose and lot alot to disk usage logged to error logs, so ensure you have a lot of disk free space.

    You can minimise this by limiting it to specific ip based client connections too
    After debugging is done, reverse the debug now by setting NGINX_DEBUG=n and recompile Nginx again

    What's output for these commands - post output wrapped in CODE tags
    Code (Text):
    php -v

    Code (Text):
    php -m

    Code (Text):
    php-config --configure-options


    Examples with strace below:
    With gdb backtrace and PHP debug compiled mode where centmin.sh has a PHPDEBUGMODE variable which you can set to PHPDEBUGMODE=y and recompile php via centmin.sh menu option 5 to enable debug mode for PHP-FPM. After troubleshooting set PHPDEBUGMODE=n and recompile php via centmin.sh menu option 5 again to disable debug mode.
    Code:
    PHPDEBUGMODE=n # --enable-debug PHP compile flag
    Centmin Mod is provide as is, so short of scripted related bugs or issues, any further optimisation to the web stack components - nginx, php-fpm, mariadb mysql, csf firewall etc or web app specific configurations are left to the Centmin Mod user to deal with. So I do not provide any free support for such.

    You'll need to tune your PHP-FPM settings and this is left up to end user to do but here's a thread for starters to enable php status page output outlined at PHP-FPM - CentminMod.com LEMP Nginx web stack for CentOS and PHP-FPM - pm.max_children | Centmin Mod Community which outlines the official PHP-FPM config documentation as well.
     
  6. Jimmy

    Jimmy Well-Known Member

    1,788
    390
    83
    Oct 24, 2015
    East Coast USA
    Ratings:
    +990
    Local Time:
    1:16 AM
    I'll try that in a few. Have to finish up a few things before I can re-compile nginx.
     
  7. Jimmy

    Jimmy Well-Known Member

    1,788
    390
    83
    Oct 24, 2015
    East Coast USA
    Ratings:
    +990
    Local Time:
    1:16 AM
    LOL! I'm trying to re-install Nginx and can't figure out what is giving me an error. Until I realize this:

    Code:
    Compiling nginx...
    Initializing download: http://nginx.org/download/nginx-5.6.22.tar.gz
    HTTP/1.1 404 Not Found
    
    Too much coffee... :wacky:
     
  8. Ahmad

    Ahmad Active Member

    209
    80
    28
    Apr 13, 2015
    Ratings:
    +150
    Local Time:
    7:16 AM
    1.9.9
    10.1.10
    Nice one lol.
    If it doesn't contain important member info (i.e. email etc.), you are free to send me the list and I'll give it a try on my dev board :)
     
  9. Jimmy

    Jimmy Well-Known Member

    1,788
    390
    83
    Oct 24, 2015
    East Coast USA
    Ratings:
    +990
    Local Time:
    1:16 AM
    Yea, it does - can't send it out.

    Interesting... I just noticed - from the debug log that I was missing the code below from the conf file:

    Code:
    http2_max_field_size 16k;
    http2_max_header_size 32k;
    Now instead of sending me to the "10.0.0.121 unexpectedly closed the connection." it just loops the page:

    Importing members...
    Please wait while we transfer you...
     
  10. Ahmad

    Ahmad Active Member

    209
    80
    28
    Apr 13, 2015
    Ratings:
    +150
    Local Time:
    7:16 AM
    1.9.9
    10.1.10
    Take a look at PHP logs, maybe it's an IPS bug.
     
  11. Jimmy

    Jimmy Well-Known Member

    1,788
    390
    83
    Oct 24, 2015
    East Coast USA
    Ratings:
    +990
    Local Time:
    1:16 AM
    I will now.

    If anyone is curious here is the debug log.

    IPS Import · GitHub
     
  12. Jimmy

    Jimmy Well-Known Member

    1,788
    390
    83
    Oct 24, 2015
    East Coast USA
    Ratings:
    +990
    Local Time:
    1:16 AM
    @eva2000 not to say this has to do with the new ssl_ciphers but I got a few bad gateway errors on both my site and this forum starting about the time you changed those ciphers, unsure if the ciphers are related to the gateway errors, but wanted to let you know. I had one pop up when I tried to create this thread originally and had a few on my dev forum... they never happened before.
     
  13. eva2000

    eva2000 Administrator Staff Member

    55,237
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,833
    Local Time:
    3:16 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    i never changed ciphers on this forum only centminmod.com site which are on separate servers so not related :)

    forums one could be side effects of linode free ram upgrade/migration to new linode kvm vps
     
  14. Jimmy

    Jimmy Well-Known Member

    1,788
    390
    83
    Oct 24, 2015
    East Coast USA
    Ratings:
    +990
    Local Time:
    1:16 AM
    @Ahmad nothing in the error logs located at /var/log/php-fpm/*
     
  15. Jimmy

    Jimmy Well-Known Member

    1,788
    390
    83
    Oct 24, 2015
    East Coast USA
    Ratings:
    +990
    Local Time:
    1:16 AM
    It only happened twice I believe and a page refresh took care of the issue.
     
  16. eva2000

    eva2000 Administrator Staff Member

    55,237
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,833
    Local Time:
    3:16 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    you enabled block.conf include file in nginx vhost centminmod/block.conf at 123.09beta01 · centminmod/centminmod · GitHub ?
    Code (Text):
    2016/06/16 21:59:02 [debug] 14356#14356: *2 http script set $block_sql_injections
    2016/06/16 21:59:02 [debug] 14356#14356: *2 http script var
    2016/06/16 21:59:02 [debug] 14356#14356: *2 http script var: "adsess=kn34qisbbq7b498btge08rcpg7&app=core&module=members&controller=members&do=import"
    2016/06/16 21:59:02 [debug] 14356#14356: *2 http script regex: "union.*select.*\("
    2016/06/16 21:59:02 [notice] 14356#14356: *2 "union.*select.*\(" does not match "adsess=kn34qisbbq7b498btge08rcpg7&app=core&module=members&controller=members&do=import", client: 10.0.0.245, server: 10.0.0.121, request: "POST /admin/?adsess=kn34qisbbq7b498btge08rcpg7&app=core&module=members&controller=members&do=import HTTP/2.0", host: "10.0.0.121", referrer: "https://10.0.0.121/admin/?adsess=kn34qisbbq7b498btge08rcpg7&app=core&module=members&controller=members&do=import"
     
  17. eva2000

    eva2000 Administrator Staff Member

    55,237
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,833
    Local Time:
    3:16 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    probably caught me restart services after linode memory upgrade :D
     
  18. Jimmy

    Jimmy Well-Known Member

    1,788
    390
    83
    Oct 24, 2015
    East Coast USA
    Ratings:
    +990
    Local Time:
    1:16 AM
    Yea, I have block.conf enabled in this block. Though, I remember trying the import once with that disabled. Maybe I should try again. You think that might be the issue?

    Code:
        location / {
    
            # block common exploits, sql injections etc
            include /usr/local/nginx/conf/block.conf;
    
            # Enables directory listings when index file not found
            # autoindex  on;
    
            # Shows file listing times as local time
            autoindex_localtime on;
    
            try_files    $uri $uri/ /index.php;
    
        }
     
  19. Jimmy

    Jimmy Well-Known Member

    1,788
    390
    83
    Oct 24, 2015
    East Coast USA
    Ratings:
    +990
    Local Time:
    1:16 AM
    Tried removing the block.conf and still had the issue.
     
  20. eva2000

    eva2000 Administrator Staff Member

    55,237
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,833
    Local Time:
    3:16 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Probably need to look more closely in the log though just got alert of nginx source updates for bug fixes related to uploads not sure if related nginx: log