Discover Centmin Mod today
Register Now

Xenforo ForumBackup addon transfers corrupted file to centminmod host

Discussion in 'Forum software usage' started by SneakyDave, May 21, 2015.

  1. SneakyDave

    SneakyDave Member

    68
    11
    8
    Jul 24, 2014
    Ratings:
    +16
    Local Time:
    9:14 PM
    1.0.15
    I really don't think this is a centminmod problem, but I thought I'd post it here because bigger brains and all.

    I wrote a ForumBackup addon for XenForo that has an option to it to SFTP a local backup file to a remote server. It uses the phpssh2 library to accomplish this. It is a pretty simple step to provide a hostname, user, password, remote directoy, and the appropriate permissions for the remote file.

    One user of the SFTP process is seeing that recently (April 28th), their backups (compressed gzip files) were being transferred to the backup host (running centminmod), but that the file sizes on the remote backup server were less than the local server. Also, it appears that the transferred gzip files are corrupted, they won't decompress.

    So, the question is, why would those sftp transfers start failing on April 28th? Traditional SFTP transfers appears to work ok, but I'm going to try more testing with that. I've asked the user to see if there are any PHP warnings or errors on the web server side to indicate any type of transmission problem.

    The user has commented that PHP was upgraded to 5.6.8 on April 16th, so that could also be a factor. It appears that there might be a newer version of php-ssh2 to use (0.12.3) for the php 5.6 branch, but I'm not certain of that because PECL says that version 0.12 is for PHP version 6.0 and below, but it says that for all versions too, lol. I'm not sure how to pinpoint if that's the real problem or not.

    Would the centminmod server have any logs to look at, perhaps a ssh or sftp log? Would those entries be in /var/log/messages?

    On a related note, I've heard that the phpseclib might be a good alternative to phpssh2. I doubt if it would provide any answers to this problem, but was curious if anybody has used it.
     
  2. eva2000

    eva2000 Administrator Staff Member

    30,194
    6,789
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,144
    Local Time:
    12:14 PM
    Nginx 1.13.x
    MariaDB 5.5
    how is gzip compression handled ? direct from ssh command line or via php ? maybe php ran out of memory doing gzip compression ? or php timed out ?

    if it's via php, then checking php-fpm error logs outlined at How to troubleshoot Centmin Mod initial install issues | Centmin Mod Community for /var/log/php-fpm/www-error.log might be one source to troubleshoot and investigate

    and yes /var/log/messages - also you can do a reverse ascending order listing of logs at /var/log to see which logs where updated recently (most recent listed at bottom) to get an idea of what logs are updated

    Code:
    ls -lahrt /var/log
    haven't heard of phpseclib though.
     
    • Like Like x 1
  3. SneakyDave

    SneakyDave Member

    68
    11
    8
    Jul 24, 2014
    Ratings:
    +16
    Local Time:
    9:14 PM
    1.0.15
    The file is compressed in PHP using gzip via an exec() call first, then the file is transferred via php-ssh2 as a separate step. The compressed file on the local machine is fine, but the corruption happens when it it transferred to the remote host.

    I would agree that it could be a timeout/disconnect with the remote host, but I'm not sure where I'd see that, or why it would happen. I'll do some digging.

    I'll see if I can get access to both machines to run some tests.
     
  4. eva2000

    eva2000 Administrator Staff Member

    30,194
    6,789
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,144
    Local Time:
    12:14 PM
    Nginx 1.13.x
    MariaDB 5.5
    • Like Like x 1
  5. eva2000

    eva2000 Administrator Staff Member

    30,194
    6,789
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,144
    Local Time:
    12:14 PM
    Nginx 1.13.x
    MariaDB 5.5
    curious what was prior php version before the upgrade ?
     
  6. SneakyDave

    SneakyDave Member

    68
    11
    8
    Jul 24, 2014
    Ratings:
    +16
    Local Time:
    9:14 PM
    1.0.15
    It was PHP 5.6.4, so not a big leap to 5.6.8, which is what he upgraded to
     
  7. SneakyDave

    SneakyDave Member

    68
    11
    8
    Jul 24, 2014
    Ratings:
    +16
    Local Time:
    9:14 PM
    1.0.15
    The php-ssh2 update didn't seem to fix the issue, I'm trying to dig into it more to find out if which server is really causing the problem, or if it's a disconnect type of thing. The tasks doesn't run more than a couple of seconds, so it hardly appears to be a time out situation.

    Previously, I stated that centminmod was on the remote backup server, but actually, it's on the web server. They updated php-ssh2 via your instructions, and the same issue happens, although libssh2 is version 1.4.2, not your version 1.4.3. I assume that's because you are using a newer or beta centminmod?

    Code:
    [root@server ssh2-master]# php --ri ssh2
    
    ssh2
    
    SSH2 support => enabled
    extension version => 0.12+dev
    libssh2 version => 1.4.2
    banner => SSH-2.0-libssh2_1.4.2
    [root@server ssh2-master]#
     
  8. eva2000

    eva2000 Administrator Staff Member

    30,194
    6,789
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,144
    Local Time:
    12:14 PM
    Nginx 1.13.x
    MariaDB 5.5
    libssh2 1.4.3 was on CentOS 7.1
     
  9. SneakyDave

    SneakyDave Member

    68
    11
    8
    Jul 24, 2014
    Ratings:
    +16
    Local Time:
    9:14 PM
    1.0.15
    Well, doing some more testing, this web server works fine to transfer the files to the backup server.
    centminmod with PHP 5.4.39 and phpssh2 0.12 (libssh2 1.4.2)

    But this particular server configuration doesn't, transferring to the same backup server.
    centminmod with PHP 5.6.9 and phpssh2 0.12+dev (libssh2 1.4.2)

    So all that means is that it doesn't appear to be the backup server, not even sure if its a PHP issue.

    Will post an update if I find anything else out.
     
    • Informative Informative x 1
  10. eva2000

    eva2000 Administrator Staff Member

    30,194
    6,789
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,144
    Local Time:
    12:14 PM
    Nginx 1.13.x
    MariaDB 5.5
    is backup server running CSF Firewall or firewall (iptables) ? maybe the PHP 5.6.9 web server got caught in firewall block/ban on backup server ?
     
  11. SneakyDave

    SneakyDave Member

    68
    11
    8
    Jul 24, 2014
    Ratings:
    +16
    Local Time:
    9:14 PM
    1.0.15
    I'll check that out. The database is bigger than I thought on the transfer, so it takes a while for php to gzip it, and then transfer it to the backup server, I wonder if it's getting a php max execution time timeout doing all those tasks.
     
  12. eva2000

    eva2000 Administrator Staff Member

    30,194
    6,789
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,144
    Local Time:
    12:14 PM
    Nginx 1.13.x
    MariaDB 5.5
    don't use gzip it's slow, use pigz Multi-threaded compression tools (pigz, pbzip2, lbzip2, plzip, p7zip) | Centmin Mod Community ;)

    Xeon D-1540 16 cpu thread benchmarks for gzip vs pigz and other multithreaded compression tools Intel Xeon D-1540 benchmarks | Centmin Mod Community :D
     
  13. SneakyDave

    SneakyDave Member

    68
    11
    8
    Jul 24, 2014
    Ratings:
    +16
    Local Time:
    9:14 PM
    1.0.15
    Yeah, my addon supports that, I'll try that out.
     
  14. SneakyDave

    SneakyDave Member

    68
    11
    8
    Jul 24, 2014
    Ratings:
    +16
    Local Time:
    9:14 PM
    1.0.15
    Well, we seemed to have confirmed that it isn't the web server, it's the backup server. Maybe a connection problem with the route or something.

    I spun up a quick VPS with Vultr, and the user can use the addon to transfer their larger database backup to this vultr VPS, and I can transfer my own smaller XenForo databases to this server, but neither of us can transfer our backups to the shared hosting backupserver without getting a corrupted gzip file. Doesn't matter if gzip or pigz is used, the locally created file is fine, but something's awry in the transfer.
    Trying to unzip the file results in:
    Code:
    name@server [~/backuptest]# gunzip forumbackup_today.sql.gz
    
    gzip: forumbackup_today.sql.gz: unexpected end of file
    
    So there must be a disconnect, timeout, or some other transfer problem going on with the backup server.

    Thanks for your help, and thanks for the info on upgrading php-ssh2.
     
    • Informative Informative x 1
  15. eva2000

    eva2000 Administrator Staff Member

    30,194
    6,789
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,144
    Local Time:
    12:14 PM
    Nginx 1.13.x
    MariaDB 5.5
    tried compress/decompress on backup server to check if it's the software binaries having issues ?