Join the community today
Register Now

CURL Slowing down when sending multiple requests

Discussion in 'Other Web Apps usage' started by Oxide, Feb 5, 2019.

  1. Oxide

    Oxide Active Member

    534
    29
    28
    Mar 19, 2015
    Ratings:
    +59
    Local Time:
    2:36 AM
    Hi,

    I reinstalled PHP and updated curl.

    Ever since I did this, it seems like curl requests got slow.

    I have a script which sends +1500 requests via CURL PHP after each other in a foreach loop. It goes from 0.02 second response time, to 0.2 seconds in less than 10 seconds. The response time keep increasing.

    I can confirm this is not a problem with the API I am connecting to, since when I re-start the PHP Script it goes back to 0.02 again.

    Any ideas what could be wrong? It seems like I need to restart the PHP script constantly to speed it up.

    It only seems to happen if you send multiple requests, under same script instance. You can restart the script to "flush" the delay, but it keeps building up after the exact amount of requests.

    What could be the problem? Any help debugging?

     
  2. eva2000

    eva2000 Administrator Staff Member

    54,883
    12,240
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,811
    Local Time:
    2:36 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    How did you update curl exactly?
     
  3. Oxide

    Oxide Active Member

    534
    29
    28
    Mar 19, 2015
    Ratings:
    +59
    Local Time:
    2:36 AM
    Curl updater in addons.

    Then reinstalled php 7.2 and ion cube.
     
  4. eva2000

    eva2000 Administrator Staff Member

    54,883
    12,240
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,811
    Local Time:
    2:36 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    You could try removing the curl update via yum history undo command

    Run yum history list command and note the transaction id for the curl update listed. Then run next command to undo those yum transactions
    Code (Text):
    yum history list
    yum history undo XX

    Where XX is the transaction id for curl update transaction listing
     
  5. Oxide

    Oxide Active Member

    534
    29
    28
    Mar 19, 2015
    Ratings:
    +59
    Local Time:
    2:36 AM
    I'm not sure how to do that, as the curl version is already updated and I can not check the transaction id.

    Code:
    curl 7.63.0 (x86_64-redhat-linux-gnu) libcurl/7.63.0 NSS/3.36 zlib/1.2.7 libpsl/0.7.0 (+libicu/50.1.2) libssh2/1.8.0 nghttp2/1.31.1
    Release-Date: 2018-12-12
    Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
    Features: AsynchDNS IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz HTTP2 UnixSockets HTTPS-proxy PSL Metalink
    
     
  6. Oxide

    Oxide Active Member

    534
    29
    28
    Mar 19, 2015
    Ratings:
    +59
    Local Time:
    2:36 AM
    I have made a test script, it simply loops the request. After around 150 requests, it's significant slower.

    How can I downgrade CURL Version? I suspect that may be it.

    I also removed SWAP from my server, could that have anything to do with it? The installations etc? @eva2000
     
  7. Oxide

    Oxide Active Member

    534
    29
    28
    Mar 19, 2015
    Ratings:
    +59
    Local Time:
    2:36 AM
    Fixed by downgrading CURL, finally figured out how.
     
  8. eva2000

    eva2000 Administrator Staff Member

    54,883
    12,240
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,811
    Local Time:
    2:36 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Good to hear.. you used yum history undo as outlined above right ? That would be the safe way of downgrading to previous version.
     
  9. Oxide

    Oxide Active Member

    534
    29
    28
    Mar 19, 2015
    Ratings:
    +59
    Local Time:
    2:36 AM
    Yes I did.
    Do you think it's just CURL that is slower in general for php after that upgrade, or some new settings?