Learn about Centmin Mod LEMP Stack today
Become a Member

Security Distributed sshd attacks : "top" flooded with csf and lfd processes server down

Discussion in 'System Administration' started by Benjamin74, Aug 14, 2019.

  1. Benjamin74

    Benjamin74 Premium Member Premium Member

    46
    4
    8
    May 2, 2016
    Ratings:
    +10
    Local Time:
    2:40 AM
    Hello guys,

    I thought I would report this here, in case someone has any idea.

    Basically one of my smaller VPS (1Go RAM, 1 CPU) went down and after running some commands (the few times I could even SSH into my VPS), it looked like I was attacked through some brute force sshd attacks.

    e.g. running:

    Code:
    ps aux | sort -nrk 3,3 | head -n 5
    Would give me back something like that:

    Code:
    root     23187  4.1  2.2 169760 23100 ?        R    13:38   0:00 /usr/bin/perl /usr/sbin/csf -d 200.157.34.169 lfd: 200.157.34.169 (BR/Brazil/host169.databras.com.br), 5 distributed sshd attacks on account [lehranstalt] in the last 3600 secs
    root     23183  4.1  2.2 169760 22876 ?        R    13:38   0:00 /usr/bin/perl /usr/sbin/csf -d 103.1.184.127 lfd: 103.1.184.127 (AU/Australia/viking-lagoon.bnr.la), 5 distributed sshd attacks on account [client] in the last 3600 secs
    root     23202  4.0  2.2 169760 22956 ?        R    13:38   0:00 /usr/bin/perl /usr/sbin/csf -d 139.199.25.110 lfd: 139.199.25.110 (CN/China/-), 5 distributed sshd attacks on account [laravel] in the last 3600 secs
    root     23201  4.0  2.2 169760 22964 ?        R    13:38   0:00 /usr/bin/perl /usr/sbin/csf -d 94.177.233.182 lfd: 94.177.233.182 (FR/France/host182-233-177-94.static.arubacloud.fr), 5 distributed sshd attacks on account [kevin] in the last 3600 secs
    And running "top" command would give me a gazillions of CSF and LFD processes which are probably the reason why my VPS went down at the end:

    <a href="https://imgur.com/R1sDhhw"><img src="https://i.imgur.com/R1sDhhw.png" title="source: imgur.com" /></a>

    Is there any way to avoid this?

    Thanks,
     
  2. eva2000

    eva2000 Administrator Staff Member

    42,346
    9,560
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +14,732
    Local Time:
    11:40 AM
    Nginx 1.17.x
    MariaDB 5.5/10.x
    checked your memory and cpu usage during that time ? could be you ran out of memory ? doing down isn't very descriptive unfortunately

    more info might be helpful
    1. What version of Centmin Mod ? .08 stable or .09 beta ? If .09 beta when was it installed and when was last time you updated ?
    2. What's your VPS/Server hardware specifications ? cpu type ? memory available ? disk space ? openvz or kvm/xen ?
    3. Who's your web host ? Different hosts have different limits for server resource usage and some are more restrictive that others so it could just be their resource usage policy you tripped which may or may not be restrictive. If restrictive, then real solution would be finding a better web host. If not restrictive, then it's finding out what caused high cpu load.
    4. If running Centmin Mod 123.09beta01 or higher, you will have access to a cminfo top command which can also provide a summary of statistics
    You can change the sshd listening port from default 22 to a number below 1024 via centmin.sh menu option 16.

    Running centmin.sh menu option 16, will change your sshd listening port by
    • first prompting to you enter existing sshd port = 22
    • and then prompt you for your desired new sshd port and will make appropriate changes in CSF Firewall
    • once change is made, DO NOT exit your current SSH session but open a new SSH session test connection to this server making sure to have changed your SSH session/profile's SSH port from 22, to the new port number you selected. See if the new separate SSH session can connect to the new sshd port. If it can, you should be good to go.
     
  3. Benjamin74

    Benjamin74 Premium Member Premium Member

    46
    4
    8
    May 2, 2016
    Ratings:
    +10
    Local Time:
    2:40 AM
    Thanks for the tips.

    I'm sure it's the CPU that was maxed out because I'm running Hetrixtools to monitor it and could see that before the server went down / stopped responding.

    I logged in a few hours ago and logged in again and :"There were 9372 failed login attempts since the last successful login."...

    The VPS is an ArubaCloud (VMWare virtualization), VPS 1Go RAM, 1CPU, 20Go disc space, Intel(R) Xeon(R) CPU E5-2650L v4 @ 1.70GHz.

    It's running centos 7 with CMM beta 9.

    I'm not 100% sure when it was last updated, probably 4-5 months ago.

    I'll probably try changing the SSH port to see if it helps and update it too.