Want more timely Centmin Mod News Updates?
Become a Member

Security Countless Servers Are Vulnerable to Apache Log4j Zero-Day Exploit (Log4Shell)

Discussion in 'System Administration' started by Revenge, Dec 11, 2021.

  1. eva2000

    eva2000 Administrator Staff Member

    49,741
    11,437
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +17,772
    Local Time:
    12:10 PM
    Nginx 1.21.x
    MariaDB 10.x
    :LOL: @Jimmy

    Cloudflare WAF Firewall picked up more log4j attack attempts including user agent and query string populated ones and even some tailored specifically to my domain with their subdomain naming :)

    cf-waf-log4j-firewall-events-jndi-user-agent-01.png

    cf-waf-log4j-firewall-events-jndi-user-agent-02.png


    cf-waf-log4j-firewall-events-jndi-user-agent-03.png

    cf-waf-log4j-firewall-events-jndi-user-agent-04.png

    cf-waf-log4j-firewall-events-jndi-user-agent-05.png
     
  2. eva2000

    eva2000 Administrator Staff Member

    49,741
    11,437
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +17,772
    Local Time:
    12:10 PM
    Nginx 1.21.x
    MariaDB 10.x
    My mistake Plesk doesn't use elasticsearch search, I had installed elasticsearch on this Plesk Ubuntu server as I was developing a Plesk server info diagnostic script for my paid consulting work which scans and collects all Plesk server/environment info for a quick overview of a Plesk system resource usage/configurations :) Similar to Centmin Mod's cminfo command :D

    excerpt - does more though inspects all apache, nginx, PHP, MariaDB Mysql systems etc.
    Code (Text):
    ./plesk-info.sh inspect sitedomain_name
    ---------------------------------------------------------------
    Plesk Info hostname - Sun Dec 12 05:40:32 UTC 2021
    ---------------------------------------------------------------
    System Up Since:        2021-11-06 12:54:56
    System Uptime:          up 5 weeks, 16 hours, 45 minutes
    ---------------------------------------------------------------
    Product version: Plesk Obsidian 18.0.39.2
         OS version: Ubuntu 20.04 x86_64
         Build date: 2021/11/17 18:00
           Revision: bbbc28fd54b0747027d82997dd24710e1f05caba
    ---------------------------------------------------------------
    Upgrade to following versions is supported: 18.0.40
    ---------------------------------------------------------------
    

    Code (Text):
    ---------------------------------------------------------------
    Get PHP Handler Info sitedomain_name
    ---------------------------------------------------------------
    +--------------------+--------+------+-----------------+---------+------------------+
    | name               | dom_id | php  | php_handler_id  | fastcgi | Served as nginx? |
    +--------------------+--------+------+-----------------+---------+------------------+
    | sitedomain_name    |      1 | true | plesk-php74-fpm | true    | false            |
    +--------------------+--------+------+-----------------+---------+------------------+
    ---------------------------------------------------------------
    /opt/plesk/php/7.4/bin/php -v
    PHP 7.4.26 (cli) (built: Nov 18 2021 12:28:06) ( NTS )
    Copyright (c) The PHP Group
    Zend Engine v3.4.0, Copyright (c) Zend Technologies
        with the ionCube PHP Loader + ionCube24 v10.4.5, Copyright (c) 2002-2020, by ionCube Ltd.
        with Zend OPcache v7.4.26, Copyright (c), by Zend Technologies
    ---------------------------------------------------------------
    /opt/plesk/php/7.4/bin/php --ini
    Configuration File (php.ini) Path: /opt/plesk/php/7.4/etc
    Loaded Configuration File:         /opt/plesk/php/7.4/etc/php.ini
    Scan for additional .ini files in: /opt/plesk/php/7.4/etc/php.d
    Additional .ini files parsed:      /opt/plesk/php/7.4/etc/php.d/00-ioncube-loader.ini,
    /opt/plesk/php/7.4/etc/php.d/10-opcache.ini,
    /opt/plesk/php/7.4/etc/php.d/20-mysqlnd.ini,
    /opt/plesk/php/7.4/etc/php.d/50-xdebug.ini,
    /opt/plesk/php/7.4/etc/php.d/bcmath.ini,
    /opt/plesk/php/7.4/etc/php.d/curl.ini,
    /opt/plesk/php/7.4/etc/php.d/dba.ini,
    /opt/plesk/php/7.4/etc/php.d/dom.ini,
    /opt/plesk/php/7.4/etc/php.d/enchant.ini,
    /opt/plesk/php/7.4/etc/php.d/fileinfo.ini,
    /opt/plesk/php/7.4/etc/php.d/gd.ini,
    /opt/plesk/php/7.4/etc/php.d/hideheader.ini,
    /opt/plesk/php/7.4/etc/php.d/imagick.ini,
    /opt/plesk/php/7.4/etc/php.d/imap.ini,
    /opt/plesk/php/7.4/etc/php.d/intl.ini,
    /opt/plesk/php/7.4/etc/php.d/json.ini,
    /opt/plesk/php/7.4/etc/php.d/ldap.ini,
    /opt/plesk/php/7.4/etc/php.d/mbstring.ini,
    /opt/plesk/php/7.4/etc/php.d/mysqli.ini,
    /opt/plesk/php/7.4/etc/php.d/odbc.ini,
    /opt/plesk/php/7.4/etc/php.d/pdo.ini,
    /opt/plesk/php/7.4/etc/php.d/pdo_mysql.ini,
    /opt/plesk/php/7.4/etc/php.d/pdo_odbc.ini,
    /opt/plesk/php/7.4/etc/php.d/pdo_pgsql.ini,
    /opt/plesk/php/7.4/etc/php.d/pdo_sqlite.ini,
    /opt/plesk/php/7.4/etc/php.d/pgsql.ini,
    /opt/plesk/php/7.4/etc/php.d/phar.ini,
    /opt/plesk/php/7.4/etc/php.d/posix.ini,
    /opt/plesk/php/7.4/etc/php.d/pspell.ini,
    /opt/plesk/php/7.4/etc/php.d/redis.ini,
    /opt/plesk/php/7.4/etc/php.d/snmp.ini,
    /opt/plesk/php/7.4/etc/php.d/soap.ini,
    /opt/plesk/php/7.4/etc/php.d/sodium.ini,
    /opt/plesk/php/7.4/etc/php.d/sqlite3.ini,
    /opt/plesk/php/7.4/etc/php.d/sysvmsg.ini,
    /opt/plesk/php/7.4/etc/php.d/sysvsem.ini,
    /opt/plesk/php/7.4/etc/php.d/sysvshm.ini,
    /opt/plesk/php/7.4/etc/php.d/tidy.ini,
    /opt/plesk/php/7.4/etc/php.d/xmlreader.ini,
    /opt/plesk/php/7.4/etc/php.d/xmlrpc.ini,
    /opt/plesk/php/7.4/etc/php.d/xmlwriter.ini,
    /opt/plesk/php/7.4/etc/php.d/xsl.ini,
    /opt/plesk/php/7.4/etc/php.d/zip.ini
    

    Code (Text):
    ---------------------------------------------------------------
    PHP-FPM limits
    
    php-fpm pid: 508951
    Limit                     Soft Limit           Hard Limit           Units    
    Max cpu time              unlimited            unlimited            seconds  
    Max file size             unlimited            unlimited            bytes    
    Max data size             unlimited            unlimited            bytes    
    Max stack size            8388608              unlimited            bytes    
    Max core file size        0                    unlimited            bytes    
    Max resident set          unlimited            unlimited            bytes    
    Max processes             3789                 3789                 processes
    Max open files            524288               524288               files    
    Max locked memory         65536                65536                bytes    
    Max address space         unlimited            unlimited            bytes    
    Max file locks            unlimited            unlimited            locks    
    Max pending signals       3789                 3789                 signals  
    Max msgqueue size         819200               819200               bytes    
    Max nice priority         0                    0                  
    Max realtime priority     0                    0                  
    Max realtime timeout      unlimited            unlimited            us      
    ---------------------------------------------------------------
    {
      "id": "plesk-php74-cgi",
      "displayname": "7.4.26",
      "fullVersion": "7.4.26",
      "version": "7.4",
      "type": "cgi",
      "path": "/opt/plesk/php/7.4/bin/php-cgi",
      "clipath": "/opt/plesk/php/7.4/bin/php",
      "phpini": "/opt/plesk/php/7.4/etc/php.ini",
      "custom": true,
      "status": "disabled"
    }
    {
      "id": "plesk-php74-fastcgi",
      "displayname": "7.4.26",
      "fullVersion": "7.4.26",
      "version": "7.4",
      "type": "fastcgi",
      "path": "/opt/plesk/php/7.4/bin/php-cgi",
      "clipath": "/opt/plesk/php/7.4/bin/php",
      "phpini": "/opt/plesk/php/7.4/etc/php.ini",
      "custom": true,
      "status": "enabled"
    }
    {
      "id": "plesk-php74-fpm",
      "displayname": "7.4.26",
      "fullVersion": "7.4.26",
      "version": "7.4",
      "type": "fpm",
      "path": "/opt/plesk/php/7.4/sbin/php-fpm",
      "clipath": "/opt/plesk/php/7.4/bin/php",
      "phpini": "/opt/plesk/php/7.4/etc/php.ini",
      "custom": true,
      "status": "enabled"
    }
    {
      "id": "plesk-php74-fpm-dedicated",
      "displayname": "7.4.26",
      "fullVersion": "7.4.26",
      "version": "7.4",
      "type": "fpm",
      "path": "/opt/plesk/php/7.4/sbin/php-fpm",
      "clipath": "/opt/plesk/php/7.4/bin/php",
      "phpini": "/opt/plesk/php/7.4/etc/php.ini",
      "custom": true,
      "status": "enabled"
    }
    
     
  3. eva2000

    eva2000 Administrator Staff Member

    49,741
    11,437
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +17,772
    Local Time:
    12:10 PM
    Nginx 1.21.x
    MariaDB 10.x
    One of the better explanations for log4j vulnerability I've come across https://www.fastly.com/blog/digging-deeper-into-log4shell-0day-rce-exploit-found-in-log4j

    log4j-diagram-fastly-01.png
     
  4. deltahf

    deltahf Premium Member Premium Member

    543
    244
    43
    Jun 8, 2014
    Ratings:
    +443
    Local Time:
    9:10 PM
  5. eva2000

    eva2000 Administrator Staff Member

    49,741
    11,437
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +17,772
    Local Time:
    12:10 PM
    Nginx 1.21.x
    MariaDB 10.x
  6. buik

    buik “The best traveler is one without a camera.”

    1,764
    480
    83
    Apr 29, 2016
    Flanders
    Ratings:
    +1,511
    Local Time:
    3:10 AM
    It doesn't matter. Because mission critical systems are always not directly connected on the free Internet (though I've seen worse).
     
  7. Jimmy

    Jimmy Well-Known Member

    1,761
    381
    83
    Oct 24, 2015
    East Coast USA
    Ratings:
    +967
    Local Time:
    9:10 PM
    1.17.x
    MariaDB 10.3.x
    This could be an episode of The Big Bang Theory!

     
    Last edited: Dec 13, 2021
  8. eva2000

    eva2000 Administrator Staff Member

    49,741
    11,437
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +17,772
    Local Time:
    12:10 PM
    Nginx 1.21.x
    MariaDB 10.x
    True - but how interconnected is all the systems to systems on Earth? Who knows!

    :LOL:

    More reading

    https://www.akamai.com/blog/news/CVE-2021-44228-Zero-Day-Vulnerability

    and dealing with Docker image containers Apache Log4j 2 CVE-2021-44228 - Docker Blog

     
  9. eva2000

    eva2000 Administrator Staff Member

    49,741
    11,437
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +17,772
    Local Time:
    12:10 PM
    Nginx 1.21.x
    MariaDB 10.x
    log4j vulnerability POC (proof of concept) via Docker container you can test GitHub - justincormack/log4jpoc

     
  10. Jimmy

    Jimmy Well-Known Member

    1,761
    381
    83
    Oct 24, 2015
    East Coast USA
    Ratings:
    +967
    Local Time:
    9:10 PM
    1.17.x
    MariaDB 10.3.x
    I knew I was waiting to setup that Docker project for a reason. Now I remember.

     
    Last edited: Dec 13, 2021
  11. eva2000

    eva2000 Administrator Staff Member

    49,741
    11,437
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +17,772
    Local Time:
    12:10 PM
    Nginx 1.21.x
    MariaDB 10.x
    For folks testing my Centmin Mod fail2ban implementation @Matt I've updated it to detect log4j vulnerability scans for Centmin Mod Nginx log inspection. Details in at Update to support log4j vulnerability scans · Issue #2 · centminmod/centminmod-fail2ban. You can see an example of fail2ban detecting log4j vulnerability scans on Centmin Mod Nginx server with my fail2ban implementation installed at GitHub - centminmod/centminmod-fail2ban: fail2ban setup for centminmod.com LEMP stack with CSF Firewall. FYI, while doing testing, it actually caught a real vulnerability scan too!

    There is no support for Centmin Mod fail2ban implementation, so you'd generally be on your own to configure and troubleshoot for your needs.
     
  12. eva2000

    eva2000 Administrator Staff Member

    49,741
    11,437
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +17,772
    Local Time:
    12:10 PM
    Nginx 1.21.x
    MariaDB 10.x
    Seems like Cloudflare WAF rules for log4j is more about protecting themselves seeing as Cloudflare would be vulnerable due to their usage with log4j. Added benefit is protecting CF customers :)

    My custom CF Firewall caught some too

    cf-waf-log4j-rules-referer-useragent-fulluri-path-query-strings-02.png

    cf-waf-log4j-rules-referer-useragent-fulluri-path-query-strings-02-events.png
     
  13. eva2000

    eva2000 Administrator Staff Member

    49,741
    11,437
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +17,772
    Local Time:
    12:10 PM
    Nginx 1.21.x
    MariaDB 10.x
    Updated custom Cloudflare Firewall rule and also fail2ban regex for jndi:iiop pattern matching too :)

    cf-waf-log4j-rules-referer-useragent-fulluri-path-query-strings-03.png
     
  14. buik

    buik “The best traveler is one without a camera.”

    1,764
    480
    83
    Apr 29, 2016
    Flanders
    Ratings:
    +1,511
    Local Time:
    3:10 AM
    Fair is fair but I don't understand the media hype towards CVE-2021-44228.
    With slogans as far as "The Internet Is on Fire".

    CVE-2021-44228 is clearly described and you know what to do about it.
    Patch code is only a few lines of code, so it can be stress tested quickly and because little code changes. It has 'stress test wise' little impact on the rest of the software, hardware and infrastructure.

    Fishing for the purpose of encrypting your data is much more dangerous, with a continuous danger, without a clear manual, patch or prompt.
    Think of it as an invisible enemy that can strike at any time.

    You don't hear the media about that.
    But now the snowflake internet is 'on fire' with only a simple and clear CVE.
     
  15. eva2000

    eva2000 Administrator Staff Member

    49,741
    11,437
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +17,772
    Local Time:
    12:10 PM
    Nginx 1.21.x
    MariaDB 10.x
    Time will tell how many folks get owned by this though.

    You can actually check some of the Github repos which depend on log4j at Github Network Dependents · apache/log4j. 469,880 Github repos use, reference or depend on log4j !
     
  16. buik

    buik “The best traveler is one without a camera.”

    1,764
    480
    83
    Apr 29, 2016
    Flanders
    Ratings:
    +1,511
    Local Time:
    3:10 AM
    Mah. The time of set and forget is over. Those days where a single server stood in the uncooled broom closet. Nowadays, you have to keep a constant eye on your infrastructure. There has been enough time to intervene.

    These servers that are left now are managed without a service contract (or rather not managed) or managed without knowledge and skills. And yes, unfortunately, there are many of them. For many managements, ICT is still just a nuisance and a mere expense.
     
  17. eva2000

    eva2000 Administrator Staff Member

    49,741
    11,437
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +17,772
    Local Time:
    12:10 PM
    Nginx 1.21.x
    MariaDB 10.x
    Last edited: Dec 14, 2021
  18. pamamolf

    pamamolf Well-Known Member

    4,000
    420
    83
    May 31, 2014
    Ratings:
    +812
    Local Time:
    4:10 AM
    Nginx-1.17.x
    MariaDB 10.3.x
    matches regex doesn't exist on Pro account :(

    contains
    equals
    does not equal
    does not contain

    only :)
     
  19. eva2000

    eva2000 Administrator Staff Member

    49,741
    11,437
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +17,772
    Local Time:
    12:10 PM
    Nginx 1.21.x
    MariaDB 10.x
    Yep, Cloudflare Firewall regex rules are only available in Cloudflare Business and Enterprise plans :) For other plans, you'd have to use some creative longer and/or rules for each possible match.
     
  20. pamamolf

    pamamolf Well-Known Member

    4,000
    420
    83
    May 31, 2014
    Ratings:
    +812
    Local Time:
    4:10 AM
    Nginx-1.17.x
    MariaDB 10.3.x
    I replaced matches regex with contains...

    Hope that should be ok :)

    Unless that "contains" means any letter, number or symbol in the field.

    Then i really mess it :p