Security Countless Servers Are Vulnerable to Apache Log4j Zero-Day Exploit (Log4Shell)

eva2000 · Dec 12, 2021

@Jimmy

Cloudflare WAF Firewall picked up more log4j attack attempts including user agent and query string populated ones and even some tailored specifically to my domain with their subdomain naming

eva2000 · Dec 12, 2021

My mistake Plesk doesn't use elasticsearch search, I had installed elasticsearch on this Plesk Ubuntu server as I was developing a Plesk server info diagnostic script for my paid consulting work which scans and collects all Plesk server/environment info for a quick overview of a Plesk system resource usage/configurations

Similar to Centmin Mod's cminfo command

excerpt - does more though inspects all apache, nginx, PHP, MariaDB Mysql systems etc.

Code (Text):

./plesk-info.sh inspect sitedomain_name
---------------------------------------------------------------
Plesk Info hostname - Sun Dec 12 05:40:32 UTC 2021
---------------------------------------------------------------
System Up Since:        2021-11-06 12:54:56
System Uptime:          up 5 weeks, 16 hours, 45 minutes
---------------------------------------------------------------
Product version: Plesk Obsidian 18.0.39.2
     OS version: Ubuntu 20.04 x86_64
     Build date: 2021/11/17 18:00
       Revision: bbbc28fd54b0747027d82997dd24710e1f05caba
---------------------------------------------------------------
Upgrade to following versions is supported: 18.0.40
---------------------------------------------------------------

Code (Text):

---------------------------------------------------------------
Get PHP Handler Info sitedomain_name
---------------------------------------------------------------
+--------------------+--------+------+-----------------+---------+------------------+
| name               | dom_id | php  | php_handler_id  | fastcgi | Served as nginx? |
+--------------------+--------+------+-----------------+---------+------------------+
| sitedomain_name    |      1 | true | plesk-php74-fpm | true    | false            |
+--------------------+--------+------+-----------------+---------+------------------+
---------------------------------------------------------------
/opt/plesk/php/7.4/bin/php -v
PHP 7.4.26 (cli) (built: Nov 18 2021 12:28:06) ( NTS )
Copyright (c) The PHP Group
Zend Engine v3.4.0, Copyright (c) Zend Technologies
    with the ionCube PHP Loader + ionCube24 v10.4.5, Copyright (c) 2002-2020, by ionCube Ltd.
    with Zend OPcache v7.4.26, Copyright (c), by Zend Technologies
---------------------------------------------------------------
/opt/plesk/php/7.4/bin/php --ini
Configuration File (php.ini) Path: /opt/plesk/php/7.4/etc
Loaded Configuration File:         /opt/plesk/php/7.4/etc/php.ini
Scan for additional .ini files in: /opt/plesk/php/7.4/etc/php.d
Additional .ini files parsed:      /opt/plesk/php/7.4/etc/php.d/00-ioncube-loader.ini,
/opt/plesk/php/7.4/etc/php.d/10-opcache.ini,
/opt/plesk/php/7.4/etc/php.d/20-mysqlnd.ini,
/opt/plesk/php/7.4/etc/php.d/50-xdebug.ini,
/opt/plesk/php/7.4/etc/php.d/bcmath.ini,
/opt/plesk/php/7.4/etc/php.d/curl.ini,
/opt/plesk/php/7.4/etc/php.d/dba.ini,
/opt/plesk/php/7.4/etc/php.d/dom.ini,
/opt/plesk/php/7.4/etc/php.d/enchant.ini,
/opt/plesk/php/7.4/etc/php.d/fileinfo.ini,
/opt/plesk/php/7.4/etc/php.d/gd.ini,
/opt/plesk/php/7.4/etc/php.d/hideheader.ini,
/opt/plesk/php/7.4/etc/php.d/imagick.ini,
/opt/plesk/php/7.4/etc/php.d/imap.ini,
/opt/plesk/php/7.4/etc/php.d/intl.ini,
/opt/plesk/php/7.4/etc/php.d/json.ini,
/opt/plesk/php/7.4/etc/php.d/ldap.ini,
/opt/plesk/php/7.4/etc/php.d/mbstring.ini,
/opt/plesk/php/7.4/etc/php.d/mysqli.ini,
/opt/plesk/php/7.4/etc/php.d/odbc.ini,
/opt/plesk/php/7.4/etc/php.d/pdo.ini,
/opt/plesk/php/7.4/etc/php.d/pdo_mysql.ini,
/opt/plesk/php/7.4/etc/php.d/pdo_odbc.ini,
/opt/plesk/php/7.4/etc/php.d/pdo_pgsql.ini,
/opt/plesk/php/7.4/etc/php.d/pdo_sqlite.ini,
/opt/plesk/php/7.4/etc/php.d/pgsql.ini,
/opt/plesk/php/7.4/etc/php.d/phar.ini,
/opt/plesk/php/7.4/etc/php.d/posix.ini,
/opt/plesk/php/7.4/etc/php.d/pspell.ini,
/opt/plesk/php/7.4/etc/php.d/redis.ini,
/opt/plesk/php/7.4/etc/php.d/snmp.ini,
/opt/plesk/php/7.4/etc/php.d/soap.ini,
/opt/plesk/php/7.4/etc/php.d/sodium.ini,
/opt/plesk/php/7.4/etc/php.d/sqlite3.ini,
/opt/plesk/php/7.4/etc/php.d/sysvmsg.ini,
/opt/plesk/php/7.4/etc/php.d/sysvsem.ini,
/opt/plesk/php/7.4/etc/php.d/sysvshm.ini,
/opt/plesk/php/7.4/etc/php.d/tidy.ini,
/opt/plesk/php/7.4/etc/php.d/xmlreader.ini,
/opt/plesk/php/7.4/etc/php.d/xmlrpc.ini,
/opt/plesk/php/7.4/etc/php.d/xmlwriter.ini,
/opt/plesk/php/7.4/etc/php.d/xsl.ini,
/opt/plesk/php/7.4/etc/php.d/zip.ini

Code (Text):

---------------------------------------------------------------
PHP-FPM limits

php-fpm pid: 508951
Limit                     Soft Limit           Hard Limit           Units    
Max cpu time              unlimited            unlimited            seconds  
Max file size             unlimited            unlimited            bytes    
Max data size             unlimited            unlimited            bytes    
Max stack size            8388608              unlimited            bytes    
Max core file size        0                    unlimited            bytes    
Max resident set          unlimited            unlimited            bytes    
Max processes             3789                 3789                 processes
Max open files            524288               524288               files    
Max locked memory         65536                65536                bytes    
Max address space         unlimited            unlimited            bytes    
Max file locks            unlimited            unlimited            locks    
Max pending signals       3789                 3789                 signals  
Max msgqueue size         819200               819200               bytes    
Max nice priority         0                    0                  
Max realtime priority     0                    0                  
Max realtime timeout      unlimited            unlimited            us      
---------------------------------------------------------------
{
  "id": "plesk-php74-cgi",
  "displayname": "7.4.26",
  "fullVersion": "7.4.26",
  "version": "7.4",
  "type": "cgi",
  "path": "/opt/plesk/php/7.4/bin/php-cgi",
  "clipath": "/opt/plesk/php/7.4/bin/php",
  "phpini": "/opt/plesk/php/7.4/etc/php.ini",
  "custom": true,
  "status": "disabled"
}
{
  "id": "plesk-php74-fastcgi",
  "displayname": "7.4.26",
  "fullVersion": "7.4.26",
  "version": "7.4",
  "type": "fastcgi",
  "path": "/opt/plesk/php/7.4/bin/php-cgi",
  "clipath": "/opt/plesk/php/7.4/bin/php",
  "phpini": "/opt/plesk/php/7.4/etc/php.ini",
  "custom": true,
  "status": "enabled"
}
{
  "id": "plesk-php74-fpm",
  "displayname": "7.4.26",
  "fullVersion": "7.4.26",
  "version": "7.4",
  "type": "fpm",
  "path": "/opt/plesk/php/7.4/sbin/php-fpm",
  "clipath": "/opt/plesk/php/7.4/bin/php",
  "phpini": "/opt/plesk/php/7.4/etc/php.ini",
  "custom": true,
  "status": "enabled"
}
{
  "id": "plesk-php74-fpm-dedicated",
  "displayname": "7.4.26",
  "fullVersion": "7.4.26",
  "version": "7.4",
  "type": "fpm",
  "path": "/opt/plesk/php/7.4/sbin/php-fpm",
  "clipath": "/opt/plesk/php/7.4/bin/php",
  "phpini": "/opt/plesk/php/7.4/etc/php.ini",
  "custom": true,
  "status": "enabled"
}

eva2000 · Dec 12, 2021

One of the better explanations for log4j vulnerability I've come across https://www.fastly.com/blog/digging-deeper-into-log4shell-0day-rce-exploit-found-in-log4j
The Fastly Security Research Team has also successfully recreated an arbitrary ability to force DNS queries to be executed by the vulnerable Log4j instance through the use of the string:
Code (Text):
${jndi:dns://<dns server>/<TXT record query string>}
At this time, it is not clear that DNS provides a path to arbitrary code execution, but it can be used to scan for the vulnerability, or even to tunnel data through DNS when other security controls block communication (like an egress firewall rule).
Click to expand...
What we’ve seen
The jndi: URI trigger must be logged by Log4j to exploit the bug. We have observed attackers inserting the string into a variety of HTTP Headers to perform this, with User-Agent being by far the most common location. But we’ve also observed attackers attempting the offending insertion in every header that can contain arbitrary strings — and even in the URI Path itself — during the first 24 hours post-disclosure. We have even observed the jndi: string in POST bodies, which are logged less often.

All in all, it means attackers are clearly attempting all possible outlets to look for callbacks (i.e., successful control through the attacker-supplied arguments). When attackers are so actively and rapidly investing R&D into exploiting a vuln (which is not true of every 0day that becomes public), it is reasonable to assume that attack campaigns capable of exploiting the vuln in a scalable or automated way will swiftly emerge.
Click to expand...

deltahf · Dec 12, 2021

It looks like even the Mars Ingenuity rover is vulnerable!

https://twitter.com/Scott_Helme/status/1469774750083858443?s=20

eva2000 · Dec 12, 2021

deltahf said: ↑

It looks like even the Mars Ingenuity rover is vulnerable!

https://twitter.com/Scott_Helme/status/1469774750083858443?s=20
Click to expand...

Woah - didn't see that one coming! Someone hack the Mars rover to do doughnuts on the Mars

EDIT: oh it's a helicopter not Rover Mars Helicopter

buik · Dec 12, 2021

eva2000 said: ↑

Woah - didn't see that one coming! Someone hack the Mars rover to do doughnuts on the Mars

EDIT: oh it's a helicopter not Rover Mars Helicopter
Click to expand...

It doesn't matter. Because mission critical systems are always not directly connected on the free Internet (though I've seen worse).

Jimmy · Dec 13, 2021

This could be an episode of The Big Bang Theory!

eva2000 · Dec 13, 2021

buik said: ↑

It doesn't matter. Because mission critical systems are always not directly connected on the free Internet (though I've seen worse).
Click to expand...

True - but how interconnected is all the systems to systems on Earth? Who knows!

Jimmy said: ↑

This could be an episode of The Big Bang Theory!
Click to expand...

More reading

https://www.akamai.com/blog/news/CVE-2021-44228-Zero-Day-Vulnerability
Servers hosting internet-facing applications may already be compromised. To identify indicators of compromise, the following command can show exploitation attempts
Code (Text):
sudo egrep -i -r '\$\{jndi:(ldap[s]?|rmi|dns)://' /var/log
Click to expand...
and dealing with Docker image containers Apache Log4j 2 CVE-2021-44228 - Docker Blog
The configuration for the docker scan command previously shipped in Docker Desktop versions 4.3.0 and earlier unfortunately do not pick up this vulnerability on scans. Please update to Docker Desktop 4.3.1+ with docker scan 0.11.0+, which we released today, 11 December 2021.

If you are using docker scan from Linux you can download binaries from GitHub and install in the plugins directory as explained in the instructions here. We will soon update the Linux CLI version to include the updated docker scan.

If you use the updated version, you should see a message in the output log like this:
Code (Text):
Upgrade org.apache.logging.log4j:log4j-core@2.14.0 to org.apache.logging.log4j:log4j-core@2.15.0 to fix
  ✗ Arbitrary Code Execution (new) [Critical Severity][https://snyk.io/vuln/SNYK-JAVA-ORGAPACHELOGGINGLOG4J-2314720] in org.apache.logging.log4j:log4j-core@2.14.0
   introduced by org.apache.logging.log4j:log4j-core@2.14.0
Click to expand...
To test this, you can check a vulnerable image, for example this image contains a vulnerable version.
Code (Text):
docker scan elastic/logstash:7.13.3
or to cut out all the other vulnerabilities
Code (Text):
docker scan elastic/logstash:7.13.3 | grep 'Arbitrary Code Execution'
For more information about docker scan, see the documentation.
Click to expand...

eva2000 · Dec 13, 2021

log4j vulnerability POC (proof of concept) via Docker container you can test GitHub - justincormack/log4jpoc

Reproduce the vulnerability
You need to run a docker image that will safely fetch and run the vulnerable code without you having to install java or any java dependencies

Working around the vulnerability
With this repo you can play with possible workarounds and fixes, notably by moving to 2.15.0 of log4j, you can do this in the pom.xml or you can experiment with the LOG4J_FORMAT_MSG_NO_LOOKUPS=true environment variable that you can test in the included Dockerfile
Click to expand...

Jimmy · Dec 13, 2021

I knew I was waiting to setup that Docker project for a reason. Now I remember.

eva2000 · Dec 13, 2021

For folks testing my Centmin Mod fail2ban implementation @Matt I've updated it to detect log4j vulnerability scans for Centmin Mod Nginx log inspection. Details in at Update to support log4j vulnerability scans · Issue #2 · centminmod/centminmod-fail2ban. You can see an example of fail2ban detecting log4j vulnerability scans on Centmin Mod Nginx server with my fail2ban implementation installed at GitHub - centminmod/centminmod-fail2ban: fail2ban setup for centminmod.com LEMP stack with CSF Firewall. FYI, while doing testing, it actually caught a real vulnerability scan too!

There is no support for Centmin Mod fail2ban implementation, so you'd generally be on your own to configure and troubleshoot for your needs.

eva2000 · Dec 13, 2021

Seems like Cloudflare WAF rules for log4j is more about protecting themselves seeing as Cloudflare would be vulnerable due to their usage with log4j. Added benefit is protecting CF customers

My custom CF Firewall caught some too

eva2000 · Dec 13, 2021

eva2000 said: ↑

My custom CF Firewall caught some too
Click to expand...

Updated custom Cloudflare Firewall rule and also fail2ban regex for jndi:iiop pattern matching too

buik · Dec 13, 2021

Fair is fair but I don't understand the media hype towards CVE-2021-44228.
With slogans as far as "The Internet Is on Fire".

CVE-2021-44228 is clearly described and you know what to do about it.
Patch code is only a few lines of code, so it can be stress tested quickly and because little code changes. It has 'stress test wise' little impact on the rest of the software, hardware and infrastructure.

Fishing for the purpose of encrypting your data is much more dangerous, with a continuous danger, without a clear manual, patch or prompt.
Think of it as an invisible enemy that can strike at any time.

You don't hear the media about that.
But now the snowflake internet is 'on fire' with only a simple and clear CVE.

eva2000 · Dec 13, 2021

buik said: ↑

Fair is fair but I don't understand the media hype towards CVE-2021-44228.
Click to expand...

Time will tell how many folks get owned by this though.

You can actually check some of the Github repos which depend on log4j at Github Network Dependents · apache/log4j. 469,880 Github repos use, reference or depend on log4j !

buik · Dec 14, 2021

eva2000 said: ↑

Time will tell how many folks get owned by this though.
Click to expand...

Mah. The time of set and forget is over. Those days where a single server stood in the uncooled broom closet. Nowadays, you have to keep a constant eye on your infrastructure. There has been enough time to intervene.

These servers that are left now are managed without a service contract (or rather not managed) or managed without knowledge and skills. And yes, unfortunately, there are many of them. For many managements, ICT is still just a nuisance and a mere expense.

eva2000 · Dec 14, 2021

Here is hoping enough time to intervene.

Interesting usage for Cloudflare Gateway Responding to the Log4j 2 vulnerability (CVE-2021-44228)

I already use Cloudflare Gateway for home devices so extending to servers is a cool idea!

pamamolf · Dec 14, 2021

matches regex doesn't exist on Pro account

contains
equals
does not equal
does not contain

only

eva2000 · Dec 14, 2021

pamamolf said: ↑

matches regex doesn't exist on Pro account

contains
equals
does not equal
does not contain

only
Click to expand...

Yep, Cloudflare Firewall regex rules are only available in Cloudflare Business and Enterprise plans For other plans, you'd have to use some creative longer and/or rules for each possible match.

pamamolf · Dec 14, 2021

I replaced matches regex with contains...

Hope that should be ok

Unless that "contains" means any letter, number or symbol in the field.

Then i really mess it

Log in / Register

Forums

Welcome to Centmin Mod Community

Security Countless Servers Are Vulnerable to Apache Log4j Zero-Day Exploit (Log4Shell)

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

deltahf Premium Member Premium Member

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

Jimmy Well-Known Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

Jimmy Well-Known Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

eva2000 Administrator Staff Member

pamamolf Premium Member Premium Member

eva2000 Administrator Staff Member

pamamolf Premium Member Premium Member

.

Log in / Register

Useful Searches

Welcome to Centmin Mod Community

Security Countless Servers Are Vulnerable to Apache Log4j Zero-Day Exploit (Log4Shell)

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

deltahf Premium Member Premium Member

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

Jimmy Well-Known Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

Jimmy Well-Known Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

eva2000 Administrator Staff Member

pamamolf Premium Member Premium Member

eva2000 Administrator Staff Member

pamamolf Premium Member Premium Member

.