Join the community today
Become a Member

Beta Branch update prep for ModSecurity v3.0

Discussion in 'Centmin Mod Github Commits' started by eva2000, Aug 5, 2017.

  1. eva2000

    eva2000 Administrator Staff Member

    54,106
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    3:47 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    update prep for ModSecurity v3.0

    - update prep work for ModSecurity v3.0 new module architecture for Nginx ModSecurity support
    - move routine to own inc/mod_security.inc include file
    - routine change to install libmodsecurity library and nginx connector for ModSecurity, ModSecurity-nginx dynamic module


    Continue reading...

    123.09beta01 branch
     
  2. eva2000

    eva2000 Administrator Staff Member

    54,106
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    3:47 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Just making sure ModSecurity v3.0 Nginx connector module compiles properly as a Nginx dynamic module on CentOS 7.3 64bit OS running Centmin Mod 123.09beta01 LEMP stack installer.
    contents of /usr/local/nginx/conf/dynamic-modules.conf which load nginx dynamic modules
    Code (Text):
    # place custom load_module lines in this dynamic-modules-includes.conf
    # file so that they persistent i.e. for manually dropped in dynamic modules
    include /usr/local/nginx/conf/dynamic-modules-includes.conf;
    load_module "modules/ngx_http_modsecurity_module.so";
    load_module "modules/ngx_http_image_filter_module.so";
    load_module "modules/ngx_http_fancyindex_module.so";
    load_module "modules/ngx_stream_module.so";

    list of nginx dynamic modules
    Code (Text):
    ls -lah /usr/local/nginx/modules | grep -v .old
    total 884K
    drwxr-xr-x.  2 root root 4.0K Aug  4 18:27 .
    drwxr-xr-x. 12 root root 4.0K Aug  4 18:23 ..
    -rwxr-xr-x   1 root root  29K Aug  4 18:27 ngx_http_fancyindex_module.so
    -rwxr-xr-x   1 root root  29K Aug  4 18:27 ngx_http_image_filter_module.so
    -rwxr-xr-x   1 root root  35K Aug  4 18:27 ngx_http_modsecurity_module.so
    -rwxr-xr-x   1 root root 196K Aug  4 18:27 ngx_stream_module.so
    

    contents of /usr/local/nginx/modsec/modsecurity.conf
    Code (Text):
    # -- Rule engine initialization ----------------------------------------------
    
    # Enable ModSecurity, attaching it to every transaction. Use detection
    # only to start with, because that minimises the chances of post-installation
    # disruption.
    #
    SecRuleEngine DetectionOnly
    
    
    # -- Request body handling ---------------------------------------------------
    
    # Allow ModSecurity to access request bodies. If you don't, ModSecurity
    # won't be able to see any POST parameters, which opens a large security
    # hole for attackers to exploit.
    #
    SecRequestBodyAccess On
    
    
    # Enable XML request body parser.
    # Initiate XML Processor in case of xml content-type
    #
    SecRule REQUEST_HEADERS:Content-Type "(?:application(?:/soap\+|/)|text/)xml" \
         "id:'200000',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=XML"
    
    # Enable JSON request body parser.
    # Initiate JSON Processor in case of JSON content-type; change accordingly
    # if your application does not use 'application/json'
    #
    SecRule REQUEST_HEADERS:Content-Type "application/json" \
         "id:'200001',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=JSON"
    
    # Maximum request body size we will accept for buffering. If you support
    # file uploads then the value given on the first line has to be as large
    # as the largest file you are willing to accept. The second value refers
    # to the size of data, with files excluded. You want to keep that value as
    # low as practical.
    #
    SecRequestBodyLimit 13107200
    SecRequestBodyNoFilesLimit 131072
    
    # Store up to 128 KB of request body data in memory. When the multipart
    # parser reachers this limit, it will start using your hard disk for
    # storage. That is slow, but unavoidable.
    #
    SecRequestBodyInMemoryLimit 131072
    
    # What do do if the request body size is above our configured limit.
    # Keep in mind that this setting will automatically be set to ProcessPartial
    # when SecRuleEngine is set to DetectionOnly mode in order to minimize
    # disruptions when initially deploying ModSecurity.
    #
    SecRequestBodyLimitAction Reject
    
    # Verify that we've correctly processed the request body.
    # As a rule of thumb, when failing to process a request body
    # you should reject the request (when deployed in blocking mode)
    # or log a high-severity alert (when deployed in detection-only mode).
    #
    SecRule REQBODY_ERROR "!@eq 0" \
    "id:'200002', phase:2,t:none,log,deny,status:400,msg:'Failed to parse request body.',logdata:'%{reqbody_error_msg}',severity:2"
    
    # By default be strict with what we accept in the multipart/form-data
    # request body. If the rule below proves to be too strict for your
    # environment consider changing it to detection-only. You are encouraged
    # _not_ to remove it altogether.
    #
    SecRule MULTIPART_STRICT_ERROR "!@eq 0" \
    "id:'200003',phase:2,t:none,log,deny,status:400, \
    msg:'Multipart request body failed strict validation: \
    PE %{REQBODY_PROCESSOR_ERROR}, \
    BQ %{MULTIPART_BOUNDARY_QUOTED}, \
    BW %{MULTIPART_BOUNDARY_WHITESPACE}, \
    DB %{MULTIPART_DATA_BEFORE}, \
    DA %{MULTIPART_DATA_AFTER}, \
    HF %{MULTIPART_HEADER_FOLDING}, \
    LF %{MULTIPART_LF_LINE}, \
    SM %{MULTIPART_MISSING_SEMICOLON}, \
    IQ %{MULTIPART_INVALID_QUOTING}, \
    IP %{MULTIPART_INVALID_PART}, \
    IH %{MULTIPART_INVALID_HEADER_FOLDING}, \
    FL %{MULTIPART_FILE_LIMIT_EXCEEDED}'"
    
    # Did we see anything that might be a boundary?
    #
    SecRule MULTIPART_UNMATCHED_BOUNDARY "!@eq 0" \
    "id:'200004',phase:2,t:none,log,deny,msg:'Multipart parser detected a possible unmatched boundary.'"
    
    # PCRE Tuning
    # We want to avoid a potential RegEx DoS condition
    #
    SecPcreMatchLimit 1000
    SecPcreMatchLimitRecursion 1000
    
    # Some internal errors will set flags in TX and we will need to look for these.
    # All of these are prefixed with "MSC_".  The following flags currently exist:
    #
    # MSC_PCRE_LIMITS_EXCEEDED: PCRE match limits were exceeded.
    #
    SecRule TX:/^MSC_/ "!@streq 0" \
            "id:'200005',phase:2,t:none,deny,msg:'ModSecurity internal error flagged: %{MATCHED_VAR_NAME}'"
    
    
    # -- Response body handling --------------------------------------------------
    
    # Allow ModSecurity to access response bodies.
    # You should have this directive enabled in order to identify errors
    # and data leakage issues.
    #
    # Do keep in mind that enabling this directive does increases both
    # memory consumption and response latency.
    #
    SecResponseBodyAccess On
    
    # Which response MIME types do you want to inspect? You should adjust the
    # configuration below to catch documents but avoid static files
    # (e.g., images and archives).
    #
    SecResponseBodyMimeType text/plain text/html text/xml
    
    # Buffer response bodies of up to 512 KB in length.
    SecResponseBodyLimit 524288
    
    # What happens when we encounter a response body larger than the configured
    # limit? By default, we process what we have and let the rest through.
    # That's somewhat less secure, but does not break any legitimate pages.
    #
    SecResponseBodyLimitAction ProcessPartial
    
    
    # -- Filesystem configuration ------------------------------------------------
    
    # The location where ModSecurity stores temporary files (for example, when
    # it needs to handle a file upload that is larger than the configured limit).
    #
    # This default setting is chosen due to all systems have /tmp available however,
    # this is less than ideal. It is recommended that you specify a location that's private.
    #
    SecTmpDir /tmp/
    
    # The location where ModSecurity will keep its persistent data.  This default setting
    # is chosen due to all systems have /tmp available however, it
    # too should be updated to a place that other users can't access.
    #
    SecDataDir /tmp/
    
    
    # -- File uploads handling configuration -------------------------------------
    
    # The location where ModSecurity stores intercepted uploaded files. This
    # location must be private to ModSecurity. You don't want other users on
    # the server to access the files, do you?
    #
    #SecUploadDir /opt/modsecurity/var/upload/
    
    # By default, only keep the files that were determined to be unusual
    # in some way (by an external inspection script). For this to work you
    # will also need at least one file inspection rule.
    #
    #SecUploadKeepFiles RelevantOnly
    
    # Uploaded files are by default created with permissions that do not allow
    # any other user to access them. You may need to relax that if you want to
    # interface ModSecurity to an external program (e.g., an anti-virus).
    #
    #SecUploadFileMode 0600
    
    
    # -- Debug log configuration -------------------------------------------------
    
    # The default debug log configuration is to duplicate the error, warning
    # and notice messages from the error log.
    #
    #SecDebugLog /opt/modsecurity/var/log/debug.log
    #SecDebugLogLevel 3
    
    
    # -- Audit log configuration -------------------------------------------------
    
    # Log the transactions that are marked by a rule, as well as those that
    # trigger a server error (determined by a 5xx or 4xx, excluding 404,
    # level response status codes).
    #
    SecAuditEngine RelevantOnly
    SecAuditLogRelevantStatus "^(?:5|4(?!04))"
    
    # Log everything we know about a transaction.
    SecAuditLogParts ABIJDEFHZ
    
    # Use a single file for logging. This is much easier to look at, but
    # assumes that you will use the audit log only ocassionally.
    #
    SecAuditLogType Serial
    SecAuditLog /var/log/modsec_audit.log
    
    # Specify the path for concurrent audit logging.
    #SecAuditLogStorageDir /opt/modsecurity/var/audit/
    
    
    # -- Miscellaneous -----------------------------------------------------------
    
    # Use the most commonly used application/x-www-form-urlencoded parameter
    # separator. There's probably only one application somewhere that uses
    # something else so don't expect to change this value.
    #
    SecArgumentSeparator &
    
    # Settle on version 0 (zero) cookies, as that is what most applications
    # use. Using an incorrect cookie version may open your installation to
    # evasion attacks (against the rules that examine named cookies).
    #
    SecCookieFormat 0
    
    # Specify your Unicode Code Point.
    # This mapping is used by the t:urlDecodeUni transformation function
    # to properly map encoded data to your language. Properly setting
    # these directives helps to reduce false positives and negatives.
    #
    SecUnicodeMapFile unicode.mapping 20127
    
    # Improve the quality of ModSecurity by sharing information about your
    # current ModSecurity version and dependencies versions.
    # The following information will be shared: ModSecurity version,
    # Web Server version, APR version, PCRE version, Lua version, Libxml2
    # version, Anonymous unique id for host.
    SecStatusEngine On
    


    test ModSecurity rule in SecRuleEngine DetectionOnly mode with custom rule in /usr/local/nginx/modsec/main.conf
    Code (Text):
    # Basic test rule
    SecRule ARGS:testparam "@contains test" "id:1234,deny,status:403"
    

    full contents of /usr/local/nginx/modsec/main.conf
    Code (Text):
    # Edit to set SecRuleEngine On
    Include "/usr/local/nginx/modsec/modsecurity.conf"
    
    # OWASP CRS v3 rules
    Include "/usr/local/nginx/owasp-modsecurity-crs-3.0.2/crs-setup.conf"
    Include "/usr/local/nginx/owasp-modsecurity-crs-3.0.2/rules/*.conf"
    
    # Basic test rule
    SecRule ARGS:testparam "@contains test" "id:1234,deny,status:403"
    

    Code (Text):
    curl -I localhost?testparam=test
    

    Code (Text):
    tail -100 /var/log/modsec_audit.log
    
    ---H2uPlUVD---A--
    [04/Aug/2017:18:44:19 +0000] 150187225955.446369 127.0.0.1 26364 127.0.0.1 80
    ---H2uPlUVD---B--
    HEAD /?testparam=test HTTP/1.1
    User-Agent: curl/7.29.0
    Host: localhost
    Accept: */*
    
    ---H2uPlUVD---D--
    
    ---H2uPlUVD---F--
    HTTP/1.1 200
    X-Powered-By: centminmod
    Connection: keep-alive
    ETag: "597d5cfe-ed9"
    Last-Modified: Sun, 30 Jul 2017 04:13:50 GMT
    Vary: Accept-Encoding
    Content-Type: text/html
    Content-Length: 3801
    Date: Fri, 04 Aug 2017 18:44:19 GMT
    Server: nginx centminmod
    Server: nginx centminmod
    Accept-Ranges: bytes
    
    ---H2uPlUVD---H--
    ModSecurity: Warning. Matched "Operator `Contains' with parameter `test' against variable `ARGS:testparam' (Value: `test' ) [file "/usr/local/nginx/modsec/main.conf"] [line "207"] [id "1234"] [rev ""] [msg ""] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [ref "o0,4v17,4"]
    
    ---H2uPlUVD---I--
    
    ---H2uPlUVD---J--
    
    ---H2uPlUVD---Z--
    

    testing OWASP Modsecurity v3 Core Ruleset (CRS). CRS recognizes requests from scanners, including Nikto via User‑Agent header. The CRS comes preconfigured to block requests that have the default User‑Agent header for Nikto (Nikto).
    Code (Text):
    curl -H "User-Agent: Nikto" http://localhost
    

    Code (Text):
    tail -100 /var/log/modsec_audit.log
    
    ---tgi4oVmS---F--
    HTTP/1.1 200
    X-Powered-By: centminmod
    Connection: keep-alive
    ETag: "597d5cfe-ed9"
    Last-Modified: Sun, 30 Jul 2017 04:13:50 GMT
    Vary: Accept-Encoding
    Content-Type: text/html
    Content-Length: 3801
    Date: Fri, 04 Aug 2017 19:18:45 GMT
    Server: nginx centminmod
    Server: nginx centminmod
    Accept-Ranges: bytes
    
    ---tgi4oVmS---H--
    ModSecurity: Warning. Matched "Operator `PmFromFile' with parameter `scanners-user-agents.data' against variable `REQUEST_HEADERS:User-Agent' (Value: `Nikto' ) [file "/usr/local/nginx/owasp-modsecurity-crs-3.0.2/rules/REQUEST-913-SCANNER-DETECTION.conf"] [line "17"] [id "913100"] [rev "2"] [msg "Found User-Agent associated with security scanner"] [data "Matched Data: nikto found within REQUEST_HEADERS:User-Agent: Nikto"] [severity "2"] [ver "OWASP_CRS/3.0.0"] [maturity "9"] [accuracy "9"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-reputation-scanner"] [tag "OWASP_CRS/AUTOMATION/SECURITY_SCANNER"] [tag "WASCTC/WASC-21"] [tag "OWASP_TOP_10/A7"] [tag "PCI/6.5.10"] [ref "o0,5v55,5t:lowercase"]
    ModSecurity: Warning. Matched "Operator `Ge' with parameter `%{tx.inbound_anomaly_score_threshold}' against variable `TX:ANOMALY_SCORE' (Value: `5' ) [file "/usr/local/nginx/owasp-modsecurity-crs-3.0.2/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "36"] [id "949110"] [rev ""] [msg "Inbound Anomaly Score Exceeded (Total Score: 5)"] [data ""] [severity "2"] [ver ""] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-generic"] [ref ""]
    ModSecurity: Warning. Matched "Operator `Ge' with parameter `%{tx.inbound_anomaly_score_threshold}' against variable `TX:INBOUND_ANOMALY_SCORE' (Value: `5' ) [file "/usr/local/nginx/owasp-modsecurity-crs-3.0.2/rules/RESPONSE-980-CORRELATION.conf"] [line "61"] [id "980130"] [rev ""] [msg "Inbound Anomaly Score Exceeded (Total Inbound Score: 5 - SQLI=0,XSS=0,RFI=0,LFI=0,RCE=0,PHPI=0,HTTP=0,SESS=0): Found User-Agent associated with security scanner'"] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [tag "event-correlation"] [ref ""]
    
    ---tgi4oVmS---I--
    
    ---tgi4oVmS---J--
    
    ---tgi4oVmS---Z--
    

    OWASP ModSecurity v3 Core Rule Sets listed in /usr/local/nginx/owasp-modsecurity-crs-3.0.2/rules/
    Code (Text):
    ls -lah /usr/local/nginx/owasp-modsecurity-crs-3.0.2/rules/
    total 572K
    drwxrwxr-x 2 root root 4.0K May 12 17:11 .
    drwxrwxr-x 6 root root 4.0K Aug  4 19:05 ..
    -rw-rw-r-- 1 root root  659 May 12 17:11 crawlers-user-agents.data
    -rw-rw-r-- 1 root root  551 May 12 17:11 iis-errors.data
    -rw-rw-r-- 1 root root  264 May 12 17:11 java-code-leakages.data
    -rw-rw-r-- 1 root root  240 May 12 17:11 java-errors.data
    -rw-rw-r-- 1 root root  30K May 12 17:11 lfi-os-files.data
    -rw-rw-r-- 1 root root 5.3K May 12 17:11 php-config-directives.data
    -rw-rw-r-- 1 root root 9.0K May 12 17:11 php-errors.data
    -rw-rw-r-- 1 root root  589 May 12 17:11 php-function-names-933150.data
    -rw-rw-r-- 1 root root  21K May 12 17:11 php-function-names-933151.data
    -rw-rw-r-- 1 root root  224 May 12 17:11 php-variables.data
    -rw-rw-r-- 1 root root 7.3K May 12 17:11 REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf.example
    -rw-rw-r-- 1 root root 9.9K May 12 17:11 REQUEST-901-INITIALIZATION.conf
    -rw-rw-r-- 1 root root  13K May 12 17:11 REQUEST-903.9001-DRUPAL-EXCLUSION-RULES.conf
    -rw-rw-r-- 1 root root  19K May 12 17:11 REQUEST-903.9002-WORDPRESS-EXCLUSION-RULES.conf
    -rw-rw-r-- 1 root root 1.4K May 12 17:11 REQUEST-905-COMMON-EXCEPTIONS.conf
    -rw-rw-r-- 1 root root  10K May 12 17:11 REQUEST-910-IP-REPUTATION.conf
    -rw-rw-r-- 1 root root 2.7K May 12 17:11 REQUEST-911-METHOD-ENFORCEMENT.conf
    -rw-rw-r-- 1 root root 9.1K May 12 17:11 REQUEST-912-DOS-PROTECTION.conf
    -rw-rw-r-- 1 root root 7.5K May 12 17:11 REQUEST-913-SCANNER-DETECTION.conf
    -rw-rw-r-- 1 root root  48K May 12 17:11 REQUEST-920-PROTOCOL-ENFORCEMENT.conf
    -rw-rw-r-- 1 root root  13K May 12 17:11 REQUEST-921-PROTOCOL-ATTACK.conf
    -rw-rw-r-- 1 root root 6.3K May 12 17:11 REQUEST-930-APPLICATION-ATTACK-LFI.conf
    -rw-rw-r-- 1 root root 5.7K May 12 17:11 REQUEST-931-APPLICATION-ATTACK-RFI.conf
    -rw-rw-r-- 1 root root  47K May 12 17:11 REQUEST-932-APPLICATION-ATTACK-RCE.conf
    -rw-rw-r-- 1 root root  31K May 12 17:11 REQUEST-933-APPLICATION-ATTACK-PHP.conf
    -rw-rw-r-- 1 root root  41K May 12 17:11 REQUEST-941-APPLICATION-ATTACK-XSS.conf
    -rw-rw-r-- 1 root root  56K May 12 17:11 REQUEST-942-APPLICATION-ATTACK-SQLI.conf
    -rw-rw-r-- 1 root root 5.4K May 12 17:11 REQUEST-943-APPLICATION-ATTACK-SESSION-FIXATION.conf
    -rw-rw-r-- 1 root root 3.2K May 12 17:11 REQUEST-949-BLOCKING-EVALUATION.conf
    -rw-rw-r-- 1 root root 3.8K May 12 17:11 RESPONSE-950-DATA-LEAKAGES.conf
    -rw-rw-r-- 1 root root  20K May 12 17:11 RESPONSE-951-DATA-LEAKAGES-SQL.conf
    -rw-rw-r-- 1 root root 3.8K May 12 17:11 RESPONSE-952-DATA-LEAKAGES-JAVA.conf
    -rw-rw-r-- 1 root root 5.1K May 12 17:11 RESPONSE-953-DATA-LEAKAGES-PHP.conf
    -rw-rw-r-- 1 root root 5.9K May 12 17:11 RESPONSE-954-DATA-LEAKAGES-IIS.conf
    -rw-rw-r-- 1 root root 2.8K May 12 17:11 RESPONSE-959-BLOCKING-EVALUATION.conf
    -rw-rw-r-- 1 root root 4.4K May 12 17:11 RESPONSE-980-CORRELATION.conf
    -rw-rw-r-- 1 root root 3.0K May 12 17:11 RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf.example
    -rw-rw-r-- 1 root root  713 May 12 17:11 restricted-files.data
    -rw-rw-r-- 1 root root  216 May 12 17:11 scanners-headers.data
    -rw-rw-r-- 1 root root  418 May 12 17:11 scanners-urls.data
    -rw-rw-r-- 1 root root 4.0K May 12 17:11 scanners-user-agents.data
    -rw-rw-r-- 1 root root  717 May 12 17:11 scripting-user-agents.data
    -rw-rw-r-- 1 root root 1.9K May 12 17:11 sql-errors.data
    -rw-rw-r-- 1 root root 2.0K May 12 17:11 sql-function-names.data
    -rw-rw-r-- 1 root root  943 May 12 17:11 unix-shell.data
    -rw-rw-r-- 1 root root 3.9K May 12 17:11 windows-powershell-commands.data
    

    The ruleset for Nikto located in /usr/local/nginx/owasp-modsecurity-crs-3.0.2/rules/scanners-user-agents.data
    Code (Text):
    grep -rin Nikto /usr/local/nginx/owasp-modsecurity-crs-3.0.2/rules/
    /usr/local/nginx/owasp-modsecurity-crs-3.0.2/rules/scanners-user-agents.data:106:# https://cirt.net/Nikto2
    /usr/local/nginx/owasp-modsecurity-crs-3.0.2/rules/scanners-user-agents.data:107:nikto
    

    Code (Text):
    grep -C5 Nikto /usr/local/nginx/owasp-modsecurity-crs-3.0.2/rules/scanners-user-agents.data
    nessus
    # vuln scanner
    # https://www.netsparker.com/web-vulnerability-scanner/
    netsparker
    # vuln scanner
    # https://cirt.net/Nikto2
    nikto
    # vuln scanner
    nmap nse
    nmap scripting engine
    nmap-nse
    
     
  3. yves

    yves New Member

    16
    3
    3
    Jan 17, 2016
    Ratings:
    +4
    Local Time:
    6:47 PM
  4. eva2000

    eva2000 Administrator Staff Member

    54,106
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    3:47 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    cheers pull request merged :)

    FYI, in future do the mods on 123.09beta01 branch so i can merge into 123.09beta01. I only do 123.09beta01 to master merges once a week usually as development is actually on 123.09beta01 branch first :)

    Added contributing guidelines file just to clarify stuff :)
     
    Last edited: Aug 30, 2017
  5. yves

    yves New Member

    16
    3
    3
    Jan 17, 2016
    Ratings:
    +4
    Local Time:
    6:47 PM
    Thanks for merging the PR.
    Noted for contribution guidelines ;)

    Regarding Modsecurity install, I've noticed that I've had to manually enter entries (like "modsecurity on" in Nginx conf files).
    Yet it seems that this should be done automatically (at least for default host) according to modsec install inc file.
    The thing is, *I think*, if I understand things properly (still trying to discover how these installs interacts) that Modsec install occurs before Nginx install occurs.
    Hence Nginx conf files aren't yet created when Modesc install should insert specific entries in Nginx conf files.
    Would this be the reason why default modsec entries are missing in Nginx conf ?

    Maybe modsec install should create Nginx default conf files if they don't exist (yet) ?
     
  6. eva2000

    eva2000 Administrator Staff Member

    54,106
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    3:47 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    where you looking ? which files ? only the main vhost /usr/local/nginx/conf/conf.d/virtual.conf has setup modsecurity on/off directive. It's not included in every vhost config file. I left that up to end users who know what they're doing to do. The modsecurity nginx implementation is still a work in progress to make it more end user friendly. The above is just prep work to at least make sure modsecurity installs and works for now :)
     
  7. yves

    yves New Member

    16
    3
    3
    Jan 17, 2016
    Ratings:
    +4
    Local Time:
    6:47 PM
    Yes so far I've just searched in /usr/local/nginx/conf/conf.d/virtual.conf
    But the thing is that I've had to manually add following directives in virtual.conf file
    modsecurity on;
    modsecurity_rules_file /usr/local/nginx/modsec/main.conf;

    No worry, this is not a big deal for me and I take notes that this is just a prep work (and also understand that it 's up to each user to configure everything to their need ;)).
    However, unless I don't get it right, I seem to have noticed that mod_security.inc install is supposed to add these directives in virtual.conf (lines 116 and following )

    I think the reason why for this is that modsec install may happen before this virtual.conf is created by nginx install to start with, so it cannot modify this non existing file.
     
  8. eva2000

    eva2000 Administrator Staff Member

    54,106
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    3:47 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Yeah it should add it but will double check the order of things :)

    does it get added if you do a nginx recompile via centmin.sh menu option 4 ?
     
  9. yves

    yves New Member

    16
    3
    3
    Jan 17, 2016
    Ratings:
    +4
    Local Time:
    6:47 PM
    Yes indeed, directives get added if I recompile Nginx.
     
  10. eva2000

    eva2000 Administrator Staff Member

    54,106
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    3:47 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    yeah iirc, that's how it's intended right now when i added it for testing it was meant for nginx recompiles not fresh centmin mod installed nginx. Will look at fresh install nginx further down the track :)
     
  11. yves

    yves New Member

    16
    3
    3
    Jan 17, 2016
    Ratings:
    +4
    Local Time:
    6:47 PM
    Oh I see.
    Well, then everything is working as expected :)
     
  12. noly

    noly Member

    106
    16
    18
    Jul 24, 2017
    Germany
    Ratings:
    +28
    Local Time:
    6:47 PM
    1.18.x
    10.2.x
    In order to enable ModSecurity one has to add

    Code:
    NGINX_MODSECURITY='y'
    to the custom_config.inc, right?
     
  13. eva2000

    eva2000 Administrator Staff Member

    54,106
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    3:47 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    yes but unless you know how to configure and tweak modsecurity not a good idea to enable ;)
     
  14. enderst

    enderst Member

    38
    7
    8
    Dec 12, 2017
    Ratings:
    +10
    Local Time:
    10:47 AM
    Looking for info to get modsecurity installed.
     
  15. eva2000

    eva2000 Administrator Staff Member

    54,106
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    3:47 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    modsecurity isn't 100% officially supported yet but you can test it on 123.09beta01+ and higher as outlined by @noly here Beta Branch - update prep for ModSecurity v3.0 by setting in persistent config file at /etc/centminmod/custom_config.inc , the variable NGINX_MODSECURITY='y' and then run centmin.sh menu option 4 to recompile nginx.

    Then general outline of config files is at Beta Branch - update prep for ModSecurity v3.0 that you use to configure for your vhosts

    but modsecurity configuration and setup is let up to end users to do. CentminMod allows you to add modsecurity nginx module to Nginx server, the rest is up to you so you need to know how to configure and set it up.

    Just did a quick test to see if nginx 1.15.0 compiles with modsecurity v3 module and looks like it deos

     
  16. EckyBrazzz

    EckyBrazzz Active Member

    916
    189
    43
    Mar 28, 2018
    >>>>Click here<<<< i'm nearby......
    Ratings:
    +362
    Local Time:
    2:47 PM
    Latest
    Latest
    Just did a quick test, added
    Code:
    NGINX_MODSECURITY='y'
    But it did not get compiled into nginx....
    Missing the --add-dynamic-module=../ModSecurity-nginx

    And Curl -I give a 404, error with nginx -t look like this

    Changed it quickly back to
    Code:
    NGINX_MODSECURITY='n'
    Recompiled nginx, but this won't remove the nginx -t error. So manuly commenting out in the virtual.conf resolved it.
    Code (Text):
    #    modsecurity on;
    #    modsecurity_rules_file /usr/local/nginx/modsec/main.conf;
    #
    #        location /nginx_status {
    #        stub_status on;
    #        access_log   off;
    #        allow 127.0.0.1;
    #        #allow youripaddress;
    #        deny all;
    #        }
    
    


    Too bad I'm always running out of time, but really gonna use it in the future! Looks to me a must have on CMM.
     
    Last edited: Apr 30, 2019
  17. eva2000

    eva2000 Administrator Staff Member

    54,106
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    3:47 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    To troubleshoot, you need to check the full nginx upgrade log at /root/centminlogs and instructions under Sharing logs and errors heading for using Pastebin.com or Gists to share a sanitised version of the contents of the nginx_upgrade.log log. You can see full details at How to troubleshoot Centmin Mod initial install issues

    if you type this command it lists all logs in date ascending order so latest log at bottom - for you as you recompiled nginx twice, the modsecurity one would be the 2nd last nginx_upgrade.log instead of last nginx_upgrade.log
    Code (Text):
    ls -lArt /root/centminlogs

    so copy the entire contents of latest nginx_upgrade log to gist.github.com or pastebin.com

    you can use grep to filter the logs, i.e. look for nginx_upgrade in log name
    Code (Text):
    ls -lahrt /root/centminlogs/ | grep nginx_upgrade
    -rw-r--r--  1 root root 3.2M Oct 11 15:55 centminmod_1.2.3-eva2000.09.001_111016-155345_nginx_upgrade.log
    -rw-r--r--  1 root root 672K Oct 11 22:06 centminmod_1.2.3-eva2000.09.001_111016-220515_nginx_upgrade.log

    So the last nginx upgrade log was named centminmod_1.2.3-eva2000.09.001_111016-220515_nginx_upgrade.log and located at /root/centminlogs/centminmod_1.2.3-eva2000.09.001_111016-220515_nginx_upgrade.log

    then use cat command to output the contents of that log

    clear your ssh window buffer/screen and type
    Code (Text):
    cat /root/centminlogs/centminmod_1.2.3-eva2000.09.001_111016-220515_nginx_upgrade.log

    then select and copy and paste output to pastebin.com or gist.github.com file to share. If your SSH client's scroll buffer isn't large enough using cat might not output the entire log file contents, so you may need to download the log and use local text editor to open and copy and paste.

    so only need content of one specific log, in this case most recent nginx_upgrade.log log
     
  18. eva2000

    eva2000 Administrator Staff Member

    54,106
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    3:47 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Ok I think I know why modsecurity failed to install libmodsecurity due to GCC compiler version used in modsecurity_install function. I updated 123.09beta01 branch with modsecurity for nginx fixes to use GCC 8/7 instead of 6/4 https://community.centminmod.com/th...urity_install-function-in-123-09beta01.17359/.

    Enable nginx modsecurity module via NGINX_MODSECURITY='y' set in persistent config file /etc/centminmod/custom_config.inc BEFORE running centmin.sh menu option 4 to recompile Nginx 1.15.12
    Code (Text):
    nginx -t
    nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
    nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful
    

    dynamic modules load include file /usr/local/nginx/conf/dynamic-modules.conf contents
    Code (Text):
    # place custom load_module lines in this dynamic-modules-includes.conf
    # file so that they persistent i.e. for manually dropped in dynamic modules
    include /usr/local/nginx/conf/dynamic-modules-includes.conf;
    load_module "modules/ngx_http_modsecurity_module.so";
    load_module "modules/ngx_http_image_filter_module.so";
    load_module "modules/ngx_http_headers_more_filter_module.so";
    load_module "modules/ndk_http_module.so";
    load_module "modules/ngx_http_set_misc_module.so";
    load_module "modules/ngx_http_echo_module.so";
    load_module "modules/ngx_http_fancyindex_module.so";
    load_module "modules/ngx_http_vhost_traffic_status_module.so";
    load_module "modules/ngx_pagespeed.so";
    load_module "modules/ngx_http_brotli_filter_module.so";
    load_module "modules/ngx_http_brotli_static_module.so";
    

    nginx dynamic modules themselves installed at /usr/local/nginx/modules where modsecurity dynamic nginx module = ngx_http_modsecurity_module.so
    Code (Text):
    ls -lah /usr/local/nginx/modules | grep -v .old
    total 41M
    drwxr-xr-x.  2 root root 4.0K Apr 30 10:45 .
    drwxr-xr-x. 13 root root 4.0K Apr 30 10:39 ..
    -rwxr-xr-x   1 root root 104K Apr 30 10:45 ndk_http_module.so
    -rwxr-xr-x   1 root root  97K Apr 30 10:45 ngx_http_brotli_filter_module.so
    -rwxr-xr-x   1 root root  90K Apr 30 10:45 ngx_http_brotli_static_module.so
    -rwxr-xr-x   1 root root 532K Apr 30 10:45 ngx_http_echo_module.so
    -rwxr-xr-x   1 root root 114K Apr 30 10:45 ngx_http_fancyindex_module.so
    -rwxr-xr-x   1 root root 222K Apr 30 10:45 ngx_http_headers_more_filter_module.so
    -rwxr-xr-x   1 root root 108K Apr 30 10:45 ngx_http_image_filter_module.so
    -rwxr-xr-x   1 root root 291K Apr 30 10:45 ngx_http_modsecurity_module.so
    -rwxr-xr-x   1 root root 643K Apr 30 10:45 ngx_http_set_misc_module.so
    -rwxr-xr-x   1 root root 781K Apr 30 10:45 ngx_http_vhost_traffic_status_module.so
    -rwxr-xr-x   1 root root  18M Apr 30 10:45 ngx_pagespeed.so
    

    in virtual.conf main hostname
    Code (Text):
        modsecurity on;
        modsecurity_rules_file /usr/local/nginx/modsec/main.conf;
    

    contents of /usr/local/nginx/modsec/main.conf
    Code (Text):
    # Edit to set SecRuleEngine On
    Include "/usr/local/nginx/modsec/modsecurity.conf"
    
    # OWASP CRS v3 rules
    Include "/usr/local/nginx/owasp-modsecurity-crs-3.0.2/crs-setup.conf"
    Include "/usr/local/nginx/owasp-modsecurity-crs-3.0.2/rules/*.conf"
    
    # Basic test rule
    SecRule ARGS:testparam "@contains test" "id:1234,deny,status:403"
    

    directory contents for /usr/local/nginx/modsec
    Code (Text):
    ls -lah /usr/local/nginx/modsec
    total 76K
    drwxr-xr-x   2 root root 4.0K Apr 30 10:39 .
    drwxr-xr-x. 13 root root 4.0K Apr 30 10:39 ..
    -rw-r--r--   1 root root  327 Apr 30 10:39 main.conf
    -rw-r--r--   1 root root  10K Apr 30 10:39 modsecurity.conf
    -rw-r--r--   1 root root  52K Apr 30 10:39 unicode.mapping
    

    directory contents for /usr/local/nginx/owasp-modsecurity-crs-3.0.2
    Code (Text):
    ls -lah /usr/local/nginx/owasp-modsecurity-crs-3.0.2
    total 196K
    drwxrwxr-x   6 root root 4.0K Apr 30 10:39 .
    drwxr-xr-x. 13 root root 4.0K Apr 30 10:39 ..
    -rw-rw-r--   1 root root  45K May 12  2017 CHANGES
    -rw-rw-r--   1 root root  192 May 12  2017 CONTRIBUTORS
    -rw-r--r--   1 root root  30K Apr 30 10:39 crs-setup.conf
    -rw-rw-r--   1 root root  30K May 12  2017 crs-setup.conf.example
    drwxrwxr-x   3 root root 4.0K May 12  2017 documentation
    -rw-rw-r--   1 root root  253 May 12  2017 .gitignore
    -rw-rw-r--   1 root root  362 May 12  2017 .gitmodules
    -rw-rw-r--   1 root root  460 May 12  2017 IDNUMBERING
    drwxrwxr-x   2 root root 4.0K May 12  2017 id_renumbering
    -rw-rw-r--   1 root root  17K May 12  2017 INSTALL
    -rw-rw-r--   1 root root 2.8K May 12  2017 KNOWN_BUGS
    -rw-rw-r--   1 root root  12K May 12  2017 LICENSE
    -rw-rw-r--   1 root root 1.8K May 12  2017 README.md
    drwxrwxr-x   2 root root 4.0K May 12  2017 rules
    -rw-rw-r--   1 root root  216 May 12  2017 .travis.yml
    drwxrwxr-x  12 root root 4.0K May 12  2017 util
    

    Then test disable nginx modsecurity module via NGINX_MODSECURITY='n' set in persistent config file /etc/centminmod/custom_config.inc BEFORE running centmin.sh menu option 4 to recompile Nginx 1.15.12

    check virtual.conf should now have the 2 modsecurity directives commented out
    Code (Text):
        #modsecurity on;
        #modsecurity_rules_file /usr/local/nginx/modsec/main.conf;
    
     
  19. EckyBrazzz

    EckyBrazzz Active Member

    916
    189
    43
    Mar 28, 2018
    >>>>Click here<<<< i'm nearby......
    Ratings:
    +362
    Local Time:
    2:47 PM
    Latest
    Latest
    Latest log is here.
     
    Last edited by a moderator: Apr 30, 2019
  20. eva2000

    eva2000 Administrator Staff Member

    54,106
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    3:47 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    that seems to be log for NGINX_MODSECURITY='n' run right which is latest log, the log prior to that would be for NGINX_MODSECURITY='y'. Either way I believe I fixed it so if you do a cmupdate to update 123.09beta01 then set NGINX_MODSECURITY='y' and centmin.sh menu option 4 recompile nginx, it should successfully install modsecurity for nginx

    cmupdate
    Code (Text):
    cmupdate
    Saved working directory and index state WIP on 123.09beta01: 56a7a13 part 6 update centmin.sh menu option 22 wordpress routines in 123.09beta01
    HEAD is now at 56a7a13 part 6 update centmin.sh menu option 22 wordpress routines in 123.09beta01
    remote: Enumerating objects: 4, done.
    remote: Counting objects: 100% (4/4), done.
    remote: Compressing objects: 100% (3/3), done.
    remote: Total 4 (delta 1), reused 2 (delta 1), pack-reused 0
    Unpacking objects: 100% (4/4), done.
    From https://github.com/centminmod/centminmod
       56a7a13..eb6d40c  123.09beta01 -> origin/123.09beta01
    Updating 56a7a13..eb6d40c
    Fast-forward
     inc/mod_security.inc | 25 +++++++++++++++++--------
     1 file changed, 17 insertions(+), 8 deletions(-)