Beta Branch autoprotect.sh - apache .htaccess check & migration to nginx deny all

Nginx web servers do not support Apache web server .htaccess files. So any web application which relies on .htaccess to protect directories will be fully open to the public on Nginx web servers unless your Nginx vhost config file has specific location directory matches to deny and/or password protect them.

---

I'm developing an autoprotect.sh script (/usr/local/src/centminmod/tools/autoprotect.sh) which basically checks entire Centmin Mod LEMP stack server's Nginx vhost site's public web accessible directories for .htaccess files that exist. Then grabbing their directory path to .htaccess file and auto generating an Nginx equivalent location match and deny all setup when a web url curl check reveals that the HTTP status code of the directory path is not matching 403 or 404. Which means if any directories which already have equivalent Nginx deny all (403) or 404 HTTP status code based location matches will be skipped and no auto generated nginx location auto protect rules will be made. Update had to disable curl 403/404 prechecks though as running subsequent autoprotect.sh runs would empty out the domain's include auto protect file as 403 is detected from previous autoprotect.sh run. So instead all .htaccess 'deny from all' detected directories now get auto generated Nginx equivalent location match and deny all setups except if you want to manually bypass the directory from auto protection via a .autoprotect-bypass file - details below here.

An example of Apache directory's .htaccess deny all file at /privatedirectory/.htaccess

Code (Text):

Order deny,allow
Deny from all

An example of Nginx directory at /privatedirectory having deny all set for 403 permission denied

Code (Text):

location ~* ^/privatedirectory { deny all; }

Running autoprotect.sh will transverse through all Nginx vhost sites at /home/nginx/domains/ looking for any .htaccess files which do not already return a 403 or 404 HTTP status and generate the corresponding Nginx location match deny all config in an include file specific for each domain i.e. /usr/local/nginx/conf/autoprotect/domain.com/autoprotect-domain.com.conf where domain name = domain.com

Code (Text):

/usr/local/src/centminmod/tools/autoprotect.sh
generated nginx include file: /usr/local/nginx/conf/autoprotect/domain.com/autoprotect-domain.com.conf

autoprotect.sh run completed...

contents of include file /usr/local/nginx/conf/autoprotect/domain.com/autoprotect-domain.com.conf where autoprotect.sh detected a matching .htaccess file at
/home/nginx/domains/domain.com/public/privatedirectory/.htaccess which needs a nginx equivalent location match deny all rule generated

Code (Text):

# /home/nginx/domains/domain.com/public/privatedirectory
location ~* ^/privatedirectory { deny all; }

Then in your domain.com nginx vhost file you'd have the autoprotect include file placed above the root definition i.e.

Code (Text):

  include /usr/local/nginx/conf/autoprotect/domain.com/autoprotect-domain.com.conf;
  root /home/nginx/domains/domain.com/public;

setup a cronjob to run autoprotect.sh every 4 hours or whatever you want frequency wise. The autoprotect.sh script will also nginx service reload when triggered so any updated newly found .htaccess files will re-populate each individual site autoprotect include files i.e. /usr/local/nginx/conf/autoprotect/domain.com/autoprotect-domain.com.conf with updated rules.

Code (Text):

0 */4 * * * /usr/local/src/centminmod/tools/autoprotect.sh

/usr/local/src/centminmod/tools/autoprotect.sh is not available yet, still doing internal testing for now.

 

Added to 123.09beta01 latest code

• Beta Branch - add tools/autoprotect.sh to 123.09beta01 | Centmin Mod Community

 

Update tools/autoprotect.sh to add .autoprotect-bypass file option

Give end user the option to manually bypass autoprotect.sh script and NOT create a nginx deny all location match by manually creating a .autoprotect-bypass file within the directory you want to bypass and exclude from autoprotect.sh. You may want to do this if your nginx deny location match for a directory involves whitelisting ip addresses' access to the directory.

The autoprotect.sh has no way of detecting if you already setup a nginx deny location match for a directory but also added whitelisted ips for the nginx location so will try to generate a nginx deny all rule if .htaccess file is in the directory with contents of 'deny from all' text. If you manually create a /privatedirectoryname/.autoprotect-bypass file, autoprotect.sh script skips generating the nginx deny all rule.

Tip: if you need a complete list of directories which contain .htaccess deny from all files, you can just run tools/autoprotect.sh manually and then look in the contents of generated autoprotect include files i.e. for domain.com include file at
/usr/local/nginx/conf/autoprotect/domain.com/autoprotect-domain.com.conf. Then see which directories you need to drop a .autoprotect-bypass file to exclude from autoprotecting and re-run tools/autoprotect.sh to exclude them.

Can also grep for word location within /usr/local/nginx/conf/autoprotect/domain.com/autoprotect-domain.com.conf to see the directories too.

Code (Text):

grep location /usr/local/nginx/conf/autoprotect/domain.com/autoprotect-domain.com.conf

Notes

Using Nginx with any web app like wordpress requires understanding the web app's author intention to block or prevent sensitive files or directories with .htaccess and then understanding that Nginx doesn't support .htaccess so you need to write a custom location match/protection for each of those directories with .htaccess files provided by web app's author for security.

The tools/autoprotect.sh Beta Branch - autoprotect.sh - apache .htaccess check & migration to nginx deny all aims to make it easier to identify and alert owners to which directories/files you need to more closely look at to manually write up a nginx custom rule to do the equivalent task as intended by .htaccess file. As per above outline, you can check with grep command which directories you need to pay closer attention to with command (and bypass if needed)

where domain.com is yourdomain.com name

Code (Text):

grep location /usr/local/nginx/conf/autoprotect/domain.com/autoprotect-domain.com.conf

Once, you have the nginx custom rule placed in nginx vhost or use tools/autoprotect.sh provided rule if it works, you can setup .autoprotect-bypass file in the relevant directory + re-run /usr/local/src/centminmod/tools/autoprotect.sh Beta Branch - autoprotect.sh - apache .htaccess check & migration to nginx deny all to exclude tools/autoprotect.sh from auto creating a rule as you manually added one in nginx vhost will do the equivalent task as intended by .htaccess file.

 

Disabling tools/autoprotect.sh

If you want to totally disable tools/autoprotect.sh, you need to comment out the cronjob and include file within nginx vhost config.

comment out cronjob with hash # in front

Code (Text):

#0 */4 * * * /usr/local/src/centminmod/tools/autoprotect.sh

and nginx vhost include file comment out with hash # in front

Code (Text):

#include /usr/local/nginx/conf/autoprotect/domain.com/autoprotect-domain.com.conf;

then restart nginx server

Code (Text):

ngxrestart