2020-12-29

Setup fail2ban to block bots and script kiddies accessing a Node.js app behind an nginx reverse proxy

Setup fail2ban to stop bot HTTP and HTTPS connections on nodejs apps running through nginx proxy_pass

Have a nodejs app running behind nginx. Kept seeing lots of log entries from bots trying to hit URIs like /index.php, which were generating 404 errors in the Node app.
I wanted these to be tracked in the nginx logs, so I could then use fail2ban to stop these bots.
I am running our node app on CentOS 7, but the only significant difference between these directions and those for other Linux/BSD/mac OSes would the install steps and a few paths. The general philosophy still applies.

Install fail2ban: sudo yum install fail2ban Installs fail2ban and a few other packages it requires
Ensure nginx is configured properly I find it very helpful to make sure the $server_name is added to each line of the access.log file. I add it at the end of the line to match the pre-defined ‘combined’ format:

log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                  '$status $body_bytes_sent "$http_referer" '
                  '"$http_user_agent" "$http_x_forwarded_for" $server_name';

Next, find the pertinent section for your node app. Here’s an example:

server {
    listen 443 ssl; # managed by Certbot
    listen [::]:443 ssl;
    # ...SSL info here managed by Certbot

    server_name  subdomainfornodeapp.example.com;
    
    location / {
        # THIS IS THE IMPORTANT LINE, it ensures the info is logged back to access.log
        proxy_intercept_errors  on;
        
        proxy_pass              http://localhost:8080;
    }
}

Create the custom fail2ban filter I block all 3xx, 4xx, and 5xx connections, because I don’t return those in my app, ever. You could modify the error codes inside the regex if you wanted to be more specific. Of note: numeric ranges in regex are complex, just be sure you test. Create a new file in the fail2ban filter.d directory: sudo nano /etc/fail2ban/filter.d/nginx-proxyerror.conf Paste this in:

# Fail2Ban filter to match web requests for selected URLs that don't exist
#

[INCLUDES]

# Load regexes for filtering
before = botsearch-common.conf

[Definition]

failregex = ^<HOST> \- \S+ \[\] \".*?\" [3-5]\d\d .+$

ignoreregex =

datepattern = {^LN-BEG}%%ExY(?P<_sep>[-/.])%%m(?P=_sep)%%d[T ]%%H:%%M:%%S(?:[.,]%%f)?(?:\s*%%z)?
              ^[^\[]*\[({DATE})
              {^LN-BEG}

# DEV Notes:
# Based on nginx-botsearch filter
# 
# Author: Neil A. Beardsley

Configure the new filter in your local jail: sudo nano /etc/fail2ban/jail.local Paste this section into your config file:

[nginx-proxyerror]

enabled  = true
port     = http,https
logpath  = %(nginx_access_log)s
maxretry = 2
bantime  = -1
action = %(action_mw)s

Test against your access.log file:

sudo fail2ban-regex --print-all-missed /var/log/nginx/access.log /etc/fail2ban/filter.d/nginx-proxyerror.conf >~/fail2bantest.output

Open the .output file and make sure you like the results. For me, I wanted all 3xx, 4xx, and 5xx responses to be matched (ban them!).

Restart the fail2ban service sudo systemctl restart fail2ban
Test to make sure you get banned From another computer, run a test to make sure you get banned. For my test, I just curled a known bad URL twice, and then I was banned: curl https://subdomainfornodeapp.example.com/manager/html2/ {"name":"NotFound","message":"Page not found","code":404,"className":"not-found","data":{"url":"/manager/html2/"},"errors":{}} curl https://subdomainfornodeapp.example.com/manager/html2/ {"name":"NotFound","message":"Page not found","code":404,"className":"not-found","data":{"url":"/manager/html2/"},"errors":{}} curl https://subdomainfornodeapp.example.com/manager/html2/ curl: (7) Failed to connect to mom.hatchfarmstudios.com port 443: Connection refused
Unban yourself after a successful test sudo fail2ban-client set nginx-proxyerror unbanip X.X.X.X
You can also manually ban IPs simlarly: sudo fail2ban-client set nginx-proxyerror banip X.X.X.X
To just outright block subnets, use iptables: sudo iptables -I INPUT -s ${subnet} -j REJECT Or from a file:

for subnet in `cat neil-list.txt`; do sudo iptables -I INPUT -s ${subnet} -j REJECT; done

To remove full rules from iptables: sudo iptables -D INPUT -s {cidr} -j REJECT --reject-with icmp-port-unreachable

Reference for determining CIDR subnet masks: http://www.subnet-calculator.com/cidr.php