If you use Fail2ban then you are probably aware of the fact that you must add a rule number to the ipfw deny rule for actionban in ipfw.conf. If you don’t add a rule number then there is no way for fail2ban to delete the rule after it expires. The problem lies in that you can easily set a different rule number for each filter but if the filter adds many rules within it’s ban time then when that first actionunban gets triggered all rules with the same number are removed, even if there full ban time has not transpired.
I was looking for an elegant solution to this and finally figured out how to do it myself. What I’ve done is in the ipfw.conf file I’ve added a variable that will create a random number between 10000 and 12000 to use as the rule number.
The code is pretty simple.

echo $((RANDOM%2000+10000))

There needs to be an extra % in there for it to work. I think it has something to do with python. So far it seems to be working pretty good here. While it is possible that I could get a duplicate rule number, it’s unlikely.
I’ve modified my installation of Fail2ban significantly; but only by adding filters, jails, etc. Here’s a bundled version of all of my modifications. Here are instructions for using my modifications. So far everything seems to be working great. I’ve had to add a few items to ignoreregex so I don’t ban people using their iPhones on 3G or at home from certain dynamic IP cable providers.
What I’ve done is a host lookup on the IP that’s banned and if I find it’s a local ISP, like Verizon or Time Warner Cable, I add part of their host lookup to the ignoreregex list. So far it seems to be doing the trick.

I’ve just stumbled across a great command in Fail2ban to check whether or not your filter will actually score a hit from your log file.
From the command line.
[code lang=bash]
$ fail2ban-regex /path/to/logfile /etc/fail2ban/filter.d/myfilter.conf regex_to_ignore
[/code]
As an example.
[code lang=bash]
$ fail2ban-regex /var/log/secure.log /etc/fail2ban/filter.d/sshd.conf (myusername|myIPaddress)
[/code]
This seems like a great way to test whether changes to your filters are correct, rather than just waiting to see if anything shows up in fail2ban.log.

So here I am running my own server — for almost 2 years now. It’s been a real learning experience and I’ve tried to share. My latest add-on has been fail2ban. I got tired of looking into my logs and seeing where script kiddies or bots were trying to take control of my server. Fortunately, none have.
After a bit of googling, I found fail2ban. It’s a collection of python scripts.

Fail2ban scans log files like /var/log/pwdfail or /var/log/apache/error_log and bans IP that makes too many password failures. It updates firewall rules to reject the IP address.

There are a few tricks I’ve discovered along the way to make it work on my installation and likely on Mac OS X Server in general.
First is that fail2ban creates a PID and socket file in a directory that it fails to create. Yeah, that’s a bug. Since I didn’t want to mess around with the actual scripts in the program, I created a plist that issues the mkdir /var/run/fail2ban command. I placed this in /System/Library/LaunchDaemons and set it to Run at Load. Lingon is your friend, but’s now inactive. 🙁
After creating the file you have to use the command line to move it to the /System/Library/LaunchDaemons directory. I also created another launchd plist to reload fail2ban every day. I did this because I run multiple virtual websites and the error logs for those sites get rotated and the names have some time code or something tacked on the end of the filename.
OK, problem 1 solved. Next I discovered that since fail2ban is really running on a multitude of linux boxes all the different methods of IP tracking, sorting etc. were really useless on my OS X Server. I run ipfw firewall and fortunately there’s a module for that in fail2ban. Unfortunately it’s not quite set up correctly, at least it wasn’t for me. I had to tweak it a bit.
What this means is that your action is always going to be ipfw. I tweaked the ipfw.conf file a bit. Now it does the following.

1. Logs it’s action to ipfw.log
2. Adds a rulenum to the ipfw command. I did this because some other rule in my setup was allowing the IP before my deny could take effect. By lowering the rulenum my deny now fires off first.
3. Abstracted the protocol (tcp, udp) to pass as a variable. Just in case something you want to block isn’t tcp.

I also created another filter as I found many times some machine would excessively hit my Apache server looking for nonexistent files. Since it sounds like something a bot would do I decided to ban it. This was the simple creation of a new filter.
I created a jail.local file to hold all my prefs and through trial and error discovered that the examples of how to call for a jail weren’t working for me. Perhaps I just didn’t understand the examples. I soon discovered that parameters for the jail action needed to be passed inside of square brackets in the prefs.
I’m sure, if you’ve gotten this far that you’re either very confused by this whole post or that you’ve had an epiphany. To further the epiphany along I’ve uploaded my file changes.
You should be able to figure out what file goes where from the folder structure of the upload.
A couple of things in summary to remember. First, turn on your server’s firewall. Then make sure you change your server’s local IP address in the files to match your own. That’s the setting for localhost.
Good luck. If you have any questions leave a comment.