Fail2ban Problems and Solutions

If you use Fail2ban then you are probably aware of the fact that you must add a rule number to the ipfw deny rule for actionban in ipfw.conf. If you don’t add a rule number then there is no way for fail2ban to delete the rule after it expires. The problem lies in that you can easily set a different rule number for each filter but if the filter adds many rules within it’s ban time then when that first actionunban gets triggered all rules with the same number are removed, even if there full ban time has not transpired.
I was looking for an elegant solution to this and finally figured out how to do it myself. What I’ve done is in the ipfw.conf file I’ve added a variable that will create a random number between 10000 and 12000 to use as the rule number.
The code is pretty simple.

echo $((RANDOM%2000+10000))

There needs to be an extra % in there for it to work. I think it has something to do with python. So far it seems to be working pretty good here. While it is possible that I could get a duplicate rule number, it’s unlikely.
I’ve modified my installation of Fail2ban significantly; but only by adding filters, jails, etc. Here’s a bundled version of all of my modifications. Here are instructions for using my modifications. So far everything seems to be working great. I’ve had to add a few items to ignoreregex so I don’t ban people using their iPhones on 3G or at home from certain dynamic IP cable providers.
What I’ve done is a host lookup on the IP that’s banned and if I find it’s a local ISP, like Verizon or Time Warner Cable, I add part of their host lookup to the ignoreregex list. So far it seems to be doing the trick.

Checking Fail2ban regex

I’ve just stumbled across a great command in Fail2ban to check whether or not your filter will actually score a hit from your log file.
From the command line.
[code lang=bash]
$ fail2ban-regex /path/to/logfile /etc/fail2ban/filter.d/myfilter.conf regex_to_ignore
As an example.
[code lang=bash]
$ fail2ban-regex /var/log/secure.log /etc/fail2ban/filter.d/sshd.conf (myusername|myIPaddress)
This seems like a great way to test whether changes to your filters are correct, rather than just waiting to see if anything shows up in fail2ban.log.

Server-Side Email Filtering With Sieve

Another post for the peripheral brain.
When I first set up my own server lo these years ago, I never really thought about email message filtering. After all, I had rules in that would send my incoming message to wherever I wanted them. Besides, I was much more concerned with eliminating spam.
Well, that was then and spam seems under control. I was prompted to look at server-side message filtering mostly to help out my mother, who seems determined to have every single store, travel and other consumer site that will happily take your email address and send you messages daily — or more often, have a more controllable experience on her iPhone. When we originally set up her iPhone she told me she didn’t want to use it for email. Silly me, I listened and set her up with a POP account. Well now she wants email. What’s a good son to do. 😉
I changed her POP account to IMAP, copied over all her messages to her new IMAP folders and thought I’d need to solve her impending problem of 100 or so messages every other day choking her inbox.
After a bit of Googling I found Sieve. I’d actually heard of it before but never really thought about it. The Apple Discussion Forum had a nice start and pointed me on to sources I used to set it up.
Here are the salient points. From the terminal…

  1. Add the following lines to /etc/services
    sudo pico /etc/services

    Insert the following lines.

    callbook 2000/udp # callbook
    callbook 2000/tcp # callbook
    + sieve 2000/udp # sieve mail filtering
    + sieve 2000/tcp # sieve mail filtering

    You can check to see if it’s running by running

    netstat -an | grep 2000

    with results

    tcp4 0 0 *.2000 *.* LISTEN
    tcp6 0 0 *.2000 *.* LISTEN
  2. Create /usr/sieve
    sudo mkdir /usr/sieve
    sudo chown _cyrus:mail /usr/sieve
  3. Restart mail services
    sudo serveradmin stop mail
    [ some stuff ]
    sudo serveradmin start mail
    [ some stuff ]
  4. Since I’m using OS X Server and SquirrelMail is already running, next was installing and configuring avelsieve.

I really did try installing the latest development version — 1.9.9 alpha. That should have been a clue. After spending way too much time with it I installed the stable version – avelsieve 1.0.1. Once copied into /usr/share/squirrelmail/plugins run sudo perl /etc/squirrelmail/config/ and activate the plugin.
Then it’s back to the terminal. These instructions are from AFP548.

    cd /usr/share/squirrelmail/plugins/avelsieve
    sudo cp config-sample.php config.php

Now set the correct authentication matching SquirrelMail.
Edit /etc/squirrelmail/plugins/avelsieve/config.php and change:

$preferred_mech = "PLAIN";


$preferred_mech = "CRAM-MD5";

You should be running SquirrelMail with CRAM-MD5 authentication anyway.
Finally, edit the /etc/squirrelmail/plugins/avelsieve/lib/sieve-php.lib.php file.
Find the line:

fputs($this->fp, "PUTSCRIPT "$scriptname" {$len+}rn");

and change it to :

fputs($this->fp, "PUTSCRIPT "$scriptname"".' {'."$len+".'}'."rn");

This fixes an error in the script allowing you to save your changes to the filters. Now go login to webmail and click on the Filter link to start creating your Sieve filters.

Poof it's Gone

OK, I’m only really posting this cause I need it and I just want to put it somewhere to remember it. 😉
To make a file or folder invisible issue the following from the CLI.
SetFile -a V path/to/fileOrFolder
To make it visible again…
SetFile -a v path/to/fileOrFolder
That’s it. This can be especially useful on shared drives to keep others out of specific folders, etc. BTW, the file or folder still shows up in `ls`, it’s just not visible in Finder.

Don’t Send iCal Replies

I came across a great hint in Mac OS X Hints today. It seems that there’s an easy way to interrupt the process of sending an email reply when accepting iCal invites. As I tend to play around with iCal invites a lot (adding and deleting the same event ad nauseum) — I love this.
I wrote up a modified script like in the example and bundled it with a shell script, to install and uninstall the modification. You have to run this shell script using sudo from the CLI (Command Line Interface aka
The zip file contains the shell script, the modified Mail.scpt AppleScript, and the original Mail.scpt AppleScript.
To install run sudo /path/to/ install
To uninstall run sudo /path/to/ revert
To check usage and status, run /path/to/
If you don’t like messing with the CLI then there’s a great little shareware app, iCal Reply Checker that does it all, and more.
It seems that neither method interferes with the code signing of iCal as the script in question is not code signed.
It appears that if you’re using an Exchange account in that this script is being bypassed and this hint won’t work for you. 🙁

Fail2ban on Leopard Server

So here I am running my own server — for almost 2 years now. It’s been a real learning experience and I’ve tried to share. My latest add-on has been fail2ban. I got tired of looking into my logs and seeing where script kiddies or bots were trying to take control of my server. Fortunately, none have.
After a bit of googling, I found fail2ban. It’s a collection of python scripts.

Fail2ban scans log files like /var/log/pwdfail or /var/log/apache/error_log and bans IP that makes too many password failures. It updates firewall rules to reject the IP address.

There are a few tricks I’ve discovered along the way to make it work on my installation and likely on Mac OS X Server in general.
First is that fail2ban creates a PID and socket file in a directory that it fails to create. Yeah, that’s a bug. Since I didn’t want to mess around with the actual scripts in the program, I created a plist that issues the mkdir /var/run/fail2ban command. I placed this in /System/Library/LaunchDaemons and set it to Run at Load. Lingon is your friend, but’s now inactive. 🙁
After creating the file you have to use the command line to move it to the /System/Library/LaunchDaemons directory. I also created another launchd plist to reload fail2ban every day. I did this because I run multiple virtual websites and the error logs for those sites get rotated and the names have some time code or something tacked on the end of the filename.
OK, problem 1 solved. Next I discovered that since fail2ban is really running on a multitude of linux boxes all the different methods of IP tracking, sorting etc. were really useless on my OS X Server. I run ipfw firewall and fortunately there’s a module for that in fail2ban. Unfortunately it’s not quite set up correctly, at least it wasn’t for me. I had to tweak it a bit.
What this means is that your action is always going to be ipfw. I tweaked the ipfw.conf file a bit. Now it does the following.

  1. Logs it’s action to ipfw.log
  2. Adds a rulenum to the ipfw command. I did this because some other rule in my setup was allowing the IP before my deny could take effect. By lowering the rulenum my deny now fires off first.
  3. Abstracted the protocol (tcp, udp) to pass as a variable. Just in case something you want to block isn’t tcp.

I also created another filter as I found many times some machine would excessively hit my Apache server looking for nonexistent files. Since it sounds like something a bot would do I decided to ban it. This was the simple creation of a new filter.
I created a jail.local file to hold all my prefs and through trial and error discovered that the examples of how to call for a jail weren’t working for me. Perhaps I just didn’t understand the examples. I soon discovered that parameters for the jail action needed to be passed inside of square brackets in the prefs.
I’m sure, if you’ve gotten this far that you’re either very confused by this whole post or that you’ve had an epiphany. To further the epiphany along I’ve uploaded my file changes.
You should be able to figure out what file goes where from the folder structure of the upload.
A couple of things in summary to remember. First, turn on your server’s firewall. Then make sure you change your server’s local IP address in the files to match your own. That’s the setting for localhost.
Good luck. If you have any questions leave a comment.