Intrusion detection?
Over on Mezzoblue, Dave Shea found out that his website had been compromised subtly. The attacker had exploited some (as yet unknown) security hole and quietly modified his website to link to the standard spam sites.
Within a few hours there were tens of posts from people who’d checked their websites and found similar modifications that had been sitting their un-noticed, with people pointing the finger either at old WordPress installations or a guesses that their hosting service had been compromised.
This shows us once again, that any software you run on your website needs to be kept up-to-date immediately, but what shocked me was that so many people out there running websites and are not watching them for file changes. They had no idea that their sites had been hacked until they went and looked for it.
So – in bold: Anyone running a website or webserver of any type needs to watch out for unexpected access and changes.
The easiest way to do this is to use some intrusion detection software (IDS). This sounds complex, but it’s actually quite easy to do. All these programs do is to monitor your files and warn you when they change. This would have immediately spotted this type of attack.
Because of this, I’ve decided to write up an easy guide to simple file alteration monitoring – here it is.
Choose your weapon:
There are plenty of intrusion detection/file modification apps out there – some of the better known ones include AIDE, Samhain and Tripwire. These are all very cool, and highly powerful, but are also quite complex and hard to install, especially on cheap shared hosting.
Therefore, my weapon of choice, for the last few months has been a lightweight and fast application called integrit, so I’m going to tell you how to install it here.
Before we start: Do make sure that you’re not compromised right now, there’s no point running an IDS if you’re already hacked. While you’re at it, make sure everything’s upgraded too.
Step 1: Make a place to store integrit:
Since you’re on shared hosting, you can’t install integrit properly into /usr, but you need to to put it somewhere anyway.
I decided to install it to a directory called “integrit” inside my home dir, so:
mkdir ~/integrit
In the following commands, do remember to replace any mention of ~/integrit with the directory you used.
Step 2: Download and install integrit:
The integrit webpage is at: http://integrit.sourceforge.net/, so go there and get the latest version (currently 4.1), or you could cut’n’ paste this:
cd ~/integrit
wget http://optusnet.dl.sourceforge.net/sourceforge/integrit/integrit-4.1.tar.gz
Once the integrit archive file is in your ~/integrit directory, we need to decompress it and install it:
cd ~/integrit
tar -zxvf integrit-4.1.tar.gz
cd integrit-4.1
./configure
make
Next we need to put the integrit binary somewhere where we can get it, here I’ve just dumped it into the ~/integrit directory, but you could put it in ~/bin or something nicer if you want:
cp integrit ~/integrit
Step 3: Set up integrit:
Look in the integrit-4.1/examples directory and make a config file from the example.
You need three things at least:
- The known file database – this is where the integrit database is stored.
- The current file database – this is where integrit stores the modified info.
- A root directory to monitor – this is the full path to the directory we want to watch.
We can also tell integrit to ignore directories, by listing with an exclamation mark at the start of the line. We want to ignore the ~/integrit directory, and on dreamhost, we’ll need to ignore the webserver log directory (because it changes a lot, and parts of it our user can’t access which will cause errors).
All in all, it’ll look something like this:
# database locations (FULL PATHS!)
known=/home/simon/integrit/src_known.cdb
current=/home/simon/integrit/src_current.cdb
# What do we want to check (no trailing slash!)
root=/home/simon
# ignore the integrit dir:
!/home/simon/integrit
# ignore the webserver logs dir:
!/home/simon/logs
# oh, and the bash logfile
/home/simon/.bash_history
Note: You’ll need to change “simon” to your user name, and “integrit” to where you installed integrit to in step 2.
Step 4: Create integrit database
We need to get integrit to store a list of the current files, and their vital statistics, so run this command:
~/integrit/integrit -C home.conf -u
– where “home.conf” is the integrit configuration file that you generated in step 3.
If all goes well, you’ll see something like this:
integrit: -- integrit, version 4.1 -------
integrit: output : human-readable
integrit: conf file : home.conf
integrit: known db : /home/simon/integrit/src_known.cdb
integrit: current db : /home/simon/integrit/src_current.cdb
integrit: root : /home/simon
integrit: do check : no
integrit: do update : yes
integrit: current-state db RMD160 ------
integrit: 3d6b135343a5031d357b5bb2d7d7dc39c7ab5646 /home/simon8/integrit/src_current.cdb
Once that’s done, copy the newly created database to the known database location:
cp ~/integrit/src_current.cdb ~/integrit/src_known.cdb
Step 5: Test that integrit’s working:
So lets make sure that integrit’s working properly. To do this, we can add an empty file somewhere and see if integrit spots it.
cd ~
touch foo
Now we can run integrit:
~/integrit/integrit -C ~/integrit/home.conf -c
If all goes well, you’ll see something like this:
integrit: -- integrit, version 4.1 -------
integrit: output : human-readable
integrit: conf file : /home/simon8/integrit/home.conf
integrit: known db : /home/simon8/integrit/src_known.cdb
integrit: current db : /home/simon8/integrit/src_current.cdb
integrit: root : /home/simon8
integrit: do check : yes
integrit: do update : no
new: /home/simon8/foo p(664) t(100000) u(767504) g(203016) z(0) m(20070605-162156)
new: /home/simon8/foo s(9c1185a5c5e9fc54612808977ee8f548b2258d31)
integrit: not doing update, so no check for missing files
Notice how integrit’s spotted the foo file that’s not in the database? If one of the files has changed, you’ll get much the same output, with “changed:” instead of “new:”. So – remove the dummy file:
rm ~/foo
Step 7: Get integrit to run daily:
Now, we want to set up a cron job, so that integrit is run automatically for us. First of all, we should make a quick little shell script to run integrit and email the results to us:
#!/bin/bash
DATE=`/bin/date "+%F"`
/home/simon/integrit/integrit -C /home/simon/integrit/home.conf -cu | /usr/bin/mutt -s "integrit -- $DATE" email@example.com
Paste the above into a file called “run_integrit.sh” (a good place to put it would be in your ~/integrit directory), edit the paths to match your setup, and change the email address. Finally, make this file executable:
chmod +x ~/integrit/run_integrit.sh
Now all we have to do is to add that to our crontab.
crontab -e
– and add a line that looks something like this:
59 21 * * * /home/simon/integrit/run_integrit.sh
This will run integrit at 21.59 every day, if you don’t know what that means, then have a google for “crontab tutorial”.
Save the file, and you’re off.
Updating the database after valid changes:
When you’ve changed or added a file yourself, then you’ll need to update your known database with these changes. To do this, just generate a current database, and copy it over the old one. The script I’ve got above will automatically generate a current one, so you can just use that version, or repeat Step 4.
Final considerations:
Note: If you can, you should run integrit (that is BOTH the database files and the binary files) off a “safe” partition, that’s not writable. Unfortunately, most of us on shared hosting don’t have that privilege, so just be aware that if a really clever attacker does get you, then they’re likely to disable or modify the IDS if they can.
A good way of dealing with this is to copy your known file database off the webserver and make sure that the one on the server matches this one every so often.
–Simon