Category Archives: Linux

Netios 0.71

I release a new version of Netios : 0.71.

There are a lot of changes, starting with cosmetics, but the biggest one is the support of multiprocessing.

It is now able to process several routers at the same time, so using it on a large list of machines results in a big speed up.

A downside is that it now requires at least Python 2.6, as multiprocessing started to be supported with this version only. Most Linux distributions now include Python 2.6, but still not all. Anyway it will be more and more the case. If you can’t uprade your distribution, you can stick with 0.60 which still do most of the work fine.

It is also now able to fetch a configuration file remotly, but it requires more testing before I feel confident in the way it works.

The complete changelog :

2009-12-20  (0.71) phocean ;

2009-12-20  (0.71) phocean ;

* ciscoclass.py : handle correctly the cisco pager — More — so that “show run” mode should work even with large config files
* sshclass.py : allow to override terminal size system settings (make use of the cisco pager to avoid filling the buffer)

2009-11-16  (0.70) phocean (private release)

2009-11-16  (0.70) phocean (private release)

* implement multiprocessing
* improve code documentation
* clean up UI
* reduce useless logging
* netios.py : bug : missing startTime parameter in f_skip_error and f_command functions

I cross my fingers so that there are not too many bugs, but if so, please don’t forget to report it to me.

Check there (tools page) for more details and a download link.

ModSecurity 2.5 review

I finished reading the ModSecurity 2.5 book, written by Magnus Mischell and published by Packt Publishing.

I found a lot of interest reading it as I was already using ModSecurity – and I think anyone exposing an Apache web server should.
I was actually using it partially. It is not trivial to secure a web application, and the rule engine of ModSecurity is very powerful but it is also quite complex.

So this book was a good opportunity for me to dig into it further.

The book covers all topics : from the set-up to a real use-case.
The author explains how to write rules, how to deal with the performance impact, logging and gives us a range of various core rules to implement to get a good security basis.

The difficulty goes up progressively and the author doesn’t forget the beginners.
The set-up of the module is precisely described. All requirements are also explained and there are some good recalls about regular expressions, common attacks on systems, server and client sides, and other stuff like that.

After reading the book, I could harden my rules, reorganize and optimize them for better performance – something I hadn’t cared about before.

So I have nothing else to say but to recommend this book.
It is definitely a great handbook about ModSecurity that’s worth having next to you. The variety of configuration patterns makes it a reference.

Check it there. I also appreciated the availability of PDF version, so that I can carry it everywhere with my laptop and index it with Beagle.

Dovecot LDA vs Procmail

I have a mail server configuration based on Postfix for the smtp, Dovecot for the imap and virtual users receiving e-mails in maildir boxes.

I am also using Amavis and Spamassassin for content filtering.

I am not going now to describe this configuration, I think there are already a lot of very good tutorials about it all over the web. Moreover, the openSUSE maintainers made such a configuration quite easy : a sensible part of the work just consists in commenting out some line in the configuration files.

However, with the basic setup, I had an issue with permissions :  all e-mails delivered by Postfix were created with permissions set to 600.

A typical use case with which I got into trouble was spam learning, done with a cron script with a dedicated account (“vscan”, you don’t want to execute such a script with root, right ?).

In that case, what I need is files to be created with permissions 660.

It seems easy and rather obvious at first, but actually there is not such a setting in Postfix.

Actually, it is not really the job of the MTA to do it, so in the case of Postfix it doesn’t bother with the transmission of such a parameter.
Then, I tried to use Procmail and set UMASK in /etc/procmailrc, but this just didn’t have any effect.

After searching and trying in vain a couple of hours, I found out that Dovecot can also deliver e-mails from the MTA to the maildir with Dovecot LDA.

So I tested it out. The configuration is pretty straightforward.

Add the line in bold to the virtual user configuration in /etc/postfix/main.cf :

[...]
virtual_mailbox_domains = domain.com
virtual_mailbox_maps = hash:/etc/postfix/vmailbox
virtual_mailbox_base = /home/vmail
virtual_minimum_uid = 100
virtual_gid_maps = static:1002
virtual_uid_maps = static:1001
virtual_transport = dovecot
[...]

Now, add these lines in /etc/postfix/master.cf :

[...]
# Dovecot LDA
dovecot   unix  -       n       n       -       -       pipe
   flags=DRhu user=vmail argv=/usr/lib/dovecot/deliver -d ${recipient}
[...]

Finally, configure /etc/dovecot/dovecot.conf with these sections :

[...]
protocol lda {
  # If you wish to use plugins you need to specify plugin directory
  # For example quota enforcing is implemented by plugin
  module_dir = /usr/lib/dovecot/modules/lda
  # Address from LDA should send MDNs like out of quota
  postmaster_address = postmaster@domain.com
  # UNIX socket path to master authentication server to find users.
  auth_socket_path = /var/run/dovecot/auth-master
}
[...]
auth default {
[...]
  socket listen {
	master {
		# Master socket provides access to userdb information. It's typically
		# used to give Dovecot's local delivery agent access to userdb so it
		# can find mailbox locations.
	      	path = /var/run/dovecot/auth-master
      		mode = 0660
		user = vmail
		group = vmail
	}
	client {
		# The client socket is generally safe to export to everyone. Typical use
		# is to export it to your SMTP server so it can do SMTP AUTH lookups
		# using it.
		path = /var/run/dovecot/auth-client
		mode = 0660
	}
  }
}
[...]

And that’s all !
Restart Postfix and Dovecot, check the log to ensure that everything works fine.
Now all new mails should come out in the maildir folder with permissions set to 660.

Definitely, in my opinion, Dovecot LDA is the way to go : simple and extensible. Good bye Procmail and your cluttered configuration file.

OpenSSL : CVE-2009-3555 security fix and mod_ssl client authentication breakage

A security advisory on OpenSSL has recently been published. Details are there and there.

It is vulnerable to a MiTM attack where the attacker can intercept and retrieve the credential to a trusted HTTPS website, by intercepting the session cookie sent back to the client.

A proof of concept of an attack against Twitter was made.

Fine. But so far, the answer was to just disable any renegociation.

This actually causes some issues with SSL session timeout and totally broke client authentication.

I got into problems because of the latter. I am using client authentication for some location of my web server, and I recently could not connect anymore to these with the following log in apache :

[Tue Nov 24 16:56:15 2009] [debug] ssl_engine_kernel.c(1912): OpenSSL:Exit: error in SSLv3 read client hello A
[Tue Nov 24 16:56:15 2009] [error] [client x.x.x.x] Re-negotiation handshake failed: Not accepted by client!?

I first was not aware of the openssl patch and tried almost anything possible. My focus was, of course, on the certificate and the client.
But, a nice guy on IRC #suse, Stittel, had a good hunch and suggested me to look at the CVE-2009-3555 fix.

After more tests, it was quickly confirmed to work well with older versions of OpenSSL (as shipped in Debian Lenny).
Finally, I downgraded the OpenSSL version on my openSUSE box to a version prior to the CVE-2009-3555 fix and it just worked fine.

Then, I dig into it and found a lot of interesting reports there and there. So far it is a real mess.
In short, the breakage will stay as long as browsers don’t also include a patch to avoid renegotiation.
So far, I could not find a browser that does include a patch.
If anyone reading it knows a version that does it, please let me know.

Meanwhile, you have actually the choice between :

  • low security by deactivating client authentication on your server
  • low security by keeping a vulnerable version of OpenSSL

As my server is not very exposed, I chose the latter, but that’s not satisfying.  It is not recommended, but if like me you need to use client authentication with mod_ssl on openSUSE 11.2, do :

% zypper install --from repo-oss openssl openssl-certs libopenssl0_9_8 libopenssl0_9_8-32bit

where repo-oss is the alias to the 11.2 release (without updates) on your system.

What a brutal way to fix an issues without much notification and consideration to the users ! Even the log message is wrong and just confusing the administrator…

PS 1 : thanks again to Stittel for the good hint (I hope you will come by here) and to the always nice and helpful #suse channel in general ;)

PS 2 : bug reported on openSUSE bugzilla

openSUSE 11.1 and /boot on RAID 1

I tried yesterday to set up a home server with two disks in a RAID 1 array.

My intention was to have everything on the RAID volume, including /boot, so that if a disk crash, I still can boot on the second one.
That’s the way I think it should work anyway, despite the number of tutorials or forum posts advising not to do so.

The openSUSE partitionner, during the installation process, warned me that having /boot on the RAID 1 array may not work with grub.
And indeed, at the end of the installation, I could not be able to set grub properly :

  • the installer seemed to be lost, writing wrong devices in the various grub config file like menu.lst and grub.conf
  • I tried to use the manual edition mode offered by the installer, but either because I missed something or there is a bug somewhere, I could never save my modifications.

Conclusion : there was a no go.

I became curious to see how a few other distributions could handle it.

At first, I thought they would all fail, because I have always found the openSUSE installer to be the best out there.

So I took a Debian Lenny CD, which is my second favorite distribution, and looked at it. The Debian installer is straightforward, but the partionner has always lacked of flexibility. It is fine when you are doing something pretty standard, but you get quickly limited when you want to do something more complicated.
There, no way to have /boot on a RAID volume, and the interface was really painful to use. Any mistake almost oblige you for sure to restart all from scratch.

Then, I looked at Fedora 11… without expecting much. The last time I tried it on a machine, it just froze every time it read the disk configuration.
But… surprise ! All I wanted was supported out of the box. The partionner was as pleasant to use as the one of openSUSE, and setting grub on /dev/md0 was just a matter of checking a box.

I have been quite impressed this time and Fedora will stay on this machine for a while.

I haven’t tested openSUSE 11.2 yet because the purpose of this machine is to be a server, so I care about stability.

But I will, and I wish it can support this feature also, or that it will be the case some day.

Automatic backup when inserting a drive

I bought a 500 GB 2.5″ external disk drive to backup the data of my laptop. It is small, quiet, easy to move and far enough for the important data I want to backup, mostly documents, e-mails or script from work.

Being lazy, it happened that I did not backup my data. Yes, it is a shame, but inserting a drive and launching the commands to rsync the discs was preventing me from this best practice.

So, I decided to make it automatic. The goal was that the only thing I would have to do would be to insert the drive, and then remove it when it is done.

Thanks to the magic of Gnu/Linux, it had been very easy. I will show below how I did it, thought they are many things that could be improved (but I haven’t felt the need so far).

Udev

Udev not only allows to create /dev entries dynamically, but offers a lot of triggers to perfom all kind of actions when some hardware is inserted.

The udevinfo command will show you a lot of output concerning your drive. What we want is a unique way to differenciate the backup drive from any other drive that will be inserted in the future.

What would be better than the manufacturer serial ?

So let’s look for it :

$ udevinfo -a -p /sys/block/sdc | grep serial

*UPDATE 06/2011* It seems that on recent versions, the syntax of this command slightly changed into this :

$ udevadm info -a -p /sys/block/sdc | grep serial

Copy the serial.

Now we have to create a rule file, that will tell to udev what to do when this particular drive is inserted.

This is done in the /etc/udev/rules.d folder. Let’s create a file 30-mnt.rules or anything you like.

We edit this file so that it contains :

ACTION=="add",KERNEL=="sd*",SUBSYSTEMS=="usb", ATTRS{serial}=="57442D57584E3430394C5A38", RUN+="/home/jc/bin/backup/bckp-home.sh %k"

ACTION==”add” will tell udev that this action must be triggered when the drive is inserted.
SUBSYSTEMS could be changed according to the drive you are using (scsi, usb, …).
ATTRS{serial} must contain the serial you just grabbed.
RUN+=”/path/to/bin/backup.sh %k” tells udev to launch the backup script. %k, which contains the device name, sdc, is passed as an argument.

Optionally, it is quite convenient, you may want to make a symlink to the /dev/sd? device, with :

KERNEL=="sd*",SUBSYSTEMS=="scsi", ATTRS{model}=="GJ0250EAGSQ     ", SYMLINK+="ultrabay%n"

The shell script

Now, the script itself :

#!/bin/sh
LOGFILE=/PATH/TO/bckp.log
echo "--- BCKP - INFO : \$1=_${1}_" >> $LOGFILE
[[ $1 ]] || { echo "ERROR : missing parameter">>$LOGFILE; exit 1; }

# give time for the user, if needed to kill the process
sleep 6
MOUNT_PATH=$(grep $(echo $1) /etc/mtab | awk '{print $2}')
[[ $MOUNT_PATH ]] || {
  echo "ERROR fretching mount point">>$LOGFILE;
  exit 1;
}
echo " Synchronizing $MOUNT_PATH)">>$LOG

# add here all you rsync commands
rsync -av --delete /PATH/TO/DATA $MOUNT_PATH/backup/
...
exit 0

Testing it

Now, let’s reload udev :

$ sudo udevadm control --reload-rules

To test if it works :

$ sudo udevadm trigger

or maybe :

$ /etc/init.d/boot.udev restart

Plug off/in your drive, and the script should be executed as expected.

Optional : setting more options with Hal

It is not necessary at all for the backup script to work, but it would be very practical to have  a fixed mount point for a drive.
For instance, I use a second drive (in the untrabay slot of my thinkpad) that contains all my virtual machines.

The benefice is to prevent a performance drain of the system when many virtual machines are doing I/O like swapping or anything else.

Create a file like /etc/hal/fdi/policy/15-static-mount.fdi, containing :

<?xml version="1.0" encoding="ISO-8859-1"?>
<deviceinfo version="0.2">
  <device>
    <match key="volume.uuid" string="aa0019ef-86e0-4011-b996-31ef3e7174c8">
      <merge key="volume.policy.should_mount" type="bool">true</merge>
      <merge key="volume.fstype" type="string">ext4</merge>
      <merge key="volume.policy.desired_mount_point" type="string">ultrabay</merge>
      <merge key="volume.label" type="string">Fuji</merge>
      <merge key="volume.policy.mount_option.noatime" type="bool">true</merge>
      <merge key="volume.policy.mount_option.acl" type="bool">true</merge>
    </match>
  </device>
</deviceinfo>

The drive is matched by it uuid. You can get the uuid of your disk with :

$ ls -la /dev/disk/by-uuid/

You can, if you want, set the volume label and specify several options of the file system.

However, the most interesting option is the “desired_mount_point” one which allow you to fix the mount point. In the example, the disk will always be mounted in

/media/ultrabay

, and not the system disk, or disk_1, etc.

Coming next !

That’s all for today folks. Let me know if there are some things not clear or that can be optimized.

Next time, we will see how to run the same script from Hal instead. We will also use Zenity to get a nice GUI prompt when the disk is inserted.