Archive for September, 2005

TNT delivered my Thinkpad T30 back today. A quick check showed that it was a new motherboard and it has the latest BIOS and embedded controller software, which means that it’s one of the newer boards without the memory slot problems. My excitement didn’t last long because soon after I booted into Linux the screen went magenta and fuzzy. My first guess was they must have changed the video chipset and I would just need to replace the driver, but when I shutdown the machine the problem remained after X quit, and even more worryingly it was still there when I restarted went into the BIOS.

When I booted the machine up later the problem was gone, but after a while it came back, but then went away again. An intermittent problem with no apparent cause. A search on Google didn’t turn up much, the closest was ‘Problem with garbled screen‘ on Thinkwiki. But the screenshots don’t look like the same problem, maybe there are different levels of severity.

I took some photos of the problem. First one is of the BIOS screen, it’s hard to see the problem because it only really shows up on white areas of the screen and the photo looks a lot better than what is actually displayed. I’ve circled a couple of areas where you can see the problem:

Once you’re into Linux the problem is very obvious as you can see here:

I guess it’s back on the phone to IBM technical support tomorrow morning, sigh.

It looks like there’s another workaround for the Berkeley DB version clash on Dreamhost’s servers, use MySQL as the database instead. I didn’t realise this was an option and came across it on DH’s forums. Someone’s written up a guide to installing SA on DH:

http://www.unsaturated.com/projects.spamassassinMySQL.html

But I didn’t want to install from scratch and thankfully you can import your existing bayes data. From the README.bayes file that is distributed with SA:

1. Run ’sa-learn –backup > backup.txt’ which will backup your bayes data into a text file.
2. Optionally you can run ’sa-learn –clear’ to remove the DBM based bayes files.
3. Modify your local.cf file according to the directions above.
4. Run ’sa-learn –restore backup.txt’ to restore your bayes data to the SQL database.

This went without a hitch and my spam filtering is decent again!

The root of the problem is the version of Berkeley DB that’s been used to store the bayes files. On Sarge it’s 4.3. The problem is that the DreamHost mail servers haven’t been upgraded yet so they’re still using Perl 5.6.1 and DB 3.x.

This would explain why it was previously able to filter my mail but failed when training, because the training was taking place on a Sarge machine. Once I upgraded the files to the 4.3 format (db4.3_upgrade) the filtering failed. All my newly created bayes files are in 4.3 format so they won’t work at filtering. I had a quick stab at trying to downgrade them without any success. Since DH is supposed to upgrade the mail servers any time now it’s hardly work trying to get it to work in the mean time.

So this message:

warn: bayes: cannot open bayes databases /home/.../.spamassassin/bayes_* R/O: tie failed:

means your bayes file have the wrong DB version. Check this page for more details:

http://wiki.apache.org/spamassassin/DbDumpAndLoad

Another annoying thing that stumped me for ages is that procmailrc doesn’t appear to let you set the value of PERL5LIB. I thought that would have fixed my problem with the DB issue but also with a DNS version problem. I also see this in my logs:

warn: dns: Net::DNS version is 0.19, but need 0.34 at /home/.../usr/share/perl/5.8.4/Mail/SpamAssassin/Dns.pm line 589.

This is because DH is using some ancient version of Net::DNS. It’s not a fatal error but a lot of useful tests need it. So without it and bayesian testing I guess I’ll be seeing a lot more spam in the couple of weeks until it all gets resolved.

I’ve had this post sitting in my drafts folder for ages, I don’t think I’ll ever get around to finishing it because I’ve forgotten the steps I took and I found someone else has already written up a guide.

I followed the instructions here:

http://wiki.apache.org/spamassassin/SingleUserUnixInstall

and the gist is, you have to install local copies of all the Perl libraries you need, i.e.:

perl Makefile.PL PREFIX=$HOME/usr && make && make install

The packages required and the order to install is:

1. IP-Country (2.20)
2. razor-agents-sdk (2.07) Also provides (Digest, Digest::SHA1, File::Spec, MIME::Base64, Test::Harness, Test::More, Test::Simple, Time:HiRes, URI)
3. razor-agents (2.77)
4. Net-Ident (1.20)
5. Socket6 (0.19)
6. IO-Socket-INET6 (2.51)
7. Net-IP (1.23)
8. Net-DNS (0.53)
9. Sys-Hostname-Long (1.4)
10. Mail-SPF-Query (1.997)
11. pyzor (0.4) (Didn’t install correctly)
12. dcc-dccproc (1.3.16)
13. Mail-SpamAssassin (3.1.0)

Check out SpamAssassin 3.1.0 for Dreamhost or the DreamHost Wiki for more instructions.

I did a clean install of SpamAssassin 3.1.0 with the current version of Perl (5.8.4) and retrained the bayesian bit and it still doesn’t work. Upgrading the DB format didn’t make a difference either. I started a post on how to install SA, but I figured it’s pointless to finish it until I know I can get it all working again, so I’m not going to post it now.

Since I can train it and the bayesian checking works from the command line I know it’s all working properly and it going to be some stupid little detail. My first guess is permissions because the bayes files are 600 so procmail might be running as another user or something.

When I tried to turn on my Thinkpad from a hibernation this morning it reported that my hardware had changed. I quick check in the BIOS confirmed my worst fears, the second memory slot was no longer being detected. This means when IBM replaced the board (I confirmed it was changed when the internal network card’s MAC address changed) they replaced it with one that had the same problem but didn’t bother to do any of the fixes to prevent the problem occurring again. My opinion of IBM has gone down somewhat.

I’ll give them a call tomorrow morning because my warranty is running out fast. I’ll ask them about the new motherboard version that the American fixes get that permanently fixes the problem. There’s little point in getting another old board because it’ll just break again. I guess the alternative solution is to get a 1GB RAM stick and be content with that.

Dreamhost has recently upgraded to Debian Sarge. Along with that upgrade was a new version of Perl (5.8.4) this appears to have broken the bayesian filtering part of my SpamAssassin setup. This Usenet post describes some of the problems I’m seeing. But even after upgrading the database format I’m still having trouble. I’m getting this message in my procmail logs:

Cannot open bayes databases /.../.spamassassin/bayes_* R/O: tie failed:

Which was the error message I got when trying to use sa-learn before upgrading. So far I’ve tried retraining from scratch and even installing a local copy of Perl 5.6.2, but none have solved the problem. What’s really annoying is that if I save a spam message and run it through SpamAssassin on the command line it works fine.

Unless tomorrow’s Google searches turn up anything useful I’ll just try rebuilding all my local Perl packages with the new version and hope that makes a difference.