Spammers... damned spammers

Here's a screenshot of a little python application that I wrote to manage some spammers on the system. I've 'sed'ed their domains to example.com, but it basically shows how many messages are currently queued on one of our outbound smtp servers.

Every shift, I run an application that monitors the queues, and when an individual account exceeds the maximum number of allowed egress messages, my application logs it, and removes it from the queue.

But, spammers are smart little buggers, so they of course create multiple accounts from which they spam a little bit from.

My software is smart enough that it keeps track of all found spammers from the past 30 days, and simplifies their email address, then compares that to all of the email addresses in the current out queue. If the simplified "known spammer" address matches any of the current addresses, then those current addresses are treated as a single address and their queues are aggregated. If the aggregate exceeds the threshold, then all messages are removed and the synonymous accounts are flagged as spammers for later termination.

Note: The addresses in the above screen cap are the ones that come out of the filter against the known/previous offenders. Only usernames that have been previously declared spammers are aggregated.

Oh, and here's a little graph showing the outbound message queue on a single server. The drop was when I started work that day.

New phone

Just replaced the broken phones with some non-broken ones. Attached are some pictures taken on them.

Python: Functional programming

I'm trying to get used to functional programming and I must admit, it's kinda fun!

Here's a trivial example using both generators, and built in functions to acheive the same result:

Leslie Lamport

I just stumbled upon Leslie Lamport's website, which has electronic copies of most of his published work.

I've studied some of Lamport's work in the past (mainly in regards to Lamport Logical Clocks & totally ordered multicasting, etc) and was recently doing some digging to find out how & why he created the LaTeX extensions to TeX.

That was when I came upon his papers.. wow.

Graduate school

Katharine has submitted her final MRP to the school of graduate studies, and will do her oral defence in just under a week. Should all go well, then she'll have her MA.

This has gotten me thinking about my own future possibilites in graduate school. I have the marks, and just have to maintain my average with the requisite math courses (Calculus, Linear Algebra, Statistic) and my current courses.

I spoke with a professor with whom I'm aquainted, and I must admit that he's piqued my interest in pursuing a M.Sc at Trent.

This means that I'll have to pay higher attention to my studies, since I've (until now) been treating my B.Sc. as a terminating degree.

First day of class

New semester started up yesterday and I captured the telemetry of my drive. Below is a screencap from an internal application used to inspect data drom our onboard electronic recording device (nice name, eh?).

kd-tree creation tutorial

I've started work on the walkthrough mentioned in my previous post. It doesn't touch on topics such as searches of the completed tree, but sticks to the theory behind the creation and balancing of the resulting tree.

The graphic(s) for the walkthrough were created using graphviz. I wrote a small utility module in python that is used by the tutorial to generate the images.

New school semester

Starting a new semester at school next week. Will be at school three days and at work three days, with Sunday off.

Will post more once classes have begun.

---

Here's a screenshot of some trip data aquired from a test device installed in my car.

---

I hope to post a demonstration of kd-trees using Python in the not too distant future. It should be fun.

Cisco Switch -> NIC problems

The other day I noticed terrible transfer rates between two UNIX servers at work. Both were autonegotiating to 100Mbit/full-duplex, but the switch port that the slow server was on was showing a lot of 'orange blinks'.

I figured that the switch wasn't setup for 100Mbit/full, so I sent an email to the manager of that switch and he replied that all ports were forced to 100-fd.

After googling, I came accross this table which outlines all of the possible mis-matches in switch/nic configuration that could cause duplex-mismatch and create a ton of collissions.

I forced the server's nic to 100fd and was able to transfer at 80Mb/s again.

Then I started thinking...At our remote location we were experiencing terrible transfer rates of ~200Kb/s on the Internet. I double checked, and found that the firewall's (a freebsd 6.1 machine) internal nic which is serving as an 802.1q vlan trunk had autonegotiated to 100Mb-half-duplex!

I logged into the switch at the co-location and forced all ports to 100-full then did the same with all nics on the server.

After the switch & firewall renegotiated, I was able to get 40Mb/s transfer rates through the firewall to the internal servers.

Unbelievable.

How not to make coffee

In the past few weeks we've had a number of humourous (and dangerous!) coffee experiences such as mixing boiling water in the coffee grinder, turning the coffee maker off after the first cup, etc etc.

Here's a picture of today's coffee.

Online storage with Box.Net

Signed up for a free account with box.net today. You get 1GB of storage space with the limitation of 10MB files.

This size limitation on the "free" account is a little annoying, but it's easy enough to split my files first.

We'll see if I'm still using the service in a month.

Some benchmarks

Did a few simple benchmarks on a couple of FreeBSD machines at work that are used to VPN our Hamilton & Toronto locations.

tpgvpn1:


OpenSSL 0.9.7e-p1 25 Oct 2004
built on: Wed Apr 12 10:24:56 EDT 2006
options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) aes(partial) blowfish(idx)
compiler: cc
available timing options: USE_TOD HZ=128 [sysconf value]
timing function used: getrusage
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
md2 1638.20k 3516.44k 4954.45k 5548.25k 5704.30k
mdc2 5364.32k 6914.55k 7521.42k 7624.97k 7705.83k
md4 12433.45k 47035.26k 136420.21k 306165.92k 481776.95k
md5 11107.62k 38561.00k 109706.75k 210811.74k 287746.12k
hmac(md5) 14013.40k 49112.06k 131354.82k 229386.40k 291448.39k
sha1 9059.35k 25369.27k 57307.40k 88381.24k 105989.38k
rmd160 7511.49k 22129.82k 46165.80k 67428.84k 78017.10k
rc4 93653.74k 109422.32k 113919.20k 115308.83k 115680.63k
des cbc 58475.17k 61181.88k 62006.46k 62025.47k 61894.47k
des ede3 21643.57k 21967.56k 22083.05k 22130.20k 22114.93k
idea cbc 0.00 0.00 0.00 0.00 0.00
rc2 cbc 20740.90k 22695.78k 22893.82k 23239.87k 22849.82k
rc5-32/12 cbc 147578.81k 158957.43k 168257.95k 171201.41k 171239.86k
blowfish cbc 94637.96k 101551.60k 102807.14k 103450.35k 103305.91k
cast cbc 83145.40k 88404.94k 89604.76k 90079.13k 89946.25k
aes-128 cbc 62447.41k 62234.48k 64136.66k 64602.90k 63681.85k
aes-192 cbc 55820.88k 55668.59k 56812.85k 56763.62k 56988.42k
aes-256 cbc 50017.98k 50337.99k 51038.58k 50574.14k 51127.44k
sign verify sign/s verify/s
rsa 512 bits 0.0009s 0.0001s 1115.8 11984.3
rsa 1024 bits 0.0044s 0.0002s 228.6 4268.8
rsa 2048 bits 0.0265s 0.0008s 37.8 1308.3
rsa 4096 bits 0.1756s 0.0027s 5.7 368.7
sign verify sign/s verify/s
dsa 512 bits 0.0007s 0.0009s 1370.6 1118.1
dsa 1024 bits 0.0022s 0.0027s 455.8 370.8
dsa 2048 bits 0.0073s 0.0087s 137.3 114.3


core1:


OpenSSL 0.9.7e-p1 25 Oct 2004
built on: Sun May 7 03:08:51 UTC 2006
options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) aes(partial) blowfish(idx)
compiler: cc
available timing options: USE_TOD HZ=128 [sysconf value]
timing function used: getrusage
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
md2 689.54k 1497.71k 2116.48k 2362.59k 2444.21k
mdc2 2582.80k 3138.69k 3315.10k 3368.65k 3376.00k
md4 5538.51k 20431.26k 66112.68k 150588.74k 239281.97k
md5 4837.07k 17335.76k 52498.88k 106514.46k 152909.54k
hmac(md5) 8870.64k 29192.61k 75483.85k 126350.29k 157551.74k
sha1 4349.97k 13238.35k 34171.43k 56445.42k 69773.18k
rmd160 4158.92k 13428.72k 33624.91k 53853.32k 65470.42k
rc4 131339.75k 149218.64k 159794.61k 162431.71k 163566.16k
des cbc 29754.63k 30797.61k 31205.75k 31267.00k 31324.31k
des ede3 10977.26k 11112.41k 11175.22k 11194.61k 11194.92k
idea cbc 0.00 0.00 0.00 0.00 0.00
rc2 cbc 11394.78k 11796.30k 11897.23k 11932.45k 11929.00k
rc5-32/12 cbc 82203.59k 94505.04k 100379.68k 101657.50k 102098.28k
blowfish cbc 45409.02k 48400.70k 49780.22k 50007.64k 50007.22k
cast cbc 44914.14k 47809.21k 49023.44k 49306.77k 49333.89k
aes-128 cbc 30723.93k 31820.69k 32113.55k 32244.90k 32282.16k
aes-192 cbc 26401.29k 27386.46k 27623.19k 27743.82k 27753.52k
aes-256 cbc 23762.41k 24460.65k 24633.90k 24730.52k 24737.95k
sign verify sign/s verify/s
rsa 512 bits 0.0012s 0.0001s 856.6 8413.9
rsa 1024 bits 0.0053s 0.0003s 188.3 3317.8
rsa 2048 bits 0.0305s 0.0009s 32.8 1069.9
rsa 4096 bits 0.1997s 0.0031s 5.0 320.0
sign verify sign/s verify/s
dsa 512 bits 0.0009s 0.0011s 1095.4 921.7
dsa 1024 bits 0.0026s 0.0031s 390.3 321.2
dsa 2048 bits 0.0084s 0.0103s 119.4 96.9

And now for the bandwidth:

[peterb@tpgvpn1] netperf> /usr/local/netperf/netperf -H 204.15.193.30 12865
TCP STREAM TEST to 204.15.193.30
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec

65536 32768 32768 10.04 5.75

Been offline for some time

Just writing to say that I haven't abandoned this blog. Just with work full time, and family full time, I haven't found the time (or motivation?) to sit down and write anything of substance or worth in a while. Here's a quick list of things that have happened in the past few months:

• Winnifred started pre-school
• Started work full-time
• ...
• That's about it

Well, the list is rather small and not very impressive, but its contents have been keeping me very busy!

A finish to exams

Just finished my last exam. Now I just have to finish writing a report on AFS and my semester is complete.

I've backed up all of my unix/netware files to my new flash disk (1gig) that I bought last night along with a new laser printer (brother hl-2040) for Katharine. We've been without a printer for ~6months, and it's been a real pain in the neck. After being able to print out all of my course notes to study with last night, I was regretting having put off the printer purchase for so long.

If anyone is in the market, I highly recomend getting a monochrome laser. The quality is beautiful, and how often are you really printing colour?

Davis' Computer is Vegetarian

I recently did a 'w' command on a shell server to see who was online, and my friend's remote hostname came up as: "CPEbadbadbeeeef...".

Now, that hostname belongs to Rogers internet, and they (appear to) use a combination of strings and hex values to denote their customers' address.

I just find this (arbitrary) address funny.

Note: http://en.wikipedia.org/wiki/0xdeadbeef is a funny little bit of trivia

Messy Girls

Since Fin is tube-fed, she hasn't really developed much of an appetite, but she still likes to get messy with the rest of us during meals.

Attached are a few pictures of her eating palenta & pesto.

Oh, and Fin's learning to crawl:

Personal Web News Aggregator

I've just written a little python programme to pull a number of news xml files, and present them on my website for viewing.

It by no means compares with certain browser based news aggregators, but there are times when I'm at a computer that cannot run such aggregators, so it's nice to be able to simply go to my website and view them there.

There's also a link at the bottom my news page to the script's source code.

Deletion in RET Gene / Fin's Gear

Today the geneticist called with results from her DNA tests which confirmed the suspicion that her RET gene was "garbled" in some manner. Our next step is for Katharine & I to get tested to determine if we're carriers of the faulty gene, or if Fin developed the gene deformity all on her own.

It really doesn't make a difference to us, but it helps the greater research in Hirschprung's & genome mapping.

---

I've decided to post a couple of pictures of Fin and her gear for a few people who've shown interest in what kind of gear she sports.

Fin's Ostomy:



Fin's G-Tube (her feeding tube):



Fin all dressed:



Just Fin:

Dinner

On Friday we had a friend over for a nice light dinner. Now that it's spring time, we decided that a lighter dinner was in order (plus, it leaves more room for desert!).

Stephanie is always a good person to have over because she never gives any of our vegan recipes a funny look.

I think that the salad was a hit with everyone. Also, we had some neat sodas.

Down with protected class variables?

I'm not sure.

Kasia writes in her blog entry:

If a variable is not final then it should be private. If you need to make it accessible to another class and there is no accessor then you're probably not doing something right.

While true, that making fields/attributes publicly available via an interface of sorts (not "Interface in the sense of an abstract class, but just in the sense of certain known methods), and not directly from a "friend" class/function or a derived sub-class.

But, unless the compiler is doing some inlining for you, then the performance hit on making those extra accessor calls, and the extra layer of indirection seems prohibative.

But, then perhaps I'm forgetting that computers are constantly evolving and these kind of pre-conceived bottlenecks aren't actually anything to worry about.

Programming languages and their relationship styles

http://maradydd.livejournal.com/293666.html

I particularly like the comment on C:

I don't think C gets enough credit. Sure, C doesn't love you. C isn't about love--C is about thrills. C hangs around in the bad part of town. C knows all the gang signs. C has a motorcycle, and wears the leathers everywhere, and never wears a helmet, because that would mess up C's punked-out hair. C likes to give cops the finger and grin and speed away. Mention that you'd like something, and C will pretend to ignore you; the next day, C will bring you one, no questions asked, and toss it to you with a you-know-you-want-me smirk that makes your heart race. Where did C get it? "It fell off a truck," C says, putting away the boltcutters. You start to feel like C doesn't know the meaning of "private" or "protected": what C wants, C takes. This excites you. C knows how to get you anything but safety. C will give you anything but commitment

In the end, you'll leave C, not because you want something better, but because you can't handle the intensity. C says "I'm gonna live fast, die young, and leave a good-looking corpse," but you know that C can never die, not so long as C is still the fastest thing on the road.

I love this:

"You start to feel like C doesn't know the meaning of "private" or "protected": what C wants, C takes. This excites you. C knows how to get you anything but safety."

I love it!

It's very much worth the read.

C++ Exceptions: Continuance

In designing a language that supports exception handling, one must consider the aspect of continuance: where does execution resume after the exception has been raised?

I recently came upon an online conversation of Bjarne Stroustrup, in which he discusses why resumption directly after the statement that raised the exception wasn't implemented.

Basically, someone resuming from an exception handler can never be sure that the code after the point of throw was written to deal with the excecution just continuing as if nothing had happened. An exception handler cannot know how much context to "get right" before resuming. To get such code right, the writer of the throw and the writer of the catch need intimate knowledge of each others code and context. This creates a complicated mutual dependency that wherever it has been allowed has led to serious maintenance problems.

This is probably of very little interest to many people, but I find it rather facinating to see why certain decisions were made.

The Art of UNIX Programming

The current quote above ("When in doubt, use brute force") was retreived from the following book:

The Art of UNIX Programming

It's an interesting read (and free too!). It isn't a technical manual on how to write applications for the UNIX environment, but rather a philosphical approach for programming according to the (so called) UNIX philosophy.

The 17 rules are good rules to follow iregardless of the platform/language/environment that you work with. Also, the Interface Pattern page has a lot of insight on how to organise your tools. Make a number of small, flexible tools that can be used in conjunction to accomplish a number of tasks, instead of making a monolithic utility that tries to do everything itself.

I could go on & on, but instead you should go and check out the book for yourself. (Even if you're not a UNIX person)

Gak!

Over the weekend, Fred Fin and I made Gak. It was our first time, and everyone had a lot of (messy) fun. Fin really got into it and required a little more cleaning off than the rest of us.

Software Eng.: FIT & JUnit

Check out this article on IBM regarding a unit testing framework that enables business people to develop test cases using tools like excel.

SlickRun deserves its name

Thanks to my friend Mirek, I've installed SlickRun on both my home & work Windows workstations.

It's quite neat -- after setting up some "magic words" for my most frequently used applications & websites, I rarely use the start button anymore.

Very neat.

Back in the hospital

Today Fin started outputing more than usual. Katharine picked up some paedialites at the pharmacy & began replacing Fin's food with the paedialites 1:1.

At three-ish, I left work early to pick Katharine & the girls up and together we headed to Emergency at SickKids. She's been put on IV fluids & taken off of foods to give her stomach a rest while her virus runs its course.

It's 2am, and Winnifred & I just recently arrived at home -- unfortunately Katharine and Findley are still in observation until around 3 when paediatrics will finish their rounds and admin Findley into a room.

While we were waiting in Emergency, Winnifred and I were looking at a poster of cats when we heard:

"What, are, you, doing, here?"

We turned around to see Findley's surgeon, Dr. Annie Fecteau, who was on call that evening. She looked a little horrified to see us standing there, so I brought her up to speed on Findley's condition.

The thing about Findley & Dr. Fecteau is that as soon as Dr. Fecteau first operated on Findley, she, Dr. Fecteau became Findley's primary physician. Findley still has a Paediatrician, a GI team, a Nutruitionist, a Geneticist, a General Practitioner, and a few other teams that I'm forgetting at the moment. But, even if Findley develops a cold, Dr. Fecteau is her primary physician.

Imagine that, having a transplant surgeon being the first person you see as soon as you get sick.

Hopefully this stay at the hospital won't last more than five days or so. It will be interesting to find out what a stay in Paediatrics will be like. Most of our experience is on the surgical ward.

---

That said, here are a few comparison pictures of Fin in August, and her in January:

Spring Break

Over spring break Katharine, Winnifred, Findley and I packed up the car and headed north the Mattawa to visit Katharine's grand father before driving west to Sudbury for two nights.

I hadn't been to Sudbury in about 14 years, so the visit experience was completely new to me. We all got a kick out of Science North, and seeing the Big Nickel was fun. But I think that the biggest hit of all was the swimming pool at the hotel. I didn't know it was so easy to impress a three year old.



After two nights in Sudbury, we got back into the car and headed south to Katharine's family farm for a stay at the cabin. The snow was about 2-3 feet deep, so we had to park the car at the house and walk all of our stuff back to the cabin. It was nice and insular; Having a wood burning stove was super cozy.

Unfortunately the girls were asleep at the time, but at about 5:30 - 6:00am when I was up putting more wood on the fire I saw some wild turkeys and a deer out of the window. My camera was burried deep in the bag so I wasn't able to snap any shots for Winnifred. Although later on, she and I treked through the snow looking at all of the animal tracks.

Most people don't head to Central/Northern Ontario on their Spring Break (in February), but it was quite fun.

Kick Sun out

My school's SunLab is a collection of aging UltraSparc5 machines, and an Enterprise server running NFS & NIS for the workstations.

The workstations run fine if you limit your session to CDE, but if you try to log in with JavaDesktop (a Gnome variant) the systems quickly run out of their 256Megs of RAM and begin to thrash the swap space.

The funny thing is that my school has no Linux/FreeBSD/whatever workstations available (that I know of), but is content to limit its UNIX activities to the 20-something Ultra5s that are rotting away.

My solution would be to install a high-end x86 Linux server, offering account management via LDAP, and a number of mid-range servers running KDE as remote sessions.

The Ultra5s would be upgraded to Solaris 10 (currently sitting at 7) and be able to either:

1. Log in natively using CDE or whatever
2. Log in remotely to the Linux/KDE servers and display the applications locally

Additionally, the school should offer a customized Linux BootCD with which students could place into any Windows XP workstation on campus and boot into Linux, and connect to the remote KDE servers.

The only additional hardware requirements would be the Authentication Server, and the Application Servers. The school could keep their crappy Ultra5s (they have nice monitors) and expand their UNIX footprint with Linux/KDE applications etc.

That said, a place like that would never be implemented.

Chubby Fin

Findley's doing much better. She was only in hospital for about a week. It turned out that her inability to hold down food wasn't surgically related, but instead she had (at some point) contracted a stomach virus that was causing her to throw up.

The surgeons had her on an IV to keep her fluids up, and after a week, the virus seemed to have run it course, so she was discharged. She has since been putting on weight again, and is finally back on her growth-chart curve!!

It's nice to have the girl back at home.

Time away from computers

It's been a month since my last post and a number of things have happened since then:

1. Finished exams
2. Findley had surgery to have her G-Tube inserted
3. Started winter classes
4. Findley re-admitted to surgery five days after being discharged due to an inability to tollerate her foods

Number one was good. Two was good. Three was good. Four, not so good.

As it turned out, Findley had contracted a stomach virus which was causing her to reflux and dump her meals/fluids. So she's currently back at SickKids on an IV for fluids while they monitor her & her fluid retention. I hesitate to say that she'll be home soon, because discharge dates always change as soon as you become attached to them. She is doing much better and the virus has almost completely run its course. We'll hopefully have her at home again soon.