Fnord.

I woke this morning to a slew (here defined as ‘62′) of ssh dictionary attacks:

There were already 20 or so last night. Looks like a new botnet/attack wave or similar. I’m using DenyHosts and quite frankly, you should be too.

If you’re running Debian, there’s a nice package for it that I use and recommend. I’ve set mine to trigger on 3 attempts, but I’ve few users and most use ssh keys and not keyboard auth.

Might be a good time to run chkrootkit and change some passwords!

Inspired by the recent news that Google now has free sync of calendars and address books, I set off to try and get the following working:

1. I want to share a group called ’shared’ from my address book to Chris and her iPod touch. (There’s a bunch of other entries she doesn’t want.)
2. We really need a family calendar that we can both easily view and edit, for stuff like daycare outages, pediatrician visits, house guests and the like. If at all possible, push sync to iPhone/iPod would rock.

We use Bittorrent to download open-source software and also broadcast TV. (Just programs that are over-the-air free, as I don’t want the RIAA/MPAA mafia on my case). Unfortunately, bittorrent traffic really slows throughput on our network, and I can’t tell if the router or the DSL modem is to blame. I’ve tried setting BT as the lowest priority on the router, and also using the speed limiting features of the Transmission client we use. Neither solves the problem.

From googling and reading, it seems that the large number of TCP connections opened is the most likely culprit. However, that’s inherent to the BT protocol, so although I can limit the number of connections it’s not really a solution. From everything I’ve read, there simply doesn’t exist a SOHO router really built to handle the hundreds or thousands of connections.

After reading quite a bit and thinking about it, here’s my wishlist:

1. Wired-only router with 10/100 ports. Gigabit is nice but not useful given my connection speed.
2. Fanless, low power, no moving parts. This rules out using an old desktop machine.
3. VPN endpoint with at least PPTP support.
4. SNMP reporting ability.
5. IPv6 dual-stack or 6to4 support.
6. QoS/priority by protocol

1. Web GUI for setup and configuration
2. More than one IP and port on the LAN side, so I could setup multiple subnets
3. syslog support
4. Ability to have multiple WAN ports – someday I want to try BGP and similar routing.

M0n0wall is really close, but OpenBSD’s pf has a few unique tricks I don’t think Monowall can do:

1. Packet filtering based on TCP fingerprint. This is invaluable for spam blocking, as a rule that says ‘no Windows machines can connect to inbound SMTP’ is a nearly-complete block for botnet-generated spam.
2. Simple anti-spoofing, activated with a single line.
3. Packet scrubbing. (Read all about it)

After a lot of reading, my current solution looks like this:

That’s a Soekris 5501, which at 433 or 500MHz could actually run most of my server software too – more than a little overkill for this. Then again, for $320 its 2 to 3x the price of a SOHO router like my RV042. Liveable, and it’s worth it to get the capacity overhead.

Update 11/25/08: The m1m1wall firewall from Netgate looks better – $205 instead of 320, preinstalled with pfsense or m0n0wall. 256 vs 512MB, which is no problem. Best of all, you can get it in red!

For software:

1. OpenBSD plus pf, install via OpenSoekris. Update – see below, pfsense may be our winner.
2. pfw for GUI management and setup of firewall rules
3. SNMP from OpenBSD is looking pretty easy, and of course syslog and IPv6 are built in.

I like it. I think I’ll ebay off the 2U Cisco router, with its fans and 200+ watt heat, and buy one of these instead.

More reading if you’re curious:

1. Slashdot thread on routers, bittorrent, TCP connection limits.
2. Blog post OpenBSD on Soekris
3. SmallNetBuilder on Untangle, which is similar but Linux-based
4. OpenBSD on Soekris plus state-preserving failover (CARP, more advanced than I can use here.)
5. Soekris product page.
6. Soekris also sells a HiFn-based VPN accelerator chip, which can be added later if I need it. Nice.
7. IPv6 intro and IPv6 wiki.

As an amusing footnote, I was researching gigabit switches with SNMP support, so that I could monitor my backbone using Cacti. You can now get 24 ports of SNMP-managed gigabit for $300 from Netgear (GS724T), but the amusing part is that the 16-port version costs the same! Ahh, someday when I outgrow my 8-port maybe.

Update 11/18/08: Commenter timf points me to pfsense, which I had missed, and appears to meet every one of my wishes. Oops. OK, less work for me. Thanks!

Update 11/25: Added m1m1wall hw/sw combo.

Right now the home network is pretty simple. I’ve consolidated all of the server functions (web, dns, email, blogs, database, VoIP, etc) into a single Debian box, behind the Linksys RV042 router/firewall. Wireless is provided by a couple of Apple Airport Express boxes (one for G, one for N) and there’s also a gigabit switch as a backbone. Other than the Debian box, the rest are fanless and silent, which has been a continuing goal once we moved to a much smaller house!

(There’s other gear on the net, but it’s not relevant here.)

The Debian box has 4 drives in it right now, 1 for Linux and 3 250s in a RAID5 hardware array:

This works, and is stable and quiet-er. Or ish. It’s definitely not silent, though it’s hard to measure noise in any useful way. (I have a decibel meter, and I tried, sorry. Too many variables.)

One idea I’ve been kicking around is this: Other than disk, everything served by Debian now could run on much smaller machine, perhaps even (gasp!) one without fans.

Check this out: It’s an Asus EEE Box, a Atom-based PC for under $300:

(Wikipedia page is here for more info)

I was just reading a bit about IPv6 and researching what it’d take to add it to the home network, when I found smallnetbuilder.com. It seems to be a really good site for material like this, as well as NAS, wireless and router reviews. S/he likes the Untangle software bundle, which is a polished firewall/filter/router system. I personally prefer OpenBSD for firewalls, as pf can still do the most tricks, but I gotta admit that the GUI polish of Untangle is compelling. Here’s a screenshot from the Untangle site:

The files currently on the Debian RAID have mostly migrated to the iMac now (iTunes is quite nice for music management), so losing the RAID array is do-able. I’ve also got a co-worker with a hand-me-down NAS RAID that might work. So perhaps Untangle or something Debian-based on the Eee Box might work. 

From what I can find, the Linksys RV042 I’m using now lacks IPv6, so something like this is compelling.

Amazing times that we live in, eh? 

It used to be that, if you had corporate funding, you’d buy ‘enterprise grade’ routers and switches that sported a remote management protocol called SNMP, for Simple Network Management Protocol. Typically, you’d buy a copy of HP’s OpenView (at around 50,000$) and use it to monitor all of your SNMP devices. Since SNMP actually asks the routers what they’re doing, it’s precise and has very little network impact. Sorted, as the Brits would say.

Those of us who network as a hobby/home/hacker were left out of this particular nerdvana, and turned to indirect solutions like Smokeping, and/or free software like syslog. Workable but not as good.

However (you knew this was coming, right?) the world has moved on. Prosumer-grade routers like my Linksys RV042 (at all of $149) and Airport Express (the N version, $99, more on this later) now have SNMP support built in! So that’s half of the puzzle. There have also been significant advances on the software side:

That’s Cacti, a web-based SNMP monitor running on my Debian box. It’s one of the nicest pieces of programming I’ve seen in some time, very polished and amazingly easy to use considering the complexity of what it does. You have access control, easy device configuration, many many graph options, and the ability to monitor non-SNMP devices like, say, system load. I’ve setup a guest account, password guest, that you can experiment with here. The data is real, monitoring my firewall’s traffic and part of my wireless as well. The guest account is configured to be read-only, so feel free to poke around and experiment.

Let me back up a bit. First off, you need

1. SNMP-capable hardware. My newer Airport Express (b/g/n, part number MB321LL/A) has full support, but the previous b/g (part number M9470LL/A) does not. My RV042 has SNMP, but the more common WRT54 series doesn’t unless you install the third-party Tomato firmware. (More Airport info is here and here.)
2. A server to run Cacti.

Next up is the firewall, very similar:

On the negative side there’s a few of Cacti limitations to note:

• Cacti doesn’t support SNMP traps, or asychronous alerts. This is a showstopper for a serious monitoring system, as you won’t get notified when something dies or fails.
• Since it doesn’t have MIB support, using vendor or device-specific features requires writing a data query, which I’ve not yet tried.
• You can’t issue SNMP commands from Cacti.

 

While I was learning all of this, I also took detours into MRTG and snmpd and found this nifty Dashboard widget for OSX called iEyeNet:

Which requires no server, so if you just want a way to peek now and then iEyeNet is a great solution.

Overall I’m thrilled to finally get to play with SNMP, and having views into the state of my network is marvelous. If you poke around, you’ll see that I’ve also setup entries for monitoring non-SNMP hosts via simple pings, which is also nice but less complete. Overall Cacti and iEyeNet are both useful, free and easy to get running.

My wish list looks like this right now:

1. Alerts on failures or traps
2. Ability to look by traffic type, e.g. “Is bittorrent why I’m so slow?”

Overall, SNMP is highly recommended. I’d say flat out not to buy any new network hardware without SNMP built in. It’s just too useful. 

Update 11/17/08: Cacti works on the iPhone browser too:

The situation: We have a home wiki installed on Debian using MoniWiki. MoniWiki has some limitations, a borked search engine and has generally been annoying me, so I want to change wiki software. Current leading candidate is MediaWiki, the basis of Wikipedia.

The problem: MoniWiki is filesystem-based, where each page is a file. MediaWiki is database-hosted. I can’t find a single page on the net where someone moves the contents of a wiki from one to another. It seems to be the classic captive-data trap.

Weak.

Any suggestions out there? The wikimatrix site suggests that there’s a raw export for MoniWiki, but I can’t find a mention of how to import same without cut and paste or similar.

Links:

• MediaWiki on Debian page
• MoniWiki home page
• MediaWiki home page
• Wiki comparison site, very nice

apt-get broke something in my theme just now. I’m on travel, so fixing it will take more time than usual, sorry…

Update 9/9/08: Fixed!

This morning, in my inbox, this is what I saw:

That is, 37 reports that a host had tried to break in to my SSH port, and failed more than one password attempt. If I weren’t running DenyHosts, the dictionary attacks would have a lot greater chance of succeeding. As it is, I’ve seen at least 50 hosts blocked today; usually I get one every few days. Sadly, I’ve not got over sixty five thousand hosts denied via DenyHosts, which is a sad commentary on how many cracked Windows machines are out there.

Patch and monitor those boxes, people!

Non-geek summary: Email was down to phfactor for a few days due to a local problem. It’s back now, and I have tools to keep an eye on things.

Geeky details: I run exim4 on Debian, and had tweaked the config for dual-port (25/587) for SMTP inbound and outbound. One of the recent Debian updates borked something, such that

 -oX 25:587 -oP /var/run/exim4/exim.pid

was no longer valid, and thus exim refused to run. Oops. As a workaround, I’ve removed the port settings (meaning I have to use a different SMTP server when out and about, so this’ll get fixed later as time permits) and it’s back to running.

Having been bitten numerous times by similar problems, where one of many Debian updates will break a service, I spent some hours today researching server monitoring software beyond the Smokeping I have running already. I wanted something that could query a running service over TCP/IP and see if all was good. I came up with Monit via this post, and have it up and running now. Here’s a screenshot:

(click for full size)
Seems pretty good, decent list of native protocols that it understands. Right now it’s passworded off, not sure if I’ll remove the login or not. Seems mostly harmless to publish, since most of the monitored stuff is only accessible from this side of the router.

Update: Password will remain, as the webpage allows you to stop/start services! What a vulnerability that is, yeesh.

If you’re geeky enough to read the runes in the screenshot, you’ll see that usul (the phfactor.net do-everything Debian server) is now a Pentium 4 621, 3.2 GHz dual-core, with 2GB of memory. I just upgraded from a Athlon 64 3200+, 1GB, so this is a nice boost in capability. About the only negatives are analog VGA onboard (no DVI), and 10/100 instead of gigabit. A spare 3c996 solved that, and my old monitor has analog, so we’re sorted. Do email or call if you see any oddities!