Fnord.

I’ve been an RSS fan for years now, and with the advent of the iPhone/Google Reader combo I read even more. I used to use NetNewsWire, but by using Google Reader I’m always in sync on mobile and desktop. Their iPhone interface is really good, even on EDGE data rates.

Yesterday, I checked out Google Reader Trends, and found the following:

Umm, that’s a lot, right?

Kinda interesting. What I’m finding is that the iPhone allows me to do short newsfrouping sessions when I have a few spare minutes, so I end up reading more than I did before. I also really like that the load of loading feeds and pages is moved to Google, resulting in less laptop/PDA CPU usage and faster overall performance. It is, in a word, magnificent.

Even if you don’t have an iPhone or iTouch, Google Reader is still most excellent. If you do, fire up Safari and start keeping up with more of the world than previously possible!

Highly recommended.

P.S. I have several hundred feeds, with an unusual organization that I plan to post later. I’m also happy to share the list with you in OPML format if you want to jump-start your Reader setup.

Update: Here’s reading by time of day and day of week. Lots of late-night!

One of the minuses of urban life is traffic. It can be hellish, even though the locally-absurd gas prices have reduced traffic a bit. In particular, I am of the opinion that few things are ruder, stupider and more dangerous than a sports fan on their way to a game. Except maybe that same fan returning home after a few beers and a big loss. Locally, any time there’s a Padres game downtown, the traffic backs up highways 163, 805 and 5 for miles. The Chargers aren’t quite as bad, since Qualcomm Stadium is east a bit, but still a problem, as are things like Fleet Week.

For whatever reason, web-based traffic feeds don’t have any concept of ‘monstrous event with traffic implications,’ though I really wish they’d get a clue. Maybe add a sidebar?

Initially, I had the idea to simply subscribe to their respective iCalendar feeds, and be warned that way, but that’s pretty noisy; all you really care about are home games, but there aren’t separate calendars for that.

Recently I found a solution that seems to work quite well called FuseCal. It can read all sorts of feeds, including graphical calendars (impressive, that) and allows you to filter them via strings. So I have three feeds right now merging into a tag of ‘traffic factors’

1. Padres + “at San Diego”
2. Chargers + “Qualcomm Stadium”
3. Fleet Week (all)

If you have any suggestions as to other events to add, leave a comment, also if you want I can make the feed public. So far I’m pretty happy - this is a free service, it works very well and the merge of feeds produces information that’s hard to get otherwise. Very cool.

I was reading about Google Gears, and discovered that Firefox 3 on OSX is supported. It’s a quick install, and once you do so check out Google Reader - you’ll find ‘offline mode,’ where Reader will slurp down data for use offline and re-sync fast when you’re back to civilization.

Another reason to love Reader! It makes a compelling case for Gears, too. Highly recommended.

This morning, in my inbox, this is what I saw:

That is, 37 reports that a host had tried to break in to my SSH port, and failed more than one password attempt. If I weren’t running DenyHosts, the dictionary attacks would have a lot greater chance of succeeding. As it is, I’ve seen at least 50 hosts blocked today; usually I get one every few days. Sadly, I’ve not got over sixty five thousand hosts denied via DenyHosts, which is a sad commentary on how many cracked Windows machines are out there.

Patch and monitor those boxes, people!

It’s often handy to be able to surf from a monitored network. Perhaps you want to job-search Monster from work, view (cough) inappropriate comment on your lunch break, check your bank balance in a open-wifi coffeeshop, whatever. For it to really be secure, all parts of the channel must be encrypted, including the DNS lookups. Here’s a free way to do it. (If you have the money, you can also run a VPN; this is a simpler way of building a private HTTP-only VPN.)

You need an SSH account on a server that you trust, and a copy of Firefox. That’s it! An SSH agent such as SSHKeychain is nice for not having to re-enter passwords every time, but not required.

For this example, I’m going to use my server, usul.phfactor.net, so change accordingly for your setup.

There are two parts of this: The first part is to setup an encrypted connection, which has what’s called a SOCKS proxy. This forwards your HTTP requests over the SSH connection to the remote server, which does the DNS query, HTTP fetch and returns the results to you via SSH. It turns out that SSH itself has a SOCKS proxy in it, you just have to enable it.

The second part is to tell your web browser to use the proxy, and how to find it.

Here’s the first part:

 ssh -C -D 8119 pfh@usul.phfactor.net

That connects me to my account (pfh@phfactor.net). The -C asks for gzip compression, which speeds things up a bit over slow connections. The -D 8119 sets up the SOCKS proxy, and establishes port 8119 on the local machine as the proxy connection. You’ll see a prompt after this, as the SOCKS part is invisible:

Next, you have to tell your browser about it. I’ll use Firefox, as its common and easy to use. Open up preferences, and go to Advanced/Network:

Then go to Settings:

Put in localhost/8119 for the SOCKS proxy, and use SOCKS4 and not the SOCKS5.

That’s it! Now, every web page you surf travels over SSH to the remote host and is encrypted from you to there. (In my case, until it hits usul.phfactor.net) You still have to trust the server and sysadmin who runs it, but that’s easier sometimes.

Enjoy!

Sources: This page had the key bits, before I found it I had tried twice and failed. Thanks, interweb!

Non-geek summary: Email was down to phfactor for a few days due to a local problem. It’s back now, and I have tools to keep an eye on things.

Geeky details: I run exim4 on Debian, and had tweaked the config for dual-port (25/587) for SMTP inbound and outbound. One of the recent Debian updates borked something, such that

 -oX 25:587 -oP /var/run/exim4/exim.pid

was no longer valid, and thus exim refused to run. Oops. As a workaround, I’ve removed the port settings (meaning I have to use a different SMTP server when out and about, so this’ll get fixed later as time permits) and it’s back to running.

Having been bitten numerous times by similar problems, where one of many Debian updates will break a service, I spent some hours today researching server monitoring software beyond the Smokeping I have running already. I wanted something that could query a running service over TCP/IP and see if all was good. I came up with Monit via this post, and have it up and running now. Here’s a screenshot:

(click for full size)
Seems pretty good, decent list of native protocols that it understands. Right now it’s passworded off, not sure if I’ll remove the login or not. Seems mostly harmless to publish, since most of the monitored stuff is only accessible from this side of the router.

Update: Password will remain, as the webpage allows you to stop/start services! What a vulnerability that is, yeesh.

Via TidBits, news that the new Apple Time Capsule is shipping. Its a combination device with a WiFi router and internal hard drive. The WiFi is the new 2.4/5GHz 802.11n, which increases both the speed and the range of the network. According to the Wikipedia page, going from 802.11g (which most computers made in the past 3-4 years have), you go from 38m to 70m in range and 19 to 74 megabits of usable throughput.

I’ll pause so that the nerds reading this can cease drooling. That’s a big jump in range and speed, which naturally leads to the question of ‘What do I do with that much bandwidth? I mean, my internet connection is much slower than that.’

In a word: Backups. OSX 10.5 introduced TimeMachine, integrated backup software of stunning polish and elegance. I’ve spent years with software of varying capability, so trust me when I say that stunning is not an exaggeration. cpio, tar, rsync, unison, windows backup (shudder), ufsdump, Retrospect… it’s a long list. I still use cron and rsync for Debian on this server, but it’s got serious limitations and my main defense is to replace the hard drive every two years. TimeMachine solves all these, in an automatic fashion that’s tied into the operating system itself.

Let me explain. If you use a third-party backup program, consider what happens when the drive fails. You have to reinstall the operating system itself, usually a manner of hours, then install the backup/restore program, then restore the backups. With OSX, you boot the install DVD and select the ‘Restore from TimeMachine backup’ option. Vive la difference, baby.

So, motivation established, you consider 802.11n routers. However, if you’re like me you have older machines that don’t have 802.11n and can’t afford to upgrade them all. You’re then faced with the hassle of running two wireless networks, one slow and one fast. Double the fun it ain’t.

Time Capsule (seemingly) solves a lot of these. And some others. Right now, I use an older Airport Express (802.11g) plus a LaCie NAS drive. It mostly works, but since the LaCie is RAID0 I’m actually more at risk; if either drive in the NAS fails I lose all my backups. And it’s slow as well, even over gigabit. In contrast, the Time Capsule uses a single drive, which is server-grade:

Chulani clarified that the “server-grade” drives in a Time Capsule are the same 7200 rpm drives used for Apple’s Xserve servers, and that they have a higher mean time between failure (MTBF) rating than consumer drives. The MTBF for server-grade drives is often 1 million hours (114 years), which is a measure of probability; in this case, that out of a set of drives with similar properties, an extremely high percentage will still be fully functional after several years.

(That’s from the TidBits article)

The phrase from TidBits that got me really interested was this:

AirPort Utility 5.3 also adds setup features that enable you to migrate settings from an existing base station into the Time Capsule; to set up a dual-band network, with an older base station operating at 2.4 GHz and the Time Capsule set to 5 GHz; and to set up a roaming network with multiple base stations connected over Ethernet.

If I’m reading that correctly, it sounds like the TC can run somehow with another WiFi router? Hmm. I don’t have Airport Util 5.3 and can’t find a download URL, so this is hard to verify.

The TimeCapsule costs $299 for 500GB and $499 for 1TB, which is quite good. If I can get it to replace or complement my Airport Express, it’d be fabulous.

Update: Nice review and walk-through on Gizmodo.

Gotta transfer DNS from UltraDNS (now costing $50/month!) to EveryDNS.net. Not gonna be simple.

Wish me luck…

Update: Here’s why I’m moving. My UltraDNS contract is for 10k queries/month, and we’re getting 10 times that. It’s not showing up on the page views, so I’d guess it to be spammers.

Update 1/9/08 2:30PM: Update in progress, we have to wait for Network Solutions to update the root, and also for everydns to update their databases. Expect DNS errors for a day or two, sorry.

PS Yes, I donated to EveryDNS. Karma is good.

Update 1/10/08: Done! It works! A-maze-ing.

From HPWREN, a new set of cameras capturing 20-megapixel (!!) images of the city and view:

Wow. For those of you still in the depths of winter, feel free to click through and see the awful conditions here.

First off, a nice paper that talks about how Amazon builds their system to be scalable and reliable.

(I blogged about a related item not long ago)

Next up, a bit of very good news. I’ve been bummed that the brilliant (and I do not exaggerate) Google innovation of ‘map-reduce‘ for parallelism is proprietary. As in, I can’t look at it, try it, deploy it, etc. Ditto for GoogleFS and BigTable. Bummer, that, but it makes corporate sense.

Today, I found the Apache Hadoop project, which as far as I can tell explicitly recreates map-reduce and gfs! This pic is from their site:

What’s more, it’s not a toy project either:

Hadoop has been demonstrated on clusters with 2000 nodes. The current design target is 10,000 node clusters.

The filesystem is similarly designed:

The Hadoop Distributed File System (HDFS) is a distributed file system designed to run on commodity hardware. It has many similarities with existing distributed file systems. However, the differences from other distributed file systems are significant. HDFS is highly fault-tolerant and is designed to be deployed on low-cost hardware. HDFS provides high throughput access to application data and is suitable for applications that have large data sets. HDFS relaxes a few POSIX requirements to enable streaming access to file system data. HDFS was originally built as infrastructure for the Apache Nutch web search engine project. HDFS is part of the Apache Hadoop project, which is part of the Apache Lucene project.

Killer stuff! I’ve lost track of some of this stuff, but had read on the Register about an update to IBM’s GPFS that also sounded cool. It’ll be interesting to see which is more advanced, but I’m delighted that distributed computing is once again a cool and happenin’ thing.

Hmm, it looks like Hadoop is a) Java-based and b) designed for single-computer installs, so I don’t need monster hardware to play with it:

By default, Hadoop is configured to run things in a non-distributed mode, as a single Java process.