• It was huge - 2,300 attending, even though it was “capped” at 1,500. More would have liked to attend.
• The on-site network worked very well, reliable and stable. Capped at 512KB/sec per person, which is fine. I tried out my LTE tether but didn’t need it.
• The conference hotel was good, and it was nice to be able to walk back to my room if needed. 149/night is a very reasonable rate, and the room was nice. Good chair to sit in.
• I attended two tutorials, which ran the day before the main conference. At $150 each I’m not sure I’d do so again; decent but not wow. The quality of the main talks is so high that the tutorials are just icing.
• I think next year I’ll go for a code sprint instead. I’d really like to contribute to Python or some of the libraries I use and appreciate.
• Overall, the conference felt like coming home. It’s odd, as I’ve been to quite a few conferences but this felt the most like my personality. Big expensive ones like Supercomputing are more about PR and sales, and even Software Development was kind of sterile and blah. PyCon felt like peers, getting together to hang out and do amazing stuff. And make no mistake, a lot of the people there are doing amazing and inspiring work.
• Jobs - I didn’t know this going in, but of the 134 exhibitors/sponsors, it seemed like all of them were looking for Python hackers. I’m not looking, but it’s reassuring to know the market is there for the skills.
• Very decent food for a convention this large.
• I will go again, even if I have to pay my own way. Strongly recommended.

And a few more specific notes and pointers:

• The OpenStreetMap project has advanced a huge amount. Check it out as a Google maps equivalent/replacement.
• My deep thanks to the Dropbox folks for giving out 5GB at their hiring booth. I needed that!
• NewRelic looks awesome. Magic-level app analytics.
• I’m already using Disqus here and on WatchOtaku, and stopped by to confirm with them that I had implemented it correctly there. Nice folks, they setup a board just for pycon.
• Speaking of chat, once again IRC was helpful, I used Colloquy to hang out in the #pycon channel.
• I tried to take the ‘Google at PyCon’ challenge but after passing the first level their email never made it to me. Weak.
• I really need to try the Pyramid web framework. Talked a bit to the hackers and it sounds like a nice replacement for web.py for the code I’m doing.
• The NAO robot demos were amazing, I wish I could afford one. Even at developer discount they’re 6k USD each.
• The WebCube platform might be nice; I have a friend or two needing sites that’d be well suited for this.
• The new iPython notebook looks astounding, but I wasted a few hours trying to get the stack of requirements installed to no avail. You have to install an entire Python distribution EPD to get it working.
• I was impressed by Web2py but a lot of the Pythonistas were skeptical or dubious. Interesting.
• Videos of all of the talks and posters are free to watch at PyVideo.org.
• Almost forgot PiCloud, a nifty-looking way to run Python with 2 lines of code. Up to 20 CPU hours per month for free, very much worth a try.

For next year:

• I spent some airport time hacking up a quick Bootstrap/web.py app for signing up for an Ultimate game. I think it’d be great to have during PyCon, will see if I can make it happen. A bit daunting, between the logistics and not knowing anyone, but who knows.

2010 Bordeaux red, $25 for 3L. It’s actually pretty good, an everyday wine and quite drinkable. For four bottles’ worth the price is good, and if it keeps I’ll be happy. Unlike before, the spout isn’t leaking, either.

1

perl -pi -e 's/layout: post/layout: post\ncomments: true/g' *.markdown

I think the exitwp script that I used didn’t add the comments: true field.

If it’s free, they can always go out of business, change the terms, or generally do anything they want. Those are your risks.

There has been, in the last couple of years, a recurring spate of news about Google that gives me pause. Censorship in China, the blending of Plus into search, etc, etc.

The thing that tipped me into action is their awful new ‘privacy’ policy (By the way, you should really clear your Google history before that takes effect on March 1st!). Me no like.

It’s their company. They have to make a profit, and it’s their choice how they do so. From the point of view of a random nerd, I strongly dislike the level of fine-grained data they will be gathering and selling. I am choosing to look for alternatives, and I’m happy to pay for them.

Let me share what I’ve found so far. For the most part, I’ll just mention the ones I chose and not try to enumerate all the alternatives.

Search

Still far and away the best product from Google, I must say. I’ve been using the oddly named DuckDuckGo which I found via John Gruber. It takes some getting used to, but has some new features I like too. Using ‘!so’ to search StackOverflow is handy, as are the Python and Android shortcuts. I’m warming up to it.

Search in Safari or other browsers

If, as I recommend strongly, you are running GlimmerBlocker, then adding DDG to Safari is pretty easy - see this page for instructions.

Chrome or Firefox are also tweakable; see the same page

Search on iPhone / iPad

No easy way to replace Google with DDG here. DDG does have mobile search apps for iOS and Android, but it’s not integrated into Safari.

Search on your blog

If you use the search bar here on Fnord, as of today it’ll go via DDG. Kudos to them for having Octopress instructions - major nerd plus! They also support other platforms, of course.

RSS/Atom (Google Reader)

There are several alternatives here. I found and recommend NewsBlur, which is Freemium, open source and has an iPhone app. I simply exported my OPML from Google Reader, imported it into NewsBlur and have never looked back. I paid for the Premium account, which is as I recall ~20/year; you set the donation amount.

Minuses:

• iPhone app is a bit crashy
• No iPad app yet

Overall - strongly recommended. I use the main site on the desktop and the iPhone app. Works great.

Email

I run an Exim-based mailserver here, though a few years ago I conceded the spam battle. My personal email is now forwarded to SpamCop for filtering. $30/year, fantastic. Never a single outage or problem in three years now. I have a hybrid setup, where I archive messages to my IMAP server, but the inbox is on SpamCop.

Blogs

Depending on your level of techie interest, you can self-host as I do, use Github and Octopress, or one of the other free platforms such as Tumblr, Posterous, WordPress.com and so on. I prefer to self-host, as that way I have my data locally and complete control. Plus, it’s fun to do this stuff yourself!

If you do run WordPress, there are commercial services that’ll handle all the sysadmin work for you. It’s a pain if your site gets popular.

Analytics

I paid for and like Mint. You can see my stats on this page. The price is reasonable, the install painless and the results decent. For my WatchOtaku site, I’ve been using Clicky, as it’s a bit nicer but really Mint would work there too. There are others; for my level of traffic and casual observation these both suffice.

Code and project hosting

Go Github. Don’t even consider anything else. It’s that good.

Also works for static pages like this one

If you need an in-house-hosted or FOSS solution, Gitlab is pretty darn near a complete workalike.

Maps and directions

OpenStreetMaps is excellent. Good for GIS work too.

Google Docs

I used to use this more at UCSD, these days it’s a combination of IM, Email and the rare DOCX file. No suggestions, other than to suggest using HTML for doc formatting; makes posting it easier anyway.

Maybe Dropbox?

Google apps for your domain

I use this a bit; Google is still doing the mail for WatchOtaku. I can live with this for now and will migrate off if the account starts getting a lot of email; right now its about 1 per month and I can accept that. Since that’s a domain-specific hobby/account/site, the analytics from it are gonna be bizarre.

Conclusions

Google has some excellent products and replacing them takes time and money. It’s worth it to me. I’d rather spend a few bucks then be monetized, but of course YMMV.

So today did a hack while inspired. Shared as a gist on Github, this code:

• Makes the path YYYY/MM/DD, e.g. ‘2012/02/24’
• Verifies that the directory created is writeable
• Does an upload via the filesystem
• Fetches the uploaded file via HTTP for an end-to-end check
• Computes MD5 hashes of both to make sure it made it intact
• Print out the full URL of the image, for easy use in your editor
• Copies said URL into the copy & paste buffer via pbcopy

Usage is pretty basic:

1
2

code/Image_uploader/upload.py Desktop/sorry.png
http://fnord.phfactor.net/wp-content/uploads/2012/02/24/sorry.png

In one step, you’ve uploaded the file and can go paste the URL into the editor!

I wanted to use scp, but was unable to find a way to create directories easily. It’s do-able, but since I paid for ExpanDrive I’m not averse to using it. Here’s the code in case you want it:

Gotta love Python. And Github. And Octopress.

It suggests I complete thier HRA (Health risk assessment). This is something I found useful while at UCSD, so I click the link, sign in to my account, and promptly get this:

…which has to be one of the least useful dialog boxes I’ve ever seen. At a guess, the email was form-generated, and my current employer lacks the HRA feature. But WTF, Aetna? Shouldn’t you, you know, check first?

Yeesh.

Anyway, yesterday I got an email from the IEEE reminding me that my email alias, phubbard@computer.org, was going to expire unless I renewed my membership here soon. Which lead me to think a bit and ask a simple question:

What am I getting from professional memberships and is it worth the cost?

After all, IEEE is about $200 per year and ACM is ~$100; are they worth it?

It used to be that professional societies provided human networking, training, insurance plans, and publications. These days, with LinkedIn, various websites and such, the only service I actually use is the email alias, and it’s not like that’s of great value to me.

A couple of bullet points on the resume, meh. “Member of IEEE and ACM” has never gotten a comment.

Let’s not even mention the free magazines you get. They’re worthless, full stop.

I suspect that a lot of societies, both fraternal and professional, are going to have to reinvent their utility in the age of the Internet. The old value propositions no longer hold, and they need to find new ways to be useful. For me, as a working developer, IEEE and ACM no longer make the cut. Here’s hoping I have reason to reconsider some day.

Click it and you can subscribe to daily updates via email, implemented via Feedburner. I know that some folks used to read the site that way, so I’m pleased to have it working on Octopress.

If you want to set it up on your Octopress site, there’s two steps. You have to setup Feedburner with your RSS or Atom feed, then go to the ‘Publicize’ tab and activate email. Then take the HTTP URL they give you and enter it into the _config.yml:

1

subscribe_email: http://feedburner.google.com/fb/a/mailverify?uri=phfactor/rzbg&loc=en_US

Leave a comment if you have any problems, or email me - the feed now includes my email address.

I have a semi-solution using (as with Fnord) Disqus. Some notes for other folks attempting this:

1. You have to use the raw Javascript functions of Disqus
2. On Confluence, go into Site admin / Theme / Layout editor / Page footnotes
3. Change Options to ‘Velocity then raw HTML’
4. Change the shortname to that of your site

Here’s mine so far, with the short name munged:

1
2
3
4
5
6
7
8
9
10
11
12

<div id="disqus_thread"></div>
<script type="text/javascript">
    var disqus_shortname = 'CHANGEME'; // required: replace example with your forum short name

    (function() {
        var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;
        dsq.src = 'http://' + disqus_shortname + '.disqus.com/embed.js';
        (document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
    })();
</script>
<noscript>Please enable JavaScript to view the <a href="http://disqus.com/?ref_noscript">comments powered by Disqus.</a></noscript>
<a href="http://disqus.com" class="dsq-brlink">blog comments powered by <span class="logo-disqus">Disqus</span></a>

This seems to almost work, but the variables that tell Disqus what page it is seem to be unreachable. Normally, you set

1
2

disqus_url
disqus_identifier

but I can’t tell how to read those. I’ve tried (from these docs)

1
2

$req
$req.contextPath

but those are not found.

Syntax was

1

var disqus_identifier = $req.contextPath;

which produces an error visible in the Safari error console about req not being found.

I also tried (from these docs)

1
2

$content
$content.getEntity().getRequestUrl()

but even content is not found. Perhaps some syntax error with the Velocity engine stuff, but the mixture of Velocity and Javascript is very poorly documented. I was unable to find anyone else using Disqus with Confluence, so if you have any tips do say so here!

At present, it’s almost working. The page portion loads, the Disqus portal seems happy, but due to the lack of URL is causing comments to not load on the page they should be attached to.

Update 3/10/12: I talked to one of the Disqus hackers at the PyCon convention, and he confirms that this approach works fine. If no URL or ID is set, Disqus sniffs the URL and comments will follow it as long as the URL is constant. Excellent.

Secondly, a quick tweak to Octopress - when you create a new post using rake, I added calls to add the post to git and fire up the editor, as those are what I’d do next anyway. Here’s the snippet from the Rakefile:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

# usage rake new_post[my-new-post] or rake new_post['my new post'] or rake new_post (defaults to "new-post")
desc "Begin a new post in #{source_dir}/#{posts_dir}"
task :new_post, :title do |t, args|
  raise "### You haven't set anything up yet. First run `rake install` to set up an Octopress theme." unless File.directory?(source_dir)
  mkdir_p "#{source_dir}/#{posts_dir}"
  args.with_defaults(:title => 'new-post')
  title = args.title
  filename = "#{source_dir}/#{posts_dir}/#{Time.now.strftime('%Y-%m-%d')}-#{title.to_url}.#{new_post_ext}"
  if File.exist?(filename)
    abort("rake aborted!") if ask("#{filename} already exists. Do you want to overwrite?", ['y', 'n']) == 'n'
  end
  puts "Creating new post: #{filename}"
  open(filename, 'w') do |post|
    post.puts "---"
    post.puts "layout: post
comments: true"
    post.puts "title: \"#{title.gsub(/&/,'&')}\""
    post.puts "date: #{Time.now.strftime('%Y-%m-%d %H:%M')}"
    post.puts "comments: true"
    post.puts "categories: "
    post.puts "---"
  end
  # pfh 2/10/12
  system "git add #{filename}"
  system "edit #{filename}"
end

One more note - if you use zsh, you have to escape the square brackets. Here’s the syntax when I created this post:

1

rake new_post\["New CSL Arduino posting"\]

Mildly annoying.

It’s a new site with an interesting presentation of the data. Price bands, uses model generation and not production year by default, and pulls data from other sites. Worth a try if you’re in the market for a used car.

AutoGlance

I just added it to Octopress, so the data will be worthless for a while.

LSI update

I thought I had generated native code, but after a solid ten hours of cranking I killed it. So no latent semantic analysis for us, yet. Do, uh, leave me a comment if you have an idea!

In fact, please leave a comment and let me know how you like Disqus. If people hate it and want it gone, I would understand.

In _config.yml, set

1

lsi: true

…and watch your laptop turn into a smoldering heap of rubble. My

1

rake generate

is still cranking, fans roaring, as it tries to generate related posts. It does helpfully say

Notice: for 10x faster LSI support, please install http://rb-gsl.rubyforge.org/

But rb-gsl is Ruby on top of GSL, hmm. GSL compiles and installs no prob, but rb-gsl dies:

1

gem install gsl

yields lots of compiles then

1
2
3
4
5
6
7
8

gcc -I. -I/Users/hubbard/.rvm/rubies/ruby-1.9.2-p290/include/ruby-1.9.1/x86_64-darwin11.2.0 -I/Users/hubbard/.rvm/rubies/ruby-1.9.2-p290/include/ruby-1.9.1/ruby/backward -I/Users/hubbard/.rvm/rubies/ruby-1.9.2-p290/include/ruby-1.9.1 -I. -DHAVE_NARRAY_H  -I/Users/hubbard/.rvm/gems/ruby-1.9.2-p290/gems/narray-0.6.0.1/. -I/Users/hubbard/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/site_ruby/1.9.1/x86_64-darwin11.2.0 -I/Users/hubbard/.rvm/usr/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE   -fno-common  -Wall -I../include  -I/usr/local/include  -o matrix_complex.o -c matrix_complex.c
matrix_complex.c:1525: error: conflicting types for ‘gsl_matrix_complex_equal’
/usr/local/include/gsl/gsl_matrix_complex_double.h:227: error: previous declaration of ‘gsl_matrix_complex_equal’ was here
make: *** [matrix_complex.o] Error 1


Gem files will remain installed in /Users/hubbard/.rvm/gems/ruby-1.9.2-p290/gems/gsl-1.14.7 for inspection.
Results logged to /Users/hubbard/.rvm/gems/ruby-1.9.2-p290/gems/gsl-1.14.7/ext/gem_make.out.

Drat. No fix yet.

Update: Fix is to use v1.14 of GSL, not 1.15. Cranking now…

Update 2/5/12: Compiled, but still unusable. Killed it after ten hours.

Fnord is now running on Octopress. Should be faster and more stable. Enjoy.

Why Octopress?

When I started Fnord back in 2005, WordPress was the hot ticket. Well supported, lots of add-ons and themes, and most importantly a Debian package. Since then, the ongoing upkeep has been a bit of a hassle, I’ve gotten hacked once and also just have the itch to try something else. The PHP in WordPress has too many fiddly bits, and these days something as simple as a free WordPress theme can get you hacked.

It’s time to move on.

My first impulse was to move to Confluence. I use that for WatchOtaku, where the combination of blog+wiki is perfect. I created a project and did some hacking, but the HTML parsing had too many edge cases. I should have used BeautifulSoup or similar instead of trying to parse it myself; lesson learned.

Sometime later I read several posts about the advantages of a static site: they are super efficient to serve, fast under load and much harder to hack. I spent some time trying to make Rui Carmo’s code work, but that was a pain and the docs were poor.

Then I read this post by Matt Gemmell and its link to the key missing piece, exitwp. Boom!

So I went with Octopress. Killer support for mobile and such, designed for programmers, plugins, nice fonts, twitter and github integration, etc, etc. I’m having fun, and the results look good so far.

Most importantly, Octopress is fun. I blog because I want to, and having a platform that encourages that is a major plus.

Why Not Octopress?

There are some drawbacks. It’s definitely harder to get working, oriented as it is towards software developers. Ruby, git, Markdown, rsync, Apache config; all pretty geeky. On the flip side, that niche appeal should make it less attractive towards spammers.

No nifty iOS app for blogging. That I will miss.

MarsEdit works with a bit of effort.

Comments, Categories and email subscriptions: missing features

1. One key feature I want was the dropdown list of categories. I’m trying to figure out how to replicate that here.
2. As for comments, to be honest I get about 1 real comment per month here, and several per day of varying spam. So I have no comments in place, and am not sure if I want them at all.
3. I have a few folks subscribed via email, using a wordpress plugin. I’ll see what I can do there.
4. I need to decide on stats & analytics and perhaps add Mint to the header. Or getclicky; I like that too.

A Note For Others

As noted on this page, you need set a couple of environment variables before running rake to generate the site:

1
2

export LC_CTYPE=en_US.UTF-8
export LANG=en_US.UTF-8

Otherwise you get errors like this:

1
2
3
4
5

## Generating Site with Jekyll
unchanged sass/screen.scss
Configuration from /Users/hubbard/build/octopress/_config.yml
Building site: source -> public
/Users/hubbard/.rvm/gems/ruby-1.9.2-p290/gems/jekyll-0.11.0/lib/jekyll/convertible.rb:29:in `read_yaml': invalid byte sequence in US-ASCII (ArgumentError)

Overall, Octopress is a huge win. Strongly recommended, and I plan to move the other blogs here over as time permits.

As part of the move, the entire site, with configuration and posts, is public on github. I hesitated a long time over this, but really? The site is public, the config holds no secrets, and it might help someone else get going. Enjoy!

It’s great. I continue to wish I’d gotten it sooner.

My previous firewall was the Cisco/Linksys RV042, a entry-level business unit that worked quite well for years here.

There were a few things I wanted that the RV042 couldn’t do:

Better handling of bit torrent. Nothing better for ISOs! The RV042 choked under, I think, large numbers of open TCP connections. Hard to tell if the limitation was CPU or memory or what, which leads to… Better instrumentation, visibility and metrics. I want to be able to see how it’s performing and if I’m up against limits Better filtering and traffic handling. Stretch goal: Block all inbound mail connections from any machine running MS Windows. Invariable botnet spam, and one of the niftier tricks possible with OpenBSD and pf. Turn-key-grade. I’ve less time to DIY these days, so I needed something good to go out of the box.

Running pfsense 2.0, the new firewall is superb beyond my expectations. The web-based GUI is fantastic, the 500Mhz CPU and 256MB of memory handle more traffic quite ably, and the sheer depth of functionality present is almost daunting. This wee beastie is astounding. Here are some bits I’ve liked so far:

RRD graphs for CPU, memory, TCP state tables, rules traffic, etc, etc. Updated via AJAX, no less. Nice dashboard showing status and traffic at a glance. It ships with nanoBSD installed twice, two partitions on the 4GB compact flash card. The idea is that, if you hose the firewall, you can boot into the second partition and be back up quickly. Haven’t tried that, but I love the no-moving-parts flash storage. The 2D3 version gives me an extra ethernet port, for later expansion. I got the HiFn crypto accelerator, which can do 35MB/sec of AES128. Good support for Apple - OpenVPN for laptops, and IPsec for iDevices. Nice list subscriptions, so I’ve subscribed to IP-based blacklists for compromised hosts, ad servers, spammers and the link. Transparent HTTP filtering with squid and squid guard. I could also use this for caching if I wanted, but for now it’s an easy way to block domains like 2o7.net and the like.

So here are some RRD graphs for a monster bit torrent test, around 200-400 peers.

Traffic:

TCP states, peaking around 3,000:

Traffic: Red is upload, set to a 2.0 sharing ratio:

The important one, CPU usage:

So, roughly speaking, the 20Mbit peak took about half the CPU with the current minimal rules and fire walling. That’s not astounding, but I suspect and hope that I can tune it better. Next up, I’m going to try the packet shaping to see what effect that has. Right now, ICMP goes to hell:

I’ve done a simple test with the OpenVPN, which works though I had to buy the Viscosity app to make it work. ($5). Need to get iOS working and see how well that works, that’ll be nice to have for the trip to PyCon in March.

Here’s a shot of the web interface:

(Yep, I named it ‘fratboy.’ My old firewall was ‘nail.’ The theme is ‘things that get hammered.’ I still think it’s funny.)

Other things to look forward to:

Full IPv6 support (Hi, BenC!) Operating-system-based filtering Adding a 2-line LCD screen, I want a traffic barograph, and the 2D3 has a serial port I can use.

Overall? For $300 it’s a bit steep, but frankly I now regret having bought two RV042s; shoulda gone here years ago. It out-features commercial routers up to two or three thousand bucks, and does so with silence and ~4 watts of power. Yay!

Strongly recommended.

I am in heaven. Highly recommended.

Check out this home grilling setup: coals on the left, 5 feet or so of grilling area.

Now I’m all hungry again.