As some others have mentioned, it’s that time of year where most people with blogs make their public announcements on what languages they’ll be concentrating on for the upcoming year. Last year I think my top focus was Ruby on Rails and Javascript; so safe to say I’ve gotten a little experience in those. Things have taken a bit of a turn though, and I’m going to switch things up for 2009 and concentrate a little on my weaknesses and a little on new growth.

Weaknesses? Yeah, everyone has them. Those areas where when you have to implement something you bite your lip and find a way to get through it — usually with a 10ft poll. If they’re something that ties into your daily routine in some way, this is kind of task that can be holding you back from potentially getting more work done, or higher quality work.

One of my weaknesses has always been testing. Maybe I’ve been surrounded by too many testers, but its definitely something I could improve on. I wouldn’t consider myself fully proficient in a language to the point where I’d feel confident teaching someone else until I was proficient testing in it. Some languages don’t make this easy of course. Testing Javascript isn’t the most common thing, but it’s important, especially with such a Javascript heavy application as CloudShout. There are some great new tools like FireUnit for testing in Javascript too. Testing in Python, especially within Django has to have great support — as long as you actually do it. With Rails there’s a million different ways to test; but I’d like to concentrate mostly on mochs and stubs for a swiftly running test suite. No point in having a test suite that takes so long to run that no one has the time to run it after all.

There are a few other topics of interest for the new year. Things I’d love to look into and play around with at least a least a little. XMPP and especially bosh look extreme interesting to me, and I look forward to cramming together the little I’ve learned so far into an Adogo presentation in a little over a week.

Collective intelligence looks like another interesting topic that I’ve always wanted to dig a little deeper into. The Netflix prize looks an great way to experiment a little with it in some different language — Python, C and Erlang seem to be the most commonly used for it. There’s at least one active Rails plugin for collective intelligence, acts_as_recommendable, that uses a C implementation for the heavy lifting and might be worth checking out as well.

Last year git took over in a lot of programming world thanks to an amazing killer app, Github. Getting familiar with git isn’t a big thing, but it’s worth it to have a little more shared language with your peers. Moving from CVS to Subversion wasn’t something that most people did overnight, but after the switch no one ever went back. Git is shaping up to be same, but has a long road of public support and client side tools ahead of it before it makes it that far. People that like git, really really like git. I’m ashamed to say I’ve made improvements to a few projects I’ve cloned from Github, but never got around to branching and committing my changes (or writing tests for my changes of course). This is something I’d love to get better at — more for fun and familiarity with git than anything else. That’s the goal for this year — work on projects that are fun!

There comes a time in everyones life when they have to setup a linux computer from scratch. OK, so maybe not, but if you’re a web programmer you might well be faced with the problem of putting out a colocated server, just a VPS where you have full control over the box. With full control comes full responsibility though, so I hope you’re willing to learn a little on your feet.

For the new rails site I’m putting together, I decided on Slicehost. Why Slicehost? For cheap, respectable rails hosting with a load of support it just kept coming up. You have all the default features — the ability to upgrade your account at any time, a variety of operating systems. For someone just getting started, you might also want the ability to rebuild your system at anytime, which is possible from the Slicehost web panel. If you decide you want to switch from Ubuntu 7 to 8, or from CentOS to Fedora, it’s just a matter of selecting the option (well, and then rebuilding your slice from scratch). You can also restart of course.

What really sold me was the wealth of information available by other Slicehost members. There’s a wiki with some details but the technical article repository is a fantastic source of knowledge for getting a fresh install up and running for just about anything you’d want.

I’m still in the process of getting this setup, but I’ll post about my experience at Slicehost more as I use them. If you’re looking for a quick start with Rails from scratch, whether you’re experienced in setting a box from scratch or not, Slicehost offers some very useful tools.

One of the things that’s most annoying when making sites that rely on external data is keeping them in sync. Although this usually means importing some sample data during development, eventually you’ll have to do it the right way. This evening I hit that point on a project where I needed to get a list of all arcade games from GameFAQs. Keeping these updated manually is completely out of the question, especially considering there’s over 3500 of them.

I decided to turn to Hpricot, which I’d heard about on Peepcode as well a number of other blogs. Hpricot is a very simple HTML parser for ruby.

To get started, just install the hpricot gem…

$ gem install hpricot

After that you can require it in your controller and go wild. I needed to get a list of all arcade games available on GameFaqs for this, so that means hitting 27 different pages and parsing the results (26 letters + all numbers). My Games table is extremely simple at this point with just an ID, name and gamefaqs_id. Since all I really need is to update my local games table with the data from GameFaqs, I also want to make sure I don’t insert duplicate records. One thing to note though: GameFaqs has multiple names for the same game. You might pull back 3 different games with a specific id. In my case I’m just using the first one I find, but you could switch this up easily enough.

So where’s the code?

require 'hpricot'
require 'open-uri'

class GatewayController < ApplicationController
  before_filter :check_administrator_role

  def update_games
    letters = ('a'..'z').to_a << '0'
    letters.each do |letter|
      page = Hpricot(open('http://www.gamefaqs.com/coinop/arcade/list_'+letter+'.html'))
      page.search( "//div#container/div#content/div#sky_col_wrap/div#main_col_wrap/div#main_col/div[@class='pod']/div[@class='body']/table/tr" ).each do |g|
        a = g.search( "//td:first/a").first
        name = a.inner_html
        link = a['href']
        gamefaqs_id = link.match(/[0-9]+/)[0]

        # Create this game if it doesn't exist
        if !Game.find_by_gamefaqs_id(gamefaqs_id, :select => 'true')
          Game.create(:name => name, :gamefaqs_id => gamefaqs_id)
        end
      end
    end
    render :template => "games/update"
  end
end

Not too bad for 29 lines in Rails. The letters variable contains all possible endings for the URL, with a pair of loops to do the work. The main work goes on in the page.search() part, which generates an array of td elements containing the information we need. From this you can grab the a element and then the gamefas_id and game name.