8 Awesome Uses for DropBox
Although only a very small part of my backup strategies post dealt with Dropbox, after using it for a second week I’ve fallen for it. Sure, it’s useful for syncing commonly used files between computers, and there’s many more files that it would be useful to have revision history on, but there’s more to it than just sharing files. With that in mind I’ve been on a rampage to use it for simplifying my life as much as possible. Here’s a few tricks I’ve found that help make things easier.
Backup Folders Not in Dropbox
Right now Dropbox installs itself into a pre-selected location, then everything in that folder is archived in Dropbox. Syncing outside of the Dropbox folder is on the todo list, but for now there’s a workaround to do this. Create a symbolic link within the Dropbox folder pointing to the folder you want to archive. For instance, lets say within Dropbox you want your Documents folder. To do this you’d create a sym link for this.
ln -s ~/Documents/ ~/Dropbox/Documents
Dropbox should start syncing right away, and will sync from here on out whenever. Anything that goes into your ~/Documents folder will be backed up. One very important thing to note is that if you delete something from the ~/Dropbox/Documents folder on another computer, it will be removed from your ~/Documents folder on the computer with the symlink, so keep that in mind.
Syncing Adium Log Files Between Macs
So you’re looking through chat logs for a conversation you know you had, but you can’t seem to find it. Turns out it was on a a different computer. Has this ever happened to you? There’s a convenient way of syncing Adium log files (or any other log files) with Dropbox. Adium doesn’t allow you to change your Log directory, but there’s no reason why you can’t make the default location a smylink to somewhere else. I have a ~/Dropbox/Sync directory where I store any settings/items that I want to keep in Sync between computers, so this is the perfect place for it. I created a ~/Dropbox/Sync/chat folder, quit Adium and dragged my existing Adium logs folder into Dropbox (from /Users/adam/Library/Application Support/Adium 2.0/Users/Default/Logs). All you have to do is point a folder at your Dropbox and you’re good to go! In my case, the command was:
ln -s ~/Dropbox/Sync/chat/logs/ "/Users/adam/Library/Application Support/Adium 2.0/Users/Default/Logs"
Just setup multiple computers with this same path and you should have a single repository for all your chat logs. I’m not too sure how this works with conflict resolution, but unless you’re chatting to the same person on multiple computers at the same time you should be good. If you’ve tried that though, I’d be interested to see how it works.
Backup configuration
This was mentioned earlier, but having a shared space for your backup scripts makes life easier. Dropbox is amazing, but the Dropbox application does have the ability to delete files from your computer. Lets say the worst happens and they delete your account, causing your Dropbox to be deleted on all your computers — would you mind? If you have your ~/Documents and other folders linked, this might be a huge blow to lose unless you’re backing it up somewhere. This same doomsday scenerio could happen if someone hacks into your Dropbox account and removes your files from the web interface. You’ll probably be able to restore them in that case, but then you’re relying on Dropbox to fix the Dropbox problem.
A more secure way of handling this might be to have a full copy of your Dropbox somewhere locally. With Dropbox maxing out around 55GB, making a full copy of this on some HD you have laying around isn’t a bad idea. Automating the process and backing up your Dropbox nightly is even better though! So what would something like this look like.
rsync -rpLtgoDE --delete ~/Dropbox /Volumes/Drobo/Users/adam/current
This rsync command will copy everything from my Dropbox to a local directory, in this case on a Drobo. This follow all symlinks and copy the contents of them, rather than copying the symlinks themselves. Try saving this one line in a file, backup.command. You should be able to run it by double clicking or from terminal. I store this file in ~/Dropbox/Sync/scripts/backup.command so it can be edited from anywhere.
Version your crontab
You don’t want to run that backup script manually everyday do you? To save time (and guarantee a good backup strategy), you’ll want to have this automatically run. I’ll recap real quick how to set this up. Just open up terminal on the computer you’ll be automating the backup. You can call things on a scheduling using cron, which can be opened by entering crontab -e. I’ve decided to run the above script every day, and backup my list of cron jobs every day.
@daily crontab -l > ~/Dropbox/Sync/scripts/crontab.txt
30 3 * * * sh ~/Dropbox/Sync/scripts/backup.command
The first line will save the crontab contents in the ~/Dropbox/Sync/scripts/crontab.txt file. The @daily parts means this’ll happen every night at midnight. The second line is scheduled to run every night at 3:30am as well, but usually takes a little longer. Rsync isn’t the fastest in the world, but it’ll get the job done — usually in under a minute for me.
Sync Your Things Tasks
For a while during the beta, Things was my GTD/todo list of choice. The interface has a level of polish that few apps obtain — and even fewer in the GTD relm where complexity is the norm. One downside of Things is that it doesn’t use MobileMe syncing like OmniFocus, it’s competitor. You can get around this and use your Dropbox for syncing in the same way as Adium though. Just close Things and move your Things database files folder to your Dropbox on one computer — I threw it in ~/Dropbox/Sync/Things. Create a symlink from where it used to be to the new Dropbox location and you should be good! You’ll need to do this same step on your other computer(s), but without the copying the file to Dropbox step. This does have a caviat though — you must close Things each time you switch computers. Unlike Adium, when there’s a conflict, well it’s very bad. Your Things database is a single XML file that cannot be merged. So be sure to always close Things when you switch computers to be safe. If anything goes wrong though, you can always sign into the Dropbox web interface and roll the file back.
Or you could just use Remember the Milk and not have to worry about syncing.
Host a Local Wiki
TiddlyWiki is a Wiki unlike any other I’ve heard of. It has all the storage characteristics of a normal wiki, but exists as a single html file that edits. Just copy it to your Dropbox, open it up in Firefox (you are using FireFox right?) and start making changes. Upon saving it’ll update the file in Dropbox. As with a few other tips, you probably shouldn’t have TiddlyWiki open on another when saving. If you’ve jumped on the TiddlyWiki bandwagon but you’re using a USB drive to transfer your Wiki around, Dropbox is a simple upgrade to save some time.
Sync your Passwords
I don’t personally use any sort of password manager, but LifeHacker has a good article on How to Use Dropbox as the Ultimate Password Syncer. According to the Lifehacker article, changes are pulled in right away, even when using multiple computers. This is because 1Password, the password manager used in the article, stores your data in lots of little files, making it ideal for minimal conflicts.
Sync your Firefox Bookmarks
Firefox bookmarks are another one that seem like a no-brainer to use Dropbox for. There are so many different bookmark sync services out there that can do this, but why use another service when you’re already using Dropbox? The basics for this are just copying your firefox profile to Dropbox, then starting Firefox with the -p option which allows you to specify a profile path. Just follow the directions on how to Sync Firefox Bookmarks from the Dropbox Wiki and you should be good. This tip doesn’t use symlinks like a number of others, making it a little easier to setup.
Join Dropbox
Dropbox provides 2gb of storage out of the gate, but if you signup with a referral link, you’ll get an extra 250mb of space (and I’ll get an extra 250mb too)! The desktop app works on Mac, PC and Linux with a very simple install on all of them. If you join up and have any other tips or tricks, feel free to share them here!
Building the Ultimate Media Center, Part 3: Backup
Describing what I’ve been working on so far as a media center may not be quite sufficient. Now a days term “home server” seems to do it a bit more justice, and if a home computer is on full time you’ll certainly want to get the most out of it. If you’ve ever wanted to do backups within your home, you might’ve looked into Windows Home Server or Apples Time Capsule / Time Machine setups, but neither is quite what I was looking for. First off, I don’t really want a Windows box in (mostly) mac home, so that’s out. Time Machine alone doesn’t do everything I have in mind, although it does play an important part. I’ve had a lot of backup plans in the past, but none were quite “perfect”, which got me wondering what a perfect backup plan would look like.
What do you want in backup plan?
Just like in creating software, when setting up a backup plan it’s important to know your requirements ahead of time. Otherwise what are you working towards? So what do I want this backup plan to be able to do…
- Make it so that there is no single point of failure for any important data
- Have on site and off site backups of everything
- Backup and sync core data between macs (settings, address book, calendar, keychain, etc)
- Some kind revision system for all text files. Something like git or svn where you can rollback changes
- A local redundant hard drive array to guarantee no data loss if a single drive fails
- Backup all core documents on various time intervals (daily, weekly, monthly). That way if I mess up a file and don’t realize it for a week I’ll be able to grab the version a month ago.
Settings, Calendars, Address Book and more
If you have multiple Macs Mobile Me is a good choice for easy syncing between macs. The system has had a lot of problems, and continues to struggle with performance. iDisk, the webdav interface to your MobileMe account is painfully slow, and continues to be. Just cleaning out a few hundred empty folders was enough to stall it earlier. The online image gallery MobileMe has is one of the better one’s seen, and if you use iPhoto it’s a convenient photo storage location.
Just so that all my calendars and contacts aren’t confined to a single system (the Mac ecosystem), I use Spanning Sync to keep my Google Calendars and Google Contacts in check with iCal and Address Book. My girlfriend and I share calendars with each other on Google, and this way any changes she makes show up on my iCal, and any I make on iCal show up on hers. Also, whenever I get Google Calendar invites to my email address, they show up on Google Calendar then get syncd back to iCal. 2 way syncing is the way to go!
Complete Backups With Time Machine
Time Capsule is powerful and dead simple for full system backups. The way it works is simple: all macs on a network will have Time Machine (a program that ships with Leopard) enabled and set to use the Time Capsule hardware. Every hour or so, changes made to the core and specific folders will be pushed from the client computer (in this case my laptop) to the Time Capsule. You can specify which folders to watch for this as well. You might not want it watching your Downloads folder or others that are similarly volatile, but that’s easy to change. The first time you back up everything to Time Capsule you’ll want a hardline to it rather than doing it over a home network (even if it’s wireless N). Transferring your entire computer initially could take a full day if not, so be sure to plug it in. The hourly backups after that work wirelessly though, as they are much much smaller.
Apple’s Airport Extreme has the ability to add a remote hard drive which can serve as a Time Machine drive as well. The nice part about this is that you can use an old external hard drive you have, or choose any drive out there. Leopard didn’t originally allow you to sync with drives like this, but in one of the recent updates it was enabled. For me, after the initial backup the incremental backups are very quick if not much has changed.
Running TimeMachine for the first time is a trippy experience. You “go back in time” to the point where you want to see a file, then you can restore it. This covers a few of the requirements such as versioned files locally and full system backups with the ability to restore on any Mac.
Backups to a local RAID (or Drobo)
Finally, the first step that the Media Center can help with. Whether you go with RAID, a Drobo or something else, the backup procedure to it will be about the same.
The easiest way I’ve found to do backups to this is with rsync and crontab. Here’s how it works: Your media server will have a series of scripts setup to run daily, weekly and monthly. These scripts will do all the heavy lifting of the backup — grabbing files from every computer on your network you want to backup and moving them to the redundant array.
So firstoff the media server should be setup to backup everything from the computers on the network. That’s easy actually. On each computer you want to back files up from, go into System Preferences > Sharing and enable “Remote Login”. This will allow the media center to ssh into each laptop. Backing up using rsync can be done by mapping drives in Finder and using those paths (/Volumes/adam), or by using SSH. With SSH you don’t have to map the drive beforehand, so it is the preferred method.
On the media center, open up a text file. This will contain everything you want to copy from your client computers, and the path (on your media center) that you want to store them in.
rsync -aE --delete adam@link.local:Desktop /Volumes/Drobo/Users/adam/current
rsync -aE --delete adam@link.local:Documents /Volumes/Drobo/Users/adam/current
rsync -aE --delete adam@link.local:Movies /Volumes/Drobo/Users/adam/current
rsync -aE --delete adam@link.local:Pictures /Volumes/Drobo/Users/adam/current
rsync -aE --delete adam@link.local:Processing /Volumes/Drobo/Users/adam/current
rsync -aE --delete adam@link.local:Sites /Volumes/Drobo/Users/adam/current
Note that the destination (the 2nd path) is the same. That’s just because it’ll actually store the documents folder in “/Volumes/Drobo/Users/adam/current/Documents”. The -a option specifies “Archive”, which is actually shorthand for a few other options. It will recusively copy all directories, copy symlinks, preserve file permissions, preserve owner and groups, preserve file times preserve special files. -E preserves executability, while –delete deletes extraneous files — like those that were deleted. Save this file somewhere on the media center as a .command file. I kept it simple - backup.command. You should be able to run this script by just double clicking on it now. But there is a one major problem — authentication.
Setting up SSH keys
By setting up SSH Keys you can automate the authentication from the Media Server to any clients you want to backup. The way you do this is very easy. On the Media server, run the command ssh-keygen -t dsa. Go with the defaults, but choose a passphrase you can remember. Copy this file from the Media server to the computer you want to backup FROM. Since you should already have SSH setup, something like this should do the job.
llink:~ adam$ scp ~/.ssh/id_dsa.pub adam@link.local:~/.ssh/authorized_keys2
If you already have an authorized_keys2 file, you’ll want to add the contents of id_dsa.pub to it instead of replacing it. After that’s updated, try SSHing to that client from the Media Server. You should get prompted for the passphrase (a mac prompt, not a terminal prompt). After you enter it and save it for later you should be able to SSH without any password going forward.
Setting it up to run automatically
Not worrying about backups is the whole point of this entire system, so running it automatically is essential. Luckily Unix systems have an easy way of doing this using cron jobs. To open up the list of cron jobs, just enter crontab -e in a terminal window. This will open up the cron file using VI. Editing in VI can be a bit annoying without knowing the comamnds. To edit it you don’t need to know much VI though. Just hit “i” to go into insert mode, where you can start typing. Each line you enter is a single scheduled cron job with a very specific format. For example, here’s a line to run a script every day at 3:30am:
30 3 * * * sh ~/Dropbox/scripts/backup.command
The “30 3 * * *” part deals with when it should be run, while the part after that is what to run. Here’s what those first 5 options translate to:
- 30 - minute to run job on
- 3 - hour to run job on
- * - Day of the month to run job on
- * - Which month to run job on
- * - Which day of the week to run the job on
It’s a bit tricky to remember at first, but it’s a very simple scheduling system. For example, if you want to run something on sunday (the 0th day of the week) at 5:30 am, the parameters would be “30 5 * * 0 …”. Here’s my full crontab that I’m using (some of these scripts I haven’t discussed yet, but we’ll get to later).
30 3 * * * sh ~/Dropbox/scripts/backup.command
00 5 * * 1 sh ~/Dropbox/scripts/weekly.command
00 6 1 * * sh ~/Dropbox/scripts/monthly.command
30 6 * * 1 sh ~/Dropbox/scripts/external.backup.command
The weekly and monthly scripts will just take a snapshot every month rather than every day. Nothing much to those scripts — just removing the last update and copying a snapshot. The monthly script is exactly the same but with “weekly” changed to “monthly”. The point of this is that if anything big goes wrong, you’ll have a backup locally of the original.
rm -rf /Volumes/Drobo/Users/adam/weekly/
cp -pR /Volumes/Drobo/Users/adam/current/ /Volumes/Drobo/Users/adam/weekly/
Versioned changes with Dropbox
DropBox is probably my favorite part of all this. After signing up for DropBox, you install a small program on each computer you want to use it with. It’ll create a DropBox folder at a location you pick (I went with the default ~/Dropbox). Anything you add to this folder will get synced up to the Dropbox service. What’s extra-useful about this is that all files will be versioned. I’m storing my Documents and Sites folders in here and backing up daily. Even if I haven’t committed site changes for something I’m working on, it’ll still have a daily revision of changes.
I have a few other commands in my backup.command file that copy things over to a DropBox folder on the Media Server.
rsync -aE --delete /Volumes/Drobo/Users/adam/current/Documents ~/Dropbox
rsync -aE --delete /Volumes/Drobo/Users/adam/current/Sites ~/Dropbox
One really nice part about Dropbox’s desktop client is that you can limit the upload speed. With RoadRunner my upstream is capped to about 70k/s when it’s in a good mood, and maxing it out will usually slow down any downloads. Being able to cap it to 10k/s is very useful in my situation.
Backing up your backup scripts
You’ll notice that my backup scripts where in my Dropbox: ~/Dropbox/scripts/backup.command. Not only does this mean that you’ve backed up your scripts (always a good practice), but you can change your backup procedure from anywhere you can access Dropbox! This is so simple it’s one of the most effortless things to setup. You can also copy the contents of your crontab file
Add this line to your crontab:
@daily crontab -l > ~/Dropbox/crontab.txt
“@daily” is shorthand for “everyday at midnight”. This is a nice way of archiving any changes you make to your crontab. With the other backup scripts you can make changes on any computer and the Media Center will pull them in thanks to DropBox. But for the crontab you’ll actually want to make changes directly to the Media Center using the “crontab -e” command.
External backups of everything else
DropBox is free up to 2GB making it excellent for documents, but not very realistic for media. They do offer a plan up to 50GB for $99/yr, which blows MobileMe’s 20GB plan out of the water. Most the use I get from MobileMe is in syncing computers though, so I’ll keep it going for that. For now I’ll stick with MobileMe for my larger media backup as well, although I can tell you now I’m not exactly happy about it.
One of the downsides of MobileMe is that it doesn’t automatically mount itself on startup. Normally you have to click on it in Finder to mount the drive. Only then it’ll show in “/Volumes”. You can open up Script Editor and create a small app to do this though. Just copy this and change it to use your username and password.
tell application "Finder"
mount volume "http://idisk.mac.com/myusername/" as user name "myusername" with password "mypassword"
end tell
Save it as file format “Application” with “Run Only” checked, but don’t check “Stay Open” or “Startup Screen”. Save this application somewhere then add it to your startup items.
The last part is setting up the “remote.command” script listed earlier. This will do all the work of copying everything remotely, with the exception of whatever uses Dropbox.
rsync -aE --delete /Volumes/Drobo/Users/adam/current/Documents /Volumes/myusername
rsync -aE --delete /Volumes/Drobo/Users/adam/current/Backup /Volumes/myusername
rsync -aE --delete /Volumes/Drobo/Users/adam/current/Desktop /Volumes/myusername/Documents
rsync -aE --delete /Volumes/Drobo/Users/adam/current/Movies /Volumes/myusername
rsync -aE --delete /Volumes/Drobo/Users/adam/current/Pictures /Volumes/myusername
rsync -aE --delete /Volumes/Drobo/Users/adam/current/Sites /Volumes/myusername
rsync -aE --delete /Volumes/Drobo/Books /Volumes/myusername/Documents
This just runs weekly everything here doesn’t change that much. The Pictures directly is the main one that’ll be updated, as it’s a single iPhoto file that changes often.
Woh, that’s a lot. Did we miss anything?
Well, two things were skipped — Music and Videos. I have a decent sized music collection, and I tend to archive my DVDs for use with Plex. Backing up 2TB of data remotely when you’re limited to 70k/s upload speeds isn’t exactly possible. What I am doing is storing all my media on a Drobo which protects it against a single hard drive failure (maybe more, depending on when the failures occur). My TimeMachine drive is actually a 500gb drive split - 200 for Time Machine and the rest for media backup. Every night the Media Center pushes all music from the Drobo onto this external drive. At least that way all my music is protected in case the Drobo itself fails. There’s no external backup for music, but I’m OK with that. If I was a little more paranoid I might archive weekly and leave the drive at work or something, but that’s where I draw the line.
That leaves one gaping hole - the movie collection this system was for in the first place! For now I’m content with being protected against a single HD failure with the Drobo, but if anyone has any suggestions on how to back this up I’m all ears. Also if you think of anything else I missed, or just something I could do better then please leave a comment!
Did this help?
If this article helped you, please sign up for DropBox! DropBox was in beta for a while, but now it’s open and anyone can sign up. You get 2GB of space just for signing up. They have the added bonus of increasing your space by 250mb for everyone you refer (up to 5GB total), so by signing up you’ll give me more space to experiment with. It’s easily the best backup service I’ve used, so I’m looking forward to seeing how it grows.
Building the Ultimate Media Center, Part 2: Plex
If there’s a killer app out there in the media center world, it’s Plex. Built on the long running XBMC, Plex is a fork of the project optimized to run on the Mac. XBMC has been around since the first Xbox, making one of the most established media center platforms out there. It’s long outgrown the console however, and now serves as the core for projects like Plex and Boxee. It was also featured on a recent FLOSS Weekly episode which is worth a listen. With lots of recent talk about Boxee, I thought I’d add Plex to the conversation with a little talk on why I think it’s the best media center software available.
Media Management
Like other projects based on XBMC, Plex has built in capacity for managing your media collection. This enables Plex to have an internal listing of what movies you have available in a much nicer format than the typical folder structure. Within Plex you do this by adding sources (in the form of folders containing files) and specifying what type of content resides there (movies, tv shows, music videos). Plex even allows for different sources to pull down the information on the video — from IMDB, OFDB, MovieMaze, TheTVDB, TV.com and many more. If you want to pull data from another site, it can be added using Python, so if a better movie database comes out, it’s just a matter of writing an adapter for it. Plex uses the file name or folder name to look it up, so it’s important to name your files correctly and add the year the movie/tv show was released. You can do this manually from within Plex as well, but it’ll find your media on IMDB if your files are named correctly.
One issue that always come up when crawling a media collection is varying file format and folder structure. Chances are not all of your videos will be in the same format. Some might be single file avis, mkvs, or they might be in multiple small files (disc1, disc2), or even in DVD rips (in a VIDEO_TS folder) or Blu-ray rips. If you follow Plex’s conventions, all of these can be used and you won’t even have to concern yourself with it later on. Here’s a sample folder structure for this setup:
/WALL.E (2008).mkv
/Wanted (2008)/any_file_name_disca.avi
/Wanted (2008)/any_file_name_discb.avi
/Waltz with Bashir (2008)/VIDEO_TS/* (DVD contents here)
To Plex all these will be listed the same, with IMDB show descriptions, artwork, genre listing and even actor/actresses if you want. When you play “Wanted (2008)”, it’ll queue up all files in the folder and play them in alphabetical order. I sometimes notice a short skip when it transitions to the second video, but it may not be noticeable all the time. Playing the DVD file is very similar as well. Upon selecting it to play, it’ll open you up to the DVD title menu, the same menu you’d go to if you were using the DVD itself! I’ve found there is a bit more overhead to using DVDs, as it has to put the entire menu into memory, rather than streaming the video from the start. This is really only an issue if you’re playing your media off a NAS. DVDs will still work, but they won’t be as snappy at starting.
TV shows are categorized by season number, but aside from that they work the same. Each season has it’s own folder, and within that each episode is named accordingly. You can choose to add in descriptions, or leave them blank. You can also combine episodes if multiple ones are in the same file.
/Arrested Development (2003)/Season 1/Arrested Development (2003) - S01E01 - pilot.avi
/Arrested Development (2003)/Season 1/Arrested Development (2003) - S01E02.avi
/Arrested Development (2003)/Season 1/Arrested Development (2003) - S01E03-04.avi
Like with movies, TV DVDs also work here, but you’d want to check the Plex Wiki for instructions on setting that up.
In addition to downloading cast, banners and artwork from movies and TV shows, Plex will also download TV theme songs. As you’re browsing through your TV shows you’ll hear a little snippet that Plex was able to grab for each series. One very relivieving thing to note is that Plex won’t touch your media folders or add new files to them at all. You know how you can set iTunes to manage your media library? There’s no similar setting on Plex. You have full control of your files, and Plex won’t alter them in any way. The additional files that Plex downloads will be stored your Application Support folder on the computer running Plex.
Library Searching
After all you files have been discovered by Plex that’s when the fun begins. Just browsing your movies and tv shows is a visual treat, and the Plex team has done an excellent job of making the experience similar regardless of what type of artwork is available. You can filter Movies or TV shows by year, imdb rating, director, genre, actor/actress, and even whether or not you’ve seen them or not. The ability to list what you haven’t seen, and even hide descriptions for new items, is extremely useful, serving as a bookmark for where you left off in a series. The Favorites system that was recently introduced gives one click access to some commonly used movies, shows or even apps. I’m currently re-watching a TV Series, so having in Favorites saves time. If you’re using a Harmony Remote, you can even setup a button to open favorite directly.
For instance, if you wanted to watch something with Christian Bale in it, you could filter your entire movie library to just movies he was in. Plex keeps an internal actor/actress list if you specify it when adding a folder (a source).
3rd Party Plugins
With a little background in Python, anyone can create plugins to pull video from other sites. Since this feature was added to the .7 release of Plex a few months ago, there’s been a huge flood of these apps making Plex better and better. Some of the more popular include Hulu, Joost, MTV, Comedy Central, South Park, Ted, Escapist Magazine (Zero Punctuation!), Youtube, Apple Trailers, Pandora, CNN, The Onion, Daily Show, Colbert Report — the list goes on and on. The framework for building these apps allows full screen playback and scaling of flash videos, which just about all of these are. Need to click on a close button within the video so an ad doesn’t show? The plugin developer can even do that. These plugins have quickly become one of the most useful features in Plex for expanding it into new territory.
External Applications
You can setup any application to launch from within Plex. When this application is closed, Plex is reopened. This seems like a throwaway feature at first glance — why would you need to open up something else when Plex can do so much? But from a media center standpoint it helps having one more way of extending the livingroom feel without having to go back to the mouse/keyboard setup if you get it right. One application that is a prime candidate for launching in this manner is Emulaunch, and emulator launching program. Emulaunch fits in perfectly — running full screen and launching ROMs full screen. Emulaunch isn’t an actual emulator program, for that you’ll need something like SNES9x. Emulaunch will run the rom using whatever program it’s associated to run in (whatever program would run it when you double click). Without much trouble at all I was playing some Zelda a Link to the Past launched from Plex. If you were playing a specific game repeatedly, you could also set it up to run directly from Plex by launching the emulator you’d want to use and passing the game in as an argument. Since it’s possible to use Xbox 360 controllers (both wired, or wireless if you buy the controller adapter), this could easily become a new center for gaming.
Remote Control
I mentioned this last week, but it bears repeating — Plex has excellent support for apple remotes and Harmony remotes. I’d been using an Apple remote for the past year without any trouble actually. It’s so simple to use that you could give it to someone who’s never used Plex before and they’d be able to navigate around, play videos and even browse Hulu. To be fair, you can do all this with an Apple remote from Boxee as well. One of the advantages of Plex though, is that by holding down the menu button, you can get a list of advanced features which become important in managing a media collection.
You could be perfectly happy running Plex with an Apple remote for a long time. I still plan to use it when people are over because of how easily new people take to it. For general use though, I love having a universal remote — a single remote for all my devices. That’s where the Harmony One comes in. With the One, and other higher end Harmony remotes, you set all devices through the bundled software. I was shocked when I went to add my Mac Mini here only to find Plex as a listed device. This sets up the remote with all of Plex’s default keys, getting your running with Plex right away. There is also a screen from within Plex to set this up which can be set for the Harmony line of remotes.
If you're wanting to learn more about Plex, it's worth watching the Plex Screencasts to get the hang of what it can do.
Building the Ultimate Media Center, Part 1: Hardware
If you’re a geek, chances are you’ve dreamed of having an amazing media setup that gives you a movie theater at home feel while providing all the options to extend it with your media. Well, thanks to the recent release of the Apple Mac mini, hardware and software (specifically Plex) are at a great point right now to do just about everything you could want on a single system. Since this past Tuesday when the Mac Mini came out I’ve been setting a system for this purpose. Hopefully my research can save someone else a little bit of time.
So what do I want to be able to do?
- Watch TV
- Watch DVDs/BluRay
- Watch downloaded/ripped videos of any format (avi, mkv, DVDs, hd video)
- Stream videos off the network (so we don’t have to have a NAS in the living room)
- Have some sort of way to track ratings, seen/not seen, imdb information, genres, cast, and movie/tv show information automatically
- Ability to watch 1080p video with minimal artifacts (if any)
- Play video games through an emulator
- Use a wireless Xbox 360 controller for games
- Use a single remote control for media center, dvd, blueray, cable
- Full surround sound (7.1, or 5.1 at least)
- Watch videos from a variety of web-only sources (Hulu, Joost, TED, others)
All these things and more will be possible through this system, though it will take some setting up and experimentation.
Uhh, what don’t you want to do?
This is just as important to know when it can impact the core hardware. There is one thing left out of here - the ability to record from cable on a computer (a DVR). This is one feature that has always sounded cool in theory. MythTV looks impressive, but the idea of having something I setup control whether I see my favorite shows or not is one big variable. The hardware encoding and conversion don’t interest me too much at this point, and since I don’t plan on archiving media indefinitely, there’s no need for digital copy. Likewise for the ability to watch TV through a media center. I have a DVR setup anyway, so I don’t mind switching from the media center over to it for TV. Someday maybe these could be one — someday soon actually.
Hardware Matters
This system will be running Plex at it’s core, so it will have to be a Mac. Since it’ll be in the living room, and because we’re aiming for lowest power consumption the Mini is the perfect choice. Like all Macs it also has Bluetooth and IR capabilities so we can go completely wireless. Some of the other hardware will be completely dependent on what you want as the core of your system.
Generic Hardware
The TV, Cable Box/DVR Box and Receiver could really be anything for this. There are a couple of requirements, but nothing that even middle of the line items don’t have anymore. The TV should have at least one HDMI input, and doesn’t hurt to have a Component IN. The receiver should also have inputs for at least your cable box and a HDMI connector. It should also have at least two optical audio (toslink) connections. The receiver should also have 5.1 surround (2 front, 1 center, 2 side speakers), or 7.1 surround (5.1 + 2 behind). 6.1 Surround will also work (5.1 + 1 behind). Hopefully your cable box will have component or HDMI output as well as toslink for audio for full surround and HD video. The speakers and speaker cable can be whatever; there are better places than here for information on that.
Media Computer
Ah yes, this is the core of this new system isn’t it. This should be one of the new Mac Mini’s for a few major reasons. The CPU is slightly faster 2.0/2.26Ghz rather than 1.83Ghz. It also has a faster frontside bus. The graphics card is wildly improved from 64mb to 128/256mb. Old models only had 1GB of memory and it was not officially expandable. Now though, models start at 1GB/2GB and go up to 4GB (through 2×2GB chips).
That difference in the graphics card looks like it would make a world of difference when playing videos, but it’s just not the case. There is no hardware assisted decoding when playing media files so video ram doesn’t help out one bit. Adding more RAM is what makes all the difference actually. Maxing it out at 4GB is only an extra $50, so it’s well worth it. Plex does an amazing job of rendering videos that no other video software can play, but it requires some serious RAM to make it happen.
Audio
Only a few cables are needed to get things going. To output sound you’ll need a toslink cable as well as a mini to toslink adapter. The mini uses special combination analog/optical-digital audio jacks, so with toslink cables you can get full surround sound.
Video
The Mini has a mini-DVI port and comes with a mini-DVI to DVI-D connector, which is what we’ll use for video. Your TV might have a DVI input, but the quality won’t be the same as using HDMI. I found that my TV wouldn’t go to full 1080p with the DVI connector attached, but with HDMI it would. Luckily there’s plenty of DVI-D to HDMI cables on Amazon. You shouldn’t buy HDMI cables locally at all actually. Best Buy sells them for 10x the price on Amazon (or more).
Computer Devices
I already had a Apple wireless keyboard and a wireless Might Mouse, which do the trick. There’s no way I want cables running from my media center all the way to the couch each time I need to use it, so wireless is essential — as are rechargeable batteries. The mini found both of these devices when it was first started, which was an added bonus.
Gaming Devices
The Xbox 360 Controller is absolutely amazing. It feels more natural in your hands than any other controller, has a great combination of controls that allow it work with just about any previous system. Keeping with the previous trend, we also want this to be wireless. Using a wireless controller should be a breeze these days, but Microsoft decided to use a proprietary RF format so no one else could use it. Well, not without some hoops anyways. If you have a USB Xbox 360 controller it should work once you’ve setup some drivers for it. To use the wireless controller though, you also have to buy a Xbox 360 Wireless Gaming Receiver for Windows. This is a little USB device that allows you to pair your controllers with a computer. By default it doesn’t work on the Mac, although thanks to the great work by some smart people, the controller works great. The driver adds a preference pane where you can test out if the controllers are working too, which is great for testing too.
Remote Control
Plex has amazing support for the Apple Remote. Although the Mac Mini doesn’t come with it, it’s a perfectly good remote for controlling just about anything within Plex. It doesn’t control the TV or receiver though, you’d still need multiple remote controls. That’s where a great remote like the Logitech Harmony One comes into play. In addition to working on all my devices (and just about anything out there), it comes with a special setting in for Plex! The One also has a series of macro buttons you can use to do multiple things at once. For instance, using Plex requires changing my TV to the HDMI input and changing the receiver to the PC input. Doing this and going back to Cable is a one click process. There’s a few more bells and whistles you can customize for Harmony within Plex as well, once you’re up and running. For instance, you can add one touch buttons for things like “Watch Hulu”, or “My Favorites” or even “Watch The Daily Show on Hulu”. Not bad for a single click. You can also customize the buttons per context as well. So the number 0 might do something different when you’re watching a movie than when you’re in a listing of movies. Lots of possibilities there.
That’s all there is to it
Getting everything up and running was surprisingly smooth. It’s all dependent on a very small set of core programs and some smart settings to keep memory usage low. If you’re wanting to see if your setup can handle HD video, try downloading the birds scene from Planet Earth and see how it performs. If it doesn’t even run (and chances are it might not), try downloading Plex and trying it form there. You can also benchmark within Plex by pressing ‘i’, which can serve as a good gauge of how many frames are being dropped. I’ll save that for the next entry on media software.
Adding a Lifestream to Wordpress
Sometimes a new service comes around that you know is powerful but you have trouble finding an actual use for it. Two of those that I’ve recently been looking into are FriendFeed and Yahoo Pipes. By themselves they’re nothing, but once you have a goal in mind both become powerful allies.
Adding some kind of a “lifestream” seems like all the rage these days, so I decided to give it a try using FriendFeed. I have to admit they make this extremely easy. Creating a page via a single js call is about as easy as it gets, although if the page was going to be hit often it’d be a good idea to do some kind of caching. FriendFeed is nice enough to provide some css hooks into the generated HTML that help with the finishing touches.
There is one type of service that wasn’t available on FriendFeed that I wanted to add to it though — movie reviews. Most movie sites are still walled gardens, despite the outcry from the open web community. IMDB has absolutely no API, although they do provide their data in txt files. That won’t help for getting your specific info though. FriedFeed does integrate with NetFlix, but it looks to be only for receiving queue notifications. Flixster seemed like the best choice for movie reviews, despite the awful interface of their website. Luckily you can do just about everything from Facebook. The problem is, Flixster doesn’t have any sort of API! Actually, no movie sites I’ve found provide so much as an RSS feed of your reviews (if you know of any, let me know). Enter Yahoo Pipes. Just a few weeks ago, someone created a Yahoo Pipe for transforming the XML that Flixster uses for it’s site widget into an RSS feed. Quite effective, and looks good (example). I still have my problems with Flixster, but at least it’s a start in integrating movie reviews.
The end result is a very basic lifestream page. There’s a lot more than can be done with this though. What if you were able to jump to a specific date and see activity? Or if there were check boxes to show/hide specific services? Not to mention all the styling you can do to it. Interested to see what people end up doing with their FriendFeed data.
New Languages for a New Year?
As some others have mentioned, it’s that time of year where most people with blogs make their public announcements on what languages they’ll be concentrating on for the upcoming year. Last year I think my top focus was Ruby on Rails and Javascript; so safe to say I’ve gotten a little experience in those. Things have taken a bit of a turn though, and I’m going to switch things up for 2009 and concentrate a little on my weaknesses and a little on new growth.
Weaknesses? Yeah, everyone has them. Those areas where when you have to implement something you bite your lip and find a way to get through it — usually with a 10ft poll. If they’re something that ties into your daily routine in some way, this is kind of task that can be holding you back from potentially getting more work done, or higher quality work.
One of my weaknesses has always been testing. Maybe I’ve been surrounded by too many testers, but its definitely something I could improve on. I wouldn’t consider myself fully proficient in a language to the point where I’d feel confident teaching someone else until I was proficient testing in it. Some languages don’t make this easy of course. Testing Javascript isn’t the most common thing, but it’s important, especially with such a Javascript heavy application as CloudShout. There are some great new tools like FireUnit for testing in Javascript too. Testing in Python, especially within Django has to have great support — as long as you actually do it. With Rails there’s a million different ways to test; but I’d like to concentrate mostly on mochs and stubs for a swiftly running test suite. No point in having a test suite that takes so long to run that no one has the time to run it after all.
There are a few other topics of interest for the new year. Things I’d love to look into and play around with at least a least a little. XMPP and especially bosh look extreme interesting to me, and I look forward to cramming together the little I’ve learned so far into an Adogo presentation in a little over a week.
Collective intelligence looks like another interesting topic that I’ve always wanted to dig a little deeper into. The Netflix prize looks an great way to experiment a little with it in some different language — Python, C and Erlang seem to be the most commonly used for it. There’s at least one active Rails plugin for collective intelligence, acts_as_recommendable, that uses a C implementation for the heavy lifting and might be worth checking out as well.
Last year git took over in a lot of programming world thanks to an amazing killer app, Github. Getting familiar with git isn’t a big thing, but it’s worth it to have a little more shared language with your peers. Moving from CVS to Subversion wasn’t something that most people did overnight, but after the switch no one ever went back. Git is shaping up to be same, but has a long road of public support and client side tools ahead of it before it makes it that far. People that like git, really really like git. I’m ashamed to say I’ve made improvements to a few projects I’ve cloned from Github, but never got around to branching and committing my changes (or writing tests for my changes of course). This is something I’d love to get better at — more for fun and familiarity with git than anything else. That’s the goal for this year — work on projects that are fun!
If Quicksilver isn’t working for you, you’re not using it enough
It’s taken me a while to jump on the Quicksilver bandwagon. If you’re a Mac user chances are you’ve run into QS as some point, as it seems to be one of the core apps that most hardcore mac users recommend. There are tons of great links on del.icio.us, posts by Merlin Mann and even screencasts. Safe to say it’s here to stay, and with it now open source and hosted on Google, it has a future ahead of it.
Explaining what Quicksilver is might best be left to the wikipedia entry, but to put it short Quicksilver is used to launch apps and control them using short, quick keystrokes. The hardest part about Quicksilver may just be finding out how can use it to make things you do faster. One of the best ways to handle this is to actively think about anything you do often and see if it could be done using Quicksilver. There’s a few Quicksilver tasks that I’ve recently started using that might be useful.
Add Items to a RememberTheMilk.com todo list
I’m a recent convert to RememberTheMilk and it’s related IPhone app, so linking these extremely common tasks was a top priority. There is a quicksilver plugin for remember the milk that makes this quick. Initially when the plugin is added, you’ll need to authorize it to work via the quicksilver preferences. After that it’ll import your task lists and make them available to add to directly. Putting them all together it looks like this.
Send tickets to a Lighthouse app project
There’s no magic plugins for this one — just the basics. Lighthouse allows you to create tickets by email for a specific project. Just about any bug tracking/project management app has a similar capability that can be invoked via Quicksilver. There’s a number of different ways to do this, but the basic one is to create an entry in Address Book titled something related to your app. It has the added bonus of being able to add a contact image that will show up in Quicksilver.
Get fast access to commonly used text
One feature of Quicksilver that gets a lot of coverage is the Shelf. It’s basically a way to access things fast. These “things” can be plain text, folders, applications — basically anything that shows up in quicksilver in the first pane. To enable the shelf, go to plugins in preferences and make sure the “Shelf module” is checked off. After that you can try putting something on the shelf to test it out. To do that just enter some text in the first pane, and then put on shelf
After you have at least one item on the shelf, you should be able to pull it up in the first pane using shelf. show contents is used to get a list of what’s actually on your shelf.
This Shelf > Show Contents combination gets old real fast, so you might want to consider moving this to a trigger. In Preferences > Triggers > Custom Triggers you can create a new keyboard command that jumps you straight into the shelf. I went with option+command for this. Only takes a few keystrokes!
What killer users for Quicksilver have you found? Or perhaps other uses for the shelf?
Cloudshout Alpha 2 Released!
Over at IZEA there are a lot of balls up in the air at any given time. Lately Dan and I have been concentrating on a relatively new creation — CloudShout. It fits in nicely with the blog focused atmosphere for IZEA, and we’re both pretty excited about it. It already has a lot going for it, and I hear what’s available now with it is just the tip of the iceberg. Either way it’s something new and interesting, that I think you’ll be hearing a lot more about in the months to come.
Cloudshout is, in the most basic terms, a widget that can be installed on any site that enables realtime communication across the interwebs. It provides a open way for new developers to create javascript applications that can be installed by any user or site using CloudShout. One of the best ways to see what’s possible is to check out some sample applications. Just click on application tab in the widget and it’ll show what’s installed on my blog. Here are some applications that are already available:
- A group chat application (chat has to be realtime of course)
- An app that pull recent tweets from the blog owner, and even tweets from other site visitors
- An app that pulls recent flickr photos from the blog owner, and other photos from site visitors
- Multiple games — like checkers
- A “Growl” like application that shows activity for the site (if you’re using a mac and don’t use Growl it’s worth checking out)
It’s important to note that any of these applications are updated immediately as new visitors come to the site. So lets say while you’re reading this blog post someone new comes to my site. They’ll show up immediately on the right, and you’ll be able to view tweets, or play checkers with them. Having realtime communication and the ability to send Javascript to other users really opens up a whole new world of application possibilities, so we’re thinking of, and hearing suggestions about new applications every day.
We Have the Technology
You can check for yourself how it’s all done, but it shouldn’t come as much of a surprise — ajax using script tag insertion. Since you can’t use XMLHttpRequest across domain (well until cross-site XmlHttpRequest becomes mainstream), it has to include script tags occasionally to get updates from the server. It does all of this by having the site owner include a single javascript file and creating a div with a specific ID. After that just hit the URL and you’ll see the widget.
The main hub itself and any applications you write have full access to JQuery, which I’m definitely falling for. We’re still looking into how to incorporate some kind of a library to take advantage of the huge amount of Jquery plugins and extensions that are available, but until then the basic jQuery should keep you busy.
Developing applications is easy, especially if you use Textmate. There’s a bundle available that allows editing application code locally and saving it to CloudShout. When developing you’ll get a URL to include on some sample page (usually stored locally) that will load the base widget and only your application. You can make changes and view them in this sandbox, then publish your application once it’s perfect. At that point one someone from IZEA will review it (no nasty hacks or bad code, etc) and then it’ll be available for anyone to install to their blog.
Oh there’s more?
At first some of this was sounding a bit like Google Site Widgets (which I heard about once then never again) — a way to enable some basic dynamic functionality. The difference is astounding though. CloudShout is realtime, open, extendable and just more badass in general. It’s also not specific to a single site. There’s quite a bit more to it, and for some you can watch Dans talk at BarcampTampa.
IzeaFest Wrap Up
Did you make it to IzeaFest last week? It was a fun experience, and a major source of motivation to do better as a blogger. As a general conference attendee, I was even surprised by how well everything turned out. Talks ran smoothly, things went well, they were broadcast live through Ustream, flickr and twitter feeds were buzzing throughout the event (and some up on monitors throughout the event) — just well done all around. The development team at Izea was working hard for quite a few product releases during IzeaFest as well, including some great new addons to SocialSpark and the public alpha of CloudShout (available on the Izea Blog if you want to check it out). I just started helping out on CloudShout about 2 weeks back, but I’m floored by the possibilities of it; and it’s been fun for someone as interested in Javascript as me.
One very nice thing about IzeaFest is that all the talks are available online! One of the talks I was most looking forward to didn’t let me down either — Merlin Mann from 43 Folders keynote on How to Blog. Deceptive title for a great talk. I dare anyone to listen to it and not want to start a blog right then and there. He has some other great talks out there like Inbox Zero and Getting Things Done.
Aside from that, Izea has a way of throwing some great night events. Thursday night they took over the lounge at the Grand Bohemian for an elegant night of great food and drinks. Friday and Saturday nights also had food and drink events down on Wall Street with tons of bloggers and advertisers in attendance. I still have yet to find out the details on the IzeaHunt (a photo scavenger hunt through downtown) from yesterday night, but I’m imagining it was crazy.
Orlando Events for this Week
If you’re in Orlando this week there’s a lot of cool stuff going on. Monday is our monthly Adogo meeting with a very interesting talk on Ajax by Brian LeGros. He’ll be going over some Ajax basics and move onto some “push” Ajax, a topic that’s extremely useful but rarely covered. A majority of ajax sites use polling to know when to retrieve data from the server, but with a push service, such as BlazeDS, you can cut out the polling altogether. It’s something I’ve been wanting to learn about for a while, so I’m looking forward to his talk. If you’re curious about the code, it’s available in the Adogo SVN repo, and a recording of the presentation will be available online shortly after.
Thursday is the monthly Orlando Ruby Users Group meeting, which this month will be taking a break from Ruby in favor of Seaside, a Smalltalk framework. There’s a famous quote (by DHH I believe) referencing Seaside: “I defy anyone to come up here and use any other framework to duplicate what we’re doing in Rails as quickly. Except Avi.” — Avi being the creator of Seaside. Although I haven’t messed with Seaside this looks like it’ll be an interesting talk. Don’t know if I’ll make it though, due to the BIG EVENT for the week.
Thursday night through Saturday night is IzeaFest, which is looking better and better. Aside from a fast growing list of attendees, there’s some exciting speakers including Merlin Mann from 43 Folders, Jeremy Shoemaker from ShoeMoney, John Chow a lots more. The scavenger hunt on Saturday night should make for some fun stories after the fact as well.
