What I use (3): Online sync and Backup

A while ago I wrote that I think that best way to access the right computing power at the right time is to have 3 different hardware solutions, I called them ‘Permanent Presence’ (a smart phone), Portable Solution (laptop) and Base Station (desktop/ server). At the time I noted that the downside to optimising the hardware solution is the need to synchronise information between the machines. Although there are various solutions for the large scale enterprise including Groove, I found that the solutions offered to a small business were clunky, slow and frequently simply failed to work.
The advantage that the large enterprises have, and probably the reason why the solutions work, is that someone has actually put some thought into designing an optimal solution. When you are not a storage/ backup professional the temptation is to just let a service provider sell you their latest toy and be happy with it. The problem with this is that you are no longer in control of meeting your requirements- you are reacting to someone else’s belief of what they think you need.
However, having someone do the thought for you is sometimes good and a couple of solutions have come along that allow the design of a fairly good solution. The tools are Dropbox and Mozy; both of which have been around a little while and both have won awards. So I am not some crazy evangelical here, and I am not even at the front of the wave; what I am proposing is that combining the best tools out their with some system design can yield an effective solution for virtually no cost.

So, as I was trying to come up with a better solution the first question that you need to ask is “What information do I need to keep synchronised between the machines and what information do I just need to copy to all my computers?” Examples of the first type of information include the documents you are working on, examples of the second time include photographs or MP3 files. Because what you actually need for photos is not a synchronisation service, and you don’t even want an interactive backup service- most times what you need is cold storage backup. These days even the most basic laptops often come with a DVD writer and the software to burn your photos to a DVD is almost ubiquitous. So burn away and make 2 copies of everything, then send one to some other location so that in the case of a real, biblical scale disaster (we are talking fire or flood here) you can get all your info back.
So, as we go through the files on your computer, the first thing we have done is determine that the vast majority of data (certainly for me) does not need to use a synchronisation service or an on-line backup. This is good because most times you will be charged by the Gigabyte, so saving all this space means that you are now most likely going to be under the free limit.

Next up, because it is the simplest to determine, is what needs to be synchronised. For me, where I work on a number of reasonably well defined projects, this is straight forward: Anything that is associated with a project that is ‘live’ needs to be synchronised. The net result of this is that, if like me, you file your documents according to the company you work at, then you finish up with a folder for the current work e.g. CompanyX 2008 and an archive folder CompanyX. When each project finishes, move the entire project folder into the archive (this approach works for e-mail too, but uses a different technology). So everything that is live gets stored under a special folder, cunningly called Dropbox. What Dropbox does is create a service on your machine(s) that monitors this folder and all subfolders under it; any change to any file is fairly rapidly copied to your private space (up to 1GB) on the Dropbox server. Yes, I know that this has potential privacy issues; so don’t put your medical records in your Dropbox. All your other machines are also monitoring the server (or do so as soon as they get switched on) and recognise that a change has been made and copy the relevant file(s) to their local Dropbox folder. As the blurb says on the web site “It just works” And that is the coolest thing about Dropbox- it just does it. No mess, no worries, it just gets on with ensuring that all the information you need is available on your local machine whether you are on-line or not. And it doesn’t matter if half your machines are Windows boxes and some are Mac’s- the files just magically appear.

You can achieve the same effect for e-mail by simply using IMAP instead of POP to collect your mail and then at a fixed moment, copying everything from the IMAP server to a local file and then manually copying the files to all relevant machines. This requires using the same mail client on all machines which is a bit of a blow if you have a Mac, but not everything is perfect.

Finally we come to the backup solution- now I know that some people like to backup their Program Files to recreate their machine in one go. Personally I think that there it is no bad thing to clear out the junk and garbage by going back to CD’s and reinstalling, so my backup is limited to the Dropbox (because if you do screw up, then you may still need to go back to yesterdays version) and the archive projects (because as a consultant, that archive is in fact your value to the next client). And once again, Mozy just seems to work. Obviously if you have a Mac you can use Timemachine to do this, but I don’t, so I can’t.

So, in summary:
Media: Burn DVD’s
Current documents: Sync through Dropbox, backup through Mozy
Archive documents: Backup through Mozy

I can’t say that I have had to use much of the Mozy functionality, but it does give me a warm fuzzy to know that I could.

Continuing saga of e-mail organisation

I have been eating my own dog food as it were and using the filing approach for organising my e-mail. However, I ran into another problem: I have a main desktop computer at my home office and an ultraportable laptop. Unfortunately, the ultraportable is running Vista whilst the desktop is on XP and synchronisation between the two has proven to be tricky (if I received 1 new e-mail from someone, I was having to copy 20 or 30MB of data because of the way the folders were organised). So I needed a better solution to managing my e-mail between computers, rather than within the mail system.

Fortunately there is such a protocol: IMAP. Unfortunately, neither or my e-mail providers support it (Damn you, Yahoo!) so I needed something else. Enter Google: gmail (or googlemail in the UK) has changed the way people deal with e-mail: There are no folders, just Labels and you can store massive amounts of e-mail on-line. Well, as far as I can make out, Labels are exactly the same as Tags, so that is no big deal and the storage of e-mail on-line seems to solve many of the problems of accessing my mail from multiple computers.

Except that quite a lot of the time I need to use mail in locations where I can’t get on-line (e.g. planes) so I still need an off-line set up. And I need to access all my existing mail accounts, and I don’t really want to have to tell anyone about a new e-mail address.

And wouldn’t you just know it, Google mail does all of this: So now my desktop/ laptop e-mail clients have an IMAP connection to Google mail. My Google account is set up to download all the mail from all my existing providers, I have even set up an existing account as my default (so no-one needs to know where my mail has really come from). Each label in Google is automagically converted into a folder in my mail client and I set up all my folders to be available offline.

All I need to do is make sure that I download all mail to the local PC and then at various periods (about 5GB periods in fact, as that is my mail storage limit) archive my mail to local files and then copy those local files to both my computers.

So far it seems to work. I hope it continues.