Watching a train crash

A blog is an intensely personal form of communication. It is, for the writer, an immediate and occasionally quite intimate outlet for the thoughts, feelings and emotions that is coursing through your veins. But writing a blog is also an incredibly public process. I have no control over who reads this or any other post. Furthermore, I have no control over who copies, archives or otherwise keeps a record of what I said, even if I have since deleted the post. Of course, I would like to think that anyone keeping a copy will delete any record they have when I delete my version, but I am not so naive to assume that is so.

Although most bloggers realise this, sometimes emotion overtakes us and we allow ourselves to vent without thought of the consequences. I read several blogs and most of them are well thought out and the author will be able to look back at the post in years to come with no fear or recriminations. But just occasionally, even smart people will forget; one blog I read has been describing what appears to be somewhat of an emotional breakdown of an otherwise perfectly healthy woman.

I know neither the blog author nor her friend and the first posts in the series, whilst slightly uncomfortable reading, were acceptable. The recent posts have made me genuinely fear for the friend- for her mental and even her physical well being. The feeling of impotence as I watch this train crash be described over a number of days is extremely unpleasant. Compounding the problem is that I have neither the skills nor the ability to do anything for these people. I don’t wish to be a ‘white knight’ riding in on a charger (though there are other scenarios in life where that image might be fun) but that doesn’t ease the feeling. I don’t know how that story will end, I fear that the answer is ‘not well’, and I am even afraid to look at the blog again. It feels like watching a train crash-you can’t take your eyes off it, no matter how much you want to.

The other example has less actual danger but highlights the danger of venting without thought of the recriminations. As anyone who knows me will tell I am not particularly politically correct. I tend to say what I think and frequently damn the consequences. However, I do try to care how I say things and I try to always include the requisite amount of etiquette in anything I write, say or do. OK, less on the say, but certainly when it comes to the written word I try to take care. So when I come across professional people who either publish very pointed blogs or send curt, incomplete and brusque e-mails; well I generally cringe inside. And the real problem is that by the time I get to read the words they have already been read by the intended recipient. So there is very little I can do can to prevent the damage- it is already there. In this situation the train has already crashed- all I do is to try some form of clean-up.

So with these examples out there in the blogosphere (or the world of internal e-mails) I hope I don’t ever offend anyone by the contents of this blog. If I do feel I am saying something negative I will endeavour to hide or disguise the person concerned so much that they are unaware of the point I am making. Conversely, if I am complimenting someone I will generally let them know directly (especially if I know the person). I feel I owe this to whoever I am communicating with, whether I know them or not, whatever personal feelings I may have or whatever medium I am using. Perhaps I should remind others of this, but I fear that the advice would not be taken well (but that may be due to the aforementioned lack of political correctness and subtlety in my communcation of the message)

About a blog: How to increase your hits

In the scale of navel gazing this has got to be right up there, but over the past few days I have been thinking about this blog and what I have found out over the past 15 months. So I am now going to create a blog post all about creating blog posts.

When I started to write this blog I had no clear idea what I wanted to make it into. I saw it as a forum for mainly technical items that would be a work in progress before uploading to a permanent location on another site. I also realised that there is an example of the type of entry I wished to create: It shouldn’t be a random comment on something I have seen or read, but rather should contain some original thought and try to reach a logical and sensible conclusion. As I thought about more and more I realised that I was, in a very small way, creating my version of Alistair Cooke’s venerable and much missed ‘Letter from America’

Of course the original intention of a technical forum was now pushed to the back and I posted on the vagiaries of the world. And as I posted I monitored the hits and watched what people were reading. Here is what I have learnt:

1. Most recommendations say that you need to create decent sized postings to keep people coming back. This appears to be true
2. You will create many, many postings but most visitors will only read 1 or 2 really popular posts. And you probably won’t know which post that will be until after the stats come in
3. Almost no-one clicks on the hyperlinks in a post (even when you completely expose the URL)
4. If you really want to increase your traffic then create a post with certain key words in it

This last point requires some explanation. In April 2008 I created a post that attempted to discuss the nature of art, photography and whether a photograph on a magazine cover should be an accurate representation of the image recorded by the camera. All heady stuff, except it was inspired by a TV show called ‘Dawn Gets Naked’ and considered the examples of photo manipulation performed on Faith Hill and Kate Winslet on the cover of Redbook and GQ.

So, at various points in the post there are the words “naked”, “Kate Winslet” and “Faith Hill”. Oh boy, do I get a lot of hits on that post. And I am guessing, but it is only a guess, that most people who have typed ‘faith hill naked’ into Google were not expecting to read a blog posting that discusses the evolution of painting techniques and creating accurate oil colours to depict an english monarch from 900 years ago.

My only hope is that a small number of people who arrive at the site looking for Kate or Faith read the article and stay for the thought and comment. Although, saying that, do I really want someone interested in naked pictures of Kate reading this? Well that is the beauty and terror of a blog- you have no idea who the vast majority of readers are. So welcome, one and all: Batblog is still here and still growing. I’ll keep posting, I hope you keep reading

Change is great (but not for me)

As someone who is paid to update processes, design new applications to support those processes and implement the data structures to store the necessary information I am frequently proposing that a company implement a new system. In many of these situations I am called upon to perform the sales pitch to the existing users to convince them that the new system will be better and their lives will improve. And most times I believe that I what I am saying is the truth; because as anyone who has played with me will tell you, I am a rotten poker player and it is almost impossible for me to convince someone else of something I don’t believe in.

A natural consequence of this is that I am normally close to the front of the queue to try out some new application or another. Most of the time I can work out what to do pretty quickly and if I can’t then it is probably either indicative of a poorly designed application or that I am trying to learn something completely new and I have no clue what I am doing. Yeah, you got me; most times it is the latter of these cases. Furthermore, if I am trying an upgrade to an existing application then I expect there to be little or no learning curve and only the occasional frustration when I can’t find something. I have even grown to live with Vista, though I can’t actually say I like it, I find that if you switch off enough of the new toys then it becomes a perfetly adequate OS. Of course if you leave them all on then it truly is the nightmare that so many others have written about.

So with this background and experience it was quite the shock today when I found myself so frustrated with an application that I was literally banging my head against my desk. Not only can I still feel the bruise on my forehead but it scared the crap out of a co-worker who was sat behind me. What caused this outburst? Microsoft Office 2007 and in particular Excel. I had already gone through the process of turning off the Ribbon and creating my own toolbar when I realised that I needed to control the zoom of the window. Since I started using Excel in 1992 it has been possible to do this by placing a little widget in the toolbar; but no longer. Despite there being 3 (count ‘em) different Zoom options it is not possible (as far as I can see) to put the zoom level in the toolbar. No, now you have to look down in the lower right hand corner of the screen. Sure, you can turn off the standard location, but you can’t put it where you want it to be. It was somewhere during the 5 minutes it took me to work this all out that my head and the desk came into close proximity and my colleague jumped out of his skin.

And this is just one of the myriad of changes that has been forced on me by Microsoft. I have read that people like myself make up less than 2% of the users of Microsoft Word, by which I mean that I believe I use it correctly with styles, customisation of the toolbars and personalised templates. So Microsoft, by all means create a UI that is better for the 98% of people who are not using your tools correctly but please Please PLEASE let the 2% who do like to control the look of the application so that it truly meets our needs make the customisations necessary.

All of which is a rant aimed squarely at the designers of Office. I generally like change but in this case I will be sticking with the old versions of your software. So keep making your changes, when they work for me, then I’ll join the party. Until then I’ll keep on being productive in my own way.

What I use (3): Online sync and Backup

A while ago I wrote that I think that best way to access the right computing power at the right time is to have 3 different hardware solutions, I called them ‘Permanent Presence’ (a smart phone), Portable Solution (laptop) and Base Station (desktop/ server). At the time I noted that the downside to optimising the hardware solution is the need to synchronise information between the machines. Although there are various solutions for the large scale enterprise including Groove, I found that the solutions offered to a small business were clunky, slow and frequently simply failed to work.
The advantage that the large enterprises have, and probably the reason why the solutions work, is that someone has actually put some thought into designing an optimal solution. When you are not a storage/ backup professional the temptation is to just let a service provider sell you their latest toy and be happy with it. The problem with this is that you are no longer in control of meeting your requirements- you are reacting to someone else’s belief of what they think you need.
However, having someone do the thought for you is sometimes good and a couple of solutions have come along that allow the design of a fairly good solution. The tools are Dropbox and Mozy; both of which have been around a little while and both have won awards. So I am not some crazy evangelical here, and I am not even at the front of the wave; what I am proposing is that combining the best tools out their with some system design can yield an effective solution for virtually no cost.

So, as I was trying to come up with a better solution the first question that you need to ask is “What information do I need to keep synchronised between the machines and what information do I just need to copy to all my computers?” Examples of the first type of information include the documents you are working on, examples of the second time include photographs or MP3 files. Because what you actually need for photos is not a synchronisation service, and you don’t even want an interactive backup service- most times what you need is cold storage backup. These days even the most basic laptops often come with a DVD writer and the software to burn your photos to a DVD is almost ubiquitous. So burn away and make 2 copies of everything, then send one to some other location so that in the case of a real, biblical scale disaster (we are talking fire or flood here) you can get all your info back.
So, as we go through the files on your computer, the first thing we have done is determine that the vast majority of data (certainly for me) does not need to use a synchronisation service or an on-line backup. This is good because most times you will be charged by the Gigabyte, so saving all this space means that you are now most likely going to be under the free limit.

Next up, because it is the simplest to determine, is what needs to be synchronised. For me, where I work on a number of reasonably well defined projects, this is straight forward: Anything that is associated with a project that is ‘live’ needs to be synchronised. The net result of this is that, if like me, you file your documents according to the company you work at, then you finish up with a folder for the current work e.g. CompanyX 2008 and an archive folder CompanyX. When each project finishes, move the entire project folder into the archive (this approach works for e-mail too, but uses a different technology). So everything that is live gets stored under a special folder, cunningly called Dropbox. What Dropbox does is create a service on your machine(s) that monitors this folder and all subfolders under it; any change to any file is fairly rapidly copied to your private space (up to 1GB) on the Dropbox server. Yes, I know that this has potential privacy issues; so don’t put your medical records in your Dropbox. All your other machines are also monitoring the server (or do so as soon as they get switched on) and recognise that a change has been made and copy the relevant file(s) to their local Dropbox folder. As the blurb says on the web site “It just works” And that is the coolest thing about Dropbox- it just does it. No mess, no worries, it just gets on with ensuring that all the information you need is available on your local machine whether you are on-line or not. And it doesn’t matter if half your machines are Windows boxes and some are Mac’s- the files just magically appear.

You can achieve the same effect for e-mail by simply using IMAP instead of POP to collect your mail and then at a fixed moment, copying everything from the IMAP server to a local file and then manually copying the files to all relevant machines. This requires using the same mail client on all machines which is a bit of a blow if you have a Mac, but not everything is perfect.

Finally we come to the backup solution- now I know that some people like to backup their Program Files to recreate their machine in one go. Personally I think that there it is no bad thing to clear out the junk and garbage by going back to CD’s and reinstalling, so my backup is limited to the Dropbox (because if you do screw up, then you may still need to go back to yesterdays version) and the archive projects (because as a consultant, that archive is in fact your value to the next client). And once again, Mozy just seems to work. Obviously if you have a Mac you can use Timemachine to do this, but I don’t, so I can’t.

So, in summary:
Media: Burn DVD’s
Current documents: Sync through Dropbox, backup through Mozy
Archive documents: Backup through Mozy

I can’t say that I have had to use much of the Mozy functionality, but it does give me a warm fuzzy to know that I could.

Internal economies and large companies

Life as an independent contractor is going to be a whole lot more ‘interesting’ for the next few months and possibly few years. The credit crunch (cue bad joke about the ‘credit crunch’ sounding like a bad breakfast cereal) has been having an effect on the wider economy for a number of months and from my personal survey it appears to be hitting the technical community and big companies. It may simply be that November/ December are simply exhibiting their usual end-of-year slow down and that activity will pick up again in 2009, but I don’t think so.

I have been more active than usual in monitoring job vacancies on account of the fact that a major contract ended and I had no immediate opportunities. As the financial markets started to resemble a roller coaster (up and down, but far more down than up) there were many, many contracts advertised in the financial services sector. Now I don’t have any experience in financial services, but when you don’t have any other work, you start to consider other possibilities. Of course as the financial meltdown continued the possibility of turning up on a Monday to start work and getting terminated 24 hours later grew and grew. So maybe financial services wasn’t the way to go for me, but that question became moot as within a couple of months there are almost no jobs being advertised in financial services. The companies have closed down all recruiting activity as they try to work out what on earth they are going to do to weather the storm.

Fortunately for me I managed to get a new contract with an existing client but this got me to thinking as to why I was having difficulty finding new opportunities. Obviously, the credit crunch is not helping, but I also realised that as a small contractor my network is very dependent on the people I meet. I have had a contract at a large company for over two years now and I realised that my network of contacts is actually almost exclusively internal to the company.

The company is so large it is functioning as its own mini-economy- people change jobs, move, have health care and many the other benefits that are typically assigned to a government. And they do all of this without ever actually leaving the company. Even more bizarrely is that different departments start poaching people to server their own means and consequently even turf wars erupt. When people are looking for a new role they have to advertise their own services to the rest of the company, just as you would in the ‘real world’.

I have no idea if this is the same in all really large companies, but I can imagine that it is. And here is my point- although this company is exhibiting many characteristics of a country, a government and an economy, it is not a totally related to the world outside the walls. The closest analogy I can come up with is the artificial economy that was put in place by the Soviet Union: I can clearly remember having a discussion in the early 1990’s with an ethnic Russian in his late 50’s who was convinced that the West was forcing up the price of bread. I never did work out if he thought it was all part of the closing stages of the Cold War or just some capitalists playing with his mind. What became clear was that under the soviet system he had been paying far, far less for his loaf of bread: the system was determining the price to pay, not external factors such as the price of wheat in Canada or a rise or fall in the price of oil to transport the oil.

What I find most interesting is that the very companies that are the basis of the whole free market system are functioning internally in exactly the same way as a system that they brought to its knees. I think it is going to be one hell of a ride to watch how this plays out over the next few months. But in the meantime I am going to put much more effort into developing an external network. So look out for me at a few more conferences in the next few months- I’ll see you there.

Privacy, Venture Capital and the Common Good

Some people may say that I am naive when it comes to privacy- if you Google me, you find me; I am happy to upload a photograph of what I look like to Facebook and I even publish details of my location in the world via a calendar on my web site. No, when it comes to privacy, I leave the deep thinking in the hands (and heads) of people like Pamela Dingle.

Which doesn’t necessarily mean that I don’t care about it, after all the first time I ever thought about privacy issues was when Sam Seaborn was interviewing a prospective Supreme Court Judge on the West Wing. No, I have issues with who collects information about me and what they do with it. And I have even more issues with a service that creates a picture of my entire financial life and then stores that information somwhere that I don’t have entirely under my control.

Now you can argue that anyone can break into my house, go through my filing cabinet and create a similar picture, but the crucial difference is that I would (almost certainly) know that someone has broken in, so I would be able to immediately take some remediative action. Furthermore, even if you have all the account information, unless you can break the password codes kept in the back of my diary, you still wouldn’t be able to actually do anything.

So what has prompted this post? Well it seems that there are enough people in the world willing to trust someone else with their financial life (including passwords) for the creation of another amalgamation site: Techcrunch is reporting on Kublax offering the ability to store all your financial details, your bills and even your loyalty cards.  This is the same as Mint and Gazeo, both of which have received heavy venture capital investment.

Although there is a part of me that loves seeing the VC guys fall flat on their faces, they do perform a necessary function- after all how could Cuil afford to order in lunch every day if someone wasn’t backrolling them? So the venture capital guys see a future in sites that store everything about you and reckon that they can make some money.

And this is where is starts getting really frightening: I am not convinced that enough people are aware of the potential privacy issues to use these systems safely. And just because someone can make money out of something doesn’t necessarily mean that it is a good thing- we need checks and balances, we need to truly understand the implications of providing all this information to someone else.

If we do have the correct rules in place and people are aware of what we are getting into then this can be a useful function. If not then a few people are making money, a few companies will know more about you than they ever should and we nail another nail into our on-line world.

(Re)connecting through social networks

No one can deny the massive impact that social network sites are having on the internet world. Of course there is always the bigger question of what percentage of the real world the internet world really represents, but that is a question for another day.

I have used different social network web sites for the past 2 years (except that at the start I didn’t even know what a social network web site was, I just knew that LinkedIn had huge performance issues) and have found that the ability to reconnect with old mates is one of my favourite features. The question becomes: What do you do once you have connected via the ethernet? Ping an occasional e-mail and catch up or actually meet in real-life again.

I have tried both approaches and found that the impact of actually meeting is far greater than any number of e-mails, instant messages or pings or nudges. These are the same issues that all people who meet through the internet face: Suddenly there is no where to hide, no back space key to remove ill conceived thoughts and, worst of all, no easy way to duck out of the conversation if things are going badly. The pressure of these situations is well known on the first date, but what surprised me was that it is still present when reconnecting.

The net result of this is the that meeting can rapidly head into one of two directions:

1. You realise that you really need to see this person more and can’t believe that you ever fell out of touch
2. You realise that the intervening years have taken you in markedly different directions and you really have nothing in common any more

Of course when #2 occurs it is best to retreat a quick retreat, but when #1 happens…..

A(nother) cool web 2.0 site

OK, this one might go into the ‘cool but pointless’ category or it could be the greatest thing to hit the computing cloud, I can’t decide which and I guess that only time will tell.

There appears to be 2 ways to move into the true on-line world and I think they differ in their point. The approaches are:

1. Integrate your desktop with your on-line world (not a desktop machine, but your compute desktop)
2. Migrate all the functionality on-line

Desktop Integration

The integration approach (taken by Xdrive) allows you to open a Windows Explorer window that looks just like any other- except that the drive you just opened (cunningly called X: ) is nowhere near your local area network. This allows you to run all your local, desktop applications and save documents and files to the on-line storage. The advantage of this approach is that you have pretty damned good integration with your desktop. The disadvantage is that if you use multiple local machines, you have to make sure that they are all set up identically if you wish to duplicate the experience- for example ensuring that all the applications are installed on each machine.

On-line Functionality

On-line functionality takes everything away from the desktop and runs in its own little world. Meebo takes this approach- so all your IM contact details, your chat logs etc are stored remotely. This has the advantage that everything is going to be the same no matter what computer you use to access the functionality. The downside is that (currently) most apps focus on a single piece of functionality e.g. IM or Calendar or file managment

Unfortunately, what all of this means is that many of the apps are simple re-packaging of what you can already do, just not do it on-line. This means that every on-line desktop provider I have tried (Xdrive, GMX, Omnidrive) has a folder for ‘My Music’, ‘My Photos’ etc and I have no way of taking my data from the on-line applications off-line (though you could argue why would I want to- it is just the Generation X insecurity in me)

And then I came across Jooce (www.jooce.com) that gives the whole world a shake upside down: What Jooce does is create a complete desktop, just like your main desktop, but it does it inside a browser. So now you can file your documents away nicely or just leave them on the desktop.

As I said as the start, I have no idea if this is the coolest thing to hit the way we use computers or a total waste of time. In my mind, the key to creating a cool Web 2.0 app is to try to break down what we use a computer for and then either identify something totally cool that you never knew you needed to do or make something that you do already available in a new and interesting way. I don’t see Jooce doing either of these, but I can’t deny that when I saw that I had the ability to rotate a picture on my desktop it made me smile with glee.

Jooce Desktop with a rotated picture

But then small things always did amuse me… and a good job too.

Building the complete solution

Sometimes it takes a person with a bit of perspective and distance from a problem to identify the solution. Sometimes it takes someone else to read your ramblings on a blog and ask a question to make you realise the point that you have been trying to make. I received some feedback about my recent post on hardware profile that made me realise that the combination of the posts on what I use, data portability and hardware profile were all pointing to a common solution. And as with so many solutions I think that the key to it all is the information.

What Marinko made me realise is that I own, use and define a set of information; some of the information is created or edited by me, some is delivered to me from other sources and may or may not be read-only; some of the information has its own security profile (for example, I can’t copy the file away from my work laptop) and some is available to anyone, such as a public web site like BBC Sport. But, at the end of the data, the sum of the documents, e-mails, pictures, music web sites and RSS feeds etc make up my information entity (sorry to the data modellers for using that word, but I can’t think of a better one).

So, what I need is a number of mechanisms to access my information- depending on the source of the data (public, private), the action I wish to do (edit/ read) and my location (on-line/ off-line) I will access my information using different tools (the Permanent Presence, Portable Solution or Base Station as described in the Hardware Profile).

Everything else, the file format, the internet protocol, new capabilities of a smart phone etc are all steps to move towards this goal.

Which brings me to a bit of a conclusion- there does not need to be one single format to solve all the problems of the world, but there needs to be a fairly small number and the information format must be open for all to read and all to implement. By this I mean, that though it may be tempting to say that all data, documents images etc be stored as XML because that can be parsed, transformed and displayed in different ways on different machines, this is the typical IT situation of an evangelical war over one technology or another. Sure, it would be nice for the application developer to know that all e-mail is in the same XML format, but it is not going to happen- and we already seem to have a fine working model of e-mail formats that can be accessed wherever you want. Same applies for images (JPEG/ PNG), drawings (SVG) and music (MP3).

What the alert amongst you will have noted is that I haven’t included a set of Office document formats- so here goes, time to get flamed or praised: Any document format that is dependent on a particular client to correctly display the information cannot meet my requirements. By this I mean that if your format has a bug in it so that the answer to a calculation is only correctly displayed if you use, oooh say Microsoft Excel, you are not fit to be considered an international standard certainly should not be blessed by ISO. Anyone who knows me has come across a situation where I have held an opinionated and probably unfathomable view on something or other; but at the same time I tend not to simply dismiss technology, companies or solutions out of hand simply as an act of faith. I use Microsoft Office including Visio because it does do a good job. But requiring that I use Excel or Word to view a document, even though it has been created using an ‘open standard’ fails on so many levels.

As an information or data architect I don’t understand the need to have open source software (the components and code) but I definitely see the need to have open, defined information standards. And why do I need this? Because I have not decided how I am going to access your data- and depending on the profile I am using to view it, I may or may not wish to use the same tool or see all the details of what you sent. This is the same as a web page ‘degrading gracefully’ when viewed over a phone, but if we take the premise I made back at the start- all the information that makes up my world needs to be available to me, whatever hardware profile I choose. This means that an office document, a photograph or a web page need to be treated the same. Well, for me to be happy anyway.

Hidden Beauty in Images

Once upon a time I used to be a geophysicist. Not a very good one, in fact I think I can lay claim to being the world’s most environmentally friendly oil explorer: I didn’t find any. I even got sent to look for oil in Saudi Arabia and managed to come up dry- now that, my friend, that takes some doing.

So, even though in the entirety of my exploration career I used more oil travelling to far flung parts of the world than I ever found when I got there, I still somehow manage to maintain accreditation with various professional geophysics organisations, including the Canadian Society of Exploration Geophysicists, who send me a monthly magazine. Needless to say, most of the content goes far over my head (perhaps this is why I was so singularly unsuccessful at exploration?) but every now and then there is an article that makes sense, is at least half way well written (hey, I can recognise good writing, even if I find the act of creating a well written piece a little harder) and is worthy of a wider audience.

Well this article http://www.cseg.ca/publications/recorder/2008/02feb/feb2008-amplitude-and-phase.pdf is just such a cool little piece. It shows how information can be hidden, how the traditional approaches of discovery can be effective, but sometimes you need to take an unusual tack to get significantly better results. And if you look very closely, the world can be a far, far more beautiful place than you ever imagined before.