A(nother) cool web 2.0 site

OK, this one might go into the ‘cool but pointless’ category or it could be the greatest thing to hit the computing cloud, I can’t decide which and I guess that only time will tell.

There appears to be 2 ways to move into the true on-line world and I think they differ in their point. The approaches are:

  1. Integrate your desktop with your on-line world (not a desktop machine, but your compute desktop)
  2. Migrate all the functionality on-line

Desktop Integration

The integration approach (taken by Xdrive) allows you to open a Windows Explorer window that looks just like any other- except that the drive you just opened (cunningly called X: ) is nowhere near your local area network. This allows you to run all your local, desktop applications and save documents and files to the on-line storage. The advantage of this approach is that you have pretty damned good integration with your desktop. The disadvantage is that if you use multiple local machines, you have to make sure that they are all set up identically if you wish to duplicate the experience- for example ensuring that all the applications are installed on each machine.

On-line Functionality

On-line functionality takes everything away from the desktop and runs in its own little world. Meebo takes this approach- so all your IM contact details, your chat logs etc are stored remotely. This has the advantage that everything is going to be the same no matter what computer you use to access the functionality. The downside is that (currently) most apps focus on a single piece of functionality e.g. IM or Calendar or file managment

Unfortunately, what all of this means is that many of the apps are simple re-packaging of what you can already do, just not do it on-line. This means that every on-line desktop provider I have tried (Xdrive, GMX, Omnidrive) has a folder for ‘My Music’, ‘My Photos’ etc and I have no way of taking my data from the on-line applications off-line (though you could argue why would I want to- it is just the Generation X insecurity in me)

And then I came across Jooce (www.jooce.com) that gives the whole world a shake upside down: What Jooce does is create a complete desktop, just like your main desktop, but it does it inside a browser. So now you can file your documents away nicely or just leave them on the desktop.

As I said as the start, I have no idea if this is the coolest thing to hit the way we use computers or a total waste of time. In my mind, the key to creating a cool Web 2.0 app is to try to break down what we use a computer for and then either identify something totally cool that you never knew you needed to do or make something that you do already available in a new and interesting way. I don’t see Jooce doing either of these, but I can’t deny that when I saw that I had the ability to rotate a picture on my desktop it made me smile with glee.

Jooce Desktop with a rotated picture

Jooce Desktop with a rotated picture

But then small things always did amuse me… and a good job too.

Building the complete solution

Sometimes it takes a person with a bit of perspective and distance from a problem to identify the solution. Sometimes it takes someone else to read your ramblings on a blog and ask a question to make you realise the point that you have been trying to make. I received some feedback about my recent post on hardware profile that made me realise that the combination of the posts on what I use, data portability and hardware profile were all pointing to a common solution. And as with so many solutions I think that the key to it all is the information.

What Marinko made me realise is that I own, use and define a set of information; some of the information is created or edited by me, some is delivered to me from other sources and may or may not be read-only; some of the information has its own security profile (for example, I can’t copy the file away from my work laptop) and some is available to anyone, such as a public web site like BBC Sport. But, at the end of the data, the sum of the documents, e-mails, pictures, music web sites and RSS feeds etc make up my information entity (sorry to the data modellers for using that word, but I can’t think of a better one).

So, what I need is a number of mechanisms to access my information- depending on the source of the data (public, private), the action I wish to do (edit/ read) and my location (on-line/ off-line) I will access my information using different tools (the Permanent Presence, Portable Solution or Base Station as described in the Hardware Profile).

Everything else, the file format, the internet protocol, new capabilities of a smart phone etc are all steps to move towards this goal.

Which brings me to a bit of a conclusion- there does not need to be one single format to solve all the problems of the world, but there needs to be a fairly small number and the information format must be open for all to read and all to implement. By this I mean, that though it may be tempting to say that all data, documents images etc be stored as XML because that can be parsed, transformed and displayed in different ways on different machines, this is the typical IT situation of an evangelical war over one technology or another. Sure, it would be nice for the application developer to know that all e-mail is in the same XML format, but it is not going to happen- and we already seem to have a fine working model of e-mail formats that can be accessed wherever you want. Same applies for images (JPEG/ PNG), drawings (SVG) and music (MP3).

What the alert amongst you will have noted is that I haven’t included a set of Office document formats- so here goes, time to get flamed or praised: Any document format that is dependent on a particular client to correctly display the information cannot meet my requirements. By this I mean that if your format has a bug in it so that the answer to a calculation is only correctly displayed if you use, oooh say Microsoft Excel, you are not fit to be considered an international standard certainly should not be blessed by ISO. Anyone who knows me has come across a situation where I have held an opinionated and probably unfathomable view on something or other; but at the same time I tend not to simply dismiss technology, companies or solutions out of hand simply as an act of faith. I use Microsoft Office including Visio because it does do a good job. But requiring that I use Excel or Word to view a document, even though it has been created using an ‘open standard’ fails on so many levels.

As an information or data architect I don’t understand the need to have open source software (the components and code) but I definitely see the need to have open, defined information standards. And why do I need this? Because I have not decided how I am going to access your data- and depending on the profile I am using to view it, I may or may not wish to use the same tool or see all the details of what you sent. This is the same as a web page ‘degrading gracefully’ when viewed over a phone, but if we take the premise I made back at the start- all the information that makes up my world needs to be available to me, whatever hardware profile I choose. This means that an office document, a photograph or a web page need to be treated the same. Well, for me to be happy anyway.

Hidden Beauty in Images

Once upon a time I used to be a geophysicist. Not a very good one, in fact I think I can lay claim to being the world’s most environmentally friendly oil explorer: I didn’t find any. I even got sent to look for oil in Saudi Arabia and managed to come up dry- now that, my friend, that takes some doing.

So, even though in the entirety of my exploration career I used more oil travelling to far flung parts of the world than I ever found when I got there, I still somehow manage to maintain accreditation with various professional geophysics organisations, including the Canadian Society of Exploration Geophysicists, who send me a monthly magazine. Needless to say, most of the content goes far over my head (perhaps this is why I was so singularly unsuccessful at exploration?) but every now and then there is an article that makes sense, is at least half way well written (hey, I can recognise good writing, even if I find the act of creating a well written piece a little harder) and is worthy of a wider audience.

Well this article http://www.cseg.ca/publications/recorder/2008/02feb/feb2008-amplitude-and-phase.pdf is just such a cool little piece. It shows how information can be hidden, how the traditional approaches of discovery can be effective, but sometimes you need to take an unusual tack to get significantly better results. And if you look very closely, the world can be a far, far more beautiful place than you ever imagined before.

Hardware Profile

I have not had time to post much (read anything) to the blog in a while- this is not because I have stopped having ideas of things, though they have slowed down- but because I have not had time to sit down and put electronic pen to paper. Additionally, now that I have plunged into the Twitter micro-blog pool it feels as though I am still updating my on-line life, even if the longer articles associated with a blog are less frequent.

So, why have I not had sufficient time? Well, as I posted before, occasionally real work gets in the way of doing the fun stuff, but a new toy has led to the inspiration to post.

The history of the computer can be traced back over 200 years to Charles Babbage and his Analytical machine, 50 years to the Bombe that helped to crack the Enigma codes, 35 years to the debut of Intel’s 8008 microprocessor or 17 years (at the time of writing) to the invention of the internet. If we look back through time the path to the ubiquity of computers seems clear, but looking forward from today there are clear differences in the vision of what computers will do and how we will use them: Tom’s Hardware is trying to decide whether or not to buy a laptop or desktop whilst MacUser has made the decision and recommends that if you only need a single computer, then that computer should probably be a laptop- but I simply reject their premise that a single computer can meet my needs. I have identified 3 use profiles for the tech aware, connected geek of today (and the hardware that I currently use to meet the profile):

  • Permanent Presence: Nokia e51

  • Portable Solution: Toshiba R500

  • Base Station: Twin screen desktop with serious sound system, top notch keyboard and bucket loads of RAM

The key thing about these profiles is that they need to work seamlessly together- information needs to by synchronised between all machines as well as needing to meet the particular profile. The synchronisation issue is why I am so interested in web apps and how they can also work offline. I want to be able to use a single storage location for all my files, have that sync whenever I need it, work off-line and not get in the way. I have worked at some large companies that have implemented this, but I get the impression that it is just hard. Too hard for me anyway, as my trials with Omnidrive, Meebo etc have proved. Anyway, getting on with the profiles:

Permanent Presence

This is the term I use for the computer that you have with you the whole time- for most people today this is a cell/mobile phone (be it ‘smart’ or not), though in the past this may have been a Personal Digital Assisstant (PDA) or even simply a filofax. I resisted carrying a mobile for several years, feeling that they were for posers, gits and tossers who liked to announce to the world that they were “Yuh, I’m on the train. I’ll be home in 20 minutes” to anyone in earshot (which would normally be the whole carriage). However, sometime this century mobile phones stopped being looked down upon and became all pervasive. Being good little computers, the phones followed Moore’s Law and increased their computing power each year, as did our ‘need’ to be connected. However, the prime use of the permanent presence machine is to read e-mail, text, basic news services and stay in contact through voice, text, brief e-mail and possibly video calls. For many people today this is a Blackberry or similar. Personally, I got stuck into Nokia many years ago, hence my preference for the e51. The ability to create content will vary by user, but for me that is where the next step up comes in…

Portable solution

This is where the situation starts to get interesting, as for many people this is the size of machine where they start to think of owning a computer. Now if you truly are set on owning a single solution then this will be a mid sized laptop, or even a large multimedia desktop replacement. But if you actually follow my advice (and the justification for it appears in the Base Station section) then the portable solution should be as portable as possible. By introducing the ability to take your compute power with you, you are already compromising on screen size, keyboard capability, mouse comfort and sound system quality. Buying a bigger laptop is not going to give you speakers as good as a set of Harman Kardons, nor is the keyboard going to be as good as a Logitech di Novo. So if the compromise has to be made and the system is not your one and only, I believe that the system should be as light and portable as possible.

Currently there are 4 real players in this sphere: 3 have been battling out for almost a year and one has come along more recently. The three old hands are Sony TZ series, Apple’s MacBook Air and Toshiba’s R500 and there was, in my mind, no clear winner; the MacBook Air is the thinnest, the R500 is the lightest and the Sony had the most capability built in- you just chose your favourite vendor of particular pose requirement (I am an old Tosh hand, and I like that lack of weight in the R500). Lenovo obviously learned from these three pretenders and came out with the X300 which is now clearly the best portable solution. So why isn’t everyone using these machines? Well, most obviously they are hideously expensive; every 50g or 1cm reduction costs more. Further these machines are only really usable when you have a 2nd machine- the advantage of a single machine is that there is no need to synchronise separate machines. If you have a docking station and simply plug the machine in, then you are using the same hard drive- hence the lack of need to sync. So the only people likely to be driving a truly portable machine are not constrained (too much) by budget, are tech aware to keep things in sync and probably like to pose a little bit with their new toy. Yup, sounds like I fit that bill 😦

Base Station

Now we get to the real justification for using multiple machines. I like twin (or more) screens. I know it is a little extravagant, but honestly once you get used to that much screen real estate, you will not be prepared to give it up. To paraphrase a quote “once you go black, you won’t go back”. I like to think that the quote was about coffee, but read into it what you will. I currently have two 21” screens giving me a desktop that is 3200*1200 pixels across; that equates to 1m * 36cm. Or as my brother put it: “That’s a piggin’ big screen!”. Now admittedly this requires a large desk, though not nearly as large as it used to be when the screens were CRTs! And there are arguments that can be made that a single 30” screen is a better solution, but not only were they not available when I bought my system, but they are even more expensive than the ultra-portable laptop. Somewhere budget has to come into it 🙂


So, if you can solve the synchronisation issues and can afford it, then using machines designed for each particular purpose is truly the best approach. And if you travel a lot and are able to justify not using the 7lb monster, your back will thank you too.

Take foot, place in mouth

So a little more digging and suffering with web storage, in particular Omnidrive just shows how dangerous it is to say anything. The minor teething issues I was having with getting Omnidrive to work turn out to be probable symptoms of something far worse: In the words of Dr McCoy; “It’s worse than that, Jim, it’s dead”. Well maybe not quite dead yet, but far from the glowing reviews that Omnidrive has received from Techcrunch, extremetech and many others, the truth for the average user seems far less rosy.

Matthew Ingram wonders if Omnidrive is heading for the deadpool, which my current experience appears to back up. Wandering around the various forums and chat rooms seems to indicate that all is not well in the Omnidrive world- there are many technical issues and these are not being solved mainly because of financial difficulties. And yet most of the complaints seem to be coming from people who have paid for storage. This raises a puzzle in the Web 2.0 world: How to make money from all the tech developments? I am worried when something like on-line storage can’t make it work: surely that has one of the simpler business models and if they can’t make it work, then what hope is there for something like Meebo that has no obvious revenue stream.

So for now at least, I have to have 1 foot in with the big guys- I’ll use Xdrive for on-line storage, becuase even though it may be flaky from time to time, it seems to be the best of the bunch.

So, now the list looks like

  • IM: Pidgin/ Meebo
  • Web Mail: GMX
  • Online storage: Xdrive

And the rest stay the same

What I use (2)

Six months ago I posted about the various pieces of (mainly web) software that I use. Well, in this world of rapid software development, six months is a long time. Not least of which is because I am slowly catching up with the rest of the world as to what is the latest ‘cool’ app. I am still using the same personal social network (Facebook), professional social network (LinkedIn), photography site (Flickr), bookmarking site (del.icio.us), travel site (Dopplr) and calendar (Google Calendar) but I have now made some decisions, expanded in other directions and discovered some new apps. So, here goes with the new things that I am using:

  • Web mail: GMX. You may not have heard about GMX, but they have been dominant in Germany for years and have recently made their service available to the rest of the world. Now I know that there are lots of people who rave about Gmail and Yahoo has fought Hotmail/ LiveMail (or whatever brand Microsoft has given it this week) to be the biggest webmail on the block, but I have tried ’em all, and so far GMX is knocking them all out the park: IMAP, slick UI, fast response and no adds. Damn this is good. Oh and it can collect all your mail from all your other accounts. Just about the only thing it doesn’t do is a WAP interface, but smart phones are just downloading mail straight to your phone, so that is not as essential
  • Instant Messaging: On a desktop I have been using Adium or Pidgin, but I really like the way Meebo works on the web. The thing about IM is that there is no need to have a desktop client, because how can you chat if you don’t have an internet connection? So it would seem a perfect case of a web app. As far as I can make out, the only down side is that you can’t encrypt the conversation (e.g. with OTR)
  • Online storage: Originally there was Xdrive, then AOL bought it and it still offering the best deal (5GB for free, as opposed to Yahoo’s 30MB) but Microsoft’s Live Spaces is coming up hard on the rails. I am trying to stick with an alternative: Omnidrive. When it works it is just great: only 1GB storage, but that is OK for what I need. But I keep having issues getting it to work on my Windows machine- it works magnificently on OS X (seriously, the UI is so good it is yet another reason to switch)
  • RSS reader: Last time I didn’t know whether to go with Yahoo or Netvibes. Well, Netvibes won, but I still miss some of Yahoo’s functionality. Oh well, you can’t win ’em all

There may be more, but with those three (GMX, Meebo and Omnidrive) I think that I am starting to see a trend: the big players (Microsoft, Yahoo and Google) all offer the same functionality and all in a one-stop-shop. So why I am using all sorts of odd little players and not just stick with 1 offering? Many, many reasons including truly picking the best of breed, supporting the little guy or just trying to be different. But probably the main one is just trying to stick one to ‘the man’: It is possible to survive in America without sucking up to the blandness of corporate ubiquity- Dave Gorman proved it by driving coast to coast (with a few, OK a lot, of detours on the way) and making a film and book about it; America Unchained is well worth the read. Mind you, Dave also attempted to live exactly according to his horoscope, so maybe holding him up as a role model is not entirely a good idea.

In the meantime, I’ll keep doing what I can do be independent.

add to del.icio.us : Add to Blinkslist : add to furl : Digg it : add to ma.gnolia : Stumble It! : add to simpy : seed the vine : : : TailRank : post to facebook

Data Modelling in Practice

I try to keep my hand in at creating data models- for whatever strange reason I still enjoy sitting down with a set of requirements documents and working out the logical and then physical data model to meet those requirements (As a quick aside, I still prefer Oracle as my target database, but that is just because the oil business has been dominated by it for so long that it has become my ‘comfort food’ of database systems. Either that or the fact that there was a particularly hot woman who worked there and was one of my (many) not girlfriends. But I like to think it is the former reason)

I studied the theory behind data modelling whilst at University, but for the past 9 years, my approach has pretty much been do it this way (or that way) because it ‘feels’ right. If I had to put a formal qualification on my natural data modelling tendancies then I would have figured it was probably somewhere around 3rd Normal Form. Except that some recent work has caused me to reconsider- or at least hit the textbooks again to verify what level I normalise to.

First of all I had to compare the way an application was storing its data compared to how it used to in the old version. My first reaction was that the new way was simply correct and that is how I would have modelled it (in fact I could find an example of doing exactly that in one of my models). Imagine my surprise when I find out that the vendor’s model is actually in 5th Normal Form. Wow, so my gut feel approach actually skips over 3rd, BCNF and 4th and goes straight to 5th. Cool.

Except that now I have just finished reviewing a guideline document and apparently the simple act of adding the effective_date, expiry_date type of information that we all do means that I am now creating a 6th Normal Form. Even more cool, I am doing really smart things, without really knowing about it.

Except that when I go back to my textbooks and read up on these really rather academic discussions I am not entirely sure that I have even addressed all the subtleties of 5th Normal Form. Then I can’t even find any mention on 6NF in the text book (it was only published in 1994 after all, so there have been a number of advances since then) and have to hit the internet to find out what it means. Again, I find that I am not sure if what I do is actually meeting the subtleties of 6NF.

So I come to 1 of 2 conclusions:

  1. I am not normalising to 5NF or 6NF in which case I am being sold a bunch of horse…. by the vendors and technical authors who are trying to justify their existence, but don’t actually know what they are talking about. This is highly likely
  2. I am normalising to 5NF and 6NF and the academic understanding behind what I do is beyond my comprehension. I just understand that I do what I do

Option 2 is quite possible, after all, it is highly unlikely that Wayne Gretzky could explain why he was so much better at doing what he did on a hockey rink than anyone else, likewise for Pele on a football field or Picasso in front of a canvas etc etc. In which case I must be afflicted with a data modellers equivalent of the Photographer’s Eye. Which may well be the case- I have worked with people (1 person in particular) and we just saw the solution to a modelling problem in the same way. We couldn’t necessarily explain why we saw it that way, but invariably the solution stood up to whatever tests were thrown at it and eventually got accepted by the naysayers.

So, to bring this back to modelling in practice; if you find someone who is good at data modelling (and don’t take their word for it, get a few other data modellers to verify their work) they are probably doing many, if not all, of the things to create a model that is in 5th, 6th or even more Normal Form. They just won’t know why they are doing it that way. But, and this is crucial to the practicality bit: when I created the temporal aspect of a database, I occasionally had to go against every principle in my body and not only create effective_date and expiry_date but also an active_flag. Now, obviously there are situations where a row can be effective, not yet expired but not active- but in my case I had to add the extra attribute because the user could then simply click a check box in the UI and a trigger in the DB would populate the relevant date columns. So was I in 6th NF or not had I not even made it to 3NF? At the end of the day it didn’t matter, because the app met the user requirements, the developers could code against it and the DBA could manage the tables. And all of those issues combined is the true test of a practical data model

A club even Groucho would want to join

This may come across as sentimental but every now and then you come across a group of people that makes you realise Groucho’s famous quote on clubs (“I don’t want to belong to any club that will accept me as a member”) gets it so right that is scary. The Camabridge Ring is a group of scientists who actually think about the real problems that the computing world faces. Note that these are not necessarily computer scientists- instead they come from a wide range of backgrounds and are far more concerned with the underlying and fundamental issues that need to be solved to move computing forward.

In some ways the vision in your mind of this group is probably correct- they are largely male, a cross section of ages from mid-20’s bright eyed things to 90 year old duffers and generally graduates of Cambridge University. But the humour is gentle, self deprecating and largely doesn’t translate outside the group.

The work that I do is far removed from the theoretical thoughts that the Cambridge Ring discuss, but that doesn’t mean that I would love to sit down and simply be in the presence of these great minds. So how does Groucho apply here? Well, any society that I can join is far less interesting than the Cambridge Ring- so I wouldn’t want to be a member, but there is no way that I could ever contribute to the Cambridge Ring, thus excluding me from ever being a member of the group I wish to join.

Oh, and if you think that the Cambridge Ring is entirely removed from the commercial world- a recent estimate on the combined net worth of the 100 or so members: $5billion. OK it is not Bill Gates, but that is not bad for a bunch of academic dons. And if you want to read another appreciation of them PC Pro has published and article

IT Architecture in the Blogosphere

Say what you will about Google and its adsense, it does throw up more than the average number of ‘interesting’ links at the top of Google Mail. I stumbled across a link that looked incredibly interesting, but as a means to prevent me from being sued for libel I will refrain from publishing it. My recent post on Redefining IT Architecture shows that I am completely unsure whether I do any real architecture work- most of my work is in the design world- but from time to time I do define the rules that a solution (and very occasionally an enterprise) uses to implement a system so I think that I can call myself an architect.

But after an hour of so of clicking and following links in the IT Architect blogosphere I am confused, depressed and not a little angry at the content that supposed architects are publishing. Now I admit that most of my professional life has been in the data sphere, but I also spend a significant amount of time in the ‘business’ world- talking about governance, processes and skills and resources. In my entire experience I have never had cause to wonder about the details of language used to implement a system, the development methodology or any other deep technical issue.

Now this may simply be a case of the same word being used in multiple situations- after all, we as the IT profession stole the word from the construction industry, so we can’t really complain- but surely there is a difference in the technical skills, the soft skills and the background and experience between someone attempting to define how an multi billion dollar company sets up its governance structure for Information Management and someone implementing an (admittedly possibly large scale) application using the latest technology.

And furthermore (yes I am fully in rant mode now) I also recognise that there are significant skills required to build these systems, but according to the scale of systems complexity (I’ll insert the link to that later) an organisation is one of the most complex systems it is possible to consider; far, far more complex than any engineering design, simply because it has to deal with that most unpredictable of things- a human being. So, is the work that a technical architect does less valuable than an enterprise architect? No, because like all good constructions you need the foundations to be sound, but honestly, none of the major challenges that I face  are in the traditional technical domains (application, data or infrastructure). Far, far more complex to understand is the activity associated with working out what users really want, how we ensure that they use the systems that we build for them correctly and getting buy-in and belief from non-IT people who just see us getting in the way.

As the MD of one of my former employers would frequently say: “Guys, this is not a technical problem”. Unfortunately for that company the tech heads ruled and it finished up going bust (after, I hasten to add, I had got the heck out of Dodge as I could see the writing on the wall). OK, IT Architecture is not going to go bust, but we risk losing a lot of credibility by solving the wrong problem.

And now I need to shut up before I offend more people than normal.

Redefining IT Architecture

Every now and then someone comes along who has such a compelling message and a such a fantastic delivery of that message that the very fabric of your intellectual foundations get shaken. Jan Hoogervorst is just such a person.

Until now I looked upon IT Architecture as a largely diagram based activity- trying to stay away from the As-Is and focus on the To-Be- but fundamentally it drew pictures and was descriptive. Oh, how naive I was: In an hour long presentation Dr. Ir J. A. P. Hoogervorst turned my world on its head.

According to the presentation, and both its content and style of presentation lead me to believe that this view is entirely correct, Architecture provides normative guidance for design. That expression needs explaining: Architecture is the prescriptive process the defines (not describes) the rules by which changes may be made to an existing design to create a new design. Practically speaking then, Architecture is a consistent and coherent set of principles and standards that prescribe how a system is to be designed.

The consequence of taking this approach is the realisation that Architecture and Design (at least in the IT world) are fundamentally different concepts, and that most of the time, the activities that Solution Architects do are actually design. This is not a problem (in fact, a lot of the ‘fun’ part of my job is the design part, the architecture part makes your head hurt) as long as the managers, leaders and other stakeholders recognise what it is you do. And there are actually some good reasons for having one person act as Architect and Designer: the relationship between Architecture and the Design created using it is naturally very close. And as most organisations do not have well defined Enterprise or Solution Architectures, the poor chap (or lass) doing the work is probably going to be switching hats between Architect and Designer many times a day.

As the maturity of Architecture increases, the distinction between the Design and the Architecture will become clearer. But by then there will be a new IT paradigm and we will all be some other sort of engineers.