What do you get if you mashup Wikipedia, Google Documents, Yahoo! Pipes and Yahoo!/Google Maps?

Tony Hirst has a wonderful post on his blog showing a mashup between Wikipedia, Google Documents, Yahoo! Pipes and Yahoo!/Google maps:

"So to recap, we have scraped some data from a Wikipedia page into a Google spreadsheet using the =importHTML formula, published a handful of rows from the table as CSV, consumed the CSV in a Yahoo pipe and created a geocoded KML feed from it, and them displayed it in a Yahoo map."

I highly recommend checking it out as it gives a good overview of some of the infrastructure Yahoo! and Google are making available to all of us. Here is the end result.

Chrome updates offer a HUGE exploit window

I'm loving Google Chrome, the new browser offering from the Googleplex. However I was very worried this morning when I read that they have a proprietary (ie. not open-source) update infrastructure which downloads updates to the user's browser without telling them.

This is a major security flaw, and I'm frankly surprised by it - perhaps it is intended to be a short-term fix whilst the browser is in Beta, but many Google products (e.g. GMail) seem to never leave Beta. The real problem with the approach they've taken is their lack of transparency about security flaws and the fact they leave users in the dark:

"Users do not get a notification when they are updated ... When there are security fixes, it's crucial that we update our users as quickly as possible in order to keep them safe. Thus, it's important for us to not require user intervention."

That sounds very noble, but the problem is that it leaves users unable to reject an update as well. That means that if Google decides they want to change the browser to slap ads in your face every 30 seconds you can't stop it from doing that (you can always go back to another browser - assuming competitors still exist). More importantly it means that if some criminal did manage to take control (however briefly) of Google's update infrastructure then they could install whatever they like on your PC.

In fact any update infrastructure allows this to happen, and many are vulnerable to it. Google's not particularly different, and with the attitude they are displaying at the moment they are more likely to be exploited - because the people that could help them find problems are not being given full access to the software updates (the latest security changes do not show in the open-source changelog). Of course, not updating your software leaves you even more vulnerable.

A recent study into the update problem for browsers (found via ESJ) looked into the problem of users who do not update their browsers:

"We believe the auto-update mechanism as implemented within Firefox to be the most efficient patching mechanism of the Web browsers studied. Firefox's mechanism regularly polls an online authority to verify whether a new version of the Web browser is available and typically prompts the user to update if a new version exists. With a single click (assuming that the user has administrative rights on the host), the update is downloaded and installed. Just as importantly, Firefox also checks for many of the currently installed Firefox plug-ins if they are similarly up to date, and, if not, will prompt the user to update them. Opera's update mechanism is essentially the same procedure as a manual download and re-installation of the browser.

While Microsoft’s operating system auto-update functionality encompasses the Internet Explorer update mechanism even if the browser is not in use, the fact that patch updates (for both Internet Explorer 6 and 7) are typically only made available on a monthly basis means that updates are released less frequently (when compared to Firefox), which can result in a lower short term patching effectiveness.

Based upon our findings, we strongly recommend that software vendors embrace auto-update mechanisms within their products that are capable of identifying the availability of new patches and installing security updates as quickly and efficiently as possible - ideally enabled by default and causing minimal disruption to the user. We also recommend that these same auto-update mechanisms are capable of alerting the user of any plug-ins currently exposed through the Web browser that have newer and more secure versions available.

...

Given the state of the software industry and the growing threat of exploitable vulnerabilities within all applications (not just Web browsers), we believe that the establishment of a ”best before” date for all new software releases could prove an invaluable means to educating the user to patch or ”refresh” their software applications. The same ”best before” date information could also be leveraged by Internet businesses to help evaluate or mitigate the risk of customers who are using out of date software and are consequently at a higher risk of having been compromised."

It is worth noting that the researchers do not recommend a solution that involves automatic updating of the browser without user permission - rather they would prefer to let users know the "use by" date of their software in order to inform them that an update is required (see the paper for more details on how they suggest this could be done). Also their research paper notes that "Access to Google’s global Web server logs enabled the authors to provide the first in-depth global perspective on the state of insecurity for Web browser technologies," so it's not like some part of Google was unaware of the study.

Personally whilst I will continue using Chrome for some of my browsing I will be re-evaluating my continued use of it based on how Google responds to these sorts of security issues. I recommend you do the same.

So that's what backchannels are!

Ever had that disconnect where you learn a new concept, but don't have a name for it, and you come across a new term but don't understand the concept, only to realise the new term is actually describing the new concept?

That's happened to me recently with the concept of using Twitter + Hashtags at conferences to keep up with what's happening and the term backchannel. It turns out they are one and the same (and now you know too!).

So, why bother blogging about it? Or more importantly, so why should you care?

Good question. The interesting thing about backchannels is that they not only enrich your conference experience, but they also help you process the sessions you are currently in.

So how does it enrich the experience?

• You get to find out what's going on in other sessions.
  At Google Developer Day 2008 this helped me work out that it was worth switching streams when I found the OpenSocial one was not meeting my expectations.
• You get the benefit of others' viewpoints.
  We all know that sometimes the biggest "aha!" moments around a conference are during those in-between sessions conversations with other participants. Backchannels allows you to get some of those during the session.
• You get extra content.
  When twitterers know which link the presenter is talking about, or do a bit of impromptu research in the midst of the session, then you get the benefits of finding out more about what is being said and can often be downloading the code being discussed during the session.
• You get to give too!
  Each of the above benefits is because other people in the backchannel are sharing what they see, know and feel. You get to do the same too, and this reaps benefits in terms of reputation and new contacts. At both CodeCampOz 2008 and Google Developer Day 2008 I met new people who I would have probably not gotten to know except through the conference backchannel. Be aware that the conference speakers may check out the backchannel after their sessions, and set the tone of your comments appropriately.
• You've got something constructive to do in boring sessions.
  You can always surf the web, but that takes you out of the flow of the conference, being able to participate in a backchannel discussion means that you can sometimes re-assess the usefulness of the current session or at least plan which pub to meet everyone with afterwards more easily!

The other element is more about how you process the sessions in order to internalise the knowledge and practices being shared. Sometimes the fact that you are researching topics in the midst of the session means you can multiply your understanding about a topic at the moment when you're most interested in it.

I certainly found at CodeCampOz that some of the sessions might not have made sense to me were I not pushing myself to find relevant links about them in order to add them to tweets sent during the sessions (yes, backchannels can make you competitive!). It does mean you are paying less attention to the speaker, but sometimes that would have occurred anyway (like when something goes over your head, is boring or relates to a question you already know the answer to).

However, I don't think that I would have grokked the advantages of backchannels had Craig Bailey not pushed me to use Twitter before Code Camp Oz 2008, and then the organisers of that conference not provided free power and wireless internet access. I highly recommend you give it a go yourself.

Why not drop me a comment here if you find it has worked for you, or alternatively if you thought it was a waste of time?

Google Developer Day 2008 (Sydney)

I spent yesterday at the Australian Google Developer Day 2008 at Wharf 8 in Sydney. It was an interesting event to attend and I met up with some great people. Like most events there were both positives and negatives:

• Positives:
• Good turnout, lots of interesting people to meet and chat with (big shout out to delic8genius, dcw303, stuandgravy, garthk and Dr Yogesh).
• Great food and drink.
• Great keynote session with cameos from a bunch of different developer advocates and product managers who gave us interesting peeks at their developer products.
• Lots of nice new additions to Google Maps API, the Google Earth API was especially sexy.
• The Android keynote demo was cool, especially the Compass feature that allowed StreetView to track the physical movement of the mobile device.
• The App Engine sessions got me interested in using IronPython as it has a syntax similar to Ruby and might even be compatible with App Engine (the App Engine guys weren't sure about this one).
• The event was free (as in beer).
• Negatives:
• No caffeinated soft drink. Negates all the good food and drink. Coffee lovers were looking for better sources of that beverage too. It would have paid off to hire a decent coffee stand for the day and allowing delegates to pay for it themselves.
• Nobody mentioned how to access the wifi network (I talked to the events manager and Alan Noble was supposed to in his keynote speech ... hehe).
• Audio bleedover between sessions, the Blue and Green rooms shared a flimsy wall that allowed sound through, particularly annoying when in the back of the Blue room.
• Presentations became less interesting as the day wore on. The first OpenSocial session was a complete bore, basically a project report.
• I missed the code labs, but up to that point the last 2 sessions had been both very short, using only half the time allotted. Remember the Milk forgot to turn up to explain how they use Google Gears, so that explained why their sessions was short. I spent that time tweeting and mingling, but would have preferred more content.
• It's all Beta. Many of the most interesting things mentioned are still unavailable, that's probably unavoidable at this sort of event, but it was a bit galling to hear yet another "this is only in prototype" explanation.
• It started too late, 10am is a ridiculously late time to start, mind you, I enjoyed playing Halo 3 on the XBox 360 they nicked from Google's Sydney office. It also ended late at 6:30pm, which meant I missed the last 2 sessions (mostly code labs). I might have stuck around, but my wife had a prior commitment I needed to be home for.
• The event took a whole day (as in working day), but really only offered half a day's worth of content.

You can check out the #gdd08au tag for the tweets from the conference. I was a bit lonely there for a while, but it eventually took off.

OpenSocial was the big disappointment for me, the material we were shown in the sessions I attended covered client-side gadgets, hardly an exciting line of development especially given the increasingly popular view that most social mini-applications are no more than viruses. Two thirds of the first session was spent looking at differences between v0.7 and v0.8 - only after we were all bored to death did the presenter ask how many were already using the API, and then only 3 hands went up! Perhaps I should have hung around for the Apache Shindig session to see server-side details, but the first session was too boring for it to earn more of my attention.

The App Engine sessions were the surprise bonus for me. Firstly there was the news that Python is only the first language they intend to support (star this support issue to see C# supported). Then I realised that Python reminds me of Ruby, a lot. Nice.

The best bit about App Engine was that we had good speakers, Tom Stocky (leader of the product managers that work on Google's developer products) and Brett Slatkin (an engineer on the App Engine team). They made it very easy for us to understand what App Engine was all about, and particularly why we should care about it (it makes it easy to start an app for free, and then scale it easily if it takes off).

The discussion about App Engine's Bigtable datastore was gold, especially the coverage of some basic issues that frequently catch out relational database developers (namely there is no count kept of the table). This really brought home to me the smarts behind data shards (which I might blog about soon) as well as providing us with the 'right' way to handle problems like global counters and list paging. This is core to Google's success and they obviously have some great lessons to share with the developer community about how to build scalable web applications.

Tom and Brett had very strong messages (repeated often) about how much they value the privacy of your data and code, and an awareness that for App Engine to take off they must make it as easy for people to get data out of the system as into it. Clearly if you design your applications to fit App Engine you will find it relatively easy to get started on any distributed LAMP hosting platform, although you will need to build some of the infrastructure bits for yourself - like Bigtable. However the premise is that if you build up your application on App Engine then you avoid the sort of scaling problems Twitter has experienced.

I missed most of the first Gears presentation (I was stuck in OpenSocial's yawn-inducing first session), but the 10 minutes I did catch was interesting. They made the point that Gears was all about unlocking the capabilities of the client machine, not necessarily working offline (although Google Docs seems to support this just fine!). This includes utilising the client's power to process data (searching, sorting and indexing) and accessing client information (such as geolocation information).

One great question was how was it different from Adobe Air, and other than being open-source, the main difference is that Gears seeks to enhance HTML 5 and JavaScript, Air is creating a separate (proprietary) platform. In this sense Air is pretty similar to Microsoft's Silverlight model.

The MySpace presentation on how they have used Google Gears was very interesting. They handle roughly 160 million messages a day, fortunately they do not have Twitter's distributed messaging model, but it still is a huge number to handle. Each of their physical databases handles about 1 million users, with approximately 1 Terabyte of data in each. A big issue for them is how to enhance the messaging experience for users with large numbers of messages (some have hundreds of thousands). Offering searching and sorting is an obvious feature, but one they cannot easily handle given there large volume of users and messages.

MySpace's solution was to offer Google Gears to users with over 5,000 messages (soon they will lower the limit to 2,000 messages). For these users Gears could locally store their messages, indexing them on the client machine and then enhancing the (online) website so that searching and sorting could be offered. For these users the page data would be drawn from a mix of their standard online datastores and the local one - providing much faster response times, and putting less load on the MySpace servers. They are also looking at how to help users with large numbers of friends (some have 1 million+ friends!)

However implementing Gears was obviously not a trivial change, and I would hesitate to recommend it unless:

• It was inside a corporate environment (where you have strong control of client machines); or
• There was a very well funded business case for moving processing off the server architecture; or
• The functionality of offline access or geolocation was key to your application.

Remember the Milk was going to turn up for the second half of this sessions, but for some reason were not there (and AFAIK no announcement was made about their absence - maybe they were busy at Apple's Worldwide Developer's Conference?). They are an Australian startup that offers a free and simple To Do list function for users anywhere. They have been quite popular globally (half a million users) so it would have been interesting to see what they did different to MySpace (update: they have blogged about using Gears and it sounds like they use it for offline access, would have been very interesting).

Overall I'd give this a 7/10, and I would definitely go again, but I expected Google would do better. It goes to show how hard it is to run a good conference, especially when expectations are so high.

UPDATE: Got the hashtag wrong, it's now correct.