Architecture at the BIG end of town

Dan Pritchett on EBay's attitude towards architecture:

“I now need 3 mid tier boxes just so that we can have this widget on this page”

“One of the challenges that I often face with the product organization is that they will see a cool feature on a site that is parallel to our industry but they really want it. And it's like "they are doing this" and that's ok, but they are doing 500.000 page views a day and you want to put it on a page that's doing 300 million page views a day, and that involves 5 database round trips, in order for us to implement that feature so you are talking about adding 1.5 billion SQL executions a day to a database. A host that can do 600 million host a day, I now need 3 mid tier boxes just so that we can have this widget on this page. These are the kind of problems that you face as an architect at EBay.”

They also have an interesting definition of a "high-volume" page:

“(if) you are a developer at EBay you'd better know how to do a 4 million page views a day page”

“Another cool thing is that what a lot of organizations consider a scaling problem, we don't assign architects to because it's such a trivial volume of data that we expect the developers to be able to handle it. The 3 or 4 million per day per page site is just something that we will apply our pattern to. You don't need approvals, just go build it the way it is supposed to be built, we know it will run. There are people spending hours of their day trying to figure out how to get the 4 million page views a day. To us it's a cast aside problem, you are a developer at EBay you'd better know how to do a 4 million page views a day page.”

This is a good insight into the sorts of problems a truly globally popular site faces. Other problems that they have are how to maximise “transactions per watt”, a problem most organisations are not even close to facing up to.

On the other hand there is another InfoQ article covering the shift from machine-performance to human-performance which looks at the other end of the spectrum, the average application where it has become more important to maximise (expensive) developer productivity over (cheap) processor performance, coming out of a discussion about the marriage of Ruby (language) and Agile software development. There is an accompanying video in Japanese with English subtitles:

Agile references for project managers

Jimy Bogard has some great agile references for project managers on his GrabBag blog.

From what I've read and experienced of agile processes it is easier to change at the developer's end than the project manager's. Agile software development challenges some of the holy cow's of traditional software projects and it is not easy to find a solution.

Rugby World Cup 2007

The Rugby World Cup 2007 is nearly upon us, and I'm looking forward to a few late nights (~midnight) and early mornings (~5am) watching the Australian games. I was planning on doing it with friends, but the weird hours mean it will probably something I end up doing alone. However it is still exciting for all that.

The Wallabies go into the cup as underdogs, just the way we like it. Most people expect New Zealand to win, but an upset by South Africa, or one of the Northern Hemisphere teams is on the cards - especially France as they will be playing at home.

Anyway, check out this cool Rugby World Cup 2007 ad:

More cool videos on Infini-Nut.

Easy find and replace in SQL Text fields

As you may know, you can only use the REPLACE SQL function on varchar or char fields, which becomes a problem when you have a text field to update (as we often do). In my case I had a text field with values up to 14,000 bytes, far too long to do a CAST to varchar (with a maximum of 8,000 bytes).

The traditional solution to this problem is to either use the UPDATETEXT function in a cursor, with lots of nasty code, or to whip up a small throwaway .NET application to iterate through the database records using the VB.NET Replace function.

SQL Server 2005 introduces the idea of large-value data types, which allows you to specify a data type that can store values up to 2^31 (2,147,483,648) bytes of character, binary or Unicode data (equal to the String data type in .NET). The importance of this change is obvious from the help text on these:

“The built-in string functions that can operate on character and binary data are enhanced to support large-value data types as arguments”

The syntax for the type declaration is:

• varchar(max)
• nvarchar(max)
• varbinary(max)

This allowed me to write the following update statement:

UPDATE ItemContent
SET ItemContent = CAST(REPLACE(CAST(ItemContent AS VARCHAR(max)), '#ff9900', '#8b0b04') AS TEXT)

Much easier than messing around with textpointers and offset values with the UPDATETEXT function!!

KEVIN07 - update

Thanks again to everyone who commented on my original KEVIN07 post. The newsfeeds are full of interesting points of view about the site:

For a start I think it's un-Australian to shorten Kevin Rudd's name to just Kevin. No, it needs to become the truly iconic Kev for him to have a real chance to win the election in 2007.

Anyway, back to the commentary:

Craig Sherborne at the Herald Sun points out the American elements in the site:

“Lashings of American-style red, white and blue, which are also Labor Party colours, create a bold backdrop for video links and a quick, simple manifesto for hot topics from job security to climate change.”

The Herald Sun also wonders where the KEVIN07 t-shirts are made:

“Kevin Rudd's "prototype" campaign T-shirts may have been made in the sweatshops of Asia.

A "Kevin07" T-shirt obtained by the Herald Sun had its label cut out.

A Labor spokesman could not rule out that it had been made overseas. ”

NineMSN have attracted hundreds of user comments about the KEVIN07 website. Most pointed was this one:

“Hypocrisy best describes this lastest stunt. Kevin Rudd is my local member and before he became opposition leader I sent an email for him to address some issues on my behalf...........I am still waiting for a reply!!!!!”

Finally, Lachlan Connor (independent for Victoria) gave us the following hacked image of Kev and his crew:


I'm looking forward to more good parody images coming out over the next week or so ... mind you being parodied and made fun of never did John Howard any harm, so it may all come good for Kev in the end.

KEVIN07 - are Labor out of their minds?

We've hit a new low in Australian political life with the launch of KEVIN07, the campaign website for Kevin Rudd - oh, and the Labor party. It is full of the mock enthusiasm so prevalent in American politics, and the similarities are made more obvious by the use of red, white and blue, the presence of bumper stickers, and the ability to buy KEVIN07 gear ...


In a blatant acknowledgement that they must fight the Liberals on personality alone, the site launch represents the rock-bottom for Australian politics - that is, until we have the Rudd girl battling the Howard girl online (like the yanks do with Obama girl vs Hilary girl). Mind you Kev looks like he's just swallowed a prune in this photo, so he probably needs some cheering up.


The buzz around the office is that this is likely to backfire. It is all too easy to parody and rip something like this off, not to mention the average Aussie's instinctive dislike of anyone so blatantly espousing American qualities and values.


We might like working with and cooperating with yanks, but that doesn't mean we want our politicians to become them!

The buzz in the blogosphere is that Labor has made a bold play, but might just have shot themselves in the foot with this one. I think the idea, using the web effectively, is a smart one - but the out and out cheesiness of the site backfires badly.