Lean Coffee @ Agile Ottawa

Last night I facilitated our first Lean Coffee meetup at Agile Ottawa. Thanks again to all who made it out, and to You.i TV for the venue, food & drinks! :)

We covered a lot of interesting topics:

  • How to sell executive sponsors on addressing technical debt?

  • How to transition slow adopters from waterfall to agile when others have moved on?

  • Wardley Maps - how they can be used to decide when to do Agile.

  • How to estimate user stories?

  • How to engage an overloaded Product Owner more with the team?

  • Adopting [agile] with non-agile clients.

  • Scrum & Continuous Delivery - are they compatible?

See the pictures below for key points covered.

We also had some great feedback on the session:

  • Interesting discussions (x6!)

  • Loved it, do it again (x4!)

  • Well moderated (aside: thanks! :)

  • Got a lot of good ideas

I’m already looking forward to our next Lean Coffee :)

Here are the pictures I took of our whiteboard:

That's it, I'm moving to Canada.

As you may know, last year we moved to Canada… I'd been threatening to for years, and finally got around to it ;-).

A lot of my friends from London wondered why we'd want to move even further north, to which I'd reply: "but we're not!". Just so I don't have to explain it anymore: most of the population in Canada lives within 1-200 km of the US border. We don't all live in igloos, keep penguins, or wrestle polar bears during our lunch breaks. And yes, Ottawa really is further south than London. And sunnier. And … well, if you still don't believe me, look at a map.

Over the past year I've had the chance to do a few home renos, and design & install a 12v renewable energy system (used every day!). As part of that I've done some more LED research, and I'm pleased to see stores like Home Depot & Canadian Tire carrying decent LED stock these days.

In the summer we had a garden. No, I mean a real garden, not some piddly 2 foot square plot out the back door. We still have food from it. That kind of garden.

This winter we've been doing a lot of XC Skiing (Gatineau Park has great trails!) & some skating on the Rideau Canal to stay in shape. It really is strange not to be cycling to work every day!

More recently I've been playing around with a few technologies I've always wanted to learn but never had the time: Node.js & HTML5 (notably canvas), and Android. I finally got around to moving a few of my Perl projects to Github, and updated my website: www.spurkis.org. I'm also thinking of starting up a new business.

So I'm going to try to get this blog back up & running with a few posts on these, and see where it goes from there!

Trying out Google Apps for mail

So I've bitten the bullet and moved my mail into the cloud - in particular, Google's cloud: Google Apps.

For several years, I've figured there wasn't much point in running my own mailserver anymore. Almost all hosting companies these days can provide the basics (smtp, imap, spam filtering, backups). All was going well until I was forced into it again last year when the hosting provider I had just moved to screwed everything up - I needed a solution quickly, and so I ended up sharing a Slicehost with a friend. This worked out well - Slicehost have a great product offering! - but it still meant more maintenance headaches for me... configuring spamassassin, adding accounts, hearing complaints about the slow web interface, worrying about backups or accidentally taking the server down...

Moving to Google Apps (something I never thought I'd do 2 years ago!) has been in the cards for several months now. They offer everything we need and more for free. It will take some getting used to - for example, I inevitably end up joining umpteen mailing lists, and like to filter them... until I came across karltk's post on Filtering on List-Id in Gmail I was really second-guessing my decision... there's also the oddity in 'archiving' mail (ie: moving it out of the inbox) and giving something a 'label' (ie: sticking it in a folder). These are arguably more advanced ways of working with email, but you still need to change the way you think, which I'm sure will confuse some users.

The saving grace is that you can use the same old mail reader you've used for donkey's years, and use it like you always have. What's more, the web-interface is great.

The only thing I'm worried about is uptime... hopefully we won't see many more outages in the future. Still, it is a free service after all.

Highlights: FOWA Expo 2008

Nika and I had a great time at FOWA London 2008 this past week. Carsonified wanted to create a conference where people could learn, be inspired, and meet others... I think they succeeded on all 3!

If you missed the conference it's worth pointing out that the talks are being made available online! Highlights for me:


This is the main reason I went to FOWA, and I'm glad I went. I had the chance to meet some really interesting people and find out about what they were working on. The Expo area was laid out well - not too crowded, and easy to browse around & hear about people's products. There was a good turnout at the evening sessions afterwards.

General Impressions

  • It's great to see commodity computing coming on strong - I think it's starting to hit the mainstream now, a lot of the people I talked to were either already using or interested in using commodity services. Indeed, Facebook's founder mentioned he would be interested in using such services for new projects, and commented on how much the landscape had changed since he started the company.

  • Everyone's getting more & more interested in mobile devices, especially since the release of the iPhone 3g (think: app store) & Android

  • Social computing (if you will) is still on the rise. Everybody's still pissed off about walled gardens & the lack of wide-spread adoption of open standards like OpenID, oAuth for sharing data between social networking sites (but hey - adoption of OpenID is growing, so we're getting there). And OpenSocial is still being pushed by Google as part of the solution.

  • How will the credit crunch affect us? No-one knows yet, but some (like Tim Bray) fear the worst, and yet still see opportunities. Gavin Starks' take on this was particularly interesting to me, even if it is unpopular: maybe less consumption is actually a good thing.

Technologies to watch

  • AMEE

    Gavin Starks' company, AMEE, has a pretty ambition goal: to collect energy consumption data about, well, everything on Earth. To enable this, it will rely on users to voluntarily share consumption data with them through their web API. Privacy will be protected at source. I'm not sure how many devices are able to capture this kind of info at the moment, but it's certainly an interesting challenge... and imagine what could be done with this kind of data! Promoting energy efficiency, singling out inefficient products, monitoring global consumption demands to analyzing what happens after you charge & unplug your phone... making people conscious of their consumption patterns can be both painful (I use that much?) and rewarding. Certainly something that needs to happen.

  • XCalibre's Flexiscale

    It's great to see a strong competitor to Amazon's EC2 service! I've signed up for a trial and can't wait to try them out.

  • Sun's Zembly

    I spoke to Todd from Zembly at great length... their utility computing platform reminds me of what we what we were trying to do with Zimki, and with what James is currently doing with Reasonably Smart. Though Zembly is more specialized (it focuses on building services & widgets, primarily for social networking apps), I think the platform has a lot of potential and I'm keen to give it a whirl.

  • Fluidinfo

    I spoke to Terry from Fluidinfo about the way they're trying to redefine storage. Web-based API's, and bizarrely, open access to CRUD that could actually work. Would be cool to see a demo of this.

  • Frog

    I spoke to Gareth from FrogTrade about their product... a collaboration tool that could easily be used as the driving component of company intranets. Again, would love to see a demo. They're currently focused at the education sector, but if they break into other markets it could prove a strong competitor to Microsoft's Sharepoint, wiki's, and similar products.

Interesting Talks

Here are the talks that stood out for me, in order of appearance:

(will link to video as & when available.)

If you had time to watch only 3, I'd recommend: Simon Wardley's, Gavin Starks, and Kathy Sierra's. Watch them all if you can ;-)

The only thing I think the organizers could have done better is in the acoustics of the venue - curtains don't keep sound in or out very well, and there was a constant high-pitched whine of (I'm guessing?) the aircon that was a bit distracting. Certainly not a show-stopper.

Kudos to Carsonified & all the presenters for putting together a good event!

Test Anything: TAP & HTML

I've just released TAP-Formatter-HTML to CPAN:


It's meant to make reading test results easier, giving you a visual summary of your test suite and letting you drill down into individual failures (which will hopefully make testing more likely to happen at your organization ;-).

It's based on Yuval Kogman's Test::TAP::HTMLMatrix which though useful, has a few limitations imposed on it by Test::Harness.

Over the past few years, after the evolution of the Test Anything Protocol (TAP), Andy Armstrong et. al. have produced a sorely-needed replacement for Test::Harness called TAP::Harness. It comes with a great little command-line utility called prove which makes it really easy to run your tests.

With TAP::Formatter::HTML you can now produce your test reports in HTML:

% prove -m -Q --formatter=TAP::Formatter::HTML >output.html

Give it a whirl & let me know what you think!

GPX editor

I've been hacking on a GPX editor recently... it started off as a throwaway project to learn Catalyst, DBIx::Class & jQuery, but now it's starting to get close to usable I'm tempted to keep going with it...

If nothing else, I've certainly learned a lot. In addition to the above, it uses Geo::Gpx to do the GPX parsing, JSON to translate between JS & Perl, and draws on a few jQuery plugins (mainly from the UI suite, but also the form plugin and a completely reworked version of jqueryFileTree.

The functionality as it stands is pretty limited - you can upload a GPX file, browse through its tracks, segments & points, see where they are on a map, and edit/move them. Waypoints & routes are not yet supported mainly because I can't get them off my GPS over a simple USB connection, but if all goes well I'll add them in later. And there's a bug/feature list longer than.. uhh.. a really long thing.

But its certainly getting there. And there have been a few useful spin-offs already (I've just released Geo::Google::PolylineEncoder & Test::Approx to CPAN, and I'll try & package up jquery.tree.js & release that too).

Feel free to try it out, and let me know what you think!

On Hadoop, MapReduce, Databases & Mobile Agents...

One of my friends pointed me to Tom White's Disks have become tapes post over the weekend... very cool idea - I remember coming across Hadoop several years ago in one of my searches of clustering technologies. I must admit, I don't think the simplicity (and moreover, the effectiveness) of MapReduce really clicked in my head back then. It certainly has now!

In Tom's post, Learning MapReduce, it struck me that there is a parallel with mobile software agents - obviously not from the intelligent, adaptive, social agent side, but more from the simple idea of bringing the code to the data. Aside from the idealistic dream of emerging artificial intelligence, that was one of the practical goals behind the Agent.pm experiment. And that's exactly one of the Hadoop design assumptions:

    “Moving Computation is Cheaper than Moving Data”
    A computation requested by an application is much more efficient if it is executed near the data it operates on. This is especially true when the size of the data set is huge. This minimizes network congestion and increases the overall throughput of the system. The assumption is that it is often better to migrate the computation closer to where the data is located rather than moving the data to where the application is running. HDFS provides interfaces for applications to move themselves closer to where the data is located.

Of course, you could argue that database stored procedures do just that, but you'd be missing the point. There's always a tradeoff as explained in the MySQL manual:

    Stored routines can provide improved performance because less information needs to be sent between the server and the client. The tradeoff is that this does increase the load on the database server because more of the work is done on the server side and less is done on the client (application) side. Consider this if many client machines (such as Web servers) are serviced by only one or a few database servers.

So you're just shifting the problem. What I like about MapReduce & Hadoop is the new approach to solving the problem. It's really interesting to see the idea of mobile code being re-applied here.

While I don't think MapReduce will replace databases & file storage systems (and they don't claim to), I do think technologies like Hadoop are worth investigating, and potentially adding to the scalability-on-commodity hardware arsenal, alongside the likes of MogileFS, memcached, drbd, heartbeat and LVS.

Innovation, Commoditization and Management in IT

Simon Wardley recently gave a talk at the Butler Group on innovation & commoditization of IT.

I've talked to Simon a number of times on the subject, and indeed, he's talked at a number of conferences on it. What I find particularly interesting about this one is his foray into the management of organizations. I quite like his S-curve visualization of the type of activities an organization goes through as they progress from the innovative and transition to the ubiquitous. He makes the key point that:

    "[An organization] need[s] to use different methodologies for different stages of an activity's lifecycle."

Interestingly, I've seen this a number of times in IT startups I've worked for - as they become successful and grow, they always change irrevocably. This is simply because the nature of their activities change. From my experience, here's a quick overview of a typical startup building a SaaS product going through Simon's 3 stages:

  • Innovation: They typically start off with the bare minimum needed to build an innovative v1.0. Half of the time, the founders are helping to build the product, if not then you can bet they're working directly with those who are. Management structure is pretty flat, and development methodology is typically Agile (or non-existent).
  • Transition: As the product grows & becomes more successful, the company hires more employees. Different teams are created, and inevitably middle management is introduced, along with some sort of formal management methodology. The new management structure can take many attempts to become establish, eating up several months to several years. In some cases Waterfall may replace Agile. During this time some of the older employees will become fed-up with the new bureaucracy and leave.
  • Commodity: The product line is well established, highly used, and doesn't change much. Overall management structure is well-established and doesn't change much. In general, the cost of change is high, and the business' goals are focused on getting the most out of their existing products. Management will likely have introduced PRINCE2 to start running larger projects for their clients.

I believe the Transition phase is the most challenging for a startup - it can make or break a company. For example: one growing SMB I worked for was in its transition phase and had recently adopted an overall organizational management structure that was good for its existing activities, but unfortunately stifled innovation and seriously threatened the company's future. It was only through an external source of innovation that the company succeeded.

Needless to say, at large companies this lesson is potentially gold dust. Indeed, some have already learned - as Marissa Mayer, Google's VP of Search & UX, noted in an interview with Fortune Magazine in 2006:

    "Most of the teams at Google are three to ten people. ... They operate like small companies inside the large company. Google is a lot like managing a VC firm, because you're placing bets on different teams. Our organization mirrors the Internet. It looks more like a network than a hierarchy."

Certainly a valuable lesson for all those employed in IT & involved in change. Which is pretty much all of us.

Sun to buy MySQL - why?

Wow... I didn't see this coming! (Unfortunately the original announcement wasn't loading for me). From an Open Source POV, it makes sense as Sun has long been a supporter of & contributor to OSS. It's just strange to see what I picture as essentially a hardware company buying a database company... And besides, if they had IPO'd I'd have happily bought shares ;-).

So my question is: where are the competitive advantages for Sun in this move? MySQL AB is a profit making company, so there's one. It's also an Open Source product, so there's another. But both are tenuous IMO - not enough reason to buy. Being a hardware company, I'm wondering what their strategy is with this... Much like Apple and their software - it really exists to drive sales of their hardware. How do Sun plan to benefit from this?

I suppose they could fine-tune MySQL to run on their hardware & Solaris better than any other platform to try & drive sales... And maybe then they could create a solution to the industry's HA woes using a commonly used product that doesn't involve re-inventing the way data is stored. (Or maybe I'm just smoking crack ;-) ).

At any rate, it'll be interesting to see where this leads - more for Sun than MySQL, IMO.