Last week I attended a fantastic and fascinating QASIG talk by James Whittaker of Google. The talk was about how test engineering is done at Google, but it really wasn’t about testing at all. It was about quality. As James points out, people care about quality. They don’t really care about testing. James listed 10 things that Google does to dramatically increase quality, and few of them are what we would call testing. James laid out a vision of where computing is going, and where Google already is. The tools he describes are all open sourced, by Google. At the moment, they are all aimed at web apps, but Google is seriously looking at the Android market as well for these system interventions. You can see the talk here:
- Video: James Whittaker at QASIG Seattle
- Slides: James Whittaker’s slides for the QASIG Seattle talk – you will need to follow along with the slides, since they don’t show on the video
Certainly, not everyone can use the approaches that Google is using, especially if they have legacy architectures that are not amenable to those types of approaches. But lots of startups could use these approaches. I bet lots of other organizations are using similar approaches. It’s about scalability. When you are releasing a new version of a web application every day, or multiple times a day, and that application is used by hundreds of millions of people, you need to reduce the number of bugs that get released to near zero and very rapidly roll back when you do find any bugs. This is what Google does. His book is available online here: How Google Tests Software book - it’s not out in print yet, but available in Safari Rough Cuts in eBook form.
James mentioned that Google uses uTest, crowd sourced testing startup, instead of dedicated testers, for some of their projects – a fantastic idea. James also talked about a bunch of Open Source software from Google in the talk:
- Test Analytics (ACC) Tool – replaces test plans
- Quality Bots – compare web page layouts, even across browsers – much is done automatically, but it does forward to humans when necessary