What a cool idea Google has: bucket testing.

The company regularly tests out unannounced new features across a small percentage of its users to see if they are worth implementing on a wider scale (this is called bucket testing).

As a user experience professional, I remember my job isn’t just designing user experiences. I’m not simply an advocate behind company walls, lobbying for our users and their needs, pushing back when requirements or features get “too functional,” in the sense that they’re so functional they alienate and introduce hard-to-use UI.

My job is also to user test our features and designs, to get feedback as part of the design process to make sure our assumptions are validated by real, living and breathing people. So, what Google does is cool… bucket testing. It’s food for thought: kind of a shotgun usability test, automated instead of personal. One drawback I see is I’d be basing design decisions on self-reported claims about their needs instead of observing what they’re actually doing. All sorts of people write about this distinction, notably Jakob Nielsen and another usability guru, Steve Krug.