Thursday, March 18, 2010

Regression Testing for Usability

I just had a bad user experience at my bank's ATM. I'm not blogging to whine, they're a good bank, but I want to understand what went wrong with the experience and more importantly, the design process that led to it.

I drive up to the ATM and insert my card. New screen, bright and shiny oooooooh.

First new thing, it tells me to cover the keypad as I enter my PIN. Hmmmm, Not sure how one does that while sitting in a car, so I pointed to the sky and yelled, "Hey look, it's the Goodyear blimp" hoping to distract any lurker who might be there to steal my PIN.

OK, I got my PIN entered and the next screen asked me what I wanted, and I pressed "Get cash."

Then something happened that's never happened before, it gave me a list of accounts to select from. One said "Savings-123456," another "Visa Platinum-7654321," and then one said "CRWN-987654."

Nothing said "Checking." I figured the 1st one was my savings account and thought the second one was probably my credit card. That left me concluding that CRWN-987654 was my checking. We're talking money here and all of a sudden my ATM is giving me practice questions for the SAT. If all men eat turnips and John is a man, does John eat turnips?

So to be on the safe side, I decide to check the numbers on my card to see if they match 987654. Oops, card is in ATM. Cancel transaction to read numbers off card. Ooops, numbers no longer displayed on screen because I canceled transaction.

I'm in the business so I know what happened here. Product management decided to make my membership more valuable by now allowing me to select from multiple accounts when I withdraw cash. That's a good thing. But in doing so, the product has disrupted my familiar experience--turning a satisfier into a dis-satisfier.

When I was at CheckFree, whenever we introduced an enchancement to our online bill pay, we did what I called Usability Regression Testing. In QA, regression testing is when you make sure that a new feature doesn't break existing functionality. I think you have to do the same thing with usability, make sure that new features do not disrupt the comfort and familiarity of the user's current user experience.

Had they tested it they would have seen the whole number on the card in the ATM vs number on the screen thing.

They would have also figured out that CRWN did not mean "checking" to me. By the way, I checked it out with my wife, and apparently that's the marketing name for our service "Crown Checking." Marketing people are bad about that, they assume we are all in love with their product names and therefore familiar with them. Also, it was a huge screen with only 3 accounts, they could have said "Crown checking account" and I would have been OK.

Designers beware. When we improve the feature set, we run the risk of breaking a comfortable user experience. A little regression testing is always good.


Larry said...

I recognize enough of these UI prompts to know that we're both customers of the same bank. The first time, I happened to remember the "Crown Checking" name and guessed that CRWN was the right account. But after dozens of visits to the ATM I still pause each time, not completely sure that I've got it right.

The "cover the keypad" thing is relatively new, and I'm sure its purpose is to indemnify the bank in case my identity gets stolen. I wish you'd been there with me when I first encountered this one: Not being clever enough to think of the Goodyear Blimp ruse, I (gasp) left the pad uncovered. Next time I'll bring a blanket.

It's a nice bank. Its reputation for good customer service is actually well deserved. But they designed the ATM prompts such that they make perfect sense to bank personnel. To customers? Not so much.

MattBNH said...

First thing I thought was that 'Crown Checking" was too many characters. But it is 14 and "Visa Platinum" is 13. So it isn't that simple, because "CrownChecking" would fit and be a lot more intelligible than 'CRWN'.
Not to make any excuse for them, either way it is a lack of customer focus.

Chris M. said...

Glad I found your blog, and my friends at IT'S BROKEN liked your post as well (click on my name to see the reference to this article in their new blog post).

Michael Hughes said...

Thanks for the shout-out, Chris.