UX Candidate Design Challenge – v4

As I am now hiring more people on the design team, I came up with a new design exercise. However, to jump to the end, it was too damn hard. So I only used it about 6 times before deciding to put it out to pasture.

So, as I do with all retired design exercises, I share them with the world.

Design Challenge

Treasure Data is a system to manage many databases in an enterprise. There is data everywhere and it’s not consistent at all. Sometimes field data is invalid or in various formats. We are going to zoom in on a field in two databases.

Database A and Database B has a field called Zip Code. There are 1 million records in the database. Here is a sampling of what you find if you compared the data.

Database ADatabase BWhat should we do?
9440394403Nothing, they are identical
94403-null-Copy A–> B
94403XYZFOOCopy A–> B
9440394403-12234Copy B–>A
blahgbleepErase both
9440394402Valid but different, hmm?

There are lots of possible scenarios, but not infinite.

The mission to design a UI that a user can create rules of what is valid and invalid and also what to do if there are problems. It should be an ongoing rule.

Why it was a bad design exercise?

First off, it is just too complicated. The whole idea of different databases, different fields, pattern matching, rules; it’s overwhelming. One candidate didn’t understand what a zip code was. Candidates spent way too much time asking me questions about databases and almost zero time designing interfaces.

Plus, it wasn’t a collaborative discussion. It was guessing and struggling the whole time. I thought, “Well maybe its good to see how they deal with pressure”, but ultimately, I realized it was just a bad design exercise. It created tension and confusion and didn’t help me understand how they would be as designers on the team. In fact, one candidate (whom we liked!) withdrew based on this. Bummer, I know. This is why a well-designed exercise is important.

Some candidates had bad logic and others just got nervous and panicked. All in all, it was an Epic Learning Experience. (#NoFailure)

What were the good ideas?

Pattern matching like #####-##### could help a user describe a valid input. If you look at the jQuery Input Masking plugin, it shows how easy symbology could be used to describe a valid input. This was a great insight to define what is valid and invalid.

Additionally, ideas around doing the easy parts first and reduce the problem down to the long tail were solid. There were some good ideas around a chatbot to give a more natural language UI.

What were not so good ideas?

Certainly sitting at the whiteboard and asking questions for 40 minutes is not a good idea. Always know what the measures of success are and try to approach the problem from that angle. I communicated that interaction design was the goal, but some people never got to that phase. (Probably because of the difficulty)

I know design schools teach a methodology about design thinking, but sometimes it just seems to get in the way. Sometimes, you just want to dive in.

New design exercise v5!

Well, I don’t want to give it away, but here are some hints. I want candidates to sit at a table with us and discuss the problem together. #NoWhiteBoard! I want candidates to show imagination and ideas. I want them to collaborate with us and most importantly “be in the moment”.

The new design exercise is a little unconventional, but hopefully it will show us what it’s like to work together. It might not be good, in which case I will post the details of it.

Bonus Hints

Too many designers have bad answers to my question “How do you improve your interaction design skills?” The answer of shrug “Umm, Medium articles?” is pretty lame.  Try reading About Face by Alan Cooper and Information Dashboard Design by Stephen Few. You probably read Design of Everyday Things by Don Norman, but check out Emotional Design too!

And apply to be on my team! (Contact me via my blog and Ill get you into the system)


Whatya think?