UX Candidate Design Challenge – v2

I had previously posted my UX Designer interview challenge.  I just wrote a new version, so I figured I could post the one I was using here. It is a continuation of the previous challenge.

Scenario

A famous comedian is the sponsor of the project.  He is working with 4 other comedians to rate and categorize a large set of jokes.  Each comedian works on different jokes, but they work at the same time and talk to each other via video conferencing.

iPadUI

Challenge Part I

Brainstorm an gamification strategies for this UI to help keep the comedians working without getting bored.

The game should not fundamentally change this UI, although minor modifications are acceptable.  It should be competitive and fun and encourage them to keep working.

IMPORTANT: the game cannot significantly sacrifice productivity or quality.

Common Mistakes

  1. Total Score.  The first thing people jumped to was a total score of how many jokes were categorized.  This is just short-sighted.  Total score actually DE-MOTIVATES the players once someone is winning.  The thought bubble is “Well, I am getting my ass kicked, so what is the point?”  Plus, once the winning person is in the lead, they let up because it’s over.

    Lesson: Games need to always be close.  The loser needs to become a winner soon.  Parity is important.  This is why baseball is stupid (see chart).

  2. Failure to come up with ideas.  I didn’t think this was that hard, however, most candidates just failed to come up with anything after that.
  3. Sacrificing productivity or rating quality.  It says it’s important, but people miss it 90% of the time.  If you set up a game which is a race to do N jokes in the shortest amount of time, then the outcome will be a significant decrease in quality.  In other words, people will rush though the jokes.  I hate it when candidates deny this reality.  Games = incentives.  Incentives work in unexpected ways. Read Practical Wisdom by Barry Schwartz for more detail.

Challenge Part II

After brainstorming for 10 minutes, I gave them a possible answer and asked them to do the interaction design for it.  Here is the text I hand them:

Game requirements: (not complete)

  1. There should be several different games that can be started by any player. (One game at a time)  Games include:
    • Best joke
    • Biggest groaner
    • Most intellectual
    • Darkest joke
  2. The system picks 25 random jokes and assigns them to each player.
  3. The players need to categorize and rate their jokes.
  4. During the game, they pick out 1 joke to “play”.
  5. When they are all finished, the jokes of the four players are presented to the group.
  6. They all rate 1-10 for how well suited the joke is to the game chosen. (Best, darkest, intellectual, etc)
  7. The highest combined rating wins that game.

Sketch it on the whiteboard. You can modify the base admin UI, but only minimally to make room for your game.  No functionality can be removed.  Don’t spend time re-designing the base UI.  Focus on the game.  Reminder: This UI is on a modern iPad.

Common Mistakes

  1. Failure to leverage gestures.  iPad interfaces should work differently than browser interfaces.  Only once have I designed something for an iPad, but my thinking immediately jumped to gestures.  A keyboard and mouse is just really different and people kept designing mouse UIs when hand gestures were more appropriate.  Don’t say click when you mean tap or swipe.
  2. Failure to understand the situation. I didn’t think it was that hard, but apparently it was.  People couldn’t grok the idea that a player starts a game and the others need to respond.  I thought it was obvious to use the video area on the right to show the status of people, but zero candidates utilized that area.
  3. Failure to use screen real estate dynamically.  For some reason the idea that information can be pulled on and off the screen was lost.  Almost all candidates botched using the screen real estate effectively.

It’s frustrating to me because this didn’t seem hard at first, but it clearly stymied the candidates.  Once of the problems is that there is only 30 minutes to do the exercise.  Maybe it’s just too much for someone.  I might be losing good people by having too hard of a test.

I’m changing the design exercise again.  I think I can continue to use the comedian, but I need to change the situation to make it easier to understand.  Clearly, I want to continue to test collaboration systems and interaction design.

If you read this and are going to an interview with me, keep in mind, I will be posing an interaction design challenge.  Stay calm.  Be creative and think out of the box.  Don’t just get task completion.  Make it fun. Be confident.  I believe in you.

Good Luck.


Comments

7 responses to “UX Candidate Design Challenge – v2”

  1. Ah, this is the best feedback ever! I wish I could get insights like this on every design challenge I do.

    The good thing about this challenge is that it helps filter out candidates who are very quick on their feet and can go from 0 to 100 in under a minute.

    This is probably just a personality thing but I would make sure that I ramp up the intensity of the challenge a bit slower.

    Also, design is very collaborative and divergent. It’s not highly practical to expect everyone to think about specific problems you want to hear about, for e.g. “total score” . Maybe people start thinking about the problem from a different constraint and will get to other problems later.

    That being said, I like how focused this challenge is and shows that you know exactly who you’re looking for. Unfortunately, this specificity is not common in my experience :/ I wish more people start doing design challenges this way.

  2. Personally I think this is a very skewed test. Some very good UX designers (or in my case product managers) may not be into “gaming” but might still be awesome talent and you could turn them off just because they don’t have gaming experience. In fact this test seems biased toward a younger crowd. I’d be more inclined to do a more generic registration process or a job application, or maybe even redesign a well-known feature like Amazon’s “Other people who bought this…” widget.

    I’d be curious to know what drove you to the gaming angle, and how you measure the results of the challenge.

    1. Glen Lipka Avatar

      I post these after I have moved on to another test, so I dont use this one anymore. However, a good designer can design anything. Its not about gsming, its about psychology.

  3. So played around with this one too. I liked your idea for gaming the project but used the previous app layout for a phone that I had created in your v1. Am thinking of using this as a side project idea so wanted to keep with the same idea I started out with in v1. Once again, loving these and looking forward to working on v3 and would always appreciate crit an feedback. I also went with the idea that the game is part of their process and isn’t something that is initialized by someone on the team each time. The app would randomly choose a category from a preselected list, seems a lot better than expecting one of the team members to remember to create a game each time. https://www.dropbox.com/s/enyx86ieabvq4g0/Jokes%20App%20Idea%20v2.jpeg?dl=0

    1. Glen Lipka Avatar

      Its a little hard to tell what is happening in your UI. I use storyboards to help understand. http://commadot.com/storyboards-or-prototypes/

      1. Will do Glen. Thank you for the feedback. Am going through as many design challenges as I can online before this interview.

  4. Glen Lipka Avatar

    Good Luck!

Whatya think?