…“This guy threw lasers, he could throw under tight spots, he had the arm strength, he had the size, he had the intelligence.” Shonka got as misty as a two-hundred-and-eighty-pound ex-linebacker in a black tracksuit can get. “He’s a concert pianist, you know? I really—I mean, I really—liked Joey.” And yet Harrington’s career consisted of a failed stint with the Detroit Lions and a slide into obscurity. Shonka looked back at the screen, where the young man he felt might be the best quarterback in the country was marching his team up and down the field. “How will that ability translate to the National Football League?” He shook his head slowly. “Shoot.”This is the quarterback problem. There are certain jobs where almost nothing you can learn about candidates before they start predicts how they’ll do once they’re hired. So how do we know whom to choose in cases like that?”
That’s the same problem as in technical teams, which is why we try to use auditions. But even an audition alone in front of one person or with a whiteboard is no guarantee of on-the-job success.
Read the whole article, because Gladwell relates this problem to the teacher problem: how do we detect great teachers: it’s not their degrees or strictly technical competence in their field–it’s more about how they engage everyone in the room and how they give feedback (and take feedback, although that’s just implied in the article).
Does that sound familiar to you? Working in a technical team partly about technical competence, because that’s how you get in. But that’s not how you stay in or become successful. You become successful in a job because you know how to help a team to evaluate and make a good decisions, to take and give feedback to peers, to use good judgement. These interpersonal skills are key to becoming successful in a technical job.
You can still be successful technically if you’re not superb at these interpersonal skills. But you can’t manage anything well unless you master enough of these (and other interpersonal) skills. Pay attention to your interpersonal skills in addition to your technical skills.
George asked in a comment what a fake audition was. Finally, I’m ready to discuss this. (Sorry for the delay, George.)
A fake audition is when the audition is incongruous with the situation. In the situation described in A Second Chance Audition, the candidate cared about the outcome of the first audition, but the audience did not. The interviewer was not balancing his needs to see the candidate with the candidate’s need to work in a real situation. Although the candidate felt as if this was a real work situation, the audience did not. That kind of unevenness, a lack of congruence is what makes a fake audition.
George’s comment explains how the audition is fake for everyone, which is congruent. I’m curious to know how well the audition is working over time. I’m not fond of role-plays as auditions, but if everyone is playing a role, that works better than when one person is doing some work for real and the others are role-playing.
When you create auditions, make them congruent. That is, make the situation real or fake, but be consistent. Don’t make it real for the candidate and fake for everyone else. That’s capitalizing on the interviewer’s power in the interview and many candidates’ perception of the lack of their power. It’s not respectful and it doesn’t help the candidate show their skills.
John Cook pointed me to this gem: a second look – generosity 3. It’s the story of a fake audition that was dissatisfactory and how another real audition helped the author get a great job.
Avoid those fake auditions. People don’t perform as well as they would in more real circumstances.
Organizations suffer with bad resources because they don’t know how to ask for good resources.
I wish I’d been there. I bet this was a great talk! I don’t think Pradeep had heard of my book until that conference, so he did buy one while he was there. I’m sure we will keep up our email conversation.
I ran a workshop recently about hiring for an agile team, and one of the people learning to interview said, “I want a candidate who can take criticism.” I replied, “Don’t you mean feedback?” He asked, “What’s the difference?”
Oh, boy. Plenty. Criticism is when you you’re looking at a piece of code and you say, “This seems brain dead.” But if you say, “I’m confused by this piece of code,” you’ve provided me some feedback. I guarantee you, you want candidates who can take feedback.
So, if you want to know if a candidate can take feedback, here are some possible interview questions:
“Have you recently been in a position where someone reviewed your work?” (wait for a yes answer.) “What happened?”
Offer to work with the candidate in an audition (possibly pairing) and review as you go.
Ask for feedback on some of you work as part of an audition and see how the candidate provides feedback.
“How do you know your work is good?” Wait and see where the question goes. You might be able to follow up with a question such as, “Is there a way you prefer feedback on your work?”
Asking candidates about their ability to take feedback is useful. Asking about criticism is not.
A behavioral audition is one where youw ant to see some specific candidate behaviors. Management auditions typically fall into this category. But especially if you’re hiring for an agile team, you might want to see how a candidate exhibits behaviors, such as coaching, feedback, how the person participates in a standup meeting or in a retrospective.If you want to see some specific behaviors, first, return to your job analysis. What are those essential technical and non-technical skills? Can you make a behavioral audition around those skills?You might want to see if the candidate will be a smooth addition to a team or a disruptive addition. In that case, you ask the candidate to do some work with one or more members of the team. You’ll ask the candidate how that work went, and you’ll need to debrief the members of your team (with open-ended questions) about what it was like to work with the candidate.Try a focused conversation for the debrief:
What stood out for you?
Where were you challenged?
What insights do you have about this candidate?
What do you recommend we do with this candidate?
Behavioral auditions are the most difficult to design. In my experience, you can see the behaviors if you use a technical audition, you are likely to see the behaviors you want to see. But to know what you want to see, you’ve got to do a job analysis.
A management audition is similar to a technical audition, but because the functional skills are so different, and because senior technical people may also require some management capability, I see these as two different kinds of auditions.As usual, your mileage may vary. Here are some possible management auditions:
Facilitate a meeting
Give a presentation
Analyze some budget information. Don’t forget to ask what they would do with that budget and why
Organize a project portfolio
Develop an audition for a manager. (ok, that’s slightly evil
If you have other good ideas, please comment away.
In his comment, John asked about auditions for a business analyst. The audition will vary, depending on the kind of projects you do. Here are some possibilities:
For a more serial lifecycle, you could ask a BA to help elicit requirements. You’d listen to the way the candidate interviewed, if the candidate interrupted the speaker, if the candidate asked meta questions.
For a more agile lifecycle, you could ask the candidate to make user stories from bullet lists of requirements.
For any lifecycle, ask the candidate to explain his or her favorite way to express requirements. Then take a product and ask the candidate to write down the requirements for that product.
Make sure your audition matches your environment. I might not have described something useful for you, so adapt my suggestions.
If you’re thinking about auditions, consider a “technical” audition for any of your open positions. A technical audition provides you a little insight into a candidate’s functional skills.If you’re hiring a developer, ask the candidate to extend a design, or find a problem and design a solution. You could ask a tester to test a piece of a product, or a particular set of features.A particularly good technical audition is one where you always ask the candidates (for a certain role) to perform the same kind of work. Ask each developer to write some kind of code, or ask a tester to test some specific product. (Don’t vary the task between candidates.)You can use these auditions for a phone screen or an in-person interview.Consider technical auditions the minimum entry point for auditions. You’ll learn a lot about a candidate with this audition.
I’m planning to do a series on audition, and the first one is the idea that every candidate enjoys an audition of the hiring manager, team, and company. That’s because every time you organize an interview, ask a question, or do anything at all, the candidate can watch you work. That’s an audition.
