Posts filed under 'interview question'
I taught my “Hiring for Agile Teams” workshop at ADP today, and finally have words for something I’ve seen for a while. When I ask people to describe qualities, preferences, and non-technical skills, they say things like “easy-going” or “intuitive” or something else that describes behavior. Since I love behavior-description questions, you’d think this would be perfect, right? Nope. They’re not describing abilities, which is the key.
To change “easy-going” into abilities, I asked what easy-going looked like. The person said, “Relaxed in the interview.” I asked if the person would just interview or do other work. “Do other work.” We went back and forth for a bit. So then I asked “Would this be more accurate: able to keep his or her head in the midst of chaos?” Yes, that was it.
That’s different than easy-going. It’s something specific to the organization (which is good), and you can ask for examples in behavior-description questions.
So if you see adjectives, think about the deliverables and activities the candidate will have to do. Then see how to describe that in terms of abilities. You’ll have a better description and be able to ask better questions.
November 11th, 2008
I’ve got election fever, I admit it. In the VP debate last week, the moderator asked a useless question: “What is your achilles heel?” (I’m probably paraphrasing the question.) Both candidates treated it as the weakness question, and didn’t answer the question. They each turned the question around to their strengths. What a surprise (not!).
But in the presidential debate last night, one of the questions was (I’m paraphrasing again): “How do you know what you don’t know and how will you learn it?” Ok, it’s a hypothetical question, not something I would use in a town meeting setting, but was a great opening for the candidates.
If you’re hiring a senior person, this is a good question. It can help you see the difference between general arrogance (”I know everything”) and a smart person who’s introspective enough to learn from past behavior.
October 8th, 2008
I was thinking about the election. (How can anyone in the US avoid it?) I read Seth’s piece, Politics!, and thought that nightly debates might be a great way to discover who the smartest people are. Maybe. But a lot closer than the sound bites we get now.
Since we’re not going to have nightly debates, here are some of my questions for the candidates:
- Tell me how you’ve chosen at least four of your advisors. What do they advise you on, and how did you choose them?
- Give me an example of a time you inherited a budget that wasn’t balanced. What did you do? (I might need to ask when that was, because we’ve had different economic cycles and unbalanced budgets.)
- How have you affected health care in your state in the last year? two years? five years? (All of the candidates are either senators or governors.) If not the state, the Senate. Explain the effect of your actions on the people affected.
- Give me an example of how you’ve worked with people who did not share your philosophy or values. (I might ask for half a dozen examples!)
- Give me at least four references that can discuss your integrity. What would you like to tell me about your integrity?
I suspect I need more questions than these (!), but I would start here. The President’s job is too difficult to take people who don’t think and act clearly.
BTW, you could these questions when hiring for managers, too. Instead of the health care question, change it to a question that addresses a significant cost in your organization, such as project management.
October 2nd, 2008
I’ve been in email contact with Pradeep Soundararajan for a few months now. He was recently at a conference in Toronto, and has posted his The (bad) state of software testing interviews in India, which includes a pdf of a talk he gave about interviewing. He has several wonderful ideas, including:
- Candidate and interview myths
- Frequently asked (horrible) questions
- Today’s candidates are tomorrow’s interviewers
- Organizations suffer with bad resources because they don’t know how to ask for good resources.
I wish I’d been there. I bet this was a great talk! I don’t think Pradeep had heard of my book until that conference, so he did buy one while he was there. I’m sure we will keep up our email conversation.
July 23rd, 2008
Jurgen has The Perfect Job Interview Question. It’s
When reviewing somebody else’s code, what is it that you usually find most disturbing?
This is a good question. If someone doesn’t review code, you’ll hear that. Jurgen goes on to discuss the syntax answers vs. the design/architecture answers.
He has a point. I would add some more questions, such as:
- When was the last time you reviewed someone else’s code? I want to know how recent this experience is
- When have you been prohibited from reviewing code? … What were the circumstances?
- When did you fight to review code?
- Have you ever reviewed unit tests and code? (a closed question to understand the context) What did you do? (a behavior-description question to understand what actions the candidate took or didn’t)
There may be even more questions to learn more about how a candidate is accustomed to working, and how well that experience might fit for you.
I don’t think this is the one and only perfect interview question, even for developers. But it’s a good start.
April 25th, 2008
Bruce Eckel has a nice post, What Questions Would You Ask?, especially for developers. I really like the question about books :-)Here’s how I would expand on that for project managers and testers:
- How do we you make project decisions? Can you give me an example?
- What happens at the end of the project? or Do you ever have crunch time at the end of the project? What do you do?
- When was the last time you had a one-on-one with any of your staff (for a manager or project manager)?
- How do you do career development?
Got other favorite questions? Please comment.
September 29th, 2006
A few weeks ago at the “Hiring for an Agile Team” session, the group generated a number of behavior-description questions. I promised I would post them, so here they are:
- “Tell me about a time you made a mistake.”
- Using the context of shipping/releasing a product late: “What did you change?” (Notice the past tense of “did.” When interviewing for an agile team, the group expected that the candidate would take some initiative for change. And, by using the word “did,” the candidate is less likely to take this as a hypothetical question.)
- “Tell me about a time you/your team changed course.”
- “Tell me about a time you performed another role on a project.” (looking for people who are comfortable roaming around a team, doing what needs to be done.)
- “Tell me about a time you worked with people on their projects.” (Looking for people who can coach.)
- “Tell me about a time you made a difference.”
- “Tell me about a time your daily priorities changed.”
There were tons of other great questions. If you participated in that session (or even if not!) and you have some questions you have found have helped you looking for people to fit into an Agile team, please leave a comment here.
August 14th, 2006
I received some great comments on Why Puzzles and Riddles Discriminate. Adam has a terrific list of the things he’s looking for when he uses “puzzles and/or brainteasers and/or random programs to test”:
- Do they give up right off the bat?
- Do they ask questions or sit silent pondering?
- Do they make different attempts or approaches?
- What areas (if any) do they get stuck on?
- Can they explain what they are thinking? If yes - great. If no - what’s the reason?
And, Craig and Eric disagree with my original post. I may not be able to convince Craig and Eric, but let me use Adam’s list and explain how the answers might be different when the interviewer uses a puzzle/riddle vs. using an audition.In the interview, the context is key to seeing how a person succeeds at work. If the context of the interview is congruent with the workplace, the interview can be a reasonable representative of how the person will interact with people at work. But if the context of the interview is different from the work, the kinds of questions and auditions are less likely to be useful.
I was working with a hiring manager and the interviewing team recently who liked giving what they called “technical puzzles” to candidates. But they’d had several problems recently: two recent hires quit, and one candidate walked out in the middle of the interview. I asked what they were doing for these technical puzzles, and was told they had several–and all of them were about developing a recursive algorithm. The problem was these folks didn’t do any recursion at all. In fact, they had a huge data transformation application, so the context of the interview didn’t match the context of the job. Once new hires realized they’d made a mistake, they reactivated their job search and left. And one candidate didn’t bother waiting until the end of the interview. He’d flipped the bozo bit on the interviewing team.
At another client, the interviewing team liked word riddles. The team traded off who got to ask the puzzle/riddle question during the interview. (That team had 5 people, each of whom had a different puzzle/riddle.) Because the interviewing team were not great at asking behavior-description questions or using real auditions, they missed bunch of people who were hired by other managers in the company (the new hires were successful). I was able to talk to the new hires and ask them questions to understand what the interviewing team was missing. Several of the new hires misunderstood the logic puzzle because English wasn’t their first language, and they didn’t hear the puzzle correctly. Two of the new hires decided that if that’s how the team evaluated potential candidates, they didn’t want to work with that team. And one of the new hires had attempted to explain why there was more than one solution to the riddle, but the interviewer couldn’t hear that.You might say that this interviewing team was able to hire into their culture, and you’d be partly right 
But in contrast to their interview questions, this team actually had a culture of looking into a problem several ways, debating how to do things, prototyping to see what solutions could look like–a much more collaborative way of working than their puzzle/riddle questions probed. The interviewing team shortchanged themselves by not asking questions/doing auditions that reflected the way they actually worked.
You’ve probably noted I haven’t addressed the discrimination issue here. No one would talk to me about that, because they were afraid (both the new employees and the interviewing teams) that I would be forced to go to a lawyer if I had data that said they were discriminating. But I did notice that in my clients’ large North American cities, those clients who rely heavily on puzzles and riddles have primarily-white, primarily male staffs (this is in contrast to other organizations who have much more diversity). I understand about the problem attracting women to the field in general (see MusingsonWomenandIt. But I could not understand the lack of Asians and other people who aren’t white.Our choices of questions that are not directly related to the job (and no matter how you slice it, puzzles and riddles are only indirectly related to the job) reflect our culture. It’s quite clear that the puzzles and riddles interviewers choose reflect their individual culture. And without meaning to, that culture primarily selects for people just like themselves.I do want to ask questions or observe behaviors similar to Aaron’s questions. But I want to see them in the context of what people do at work, specifically work in this organization. So I want to make all my questions and auditions as context-sensitive as I can. I want to know that candidates will succeed here, in this context.
August 9th, 2006
At last week’s Agile 2006 conference, I led a tutorial called “Hiring for an Agile Team.” I made a statement that some of the participants challenged:
Using puzzles and riddles discriminate against anyone who isn’t a (middle-upper class) white American suburban male.
(I’d forgotten the middle-upper class part when I was leading the session.) So, what everyone wants to know is: Where is my data?
First, let me explain why I said this. (This post is a bit of a rant.)
- Girls, for example, do not have access or the alone time to spend doing books of puzzles and riddles. It is socially unacceptable for even the geekiest girl you know to do this. A girl who spends time pursuing puzzles and riddles for her own pleasure runs the colossal risk of being ostracized from all the other girls. Boys tend to discover puzzles and riddles during middle school and continue to pursue them through high school. Middle and high school for girls is much more about social ability and social connections.
- The people who can afford to buy the puzzle and riddle books (to practice them and become better at them) are middle-to-upper economic class.
- Puzzles and riddles appeal to a limited number of personality types–types frequently found in high-tech jobs. In particular, they appeal most often to NTs, especially INTJ and INTP types. (NTs are visionaries in Do Your Interview Questions Discriminate For or Against Your Needs?
So, the people who practice with puzzles and riddles are the people who have the disposable income, time, and social acceptability to do so. I don’t know enough about middle class high school boys from other cultures, but I see this all the time in the US.If you’d like to see some other information, take a look at How Would You Move Mount Fuji?. There are some hints at the articles at Career Resource Center but no reference. I also wrote Brainteaser Interviews Showcase Lack of Interviewer Skill, not Candidate Expertise. And, take a look at Brainteasers Inappropriate for Job Interviews by John Kador, who literally wrote the book.
So, have I convinced you yet? Maybe not. Maybe you still think you need a way to know if this candidate is smart enough for you, or you don’t feel competent to start a conversation with something like a puzzle or a riddle to dissect.If you need to know whether a person is smart enough, use behavior-description questions and auditions. Don’t be afraid to ask the “What did you learn from that experience” question as a follow up from your behavior-description questions. If you need people who are smarter than the people we normally have in our field, use the job analysis to understand why. (I bet you don’t, but that’s another rant.)If you need a conversation starter, I have some ideas on building rapport already in this blog, but I’ll write another piece about building rapport. Tongue-in-cheek: Really, if you’d been socializing in high school instead of using puzzles and riddles to avoid social interactions, you might already know how to do this.
Joel discusses ways to really interview in The Guerrilla Guide to Interviewing and he discusses auditions in The Perils of JavaSchools. So, if you’re looking for real discrimination data, sorry, I don’t have it. I only have my common sense.
Remember, the goal of an interview is to evaluate how well a candidate will do at work, specifically in your workplace. Puzzles and riddles may evaluate a certain kind of personality, and possibly even raw IQ. But they won’t tell you how well a candidate can work with your group or on your products. And that’s the point of an interview.
August 2nd, 2006
Hiring managers love the ‘Why did you leave your last job’ question. As an interviewer, too many people are willing to tell all, and those stories explain a lot about a candidate.Anthony in Been Fired Lately? says it’s not a big deal–everyone’s been fired. And for many candidates, I would agree. But here are things I’ve heard as a hiring manager:
- “Well, I didn’t like the work my boss gave me, so I stopped coming in.”
- “I had too much school work, so I stopped working.”
- “I didn’t agree with my boss.”
I like asking the question to see what kind of an answer I get, to see if it’s something I want to check in the interview or in the reference.
And for the record, I was fired twice and laid off once. The firings were called layoffs, but since I was the only one laid off, I’m pretty sure I was fired. I have not had to fire myself yet
January 18th, 2006
Previous Posts