Via the New York Times, Jason Dana, a Yale School of Management professor, writes on interviews:
...This is a widespread problem. Employers like to use free-form, unstructured interviews in an attempt to “get to know” a job candidate. Such interviews are also increasingly popular with admissions officers at universities looking to move away from test scores and other standardized measures of student quality. But as in my friend’s case, interviewers typically form strong but unwarranted impressions about interviewees, often revealing more about themselves than the candidates.
People who study personnel psychology have long understood this. In 1979, for example, the Texas Legislature required the University of Texas Medical School at Houston to increase its incoming class size by 50 students late in the season. The additional 50 students that the school admitted had reached the interview phase of the application process but initially, following their interviews, were rejected. A team of researchers later found that these students did just as well as their other classmates in terms of attrition, academic performance, clinical performance (which involves rapport with patients and supervisors) and honors earned. The judgment of the interviewers, in other words, added nothing of relevance to the admissions process.
Research that my colleagues and I have conducted shows that the problem with interviews is worse than irrelevance: They can be harmful, undercutting the impact of other, more valuable information about interviewees....Professor Dana suggests that, while interviews are not a very good tool, asking all the candidates the same questions is probably a good way of getting better comparisons. I could believe this, but I'm never quite sure.
(Personally, I have always thought that the "new" (probably 20 years old?) style of "behavior-based interviews" ("tell me about a time you've solved a problem") is will someday engender "Reservoir Dogs"-style fake anecdotes where you make up a story where you are the hero. That's probably because I am a bad person or something.)
Ah, unstructured interviews. One of my first interviews out of grad school was with Dow R&D, and I had someone go through the entire interview with a plastic do-hicky attached to her glasses. I guess I was supposed to ask her about the do-hicky? (Candidate shows curiosity!) Or maybe not? (Candidate is preoccupied with appearances and norms.) Maybe she was just screwing with me? Or maybe she was trying to break the ice?
ReplyDeleteAfter all these years, it's left a bad taste in mouth about the amateur psychology of interviews...and Dow.
Over the past 5 years, I've done a lot of recruiting and interviewing for Dow. I've never used or been encouraged to use such gimmicks. That being said, we do use a formulaic, behavior-based interview process. The reason for doing so is to ensure that all candidates get asked the same questions and can be scored in several key areas and compared to one another. The system isn't perfect, but it certainly helps us narrow the candidate pool for onsite invitations.
DeleteI worked in Dow R&D 2007-2011 and know this person well. The plastic bit on her glasses was just her way of being her. It had nothing to do with you.
DeleteAlso, I can echo the previous comment about the process as I interviewed many candidates at Dow. The best question was always, "Describe your contribution to your PhD" and you would be amazed at the number of candidates that struggled with this.
Original anonymous here. Thank you to the third anonymous in the chain. That bit of plastic has bothered me for years. (I guess we all know who's the real weirdo.)
DeleteI have to take exception to your "best question." What's the right answer? A polymath who did everything? (Or, more likely, has forgotten all the help received along the way.) A mastermind who built a cooperative team but did little work compared to other interviewees? "Get Ph.D." is an open problem with infinite solutions, and frankly, a preference for one type of solution doesn't seem like a sound method for staffing decisions.
Anyway, overthinking questions probably is what earned me a rejection letter. I guess not much has changed.
In addition to the single unreliable narrator as in Reservoir Dogs, there's also the idea from Rashomon that we're all unreliable narrators. Why do we continue to make hiring decisions based on anecdotes from people who, ignoring their incentive to lie, are probably going to color the facts anyway?
ReplyDeleteRashomon. Great movie. Next step in interviewing: hiring a psychic to channel spirits who may have witnessed your (supposed) accomplishments in past roles.
DeleteHiring is a crapshoot at best.
ReplyDeleteHad an two phone interviews for essentially the same job with one large company in two different locations (and thus, two different hiring managers and HR reps). They both asked the same type of questions and I gave the same answers. One of the positions gave me an on-site interview and the other gave hilariously negative feedback about how I never answered the questions, but I just "talked" and of course, I was not granted an on-site interview. I'm still baffled by this and naturally, the one that I apparently gave negative answers was in a location I'd prefer to stay the rest of my life.
ReplyDeleteMy life in a nutshell...
Can confirm. At my institution they hired 5 people in the last 2 years, and 3 of them were so godawful that they were fired after a year. You'd think that nowadays the applicant pool was so deep that you could pick a CV at random and that person would be decently competent at their task. But bad leadership invariably makes bad choices. The interviewing process here is way worse than random choice.
ReplyDeleteI'm not convinced that the hiring pool is as deep as we like to think. Lots of people look good on paper, then turn out to have zero soft skills or are just a**holes. I'll agree that there are lots of talented scientists out there, but that's not enough anymore.
DeleteOne of the three was clearly part of a citation ring, another wrote a really appalling thesis. The third one, a five-minute conversation would have been sufficient to recognize that they had no clue. It wasn't a lack of soft skills, they looked bad even on paper, God knows what possessed the committee.
Deletemaybe some key phone calls from the candidates' advisors?
DeleteMaybe someday they will just use our social network pages to access whether or not we have the right psychology. A chemist who used to work at Chevron told me that at their interviews they look for people who are shy and nice, the idea being that shy people won't give problems. I confirmed with this person's boss. He said they got hired because the other candidate seemed like he wanted to move up in the company (and he said it in a way that made it sound like the candidate is a jerk for wanted to advance). The awesome thing is that the chemist got her MBA paid for by Chevron and then left.
ReplyDeleteOn the other hand I had been or interviewed at places that appeared highly organized in interview process that assigned interviewers particular questions, occasionally duplicated or slightly varied probes, where I did not see any any more effective than free from mode. Unless a real effective liar/psychopath most people can not hide their natures and thus get a good read on who they are and how likely to fit regardless of the approach but it is also critical to check references.
ReplyDeleteA great interview would be one where the candidate enters a dimly lit room and is seated at a table with a hookah on it while the Jefferson Airplane's "White Rabbit" plays softly in the background ... .
ReplyDeleteRead qvxb's comment and immediately imagined Laurence Fishburne offering me the red pill or the blue pill - technical ladder or management ladder. I don't know why, just did.
DeleteI hate behavior-based interviewing. It was done where I worked, all the interviewers were given the usual list of questions to ask. I could never figure out why they would waste researchers' time doing candidate interviews if we were supposed to ask questions from a list. One year at the annual recruiting kickoff I asked the organizers that if they were going to insist on providing the questions, could I please have the list of correct answers to go with it? They looked at me blankly, then rather angrily, and I was labeled a troublemaker. That said, I would put my success/failure rate of candidate selection up against anyone else there, and I know for sure I recommended against some spectacularly horrible hire decisions (and who apparently did really well at behavioral based interviewing)
ReplyDeleteBehavior type questions lead to a lot of planning and rehearsing and probably doesn't even get to the best or most demonstrative answer. I'd say they're best at indicating preparation on the part of the candidate. Just about anyone has problem solved (especially if you made it through grad school) or had to resolve a conflict between themselves and their coworkers or boss, but fewer people have turned that into anything approaching a coherent narrative. Not flubbing a question like that is more indicative of due diligence about interviewing.
ReplyDeleteAnon 2:55: couldn't agree with you more! I spent two years looking for a job after getting my phd. At the end, I googled "most common interview questions", wrote a paragraph answer to about 50 of them, put them on flash cards, and spent a weekend at a park rehearsing them. Got a very nice job offer shortly afterward...
ReplyDelete