When Candidates Use Generative AI for the Interview

Generative artificial intelligence is transforming how candidates prepare for interviews, making it more complicated for hiring managers to accurately assess their true expertise. By inputting role-specific details, organizational information, and their resumes into GenAI tools, candidates can prompt the technology to generate potential interview questions along with personalized answers. In fact, recruiters, consultants, and other job seekers widely recommend preparing for interviews in this way.

However, there is growing concern among hiring professionals that candidates using generative AI are gaming the interview process. Research suggests that GenAI use has a material influence on hiring decisions: In one recent study, candidates who used such tools to prepare received higher overall interview performance ratings compared with those who were unassisted by GenAI.

If generative AI helps candidates acquire new expertise or reinforce existing job-relevant knowledge, skills, abilities, and other characteristics (KSAOs), it will contribute to their future performance. But if it produces polished, contextualized responses that candidates merely parrot during interviews, hiring managers may mistakenly attribute KSAOs to them that they don’t actually possess. In such instances, their interview performance will not translate to their future job performance.

What, then, can hiring managers do in the wake of generative AI to ensure that they are getting an accurate representation of candidates from their interviews? For many reasons, there is already a preference for in-person interviews, including the possibility that candidates can use transcription software during virtual interviews to feed questions into GenAI and produce answers in real time.

Strategically, interviewing in the age of AI also requires effective follow-up questions to uncover deeper indicators of genuine expertise. Can they explain how to do something, not just what to do? Do they know why something works? Do they know when, where, and for whom something is more effective? Have they considered other approaches? And are they aware of the drawbacks of their approach? Those types of questions can push potential hires to go beyond their rehearsed or surface-level answers.

Barriers to and Benefits of Follow-Up Questions

The best practices for designing and conducting behavior-based interviews are well established. These include conducting a job analysis to identify role-specific KSAOs, training interviewers on what to look for, and asking consistent questions across candidates. Since past behavior is a strong predictor of future behavior, candidates who have demonstrated a job-relevant KSAO in the past are likely to do so again in the future.

Well-structured interviews are the most valid method to predict candidates’ future job performance. However, that approach is widely familiar to candidates, and so most come to interviews having prepared with help from standard frameworks, such as STAR (situation, task, action, and result) — and, potentially, GenAI — to answer behavioral questions. But with AI as a factor, the prevailing guidance for hiring managers on how to systematically and reliably gain deeper insights beyond candidates’ rehearsed answers falls short. Some hiring managers may also mistakenly equate a structured interview with a stilted interview, thus avoiding follow-up questions altogether for fear of adding noise to the process rather than enhancing their ability to assess candidates’ true expertise.

But by strategically probing candidates’ initial answers, interviewers can go beyond simply what candidates say they have done in the past to uncover the underlying thought processes behind their decisions and actions, which is crucial for two reasons.

First, only those who have internalized their KSAOs can provide insightful answers that genuinely reflect their abilities and potential job performance. Therefore, assessing these deeper indicators is important regardless of whether candidates have used generative AI for interview preparation, though such queries can certainly help interviewers spot candidates who are reciting AI-generated responses without comprehending them.

Second, these deeper indicators reflect skills that are among the most important for workers globally, according to the World Economic Forum’s “Future of Jobs Report 2023.” Whether termed critical thinking, reasoning, or judgment and decision-making, these are uniquely human capabilities that AI cannot replicate. Ensuring that candidates possess abilities that add value beyond what technology can currently achieve is fundamental to any hiring process.

Follow-Up Questions That Help Assess Expertise

The five types of follow-up questions below, synthesized from decades of psychological research and professional insights, can help you assess whether candidates possess a genuine understanding of their KSAOs.

1. A Breakdown of Their Process

The first set of questions is aimed at getting candidates to show their work. To begin, you should paraphrase what you have already heard to build rapport and demonstrate active listening. Next, ask a variation or combination of the following:

• “Walk me through, in greater detail, what you did to achieve the outcome.”
• “If someone unfamiliar with your approach — someone on a different team or without your domain expertise, for example — had to replicate your process, how would you explain it to them?”
• Provide the candidate with a challenge unique to their future role, then ask, “How would you adjust your approach/process?”

These types of questions help in evaluating the extent to which candidates possess procedural knowledge. Do they really know how to execute a task or process? Are they providing sufficient details on their process and/or actions? Are they able to adjust their approach based on new information? A candidate with procedural knowledge will be able to provide detailed descriptions that demonstrate a deep understanding of their process, not just vague or buzzword-filled answers. This is particularly important in the context of generative AI because the answers it produces are plagued with a deliberation problem: At first blush, the tools’ output seems to be well reasoned, but closer investigation reveals that it is all style and no substance.

2. Their Rationale

The second set of follow-up questions asks candidates to articulate the why behind what they did. Ask a variation or combination of the following:

• “What were the underlying principles that guided your decisions?”
• “Why do you think your actions led to the outcome you achieved?”
• “What factors outside of your control contributed to the outcome?”

These types of questions help in evaluating the extent to which candidates possess causal reasoning. Can they explain how or why something works? Do they understand the underlying principles of their process or actions, or do they just routinely apply a set of practices? When a candidate is able to explain the mechanisms that connect their actions and results, it indicates their ability to apply causal reasoning.

3. Details on the Context

The third set of follow-up questions asks candidates to consider how different circumstances might affect their work process. Ask a variation or combination of the following:

• “In what situations (teams, organizations, industries) would your approach not work as effectively?”
• Provide the candidate with a context unique to their future role, then ask, “How would the outcome change if you were working with a different colleague or client?”
• “How would you adjust your approach if you had fewer or more (financial, time) resources?”

These types of questions help you evaluate the extent to which candidates possess conditional knowledge: Can they identify when, where, and for whom something works better or worse? Do they demonstrate nuanced thinking, or do they adopt blanket approaches for all contexts? A candidate with conditional knowledge will be able to highlight the situational variables that change the effectiveness of their actions.

4. Roads Not Taken

The fourth set of follow-up questions asks candidates to weigh potential options. Ask a variation or combination of the following:

• “What other approaches did you consider?”
• “Under what circumstances would these alternative approaches be more effective?”
• “Who else did you consult before deciding on your approach?”

These types of questions help in evaluating the extent to which candidates have explored complementary alternatives: Did they consider other approaches before landing on theirs? Can they explain how or why those approaches would be more or less effective? Can they justify why they ultimately decided against them? A candidate who has thoughtfully considered other options will be able to explain their rationale for rejecting them.

5. Challenges to Their Approach

The final set of follow-up questions asks candidates to act as a defense attorney. Ask a variation or combination of the following:

• “What are the strongest counterarguments to your approach, and how do you respond to them?”
• “What evidence did you gather to ensure that your approach was correct? What about that evidence could have been misleading?”
• “If you were to face this situation again, what would you do similarly and differently?”

These types of questions help in evaluating the extent to which candidates have sought out disconfirming information: Do they see the drawbacks to their process? Can they defend why, despite those limitations, they still went with their approach? A candidate who is self-critical will be able to reflect on the trade-offs of their actions, why it was still the right decision at the time, and what they would do differently in the future.

---

There is no way to stop candidates from using generative AI for interview preparation — nor should we try. In fact, companies themselves use ChatGPT in their hiring processes. GenAI is simply a tool, and embracing rather than resisting it acknowledges the importance of technological innovation.

In this early-adoption phase, candidates who use GenAI for interview preparation will likely have an edge over those who do not, reflecting the adage that “humans with AI will replace humans without AI.” However, as generative AI use becomes more widespread, the playing field will level, and candidates’ differentiators will be their depth of expertise and critical-thinking abilities. The five sets of follow-up questions outlined here offer a straightforward and powerful way for hiring managers to strengthen their interview process by pinpointing fundamental human attributes that no AI can currently replicate.