The Experience Detector

GenAI has changed the interview game forever. Traditional technical questions can now be answered by anyone with a ChatGPT prompt. So how do we separate genuine expertise from AI-assisted answers?

The Death of Traditional Technical Interviews

For years, technical interviews followed a predictable pattern:

• "Explain how a random forest works"
• "What's the difference between supervised and unsupervised learning?"
• "How would you handle class imbalance?"

These questions were effective gatekeepers. But now, anyone can paste these into ChatGPT and get near-perfect answers. The playing field has leveled—but not in the way we hoped.

What GenAI Can't Fake (Yet)

While AI can provide textbook answers, it struggles to replicate:

1. Real-World Experience

Genuine expertise comes from battle scars—projects that went wrong, trade-offs that backfired, and lessons learned the hard way. These stories are deeply personal and context-specific.

2. Decision-Making Under Ambiguity

AI gives confident answers. Experienced practitioners know when to say "it depends" and can articulate what it depends on.

3. Intuition and Pattern Recognition

After years in the field, experts develop an intuition for what will work and what won't. This isn't something you can prompt engineer.

The New Interview Framework: Behavioral + Situational

Instead of asking "what" questions, I now focus on "when" and "why" questions:

Example Questions

• "Tell me about a time when a model you deployed performed worse in production than in testing. What did you learn?"
  Why it works: This requires a specific, lived experience. An AI-generated answer will be generic.
• "Walk me through a recent project where you had to choose between model accuracy and interpretability. How did you make that decision?"
  Why it works: Real practitioners have faced this trade-off. They'll describe stakeholder conversations, business constraints, and regrets.
• "Describe a situation where you disagreed with a stakeholder about the feasibility of a project. How did you handle it?"
  Why it works: This tests communication skills, conflict resolution, and domain judgment—none of which ChatGPT can simulate convincingly.
• "What's a technical decision you made that you'd do differently now? Why?"
  Why it works: This reveals self-awareness, growth, and humility—traits that only come from real experience.

Red Flags in the AI Era

When interviewing, I now watch for:

• Overly polished, textbook answers: Real experience is messy. If every answer sounds like a tutorial, be suspicious.
• Lack of hesitation or reflection: Genuine experts pause to think, especially for nuanced questions.
• No stories of failure: If a candidate has never made a mistake, they're either lying or haven't done enough.
• Generic examples: Real projects have quirky details. "I used a neural network to predict customer churn" is generic. "I built a churn model but realized our data only captured users who explicitly canceled, not those who ghosted" is real.

Probing for Depth

When a candidate gives an answer, dig deeper:

• "What made you choose that approach over alternatives?"
• "What would you do differently if you had to do it again?"
• "How did you convince stakeholders to go along with your recommendation?"
• "What was the pushback you received, and how did you address it?"

These follow-ups force candidates to go beyond surface-level knowledge and reveal the process behind their thinking.

The Role of Technical Assessments

Technical questions still have value, but they must be:

• Open-ended: "Design a recommendation system for this use case" beats "What is collaborative filtering?"
• Time-constrained live tasks: Real-time problem-solving under observation is harder to fake.
• Code review exercises: Ask candidates to critique existing code. This tests judgment, not just implementation skills.

Embracing the Reality

Here's the truth: GenAI has made it easier for less experienced candidates to sound knowledgeable. But this isn't a crisis—it's an opportunity to refine what we truly value.

We should be hiring for:

• Judgment over memorization
• Experience over textbook knowledge
• Adaptability over rigid expertise
• Communication over technical jargon

Final Thoughts

GenAI hasn't broken interviews—it's just made bad interview practices obsolete. If your interview process can be gamed by a chatbot, it wasn't testing the right things to begin with.

The best interviewers have always known: You're not hiring someone who knows the answers. You're hiring someone who knows how to find them—and more importantly, how to ask the right questions.