Notes from the battlefield

Your next interviewer might not be human!

You have prepared your frameworks, practised your answers and picked an outfit that suggests you understand both spreadsheets and people. Then someone introduces you to a mayor. The mayor has a city to run, a development project to discuss and a few concerns about your recommendations.

The mayor also doesn’t exist.

What is McKinsey’s AI mayor actually doing?

The New York Post reported on October 14 that McKinsey is piloting a client simulation in which a candidate might advise a fictional mayor on a stadium project. The candidate proposes a recommendation. The avatar can introduce further concerns.

According to The Wall Street Journal, the exercise lasts roughly ten minutes and is being tested with undergraduates applying for business analyst roles at some North American offices. Human assessors observe. The avatar does not score the candidate.

That distinction matters. This is an AI stakeholder simulation within an assessment led by people. It is not an autonomous AI hiring decision.

McKinsey’s own careers page says it is piloting AI interview experiences and that recruiters and interviewers decide who progresses. So the mayor in the headline has a role to play, not a hiring budget to approve.

The interesting part is the interruption

A prepared answer can be impressive. What interests me is what happens when the situation changes halfway through it. Consider a hypothetical version of the exercise. You recommend a stadium location because it looks sensible on cost and transport. Then the mayor raises a concern about the people living nearby.

Do you ask what you are missing? Revisit the tradeoff or continue delivering the answer you polished on the way in? A good consultant should be able to explain a recommendation. A better one should notice when new information makes that recommendation less useful.

The point of a simulation, as I see it, should be to make that difference observable. Listening is difficult to demonstrate when the entire interview is an uninterrupted performance of how well you listen. A fictional mayor could be useful if the exercise gives candidates a reason to stop presenting and start responding.

Of course we will try to beat the game

Give people a predictable test and I would expect a preparation market to appear around it. Someone will offer a framework for handling the mayor. Someone else will sell a better framework, with a premium tier for municipal confidence.

That is a prediction, not a claim about this pilot. But it is the design problem I would worry about. Preparation is not the enemy. People deserve to understand an assessment and practise the skills it requires. The problem starts when mastering the format becomes easier to reward than understanding the problem.

A candidate can learn to acknowledge a concern without taking it seriously. They can perform curiosity, ask an impressive question and ignore the answer. A polished conversation is still only evidence. Somebody has to decide what it demonstrates.

I’ve written about the interview where two bots spoke to each other. This experiment raises a different question: when one side is simulated, what should the human assessor be looking for?

Here is what I would actually assess

If I were designing the rubric, I would start with four things.

First, did the candidate identify the new constraint? I would want them to explain what changed, rather than simply thank the mayor for the feedback.

Second, did it change their reasoning? Keeping a recommendation can be sensible. Defending it without considering new information is something else. I would look for the connection between the concern and the next decision.

Third, could they explain uncertainty? A useful answer might be that they need another piece of information before committing. I would want to distinguish an honest limit from an elaborate attempt to sound certain.

Fourth, did they check whether the recommendation addressed the stakeholder’s concern? A conversation should have consequences for the work, not just a pleasant closing sentence.

Those are my proposed criteria, not McKinsey’s published scoring rubric. I would also ask assessors to record the behaviour behind each judgment. Otherwise a confident delivery could quietly become a substitute for evidence.

A consistent character is only the beginning

McKinsey says AI can make the client persona consistent. I can see the appeal. An assessor who does not have to act out the client can concentrate on watching the interaction.

But consistency alone would not tell me whether the assessment is useful. I would still want to know whether different candidates get comparable opportunities to demonstrate their thinking, whether communication needs are accommodated and whether the results tell us anything about doing the job.

Those are questions for evaluating a pilot, not findings that this one has failed.

The future of work will need people who can reason with tools and with other people. A simulation might help us see those skills. It might also make an old preference for a convincing performance look newly scientific.

We have already made AI useful for preparing an answer. Putting it across the table could make the questions more interesting.

The harder test belongs to the hiring process itself.

Can it recognize intelligence without confusing it for performance?

The Last Bite

We taught AI how to help us pass interviews.

Now we’re teaching AI how to interview us.

The real question isn’t whether AI can play the interviewer.

It’s whether it can recognize the right candidate.