Article AI and Hiring
How do you assess AI skills when you hire?
In short
The reliable way to assess AI skills in a candidate is to watch them use AI on a realistic task from the job. The approach works for any role, because the task comes from the job. Every candidate gets the same task, the same source data and an AI assistant, and the work they hand in is scored against a rubric that experts wrote before anyone took it. Resumes, interviews, self-ratings and quizzes about AI tools all miss the skill that matters most at work: checking what the AI produces before acting on it. A saved record of every prompt and answer lets a person on the hiring team see how each candidate worked and make the final decision.
- 51% of job postings that asked for AI skills in 2024 were for jobs outside IT and computer science, according to a Lightcast analysis of more than 1.3 billion job postings from its global archive.
- People misjudge their own work with AI. In an Aalto University study published in 2026, people who used ChatGPT overestimated their scores, and those who knew most about AI were the most overconfident.
- Checking the AI is the skill that separates candidates. In a Harvard Business School and Boston Consulting Group experiment, 84% of consultants solved a task built to mislead the AI when they worked alone, against 60 to 70% of those who used GPT-4.
- A sound AI skills assessment gives every candidate the same realistic task, the data behind it, an AI assistant, and a rubric written before anyone takes it.
- Codility AI for Business scores the work against a rubric that experts wrote, by fixed rules, and saves every prompt and answer for review. No automated hiring decisions are made by AI or machine learning, and a person on the hiring team makes the final decision.
Why can’t a resume or an interview tell you who can use AI?
Job descriptions caught up with AI faster than hiring did. Lightcast found that 51% of the job postings asking for AI skills in 2024 were for jobs outside IT and computer science, and that those postings advertised salaries about 28% higher than postings without them. Lightcast drew the 1.3 billion postings behind those figures from its global archive, which holds more than 2.5 billion. In SHRM’s 2026 survey of heads of recruiting, 78% expect AI skills to become a more common requirement on open roles.
So every resume claims it. “Expert in AI tools” costs nothing to write, and an interview mostly shows how well someone talks about AI. Each interviewer asks different questions, and nothing gets recorded that a second person could check. One talent leader put it to us plainly:
That is the gap I hear about most from talent teams. The requirement is in the job description, and no step in the process checks it. Codility built an AI skills assessment for business roles to be that step.
What does an AI skills assessment look like?
Here is one task, the one in the video below. A candidate for a marketing manager role gets a report that recommends tripling the podcast budget. The brief asks them to find 3 errors in its cost and return claims before the marketing leader acts on it. The report and the spreadsheet behind it open in tabs beside the brief, and the AI Assistant sits in the next panel, switched on, the way AI is at work.
Read the video transcript
Every job description wants AI skills now. And every resume says it has them. Easy to claim. Easy to talk up in an interview. So who can actually do the work? Find out before you hire. With AI for Business from Codility.
Every candidate gets the same task, straight from the day-to-day job. Here, a marketer checks a budget report. They get an AI Assistant, just like at work. The best candidates question it, check its numbers, and decide for themselves.
Our assessment scientists set the scoring rules. Same work. Same score. Every time. Every prompt and every answer, saved with the result. You see exactly how they got there.
Live interview? Bring in the same tasks and watch candidates use AI in real time. Sales. Marketing. Finance. Customer service. Pre-built tasks for each one. Run them with your current employees too. See who can already do the work, and who to upskill next.
Almost two decades of assessment science stand behind every score. Ready for your legal and compliance teams. Codility AI for Business. Hire people who can use AI to do the job.
Strong candidates start the way a good analyst would. They ask the AI Assistant to check the report’s cost per listener, return figure and listener total against the spreadsheet, and all 3 turn out to be wrong. A candidate who simply asks the AI Assistant what is wrong gets an offer to help them check it, because the assistant is set up to help with the work, and the candidate still has to do it.
The video above uses a sample task and sample data, built the way the tasks in the library are built: a realistic brief, the files behind it, and an assistant to work with. Some tasks deliberately plant errors in what the AI produces, and catching them is part of what is scored. General tasks work for anyone, whatever their role, and role-tailored tasks mirror the work of sales, marketing, finance, operations and customer roles.
Why don’t self-ratings and AI quizzes work?
A self-rating asks the person least able to judge. In a study led by Aalto University, published in Computers in Human Behavior in 2026, about 700 people across 2 experiments solved logical reasoning questions from the Law School Admission Test, with ChatGPT available. Their scores went up, and their estimates of their own scores went up by even more. The people who knew the most about how AI works were the most overconfident. A survey asking candidates to rate their own AI skills collects exactly that overconfidence.
A quiz about AI tools has a different problem. It checks what someone remembers about prompts and models, and remembering is the easy part. Knowing the definition of a prompt injection tells you nothing about whether a person will notice that the AI got the numbers wrong in a report their manager is about to act on.
Training records stop at attendance. They show who finished the course. The people who finished it and the people who can do the work with AI are often two different lists.
What does a candidate need to do well with AI?
AI at work is fast and often right, and the skill is knowing when it is wrong. The clearest evidence I know comes from a field experiment by researchers from Harvard Business School and Boston Consulting Group, published in Organization Science in 2026. On tasks the AI handles well, the 758 consultants using GPT-4 finished more work, faster, and at higher quality.
Then the researchers set one task designed to lead the AI to a wrong answer that sounded convincing. To get it right, a consultant had to check the story against the interview notes and the spreadsheet in front of them. 84% of the consultants working alone got it right. Among those using GPT-4, 60 to 70% did, and the AI group that had also been shown how to write prompts did worse than the AI group that had not.
So the skill is a set of habits. The strongest candidates do 4 things:
- They give the AI the context it needs: the goal, the audience and the data.
- They question it, and ask what each claim rests on.
- They check its numbers against the source, line by line where it matters.
- They decide for themselves, and hand in work they would put their name to.
How is the work scored?
Before anyone takes a task, occupational psychologists and subject experts write its rubric. The rubric sets out what a correct answer contains and how much each part is worth. The work the candidate hands in is scored against it by fixed rules. No AI or machine learning model produces or changes a score, so the same work gets the same score every time, including when a candidate or a regulator asks you to score it again.
The AI Assistant is a tool the candidate uses inside the task, and it plays no part in scoring. That matters under the EU AI Act, which treats AI used to evaluate job candidates as high-risk.
The hiring team sees the score beside the evidence behind it: the work itself, the full AI chat with every prompt and every answer, the time spent, and the same integrity signals as any other Codility assessment. A person reviews that record and makes the final decision.
The AI chat is there for the reviewer to read, and it does not change the score. We are now building an analysis of that chat. It will give the reviewer a summary of how each candidate worked with the AI, for example whether they gave it the context it needed, questioned it, and checked its numbers against the source. The summary sits beside the score and does not change it, and a person on the hiring team still makes the decision. We built it this way to stay compliant with GDPR and the EU AI Act, and I will update this piece when it ships.
Where does it fit in a hiring process?
Put it where you would otherwise screen on a resume. Send the task to a role’s candidates after they apply, and bring the strongest to an interview with the result in hand. The work, the score and the AI chat give the interviewer something specific to ask about.
The interview can use the same tasks. In Codility Interview, the hiring manager and the candidate write in one shared text editor with the AI Assistant alongside, so the hiring manager can watch the candidate use AI on a live problem and ask why they accepted or rejected what it said.
The tour below follows one hire from start to finish: a customer success manager candidate, from the task to the live interview. It takes about 2 minutes.
The tour also runs on its own page.
The AI for Business suite runs inside Codility Screen and Interview, so assessments go out and results come back through the same applicant tracking system (ATS) integrations as the rest of the platform. For the people you already employ, the same tasks run in Codility Skills Intelligence, to show who can already do the work with AI and who to upskill next. The tour for current employees shows how that works for a team.
Can you trust an assessment this new?
Only the task library is new. AI for Business uses the same assessment engine, scoring, reporting, integrations and security as the engineering assessments that large companies have run on Codility for almost two decades. Our methodology team includes occupational psychologists, and our 92-page Technical Manual, built to APA Standards, is there for your legal and assessment teams to review. Across Codility assessments, 90% of candidates say they got clear instructions on using AI, from 34,532 responses. The platform holds SOC 2 Type II and ISO 27001 certifications and operates under GDPR.
It is in use. A global enterprise software company is live in production, assessing AI skills for go-to-market and corporate-function hires under a board mandate. The person leading that rollout described the scoring to us as “completely objective and transparent.” Another software company chose its assessment platform on this capability, over two alternatives.
Where does this approach stop today?
Here is what AI for Business does not do yet, so you can plan around it.
- There are no market-wide benchmarks yet. The assessment is too new, and I won’t invent one. A first program gives you your own baseline, which is usually the number leadership asks for first.
- Tasks run in English.
- Tasks take 5 to 60 minutes each.
If you are hiring for roles that now ask for AI skills, start with one open role. Send the task to that role’s candidates, read the results and the AI chats next to your interview notes, and see whether the strongest candidates on paper are the strongest on the work.
About the author
I lead product marketing at Codility, where I own how the platform is positioned across Screen, Interview and Skills Intelligence, and I work alongside the Assessment Science team behind the Engineering Skills Model. I came to hiring from the other side of the AI question: before Codility I led product marketing for AI data and model evaluation at Toloka, serving frontier AI labs and large technology companies. I have also built marketing teams from nothing 3 times, so the hiring problems I write about are ones I have had myself. I am based in Rome.
Frequently asked questions
What is an AI skills assessment?
An AI skills assessment checks whether a person can get work done with AI. The person completes a realistic task from the job with an AI assistant available, and the work they hand in is scored against a rubric written before anyone takes the task. Every prompt and answer is saved, so a reviewer can see how they worked. It is used to hire for any role and to see where current employees stand.