Get started

Avoid a potentially biased tech hiring process. Learn how to improve the diversity of your technology workforce & download Codility’s Gender Bias Report today!

We Analyzed 1M Candidate Sessions for Our Gender Bias Report: Here’s What We Learned

Companies are striving for diversity in the workplace to drive innovation and gain competitive advantage in their market. But how can you embrace a diverse workforce if your hiring processes are potentially bias? Hiring teams should look to their own sourcing, tech screening, and shortlisting methods to ensure that they aren’t ignoring, or accidentally discriminating against diverse, qualified candidates.

What causes the performance gap?

We did a deep dive into our data to uncover whether there are differences in performance between genders on Codility online coding tests amongst different companies, and whether there are any factors that perpetuate gender bias. Why?

Because we believe it’s our responsibility to our customers to provide the best possible online skills assessment platform that helps to eliminate — rather than add — bias to an already complicated process. Although we did not collect any information about candidates’ gender to avoid further bias, we were able to automatically recognize the gender of 53% of candidates with 96% confidence.

We found that while there is a difference in performance among genders, data suggest that there is no gender bias in Codility tasks. As far as we can tell, Codility is excluded from being a causal factor to the performance gap — instead we focused on if there are any correlations where there could be some causality.

Here’s what we discovered.

Performance is correlated to the seniority gap and candidate market

The below data show the proportion of females invited to take a Codility test by recruiters — the highest proportion of females was at 58%, while the lowest proportion of females was at 2%. When looking at our larger-sized customers we can see that, on average, 88% of candidates who received tests are male. The smaller the proportion of women that enters the tech recruitment process, the lower the chances that numerous women advance to further stages of recruitment.

Percentage female invited to codility test

Interestingly, we observed that a female recruiter invited a higher proportion of females than males to take the test.

We also found that the seniority gap between genders may be a contributing factor because women tend to be more junior — about 81% of female developers have 6 years or less of professional coding experience, and therefore are less likely to be considered for senior developer roles. In looking at the data we can expect that the inequality in numbers of males versus females in programming will decrease in the future.

Codility task characteristics impact genders similarly

We studied several variables regarding the Codility tasks to try and understand performance gaps in a more meaningful way. Overall performance was extremely similar between genders — for example, we observed that adding a picture to a task helped both females and males with their scores.

To our knowledge, the only minor discrepancy was in tasks that contained math formulas. Females performed better on them — one of the reasons might be that they tend to be more comfortable solving tasks that feel familiar and have patterns that they already know.

Impact of task on candidate score and time

*Equals sign indicates no affect on score or time used. Arrow up shows correlation with higher performance (or two arrows up — significantly higher performance). An arrow down shows correlation with lower performance (or two arrows — significantly lower performance).

We also looked at whether or not the task contained difficult vocabulary or a lengthy description — but none of these were statistically relevant enough to explain differences in performance as pictured above.

Females and males have different test-taking behaviors

Our data suggest that women were more likely to “give up” on their session, with an 11% drop-off rate among female candidates versus 9% among males. Giving up is defined as using less than 20% of the allotted time and scoring less than 20% on the test.

Female candidates also edited their solutions more than male candidates, and ran coding tests more often — that could mean that the females were more likely to double check and fine-tune their code.

We did observe some behavioral similarities between males and females:

  • The more often candidates paste code, the lower score they get, and this affects both genders similarly. Pasting code often suggests a trial-and-error approach.
  • Both genders began editing their code about 8 minutes into the session, and started their Codility test about 3 days after being invited.

Companies are fully capable of creating diverse engineering teams

The biggest shift will happen after you review your existing sourcing strategies, are transparent about discrepancies, and work to establish unbiased tactics. Review your existing tech recruiting tools to determine if they actually support your diversity efforts, not hinder them. We encourage you to rethink your strategy so you can more effectively source candidates — even though we can’t facilitate inviting more of your candidates to take the Codility test, our data suggest that there is a need for diversity-oriented strategy in our current hiring landscape.

For more insights on the gender gap in tech employment, as well as learn more tech recruitment trends, download our Gender Bias Report:

Download Report

Agnieszka is involved in Product Research @Codility – bringing to the table 14 years of research experience, she focuses on product performance, new feature development, and data analysis. She uses in-depth knowledge of both qualitative as well as quantitative research methods to engage the team in insight-driven decision making about Codility developments

Connect on LinkedIn