Noor Esposito· Total Rewards Manager, Steinbach Industrie
Posted in Pay Equity & Transparency ·
We piloted a language-model tool this year to help group jobs of equal value, at a hotel and leisure group with about 9,000 employees in Spain, Portugal and the UK. We fed it 1,150 job descriptions and asked for a first-pass score on skills, effort, responsibility and working conditions.
Where it helped: it found 210 near-duplicate jobs with different titles in about a day, and its ranking of jobs within a family matched our evaluation panel around 85% of the time.
Where it embarrassed us: it consistently scored housekeeping supervisors below maintenance supervisors on responsibility and effort, because the housekeeping descriptions were shorter and used softer verbs. The descriptions carried the bias and the tool reproduced it faithfully. A panel member caught it; the tool never would have.
3 helpful · 3 insightful · 3 replies · 161 views · 4 following
Compensation Consultant, Cardinal Insurance
This matches our experience almost exactly, in a facilities company. The quality of the description decides the score, and descriptions for female-dominated jobs were on average 40% shorter in our library. We now rewrite descriptions to a common template with the job holder before any scoring, by human or machine. Tedious, and by far the most valuable step.
People Analytics Lead, Silverline Media
I am more sceptical about using it at all for the evaluation step. De-duplication, fine. But the scoring is precisely the part we will have to justify to employee representatives, and it scored it this way because of patterns in text is not a justification anyone can examine. Our works council in Germany has said they will not accept machine-generated scores as an input, even as a first pass.
Compensation Consultant, Cardinal Insurance
Understood, and we have ended up near that position. The tool's scores are no longer shown to the panel before they score, only afterwards as a check on where panel and tool disagree. Those disagreements, about 170 jobs, turned out to be a useful list of descriptions that needed rewriting. So we kept it as a way of finding weak descriptions and dropped it as an evaluator.
