Named Entities in Job Advertisements (Internship Data Science)

Introduction
Textmetrics operates in multiple countries and offers software to help companies write better job advertisements. Its tool detects age and gender discrimination, analyses readability, and supports several other quality checks. In this project, we explored whether Named Entity Recognition (NER) could help solve multiple issues within the application.

One functionality of the Textmetrics application is suggesting synonyms. That works well for general words such as adjectives, but not for entities. In the phrase “Nieuw-Beijerland”, for example, suggesting synonyms for “Nieuw” would be incorrect. Similarly, in the Italian version of the tool, many place names were labeled as difficult words even though locations usually cannot be simplified. For words that are entities, synonym suggestions and difficulty warnings are therefore often undesirable.

Task
The goal was to label specific terms in job advertisements so that they would no longer be treated as difficult words or receive synonym suggestions. To do this, we developed and evaluated a custom NER model specialized in job advertisements.

The labels we defined included organizations, locations, persons, job titles, education, tools, and an additional miscellaneous category for terms that frequently do not have suitable synonyms, such as scrum, individual choice budget, and driver’s licence B.

Results
We created a dataset of 12,200 job-advertisement sentences and annotated it with the help of colleagues. We then fine-tuned RobBERT, a BERT model for Dutch. The confusion matrix below shows how well the model can classify spans into one of our categories, where O means no entity label.

In an additional analysis, we tested how well the model performed on completely new entities, meaning terms it had not seen during training. For several categories, performance dropped from a range of roughly 0.7–0.97 to between 0.40 and 0.95, depending on the label.

Discussion
We concluded that some entity categories were recognized very well, including entirely new entities. This suggests the model learned from sentence context rather than only memorizing terms seen in training. Performance for the miscellaneous category remained poor, likely because the concept behind that class was too broad and inconsistent. Overall, the results were promising, and we expected further improvement from more training data and active learning strategies.