Extracting Entities from Handwritten Civil Records using HTR and RegExes (MSc thesis)

About the HDSC
The Historical Database of Suriname and the Caribbean (HDSC) is dedicated to making historical documents such as slave registers and civil status records available online. This makes it possible to reconstruct life histories of individuals, especially around major transitions such as the abolition of slavery in 1863. Thanks to volunteers and sponsors, the slave registers of Suriname have been viewable online since 2018, followed by those from Curaçao in 2020. Since 2023, the birth certificates of Paramaribo have also become searchable online via the website of the National Archive of Suriname. More civil status records from Suriname and the former Dutch Caribbean are currently being transcribed.

Project
In this thesis, we investigated whether computer models can accelerate historical transcription projects and support citizen scientists. Focusing on death certificates from Curaçao, we built an HTR+RegEx pipeline that automatically extracts fields from a scan. The first step takes place in Transkribus, where regions and baselines are determined and an HTR model produces a transcription. Step two uses a Python script with regular expressions to extract entities from that text.

Original layout by Transkribus (left); layout produced by our two custom models (right).

Our custom layout models were trained to create one region for the marginalia on the left and one region for the certificate itself on the right. The baseline model was trained to create full horizontal lines of text. This prevented incorrect sentence order and avoided marginalia baselines continuing into the certificate body.

Our HTR model achieved an average Character Error Rate of 5%. Unfortunately, names were much harder to read, with a CER of 16%, which means that at least one-third of all names contained a spelling error. That is problematic when linking people across different databases.

In the image above, the HTR model returned “Martha Seratina” instead of the correct “Martha Serafina”. We also observed disagreements between citizen scientists when transcribing difficult names.

We concluded the thesis with recommendations for integrating the HTR+RegEx model into citizen science workflows. At the time, the HTR quality was not high enough for fully independent use. A more realistic approach was a collaboration between model output and volunteer effort, for example by asking volunteers to revise pre-filled entities or to focus review on certificates where model output differed from human output.

Continuation
In the HANDS-RX project, we improved and scaled this work to 24,000 Curaçaoan death certificates. In the end, we chose GPT for entity extraction because it was more robust to HTR spelling errors than static regular expressions.