README.md



POPP datasets
This repository contains 3 datasets created within the POPP project (Project for the Oceration of the Paris Population Census) for the task of handwriting text recognition.
These datasets have been publised in Recognition and information extraction in historical handwritten tables: toward understanding early 20th century Paris census at DAS 2022.
The 3 datasets are called "Generic dataset", "Belleville", and "Chaussée d'Antin" and contains lines made from the extracted rows of census tables from 1926. Each table in the Paris census contains 30 rows, thus each page in these datasets corresponds to 30 lines.
This repository is a Git LFS repository containing the image files, the labels are stored in another repository. The datasets are also available on Zenodo.
The scructure of each dataset is the following:

double-pages : images of the double pages
pages:

images: images of the pages
xml: METS and ALTO files of each page containing the coordinates of the bounding boxes of each line


lines: contains the labels in the file labels.json and the line images splitted into the folders train, valid and test.
The double pages were scanned at a resolution of 200dpi and saved as PNG images with 256 gray levels.
The line and page images are shared in the TIFF format, also with 256 gray levels.

Since the lines are extracted from table rows, we defined 4 special characters to describe the structure of the text:

¤ : indicates an empty cell
/ : indicates the separation into columns
? : indicates that the content of the cell following this symbol is written above the regular baseline
! : indicates that the content of the cell following this symbol is written below the regular baseline

The split for the Generic Dataset and Belleville have been made at the double-page level so that each writer only appears in one subset among train, evaluation and test. The following table summarizes the splits and the number of writers for each dataset:


Dataset
train - # of lines
validation - # of lines
test - # of lines
# of writers


Generic
3840 (128 pages)
480 (16 pages)
480  (16 pages)
80


Belleville
1140 (38 pages)
150 (5 pages)
180 (6 pages)
1


Chaussée d'Antin
625
78
77
10