This repository contains the ATLANTIS (coAsTaL mAritime NavigaTion InstructionS) dataset files and our benchmark results on this dataset.

This work is co-financed by the Shom and the IGN and is being carried out at the LASTIG, a research unit at Université Gustave Eiffel.

Under no circumstances should any part of this work be used for real-life navigation.

ATLANTIS is a French-language dataset for nested spatial entity and binary spatial relation extraction from text. The dataset is composed of extracts from 15 different volumes of the Instructions nautiques. The Instructions nautiques are a series of books produced and published by the French Naval Hydrographic and Oceanographic Service, the Shom. Each volume contains essential information for navigating safely in the coastal waters of a specific geographic area, including instructions for entering ports and descriptions of the coastal maritime environment. We extracted the text from the PDF documents using pdfminer.six. The volumes, which are written in French, cover coastal areas in Africa, Europe, North and South America, as well as in the Indian and Pacific Oceans.

The TextMine'24 Dataset is a subset of the ATLANTIS Dataset and contains nested spatial entity annotations. It was used as the benchmark dataset for the Défi TextMine 2024, a spatial entity recognition challenge hosted by the TextMine working group which is part of EGC, the International Francophone Association for Knowledge Extraction and Management.

We annotated our dataset by hand using the brat rapid annotation tool, which allows creating nested labelled annotations and creating directed labelled links between them. The source text was split on whitespace by brat, giving a dataset of 101,400 tokens.

We use three labels to annotate unnamed and named nested spatial entities:

• geographic feature refers to common nouns that represent types of spatial entities
• name refers to pure proper nouns
• geographic name refers to the full name associated with a geographic feature

Any token can be annotated with zero or one geographic feature or name label. A token cannot be annotated with both a geographic feature and a name label. A token cannot be annotated only with a name label. A token annotated with a name label must also be annotated one or more times with a geographic name label. Any token, already labelled or not, can be annotated zero or more times with a geographic name label.

We use 19 labels to annotate binary spatial relations:

• is off the coast of refers to an entity that is off the coast of another (it is always referred to using the same words in French in our dataset: "au large de")
• is marked by refers to an entity that is marked by another (usually a navigation mark such as a buoy or a beacon)
• is an element of refers to an entity that is situated on or in another such that a bird's eye view shows the spatial footprint of one as being within or partly within the other
• is [direction] of refers to an entity that is in any one of 16 cardinal directions with respect to another: the 16 relations include four that use the cardinal directions (N, E, S, W), four that use the intercardinal directions (NE, SE, SW, NW) and eight that use the secondary intercardinal directions (NNE, ENE, ESE, SSE, SSW, WSW, NWN, NNW)

All relation annotations must link two entity annotations with the exception of name entity labels, which cannot be part of relation annotations. All relation annotations must have a direction.

The atlantis_dataset_brat directory contains the .txt source files and the corresponding .ann annotation files for the entire dataset.

The atlantis_dataset.jsonl file contains the entire annotated dataset in a JSONL format. The specific JSONL format is that of the SCIERC dataset. We used this script, modified to use the fr_core_news_sm spaCy model for tokenisation, to convert files from the brat standoff format to JSONL.

The our_split_jsonl directory contains the train, development and test JSONL files that we used for our experiments. We split the atlantis_dataset.jsonl file into three parts: train, development and test according to a 80:10:10 ratio in terms of overall number of tokens and numbers of entity labels. We also ensured that text covering each geographic area was present in all three parts. Our dataset of 101,400 tokens contains 16,777 entity labels (which can span one or more tokens) and 3,051 relation labels (which connect exactly two entity labels, with a direction). In total, 18,030 tokens are annotated with at least one entity label, which corresponds to almost one in five tokens.

We provide benchmark results for this dataset for three tasks: nested spatial entity extraction, binary spatial relation extraction, and end-to-end nested spatial entity and binary spatial relation extraction.

Our benchmark results were obtained using a supervised deep learning approach for automatic nested spatial entity and binary spatial relation extraction from text that we presented at the 7th ACM SIGSPATIAL International Workshop on Geospatial Humanities, held in conjunction with ACM SIGSPATIAL 2023. Our approach involves applying PURE (the Princeton University Relation Extraction system), made for flat, generic entity extraction and generic binary relation extraction, to the extraction of nested, spatial entities and spatial binary relations. The advantage of extracting nested spatial entities and the spatial relations between them is that it captures more information that can aid entity disambiguation.

This work was granted access to the HPC resources of IDRIS under the allocation 2022-AD011013699 made by GENCI.

Mean micro F1-score with standard deviation over five runs for varying context window ($W$) sizes for: entity extraction, relation extraction from gold entity annotations, and end-to-end entity and relation extraction (e2e) from best predicted entity annotations ($W$ = 248 for monolingual, $W$ = 200 for multilingual). For each task, the highest F1-score over all context window sizes for each base encoder is in bold, and the overall highest F1-score over all context window sizes and both base encoders is indicated with an asterisk (*).

[Prec.|Rec.|F1] / [Mono.|Multi] gives the mean [precision|recall|micro F1-score] for the [monolingual|multilingual] model over five runs for entity extraction for each entity label using the context window that gives the best overall results ($W$ = 248 for monolingual, $W$ = 200 for multilingual). For each task, the highest precision, recall and micro F1-score for all entity labels and for all relation labels over both base encoders is in bold.

[Prec.|Rec.|F1] / [Mono.|Multi] gives the mean [precision|recall|micro F1-score] for the [monolingual|multilingual] model over five runs for relation extraction for each relation label from gold entity annotations using the context window that gives the best overall results ($W$ = 0 for monolingual and for multilingual). For each task, the highest precision, recall and micro F1-score for all entity labels and for all relation labels over both base encoders is in bold.

[Prec.|Rec.|F1] / [Mono.|Multi] gives the mean [precision|recall|micro F1-score] for the [monolingual|multilingual] model over five runs for end-to-end entity and relation extraction for each relation label from the best predicted entity annotations using the context window that gives the best overall results ($W$ = 0 for monolingual and for multilingual). The highest precision, recall and F1-score for all relation labels over both base encoders is in bold.