Specific important tasks in DH require novel advanced language technologies or adaptations of existing general approaches. In this challenge, we present four technologies, solving general problems, which we adapt to specific DH tasks in the next challenge.
There is a notable need to investigate networks of people, organizations, places, and other entities across various corpora. Current Named Entity Recognition (NER) and Linking (NEL) methods (Boros et al., 2020) are not adequately adapted for less-resource languages, leaving these languages underrepresented in multilingual historical corpora (Schweter et al., 2022). This inadaptability limits the ability to build comprehensive interaction graphs for analyzing important entities and their connections across time periods. While complex network analysis can help rank and visualize these entities, utilizing pretrained, language-agnostic models like ULTRA (Galkin et al., 2024) for reasoning over these graphs and extracting meaningful, language-agnostic data can further enhance the analysis of historical and contemporary newspapers but also parliamentary transcripts and literature. By constructing and visualizing comprehensive interaction graphs, we aim to significantly improve our understanding of relationships, sentiment, and narrative structures within these texts, providing a much-needed resource for Slovenian and other less-resource languages.
To investigate networks of people, organizations, places, and other entities across various corpora, we will adapt NER and NEL methods, such as (Prelevikj and Žitnik, 2021), for Slovenian and multilingual historical corpora. This will enable the construction of interaction graphs for analyzing significant entities and their connections in both contemporary and historical contexts. Complex network analysis will rank and visualize key entities and motifs in the networks. We will use pretrained, language-agnostic models like ULTRA to reason over the graphs.
Deliverables 4.1: Interaction graphs of historical named entities (M12). Visualization of extracted named entity graphs (M24).
Automated semantic change detection is the task of identifying and analyzing how the semantic meanings of words differ in different contexts, often across several years or decades. While several methods have been developed for this purpose, they are usually applied on small, specific gold-standard datasets (Schlechtwer et al., 2020) focusing on new lexical meanings. The challenge is how to adapt them to produce explainable results for use cases in the digital humanities, where nuanced changes in meaning will be detected. The objective of this task will be to produce novel semantic change detection methods, leveraging LLMs, to support diachronic analysis in DH.
We will extend our existing diachronic analysis approaches based on contextual embeddings (Montariol et al., 2021) with generative LLMs. The recent approach by Periti et al. (2024) for generating replacement words will be adapted for usage without fine-tuning. We will generate a distribution of likely word substitutions, and each substitution will have a ‘weight’ based on the word generation likelihood. In contrast to Periti et al. (2024), which assigns equal importance to each generated word, our approach will detect more nuanced semantic differences. We will test the methods on the gold standard for semantic change for Slovene (Pranjić et al., 2024), as well as on the diachronic changes of the poverty concept.
Deliverable 4.2: Novel methodology for diachronic analysis using LLMs (M24).
Analyzing materials in DH goes beyond the text and often incorporates multimodal materials. The application of vision-language models (VLMs) in DH, particularly for less-resource languages such as Slovenian, is unexplored, leaving a gap in the effective analysis of visual materials. While building VLMs from scratch requires large amounts of aligned language-image data, the challenge is to adapt the existing VLM models to specific domains using smaller specialized resources. For example, there is a pressing need to adapt LLMs and VLMs to challenging tasks such as OCR recognition/improvement, classification, and information retrieval to effectively support DH research.
We will adapt and apply the vision-language model (VLM) developed in T1.3 on several DH downstream tasks to be applied in WP5. We will fine-tune the model for historical image retrieval and analysis and OCR recognition by leveraging aligned multilingual historical corpora as additional training data to train specialized VLMs. The downstream task will include classifying types of visual material from historical documents, content analysis of specific sources (such as images in historical newspapers), and topics (e.g., investigating and extracting the thematic illustrations from the folkloristics domain of conflicts and the role of outlaw hero).
Deliverables 4.3: Dataset of images from Slovene historical periodicals (M24). VLM adapted for selected DH tasks (M30).
Many NLP tasks deal with dynamic documents, changing over time and containing revisions of previous contents (e.g., changes in legislation). Contradiction detection is a fundamental task in these domains, requiring comparing multiple documents for contradictions; however, finding the right documents to compare against can be challenging. A perspective approach for enabling a model to better detect contradictions is to use a retrieval augmented generation (RAG) system. An additional problem that RAG systems often do not consider is the temporal component of documents, as this information can provide an important additional context when deciding which documents are potentially relevant. This challenge aims to construct an RAG system that will support a contradiction detection model with relevant candidate documents that might contain contradictions with the original document. The main challenge is creating document embeddings that highlight potentially contradicting documents.
We will develop an RAG system to handle Slovene documents with the detection of contradictions. We will develop a novel model for constructing and storing vectorized document representations capable of recognizing potential contradictions between documents. This will involve identifying key entities and relationships within the text and fine-tuning the model to encode these elements into the document embeddings. By comparing the vectorized representations, we will integrate these strategies into an RAG system to detect contradictions between new and existing documents. The methodology will be applied in the T5.3 legal challenge.
Deliverable 4.4: A new RAG system for Slovenian capable of detecting contradictions in documents (M18).
Content:
marko.robniksikonja@fri.uni-lj.si
Duration of the project:
September 2024 – September 2027
Faculty of Computer and Information Science
Večna pot 113, SI-1000 Ljubljana, Slovenia
Room: R2.06 (2nd floor)
