Few-Shot Information Extraction: Pre-train, Prompt and Entail

About the talk:

Deep Learning has made tremendous progress in Natural Language Processing (NLP), where large pre-trained language models (PLM) fine-tuned on the target task have become the predominant tool. More recently, in a process called prompting, NLP tasks are rephrased as natural language text, allowing us to better exploit linguistic knowledge learned by PLMs and resulting in significant improvements. Still, PLMs have limited inference ability. In the Textual Entailment task, systems need to output whether the truth of a certain textual hypothesis follows from the given premise text.  Manually annotated entailment datasets covering multiple inference phenomena have been used to infuse inference capabilities to PLMs.

This talk will review these recent developments, and will present an approach that combines prompts and PLMs fine-tuned for textual entailment that yields state-of-the-art results on Information Extraction (IE) using only a small fraction of the annotations. The approach has additional benefits, like the ability to learn from different schemas and inference datasets. These developments enable a new paradigm for IE where the expert can define the domain-specific schema using natural language and directly run those specifications, annotating a handful of examples in the process. A user interface based on this new paradigm will also be presented. Beyond IE, inference capabilities could be extended, acquired and applied from other tasks, opening a new research avenue where entailment and downstream task performance improve in tandem.

Flip through Eneko’s slides:

Biography

Eneko Agirre is a Professor of Informatics and Head of HiTZ Basque Center of Language Technnology at the University of the Basque Country, UPV/EHU, in San Sebastian, Spain.

He has been active in Natural Language Processing and Computational Linguistics for decades. He received the Spanish Informatics Research Award in 2021, and is one of the 74 fellows of the Association of Computational Linguistics (ACL). He was President of ACL’s SIGLEX, member of the editorial board of Computational Linguistics, Journal of Artificial Intelligence Research and Action editor for the Transactions of the ACL. He is co-founder of the Joint Conference on Lexical and Computational Semantics (*SEM). Recipient of three Google Research Awards and five best paper awards and nominations. Dissertations under his supervision received best PhD awards by EurAI, the Spanish NLP society and the Spanish Informatics Scientific Association. He has over 200 publications across a wide range of NLP and AI topics. His research spans topics such as Word Sense Disambiguation, Semantic Textual Similarity, Unsupervised Machine Translation and resources for Basque. Most recently his research focuses on inference and deep learning language models.

‍