Pre-trained Language Models for Summarisation


Pre-trained language (PLMs) models have been used to boost automatic summarisation methods, both extractive and abstractive. Extractive summarisation methods select key sentences from documents and concatenate them into a summary. Abstractive summarisation methods are more challenging since they generate informative sentences to create a consistent summary. Domain specific pre-training is important for domains such as biomedicine (BioBERT, ClinicalBERT, etc). Some of the PLM-based summarisation methods use features, fine-tuning and domain adaptation. There are several challenges such as the encoding of long documents, how to inject domain-specific knowledge into the models, interpretability, evaluation and controllable factuality of summaries (based on the interests of users) and benchmarking. This session will provide an overview of these challenges and opportunities of PLMs for text summarisation using the biomedical domain as an example.

Background Knowledge:

NLP, Deep learning methods


Sophia Ananiadou is Professor in Computer Science, Department of Computer Science, the University of Manchester. She is also Director of the National Centre for Text Mining (NaCTeM)); Deputy Director of the University’s Institute of Data Science and AI (IDSAI); Distinguished Research Fellow at the AI Research Centre of the National Institute of Advanced Industrial Science and Technology, Japan; Alan Turing Institute Fellow; Honorary Professor, University of the Aegean and Member of European Laboratory for Learning and Intelligent Systems Society. Her research interests evolved from abstract work on fragments of linguistic theory and logic to exploration of how AI systems could acquire and exploit knowledge of language, particularly in specialised domains (biomedicine, chemistry, exposome, law, public health). Research contributions include neural information extraction, text summarisation and simplification, emotion detection, terminology, development of resources (lexica, terminologies and labelled data), annotation tools and interoperable platforms for NLP workflows. She has developed tools such as the RobotAnalyst to improve evidence-based decisions, cut costs and improve efficiency and robustness of key policy decisions in public health.

Open Data Science




Open Data Science
One Broadway
Cambridge, MA 02142

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Consent to display content from Youtube
Consent to display content from Vimeo
Google Maps
Consent to display content from Google