Presentation
While LLMs show impressive performance on a wide range of knowledge-intensive tasks, they also struggle on surprisingly trivial problems. One of the central critiques of the current deep learning paradigm is that our models are unable to learn representations of basic elements and rules for combining these into more complex structures. This ability, sometimes referred to as compositionally, is core to solving problems where curve fitting over a large training dataset is arguably not the best strategy. This has not only inspired a lot of recent research on compositionally and neural networks, but also research challenges, such as the ARC Prize. In this talk, Sondre will give an overview of both his own research on compositional generalization and recent works from abroad, focusing on the scenarios where current approaches fail, possible mitigations and the way forward.
Speaker
Sondre Wold is a PhD candidate at the Language Technology Group at the Department of Informatics, researching how we can better combine language models with structured knowledge representations, such as knowledge graphs.
Program
11:30 – Doors open and lunch is served
12:00 – "Compositional Generalization in Language Models" by Sondre Wold (PhD Candidate, Department of Informatics)
This event is open for all students, PhD candidates, postdocs, and everyone else who is interested in the topic. No registration needed.
About the seminar series
Once a month, dScience will invite you to join us for lunch and professional talks at the Science Library. In addition to these, we will serve lunch in our lounge in Kristine Bonnevies house every Thursday. Due to limited space (40 people), this will be first come, first served. See how to find us here.
Our lounge can also be booked by PhDs and Postdocs on a regular basis, whether it is for a meeting or just to hang out – we have fresh coffee all day long!