IN9550 ¨C Neural Methods in Natural Language Processing

Course content

This course studies a selection of advanced techniques in Natural Language Processing (NLP), with particular emphasis on modern research findings. The focus of the course is on "deep learning", a type of machine learning techniques using artificial neural networks.?Recently, natural language understanding systems based on deep neural models?such as ChatGPT?have revolutionized many spheres of our society and IN9550 allows students to look "under the hood" of such systems and to learn how they are built.

Topics typically include representation learning for words and other linguistic units, document classification, sequence tagging, natural language generation and other NLP tasks. They are solved using techniques like Feed-Forward Neural Networks (FFNN), Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs) and Transformers with self-attention. A special focus is put on using and designing large language models. In addition the course provides an introduction into biases and sustainability of modern deep learning methods in NLP.

The course includes strong practical components and puts emphasis on NLP problems and massive datasets of central importance in current research.?In the end of the course, the students are expected to? complete an experimental exam project and submit its summary in the form of a research paper. Thus, they will be prepared to further pursue an MSc project in deep learning based Natural Language Processing and/or Artificial Intelligence.

Learning outcome

Upon completion of this course you:

• are familiar with common techniques for learning dense representations (¡®embeddings¡¯) of natural language;
• understand the basics of various types of neural networks and their applications to natural language processing;
• can apply off-the-shelf NLP tools in meaningful ways to the data preparation for representation learning;
• have basic knowledge of the concepts of transfer and multi-task learning in application to natural language problems;
• have the necessary skills to train and fine-tune large language models for language understanding or generation
• can design, excecute, analyze, and summarize large-scale experiments in common neural network toolkits;
• know how to assess the benefits and challenges of neural learning in contrast to other common approaches in NLP;
• are able to identify and critically read relevant NLP research literature;
• have the ability to conduct in-depth error analysis of experimental results and make corresponding design adjustments.

Admission to the course

PhD candidates from the University of Oslo should apply for classes and register for examinations through?Studentweb. If a course has limited intake capacity, priority will be given to PhD candidates who follow an individual education plan where this particular course is included.

PhD candidates who have been admitted to another higher education institution must?apply for a position as a visiting student?within a given deadline.

Overlapping courses

• 10 credits overlap with IN5550 ¨C Nevrale metoder i spr?kprosessering.
• 5 credits overlap with INF5820 ¨C Language technological applications (discontinued).
• 5 credits overlap with INF9820 ¨C Language technological applications (discontinued).

Teaching

Four hours of instruction per week, mostly split into two hours of lectures and another two hours with hands-on (computer) laboratory work.

Mandatory assignments must be approved in order to qualify for the final exam. Previously approved assignments remain valid for one year.

Examination

Exam consists of an oral research presentation during the semester, a practical project and summary report of the practical project.

All three parts must be passed, and they must all be passed in the same semester.

It will also be counted as one of?your three?attempts to sit the exam for this course, if you sit the exam for one of the following courses:?IN5550 - Advanced Topics in Natural Language Processing

Grading scale

Grades are awarded on a pass/fail scale. Read more about?the grading system.

Resit an examination

In this course, postponed exams are not offered for exam candidates who are ill before the exam or who become ill during the exam. A deferred submission deadline may be offered. The illness must be documented with a doctor's certificate dated no later than the ordinary submission date. You must submit the doctor's certificate to the Student Administration at IFI before the submission deadline for the home exam.

More about examinations at UiO

• Use of sources and citations
• Special exam arrangements due to individual needs
• Withdrawal from an exam
• Illness at exams / postponed exams
• Explanation of grades and appeals
• Resitting an exam
• Cheating/attempted cheating

You will find further guides and resources at the web page on examinations at UiO.