IN5550 ¨C Neural Methods in Natural Language Processing

Course content

This course studies a selection of advanced techniques in Natural Language Processing (NLP), with particular emphasis on modern research findings. The focus of the course is on "deep learning", a type of machine learning techniques using artificial neural networks.?Recently, natural language understanding systems based on deep neural models?such as ChatGPT?have revolutionized many spheres of our society and IN5550 allows students to look "under the hood" of such systems and to learn how they are built.

Topics typically include representation learning for words and other linguistic units, document classification, sequence tagging, natural language generation and other NLP tasks. They are solved using techniques like Feed-Forward Neural Networks (FFNN), Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs) and Transformers with self-attention. A special focus is put on using and designing large language models. In addition the course provides an introduction into biases and sustainability of modern deep learning methods in NLP.

The course includes strong practical components and puts emphasis on NLP problems and massive datasets of central importance in current research.?In the end of the course, the students are expected to? complete an experimental exam project and submit its summary in the form of a research paper. Thus, they will be prepared to further pursue an MSc project in deep learning based Natural Language Processing and/or Artificial Intelligence.

Learning outcome

Upon completion of this course you:

• are familiar with common techniques for learning dense representations (¡®embeddings¡¯) of natural language;
• understand the basics of various types of neural networks and their applications to natural language processing;
• can apply off-the-shelf NLP tools in meaningful ways to the data preparation for representation learning;
• have basic knowledge of the concepts of transfer and multi-task learning in application to natural language problems;
• can design, excecute, analyze, and summarize large-scale experiments in common neural network toolkits;
• know how to assess the benefits and challenges of neural learning in contrast to other common approaches in NLP;
• are able to identify and critically read relevant NLP research literature;
• have the necessary skills to train and fine-tune large language models for language understanding or generation

Admission to the course

Students admitted at UiO must?apply for courses?in Studentweb. Students enrolled in Master's Degree Programmes not belonging to IFI can, on application, be admitted to the course if this is cleared by their own study programme.

Nordic citizens and applicants residing in the Nordic countries may?apply to take this course as a single course student.

If you are not already enrolled as a student at UiO, please see our information about?admission requirements and procedures for international applicants

Recommended previous knowledge

IN4080 ¨C Natural Language Processing

Overlapping courses

• 10 credits overlap with IN9550 ¨C Nevrale metoder i spr?kprosessering.
• 5 credits overlap with INF5820 ¨C Language technological applications (discontinued).
• 5 credits overlap with INF9820 ¨C Language technological applications (discontinued).

Teaching

Four hours of instruction per week, mostly split into two hours of lectures and another two hours with hands-on (computer) laboratory work.

Mandatory assignments must be approved in order to qualify for the final exam. Previously approved assignments remain valid for one year.

Examination

Exam consists of a practical project and summary report, both of which must be passed and both must be passed in the same semester.

It will also be counted as one of?your three?attempts to sit the exam for this course, if you sit the exam for one of the following courses:?IN9550 - Advanced Topics in Natural Language Processing

Grading scale

Grades are awarded on a pass/fail scale. Read more about?the grading system.

Resit an examination

In this course, postponed exams are not offered for exam candidates who are ill before the exam or who become ill during the exam. A deferred submission deadline may be offered. The illness must be documented with a doctor's certificate dated no later than the ordinary submission date. You must submit the doctor's certificate to the Student Administration at IFI before the submission deadline for the home exam.

More about examinations at UiO

• Use of sources and citations
• Special exam arrangements due to individual needs
• Withdrawal from an exam
• Illness at exams / postponed exams
• Explanation of grades and appeals
• Resitting an exam
• Cheating/attempted cheating

You will find further guides and resources at the web page on examinations at UiO.