Seminar in Deep Neural Networks (FS 2024)
Organization
When & Where: Tuesdays 10:15, ETZ E 9.
First seminar: February 20
Last seminar: May 28
Coordinators: Benjamin Estermann, Florian Grötschla, Roger Wattenhofer.
Spots: There are still spots available, please directly send us an email if you want to participate.
Background
This is a seminar, we will focus on recent reasearch and skip most of the basics. We assume that all participants are familiar with the fundamentals on deep neural networks. If you feel like you cannot follow the discussions, please check out this playlist, this lecture, the book by Francois Chollet on Deep Learning with Python, or any other lectures or books on deep neural networks.
Seminar Timeline
- We will publish a list of papers. You can then tell us what your preferences are.
- We will assign the papers based on a first-come first-serve principle. The presentations will also be scheduled in this order. Further, you will be assigned a mentor that is familiar with the topic and helps you with the preparation of your presentation.
- In the first week of the semester, there will be no presentations. Instead, we will give an introduction to the seminar and some tips on scientific presentations.
- After that, every week two students will present their respective papers.
Preparation Timeline
- Around 4 weeks before your talk: first meeting with your mentor where you discuss the structure of the talk.
- 4 to 1 week before your talk: you meet with your mentor as often as both parties find necessary to be making progress.
- At least 1 week before your talk: The presentation is ready, and you will present your presentation (as a test run) to your mentor only.
- Your mentor will get a copy of your test run presentation slides. (These slides will not become public, but they may influence your seminar grade.)
- Your mentor will give you feedback, and you are supposed to update your final presentation based on this feedback.
- Please send us your slides at least on the day before your presentation.
Your Presentation
- Your presentation should be 30 minutes long.
- After your presentation, you should organize a lively discussion about your presentation, for up to 15 minutes.
- It may help discussions if you also try to be critical about the presented work.
- Your presentation should take into account these presentation guidelines.
- Beyond these guidelines, you may find other useful tips about good scientific presentations online, for instance here, or here.
- All work copy/pasted from others (figures, explanations, examples, or equations) must be properly referenced.
Grade
The most important part of your grade will be the quality of your presentation, both content and style. In addition, we grade how well you motivate and direct the discussions with the audience, during and after the presentation. Also, we also grade how actively you participate in the discussions throughout the semester. And finally, we also value attendance and the quality of your mentor-only test presentation.
Papers
You can find the list of available papers here. Send us an ordered list (by preference) of up to 5 papers. We try to assign the papers first-come first-serve according to your preferences, while also taking into account the availability of the supervisor. To maximize the chance that you get a paper from your list, we recommend that you diversify the papers sufficiently. If you do not have any preference, still send us an e-mail and we will assign a paper to you.
Schedule
Date | Presenter | Title | Mentor | Slides |
---|---|---|---|---|
February 20 | Benjamin Estermann | Introduction to Scientific Presentations | - | [pdf] |
February 27 | Dennis Jüni Denis Tarasov |
Simple and Controllable Music Generation Direct Preference Optimization: Your Language Model is Secretly a Reward Model |
Luca Lanzendörfer Frédéric Berdoz |
[pdf] [pdf] |
March 5 | Jiaqing Xie |
Graph Inductive Biases in Transformers without Message Passing |
Florian Grötschla |
[pdf] |
March 12 | Yumi Kim Davide Guidobene |
AudioLDM: Text-to-Audio Generation with Latent Diffusion Models Maximally Expressive GNNs for Outerplanar Graphs |
Luca Lanzendörfer Florian Grötschla |
[pdf] [pdf] |
March 19 | Guiv Farmanfarmaian Eric Nothum |
Agree to Disagree: Diversity through Disagreement for Better Transferability Mamba: Linear-Time Sequence Modeling with Selective State Spaces |
Frédéric Berdoz Mattia Segu |
[pdf] [pdf] |
March 26 | Pyrros Koussios Zixuan Chen |
Siamese Masked Autoencoders Controlling Rate, Distortion, and Realism: Towards a Single Comprehensive Neural Image Compression Model |
Mattia Segu Till Aczel |
[pdf] [pdf] |
April 02 | - | Easter Break | - | - |
April 09 | Benjamin Jäger | Flow Factorized Representation Learning | Benjamin Estermann | [pdf] |
April 16 | Marco Giordano Lara Nonino |
Ultra-Low Precision 4-bit Training of Deep Neural Networks MLP-Mixer: An all-MLP Architecture for Vision |
Peter Belcak Peter Belcak |
[pdf] [pdf] |
April 23 | Carl Allen Thomas Kiefer |
Variational Classification: A Probabilistic Generalization of the Softmax Classifier Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistles |
- Benjamin Estermann |
- [pdf] |
April 30 | - | No seminar | - | - |
May 7 | - | No seminar | - | - |
May 14 | - | No seminar | - | - |
May 21 | Amir Joudaki Johannes Herter |
Towards Training Without Depth Limits: Batch Normalization Without Gradient Explosion Exphormer: Sparse Transformers for Graphs |
- Joël Mathys |
- [pdf] |
May 28 | - | No seminar | - | - |