Distributed Computing
ETH Zurich

Seminar in Deep Neural Networks (FS 2024)

Organization

When & Where: Tuesdays 10:15, ETZ E 9.
First seminar: February 20
Last seminar: May 28
Coordinators: Benjamin Estermann, Florian Grötschla, Roger Wattenhofer.
Spots: There are still spots available, please directly send us an email if you want to participate.

Background

This is a seminar, we will focus on recent reasearch and skip most of the basics. We assume that all participants are familiar with the fundamentals on deep neural networks. If you feel like you cannot follow the discussions, please check out this playlist, this lecture, the book by Francois Chollet on Deep Learning with Python, or any other lectures or books on deep neural networks.

Seminar Timeline

Preparation Timeline

Your Presentation

Grade

The most important part of your grade will be the quality of your presentation, both content and style. In addition, we grade how well you motivate and direct the discussions with the audience, during and after the presentation. Also, we also grade how actively you participate in the discussions throughout the semester. And finally, we also value attendance and the quality of your mentor-only test presentation.

Papers

You can find the list of available papers here. Send us an ordered list (by preference) of up to 5 papers. We try to assign the papers first-come first-serve according to your preferences, while also taking into account the availability of the supervisor. To maximize the chance that you get a paper from your list, we recommend that you diversify the papers sufficiently. If you do not have any preference, still send us an e-mail and we will assign a paper to you.

Schedule

Date Presenter Title Mentor Slides
February 20 Benjamin Estermann Introduction to Scientific Presentations - [pdf]
February 27 Dennis Jüni
Denis Tarasov
Simple and Controllable Music Generation
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Luca Lanzendörfer
Frédéric Berdoz
[pdf]
[pdf]
March 5 Jiaqing Xie
Graph Inductive Biases in Transformers without Message Passing
Florian Grötschla
[pdf]
March 12 Yumi Kim
Davide Guidobene
AudioLDM: Text-to-Audio Generation with Latent Diffusion Models
Maximally Expressive GNNs for Outerplanar Graphs
Luca Lanzendörfer
Florian Grötschla
[pdf]
[pdf]
March 19 Guiv Farmanfarmaian
Eric Nothum
Agree to Disagree: Diversity through Disagreement for Better Transferability
Mamba: Linear-Time Sequence Modeling with Selective State Spaces
Frédéric Berdoz
Mattia Segu
[pdf]
[pdf]
March 26 Pyrros Koussios
Zixuan Chen
Siamese Masked Autoencoders
Controlling Rate, Distortion, and Realism: Towards a Single Comprehensive Neural Image Compression Model
Mattia Segu
Till Aczel
[pdf]
[pdf]
April 02 - Easter Break - -
April 09 Benjamin Jäger Flow Factorized Representation Learning Benjamin Estermann [pdf]
April 16 Marco Giordano
Lara Nonino
Ultra-Low Precision 4-bit Training of Deep Neural Networks
MLP-Mixer: An all-MLP Architecture for Vision
Peter Belcak
Peter Belcak
TBA
TBA
April 23 Carl Allen
Thomas Kiefer
Variational Classification: A Probabilistic Generalization of the Softmax Classifier
Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistles
-
Benjamin Estermann
-
TBA
April 30 Johannes Herter Exphormer: Sparse Transformers for Graphs Joël Mathys TBA
May 7 No seminar - - -
May 14 TBA TBA TBA TBA
May 21 TBA TBA TBA TBA
May 28 Amir Joudaki Towards Training Without Depth Limits: Batch Normalization Without Gradient Explosion - -