• Genome Data Science

    We develop methods and tools to work with tens of thousands of genomes and analyze and integrate the corresponding data.

    Model of DNA double helix in front of a student.
    © Universität Bielefeld
You are here: home » teaching » 2025summer » advattnet

392107 Advances in Attention Networks


392107 Schönhuth/Knop/Pianesi Summer 2025 Thu 12-14 in S1-146 (S) and Thu 10-12 in D2-152 (Ü)

Contact

Presentations:

- Individual presentations

- In person in X-E0-218 Via zoom: https://uni-bielefeld.zoom-x.de/j/65877858945?pwd=9GPmnng77PeDNxaOmnqDcpaaY6Z7py.1

- To last for approx. 30 minutes, followed by discussion

- Present contents of scientific paper

- Video: How to Present

Reports:

- Summary of every presentation of fellow students

- 600-800 words per summary

A presentation by Prof. Schönhuth in summer 2023 and winter 2022:

how-to-write-reports

How to write Scientific Reports

- Drafts can be submitted for discussion

- Improving drafts based on feedback

Papers - follow soon


Topic Title Authors Year Journal
SE(3)-Transformers SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks Fabian B. Fuchs et al. 2020 34th Conference on Neural Information Processing Systems (NeurIPS 2020)
Alexander Hüdepohl SE(3)-Transformers DIFFDOCK: DIFFUSION STEPS, TWISTS, AND TURNS FOR MOLECULAR DOCKING Gabriele Corso et al. 2023 ICLR
Aditya Bantwal SE(3)-Transformers EQUIBIND: Geometric Deep Learning for Drug Binding Structure Prediction HannesStärk et al. 2022 39th International Conference on Machine Learning
Maya Vienken State Space Models Efficiently Modeling Long Sequences with Structured State Spaces Albert Gu et al. 2022 ICLR
Florian Drössler State Space Models HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution Eric Nguyen et al. 2023 NeurIPS 2023
State Space Models Mamba: Linear-Time Sequence Modeling with Selective State Spaces Albert Gu et al. 2023 arXiv
Felix Erbarth Model Interpretability Missing Values and Imputation in Healthcare Data: Can Interpretable Machine Learning Help? Zhi Chen et al. 2023 Conference on Health, Inference, and Learning (CHIL)
Gregor Foitzik Model Interpretability NODE-GAM: NEURAL GENERALIZED ADDITIVE MODEL FOR INTERPRETABLE DEEP LEARNING Chun-Hao Chang et al. 2022 ICLR
Lisa Heihoff Model Interpretability Explainable Automated Coding of Clinical Notes using Hierarchical Label-wise Attention Networks and Label Embedding Initialisation Hang Dong et al. 2021 Journal of Biomedical Informatics
Valérie Witt Generative Pre-Training(GPT)/ Natural Language Processing Improving Language Understanding by Generative Pre-Training Alec Radford et al. 2018 OpenAI
Hakan Yildirim Generative Pre-Training(GPT)/ Natural Language Processing BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining Renqian Luo et al. 2022 Briefings in Bioinformatics
Julia Fischer Generative Pre-Training(GPT)/ Natural Language Processing RealFormer: Transformer Likes Residual Attention Ruining He et al. 2021 ACL-IJCNLP
Marcel Nieveler Diffusion Model Antigen-Specific Antibody Design and Optimization with Diffusion-Based Generative Models for Protein Structures Shitong Luo et al. 2022 NeurIPS
Konrad Breipohl Diffusion Model Structured Denoising Diffusion Models in Discrete State-Spaces Jacob Austin et al. 2023 NeurIPS
Diffusion Model Unsupervised Medical Image Translation with Adversarial Diffusion Models Muzaffer Öbey et al. 2023 arXiv
Achraf Halla Social Networks Rumour Detection Based on Graph Convolutional Neural Net NA BAI et al. 2021 IEEE Access ( Volume: 9)
Social Networks Semi-supervisedly Co-embedding Attributed Networks Zaiqiao Meng et al. 2019 NeurIPS

:!: You need to login in order to view the literature.

Time table seminar sessions

Date Topic
17.04.2025 Prof. Schönhuth: Organization (slides), How to present, How to write reports
24.04.2025 How to write summaries
01.05.2025
08.05.2025
15.05.2025
22.05.2025
29.05.2025 Deadline for choosing paper
05.06.2025
12.06.2025
19.06.2025
26.06.2025 Block presentations: 8 people
03.07.2025 Block presentations: 8 people
10.07.2025 Block presentations: 8 people
17.07.2025 Block presentations: 8 people
18.07.2025 End of lectures
28.08.2025 Deadline for report and paper journal
30.09.2025 End of semester