====== 392107 Advances in Attention Networks ======
\\
| [[https://ekvv.uni-bielefeld.de/kvv_publ/publ/vd?id=515982373 | 392107]] | Schönhuth/Knop/Pianesi | Summer 2025 | Thu 12-14 in S1-146 (S) and Thu 10-12 in D2-152 (Ü) |
==== Contact ====
* Prof. Dr. Alexander Schönhuth: [[mailto:aschoen@cebitec.uni-bielefeld.de|aschoen@cebitec.uni-bielefeld.de]]
* Maren Knop: [[mailto:mknop@cebitec.uni-bielefeld.de|mknop@cebitec.uni-bielefeld.de]]
* Luna Pianesi: [[mailto:lpianesi@cebitec.uni-bielefeld.de|lpianesi@cebitec.uni-bielefeld.de]]
==== Presentations: ====
- Individual presentations
- In person in X-E0-218 Via zoom: https://uni-bielefeld.zoom-x.de/j/65877858945?pwd=9GPmnng77PeDNxaOmnqDcpaaY6Z7py.1
- To last for approx. 30 minutes, followed by discussion
- Present contents of scientific paper
- Video: How to Present
{{teaching:2024summer:how_to_present.mp4}}
==== Reports: ====
- Summary of every presentation of fellow students
- 600-800 words per summary
A presentation by Prof. Schönhuth in summer 2023 and winter 2022:
{{teaching:2023summer:privacy:howtowritereports.pdf|how-to-write-reports}}
[[teaching:2022winter:graphdata:howtowrite|How to write Scientific Reports]]
----
- Drafts can be submitted for discussion
- Improving drafts based on feedback
----
==== Papers - follow soon ====
\\
| | Topic | **Title** | **Authors** | **Year** | **Journal** |
| |SE(3)-Transformers | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/se_3_-transformers-_3d_roto-translation_equivariant_attention_networks.pdf |SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks]] | Fabian B. Fuchs et al. | 2020 | 34th Conference on Neural Information Processing Systems (NeurIPS 2020)|
| Alexander Hüdepohl |SE(3)-Transformers | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/diffdock-_diffusion_steps_twists_and_turns_for_molecular_docking.pdf |DIFFDOCK: DIFFUSION STEPS, TWISTS, AND TURNS FOR MOLECULAR DOCKING]] | Gabriele Corso et al. | 2023| ICLR |
| Aditya Bantwal |SE(3)-Transformers | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/equibind-_geometric_deep_learning_for_drug_binding_structure_prediction.pdf |EQUIBIND: Geometric Deep Learning for Drug Binding Structure Prediction]] | HannesStärk et al. | 2022 | 39th International Conference on Machine Learning |
| Maya Vienken |State Space Models | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/efficiently_modeling_long_sequences_with_structured_state_spaces.pdf |Efficiently Modeling Long Sequences with Structured State Spaces]] | Albert Gu et al. | 2022 | ICLR |
| Florian Drössler |State Space Models | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/hyenadna-_long-range_genomic_sequence_modeling_at_single_nucleotide_resolution.pdf |HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution]] | Eric Nguyen et al. | 2023 | NeurIPS 2023 |
| |State Space Models | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/mamba-_linear-time_sequence_modeling_with_selective_state_spaces.pdf |Mamba: Linear-Time Sequence Modeling with Selective State Spaces]] | Albert Gu et al. | 2023 | arXiv |
| Felix Erbarth |Model Interpretability | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/missing_values_and_imputation_in_healthcare_data-_can_interpretable_machine_learning_help_.pdf |Missing Values and Imputation in Healthcare Data: Can Interpretable Machine Learning Help?]] | Zhi Chen et al. | 2023 | Conference on Health, Inference, and Learning (CHIL) |
| Gregor Foitzik |Model Interpretability | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/node-gam-_neural_generalized_additive_model_for_interpretable_deep_learning.pdf |NODE-GAM: NEURAL GENERALIZED ADDITIVE MODEL FOR INTERPRETABLE DEEP LEARNING]] | Chun-Hao Chang et al. | 2022 | ICLR |
| Lisa Heihoff |Model Interpretability | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/explainable_automated_coding_of_clinical_notes_using_hierarchical_label-wise_attention_networks_and_label_embedding_initialisation.pdf |Explainable Automated Coding of Clinical Notes using Hierarchical Label-wise Attention Networks and Label Embedding Initialisation]] | Hang Dong et al. | 2021 | Journal of Biomedical Informatics|
| Valérie Witt |Generative Pre-Training(GPT)/ Natural Language Processing | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/improving_language_understanding_by_generative_pre-training.pdf |Improving Language Understanding by Generative Pre-Training]] | Alec Radford et al. | 2018 | OpenAI |
| Hakan Yildirim |Generative Pre-Training(GPT)/ Natural Language Processing | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/biogpt-_generative_pre-trained_transformer_for_biomedical_text_generation_and_mining.pdf |BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining]] | Renqian Luo et al. | 2022 | Briefings in Bioinformatics |
| Julia Fischer |Generative Pre-Training(GPT)/ Natural Language Processing | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/realformer-_transformer_likes_residual_attention.pdf |RealFormer: Transformer Likes Residual Attention]] | Ruining He et al. | 2021 | ACL-IJCNLP |
| Marcel Nieveler |Diffusion Model | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/antigen-specific_antibody_design_and_optimization_with_diffusion-based_generative_models_for_protein_structures.pdf |Antigen-Specific Antibody Design and Optimization with Diffusion-Based Generative Models for Protein Structures]] | Shitong Luo et al. | 2022 | NeurIPS|
| Konrad Breipohl |Diffusion Model | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/structured_denoising_diffusion_models_in_discrete_state-spaces.pdf |Structured Denoising Diffusion Models in Discrete State-Spaces]] | Jacob Austin et al. | 2023 | NeurIPS |
| |Diffusion Model | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/unsupervised_medical_image_translation_with_adversarial_diffusion_models.pdf |Unsupervised Medical Image Translation with Adversarial Diffusion Models]] | Muzaffer Öbey et al. | 2023 | arXiv |
| Achraf Halla |Social Networks | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/rumour_detection_based_on_graph_convolutional_neur.pdf |Rumour Detection Based on Graph Convolutional Neural Net]] | NA BAI et al. | 2021| IEEE Access ( Volume: 9) |
| |Social Networks | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/semi-supervisedly_co-embedding_attributed_networks.pdf |Semi-supervisedly Co-embedding Attributed Networks]] | Zaiqiao Meng et al. | 2019 | NeurIPS |
:!: You need to login in order to view the literature.
==== Time table seminar sessions ====
| **Date** | **Topic** |
|17.04.2025 | Prof. Schönhuth: Organization ({{teaching:2024summer:gnnsinbio:00-Organization.pdf|slides}}), How to present, How to write reports |
|24.04.2025 | How to write summaries |
|01.05.2025 | |
|08.05.2025 | |
|15.05.2025 | |
|22.05.2025 | |
|29.05.2025 | Deadline for choosing paper |
|05.06.2025 | |
|12.06.2025 | |
|19.06.2025 | |
|26.06.2025 | Block presentations: 8 people |
|03.07.2025 | Block presentations: 8 people |
|10.07.2025 | Block presentations: 8 people |
|17.07.2025 | Block presentations: 8 people |
|18.07.2025 | End of lectures |
|28.08.2025 | Deadline for report and paper journal |
|30.09.2025 | End of semester |