====== 392107 Advances in Attention Networks ====== \\ | [[https://ekvv.uni-bielefeld.de/kvv_publ/publ/vd?id=515982373 | 392107]] | Schönhuth/Knop/Pianesi | Summer 2025 | Thu 12-14 in S1-146 (S) and Thu 10-12 in D2-152 (Ü) | ==== Contact ==== * Prof. Dr. Alexander Schönhuth: [[mailto:aschoen@cebitec.uni-bielefeld.de|aschoen@cebitec.uni-bielefeld.de]] * Maren Knop: [[mailto:mknop@cebitec.uni-bielefeld.de|mknop@cebitec.uni-bielefeld.de]] * Luna Pianesi: [[mailto:lpianesi@cebitec.uni-bielefeld.de|lpianesi@cebitec.uni-bielefeld.de]] ==== Presentations: ==== - Individual presentations - In person in X-E0-218 Via zoom: https://uni-bielefeld.zoom-x.de/j/65877858945?pwd=9GPmnng77PeDNxaOmnqDcpaaY6Z7py.1 - To last for approx. 30 minutes, followed by discussion - Present contents of scientific paper - Video: How to Present {{teaching:2024summer:how_to_present.mp4}} ==== Reports: ==== - Summary of every presentation of fellow students - 600-800 words per summary A presentation by Prof. Schönhuth in summer 2023 and winter 2022: {{teaching:2023summer:privacy:howtowritereports.pdf|how-to-write-reports}} [[teaching:2022winter:graphdata:howtowrite|How to write Scientific Reports]] ---- - Drafts can be submitted for discussion - Improving drafts based on feedback ---- ==== Papers - follow soon ==== \\ | | Topic | **Title** | **Authors** | **Year** | **Journal** | | |SE(3)-Transformers | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/se_3_-transformers-_3d_roto-translation_equivariant_attention_networks.pdf |SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks]] | Fabian B. Fuchs et al. | 2020 | 34th Conference on Neural Information Processing Systems (NeurIPS 2020)| | Alexander Hüdepohl |SE(3)-Transformers | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/diffdock-_diffusion_steps_twists_and_turns_for_molecular_docking.pdf |DIFFDOCK: DIFFUSION STEPS, TWISTS, AND TURNS FOR MOLECULAR DOCKING]] | Gabriele Corso et al. | 2023| ICLR | | Aditya Bantwal |SE(3)-Transformers | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/equibind-_geometric_deep_learning_for_drug_binding_structure_prediction.pdf |EQUIBIND: Geometric Deep Learning for Drug Binding Structure Prediction]] | HannesStärk et al. | 2022 | 39th International Conference on Machine Learning | | Maya Vienken |State Space Models | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/efficiently_modeling_long_sequences_with_structured_state_spaces.pdf |Efficiently Modeling Long Sequences with Structured State Spaces]] | Albert Gu et al. | 2022 | ICLR | | Florian Drössler |State Space Models | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/hyenadna-_long-range_genomic_sequence_modeling_at_single_nucleotide_resolution.pdf |HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution]] | Eric Nguyen et al. | 2023 | NeurIPS 2023 | | |State Space Models | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/mamba-_linear-time_sequence_modeling_with_selective_state_spaces.pdf |Mamba: Linear-Time Sequence Modeling with Selective State Spaces]] | Albert Gu et al. | 2023 | arXiv | | Felix Erbarth |Model Interpretability | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/missing_values_and_imputation_in_healthcare_data-_can_interpretable_machine_learning_help_.pdf |Missing Values and Imputation in Healthcare Data: Can Interpretable Machine Learning Help?]] | Zhi Chen et al. | 2023 | Conference on Health, Inference, and Learning (CHIL) | | Gregor Foitzik |Model Interpretability | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/node-gam-_neural_generalized_additive_model_for_interpretable_deep_learning.pdf |NODE-GAM: NEURAL GENERALIZED ADDITIVE MODEL FOR INTERPRETABLE DEEP LEARNING]] | Chun-Hao Chang et al. | 2022 | ICLR | | Lisa Heihoff |Model Interpretability | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/explainable_automated_coding_of_clinical_notes_using_hierarchical_label-wise_attention_networks_and_label_embedding_initialisation.pdf |Explainable Automated Coding of Clinical Notes using Hierarchical Label-wise Attention Networks and Label Embedding Initialisation]] | Hang Dong et al. | 2021 | Journal of Biomedical Informatics| | Valérie Witt |Generative Pre-Training(GPT)/ Natural Language Processing | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/improving_language_understanding_by_generative_pre-training.pdf |Improving Language Understanding by Generative Pre-Training]] | Alec Radford et al. | 2018 | OpenAI | | Hakan Yildirim |Generative Pre-Training(GPT)/ Natural Language Processing | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/biogpt-_generative_pre-trained_transformer_for_biomedical_text_generation_and_mining.pdf |BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining]] | Renqian Luo et al. | 2022 | Briefings in Bioinformatics | | Julia Fischer |Generative Pre-Training(GPT)/ Natural Language Processing | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/realformer-_transformer_likes_residual_attention.pdf |RealFormer: Transformer Likes Residual Attention]] | Ruining He et al. | 2021 | ACL-IJCNLP | | Marcel Nieveler |Diffusion Model | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/antigen-specific_antibody_design_and_optimization_with_diffusion-based_generative_models_for_protein_structures.pdf |Antigen-Specific Antibody Design and Optimization with Diffusion-Based Generative Models for Protein Structures]] | Shitong Luo et al. | 2022 | NeurIPS| | Konrad Breipohl |Diffusion Model | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/structured_denoising_diffusion_models_in_discrete_state-spaces.pdf |Structured Denoising Diffusion Models in Discrete State-Spaces]] | Jacob Austin et al. | 2023 | NeurIPS | | |Diffusion Model | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/unsupervised_medical_image_translation_with_adversarial_diffusion_models.pdf |Unsupervised Medical Image Translation with Adversarial Diffusion Models]] | Muzaffer Öbey et al. | 2023 | arXiv | | Achraf Halla |Social Networks | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/rumour_detection_based_on_graph_convolutional_neur.pdf |Rumour Detection Based on Graph Convolutional Neural Net]] | NA BAI et al. | 2021| IEEE Access ( Volume: 9) | | |Social Networks | [[https://gds.techfak.uni-bielefeld.de/_media/teaching/literature/2024/semi-supervisedly_co-embedding_attributed_networks.pdf |Semi-supervisedly Co-embedding Attributed Networks]] | Zaiqiao Meng et al. | 2019 | NeurIPS | :!: You need to login in order to view the literature. ==== Time table seminar sessions ==== | **Date** | **Topic** | |17.04.2025 | Prof. Schönhuth: Organization ({{teaching:2024summer:gnnsinbio:00-Organization.pdf|slides}}), How to present, How to write reports | |24.04.2025 | How to write summaries | |01.05.2025 | | |08.05.2025 | | |15.05.2025 | | |22.05.2025 | | |29.05.2025 | Deadline for choosing paper | |05.06.2025 | | |12.06.2025 | | |19.06.2025 | | |26.06.2025 | Block presentations: 8 people | |03.07.2025 | Block presentations: 8 people | |10.07.2025 | Block presentations: 8 people | |17.07.2025 | Block presentations: 8 people | |18.07.2025 | End of lectures | |28.08.2025 | Deadline for report and paper journal | |30.09.2025 | End of semester |