Chapter 7: Domain Applications in Chemistry, Biology, Physics and Geoscience

Introduction: Generative AI Across the Sciences

About This Chapter: Audience and Prerequisites

Part I: Chemistry & Materials Science

Scientific Foundations: Molecular Representations

What is SMILES?

From SMILES to Molecular Graphs

Molecular Representation Pipeline

Key Molecular Properties

Molecular Graph Learning and Diffusion Models for Drug Discovery

Full Working Code in Google Colab

1. Molecular Graph Learning for Property Prediction

GNN Message Passing Illustrated

Data Sources and Setup

Installing RDKit

Delaney ESOL Dataset (Aqueous Solubility)

Example: Loading the Dataset

1.1 Code: Molecular Graph Property Predictor

2. Molecular Generation with Diffusion Models

2.1 Graph Diffusion for Molecules

2.2 Conditional Generation

3. Geometry-Aware Molecular Generation

4. Crystal Structure Prediction with Diffusion

4.1 Crystal Diffusion Model

4.2 Training on Materials Project Data

4.3 Real-World Results and Limitations

4.4 Production-Ready Improvements

4.5 Active Learning Pipeline

5. Reaction Outcome Prediction with Transformers

6. Pitfalls and Extensions

Common Pitfalls

Extensions for Production

Summary

References

Part II: Biology & Biomedicine

AI in Healthcare: The Broader Landscape

A Landmark Milestone: AI-Designed Drug Succeeds in Phase 2a

The AI Drug Discovery Pipeline

Why This Chapter Focuses on Drug Discovery

Scientific Foundations: Proteins and Sequences

What is a Protein?

Amino Acids and Sequences

Protein Structure Hierarchy

Key Concepts for AI Models

1. Protein Structure Prediction

The Protein Folding Problem

ESM Architecture Overview

1.1 Using Pre-Trained ESMFold

1.2 Code: ESMFold Structure Prediction Wrapper

1.3 Using Smaller Models for Learning

2. Protein Sequence Generation

2.1 The Challenge and Realistic Expectations

2.2 Code: Protein Sequence Generation with ESM-2

2.3 Production-Grade Alternatives

3. Genomic Variant Analysis

3.1 The Variant Effect Prediction Problem

3.2 Code: Simplified Variant Effect Predictor

3.3 Production-Grade Variant Prediction

4. Clinical Trial Optimization

4.1 Code: Clinical Trial Design Assistant

4.2 Ethical and Regulatory Considerations

Summary: Biology & Biomedicine

References

Part III: Physics & Engineering

Learning Objectives

1. Machine Learning for Physical Systems

1.1 Domain-Specific Challenges

2. Particle Jet Classification with Graph Neural Networks

2.1 The Jet Tagging Problem

2.2 Code: Graph Attention Network for Jet Classification

2.3 Physics-Informed Features

2.4 Production Considerations for HEP

3. Spectroscopy Analysis with 1D CNNs

3.1 The Materials Characterization Problem

3.2 Code: 1D CNN for Spectroscopy Classification

3.3 Data Augmentation for Spectroscopy

Summary: Physics & Engineering

References

Part IV: Geoscience & Climate Applications

Section Overview

4.1 Hurricane Intensity Prediction

4.1.1 Motivation and Challenges

4.1.2 Data Sources and Features

4.1.3 Neural Network Architecture

Image Encoder (Satellite Data)

SST Encoder

Atmospheric Encoder

Fusion and Prediction

4.1.4 The Saffir-Simpson Scale

4.1.5 Training Considerations

4.1.6 Operational Considerations

4.2 Climate Downscaling with Super-Resolution

4.2.1 The Resolution Gap Problem

4.2.2 Super-Resolution CNN Architecture

4.2.3 Physics-Informed Constraints

4.2.4 Training Data and Evaluation

4.3 Seismic Event Detection with 1D CNNs

4.3.1 Earthquake Detection Problem

4.3.2 Seismic Waveform Detection Architecture

4.3.3 Training Strategy

4.4 Best Practices for Geoscience Deep Learning

4.4.1 Incorporating Physical Constraints

4.4.2 Data Challenges

4.4.3 Interpretability

4.4.4 Uncertainty Quantification

4.5 Exercises

Conceptual Questions

Coding Exercises

Research Extensions

Summary: Geoscience & Climate Applications

References

Part V: Cross-Cutting Applications in Deep Learning

5.1 Video Analysis for Biological Systems

5.1.1 Introduction: Behavior Recognition from Temporal Data

5.2 Baseline Model: Behavior from Perfect Poses

5.2.1 The Oracle Assumption

5.2.2 Model Architecture

5.2.3 Training Protocol

5.2.4 Results and Analysis

5.3 Realistic Model: Behavior from Raw Video

5.3.1 The Real-World Challenge

5.3.2 End-to-End Video Architecture

5.3.3 Training Results (from actual Colab)

5.4 Transfer Learning for Scientific Domains

5.4.1 The Transfer Learning Paradigm

5.4.2 Domain Adaptation with Adversarial Training

5.5 Multi-Task Learning

5.5.1 Benefits for Scientific Applications

5.5.2 Multi-Task Architecture

Summary: Cross-Cutting Applications

References