Danica J. Sutherlandshe
Assistant Professor, UBC Computer Science Canada CIFAR AI Chair, Amii
UBC Machine Learning
AML-TN
MILD (ML theory)
CAIDA (AI)
PIHOT/Kantorovich Initiative (optimal transport)
Queer in AI
Name Change Policy Working Group
dsuth[at]cs.ubc.ca
or
djs[at]djsutherland.ml
CV
orcid
github
crossvalidated
bsky
Prospective students: Like most North American schools, we only accept applications through the departmental process, deadline December 15th. There is no need to email me about admissions; due to the volume of emails, I will probably not reply.
Anyone with specific research connections / questions / etc should feel free to get in touch at any time, via email / Bluesky DM / whatever.
Trans and gender-expansive or other queer people, also please reach out whenever, about specific things or just saying hi. Consider using my personal email (the djsutherland.ml one) for privacy reasons; Bluesky DMs or Queer in AI's Slack are also good.
Publications and selected talks are listed below.
You may come across various old items referring to me with a different first name. Please only use the name Danica to cite or refer to me, and check that your old .bib entries are correct, e.g. by replacing them with the entries here.
Group
- Wonho Bae (PhD, 2020–)
- Zheng He (PhD-track MSc, 2023–)
- Bingshan Hu (joint postdoc, 2023–)
- Yi (Joshua) Ren (PhD, 2020–)
- Hamed Shirzad (PhD, 2021–)
- Aaron Wei (BSc, 2024–)
- Nathaniel Xu (PhD, 2023–)
- Achinth Bharadwaj (BSc 2023)
- Namrata Deka (MSc 2023)
- Anubhav Garg (course MSc, 2024)
- Milad Jalali Asadabadi (MSc 2023)
- Arsh Jhaj (course MSc, 2023)
- Mohamad Amin Mohamadi (MSc 2023)
Courses
Publications
Below, ** denotes equal contribution, and this colour one of my students. Also available as a .bib file, and most of these are on Semantic Scholar. If you must (but I'd rather you didn't), here's Google Scholar.
Preprints
On the Effect of Negative Gradient in Group Relative Deep Reinforcement Optimization.
Preprint
2025.
[arXiv]
Journal and Low-Acceptance-Rate Conference Papers
Uncertainty Herding: One Active Learning Method for All Label Budgets.
International Conference on Learning Representations
(ICLR)
2025.
Learning Dynamics of LLM Finetuning.
International Conference on Learning Representations
(ICLR)
2025.
Outstanding Paper Award.
Bias Amplification in Language Model Evolution: An Iterated Learning Perspective.
Neural Information Processing Systems
(NeurIPS)
2024.
[arXiv]
Generalized Coverage for More Robust Low-Budget Active Learning.
European Conference on Computer Vision
(ECCV)
2024.
[arXiv]
Exploring Active Learning in Meta-Learning: Enhancing Context Set Labeling.
European Conference on Computer Vision
(ECCV)
2024.
[arXiv]
Differentially Private Neural Tangent Kernels (DP-NTK) for Privacy-Preserving Data Generation.
Journal of Artificial Intelligence Research
(JAIR)
2024.
[arXiv]
AdaFlood: Adaptive Flood Regularization.
Transactions on Machine Learning Research
(TMLR)
2024.
Why Do You Grok? A Theoretical Analysis on Grokking Modular Addition.
International Conference on Machine Learning
(ICML)
2024.
[arXiv]
Optimistic Rates: A Unifying Theory for Interpolation Learning and Regularization in Linear Regression.
ACM/IMS Journal of Data Science
(JDS)
1, 2.
2024.
[arXiv]
Improving Compositional Generalization using Iterated Learning and Simplicial Embeddings.
Neural Information Processing Systems
(NeurIPS)
2023.
[arXiv]
Exphormer: Scaling Graph Transformers with Expander Graphs.
International Conference on Machine Learning
(ICML)
2023.
A Fast, Well-Founded Approximation to the Empirical Neural Tangent Kernel.
International Conference on Machine Learning
(ICML)
2023.
[arXiv]
Queer in AI: A Case Study in Community-Led Participatory AI.
ACM Conference on Fairness, Accountability, and Transparency
(FAccT)
2023.
Best Paper award.
[arXiv]
Efficient Conditionally Invariant Representation Learning.
International Conference on Learning Representations
(ICLR)
2023.
Selected as notable (top 5%), i.e. as an oral.
How to prepare your task head for finetuning.
International Conference on Learning Representations
(ICLR)
2023.
MMD-B-Fair: Learning Fair Representations with Statistical Testing.
Artificial Intelligence and Statistics
(AISTATS)
2023.
[arXiv]
Pre-trained Perceptual Features Improve Differentially Private Image Generation.
Transactions on Machine Learning Research
(TMLR)
2023.
A Non-Asymptotic Moreau Envelope Theory for High-Dimensional Generalized Linear Models.
Neural Information Processing Systems
(NeurIPS)
2022.
[arXiv]
Making Look-Ahead Active Learning Strategies Feasible with Neural Tangent Kernels.
Neural Information Processing Systems
(NeurIPS)
2022.
[arXiv]
Evaluating Graph Generative Models with Contrastively Learned Features.
Neural Information Processing Systems
(NeurIPS)
2022.
[arXiv]
Object Discovery via Contrastive Learning for Weakly Supervised Object Detection.
European Conference on Computer Vision
(ECCV)
2022.
One Weird Trick to Improve Your Semi-Weakly Supervised Semantic Segmentation Model.
International Joint Conference on Artificial Intelligence
(IJCAI)
2022.
[arXiv]
Better Supervisory Signals by Observing Learning Paths.
International Conference on Learning Representations
(ICLR)
2022.
Uniform Convergence of Interpolators: Gaussian Width, Norm Bounds and Benign Overfitting.
Neural Information Processing Systems
(NeurIPS)
2021.
Selected for oral presentation.
Self-Supervised Learning with Kernel Dependence Maximization.
Neural Information Processing Systems
(NeurIPS)
2021.
Meta Two-Sample Testing: Learning Kernels for Testing with Limited Data.
Neural Information Processing Systems
(NeurIPS)
2021.
Does Invariant Risk Minimization Capture Invariance?
Artificial Intelligence and Statistics
(AISTATS)
2021.
Selected for oral presentation.
On Uniform Convergence and Low-Norm Interpolation Learning.
Neural Information Processing Systems
(NeurIPS)
2020.
Selected for spotlight presentation.
Learning Deep Kernels for Non-Parametric Two-Sample Tests.
International Conference on Machine Learning
(ICML)
2020.
Learning deep kernels for exponential family densities.
International Conference on Machine Learning
(ICML)
2019.
On gradient regularizers for MMD GANs.
Neural Information Processing Systems
(NeurIPS)
2018.
Demystifying MMD GANs.
International Conference on Learning Representations
(ICLR)
2018.
Efficient and principled score estimation with Nyström kernel exponential families.
Artificial Intelligence and Statistics
(AISTATS)
2018.
Selected for oral presentation.
Bayesian Approaches to Distribution Regression.
Artificial Intelligence and Statistics
(AISTATS)
2018.
Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy.
International Conference on Learning Representations
(ICLR)
2017.
Dynamical Mass Measurements of Contaminated Galaxy Clusters Using Machine Learning.
The Astrophysical Journal
(ApJ)
831, 2, 135.
2016.
Linear-time Learning on Distributions with Approximate Kernel Embeddings.
AAAI Conference on Artificial Intelligence
(AAAI)
2016.
On the Error of Random Fourier Features.
Uncertainty in Artificial Intelligence
(UAI)
2015.
Chapter 3 / Section 4.1 of my thesis supersedes this paper, fixing a few errors in constants and providing more results.
Active Pointillistic Pattern Search.
Artificial Intelligence and Statistics
(AISTATS)
2015.
A Machine Learning Approach for Dynamical Mass Measurements of Galaxy Clusters.
The Astrophysical Journal
(ApJ)
803, 2, 50.
2015.
Active learning and search on low-rank matrices.
Knowledge Discovery and Data Mining
(KDD)
2013.
Selected for oral presentation.
Nonparametric kernel estimators for image classification.
Computer Vision and Pattern Recognition
(CVPR)
2012.
Dissertations
Scalable, Flexible, and Active Learning on Distributions.
Computer Science Department, Carnegie Mellon University. Ph.D. thesis,
2016.
[pdf]
Integrating Human Knowledge into a Relational Learning System.
Computer Science Department, Swarthmore College. B.A. thesis,
2011.
[pdf]
Technical Reports, Posters, etc.
Normalization Matters for Optimization Performance on Graph Neural Networks.
Optimization for Machine Learning
(OPT), NeurIPS
2024.
A Theory for Compressibility of Graph Transformers for Transductive Learning.
Machine Learning and Compression, NeurIPS
2024.
[arXiv]
Understanding Simplicity Bias towards Compositional Mappings via Learning Dynamics.
Compositional Learning: Perspectives, Methods, and Paths Forward, NeurIPS
2024.
[arXiv]
Differentially Private Neural Tangent Kernels for Privacy-Preserving Data Generation.
Privacy-Preserving Artificial Intelligence
(PPAI-24), AAAI
2024.
Low-Width Approximations and Sparsification for Scaling Graph Transformers.
New Frontiers in Graph Learning, NeurIPS
2023.
Grokking modular arithmetic can be explained by margin maximization.
Mathematics of Modern Machine Learning, NeurIPS
2023.
Learning Privacy-Preserving Deep Kernels with Known Demographics.
Privacy-Preserving Artificial Intelligence
(PPAI-22), AAAI
2022.
How to Make Virtual Conferences Queer-Friendly: A Guide.
Workshop on Widening NLP
(WiNLP), EMNLP
2021.
[web]
Bayesian Approaches to Distribution Regression.
Learning on Distributions, Functions, Graphs and Groups, NeurIPS
2017.
Selected for oral presentation.
Understanding the 2016 US Presidential Election using ecological inference and distribution regression with census microdata.
Technical report
2016.
List Mode Regression for Low Count Detection.
IEEE Nuclear Science Symposium
(IEEE NSS/MIC)
2016.
Linear-time Learning on Distributions with Approximate Kernel Embeddings.
Feature Extraction: Modern Questions and Challenges, NeurIPS
2015.
Grounding Conceptual Knowledge with Spatio-Temporal Multi-Dimensional Relational Framework Trees.
Technical report
2012.
[pdf]
Invited talks
Slides for conference and workshop talks directly for a paper are linked next to the paper above.
Data-efficient learning, in general and in LLM preference tuning.
February 2025.
Snowflake.
Related papers:
ECCV-24, ICLR-25, ICLR-25.
[slides]
Expander Graphs and Low-Distortion Embeddings for Learning on Graphs.
December 2024.
Mathematics of Machine Learning, CMS.
Related papers:
ICML-23, NeurIPS-24, Machine Learning and Compression 2024.
[slides]
Scaling Graph Transformers with Sparse and Sparsified Attention.
November 2024.
Learning on Graphs Vancouver Meetup
(LoG-Vancouver).
Related papers:
ICML-23, NeurIPS-24, Machine Learning and Compression 2024.
[slides]
Scaling Graph Transformers with Expander Graphs.
June 2024.
Simon Fraser University, AI seminar.
Related papers:
ICML-23, NeurIPS-24.
[slides]
Conditional independence measures for fairer, more reliable models.
February 2024.
Statistical Aspects of Trustworthy Machine Learning, Banff International Research Station.
Related papers:
ICLR-23, Preprint 2024.
Learning conditionally independent representations with kernel regularizers.
June 2023.
Lifting Inference with Kernel Embeddings
(LIKE23), University of Bern.
Related papers:
ICLR-23.
[slides]
(Deep) Kernel Mean Embeddings for Representing and Learning on Distributions.
June 2023.
Lifting Inference with Kernel Embeddings
(LIKE23), University of Bern.
[slides]
[slides (pdf)]
Learning conditionally independent representations with kernel regularizers.
June 2023.
Gatsby25.
Related papers:
ICLR-23.
[slides]
Are these datasets different? Two-sample testing for data scientists.
April 2023.
Pacific Conference on Artificial Intelligence
(PCA).
Related papers:
AISTATS-23, NeurIPS-21, ICML-20, ICLR-17.
[slides]
[slides (pdf)]
A Defense of (Empirical) Neural Tangent Kernels.
March 2023.
AI Seminar, University of Michigan.
Related papers:
ICLR-22, ICLR-23, NeurIPS-22, ICML-23.
[slides]
[slides (pdf)]
Post-Publication Name Change Policies, Why they Matter, and Whether they Work.
March 2023.
Robotics DEI Seminar, University of Michigan.
[slides]
[slides (pdf)]
In Defence of (Empirical) Neural Tangent Kernels.
March 2023.
Microsoft Research Montréal.
Related papers:
ICLR-22, ICLR-23, NeurIPS-22, ICML-23.
[slides]
[slides (pdf)]
Neural Tangent Kernels, Finite and Infinite.
February 2023.
Winter School on Deep Learning, Indian Statistical Institute.
[slides]
Name Change Policies: A Brief (Personal) Tour.
November 2022.
Queer in AI Workshop, NeurIPS.
[slides]
[slides (pdf)]
Modern Kernel Methods in Machine Learning.
October 2022.
Research School on Uncertainty in Scientific Computing, Corsica
(ETICS).
Are These Datasets The Same? Learning Kernels for Efficient and Fair Two-sample Tests.
April 2022.
Toronto Womxn in Data Science Conference.
Related papers:
NeurIPS-21, ICML-20, ICLR-17.
[slides]
[slides (pdf)]
Are These Datasets The Same? Learning Kernels for Efficient and Fair Two-sample Tests.
February 2022.
TrustML Young Scientist Seminars.
Related papers:
NeurIPS-21, ICML-20, ICLR-17.
Better deep learning (sometimes) by learning kernel mean embeddings.
January 2022.
Lifting Inference with Kernel Embeddings
(LIKE22), University of Bern.
Related papers:
NeurIPS-21, NeurIPS-21.
[slides]
[slides (pdf)]
Can Uniform Convergence Explain Interpolation Learning?
November 2021.
NYU Center for Data Science Lunch Seminar Series.
Related papers:
NeurIPS-20, NeurIPS-21.
[slides]
[slides (pdf)]
Deep kernel-based distances between distributions.
January 2021.
Kickoff Workshop, Pacific Interdisciplinary Hub on Optimal Transport.
Related papers:
ICML-20, ICLR-17, NeurIPS-18, ICLR-18.
[slides]
[slides (pdf)]
Kernel Methods: From Basics to Modern Applications.
January 2021.
Data Science Summer School
(DS3), École Polytechnique, Paris.
Can Uniform Convergence Explain Interpolation Learning?
October 2020.
Penn State, Statistics colloquium.
Related papers:
NeurIPS-20.
[slides]
[slides (pdf)]
Interpretable Comparison of Distributions and Models.
December 2019.
Neural Information Processing Systems
(NeurIPS).
Related papers:
ICML-20, ICLR-17.
With Arthur Gretton and Wittawat Jitkrittum.
Better GANs by Using Kernels.
October 2019.
University of Massachusetts Amherst, College of Information and Computer Sciences.
Related papers:
ICLR-18, NeurIPS-18.
[slides]
Kernel distances between distributions for generative models.
July 2019.
Distance Metrics and Mass Transfer Between High Dimensional Point Clouds, ICIAM.
Related papers:
ICLR-18, NeurIPS-18.
[slides]
Learning with Positive Definite Kernels: Theory, Algorithms and Applications.
June 2019.
Data Science Summer School
(DS3), École Polytechnique, Paris.
With Bharath Sriperumbudur.
Introduction to Generative Adversarial Networks.
June 2019.
Machine Learning Crash Course
(MLCC), University of Genoa.
[slides]
Kernel Distances for Better Deep Generative Models.
September 2018.
Advances in Kernel Methods, GPSS.
Related papers:
ICLR-18, NeurIPS-18.
[slides]
Better GANs by using the MMD.
June 2018.
Facebook AI Research New York.
Related papers:
ICLR-18, NeurIPS-18.
[slides]
Efficiently Estimating Densities and Scores with Kernel Exponential Families.
June 2018.
Gatsby Tri-Center Meeting.
Related papers:
AISTATS-18.
[slides]
Better GANs by using the MMD.
June 2018.
Machine Learning reading group, Google New York.
Related papers:
ICLR-18, NeurIPS-18.
[slides]
Better GANs by using the MMD.
June 2018.
Machine Learning reading group, Columbia University.
Related papers:
ICLR-18, NeurIPS-18.
No slides actually used at the talk because of a projector mishap,
but they would have been the same as
the Google talk.
Advances in GANs based on the MMD.
May 2018.
Machine Learning Seminar, University of Sheffield.
Related papers:
ICLR-18, NeurIPS-18.
Efficient and principled score estimation with kernel exponential families.
December 2017.
Approximating high dimensional functions, Alan Turing Institute.
Related papers:
AISTATS-18.
Efficient and principled score estimation with kernel exponential families.
December 2017.
Computational Statistics and Machine Learning seminar, University College London.
Related papers:
AISTATS-18.
[slides]
Evaluating and Training Implicit Generative Models with Two-Sample Tests.
August 2017.
Implicit Models, ICML.
Related papers:
ICLR-17.
[slides]
Two-Sample Tests, Integral Probability Metrics, and GAN Objectives.
April 2017.
Theory of Generative Adversarial Networks, DALI.
Related papers:
ICLR-17.