Tutorials
The call for tutorials was here and the list of accepted tutorials is below.
1. Hyperparameter Optimization and Algorithm Selection: Practical Techniques, Theory, and New Frontiers
Dravyansh Sharma (TTIC)
Machine learning algorithms operate on data, and for any task the most effective method depends on the data at hand. Hyperparameter optimization and algorithm selection are therefore crucial to ensure the best performance in terms of accuracy, efficiency, reliability, interpretability, etc. We will start by surveying common techniques used in practice for hyperparameter optimization in machine learning including Bayesian optimization, gradient-based methods, and bandit-based approaches like Hyperband. Bayesian optimization, often using Gaussian processes, is effective for expensive evaluations but struggles with high-dimensional spaces and the performance is highly sensitive to the choice of priors and internal parameters. Gradient-based methods, such as differentiable hyperparameter optimization, offer efficiency but require smooth loss surfaces. Hyperband improves efficiency by early-stopping poor configurations. We will discuss chief advantages and shortcomings of these approaches, in particular limited theoretical guarantees, often requiring strong assumptions on how the performance of the algorithm varies with its hyperparameters which do not hold in practice.
We will then discuss exciting new developments on techniques for hyperparameter tuning with strong theoretical guarantees. A growing line of work over the past decade from the learning theory community has successfully analysed how the algorithmic performance actually varies with the hyperparameter for several fundamental algorithms in machine learning including decision trees, linear regression, unsupervised and semi-supervised learning, and very recently even deep learning. This has allowed the development of techniques that take this structure into account, apply naturally to both hyperparameter tuning and algorithm selection, work well in dynamic or online learning environments, and are equipped with provable PAC (probably approximately correct) guarantees for the generalization error of the learned hyperparameter. Future research areas include integration of these structure-aware principled approaches with the currently used techniques, better optimization in high-dimensional and discrete spaces, and improving scalability in distributed settings.
2. Counterfactuals in Minds and Machines
Tobias Gerstenberg (Stanford University), Manuel Gomez-Rodriguez (Max Planck Institute for Software Systems) and Stratis Tsirtsis (Max Planck Institute for Software Systems)
“Had I started writing the paper earlier, I would have made it to the UAI deadline.” The ability to think about how things could have turned out differently from how they did in reality, often referred to as counterfactual reasoning, is a fundamental aspect of human cognition. Is counterfactual reasoning a human capacity that machines cannot have? Surprisingly, recent advances at the interface of psychology, causality and machine learning have demonstrated that it is possible to build machines that perform and benefit from counterfactual reasoning, in a way similarly as humans do. In this tutorial, we will survey the state of the art in counterfactual reasoning in psychology, causality and machine learning, including its use in AI-assisted decision making, explainability, safety, fairness and reinforcement learning. Along the way, we will also showcase a technical demo building upon a popular mathematical model of counterfactual reasoning used in the field of machine learning.
3. Introduction to Bayesian Nonparametric Methods for Causal Inference
Michael Daniels (University of Florida) and Jason Roy (Rutgers University)
Bayesian nonparametric (BNP) methods can be used to flexibly model joint or conditional distributions, as well as functional relationships. These methods, along with causal assumptions, can be used with the g-formula for inference about causal effects. This general approach to causal inference has several possible advantages over popular semiparametric methods, including efficiency gains, the ease of causal inference on any functionals of the distribution of potential outcomes, and the use of prior information. Importantly, these BNP methods capture uncertainty, not just about the distributions and/or functions, but also about causal identification assumptions. In this workshop we review BNP methods and illustrate their use for causal inference in the setting of point treatments, mediation, and semi-competing risks. We present several data examples and discuss software implementation using R. The R code and/or packages used to run the data examples will be provided to the attendees at a specific github site associated with the presenters’ recent (2023) research monograph on the topic.
4. Causal Inference with Deep Learning and Generative Models
Murat Kocaoglu (Purdue University) and Md Musfiqur Rahman (Purdue University)
The modern deep learning revolution has led to groundbreaking results in machine learning
and AI and has rendered many tasks that were deemed extremely difficult possible, e.g., playing Go, solving the protein folding problem, or passing the Turing test with chat bots. Causal inference from observational data is another challenging problem with far-reaching consequences. Many researchers proposed novel methodologies trying to leverage the capabilities of deep learning for solving causal inference problems. However, the current literature is very scattered and unstructured as they come with assumptions that are sometimes not explicated enough, which makes it difficult for outsiders to gain a deep understanding of the state of the art. In this tutorial, our goal is to provide a clear account of the landscape. We will first establish the shortcomings of the classical causal inference algorithms for various data types. We will explain how researchers leveraged deep learning to address some of these challenges. We will provide a clear account of the accompanying assumptions of these algorithms. We will then focus on image datasets and express how deep generative models are used for answering causal questions in this challenging but practical setting using modern techniques such as diffusion models.
5. Modern Approximate Inference: Variational Methods and Beyond
Diana Cai (Flatiron Institute) and Yingzhen Li (Imperial College London)
Bayesian probabilistic modeling offers a principled framework for coherent inference and prediction under uncertainty. A central challenge in this framework is computing posterior distributions and Bayesian predictive quantities—tasks that often require approximate inference. Over the past decade, the field has seen significant progress, particularly through hybrid approaches that combine the strengths of both deterministic and stochastic methods. This tutorial provides a clear and unified overview of recent advances in approximate inference, with a focus on variational inference. We will highlight key methodological developments and emerging trends, including flow-based methods, score-based techniques, and connections to diffusion models, all of which have expanded the scope and impact of approximate inference in modern machine learning.
Last updated: May 17, 2025 19:22 (UTC)
Sponsors
