Stephan Mandt, Ph.D.




Disney Research Pittsburgh
4720 Forbes Avenue
Pittsburgh, PA 15213
firstname.lastname@disneyresearch.com

[ bio | news | research | publications | talks | CV ]

Short Bio

I am a research scientist at Disney Research Pittsburgh, where I lead the statistical machine learning group. From 2014 to 2016 I was a postdoctoral researcher with David Blei at Columbia University, and a PCCM Postdoctoral Fellow at Princeton University from 2012 to 2014. My Ph.D. supervisor was Achim Rosch at the Institute for Theoretical Physics at the University of Cologne (Germany). As a graduate student, I was supported by a fellowship of the German National Academic Foundation.


News

  • Postdoc positions and internships available! If you are interested in working with me, please send me an email with your CV. You should also write 1 or 2 sentences what explicitly interests you in a corporate research lab such as ours (status: July 2016).
  • I co-organize the NIPS 2016 workshop Advances in Approximate Bayesian Inference, together with Dustin Tran, Tamara Broderick, James McInerney and David Blei. More information will appear shortly at www.approximateinference.org.
  • Accepted papers at AISTATS 2016, ICML 2016, and NIPS 2016.

Research Interests

[Keywords: variational inference; stochastic optimization; scalable MCMC; topic models; recommender systems; word embeddings; deep generative models]

I work in the field of statistical machine learning, where my research centers on probabilistic modeling. In this field, we first posit a probability model that jointly models hidden variables and observed data, then use a Bayesian inference algorithm to discover the hidden variables from the data. There are many applications of this paradigm, including language models, recommendation systems, image clustering and denoising, and statistical genetics.

The growing amount of data calls for new scalable machine learning algorithms. To contribute to this goal, I integrate probabilistic modeling with stochastic optimization. Most of my research focusses on variational approximations that have their origin in statistical physics. For example, in one of my recent research projects I introduced local, per-datapoint temperatures into probability models, and then used a variational algorithm to learn the temperature distribution form the data. This results in a statistically more robust approach for a generic class of models, and to automatic per-datapoint annealing schedules. Another recent project focusses on the interpretation and analysis of stochastic gradient descent with constant learning rates as a form of inexact MCMC algorithm, where the learning rate can be tuned such as to sample from an approximate posterior. Most recently, my collaborators and I have generalized word embeddings to other forms of data and distributions.

Besides machine learning and its applications, I have been working in the field of non-equilibrium statistical physics. This field deals with non-parametric distributions that are defined by a stochastic process and relates to the convergence behavior of stochastic algorithms. In this field I worked on quantum spin dynamics, Boltzmann equations, hydrodynamics and random matrix theory with publications in Nature Physics and Physical Review Letters, among others. My work on thermodynamics with negative absolute temperatures inspired experimental physicists to the first realization of these states with free particles.

Publications

    2016

  • Exponential Family Embeddings
    M. Rudolph, F.J.R. Ruiz, S. Mandt, and D. Blei
    Neural Information Processing Systems (NIPS 2016). [PDF]
  • Huber-Norm Regularization for Linear Prediction Models
    O. Zadorozhnyi, G. Benecke, S. Mandt, T. Scheffer, M. Kloft
    European Conference on Machine Learning (ECML 2016). [PDF]
  • A Variational Analysis of Stochastic Gradient Algorithms
    S. Mandt, M. Hoffman, and D. Blei
    International Conference on Machine Learning (ICML 2016).   [PDF][poster]
  • Variational Tempering
    (Old title: Multicanonical Stochastic Variational Inference)
    S. Mandt, J. McInerney, F. Abrol, R. Ranganath, and D. Blei
    Artificial Intelligence and Statistics (AISTATS 2016)   [PDF]

    2015

  • Continuous-Time Limit of Stochastic Gradient Descent Revisited
    S. Mandt, M. Hoffman, and D. Blei
    NIPS Workshop on Optimization for Machine Learning (OPT 2015)   [PDF]
  • Finding Sparse Features in Strongly Confounded Medical Binary Data
    S. Mandt, F. Wenzel, S. Nakajima, J. P. Cunningham, C. Lippert, and M. Kloft
    NIPS Workshop on Machine Learning in Healthcare (2015)   [PDF]
  • Sparse Estimation in a Correlated Probit Model
    S. Mandt, F. Wenzel, S. Nakajima, J. P. Cunningham, C. Lippert, and M. Kloft
    Arxiv (2015).   [arXiv]
  • Stochastic Differential Equations for Quantum Dynamics of Spin-Boson Networks
    S. Mandt, D. Sadri, A. Houck, and H. Tureci
    New Journal of Physics 17 (2015) 053018.   [PDF]

    2014

  • Smoothed Gradients for Stochastic Variational Inference
    S. Mandt and D. Blei
    Neural Information Processing Systems (NIPS 2014)   [PDF]
  • Probit Regression with Correlated Label Noise: An EM-EP approach
    S. Mandt, F. Wenzel, J. Cunningham, and M. Kloft
    NIPS Workshop on Advances in Variational Inference (2014)   [PDF]
  • Comment on "Consistent thermostatistics forbids negative absolute temperatures"
    U. Schneider, S. Mandt, A. Rapp, S. Braun, H. Weimer, I. Bloch, and A. Rosch
    Arxiv (2014).   [arXiv]
  • Damping of Bloch oscillations: Variational solution of the Boltzmann equation beyond linear response
    S. Mandt
    Physical Review A 90, 053624 (2014).   [arXiv]

    Before 2014

  • Relaxation towards negative temperatures in bosonic systems: Generalized Gibbs ensembles and beyond integrability
    S. Mandt, A. Feiguin, S. Manmana
    Physical Review A 88, 043643 (2013).   [arXiv]
  • Ultrakalt und doch heißer als unendlich heiß. Erstmals gelang es, ein Quantengas bei negativen absoluten Temperaturen herzustellen
    S. Mandt
    Popular article on negative temperatures in the monthly proceedings of the German Physical Society.
    Physik Journal 12, March edition (2013)   [PDF]
  • Transport and Non-Equilibrium Dynamics in Optical Lattices. From Expanding Atomic Clouds to Negative Absolute Temperatures
    S. Mandt
    PhD thesis, University of Cologne (2012)   [PDF]
  • Fermionic transport in a homogeneous Hubbard model: Out-of-equilibrium dynamics with ultracold atoms
    U. Schneider, L. Hackermueller, J.P. Ronzheimer, S. Will, S. Braun, T. Best, I. Bloch, E. Demler, S. Mandt, D. Rasch, A. Rosch
    Nature Physics 8, 213-218 (2012).   [arXiv]
    Press: SciTechDaily, Pro-Physik (in German)
  • Interacting Fermionic Atoms in Optical Lattices Diffuse Symmetrically Upwards and Downwards in a Gravitational Potential
    S. Mandt, A. Rapp, A. Rosch
    Physical Review Letters 106, 250602 (2011).   [arXiv]
    Press: Nature
  • Equilibration rates and negative absolute temperatures for ultracold atoms in optical lattices
    A. Rapp, S. Mandt, A. Rosch
    Physical Review Letters 105, 220405 (2010).   [arXiv]
    Press: Nature , New Scientist, Science News
    Experimental realization of T<0 based on our theory: Braun et. al., Science 2013
  • Zooming in on local level statistics by supersymmetric extension of free probability
    S. Mandt, M.R. Zirnbauer
    J. Phys. A 43 (2010) 025201.   [arXiv]
  • Symmetric Spaces Toolkit
    H. Sebert and S. Mandt
    Lecture notes, SFB/TR 12, Langeoog (2007)   [PDF]

Invited Talks

  • California Institute of Technology
    Pasadena, CA. August 2016.
  • Data Science Colloquium, Rutgers University
    Newark, NJ. April 2016.
  • Google Research
    Mountain View, CA. April 2016.
  • Microsoft
    Sunnyvale, CA. March 2016.
  • CS Colloquium, University of Rhode Island
    Kingston, RI. March 2016.
  • Disney Research
    Pittsburgh, Pennsylvania. March 2016.
  • CS Colloquium, University of Colorado
    Boulder, Colorano. January 2016.
  • National Renewable Energy Laboratory
    Golden, Colorado. January 2016.
  • Adobe Research
    San Francisco, CA. June 2015.
  • Human Longevity Inc.
    Mountain View, CA. June 2015.
  • Dagstuhl Seminar
    Machine Learning with Interdependent and Non-identically Distributed Data.
    Dagstuhl, Germany. April 2015.
  • Humboldt University of Berlin
    Machine Learning Seminar, Berlin, Germany. February 2015.
  • Technical University of Berlin
    Machine Learning Seminar, Berlin, Germany. February 2015.
  • D-Wave Systems Inc.
    Burnaby, Canada. January 2015.
  • University of British Columbia
    Machine Learning Seminar, Vancouver, Canada. January 2015.
  • IBM Research
    Physical Sciences Seminar, Yorktown Heights, USA. October 2014.
  • Emergent Phenomena in the Dynamics of Quantum Matter
    (QUANTMAT 2014), New York, USA. April 2014.
  • University of Otago
    Theoretical Physics Seminar, Dunedin, New Zealand. February 2013.
  • Princeton Center for Theoretical Science
    Princeton University, Princeton, USA. March 2012.
  • Finite Temperature Non-Equilibrium Superfluid Systems
    (FINESS 2011), Heidelberg, Germany - September 2011.
  • University of Colorado
    Theoretical Physics Seminar, Boulder, CO. September 2010.
  • École Polytechnique
    Theoretical Physics Seminar, Palaiseau, France. March 2010.