URSABench: A System for Comprehensive Benchmarking of Bayesian Deep Neural Network Models and Inference Methods
Apr 1, 2022·
,,,,,·
0 min read
Brian Jalaian, Ph.D.
Meet Vadera
Jinyang Li
Adam Cobb
Tarek Abdelzaher
Benjamin Marlin
Abstract
While deep learning methods continue to improve in predictive accuracy on a wide range of application domains, significant issues remain with other aspects of their performance, including their ability to quantify uncertainty and their robustness. Recent advances in approximate Bayesian inference hold significant promise for addressing these concerns, but the computational scalability of these methods can be problematic when applied to large-scale models. In this paper, we present URSABench (the Uncertainty, Robustness, Scalability, and Accuracy Benchmark), an open-source suite of models, inference methods, tasks and benchmarking tools. URSABench supports comprehensive assessment of Bayesian deep learning models and approximate Bayesian inference methods, with a focus on classification tasks performed both on server and edge GPUs.
Type
Publication
Proceedings of Machine Learning and Systems
Authors
Associate Professor
Dr. Brian Jalaian is an Associate Professor at the University of West Florida and a Research Scientist at IHMC, where he leads cutting-edge work at the intersection of machine learning, AI assurance, and systems optimization. His research spans large language models (LLMs), AI model compression for edge deployment, uncertainty quantification, agentic and neurosymbolic AI, and trustworthy AI in medicine and defense. Formerly a senior AI scientist at the U.S. Army Research Lab and the DoD’s JAIC, Brian has shaped national efforts in robust, resilient, and testable AI. He’s passionate about building intelligent systems that are not only powerful—but provably reliable. When he’s not optimizing AI at scale, he’s mentoring the next generation of ML engineers or pushing the boundaries of agentic reasoning.