Abstract

The successes of speech and language technology research in recent years has led to the deployment of  human language technology (HLT) in an increasing number of high-visibility, high-consequence applications.  Well-established approaches to evaluating HLT have been recognized as insufficient for understanding the performance and impacts these technologies have when moved from the lab to the real world, and this need has given new prominence to the importance of measurement in HLT and other machine learning enabled technologies.  This talk will review the foundations of metrology (the science of measurement), how metrology has been and can be further applied to measuring HLT, and current challenges in measuring HLT in practice.

Bio

Craig Greenberg is a Mathematician at the National Institute of Standards and Technology (NIST), where he oversees NIST’s Speaker Recognition Evaluation series and Language Recognition Evaluation series, and researches the measurement and evaluation of Artificial Intelligence (AI) and other topics in AI and machine learning. Prior to joining NIST, Dr. Greenberg worked as an English language annotator at the Institute for Research for Cognitive Science, as a programmer at the Linguistic Data Consortium, and as a research assistant in computational linguistics at the University of Pennsylvania. Dr. Greenberg received his PhD in 2020 from the University of Massachusetts Amherst with a dissertation on uncertainty and exact and approximate inference in flat and hierarchical clustering, his M.S. degree in Computer Science from University of Massachusetts Amherst in 2016, his M.S. degree in Applied Mathematics from Johns Hopkins University in 2012, his B.A. (Hons.) degree in Logic, Information, & Computation from the University of Pennsylvania in 2007, and his B.M. degree in Percussion Performance from Vanderbilt University in 2003. Among his accolades, Dr. Greenberg has received two official letters of commendation for his contributions to speaker recognition evaluation.