Beating the Perils of Non-convexity: Guaranteed Training of Neural Networks Using Tensor Methods – Anima Anandkumar (UC Irvine)
Baltimore, MD 21218
Training neural networks is a highly non-convex problem and in general is NP-hard. Local search methods such as gradient descent get stuck in spurious local optima, especially in high dimensions. We present a novel method based on tensor decomposition that trains a two layer neural network with guaranteed risk bounds for a large class of target functions with polynomial sample and computational complexity. We also demonstrate how unsupervised learning can help in supervised tasks. In our context, we estimate probabilistic score functions via unsupervised learning which are then employed for training neural networks using tensor methods.
Anima Anandkumar is a faculty at the EECS Dept. at U.C.Irvine since August 2010. Her research interests are in the area of large-scale machine learning and high-dimensional statistics. She received her B.Tech in Electrical Engineering from IIT Madras in 2004 and her PhD from Cornell University in 2009. She was a visiting faculty at Microsoft Research New England in 2012 and a postdoctoral researcher at the Stochastic Systems Group at MIT between 2009-2010. She is the recipient of the Alfred. P. Sloan Fellowship, Microsoft Faculty Fellowship, AFOSR & ARO Young Investigator Awards, NSF CAREER Award, IBM Fran Allen PhD fellowship, Best thesis award from ACM SIGMETRICS society, and paper awards from the ACM SIGMETRICS and IEEE Signal Processing societies.