Earlier this year, Chris Manning published an essay observing that “2015 was the year NLP felt the full force of the (deep learning) tsunami.” While expressing excitement for the success of these methods and the sudden burst of interest in NLP, he laments the turn away from the “domain science of language technology” and calls for research that further delves “into problems, approaches, and architectures.” In this talk I will present our group’s progress in developing novel approaches for capturing linguistic structure in deep learning-based models. In particular I will focus on end-to-end training of two unconstrained latent architectures designed to learn specific underlying structure: (1) a character-aware neural model that learns the morphological structure of words (Kim et al, 2016), and (2) a cluster-aware neural model that learns latent representations of discourse entities (Wiseman et al, 2016). While these are both “black-box” deep learning models, we can explore the internal representations to better understand what aspects of linguistic structure is being learned. I will present applications of these models to the tasks of language modeling, coreference resolution, machine translation, and grammar correction.

Alexander Rush is an Assistant Professor of Computer Science at Harvard University, and formerly a Post-doctorate Fellow at Facebook Artificial Intelligence Research (FAIR). He is interested in machine learning methods for natural language processing and understanding. His past work has introduced novel methods for large-scale structured prediction with applications to syntactic parsing and machine translation.