Abstract
Automated writing evaluation (AWE) is a relatively new field of study that is highly cross-disciplinary and relies heavily on theory from linguistics, cognitive psychology, and the teaching of writing, and methods from natural language processing and machine learning, and educational measurement. Since the 1999 deployment of e-rater (ETS’s AWE system) for the Graduate Management Admissions Test (GMAT), a high-stakes, large-scale writing assessment, AWE has become a provocative research area in the education, measurement, and writing communities. AWE technology is now used to score millions of responses to high-stakes assessments. With the introduction of the Common Core State Standards Initiative’s focus on college-readiness, and its focus on writing skills for source use, argumentation, and audience, advanced research in AWE has become critical to develop methods to computationally capture writing quality with regard to those more advanced skills. This talk will provide an overview of AWE, and chart its growth based on the needs and expectations of the education, measurement, and writing communities. Studies in AWE and genre research will be discussed to illustrate how these studies have contributed to the development of AWE capabilities. A recent advanced capability for the evaluation of discourse coherence quality will be presented.

Biography
Jill Burstein is a Managing Principal Research Scientist in ETS’s R&D Division. Her background and expertise is in computational linguistics with a focus on education technology for writing, reading, and teacher professional development. She holds 13 patents for educational technology inventions. For writing-based technology, her inventions include e-rater, an automated essay scoring system, an essay-based discourse analysis system, and a style feedback capability – all used in CriterionSM, ETS’ online essay evaluation product. In more recent work, she is pursuing research to develop automated systems for detection of discourse coherence and source use in test-taker and student writing. She is the Principal Investigator on a grant funded by the U.S. Department of Education, Institute for Education Sciences, entitled: Technology-Assisted Generation of Linguistically-Relevant Instructional Activities to Support ELLs in Content and Language Learning in the Content Areas. The grant research focuses on the development of a tool (the Language Muse Activity Palette) that uses NLP supporting curriculum development through automated generation of classroom activities targeting English learners in content area, K-12 classrooms. Dr. Burstein received her B.A. in Linguistics and Spanish from New York University, and her M.A. and Ph.D. in Linguistics from the Graduate Center, City University of New York.