Abstract
Mass deployment of computing technology depends upon ease-of-use and natural human/machine interfaces. Communication methods based upon the sensory modes preferred by the human–sight, sound and touch–are consequently being developed. While as yet primitive, multimodal interfaces depend centrally upon conversational interaction. And, integration of speech with visual and tactile capabilities centers on fusing simultaneous sensory inputs (which are often redundant, ambiguous or contradictory) to achieve reliable interpretation and action. This report describes one in-progress research effort in multimodal interfaces that employs simultaneous eye tracking, visual gesture, hands-free sound capture, speech recognition, text-to-speech synthesis, tactile force-feedback and manual gesture.
Biography
James Flanagan is Vice President for Research at Rutgers University. He is also Board of Governors Professor in Electrical and Computer Engineering. Rutgers is the State University of New Jersey, with an enrollment of 48,000 and a faculty and staff of 8,000. Flanagan joined Rutgers in 1990 after extended service in research and research management at AT&T Bell Laboratories. He was previously Director of Information Principles Research, with responsibilities in digital communications and information systems. Flanagan holds the S.M. and Sc.D. degrees in Electrical Engineering from the Massachusetts Institute of Technology. He has specialized in voice communications, computer techniques and electroacoustic systems, and has authored approximately 200 papers, 2 books, and 50 patents in these fields. Flanagan is a Fellow of the IEEE, the Acoustical Society of America, and the American Academy of Arts and Sciences. He has received a number of technical awards, and is a member of the National Academy of Engineering and the National Academy of Sciences. In 1996 he received the National Medal of Science at the White House.