TR-2015001: A Survey and Critique of Facial Expression Synthesis in Sign Language Animation

Sign language animations can lead to better accessibility of information and services for people who are deaf and have low literacy skills in spoken/written languages. Due to the distinct word-order, syntax, and lexicon of the sign language from the spoken/written language, many deaf people find it difficult to comprehend the text on a computer screen or captions on a television. Animated characters performing sign language in a comprehensible way could make this information accessible. Facial expressions and other non-manual components play an important role in the naturalness and understandability of these animations. Their coordination to the manual signs is crucial for the interpretation of the signed message. Software to advance the support of facial expressions in generation of sign language animation could make this technology more acceptable for deaf people. In this survey, we discuss the challenges in facial expression synthesis and we compare and critique the state of the art projects on generating facial expressions in sign language animations. Beginning with an overview of facial expressions linguistics, sign language animation technologies, and some background on animating facial expressions, a discussion of the search strategy and criteria used to select the five projects that are the primary focus of this survey follows. This survey continues on to introduce the work from the five projects under consideration. Their contributions are compared in terms of support for specific sign language, categories of facial expressions investigated, focus range in the animation generation, use of annotated corpora, input data or hypothesis for their approach, and other factors. Strengths and drawbacks of individual projects are identified in the perspectives above. This survey concludes with our current research focus in this area and future prospects.

Food Line at a New York Trade School Social Event

This black and white photograph shows a large group of men lining up for dessert at a New York Trade School social event.

Speaker at a New York Trade School Social Event

A speaker is pictured along with other important guests on the dais at a New York Trade School social event. Black and white photograph.

Guests at a New York Trade School Social Event

This photograph features a table of guests at a New York Trade School social event. Photograph is black and white.

Charles W. Merideth at City Tech event

Charles W. Merideth is shown looking at pictures in a group at an event in the Atrium Building at City Tech. Charles W. Merideth was the sixth president of the City Tech. He was formerly installed on October 19, 1990. Before coming to City Tech, Merideth had a long career in science and higher education. Under Merideth, the College expanded the number of baccalaureate programs offered by the College.

Charles W. Merideth at a Multi-Cultural Week Event

Charles W. Merideth is pictured at the "Asia at a Glance"exhibit. The exhibit was a part of the May 1991 Multi-Cultural Week at City Tech. Charles W. Merideth was the sixth president of the City Tech. He was formerly installed on October 19, 1990. Before coming to City Tech, Merideth had a long career in science and higher education. Under Merideth, the College expanded the number of baccalaureate programs offered by the College.