Funding programme:

Project description:
The goal of the E-Inclusion Research Network is to provide creators with powerful audio-video tools that will allow them to center their attention on the creative aspects of projects and thus improve the richness of the multimedia experience for the sensory-disabled. This network aims to develop audio-visual content processing tools and sensory-specific content creation methods for multimedia producers. The expertise of the network researchers in speech recognition, audio processing, descriptive video, automatic audio-visual content extraction, human factors and human-machine interaction will be applied to produce descriptive video for the visually-impaired and captioned imagery for the hearing-impaired. This research program consists of 3 projects: Audio-Visual Content Extraction and Interaction : This project aims to develop software tools for the automatic extraction of audio-visual content in video documents for simplifying content accessibility and experimenting with new audio-visual content-based creation and user interaction. The project targets 2 main applications: Automatic film summarization based on automatic audio-visual content extraction and subtitling adaptation to visual scene activity. Speech and Audio Context Recognition: This project will concentrate research on two major problems that have to be solved in order to realise the potential of speech recognition technologies to facilitate some of the most labour-intensive aspects of media production and post-production, such as post-synchronization, captioning, script correction, and subtitling. First, core speech recognition technologies have to be improved and adapted to cultural content in order to reach a useful level of accuracy. Second, especially in cultural settings, it is not only speech but the whole audio context which conveys relational and emotional content. This project will investigate techniques to identify, classify and exploit this information to improve accuracy and provide end-users with more complete access to cultural content. Interactive Adaptable Descriptive Video: In this project, evaluations will be conducted to produce usable guidelines for the descriptive video creator. These evaluations will cover production issues not yet covered by actual research such as ascertaining the levels of description necessary to adapt to the eye-sight capacity of the visually-impaired or to provide listeners specific descriptions of moving-images which could be automatically detected in the creation process.

Description of work:
The E-inclusion research program requires a wide scope of expertises: speech recognition, audio processing, descriptive video, automatic audio-visual content extraction, perceptual interfaces, human factors and human-machine interaction, multimedia content producers and creators, software developers, designers, etc. These expertises are not present in any single organization. In addition, since the program is applied-research-oriented, the involvement of various type of users’ organization is critical and cannot be realized outside a formal network. The “value added” of using a Network is thus in the synergy it creates among all the network components (researchers, developers, creators, producers and users) around a common research theme. E-Inclusion Network Research partners currently include CRIM, McGill University, Université de Montréal, Université Laval, and École de Technologie Supérieure. Network participants currently include the Canadian Hard of Hearing Association, the Canadian National Institute for the Blind, Le Regroupement des québécois pour le sous-titrage, Le Regroupement des aveugles et amblyopes du Québec, Industry Canada , the National Film Board, the Legislative Assembly of British Columbia, Ryshco Media, Oralys Inc., Audiovision Inc and the National Research Council of Canada’s Institute of Information Technology. We are looking for Research partners join the E-Inclusion Research Network to: - Design and innovate multimedia production techniques and information systems for the producers and content creators; - Design and innovate multimedia accessibility systems (e.g. descriptive video, closed-captioning, web accessibility, etc.) for the sensory-disabled; - Automate media analysis, content extraction, and media asset management; and define the next generation of media standards. We are looking for industrial partners to add their knowledge, experience, and insight in media production, editing, annotation, retrieval, and reuse to. - Steer the E-Inclusion Research Network towards the needs of the end-users, content creators, and content producers. - Productize the technologies, systems, techniques and processes produced by the E-Inclusion Research Network.