Marie-Francine Moens

Date: Wednesday, March 8 at 2 pm
Place: LORIA, room A008
Speaker: Marie-Francine Moens (KU Leuven)

Title: Acquiring Knowledge from Multimodal Sources to Aid Language Understanding

Abstract: Human language understanding (HLU) by a machine is of large economic and social value. In this lecture we consider language understanding of written text. First, we give an overview of the latest methods for HLU that map language to a formal knowledge representation which facilitates other automated tasks. Most current HLU systems are trained on texts that are manually annotated, which are often lacking in open domain applications. In addition, much content is left implicit in a text, which when humans read a text is inferred by relying on their world and common sense knowledge. We go deeper into the field of representation learning that nowadays is very much studied in computational linguistics. This field investigates methods for representing language as statistical concepts or as vectors, allowing straightforward methods of compositionality. The methods often use deep learning and its underlying neural network technologies to learn concepts from large text collections in an unsupervised way (i.e., without the need for manual annotations). We show how these methods can help, but also demonstrate that these methods are still insufficient to automatically acquire the necessary background knowledge and more specifically world and common sense knowledge needed for language understanding. We go deeper in on how we can learn knowledge jointly from textual and visual data to help language understanding, which will be illustrated with the first results obtained in the MUSTER CHIST-ERA project.

Lire la suite

Michel Vacher

Date: Wednesday, 15th March 2017 at 2pm
Place: LORIA, room A008
Speaker: Michel Vacher (CNRS – LIG)

Title: Automatic recognition of atypical speech in smart homes: application to AAL

Abstract: About one third of the French population will be over 65 by the year 2050. Due to the lack of space in dedicated institutions for the elderly, home care services are a major concern and would benefit from technological assistance to relieve the work of the caregivers. This is the goal pursued by intelligent housing, which consists in providing houses equipped with computer technology to assist their inhabitants in the various situations of domestic life as well as in terms of comfort and security. Automatic Speech Recognition (ASR) could be an essential input in critical or abnormal situations, that is, when a surveillance system is most useful.

An ASR system adapted to this use must be adapted to the vocal characteristics of these persons. This implies meeting two requirements: on the one hand recognizing the voices of elderly and even aged persons and on the other hand recognizing calls for help made by a person in distress.

After introducing the context of intelligent housing and our methodology, we will present our studies on automatic recognition concerning aged voice and expressive voice in this context. Given the lack of available data, we recorded suitable corpora in both cases. After highlighting the decline in performance introduced using a generic system, we will show some solutions using MLLR type adaptation.

Finally, experiments on voice command for home automation in intelligent houses involving potential users will highlight the challenges that still need to be addressed in order to allow the use of this technology by individuals living alone at home.

Lire la suite

Martin Heckmann

Date: Wednesday, 29th March 2017 at 2pm
Place: LORIA, room A008
Speaker: Martin Heckmann (Honda Research Institute Europe)

Title: Personalized speech interfaces

Abstract: In this presentation I will highlight recent results obtained at the Honda Research Institute Europe GmbH in the context of personalization of speech-based human-machine interfaces. I will first talk about the detection of word prominence. Thereby, I will discuss the performance of prominence detection from noisy audio signals, the contribution of additional visual information on the speaker’s face and head movements as well as different strategies to fuse the two modalities. After that I will present a method to adapt the prominence detection to an individual speaker. The method is inspired by fMLLR, a well-known method in GMM/HMM-based speech recognition systems, and adapted to the SVM-based prominence detection. Next, I will talk about an advanced driver assistance systems (ADAS) which we currently develop to support the driver in inner-city driving and which is controlled via speech. This system will allow the driver to flexibly formulate his requests for assistance while the situation develops. In particular, when facing a left turn at an intersection the driver can delegate the task of observing the right side traffic to the system as he would do to a co-driver. The system will then inform him when there is an appropriate gap in the traffic to make the turn. Results of a user study we performed show that drivers largely prefer our proposed system to an alternative visual system or driving without any assistance. In this context I will show results on the estimation of the individual driver’s left turning behavior. Based on these driver models the interaction with the driver can be personalized to further improve the usefulness of the system.

Lire la suite