Language association from human motion symbols by using latent semantics of natural language corpus

Humanoid robots are expected to work with humans. Such robots have to understand various information, for example, human language, human motion, and objects in environment. Humans can understand various information by using natural language. I will talk about the method to understand human motion by using natural language. The framework consists of three kinds of stochastic models. The first one recognizes motion. The second one associates words with human motions. The third one represent the probability of sequence of words. These models are trained with annotated motion data and big natural language corpus that are not directly related to the motions so that relationship between language and motion are represented by using probability and motion can be interpreted as natural language sentences.