| Prof. Anthony G. Cohn, School of Computing, University of Leeds |
| Thu 01 Dec 2016, 12:45 - 15:45 |
| 4.31/33, IF |
If you have a question about this talk, please contact: Steph Smith (ssmith32)
http://www.comp.leeds.ac.uk/agc/
Speaker: Prof. Anthony G. Cohn, School of Computing, University of Leeds
Title: Natural Language Acquisition and Grounding for Robotic Systems
Abstract: Prof Cohn will present a cognitively plausible novel framework capable of learning the grounding in a visual semantics and the grammar of natural language commands given to a robot in a table top environment. The input to the system consists of video clips of a manually guided robot arm, paired with natural language commands describing the action. No prior knowledge is assumed about the meaning of words, or the structure of the language, except that there are different classes of words (corresponding to verbs, spatial relations, and objects and their observable properties). The learning process automatically clusters the continuous perceptual spaces into concepts corresponding to linguistic input. A novel relational graph representation is used to build connections between language and vision. As well as the grounding of language to the perceptual semantics, the system also induces a set of probabilistic grammar rules. The knowledge learned can be used to parse novel linguistic commands involving previously unseen objects.