Efficient Constrained Inference and Structured Neural Networks for Semantic Role Labeling

ABSTRACT: First, I will describe a dynamic programming algorithm for efficient constrained inference in semantic role labeling. The algorithm efficiently captures a majority of the structural constraints examined by prior work in this area, which has resorted to either approximate methods or slow integer linear programming solvers. In addition, it allows for structured learning, with respect to constrained conditional likelihood, which leads to improved predictions over a locally learned model.

Second, I will describe how the potential functions in the graphical model corresponding to the dynamic program can be replaced with neural networks. In addition to increased modeling power and automatically induced feature combinations, this allows us to embed phrasal arguments and semantic roles jointly in the same vector space, and provides a flexible framework for multi-task learning by the embedding of semantic roles from multiple annotation schemes in a shared vector space.

With these advances, both by themselves and combined, we obtain state-of-the-art results on both PropBank- and FrameNet-annotated datasets.

BIOGRAPHY: Oscar Täckström is a research scientist at Google in New York, where he works primarily on the semantic analysis of text and question answering from structured knowledge bases. Before joining Google in 2013, he was a PhD student in the computational linguistics group at Uppsala University and a research scientist at the Swedish Institute of Computer Science. In his thesis, he explored the use of incomplete and cross-lingual supervision for learning statistical models in natural language processing. Together with Ryan McDonald and Jakob Uszkoreit, he received the IBM Best Student Paper Award at NAACL 2012.