Lexical Disambiguation Based on Distributed Representations of Context Frequency

Abstract

A model for lexical disambiguation is presented that is based on combining the frequencies of past contexts of ambiguous words. The frequencies are encoded in the word representations and define the words' semantics. A Simple Recurrent Network (SRN) parser combines the context frequencies one word at a time, always producing the most likely interpretation of the current sentence at its output. This disambiguation process is most striking when the interpretation involves semantic flipping, that is, an alternation between two opposing meanings as more words are read in. The sense of ``throwing a ball'' alternates between ``dance'' and ``baseball'' as indicators such as the agent, location, and recipient are input. The SRN parser demonstrates how the context frequencies are dynamically combined to determine the interpretation of such sentences. We hypothesize that several other aspects of ambiguity resolution are based on similar mechanisms, and can be naturally approached from the distributed connectionist viewpoint.