--- abstract: |- Existing statistical approaches to natural language problems are very coarse approximations to the true complexity of language processing. As such, no single technique will be best for all problem instances. Many researchers are examining ensemble methods that combine the output of successful, separately developed modules to create more accurate solutions. This paper examines three merging rules for combining probability distributions: the well known mixture rule, the logarithmic rule, and a novel product rule. These rules were applied with state-of-the-art results to two problems commonly used to assess human mastery of lexical semantics -- synonym questions and analogy questions. All three merging rules result in ensembles that are more accurate than any of their component modules. The differences among the three rules are not statistically significant, but it is suggestive that the popular mixture rule is not the best rule for either of the two problems. altloc: [] chapter: ~ commentary: ~ commref: ~ confdates: 10-12 September 2003 conference: International Conference on Recent Advances in Natural Language Processing (RANLP-03) confloc: 'Borovets, Bulgaria' contact_email: ~ creators_id: [] creators_name: - family: Turney given: Peter honourific: '' lineage: '' - family: Littman given: Michael honourific: '' lineage: '' - family: Bigham given: Jeffrey honourific: '' lineage: '' - family: Shnayder given: Victor honourific: '' lineage: '' date: 2003 date_type: published datestamp: 2003-09-19 department: ~ dir: disk0/00/00/31/63 edit_lock_since: ~ edit_lock_until: ~ edit_lock_user: ~ editors_id: [] editors_name: [] eprint_status: archive eprintid: 3163 fileinfo: /style/images/fileicons/application_pdf.png;/3163/1/ranlp%2D03%2Dfinal%2Dversion.pdf full_text_status: public importid: ~ institution: ~ isbn: ~ ispublished: pub issn: ~ item_issues_comment: [] item_issues_count: 0 item_issues_description: [] item_issues_id: [] item_issues_reported_by: [] item_issues_resolved_by: [] item_issues_status: [] item_issues_timestamp: [] item_issues_type: [] keywords: ~ lastmod: 2011-03-11 08:55:20 latitude: ~ longitude: ~ metadata_visibility: show note: ~ number: ~ pagerange: 482-489 pubdom: FALSE publication: ~ publisher: ~ refereed: TRUE referencetext: |- Eric Brill and Jun Wu. Classifier combination for improved lexical disambiguation. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, volume 1, pages 191-195, 1998. David J. Chalmers, Robert M. French, and Douglas R. Hofstadter. High-level perception, representation, and analogy: a critique of artificial intelligence methodology. Journal of Experimental and Theoretical Artificial Intelligence, 4:185-211, 1992. Cathy Claman. 10 Real SATs. College Entrance Examination Board, 2000. Christiane Fellbaum. WordNet: An Electronic Lexical Database. The MIT Press, 1998. Radu Florian and David Yarowsky. Modeling consensus: Classifier combination for word sense disambiguation. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002), pages 25-32, 2002. Robert M. French. The computational modeling of analogy-making. Trends in Cognitive Sciences, 6(5): 200-205, 2002. Tom Heskes. Selecting weighting factors in logarithmic opinion pools. In Advances in Neural Information Processing Systems, 10, pages 266-272, 1998. Geoffrey E. Hinton. Products of experts. In Proceedings of the Ninth International Conference on Artificial Neural Networks (ICANN 99), volume 1, pages 1-6, 1999. Robert A. Jacobs. Methods for combining experts' probability assessments. Neural Computation, 7(5):867-888, 1995. Robert A. Jacobs, Michael I. Jordan, Steve J. Nowlan, and Geoffrey E. Hinton. Adaptive mixtures of experts. Neural Computation, 3:79-87, 1991. Mario Jarmasz and Stan Szpakowicz. Roget's thesaurus and semantic similarity. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP-03), in press, 2003. George Lakoff and Mark Johnson. Metaphors We Live By. University of Chicago Press, 1980. Thomas K. Landauer and Susan T. Dumais. A solution to Plato's problem: The latent semantic analysis theory of acquisition, induction and representation of knowledge. Psychological Review, 104(2):211-240, 1997. Michael L. Littman, Greg A. Keim, and Noam Shazeer. A probabilistic approach to solving crossword puzzles. Artificial Intelligence, 134(1-2):23-55, 2002. Robert E. Schapire. A brief introduction to boosting. In Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, pages 1401{1406, 1999. Egidio Terra and C. L. A. Clarke. Frequency estimates for statistical word similarity measures. In Proceedings of the Human Language Technology and North American Chapter of Association of Computational Linguistics Conference 2003 (HLT/NAACL 2003), pages 244-251, 2003. Peter D. Turney. Mining the web for synonyms: PMI-IR versus LSA on TOEFL. In Proceedings of the Twelfth European Conference on Machine Learning (ECML-2001), pages 491-502, 2001. Peter D. Turney and Michael L. Littman. Learning analogies and semantic relations. Technical Report ERB-1103, National Research Council, Institute for Information Technology, 2003. Peter D. Turney and Michael L. Littman. Measuring praise and criticism: Inference of semantic orientation from association. ACM Transactions on Information Systems, in press, 2003. Lei Xu, Adam Krzyzak, and Ching Y. Suen. Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Transactions on Systems, Man and Cybernetics, 22(3):418-435, 1992. relation_type: [] relation_uri: [] reportno: ~ rev_number: 12 series: ~ source: ~ status_changed: 2007-09-12 16:48:44 subjects: - comp-sci-stat-model - comp-sci-lang - ling-comput - ling-sem - comp-sci-mach-learn succeeds: ~ suggestions: ~ sword_depositor: ~ sword_slug: ~ thesistype: ~ title: "Combining independent modules to solve multiple-choice synonym and analogy problems\n" type: confpaper userid: 2175 volume: ~