Using document similarity networks to evaluate retrieval systems
| Author(s) | Kailasam, Aparna | |
| Date Accessioned | 2010-09-28T05:08:09Z | |
| Date Available | 2010-09-28T05:08:09Z | |
| Publication Date | 2010 | |
| Abstract | Information Retrieval System Evaluation is important in order to determine the performance of a system’s ability in satisfying the information needs of users. Evaluation of retrieval systems requires constructing test collections, which consists of obtaining documents, topics (information needs) and a set of relevance judgments that indicate the relevant documents for a topic. For a small collection of documents, it is possible to judge every document for relevance. A large document collection will demand an enormous judging effort, which is unrealistic. We present a novel technique for evaluating retrieval systems using minimal human judgments. Our proposed solution initially selects a small ‘seed-set’ of documents which are judged for relevance. The relevance information from this set is then propagated through an adhoc network of document similarities to determine the relevance of other unjudged documents. The original judgments combined with the inferred judgments constitute the complete set of judgments that is used to compare the relative performance of different systems. Our results show that we can effectively compare different retrieval systems using very few relevance judgments and at the same time achieve a high correlation with the true rankings of systems. | en_US |
| Advisor | Carterette, Ben | |
| Degree | M.S. | |
| Department | University of Delaware, Department of Computer and Information Sciences | |
| URL | http://udspace.udel.edu/handle/19716/5649 | |
| Publisher | University of Delaware | en_US |
| dc.subject.lcsh | Information organization--Evaluation | |
| Title | Using document similarity networks to evaluate retrieval systems | en_US |
| Type | Thesis | en_US |