REINA at CLEF 2007 Robust Task

Abstract

This paper describes our work at CLEF 2007 Robust Task. We have participated in the monolingual (English, French and Portuguese) and the bilingual (English to French) subtask. At CLEF 2006 our research group obtained very good results applying local query expansion using windows of terms in the robust task. This year we have used the same expansion technique, but taking into account some criteria of robustness: MAP, GMAP, MMR, GS@10, P@10, number of failed topics, number of topics bellow 0.1 MAP, and number of topics with P@10=0. In bilingual retrieval experiments threemachine translation programs were used to translate topics. For the target language, translations were merged before performing a monolingual retrieval. We also applied the same local expansion technique. This year the results were disappointing. We think out that the reason is the difficulty to select the best measurement for robustness. Perhaps the problem is that all measurements are average results over all topics, but thehard topics are inherently hard and must be analyze separately. This year all our runs also ends up in good ranking, both base runs and expanded ones. We think that the reason is that we used a good information retrieval system, and the expansion technique is robust because it does not deteriorate significantly the retrieval performance.