arXiv · 2022-10-18
Making a MIRACL: Multilingual Information Retrieval Across a Continuum of Languages
Abrir registro de evidências
Pergunta de pesquisa
How can a retrieval benchmark cover languages with very different resource levels while using native queries and relevance judgments?
Método
An 18-language, same-language Wikipedia retrieval benchmark built with native-speaker queries and judgments; lexical, dense and hybrid baselines were compared on the released pools.
O que a evidência apoia
Evaluate each language explicitly, document corpus and judgment construction, and keep native review central. The reported hybrid result is a historical benchmark observation.
Limites e transferência
This is same-language information retrieval, not translation assessment or web SEO. Heuristic segmentation, candidate-pool gaps and unfinished test labels limit transferability; it says nothing about hreflang or Google rankings.
Classificação
- Sustentado
- Histórico
- Somente acadêmico
Influência na NAVINES: Laboratório de comparação de páginas multilíngues