One query, three ways to rank it
Search a furniture and home catalog of about 43,000 products. Each method returns its top 10. For queries that people have labelled by hand, every result shows its label and each method gets a score.
Keyword
BM25. Every word in the query must appear.
Semantic
Sentence embeddings. Closest meaning wins.
Re-ranked
Blend of keyword and semantic signals with learned weights.
How to read this
- Labels come from the open WANDS dataset: people rated products for 480 queries as Exact, Partial or Irrelevant.
- Not labelled means nobody rated that product for that query. It counts as zero in the score, so a method can lose points for showing a good product that was never rated.
- nDCG@10 runs from 0 to 1. It rewards putting Exact matches at the top of the first ten results.
- Queries you type yourself have no labels, so they show results without a score.
The scorecard has the totals across all labelled queries and the data caveats.