PROJECT 04 · SEARCH EVALUATION
SearchEval Lab
Updated: July 2026
Built a search evaluation and regression-detection platform for benchmarking retrieval quality, latency, and query-level failure behaviour across TF-IDF, BM25, and hybrid search systems.
Problem
Retrieval changes can improve aggregate relevance while hiding latency regressions or weak-query failures. SearchEval Lab evaluates those dimensions together against versioned evidence.
Technical Highlights
Python · Information Retrieval · TF-IDF · BM25 · Hybrid Search · FastAPI · CI
- Relevance and latency evaluation
- Versioned benchmark artifacts
- Query-level failure analysis
System Design
- Validated documents, queries, and relevance judgments
- TF-IDF, BM25, and hybrid retrieval through a search-engine factory
- Precision@k, Recall@k, MRR@k, and NDCG@k evaluation
- Average, minimum, and maximum latency measurement
- Versioned artifacts with configurable regression thresholds
- Weak-query analysis, Markdown reporting, CLI, FastAPI, tests, and CI
Architecture
Validated datasets feed a configurable search-engine factory. Evaluators compute relevance and latency metrics, compare versioned artifacts against thresholds, and produce query-level failure analysis and Markdown reports through CLI and FastAPI workflows.
Documented BM25 Results
- Precision@10
- 0.1900
- Recall@10
- 1.0000
- MRR@10
- 0.9500
- NDCG@10
- 0.9198
Takeaway
Search evaluation is most useful when relevance, latency, versioned evidence, and query-level failures are assessed together.