rerank_evaluate
ActiveTool of IA-QA — 130+ QA & Dev Tools for AI Agents
declared in 1.0.0
Evaluate RAG retrieval quality using the NVIDIA neural reranker (nv-rerankqa-mistral-4b-v3). Ranks passages by semantic relevance to a query and computes Precision@k and Recall@k. Optionally accepts ground-truth relevance labels to produce a PASS/FAIL CI/CD verdict.
Parameters schema
{
"type": "object",
"required": [
"query",
"passages"
],
"properties": {
"query": {
"type": "string",
"description": "The search query or question to rank against"
},
"top_k": {
"type": "integer",
"maximum": 10,
"minimum": 1,
"description": "k for Precision@k evaluation (default 3)"
},
"passages": {
"type": "array",
"items": {
"type": "object",
"required": [
"text"
],
"properties": {
"id": {
"type": "string"
},
"text": {
"type": "string"
},
"relevant": {
"type": "boolean",
"description": "Ground truth: is this passage relevant?"
}
}
},
"description": "Array of passage objects to rank (min 2, max 20)"
},
"threshold": {
"type": "number",
"maximum": 1,
"minimum": 0,
"description": "Minimum Precision@k to PASS (0-1, default 0.5)"
}
}
}Parent server
IA-QA — 130+ QA & Dev Tools for AI Agents
https://github.com/jcjamet/ia-qa
1/7 registries