consistency_check

Active

Tool of IA-QA — 130+ QA & Dev Tools for AI Agents

declared in 1.0.0

Compare multiple LLM responses to the same prompt and detect inconsistencies using Jaccard word-overlap similarity and fact drift (number comparison). Fast, deterministic, no API key needed. Limitations: relies on surface-level word matching — "Paris is the capital of France" vs "Paris is the French capital" may score low despite semantic equivalence. For true semantic consistency, use run_semantic_tests with embedding mode. Essential for determinism testing.

Parameters schema

{
  "type": "object",
  "required": [
    "responses"
  ],
  "properties": {
    "responses": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "description": "Array of 2+ LLM responses to compare (same prompt, different runs)"
    },
    "check_facts": {
      "type": "boolean",
      "description": "Check for contradictory numbers/facts across responses (default: true)"
    }
  }
}

What this tool wraps· 0 endpoints

min confidence0.70 0.50

No endpoints wrapped at confidence ≥ 0.50.

Parent server

IA-QA — 130+ QA & Dev Tools for AI Agents

https://github.com/jcjamet/ia-qa

1/7 registries

View full server →