ab_test_report

Active

Tool of IA-QA — 130+ QA & Dev Tools for AI Agents

declared in 1.0.0

Generate an A/B test report comparing two prompts or model configurations. Accepts arrays of scores and returns statistical comparison: mean, median, std deviation, winner, and improvement percentage.

Parameters schema

{
  "type": "object",
  "required": [
    "variant_a",
    "variant_b"
  ],
  "properties": {
    "variant_a": {
      "type": "object",
      "properties": {
        "name": {
          "type": "string",
          "description": "Name/label for variant A"
        },
        "scores": {
          "type": "array",
          "items": {
            "type": "number"
          },
          "description": "Array of scores (0-100)"
        }
      },
      "description": "First variant configuration with name and score array"
    },
    "variant_b": {
      "type": "object",
      "properties": {
        "name": {
          "type": "string",
          "description": "Name/label for variant B"
        },
        "scores": {
          "type": "array",
          "items": {
            "type": "number"
          },
          "description": "Array of scores (0-100)"
        }
      },
      "description": "Second variant configuration with name and score array"
    }
  }
}

What this tool wraps· 0 endpoints

min confidence0.70 0.50

No endpoints wrapped at confidence ≥ 0.70.

Parent server

IA-QA — 130+ QA & Dev Tools for AI Agents

https://github.com/jcjamet/ia-qa

1/7 registries

View full server →