agent-qa

QA agent — verifies task completion

agent-qa is a KĀDI QA / verification agent that performs automated testing, code review, and quality verification for tasks in the AGENTS ecosystem. It subscribes to review events, can leverage LLMs (Anthropic or Model Manager) to enhance semantic code review, and uses the BaseAgent platform primitives (KadiClient, ProviderManager, MemoryService) to interact with brokers, memory, and other agents.

Architecture

• Base: built on agents-library’s BaseAgent which encapsulates broker connectivity (KadiClient), provider LLM management, and local memory.
• Data flow:
  • Connects to brokering networks (local and/or remote) defined in config.toml.
  • Listens for task review events (e.g., task.review_requested) and runs validation handlers.
  • Optionally calls LLM providers (Anthropic or Model Manager) via ProviderManager for semantic checks.
  • Uses native abilities when available (ability-file-local, ability-eval) for low-latency operations.
  • Persists/logs via configured arcadedb (config & vault credentials).
• Key runtime components:
  • BaseAgent: primary runtime object created from BaseAgentConfig and used to connect to broker and abilities.
  • ProviderManager: configured via BaseAgentConfig.provider to route LLM calls (primary/fallback).
  • MemoryService: local persistent memory configured via memory.DATA_PATH.
  • Handlers: setupValidationHandler (imported from ./handlers/validation.js) that registers event handlers for QA flows.
• How it fits in AGENTS:
  • Acts as a consumer QA agent that other task-producing agents send review requests to.
  • Uses abilities exposed by the platform (file, eval, secret, vision, log) to perform its checks.

Tools / API

The agent itself uses a small set of platform APIs and local handlers. The package registers/consumes abilities (see ability table below). The source imports and uses the following platform functions/APIs — these are the primary interaction points you will use while modifying the agent.

API / Function	Description	Key parameters / notes
BaseAgent (class)	Core agent runtime that provides KadiClient, ProviderManager, MemoryService.	Construct with BaseAgentConfig (see Configuration). Use baseAgent.connect(vault) to connect.
BaseAgentConfig (type)	Configuration shape passed to BaseAgent constructor.	Fields used: agentId, agentRole, version, brokerUrl, networks, additionalBrokers, provider, memory.
loadVaultCredentials()	Loads secrets from configured vaults (falling back to env).	Returns an object with vault keys (e.g., ANTHROPIC_API_KEY).
readConfig()	Reads config.toml and returns a helper to access fields.	Methods used: cfg.string(…), cfg.has(…), cfg.strings(…).
setLogLevel(level)	Sets the global logger level for the agent.	level examples: ‘info’, ‘debug’.
setAgentTag(tag)	Sets an identifying tag used in logs.
logger (object)	Shared logger; used for logger.info/debug/warn.	logger.info(agentId, message, timer.elapsed(‘main’))
timer (object)	Simple timer utility used for startup metrics.	timer.start(‘main’), timer.elapsed(‘main’)
loadDirective(path, opts)	Loads a directive file from CWD optionally scoped to provided tools.	Called as loadDirective(process.cwd(), { tools: toolNames, agentId })
setupValidationHandler	Local handler that wires QA validation event listeners.	Imported from ./handlers/validation.js — registers the QA behavior.

Abilities (registered in agent.json)

Ability name	Purpose / exposed actions	Notes / usage pattern
secret-ability	Provide vault/secret access (credential retrieval).	loadVaultCredentials() integrates with configured vaults.
ability-file-local	Zero-latency file operations in-process.	baseAgent.client.loadNative(‘ability-file-local’) => nativeFileLocal for direct file ops.
ability-file-remote	Remote file operations via broker.	Use through KadiClient ability call when not loaded natively.
ability-eval	Evaluate code or tasks in-process (zero-latency) when loaded natively.	baseAgent.client.loadNative(‘ability-eval’) => nativeEval.
ability-vision	Vision / image processing abilities exposed via platform.	Remote ability; used for visual QA flows if needed.
ability-log	Centralized logging ability to forward logs to other systems.	Use when archiving QA results or sending structured logs.

If your local runtime can load abilities natively (see baseAgent.client.loadNative calls below), prefer native usage for performance.

Configuration

Config is read from config.toml via readConfig(). Key fields used by the agent:

• agent.ID (string) — agent identifier (example: “agent-qa”)
• agent.VERSION (string) — agent version used in startup logs
• agent.ROLE (string) — role string (example: “qa”)
• logging.LEVEL (string) — default log level (“debug”/“info”)
• broker.local.URL (string) — optional local broker URL; if present, agent will prefer local broker
• broker.local.NETWORKS (array) — networks to subscribe on local broker
• broker.remote.URL (string) — remote broker URL (example: wss://broker.dadavidtseng.com/kadi)
• broker.remote.NETWORKS (array) — networks for remote broker (example: [“qa”,“quest”,“git”,“eval”,“global”])
• provider.PRIMARY / provider.FALLBACK (string) — names of configured providers (e.g., “model-manager”, “anthropic”)
• provider.model-manager.MODEL (string) — model name used for model-manager (example: “gpt-5-mini”)
• provider.anthropic.MODEL (string) — model name used for anthropic (example: “claude-haiku-4-5-20251001”)
• memory.DATA_PATH (string) — local path for MemoryService (default in config: ”./data/memory”)
• secrets.VAULTS (array) — vaults configured for loadVaultCredentials (example: [“anthropic”,“model-manager”,“arcadedb”])
• secrets.KEYS (array) — keys to request from vaults (example keys listed below)
• arcadedb.HOST, arcadedb.PORT, arcadedb.USERNAME, arcadedb.DATABASE — arcade DB logging configuration

Environment variables and overrides used by the agent:

• KADI_BROKER_URL_LOCAL — override broker.local.URL at runtime
• KADI_BROKER_URL_REMOTE — override broker.remote.URL at runtime
• MEMORY_DATA_PATH — override memory.DATA_PATH
• ANTHROPIC_API_KEY — override or provide Anthropics API key
• MODEL_MANAGER_BASE_URL — override model-manager base URL
• MODEL_MANAGER_API_KEY — override model-manager API key

Secrets required (declared in agent.json.deploy and referenced by loadVaultCredentials):

• ANTHROPIC_API_KEY
• MODEL_MANAGER_API_KEY
• MODEL_MANAGER_BASE_URL
• ARCADE_USERNAME
• ARCADE_PASSWORD

Note: loadVaultCredentials() reads vaults configured in config.toml and returns values; process.env values take precedence if present.

Code Examples

Below are representative snippets copied from src/index.ts demonstrating key startup and ability-loading patterns.

Startup, configuration reading, and BaseAgent construction:

const cfg = readConfig();

// Agent identity + logging
const agentId = cfg.string('agent.ID');
const agentVersion = cfg.string('agent.VERSION');
const logLevel = cfg.has('logging.LEVEL') ? cfg.string('logging.LEVEL') : 'info';
setLogLevel(logLevel);
setAgentTag(agentId);

// Broker resolution
const hasLocal = cfg.has('broker.local.URL');
const hasRemote = cfg.has('broker.remote.URL');
if (!hasLocal && !hasRemote) {
  throw new Error('At least one broker required: set [broker.local] or [broker.remote] in config.toml');
}

const brokerUrl = hasLocal
  ? (process.env.KADI_BROKER_URL_LOCAL ?? cfg.string('broker.local.URL'))
  : (process.env.KADI_BROKER_URL_REMOTE ?? cfg.string('broker.remote.URL'));
const networks = hasLocal
  ? cfg.strings('broker.local.NETWORKS')
  : cfg.strings('broker.remote.NETWORKS');

const primaryProvider = cfg.has('provider.PRIMARY') ? cfg.string('provider.PRIMARY') : undefined;
const fallbackProvider = cfg.has('provider.FALLBACK') ? cfg.string('provider.FALLBACK') : undefined;

Main flow: loading credentials, building BaseAgentConfig, connecting, and loading directive:

// Load credentials: env vars take priority over vault
const vault = await loadVaultCredentials();
const anthropicApiKey = process.env.ANTHROPIC_API_KEY || vault.ANTHROPIC_API_KEY;
const modelManagerBaseUrl = process.env.MODEL_MANAGER_BASE_URL || vault.MODEL_MANAGER_BASE_URL;
const modelManagerApiKey = process.env.MODEL_MANAGER_API_KEY || vault.MODEL_MANAGER_API_KEY;

// Build BaseAgent config
const baseAgentConfig: BaseAgentConfig = {
  agentId,
  agentRole: 'programmer',
  version: agentVersion,
  brokerUrl,
  networks,
  ...(additionalBrokerUrl ? {
    additionalBrokers: {
      remote: { url: additionalBrokerUrl, networks: additionalBrokerNetworks! },
    },
  } : {}),
  ...((anthropicApiKey || (modelManagerBaseUrl && modelManagerApiKey)) ? {
    provider: {
      ...(anthropicApiKey ? { anthropicApiKey } : {}),
      ...(modelManagerBaseUrl && modelManagerApiKey ? {
        modelManagerBaseUrl,
        modelManagerApiKey,
      } : {}),
      ...(primaryProvider ? { primaryProvider } : {}),
      ...(fallbackProvider ? { fallbackProvider } : {}),
    },
  } : {}),
  memory: {
    dataPath: process.env.MEMORY_DATA_PATH ?? cfg.string('memory.DATA_PATH'),
  },
};

const baseAgent = new BaseAgent(baseAgentConfig);
await baseAgent.connect(vault);

// Load directive
const toolNames = baseAgent.client.readAgentJson().tools.map((t: any) => t.name);
const directive = await loadDirective(process.cwd(), { tools: toolNames, agentId });

Loading native abilities (preferred for low-latency ops):

// Load ability-file-local natively (zero-latency file ops, in-process)
let nativeFileLocal: any = null;
try {
  nativeFileLocal = await baseAgent.client.loadNative('ability-file-local');
  logger.info(agentId, 'Loaded ability-file-local natively', timer.elapsed('main'));
} catch (err: any) {
  logger.warn(agentId, `Could not load ability-file-local natively: ${err.message}`, timer.elapsed('main'));
}

// Load ability-eval natively (zero-latency code/task evaluation, in-process)
let nativeEval: any = null;

Hooking up QA handlers:

import { setupValidationHandler } from './handlers/validation.js';

// ... after baseAgent connected and abilities loaded
setupValidationHandler(baseAgent);

Dependencies

Direct runtime dependencies (from package.json):

• @anthropic-ai/sdk ^0.32.1 — Anthropics provider SDK (optional provider)
• @kadi.build/core latest — KĀDI platform core integrations
• agents-library ^0.1.14 — BaseAgent, readConfig, loadVaultCredentials, logger, timer, etc.
• zod ^4.1.5 — runtime validation utilities

Dev dependencies:

• typescript, tsx, vitest, eslint, @typescript-eslint, etc.

What depends on agent-qa

• Consumers: any other agents or orchestrators that emit review events (task.review_requested) or submit tasks that require QA/validation.
• agent-qa expects to be reachable via the configured broker networks so task producers (pipeline agents, programmer agents) can request review.

What agent-qa depends on at runtime

• Broker availability (local or remote configured in config.toml)
• Vaults / secrets configured via config.toml and the deployed secret delivery mechanism (see agent.json.deploy.secrets)
• LLM provider credentials (Anthropic or Model Manager) for semantic review when used
• arcadedb credentials/host if persistence/logging to ArcadeDB is used

If you need to change QA behavior, modify ./handlers/validation.js and adjust provider or ability usage in src/index.ts — the startup code shows how providers and abilities are discovered and made available to handlers.