Eval And Repair Contract

Eval results are deterministic feedback for agents. They should be path-rich and directly repairable.

Finding Shape

interface EvalFinding {
  rule: string;
  targetId?: string;
  targetPath: string;
  groupPath?: string;
  severity: "hard" | "soft" | "info";
  message?: string;
  actual?: unknown;
  required?: unknown;
  requiredMin?: unknown;
  requiredMax?: unknown;
  unit?: string;
  repairHints?: RepairHint[];
}

Severity

Severity	Meaning
hard	Candidate should not be committed as valid
soft	Candidate is usable but lower scoring or needs review
info	Informational metric, trace, or warning

Paths

Field	Purpose
targetPath	Exact entity, property, metric, or generated object that failed
groupPath	Semantic group that owns the rule or constraint
targetId	Stable ID if available

Example:

{
  "rule": "planning.setback_distance",
  "targetPath": "site.main.envelopes.primary.faces.front",
  "groupPath": "site.main.parcels.pcl_001.constraints.setbacks.front",
  "severity": "hard",
  "actual": "4200mm",
  "requiredMin": "5000mm"
}

Repair Hint Shape

interface RepairHint {
  kind: "operationSuggestion" | "paramAdjustment" | "deleteOrReplace" | "manualReview";
  op?: string;
  targetPath?: string;
  params?: Record<string, unknown>;
  rationale?: string;
}

Repair Hint Example

{
  "kind": "operationSuggestion",
  "op": "fitMassingToEnvelope",
  "targetPath": "buildings.b_001",
  "params": {
    "envelope": "site.main.envelopes.primary",
    "allowedMoves": ["shrink_footprint"]
  },
  "rationale": "The building footprint exceeds the front setback-derived envelope."
}

Agent Loop

const result = await branch.evaluate(patch, {
  scope: "affected",
  tiers: [0, 1, 2, 3],
  returnRepairHints: true
});

if (!result.valid) {
  const repairPatch = result.findings.flatMap((finding) => finding.repairHints ?? []);
  await branch.applyPatch(repairPatch);
}