Two evaluation suites run on the same realistic asset bank — 9 wardrobes, 10 style profiles, 230 scenarios. One asks "does it behave like a stylist?"; the other, adversarially, asks "where does it break, and when it breaks, does it fail safely, honestly and explainably?" Every finding is severity-tagged, and a release gate blocks on any safety failure.
Does it behave like a stylist? 10 behavioural families + a cross-cutting safety sweep, all 11/11 pass.
The adversarial inverse — where does it break, and does it fail safely? 8 attack families behind a release gate (P0 = 0).
Every finding is typed and ranked. The gate passes iff P0 = 0; lower tiers are reported as prioritised, isolated debt.
| Tier | Meaning | Examples | Status now |
|---|---|---|---|
| P0 | Safety — blocks release | unsafe-but-confident, blacklist violation, empty result, hard-fail with a positive label | 0 |
| P1 | Trust debt | silent constraint drop, explanation contradiction, avoidable hard-fail | 0 |
| P2 | Quality | candidate-coverage gap, top-K collapse, over-penalisation | 1 (debatable) |
| P3 | Polish | minor ranking preferences | 0 |
| Property | What it proves | Result |
|---|---|---|
| Safety under attack | never an unsafe-but-confident outfit, across 8 adversarial families | 0 P0 |
| Confidence calibration | the shown confidence tracks independently-measured quality | Pearson 0.83 |
| Constraint transparency | a dropped keep-item is reported with a reason, never silently ignored | 0 silent |
| Explanation grounding | the text never names a missing garment, wrong colour, or oversells a hard-fail | 10 / 10 |
| Context sensitivity | output shifts the right way when one signal (temp / weather / occasion) changes | consistency 1.0 |
| Item-swap penalty | a single bad swap is penalised, never confidently | 0.92 correct |
| Candidate coverage | appropriate, in-season items can actually enter a candidate outfit | 0.97 coverage |
| Robustness | corrupted / contradictory metadata never forces an unsafe pick; never empty / crash | graceful |
Quality is scored by an independent 0–100 yardstick computed from item attributes + scenario — deliberately not the engine's own score — so "quality" is an external check, not a tautology. Embeddings in the harness are attribute-derived (deterministic), which under-represents personalisation versus live FashionCLIP anchors.
When a requested keep_item can't be used safely (sandals in snow, a puffer at 35 °C), the engine doesn't quietly drop it — it returns a constraint_resolution block explaining what it kept, what it dropped, and why.
{
"constraint_resolution": {
"honored_keep_items": ["blazer-01"],
"dropped_keep_items": [
{ "item_id": "sandal-02", "reason": "season_mismatch",
"message": "Bu parçayı bu mevsim için uygun olmadığından öneriye eklemedim." }
]
}
}
Stylist-grade outfit intelligence — safety-checked, calibrated, explainable — through one API.