Trace grading

실행 추적 채점

Score a trace of model calls, tools, and handoffs against explicit criteria.

···
html
<div class="stage"><div class="title">TRACE GRADING</div><div class="canvas"><div class="trace"><div id="t0"><span>MODEL</span><b>intent</b></div><div id="t1"><span>TOOL</span><b>wrong source</b></div><div id="t2"><span>HANDOFF</span><b>review</b></div></div><div class="grade" id="grade">grading model call</div></div></div>
css
.stage{width:min(94vw,820px);height:min(88vh,330px);box-sizing:border-box;padding:clamp(9px,2.4vmin,18px);border:1px solid var(--line);border-radius:13px;background:var(--surface);display:flex;flex-direction:column;gap:clamp(7px,2vmin,14px);font:600 clamp(12px,2.5vmin,16px)/1.28 "Pretendard Variable",Pretendard,-apple-system,BlinkMacSystemFont,"Apple SD Gothic Neo",sans-serif;color:var(--fg);overflow:hidden}.title{color:var(--accent);letter-spacing:.035em;flex:none}.canvas{position:relative;flex:1;min-height:0;display:flex;align-items:center;justify-content:center}.muted{color:var(--muted)}.mono{font-family:ui-monospace,SFMono-Regular,monospace}.box{border:1px solid var(--line);border-radius:8px;background:var(--bg);padding:clamp(5px,1.4vmin,12px);text-align:center}.active{border-color:var(--accent)!important;color:var(--accent)!important;background:color-mix(in srgb,var(--accent) 12%,var(--surface))!important}.canvas{flex-direction:column;gap:clamp(8px,3vmin,20px)}.trace{display:flex;width:100%;max-width:650px;gap:clamp(3px,1vmin,9px)}.trace div{flex:1;min-width:0;border:1px solid var(--line);border-radius:6px;padding:clamp(5px,1.7vmin,12px);display:flex;flex-direction:column;gap:6px;text-align:center}.trace span{color:var(--muted)}.trace b{overflow-wrap:anywhere}.trace .bad{border-color:var(--accent-3);color:var(--accent-3)}.grade{color:var(--accent)}
js
let n=0;function grade(){const s=n++%3;for(let i=0;i<3;i++){const el=document.getElementById('t'+i);el.classList.toggle('active',i===s&&i!==1);el.classList.toggle('bad',i===s&&i===1)}document.getElementById('grade').textContent=['model: pass','tool choice: review','handoff: pass'][s]}grade();setInterval(grade,950)

Trace grading records an agent run’s calls and handoffs, then scores them against a rubric. It helps locate a poor decision and improve prompts or tool design.

The demo walks through three trace steps and flags a tool-choice error. Traces may contain sensitive content, and graders can misjudge, so control retention and review samples.

When to use

Use it to locate workflow failures and compare agent changes step by step.

Open as page ↗