Evaluation dataset

평가 데이터셋

Collect real failures and expected outcomes for repeatable evaluation of model changes.

···
html
<div class="stage"><div class="title">EVAL DATASET</div><div class="canvas"><div class="cases"><div id="case0">case A</div><div id="case1">case B</div><div id="case2">new failure</div></div><div class="scores"><div><span>before</span><i class="before"></i></div><div><span>after</span><i class="after" id="after"></i></div></div><div class="note" id="note">add failure to dataset</div></div></div>
css
.stage{width:min(94vw,820px);height:min(88vh,330px);box-sizing:border-box;padding:clamp(9px,2.4vmin,18px);border:1px solid var(--line);border-radius:13px;background:var(--surface);display:flex;flex-direction:column;gap:clamp(7px,2vmin,14px);font:600 clamp(12px,2.5vmin,16px)/1.28 "Pretendard Variable",Pretendard,-apple-system,BlinkMacSystemFont,"Apple SD Gothic Neo",sans-serif;color:var(--fg);overflow:hidden}.title{color:var(--accent);letter-spacing:.035em;flex:none}.canvas{position:relative;flex:1;min-height:0;display:flex;align-items:center;justify-content:center}.muted{color:var(--muted)}.mono{font-family:ui-monospace,SFMono-Regular,monospace}.box{border:1px solid var(--line);border-radius:8px;background:var(--bg);padding:clamp(5px,1.4vmin,12px);text-align:center}.active{border-color:var(--accent)!important;color:var(--accent)!important;background:color-mix(in srgb,var(--accent) 12%,var(--surface))!important}.canvas{flex-direction:column;gap:clamp(8px,3vmin,16px)}.cases{display:flex;gap:4px;width:100%;max-width:600px}.cases div{flex:1;border:1px solid var(--line);border-radius:5px;padding:clamp(5px,2vmin,11px) 1px;text-align:center}.cases #case2{border-color:var(--accent-3)}.cases #case2.in{border-color:var(--accent);color:var(--accent)}.scores{width:100%;max-width:480px;display:flex;flex-direction:column;gap:6px}.scores div{display:flex;align-items:center;gap:6px}.scores span{width:52px}.scores i{display:block;height:clamp(10px,2.6vmin,17px);border-radius:3px;background:var(--accent);transition:width .4s}.scores .before{width:45%;opacity:.45}.scores .after{width:45%}.note{color:var(--accent)}
js
let added=false;function update(){added=!added;document.getElementById('case2').classList.toggle('in',added);document.getElementById('after').style.width=added?'82%':'45%';document.getElementById('note').textContent=added?'rerun: new case passes':'add failure to dataset'}update();setInterval(update,1450)

An evaluation dataset stores representative inputs, expected outcomes, and scoring criteria. Adding real failures lets future changes check for recurrence.

The demo moves a failure into the dataset and compares results before and after a change. A narrow or stale dataset can make a higher score misleading.

When to use

Use it to compare model, prompt, or tool changes against the same cases.

Open as page ↗