Flaky test

불안정 테스트

A test that changes result across runs without a relevant code change.

···
html
<div class="stage"><div class="title">FLAKY TEST</div><div class="canvas"><div class="runs" id="runs"></div><div class="race"><div class="ready" id="ready"></div><div class="assert" id="assert"></div></div><div class="reason" id="reason">assertion ran before ready</div></div></div>
css
.stage{width:min(94vw,820px);height:min(88vh,330px);box-sizing:border-box;padding:clamp(9px,2.4vmin,18px);border:1px solid var(--line);border-radius:13px;background:var(--surface);display:flex;flex-direction:column;gap:clamp(7px,2vmin,14px);font:600 clamp(12px,2.5vmin,16px)/1.28 "Pretendard Variable",Pretendard,-apple-system,BlinkMacSystemFont,"Apple SD Gothic Neo",sans-serif;color:var(--fg);overflow:hidden}.title{color:var(--accent);letter-spacing:.035em;flex:none}.canvas{position:relative;flex:1;min-height:0;display:flex;align-items:center;justify-content:center}.muted{color:var(--muted)}.mono{font-family:ui-monospace,SFMono-Regular,monospace}.box{border:1px solid var(--line);border-radius:8px;background:var(--bg);padding:clamp(5px,1.4vmin,12px);text-align:center}.active{border-color:var(--accent)!important;color:var(--accent)!important;background:color-mix(in srgb,var(--accent) 12%,var(--surface))!important}.canvas{flex-direction:column;gap:clamp(10px,3vmin,22px)}.runs{display:flex;gap:5px;justify-content:center}.runs span{width:clamp(20px,5vmin,32px);height:clamp(20px,5vmin,32px);display:grid;place-items:center;border-radius:4px;border:1px solid var(--line)}.runs .pass{color:var(--accent);border-color:var(--accent)}.runs .fail{color:var(--accent-3);border-color:var(--accent-3)}.race{position:relative;width:min(90%,460px);height:30px;border-top:2px solid var(--line)}.ready,.assert{position:absolute;top:-8px;width:15px;height:15px;border-radius:50%;background:var(--accent);transition:left .5s}.ready{left:67%}.assert{left:32%;background:var(--accent-3)}.reason{color:var(--muted);text-align:center}
js
let n=0;const marks=[];function run(){const pass=n++%3===2;marks.push(pass?'PASS':'FAIL');if(marks.length>7)marks.shift();document.getElementById('runs').innerHTML=marks.map(x=>'<span class='+ (x==='PASS'?'pass':'fail') +'>'+ (x==='PASS'?'✓':'×') +'</span>').join('');document.getElementById('assert').style.left=pass?'82%':'32%';document.getElementById('reason').textContent=pass?'assertion waited for ready':'assertion ran before ready'}run();setInterval(run,1000)

Flaky tests come from races, fixed sleeps, shared data, or changing external systems. Rerunning red tests until they turn green can hide real regressions.

The demo shows the same assertion failing before readiness and passing afterward. Wait for an observable condition and remove the cause instead of adding arbitrary delays.

When to use

Use it when repeat runs disagree and you need to diagnose and stabilize the test.

Open as page ↗