Agent goal hijack

에이전트 목표 탈취

Keep instructions in untrusted material from overriding the user’s task.

···
html
<div class="sd"><div class="sd-head"><strong>GOAL HIJACK</strong><span id="state">live check</span></div><div class="sd-stage"><div class="goal tile good">user goal<br>summarize</div><div class="boundary" id="boundary">TRUST<br>BOUNDARY</div><div class="source tile" id="source">document<br>change goal</div></div><div class="sd-foot"><span id="caption">checking boundary</span><span>tap to step</span></div></div>
css
.sd{box-sizing:border-box;width:min(94vw,700px);height:min(86vh,318px);padding:clamp(9px,2.3vmin,18px);border:1px solid var(--line);border-radius:14px;background:var(--surface);color:var(--fg);display:flex;flex-direction:column;gap:clamp(6px,1.8vmin,12px);font:600 clamp(12px,2.6vmin,16px)/1.25 var(--font-sans, sans-serif)}.sd *{box-sizing:border-box}.sd-head,.sd-foot{display:flex;align-items:center;justify-content:space-between;gap:8px;min-height:1.25em}.sd-head strong{color:var(--accent);letter-spacing:.06em}.sd-head span,.sd-foot{color:var(--muted)}.sd-foot{font-size:clamp(12px,2.4vmin,14px)}.sd-stage{position:relative;flex:1;min-height:0;display:flex;align-items:center;justify-content:center;gap:clamp(6px,2vmin,18px);overflow:hidden}.sd .mono{font-family:ui-monospace,monospace}.sd .tile{border:1px solid var(--line);border-radius:8px;background:var(--bg);padding:clamp(5px,1.5vmin,10px);text-align:center}.sd .good{color:var(--accent);border-color:var(--accent)}.sd .bad{color:var(--accent-3);border-color:var(--accent-3)}.sd .arrow{color:var(--muted);font-size:clamp(15px,4vmin,28px)}.sd .active{background:color-mix(in srgb,var(--accent) 16%,var(--surface));border-color:var(--accent)}.sd .blocked{background:color-mix(in srgb,var(--accent-3) 13%,var(--surface));border-color:var(--accent-3)}.goal,.source{width:31%}.boundary{border:2px dashed var(--accent-3);border-radius:6px;padding:clamp(4px,1.2vmin,10px);font:700 clamp(12px,2.4vmin,13px)/1.2 ui-monospace,monospace;color:var(--accent-3);text-align:center}.source{transition:transform .3s}.source.push{transform:translateX(-8px);border-color:var(--accent-3)}.boundary.alert{background:color-mix(in srgb,var(--accent-3) 15%,var(--surface))}
js
let n=0;function advance(){n=1-n;document.getElementById('source').classList.toggle('push',!!n);document.getElementById('boundary').classList.toggle('alert',!!n);document.getElementById('state').textContent=n?'injection blocked':'reading source';document.getElementById('caption').textContent=n?'external instruction stays data':'agent follows user goal'}advance();setInterval(advance,900);
document.querySelector('.sd').addEventListener('pointerdown',advance);

Goal hijacking places instructions in untrusted material such as search results or documents to redirect an agent from its assigned task. External content is task data, not a higher-priority command.

The demo blocks a hostile sentence at the boundary before it can replace the task goal. Track content provenance and recheck the goal and permissions before tool actions.

When to use

Consider it when an agent reads web pages, documents, or email before using tools.

Open as page ↗