VV-024mediumJailbreak2024-04-02
Many-shot jailbreaking: long context windows weaken refusals
Anthropic published research showing that filling a long prompt with many fake dialogue examples can push models into answering requests they would normally refuse.
What it is
Published 2 April 2024. The technique exploits large context windows and affected models from several vendors.
How to check
- If users can paste long text into your LLM feature, test with long adversarial inputs.
How to fix
- Cap input length where you can, and classify inputs before they reach the model.
Sources
More advisories for this tool