Respect Effective Context

The advertised window is a ceiling, not a promise. Retrieval and reasoning quality degrade well before it, and worst in the middle.

Apply:
1. Plan for roughly a third of the advertised window as reliably usable, and validate that assumption for your specific model and corpus.
2. Put the most important instructions at both the start and the end. The middle is where information goes to be ignored.
3. If you can find the relevant span and pass only that, do. Retrieval beats stuffing on both quality and cost for diffuse corpora.
4. Test retrieval quality at your actual context length, not with a synthetic needle-in-haystack. Real relevance is diffuse; needles are not.
5. Measure the cost of the window you are filling. Filling a 2M window is several dollars per request before a single output token.