Coding PROMPT
Streaming UX Latency Plan
July 26, 2026Optimized for: anyAI product responsiveness
Design the perceived-latency strategy for this AI feature. Users judge responsiveness by time to first token far more than by total generation time. Produce: 1. LATENCY BUDGET: target time to first token, target total, and where the current implementation spends its time. 2. TTFT REDUCERS ranked by impact: shorter input context, prompt caching, a smaller routing model for the opening, speculative prefetch, and moving retrieval off the critical path. 3. WHAT TO SHOW DURING THE WAIT, per phase. Skeletons, streaming partials, tool-call status, and what NOT to show because it makes the wait feel longer. 4. STREAMING FAILURE HANDLING: what the user sees when a stream dies at 80 percent, and how to make retry non-destructive. 5. THE ONE CHANGE with the best perceived-latency return. Feature: [DESCRIBE] Current measured latency: [TTFT AND TOTAL]
Optimises perceived responsiveness rather than raw throughput, including what to render during the wait and how to survive a dead stream.
Submit your own AI prompts to the community. The best ones get featured on TokenCalculator - and credited to you.