Tip
Advanced
Batch Small Requests
July 29, 2024
If you have many small, independent tasks for an LLM, batch them into a single API call if the model and API support it, rather than many separate calls. This can reduce overhead.
Category: Token Optimization
Difficulty: Advanced