Internship← AI Product Building learning pathAI Product Building · Module 9 of 10
Cost, speed and scaling
Keep AI features fast and affordable as usage grows.
📖 Key ideas
- Estimate cost: requests per day x tokens per request x price per token.
- Cache repeated answers and reuse prompt prefixes (prompt caching).
- Use batch processing for non-urgent bulk jobs at lower cost.
- Set rate limits and per-user quotas to prevent abuse.
- Monitor latency, errors and spend on a dashboard.
💡 Real-life example: Like running a canteen: know cost per plate, cook popular dishes in bulk, and limit free refills.
Prompt cachingReusing a repeated prompt prefix at lower cost
Batch APICheaper processing of many non-urgent requests
QuotaLimit on usage per user
Spend dashboardView of AI costs over time