Inference Engineering Terms Every Intern Should Know
By K. Boopathikumar, CEO & Founder · 16 November 2026 · 4 min read
In your first Inference Engineering internship meetings you will hear words that everyone else seems to know. Here they are in plain English. If you hear a term that is not on this list, write it down and ask; nobody minds an intern asking what a word means.
The terms
- Latency: time for one request.
- Throughput: requests handled per second.
- p99: the latency 99% of requests are faster than.
- Batching: processing several requests together.
- Quantisation: lower-precision weights for speed.
- ONNX: a portable model format.
- Cold start: delay when a new server or model loads.
- Cost per request: hardware cost divided by requests served.
Next steps
- Study the free Inference Engineering learning path and take its module quizzes for a free certificate.
- Follow the Inference Engineering skill track from basic to advanced.
- Do real, reviewed work in a Inference Engineering internship and keep every project in your own GitHub.