Inference Engineering Internship Project Ideas: Beginner to Advanced
By K. Boopathikumar, CEO & Founder · 16 November 2026 · 6 min read
Inference projects are measurement projects. Pick a model, measure it, optimise one thing at a time and record the results in a table.
Beginner
- CPU vs GPU benchmark: Measure an image classifier's latency on CPU and a free GPU. You learn: benchmarking basics.
- ONNX export: Export a PyTorch model to ONNX and compare speed. You learn: model formats.
- Latency percentiles: Record p50, p95 and p99 for 1,000 requests. You learn: performance metrics.
Intermediate
- Quantisation comparison: Compare accuracy and speed at fp32, fp16 and int8. You learn: precision trade-offs.
- Batching server: Add dynamic batching to a model API and measure throughput. You learn: batching.
- Load test: Use k6 or Locust to find the server's limit. You learn: capacity testing.
Advanced
- LLM serving benchmark: Compare two open-source LLM servers on tokens per second. You learn: LLM inference.
- Autoscaling demo: Scale model replicas on queue length in Kubernetes. You learn: production scaling.
- Cost per request dashboard: Track cost per 1,000 requests across configurations. You learn: cost engineering.
How to make any of these projects stand out
- Report hardware details with every number.
- Check output quality after each optimisation.
- Share scripts so results can be reproduced.
- Present results in one comparison table.
Next steps
- Study the free Inference Engineering learning path and take its module quizzes for a free certificate.
- Follow the Inference Engineering skill track from basic to advanced.
- Do real, reviewed work in a Inference Engineering internship and keep every project in your own GitHub.