Training Trains logo

Inference Engineering Internship Project Ideas: Beginner to Advanced

By K. Boopathikumar, CEO & Founder · 16 November 2026 · 6 min read

Inference projects are measurement projects. Pick a model, measure it, optimise one thing at a time and record the results in a table.

Beginner

  • CPU vs GPU benchmark: Measure an image classifier's latency on CPU and a free GPU. You learn: benchmarking basics.
  • ONNX export: Export a PyTorch model to ONNX and compare speed. You learn: model formats.
  • Latency percentiles: Record p50, p95 and p99 for 1,000 requests. You learn: performance metrics.

Intermediate

  • Quantisation comparison: Compare accuracy and speed at fp32, fp16 and int8. You learn: precision trade-offs.
  • Batching server: Add dynamic batching to a model API and measure throughput. You learn: batching.
  • Load test: Use k6 or Locust to find the server's limit. You learn: capacity testing.

Advanced

  • LLM serving benchmark: Compare two open-source LLM servers on tokens per second. You learn: LLM inference.
  • Autoscaling demo: Scale model replicas on queue length in Kubernetes. You learn: production scaling.
  • Cost per request dashboard: Track cost per 1,000 requests across configurations. You learn: cost engineering.

How to make any of these projects stand out

  • Report hardware details with every number.
  • Check output quality after each optimisation.
  • Share scripts so results can be reproduced.
  • Present results in one comparison table.

Next steps

Looking for an internship near Erode?

10 days to 6 months, online or at our Erode and Perundurai branches, with a verifiable certificate.

See internships →

More guides