All 30 tasks, level by level

The real graded LLM Engineering & Fine-Tuning tasks the portal assigns, grouped into three levels.

The portal assigns these LLM Engineering & Fine-Tuning tasks. Finish Level 1 before Level 2 and Level 2 before Level 3. Your mentor may mark some tasks as required for the certificate.

Level 1: Foundations (12 tasks)

Task	What to do
How LLMs work	In a README in a repo named llm-internship, explain tokens, the context window, next-token prediction, temperature and hallucination with your own small examples.
Count tokens	Use a tokenizer library (tiktoken or a Hugging Face tokenizer) to count tokens for 5 sentences in English and 2 in Tamil or Hindi. Note why the counts differ.
First API call from Python	Call any LLM API (Gemini free tier, Groq or a local Ollama model) from a Python script. Keep the API key in a .env file that is listed in .gitignore.
Run a model on your laptop	Install Ollama, run a small open model (such as Llama 3.2 3B or Qwen 2.5 3B) and record the answer speed in tokens per second.
System prompts	Write 3 system prompts (strict tutor, friendly support agent, JSON-only extractor) and show how the same question gets different answers.
Temperature experiment	Ask the same question 5 times at temperature 0, 0.7 and 1.2. Tabulate how much the answers change and when each setting is useful.
Few-shot prompting	Build a few-shot prompt that classifies customer messages as complaint, question or praise. Measure accuracy on 20 messages you write.
Structured JSON output	Extract name, college, phone and skills from 10 sample resumes as JSON. Validate every result with Pydantic or JSON Schema and count the failures.
Streaming chat in the terminal	Make a terminal chat that streams tokens as they arrive and keeps the last 10 messages as conversation history.
Spot hallucinations	Ask a model 10 factual questions about your own college or town. Mark each answer correct, wrong or invented and explain why models hallucinate.
Embeddings and similarity	Create embeddings for 20 sentences with sentence-transformers and print the 3 most similar sentences for any query using cosine similarity.
Cost estimate	Pick one paid LLM API, look up its input and output token prices, and estimate the monthly cost of a support bot answering 500 questions a day.

Level 2: Building real features (10 tasks)

Task	What to do
Document chatbot (RAG)	Build a RAG app over 5 PDFs: chunk, embed, store in Chroma or pgvector, retrieve the top chunks and answer with citations to the page.
Chunking comparison	Try 3 chunk sizes and 2 overlaps on the same documents. Write 15 test questions and report which setting gives the most correct answers.
Hybrid search and re-ranking	Add BM25 keyword search next to vector search, merge the results and re-rank them with a cross-encoder. Show the before and after on 10 questions.
Tool calling	Give a model two tools (a weather or currency API and a calculator). Validate the arguments in code before running each tool and log every call.
Prompt injection test	Hide an instruction like 'ignore previous rules' inside one retrieved document. Show the attack working, then add defences and show it failing.
Evaluation set	Write 30 test questions with expected answers for your RAG app and a script that scores every run. Save the scores in a CSV.
LLM as a judge	Use a model with a clear rubric to grade your app's answers for correctness and faithfulness. Compare its grades with your own on 15 answers.
Build a fine-tuning dataset	Create 200 or more instruction/response pairs in JSONL for one narrow task (for example, polite replies to college admission questions). Remove any personal data.
Fine-tune with LoRA	Fine-tune a small open model (0.5B to 3B) with LoRA or QLoRA on free Google Colab or Kaggle GPUs using Hugging Face PEFT or Unsloth. Save the adapter.
Before and after fine-tuning	Run your evaluation set on the base model and the fine-tuned model. Report the scores and show 5 example answers side by side.

Level 3: Professional work and capstone (8 tasks)

Task	What to do
Serve your model	Serve your fine-tuned or a quantized open model behind an HTTP API (Ollama, llama.cpp server or vLLM) with a /health endpoint and a request timeout.
Quantization trade-off	Run the same model in 16-bit, 8-bit and 4-bit (for example with GGUF). Measure memory, speed and quality on your eval set and recommend one.
Guardrails	Add input and output checks to your app: block off-topic questions, mask phone numbers and emails, and refuse unsafe requests. Show tests for each.
Caching and cost control	Add response caching for repeated questions and a token budget per user. Show cost and latency before and after on 100 requests.
Observability	Log every request with prompt version, model, tokens, latency and score (Langfuse or your own table). Build a small dashboard of the last day.
Preference tuning	Make 100 pairs of preferred and rejected answers and run a short DPO training with TRL. Explain what changed in the model's answers.
Model routing	Send easy questions to a small model and hard ones to a larger model using a simple classifier. Report the cost saved and any quality drop.
Capstone: production LLM app	Deploy a complete assistant for a real use (college help desk, shop FAQ or code helper) with RAG, evals, guardrails and logs. Submit the live link, repo and a 2-minute demo video.

← Tools and first-week setup Submitting tasks and earning your certificate →