All 60 tasks, level by level

The real graded AI Agents & RAG tasks the portal assigns, grouped into three levels.

The portal assigns these AI Agents & RAG tasks. Finish Level 1 before Level 2 and Level 2 before Level 3. Your mentor may mark some tasks as required for the certificate.

Level 1: Foundations (22 tasks)

Task	What to do
Explain agents vs chatbots	Write a one-page explainer with a diagram: what an AI agent is, the plan-act-observe loop, tools and memory, with two real product examples.
Embeddings hands-on	Create embeddings for 20 sentences and find the 3 most similar sentences for 5 queries using cosine similarity. Print the scores.
Chunking experiment	Split one long document with 3 chunk sizes (200, 500, 1,000 characters, with overlap). Compare which size retrieves the right passage for 10 questions.
Set up a vector store	Store 100 text chunks in a free vector database or local index (FAISS, Chroma or pgvector) and run similarity search from code.
Write tool descriptions	Define 4 tools (search, calculator, calendar, email draft) with JSON schemas and clear descriptions that an LLM can choose between.
Single tool-calling loop	Build a loop where the model may call a calculator tool until it has the final answer. Log each step.
RAG evaluation set	Write 25 questions with the expected answer and source passage for a document you choose. You will reuse it to score your RAG system.
Prompt for grounded answers	Write a system prompt that forces answers only from given context, with citations, and 'not found' otherwise. Test it on 10 questions.
`Read a framework's docs`	Pick one agent framework (LangChain, LlamaIndex, or plain SDK tool use). Summarise its core concepts and run its quick-start example.
PDF to clean text pipeline	Convert 5 PDFs to clean text, removing headers, footers and page numbers, ready for chunking. Show before/after samples.
Metadata filtering	Add metadata (source, date, section) to chunks and answer questions limited to one section or date range.
Agent safety rules	List 10 risks of agents that take actions (wrong emails, deleting data, cost loops) and a guardrail for each.
Cosine similarity by hand	Compute cosine similarity for 3 small vectors on paper and in code, and explain what the number means.
Tokenisation and chunk sizes	Count tokens for 5 documents and pick chunk sizes that fit your model's context window with room for the answer.
Compare embedding models	Embed the same 50 sentences with 2 embedding models and compare retrieval quality on 10 queries.
Agent loop on paper	Trace a plan-act-observe loop for booking a meeting, listing each tool call, result and decision.
Read a RAG paper or blog	Summarise one well-known RAG or agents article: problem, method, results and limits.
Vector index types	Explain flat, IVF and HNSW indexes and when each fits, with a small benchmark.
Document loaders	Load PDF, HTML, DOCX and CSV into clean text with metadata in one pipeline.
Prompt templates for RAG	Write and compare 3 RAG prompt templates on your evaluation set.
Agent framework comparison	Compare two agent frameworks on the same small task: code size, control and debugging.
Failure catalogue	List 15 ways RAG and agents fail (missed retrieval, loops, wrong tool) with a fix for each.

Level 2: Building real features (20 tasks)

Task	What to do
RAG chatbot over a website	Crawl 20 pages of a public site you are allowed to use, index them and answer questions with page links as citations.
Hybrid search	Combine keyword (BM25) and vector search, then rerank. Show that hybrid finds answers that vector-only misses on your evaluation set.
Research agent	An agent that searches the web (or a mock search tool), reads 3 sources and writes a cited summary. Cap it at 5 tool calls.
SQL agent	Let an LLM answer questions over a sample SQLite database by writing read-only SQL. Block any statement that is not SELECT.
Conversation memory	Add short-term (recent turns) and long-term (saved facts) memory to a chatbot. Show it remembering a fact across sessions.
Measure RAG quality	Score your RAG system on the evaluation set for retrieval hit rate and answer correctness. Improve one setting and report the new numbers.
Structured extraction agent	From 20 job emails, extract company, role, deadline and link into a spreadsheet using tool calls and validation.
Human approval step	Make an agent draft an email or calendar event but require the user to approve before the action runs.
Streaming agent UI	Build a web UI that shows the agent's steps (thinking, tool call, result) live while it works.
Multi-document comparison	Ask questions that need two documents (e.g. compare two policies). Retrieve from both and cite each source.
Calendar assistant agent	An agent that reads a mock calendar, finds free slots and drafts invites, with approval before creating events.
Query rewriting	Rewrite vague user questions before retrieval and show improved hit rate on your evaluation set.
Answer with tables	Retrieve from CSV or spreadsheet data and answer numeric questions correctly, citing the rows used.
Feedback loop	Add thumbs up/down to answers, store feedback and review the worst 10 answers to improve the system.
Parent-child chunking	Retrieve small chunks but send their larger parent section to the model; compare answer quality.
Incremental indexing	Update the index when documents change without re-embedding everything.
Multi-tenant RAG	Serve several organisations from one system with strict data separation and tests.
Graph-based retrieval	Extract entities and relations into a small knowledge graph and use it to answer multi-hop questions.
Agent memory summarisation	Summarise long conversations into compact memory and show the agent still recalls key facts.
Structured tool outputs	Make tools return typed JSON and handle tool errors gracefully in the agent loop.

Level 3: Professional work and capstone (18 tasks)

Task	What to do
Customer-support agent	An agent that answers from a FAQ knowledge base, creates a ticket through a tool when it cannot help, and escalates angry users. Deploy it.
Multi-agent workflow	Planner, researcher and writer agents that produce a report together. Log the messages between agents and the total cost.
Agent evaluation harness	Automated tests for your agent: 20 tasks with pass/fail checks, run in CI, with a results table in the README.
Observability and tracing	Trace every LLM and tool call with timing, tokens and errors, and show a trace view for one full run.
Cost and loop protection	Add max steps, max tokens, timeouts and retry limits. Show a test where a looping agent is stopped safely.
Knowledge base admin panel	Upload, re-index and delete documents from your RAG system through a small admin page, with the index updating automatically.
Capstone: production RAG app	Ship a RAG or agent app with sign-in, citations, feedback buttons and monitoring. Submit live URL, repo and a 3-minute demo.
Write the architecture doc	Document your capstone: architecture diagram, data flow, prompts, evaluation results, costs and known limits.
Agent with web browsing	Let an agent search and read pages (or a mock browser), cite sources and stop when confident or at a limit.
Role-based access in RAG	Make users see answers only from documents they are allowed to read, with tests for each role.
Cost-aware routing	Route easy questions to a small model and hard ones to a larger model; report cost and quality.
Offline evaluation report	Run a full evaluation (retrieval, faithfulness, answer correctness) and present results with charts.
Agent safety review	Red-team your own agent with 20 harmful or tricky requests, fix failures and document the results.
Ops agent for a team	An agent that reads tickets, suggests answers from docs and drafts replies for human approval.
Latency optimisation	Cut answer latency by 40% with caching, smaller models, parallel retrieval and streaming.
Evaluation dashboard	Track retrieval and answer quality over time with charts and alerts on regressions.
Agent for data analysis	Let an agent answer questions over a spreadsheet by writing and running safe pandas code in a sandbox.
Production readiness review	Review your agent for security, privacy, cost, monitoring and rollback, and fix gaps.

← Tools and first-week setup Submitting tasks and earning your certificate →