Learning path

AI engineer

Ship real AI products: agents, search over your documents, evaluation and small fast models.

41 lessons · 260 min left

Start path → What a large language model does

FoundationsLesson 2: What a large language model doesAn LLM is autocomplete that swallowed a library. Understanding that one sentence explains most of its strengths and every one of its weaknesses.Beginner4 min read🧪 Lab📝 TestUp next
FoundationsLesson 4: Tools: giving the model handsA tool is any button the model is allowed to press. Click each part of the diagram to see where trust is won and lost.Beginner6 min read🧪 Lab📝 Test
Building AI agentsLesson 196: The agent loop in codeAn agent is a loop: the model looks, decides, acts, and looks again. Write that loop yourself in about 30 lines of Python, with stop conditions and a step limit.Middle7 min read🧪 Lab📝 Test
Building AI agentsLesson 197: Designing good toolsThe model can only use your tools as well as you describe them. Learn to name, describe and shape tools so the model picks the right one and recovers from mistakes.Middle7 min read🧪 Lab📝 Test
Building AI agentsLesson 198: Structured outputs and function callingGet machine-readable JSON from the model instead of free text, check it in code against a schema, and retry cleanly when it is wrong.Middle6 min read🧪 Lab📝 Test
Building AI agentsLesson 199: Planning and task decompositionLong tasks go better when the agent breaks them into steps. Compare planning first with deciding step by step, and learn when to change the plan.Middle5 min read🧪 Lab📝 Test
Building AI agentsLesson 200: Agent memory in practiceContext windows fill up. Build the four common kinds of agent memory — a recent window, summaries, a notes file and retrieval — and decide what to forget.Middle6 min read🧪 Lab📝 Test
Building AI agentsLesson 201: Writing your own MCP serverThe Model Context Protocol lets any compatible app use your tools. Learn what a server exposes, how it is structured, and how to keep it to least privilege.Middle7 min read🧪 Lab📝 Test
Building AI agentsLesson 202: Orchestrating several agentsOne agent can hand parts of a job to others. Learn the orchestrator-worker pattern and handoffs, and when more agents help or just add cost and confusion.Middle6 min read🧪 Lab📝 Test
Building AI agentsLesson 203: Building a computer-use agentSome agents work through a screen: they look at a screenshot, then click and type. Learn the loop, why it must run in a sandbox, and where to ask for confirmation.Advanced7 min read🧪 Lab📝 Test
Building AI agentsLesson 204: When not to build an agentAgents are powerful but slow, costly and less predictable. Often a single prompt or a fixed workflow does the job better. Here is how to choose.Middle5 min read🧪 Lab📝 Test
Building AI agentsLesson 205: Evaluating and debugging agentsYou cannot improve what you don't measure. Record traces of every run, build test scenarios, measure task success, and replay runs to find what went wrong.Middle7 min read🧪 Lab📝 Test
Building AI agentsLesson 206: Cost, latency and reliability for agentsAgents make many calls, so small costs and delays multiply. Learn budgets, caching, parallel tool calls, timeouts, fallbacks and when to hand over to a person.Middle7 min read🧪 Lab📝 Test
RAG and search engineeringLesson 207: Why retrieval beats retrainingA model only knows what it saw in training. Retrieval lets it read your fresh and private documents at answer time, and show where each answer came from.Middle6 min read🧪 Lab📝 Test
RAG and search engineeringLesson 208: Embeddings and similarity searchAn embedding turns text into a list of numbers, so that texts with similar meaning land close together. Learn how similarity is measured and how search stays fast.Middle6 min read🧪 Lab📝 Test
RAG and search engineeringLesson 209: Chunking documents wellSearch finds chunks, not whole documents. How you cut the text decides what can be found. Learn size, overlap, splitting by headings and useful metadata.Middle8 min read🧪 Lab📝 Test
RAG and search engineeringLesson 210: Hybrid search: keywords plus vectorsKeyword search is great at exact words, codes and names. Vector search is great at meaning. Hybrid search runs both and fuses the results.Middle6 min read🧪 Lab📝 Test
RAG and search engineeringLesson 211: Re-ranking and query rewritingThe first search casts a wide net. A re-ranker reads each candidate closely and puts the best on top. Query rewriting fixes questions that are vague, short or in a different language or script.Middle8 min read🧪 Lab📝 Test
RAG and search engineeringLesson 212: Grounded answers with citationsFinding the right text is half the job. The other half is an answer that uses only that text, says "not found" when it must, and shows its sources so people can check.Middle6 min read🧪 Lab📝 Test
RAG and search engineeringLesson 213: Tables, PDFs and messy documentsReal documents are scanned, multi-column, full of tables and images. Most RAG failures start here, before any search happens.Middle7 min read🧪 Lab📝 Test
RAG and search engineeringLesson 214: Keeping the index freshDocuments change, get deleted and get new versions. Your index must follow, or the bot will confidently quote rules that no longer exist.Middle7 min read🧪 Lab📝 Test
RAG and search engineeringLesson 215: Knowledge graphs and structured retrievalSome questions are about counting, filtering and connecting facts, not finding a paragraph. For those, SQL or a graph can beat vector search.Advanced7 min read🧪 Lab📝 Test
RAG and search engineeringLesson 216: Measuring retrieval and answer qualityA RAG system can fail at search or at answering. Measure both: recall@k and MRR for retrieval, faithfulness and correctness for answers, on a golden question set.Middle8 min read🧪 Lab📝 Test
Evaluating AI systemsLesson 217: Benchmarks and their limitsWhat popular benchmarks such as MMLU, HumanEval, SWE-bench and LMArena actually measure, why they wear out, and why a high score does not mean the model can do your job.Middle6 min read🧪 Lab📝 Test
Evaluating AI systemsLesson 218: Contamination: when the test leakedModels learn from huge amounts of web text, which can include the very test questions used to judge them. Learn the signs, and how held-out and fresh test sets protect you.Middle6 min read🧪 Lab📝 Test
Evaluating AI systemsLesson 219: What to measure for your use caseAccuracy is only one number. Task success, faithfulness, safety, speed, cost and user satisfaction all matter. Decide what "good" means before you build.Middle6 min read🧪 Lab📝 Test
Evaluating AI systemsLesson 220: Building your own test setThe most useful evaluation is a private set of real examples from your task, with clear labels. Here is how to collect, label, size and protect it.Middle7 min read🧪 Lab📝 Test
Evaluating AI systemsLesson 221: LLM as a judgeA language model can grade other models' answers quickly and cheaply, if you give it a clear rubric, check it for known biases and compare it with human judgement.Middle7 min read🧪 Lab📝 Test
Evaluating AI systemsLesson 222: Human evaluation done rightHuman judgement is the gold standard only when it is done carefully: clear rubrics, more than one rater, a check on agreement, and blind comparisons.Middle5 min read🧪 Lab📝 Test
Evaluating AI systemsLesson 223: Regression tests for prompts and modelsA small prompt edit or a model upgrade can quietly break things that used to work. Run your eval suite automatically on every change, with clear thresholds.Middle6 min read🧪 Lab📝 Test
Evaluating AI systemsLesson 224: A/B tests and online metricsOffline evals tell you what might happen. A/B tests with real users tell you what does happen. Learn guardrail metrics, sample size intuition and novelty effects.Middle7 min read🧪 Lab📝 Test
Evaluating AI systemsLesson 225: Evaluating safety and dangerous capabilitiesSafety evals check two things at once: that the system refuses what it should, and that it does not refuse what it should not. At the frontier, labs also test for dangerous capabilities.Middle6 min read🧪 Lab📝 Test
Small and on-device modelsLesson 226: Why small models matterBig models are impressive, but small ones can be cheaper, more private, faster and able to work offline. Learn the real trade-offs so you can choose well.Middle5 min read🧪 Lab📝 Test
Small and on-device modelsLesson 227: Open-weight models and licencesDownloading a model does not mean you may use it any way you like. Learn the difference between open weights and open source, and how to read a model licence.Middle5 min read🧪 Lab📝 Test
Small and on-device modelsLesson 228: Running a model on your own computerTools like llama.cpp and Ollama let you run open-weight models on a laptop. Learn how much memory you need, when a GPU helps, and what a GGUF file is.Middle7 min read🧪 Lab📝 Test
Small and on-device modelsLesson 229: Quantisation in practiceQuantisation stores each weight with fewer bits so a model needs less memory. See exactly what 8-bit and 4-bit mean with a tiny numpy example.Middle6 min read🧪 Lab📝 Test
Small and on-device modelsLesson 230: Distillation: teaching a small modelA big teacher model can help train a small student model for one job. Learn how knowledge distillation and synthetic data work, and the licence rules that can forbid it.Advanced6 min read🧪 Lab📝 Test
Small and on-device modelsLesson 231: Fine-tuning a small model for one jobFine-tuning teaches a small model one narrow task. Learn when it is worth it, how to prepare data, how LoRA and QLoRA fit on one GPU, and how to measure before and after.Advanced7 min read🧪 Lab📝 Test
Small and on-device modelsLesson 232: AI on phones and edge devicesRunning AI on the phone or a small device keeps data local and works offline. Learn the runtimes, the battery and memory limits, and how to update models safely.Middle6 min read🧪 Lab📝 Test
Small and on-device modelsLesson 233: Serving small models cheaplyServing a model to many users costs money every hour. Learn batching, caching, CPU inference and autoscaling, and how to tell when a hosted API is simply cheaper.Advanced6 min read🧪 Lab📝 Test
Securing AI applicationsLesson 135: The OWASP Top 10 for LLM applicationsA shared list of the ten biggest risks in apps built on large language models. Learn each one in a line, and where this course covers it.Middle7 min read🧪 Lab📝 Test