Evaluate AI systems before they reply or act

Recognize meaningful AI failures. Decide whether the evidence is enough to release.

Start free — the primer See what you will learn

An AI product can give a polished answer and still be wrong, unsafe, inconsistent, or unsupported; an AI agent can take the wrong action. Learn to identify the failure, choose a relevant check, read conflicting results, and explain why a release should proceed or stop.

One Lumina course organizes a repeatable evaluation process into eight tracks: the Full Judge-Driven Harness. No programming is required for the core learning path. CORE is the anchor: you first practise with cases and release decisions. Seven applied tracks then use the same method for systems and products that answer, retrieve, generate, or act. If you want to implement it, optional Python labs show how a judge — a check with an explicit criterion — and a harness — the repeatable process that runs those checks — work as code.

58HTML lessons
57runnable labs
40+sources cited
8tracks

Course hero under final review

The Judge-Driven Harness · bilingual course hero

AI-generated fictional presenter

One method, eight applications

Generate → judge → gate → retry

The anchor course teaches the shared method. Each applied track uses it for a different AI system or product: one that answers, retrieves, generates, takes actions, or supports a decision. Start at the anchor, then choose the application closest to your work.

generate judge (as code) gate (go / no-go) retry · promote · hold · rollback
Why this isn't a free blog post

Proof, not vibes

Three things no marketplace course bundles together.

◆ published research

Research-backed

Every claim traces to published, public research — OWASP LLM & Agentic Top 10, MITRE ATLAS, RAGAS, MT-Bench, DORA, NIST AI 600-1, C2PA — cited and credited, never hand-waved.

▶ runnable

57 deterministic labs

The 58 lessons are paired with 57 Python labs that run offline, stdlib-only, with the same output every time. Not slides — code you run, then point at your own system.

✓ honest

Shipped vs backlog

Every lab metric — a numeric result from an illustrative synthetic example — is labelled. We teach you to measure your own system, never claim that ours “hits X%.”

The eight tracks

The method, applied to your domain

Start with the Harness anchor, then add the application track that matches your work — or take the whole Harness.

CORE · the method

The Judge-Driven Harness

The domain-agnostic spine: discover failures, write judges as code, validate them against a compact labeled set, gate, retry, calibrate, wire into CI.

free primer + 9 modules · 8 labs
EVAL · eval

Ship LLM Apps on Evidence

Start from trace logs (OpenTelemetry): inspect retrieved sources and answers with the RAG triad, keep a reference set, then use reproducible CI checks. A Pareto frontier makes the cost–quality trade-off visible instead of hiding it in one score.

7 modules · 7 labs
AGENT · agentic coding

Agentic Coding Harness

Claude Code in production: AGENTS.md as contract, specialized subagents, hardening & determinism, red-team your own agents, judge-driven review — governance, not a feature tour.

7 modules · 7 labs
SEC · security

Securing LLM Apps

Turn prompt-injection risks into repeatable adversarial tests, then into detection rules for a SIEM: the system that centralizes logs and raises alerts. OWASP and MITRE ATLAS provide the public risk vocabulary.

6 modules · 6 labs
DOG · dogfooding

Dogfooding

Validate your own product: run it as the eval, orthogonal quality dimensions, calibrated gates, the living question bank, and dogfooding wired into CI — ship on evidence, not "looks fine."

7 modules · 7 labs
SIM · synthetic users

Synthetic Users & Personas

Goal-directed synthetic-user simulation (an NPC is a simulated user with a goal), multi-turn exploration, and the honest breadth/mass validity boundary: personas widen the behavior space, but do not replace calibration with real users.

8 modules · 8 labs
DATA · the data layer

The Data Flywheel

Build an evaluation bank from real failures: cover important cases, remove duplicates, version the set, and retain regressions. κ agreement is the check that independent reviewers label the same case similarly.

7 modules · 7 labs
COST · the efficiency layer

The Cost & Latency Harness

Make the quality, cost, and latency trade-off visible. Compare latency percentiles rather than only averages, route simple requests economically, and use a Pareto frontier to show which cost–quality options are genuinely different.

7 modules · 7 labs
Standalone public preview

Start with one bounded agent action

This course is outside the eight-track Harness offer. Its public preview opens the course map and GOV.1; enrollment and checkout are not available yet.

GOV · public preview

Governing agent actions

Turn one tool-using agent action into a reviewable decision: role, resource, scope, confirmation, evidence, and an explicit return path.

6 modules · 6 labs · GOV.1 open to read
Explore the course
Pre-launch offers · individual self-study

Explore the scope and price of each course offer.

The prices below describe three planned offers: the anchor method, the complete program, and a later anchor-plus-track bundle. Offers are localized in BRL for Brazil and USD internationally. Enrollment remains unavailable until the Hotmart purchase and delivery journey passes preflight.

Core
The anchor method — the spine every track builds on.
US$150
Launch: US$120
  • Free primer + 9 anchor modules
  • 8 runnable labs
  • The full generate→judge→gate→retry method
  • 12 months of package updates
After initial launch
Complete bundles
The anchor + one track, focused on your domain.
US$300
Planned offer: US$240
  • Anchor + one full track (or the Validate pair: dogfooding + synthetic users)
  • All that track's labs + briefs; Coding Complete pairs the AI Coding OS pack
  • À la carte — security · eval · agentic coding · dogfooding · synthetic users · data · cost
  • Free primer included · 12 months of package updates
15-day guaranteeRequest through Hotmart under the rules shown at checkout.
Yours to keepPersonal license for the downloaded release, with 12 months of package updates.
Offline deliveryDownloadable bilingual ZIP in your Hotmart purchase library after payment confirmation.
Focused supportEmail help for access, download, and a reproducible first-run issue.

Study time is not published as a promise yet. We will state a range only after observing beginner sessions, rather than guessing from the author’s pace.

This route shows the international USD offer. The checkout total is authoritative; local taxes, payment-method costs, or conversion into another currency may change the amount shown before confirmation. The Level-0 primer remains free.
The AI Coding OS pack → is a separate companion toolkit with 18 install-ready skills — not a sixth course or a ninth Harness track.

About the offers

Straight answers

Is this beyond a free OWASP page / blog post?
Yes. The free material tells you the risks exist. This teaches you to build the harness that catches them at scale — red-team as a repeatable suite, detection-as-code, judge-driven gates — with runnable labs and paper provenance. The differentiator is the method + the code, not a list.
Are the labs real, or toy problems?
Real and runnable: 57 Python labs, offline, stdlib-only, deterministic (same output every run). Numbers are illustrative by design so you learn the pattern, then swap the single HOOK for your own model and measure your own system.
Is this current for 2026?
The content tracks the 2026 frontier: OWASP LLM (2025) + Agentic (2026) Top 10, current OpenTelemetry GenAI conventions, MT-Bench judge-bias findings, NIST AI 600-1, C2PA + SynthID, the EU AI Act Article 50 timeline. Tool status (e.g. OpenAI's announced agreement to acquire Promptfoo) is flagged with OSS alternatives.
Isn't Claude Code already taught free by Anthropic?
The free courses teach the features — subagents, MCP, skills, hooks. The Agentic Coding track teaches what they don't: governance, hardening, red-team, and quality gates — Claude Code in production, by people who ship with it — with runnable judge-labs and a starter you keep. It sits above the feature tier, not next to it.
Individual only — is there a team version?
Today every tier is individual self-study, priced accordingly. A team tier with progress tracking and seats is on the roadmap, not shipped — so we don't charge for it yet.
How do I receive it, and does it work offline?
After payment confirmation, the Hotmart library delivers a .zip with an index.html launcher. Open it in any browser, read on any device, and run the labs on a desktop. The course itself does not phone home; package updates remain available in the purchase library for 12 months.
Does the purchase include a certificate or live support?
No. This launch is individual self-study: it does not include a certificate, academic credit, live tutoring, statistical consulting, or implementation consulting. Email support covers access, download, and reproducible first-run issues.
Which payment methods and installments are available?
Hotmart may offer Pix, boleto, and card depending on the buyer and location. The payment methods, installments, taxes, conversion, and final total shown in checkout are authoritative.