wunder beta

📘 How do you manage risk in an AI product?

Hazards, fairness, and NIST RMF—how AI product risk is mapped, measured, and owned.

4
lessons
~20 min
to learn
Adults
level
Start the course →

What you’ll learn

  1. Mapping AI Product RisksIdentify and distinguish the major categories of risk that AI products can introduce.AI products carry overlapping risks: bias and fairness harms, privacy and data-protection failures, security threats including prompt injection and data leakage, hallucination and misinformation, safety in high-stakes use, and deliberate misuse. These categories often interact, so a single incident may belong to several at once. Naming them early enables systematic, proactive reasoning rather than after-the-fact reaction. This vocabulary is the foundation for the fairness, framework, and documentation lessons that follow.
  2. Fairness Concepts and Trade-offsExplain core fairness concepts and why competing fairness definitions cannot all hold simultaneously.Fairness in machine learning is a family of competing definitions spanning group, individual, and procedural notions, each encoding different values. Disparate impact describes disproportionately harmful outcomes for a protected group without requiring intent. A well-known impossibility result shows that when group base rates differ, common criteria such as calibration and equal error rates cannot all be satisfied at once. Because no purely technical fix exists, teams must explicitly choose and justify a fairness definition appropriate to their context.
  3. The NIST AI Risk Management FrameworkDescribe the four functions of the NIST AI RMF and how they structure responsible AI risk management.The NIST AI Risk Management Framework (AI RMF 1.0, 2023) is a voluntary, adaptable guide for managing AI risks across the lifecycle, oriented around trustworthiness characteristics like safety, security, privacy, fairness, and transparency. Its four core functions are Govern, Map, Measure, and Manage. Govern is a continuous, cross-cutting function for policy and accountability, while Map, Measure, and Manage form an iterative cycle of contextualizing, analyzing, and treating risks. The framework provides a shared structure and language for cross-functional teams rather than a rigid checklist.
  4. Build a Risk Case MemoApply documentation, guardrails, and framework language to write a clear AI risk case memo.This guided project synthesizes the course into a written deliverable. Transparency tools such as model cards (Mitchell et al., 2019) and datasheets for datasets (Gebru et al., 2021) supply key inputs, while operational controls (content moderation, input validation, data minimization, consent, monitoring, and incident response) implement the Manage function. A strong case memo states the system and intended use, maps top risks, evaluates likelihood and severity, and recommends mitigations tied to owners. It closes with an actionable, non-alarmist recommendation grounded in residual risk.

Questions this course answers

A language model is tricked by hidden text in a document into ignoring its instructions and revealing data it was given. This is best described as:

Manipulating a model through injected instructions is prompt injection, a security risk. When it causes the model to disclose sensitive data, it also becomes a data-leakage/privacy issue, showing how categories overlap.

Which scenario is an example of misuse rather than bias or hallucination?

Misuse is deliberate use of a system for harm. Fabricated citations are hallucination, and the demographic-rejection and accent examples are bias/fairness harms.

Why is it useful to name risk categories early in an AI product's development?

Naming categories provides structure for proactive analysis. It does not eliminate risk, replace testing, and is not a universal legal mandate.

What does the fairness impossibility result tell practitioners?

When base rates differ between groups, a classifier generally cannot be both calibrated and have equal false-positive and false-negative rates. This forces an explicit, justified choice rather than implying fairness is unattainable.

A hiring model with no demographic inputs still accepts far fewer applicants from one protected group. This is most precisely an example of:

Disparate impact is a disproportionately harmful outcome for a protected group without requiring discriminatory intent, which matches a neutral-seeming model producing unequal acceptance rates.

Why is choosing a fairness definition considered a normative, not purely technical, decision?

Competing fairness notions embed different value judgments, so selecting one expresses priorities about acceptable harms and cannot be settled by technical means alone.

Grounded in trusted sources

  • NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0), 2023
  • Barocas, Hardt & Narayanan, Fairness and Machine Learning (fairmlbook.org)
  • NIST AI RMF Playbook (NIST Trustworthy & Responsible AI Resource Center)
  • Mitchell et al., Model Cards for Model Reporting, FAT* 2019
  • Gebru et al., Datasheets for Datasets, Communications of the ACM, 2021
  • NIST — Artificial Intelligence Risk Management Framework (AI RMF 1.0)

Every Wunder lesson is built from real, reputable sources — never invented.

Related courses

📘How do you power an experiment?📘How do you choose experiment metrics?📘How does an ETL pipeline stay reliable?📘How should a data warehouse be modeled?📘How does streaming beat the nightly file?📘Keep stakeholder from meaning everyone📘Roadmaps: Product Foundations📘How do you write a brief before you build?

Wunder is a personalized learn-anything platform — tell it any topic and it builds a beautiful, fact-checked course in minutes, with narration, a knowledge check, and a college-style University track.

All topics · Home

© 2026 Wunder Learning LLC · Terms & Privacy