9 minutes
How we know — and what a gap cannot prove Audit studies, administrative data, surveys. Triangulate. Skepticism is the right instinct.
What you’ll be able to do Identify the practical implication of “How we know — and what a gap cannot prove” and apply it to one live DHS decision connected to The Record. Compare the responses to this question and explain your choice: How should the team use the concept? Document a next step for How we know — and what a gap cannot prove: Choose one live work object, name a change or formal escalation, assign an owner, and set the date the result will be reviewed. Bertrand and Mullainathan (2004): identical resumes, names randomized. White-sounding names got significantly more callbacks. That isolates perceived race in one setting. It does not settle a legislature.
Three tools, each with limits Audit studies randomize a racial signal (a name, an appearance) and hold the rest constant. Clean causal evidence in a narrow market (hiring, housing). Not a picture of a whole life.
Administrative data (wait times, removals, imprisonment) can show large disparities. They rarely isolate a single cause. A gap can be current discrimination, accumulated history, offending, eligibility, or a mix.
Attitude surveys measure what people say they believe. They do not measure what a portal does at 7 p.m.
No single method is decisive. The rigorous move is to triangulate: where independent methods point the same way — housing, lending, hiring — the finding is mainstream, not fringe.
In this lesson you will Give skepticism its due.
Know what an audit study can and cannot do.
Refuse to treat a dashboard gap as a completed causal story — and refuse to treat it as noise.
You cannot introspect your way to fairness. You measure. Sincerity is not a substitute for evidence.
DHS close — how this changes a packet You have a blended average You do not yet have a disparity. Ask for the breakout. “We don’t have it” is a finding.
You have a breakout You have a disparity. You still need a story about mechanism before you write “discrimination” as if a court had spoken.
You have an audit or a lived-experience pattern plus a number Now you are triangulating. Change the design. That is Adaptation. That is also the toolkit.
Your turn On one live metric: do you have a blend, a breakout, or both?
What would triangulation look like this month (a number plus who you hear plus one design test)?
Put this lesson to work What you can change You control whether how we know — and what a gap cannot prove changes a live DHS process connected to The Record or remains information on a page.
What to watch for The team completes the learning, keeps the same default, assigns no owner, and never checks who carries the burden.
Your next step Choose one live work object, name a change or formal escalation, assign an owner, and set the date the result will be reviewed.
Fictional working example
Match the claim to the method These three fictional study cards examine different evidence. Compare the observation with the conclusion it can support.
Randomized audit Otherwise matched fictional applications are randomly assigned different signals; the study compares responses.
Examine this part A well-designed audit can test the effect of the manipulated signal in the studied setting. It does not establish every cause across other places, times or stages.
Administrative records A service dataset shows longer recorded waits for one group.
Examine this part The difference matters, but the records alone may not identify the mechanism. Check definitions, missing records, process steps and other evidence.
Attitude survey Respondents describe what they believe about service fairness.
Examine this part These are reported views among respondents. They do not directly measure whether an evening portal submission succeeds or how nonrespondents experienced it.
Try it with the work A packet says a survey proves the portal is accessible. Which observation is missing?
Write a narrower supported claim and a second source that could help answer the practical question.
Carry this forward Audit studies, administrative data, surveys. Triangulate. Skepticism is the right instinct.
You control whether how we know — and what a gap cannot prove changes a live DHS process connected to The Record or remains information on a page.
Choose one live work object, name a change or formal escalation, assign an owner, and set the date the result will be reviewed.
Completion is shown by a scenario decision, a knowledge-check response and a saved one-minute commitment.
Choose one live work object, name a change or formal escalation, assign an owner, and set the date the result will be reviewed.
Browse and download only. Course notes are not typed or saved on this page.
Reset this lesson
Previous lesson Next lesson Carry this into practice What the word means, what the history built, and what the evidence still shows.
Return to the experience: What did you notice or try, whose perspective informed it, and what would you keep or adjust?
Participation and course completion in this program do not count toward DHS-required training credits unless management, a director, or DHS leadership expressly approves an exception.
