EAT Step 6 Walk-Through — Monitor, Evaluate, Adjust

Content

EAT Step 6 Walk-Through — Monitor, Evaluate, Adjust Resource type: Tier 1 Flagship — EAT Walk-Through (Methodology Education, Not Analysis) Adapted from: DHS Equity Analysis Toolkit Guide (Minnesota DHS) — Step 6: Accountability, Communication, and Evaluation Library home: L01 EAT Walk-Through Library Companion to: T02_01 (EAT Overview), T02_06 (Step 5 Implementation), T02_10 (IS/IS NOT) IMPORTANT — Scope of this resource: This walks staff through the methodology of the DHS Equity Analysis Toolkit. It does not perform equity analyses. Actual equity analyses are conducted by the appropriate authorities — your Equity Champion, the One DSD Equity Team, or the DHS Equity Director for your work area. What this step is for Step 6 is the step most likely to be skipped — and the one whose absence most undermines the equity work that preceded it. Without monitoring and evaluation, a program can implement an equity analysis recommendation, declare success, and never learn that the change did not produce the intended results, or produced new disparities, or helped some communities while leaving others behind. This walk-through teaches staff how the EAT approaches monitoring, evaluation, feedback loops, and iteration discipline. Plain-language overview Step 6 of the EAT asks: Are we achieving what we said we would achieve — for all communities, not just on average? The DHS EAT Guide (https://mn.gov/dhs/) describes Step 6 as "accountability, communication, and evaluation of results" and is clear that the equity analysis process is ongoing: "Using the tool at different phases helps identify and implement new opportunities for advancing equity." This is not a one-time activity that concludes when the implementation plan is complete. It is a continuing discipline. The Race Forward Racial Equity Impact Assessment (https://www.raceforward.org/sites/default/files/RacialJusticeImpactAssessment_v5.pdf) frames success indicators as the final question of any equity analysis: "What are the success indicators and progress benchmarks? How will impacts be documented and evaluated? How will the level, diversity and quality of ongoing stakeholder engagement be assessed?" Walk-through (the methodology) Outcome tracking: the three levels of performance measurement The EAT Guide specifies that performance level measures respond to three levels: 1. Quantity: How much did we do? This is the baseline measure — how many people received the service, how many applications were processed, how many training sessions were delivered. Quantity measures tell you the volume of activity. They are necessary but not sufficient for equity evaluation. 2. Quality: How well did we do it? Quality measures assess whether the service was delivered consistently, accessibly, and responsively. For equity purposes, quality must be assessed disaggregated by race — a service delivered consistently to white clients but inconsistently to Black clients has a quality equity problem even if its aggregate quality score is high. 3. Is anyone better off? This is the equity question. Did the communities that bore the greatest burden before the change experience improvement? Are the people most affected by the proposal — the communities identified in Step 1 and engaged in Step 3 — experiencing better outcomes? Are disparities narrowing? This third level is the one most equity analyses never reach because the data systems to answer it do not exist or have not been disaggregated. For each of the three levels, the EAT expects tracking disaggregated by race. An equity analysis that monitors "overall" outcomes without disaggregation cannot tell whether the change helped or harmed the communities it was designed to benefit. Feedback loops Feedback loops are mechanisms through which communities experiencing the change can report back on whether it is working for them. The EAT Guide requires that implementation include "ongoing data collection, public reporting, and community engagement." A feedback loop is the community engagement component of Step 6. Effective feedback loops: • Are accessible to the communities most affected. A feedback survey that is only in English, only online, and requires thirty minutes to complete will not reach the communities with the highest barriers. Design feedback mechanisms that match the access patterns of the communities you are trying to hear from. • Are responsive. If community feedback is collected but does not inform any decisions, communities will stop providing it. Close the loop: share back what you heard, what you decided, and why. • Are ongoing. A one-time post-implementation survey is not a feedback loop. Feedback loops are regular touchpoints — quarterly check-ins, standing agenda items in community partner meetings, annual community surveys — that create a continuous channel for community voice into program evaluation. • Are disaggregated. Feedback from community members should be analyzed by community to identify whether different groups are experiencing different outcomes. Aggregate satisfaction scores mask community-specific concerns. Community feedback mechanisms The King County Equity Impact Review (https://blog.homelessinfo.org/wp-content/uploads/King-County-Equity-Impact-Review-checklist_Mar2017.pdf) frames Phase 5 (Ongoing Learning) as: "Evaluate whether your action appropriately responds to community priorities and concerns. Learn with the community to adjust your action as their priorities and concerns shift. Communicate progress to all stakeholders. Plan to include community feedback into future planning." In DSD's context, community feedback mechanisms might include: • Standing advisory groups representing communities most affected by a change, with regular meeting cadences and documented feedback processes • Community-based organization partner check-ins, where advocacy partners who participated in Step 3 are invited to report on community experience of the implementation • Client satisfaction surveys, disaggregated by race and primary language • Formal community listening sessions at six-month and annual intervals post-implementation • Grievance and complaint tracking, disaggregated by community, with analysis for patterns that suggest systemic implementation failures Iteration discipline Iteration discipline means committing in advance to a process for adjusting the implementation based on what monitoring and evaluation reveal — and then actually following through. The EAT Guide asks: "Are you achieving anticipated outcomes? Do you have a plan to course-correct if needed?" Iteration discipline requires: 1. Pre-specified adjustment triggers. Before implementation begins, define what evidence would trigger a course-correction review. Example: "If the enrollment gap between Somali-origin and white applicants has not narrowed by 10 percentage points within 12 months, we will convene a review." Without pre-specified triggers, teams wait for a crisis instead of proactively adjusting. 2. A designated review body. Who reviews monitoring data and decides whether adjustment is needed? The Equity Director, the Equity Committee, or a designated implementation team should be specified in the implementation plan. Without a named body, review does not happen. 3. A documented adjustment process. When monitoring data triggers a review, what happens? Who is involved? What decision-making authority exists? How are communities engaged in the adjustment? Document this process in Step 5 so it is ready when Step 6 requires it. 4. Honest reporting. "We are not achieving our equity targets" is a finding that must be reported — to Equity Directors, to division leadership, and to the communities affected. The temptation to report only positive progress is real; the EAT framework requires honest reporting, even when the news is difficult. The ongoing nature of Step 6 The EAT Guide states that "the equity analysis process is ongoing. Using the tool at different phases helps identify and implement new opportunities for advancing equity." This means Step 6 is not a conclusion — it is a transition to the next iteration of the process. What monitoring reveals in Step 6 may trigger a new Step 1 — a re-framing of the issue in light of what was learned. Or it may trigger a new Step 4 — generating new strategies because the original strategies did not produce the intended equity impact. The EAT is a cycle, not a linear sequence with a defined end. Programs and policies change over time. Communities change. The data environment changes. Equity analysis that treats the toolkit as something to be "completed" rather than as an ongoing practice will miss the opportunities for course-correction that Step 6 is designed to surface. Worked example fragment (anonymized, methodology illustration only) Continuing the anonymized waiver program scenario at 12 months post-implementation: The Equity Champion conducts the first annual monitoring review. Quantity: Applications processed increased by 22%. Somali-origin applications increased by 31% — the targeted community is engaging more. Quality: Error rates on income documentation processing decreased overall. However, Karen-origin applications are experiencing a higher rate of processing delays — a new disparity not present before the change. Is anyone better off? Somali-origin enrollment increased; the documentation disparity is narrowing. Karen-origin enrollment has not improved, and a new processing-delay disparity has emerged. Feedback loop findings: Community-based organization partners report that Karen-origin applicants are encountering a specific staff training gap — case managers were trained on the new income documentation alternatives but not on the application navigator service that was designed to support Karen-origin applicants. Adjustment triggered: Pre-specified trigger met (Karen-origin processing delays exceeding threshold). Review convened. New action added to implementation plan: targeted staff training on application navigator service for Karen-origin applicants; monthly processing-time monitoring added for Karen-origin applications specifically. This resource walks through methodology only. The actual monitoring and evaluation would be conducted by the Equity Champion and Equity Director. IS / IS NOT Step 6 IS — Step 6 IS NOT Ongoing monitoring at three levels: quantity, quality, is anyone better off? — A one-time post-implementation survey Disaggregated evaluation — by race, not just in aggregate — Aggregate outcome reporting that masks community-specific results Community feedback mechanisms that are accessible and responsive — A feedback form that communities cannot access or that never influences decisions Iteration discipline with pre-specified adjustment triggers — Waiting for a crisis before reviewing whether implementation is working Honest reporting of equity target progress (or non-progress) — Only reporting when results are positive A transition to the next cycle of equity analysis — The final step that "completes" the process Common questions What if monitoring reveals that the implementation is not working? Document the finding, trigger the pre-specified adjustment review, and convene the designated review body. Do not delay reporting because the news is difficult. The EAT framework expects this — implementation without monitoring is just hope, not equity work. How long does Step 6 monitoring last? For as long as the policy or program exists. Step 6 is not a temporary phase; it is the ongoing operational posture. The monitoring cadence may change over time — more frequent early in implementation, less frequent once stable patterns are established — but it does not end. Who has access to monitoring data? The Equity Director, the Equity Committee, division leadership, and the communities affected. The EAT Guide is explicit that the Equity Impact Statement and monitoring results should be shared with stakeholders and posted on SharePoint. Monitoring data is not an internal document for program staff only. Connection to DSD goals Step 6 is where DSD's equity commitments become verifiable. DSD's six organizational goals are not just aspirational — they are measurable. Step 6 is the mechanism through which the organization knows whether its equity work is producing the outcomes it committed to. Without Step 6, DSD cannot honestly report whether equity is advancing or stalling. When to escalate / where to refer • If monitoring data reveals that equity targets are not being met and adjustment within the program area's authority is insufficient: escalate to division director with specific findings and resource implications. • If community feedback reveals significant concerns not captured in quantitative monitoring data: convene the review body and involve the communities in the adjustment process. • If monitoring reveals that a policy change has produced new disparities not present before the analysis: initiate a new Step 1 framing to address the new disparity. Reflective questions 1. Think of a program your area runs. Do you currently have monitoring measures that are disaggregated by race? If not, what would it take to add them? 2. What is the difference between a feedback mechanism and a feedback loop? Does your program area close the loop with communities after gathering feedback? 3. Have you ever seen a program course-correct based on equity monitoring data? What made the adjustment possible? 4. What would "pre-specified adjustment triggers" look like for a major program in your area? What evidence would you specify as the threshold for a review? 5. What does ongoing equity analysis look like in a program area that does not have dedicated equity staff time? Sources • DHS Equity Analysis Toolkit Guide — Minnesota DHS (https://mn.gov/dhs/) (primary source, Step 6) • Racial Equity Impact Assessment Guide — Race Forward (https://www.raceforward.org/sites/default/files/RacialJusticeImpactAssessment_v5.pdf) (section 10: identifying success indicators) • King County Equity Impact Review — Phase 5: Ongoing Learning (https://blog.homelessinfo.org/wp-content/uploads/King-County-Equity-Impact-Review-checklist_Mar2017.pdf) • Racial Equity Toolkit — GARE, via Othering and Belonging Institute (https://belonging.berkeley.edu/sites/default/files/ARCHIVE/gare-racial_equity_toolkit.pdf) • One DSD Program Charter v1.1 (one-dsd-program-charter) Always defer to the person in front of you.

Ask: What if monitoring reveals that the implementation is not working?

Document the finding, trigger the pre-specified adjustment review, and convene the designated review body. Do not delay reporting because the news is difficult. The EAT framework expects this — implementation without monitoring is just hope, not equity work.

Ask: How long does Step 6 monitoring last?

Ask: Who has access to monitoring data?

Ask: What is the difference between a feedback mechanism and a feedback loop? Does your program area close the loop with communities after gathering feedback?

Ask: Have you ever seen a program course-correct based on equity monitoring data? What made the adjustment possible?

Ask: What would "pre-specified adjustment triggers" look like for a major program in your area? What evidence would you specify as the threshold for a review?

For all staff, use EAT Step 6 Walk-Through — Monitor, Evaluate, Adjust to draft, structure, test, and distribute communication in usable formats; use the guidance in the immediate task, check impact and access, and ask for help when authority is unclear. Start by ask: What if monitoring reveals that the implementation is not working?

Do not replace official legal, policy, clinical, supervisory, program, or Tribal authority.

Do not infer an individual's identity, preferences, needs, or experience from group-level information.

State uncertainty, use current authoritative sources, and escalate when the decision exceeds the user's role.

IMPORTANT — Scope of this resource: This walks staff through the methodology of the DHS Equity Analysis Toolkit. It does not perform equity analyses. Actual equity analyses are conducted by the appropriate authorities — your Equity Champion, the One DSD Equity Team, or the DHS Equity Director for your work area. Step 1. Ask: What if monitoring reveals that the implementation is not working? Step 2. Document the finding, trigger the pre-specified adjustment review, and convene the designated review body. Do not delay reporting because the news is difficult. The EAT framework expects this — implementation without monitoring is just hope, not equity work. Step 3. Ask: How long does Step 6 monitoring last? Step 4. Ask: Who has access to monitoring data? Step 5. Ask: What is the difference between a feedback mechanism and a feedback loop? Does your program area close the loop with communities after gathering feedback? Step 6. Ask: Have you ever seen a program course-correct based on equity monitoring data? What made the adjustment possible? Step 7. Ask: What would "pre-specified adjustment triggers" look like for a major program in your area? What evidence would you specify as the threshold for a review?