A Power BI semantic model is ready for serious Copilot testing when its business terms resolve to owned measures, its grain and calendar logic are explicit, its AI-facing schema is focused, important questions have verified paths, permissions have been exercised by role, and changes are checked against a repeatable evaluation set. “The report works” is not enough evidence. Use this 21-point checklist to find the missing control, owner, or test before you mark a model Approved for Copilot.
Microsoft’s current preparation guidance covers model design, AI data schemas, verified answers, AI instructions, descriptions, testing, and the Approved for Copilot setting. Microsoft also warns that output is nondeterministic, so readiness is an operating discipline rather than a one-time configuration. Prepare a semantic model for AI
How to use the checklist
For each item, record one of four statuses:
- Yes—documented and tested: a named owner can show current evidence.
- Partly: the control exists, but scope, evidence, or testing is incomplete.
- No: the control does not exist for the target use case.
- Not sure: nobody can locate sufficient evidence. Treat this as an evidence gap, not a passing result.
The status language is a Refinity recommendation. It is not a Microsoft certification scale.
Apply the checklist to a bounded scope: one consequential use case, one priority semantic model, one target user group, and a representative set of business questions. A broad tenant-level answer hides the model-specific defects that matter.
1. Model structure
| Check | Passing evidence | Why it matters |
|---|---|---|
| 1. Every exposed fact table has a declared grain | Model note or data contract states what one row represents; duplicate-key test supports it | Copilot can generate a valid aggregation over an invalid combination of grains |
| 2. Relationships are intentional and tested | Cardinality, active/inactive role, and filter direction are reviewed; ambiguous paths are resolved | Generated queries depend on the same relationship graph as report queries |
| 3. User-facing tables and fields have distinct purposes | Model view shows fact/dimension separation and hides technical or unsafe fields | A crowded schema gives natural-language mapping more plausible wrong choices |
Microsoft’s star-schema guidance recommends dimension tables for filtering/grouping, fact tables for summarization, and a consistent grain within fact tables. Understand star schema
2. Business terminology
| Check | Passing evidence | Why it matters |
|---|---|---|
| 4. High-value business terms have one approved meaning for the target group | Glossary maps each term to a measure, dimension, owner, and scope | “Sales,” “customer,” or “active” can each resolve to several legitimate fields |
| 5. Synonyms and exclusions are documented | Representative user phrases and forbidden interpretations are recorded | Model authors and business users rarely use identical vocabulary |
| 6. Descriptions explain business meaning, not field names | Descriptions state definition, population, time basis, and important exclusions | Repeating the column name adds no disambiguating context |
Microsoft says descriptions can provide context about model data, although their effect varies by Copilot capability. Prep data for AI FAQ
3. Measures and data foundations
| Check | Passing evidence | Why it matters |
|---|---|---|
| 7. Consequential calculations use explicit, owned measures | Measure inventory includes definition, DAX, format, owner, and validation source | Raw numeric columns allow aggregations the business may not endorse |
| 8. Metric lineage is traceable | Metric → source → transformation → grain → calendar → filters can be followed | A correct-looking number is not defensible without its derivation |
| 9. Benchmark values reconcile at representative cuts | Signed-off comparisons cover totals and sensitive dimensions/periods | A model-wide total can pass while regional or calendar-specific results fail |
Microsoft distinguishes explicit DAX measures from implicit column summarizations and notes that explicit measures let modelers control appropriate aggregation. Star-schema measure guidance
4. AI context and instructions
| Check | Passing evidence | Why it matters |
|---|---|---|
| 10. The AI data schema is intentionally smaller than the full authoring surface | Reviewed include/exclude list tied to target questions | Microsoft says a focused subset can reduce ambiguity for schema-based questions |
| 11. Instructions address true business context | Each instruction maps to a tested ambiguity such as a local term or analysis rule | Long generic instructions can conflict and are hard to evaluate |
| 12. AI configuration has an owner and change history | Versioned schema/instruction record, reviewer, and effective date | An invisible model-level control can drift without users knowing |
Microsoft documents that AI instructions are model-level, hidden from consumers, limited in where they apply, and not expected to be persona-specific. AI instructions
5. Verified answers
| Check | Passing evidence | Why it matters |
|---|---|---|
| 13. High-value questions are candidates for verified answers | Prioritized question list identifies existing maintained visuals | A known visual can provide a controlled response path for common questions |
| 14. Trigger phrases cover real language | Phrases come from target users and include tested variations | An author’s preferred wording may miss how users actually ask |
| 15. The source visual has an owner and validation path | Visual, measure, refresh, and benchmark owners are named | A verified path is only as trustworthy as the artifact behind it |
Microsoft defines a verified answer as an author-configured, validated response that can return a selected report visual when a question matches its trigger phrases. Verified answers guidance
6. Security and governance
| Check | Passing evidence | Why it matters |
|---|---|---|
| 16. Each target role has been tested end to end | Named users or test identities exercise the actual Copilot surface and benchmark questions | Administrator-only testing cannot demonstrate a consumer’s result |
| 17. Sensitive metadata exposure has been reviewed | Model/report metadata, descriptions, examples, and field values are classified | Microsoft cautions that report metadata used for grounding can contain data points |
| 18. Approval and incident ownership are explicit | Named approver, review date, rollback action, and error-reporting path | “Approved” without an owner or expiry date becomes stale assurance |
Microsoft notes that Copilot can use report metadata as grounding data in Power BI Desktop and cautions that metadata can sometimes include sensitive values. Use Copilot with semantic models
7. Evaluation and lifecycle controls
| Check | Passing evidence | Why it matters |
|---|---|---|
| 19. A golden-question set has expected-answer contracts | Questions include variants, benchmark, filters, role, and acceptable tolerance | Natural-language quality cannot be tested against a vague expectation |
| 20. Tests cover repeatability and safe failure | Critical questions are run more than once; ambiguous or unsupported questions should clarify or decline | Nondeterministic output requires contract-based, repeated evaluation |
| 21. Model and AI changes trigger re-evaluation | Release checklist or pull request records affected questions and results | A correct answer today can regress after a measure, relationship, or instruction changes |
Microsoft recommends manual testing with known benchmarks, comparisons to trusted reports or analyses, before/after comparisons, and automated tests before deployment where practical. Validate Power BI content
Readiness gates: a useful alternative to an average score
An overall score can summarize progress, but critical controls should override the average. This gate model is a Refinity recommendation:
| Gate | Minimum evidence before wider use | Blocker examples |
|---|---|---|
| Foundation | Grain, relationships, key measures, calendar, and benchmark are documented and tested | Competing revenue definitions; mixed-grain double count; unknown fiscal-period logic |
| AI configuration | Focused schema, tested verified-answer candidates, concise instructions, owner | Whole model exposed without rationale; conflicting instructions; unowned source visual |
| Security | Each target role tested on consequential questions | RLS only tested by an admin; unexplained cross-role differences |
| Evaluation | Versioned golden questions, expected contracts, regression process | No expected answers; single successful demo treated as proof |
Do not call the model ready for wider use if any foundation or security blocker remains, even if most checklist items pass.
Evidence-gap triage
“Not sure” often identifies the fastest useful next step. Route each uncertainty to a concrete artifact:
- Which revenue definition is approved? → metric owner and decision record.
- What does one fact row represent? → grain statement and duplicate-key test.
- Which calendar does “last quarter” use? → calendar contract and expected DAX result.
- Which fields does Copilot see? → AI data-schema export or reviewed field list.
- Does RLS apply to the answer? → named-role test with captured result.
- How do we know the answer is correct? → benchmark query or governed report with reconciliation.
A practical first 30 days
- Select one high-consequence question family and the people who ask it.
- Establish the metric, source, grain, calendar, filter, and ownership contract.
- Repair foundation defects before AI configuration.
- Define the focused AI schema, verified-answer candidates, and only the instructions needed for unresolved business context.
- Test a versioned question set across user roles and prompt variants.
- Record remaining risks before considering Approved for Copilot.
The sequence is deliberately narrow. It produces evidence about one real use case before the team expands scope.
Microsoft product facts vs. Refinity recommendations
Microsoft product facts
- Prep data for AI currently includes AI data schemas, verified answers, and AI instructions.
- These settings are applied at the semantic-model level.
- Microsoft provides an Approved for Copilot setting for a semantic model.
- Copilot outputs are nondeterministic.
- Microsoft recommends evaluating known benchmarks before relying on outputs.
Refinity recommendations
- Scope readiness to one model, use case, user group, and question set.
- Require evidence for a “yes”; treat “not sure” as a gap.
- Use foundation and security blockers to override an attractive average score.
- Version the expected-answer contracts alongside model changes.
- Time-bound approval and assign an accountable owner.
This checklist is a directional, self-assessed diagnostic. It is not a certification, audit opinion, or guarantee of answer accuracy.
Primary CTA: Complete the ungated Power BI Copilot readiness scorecard.
Related reading: