A semantic layer that made 'talk to your data' reliable

A large government department wanted to let staff ask questions of their data in plain English and get answers, often with simple charts, without writing SQL or waiting on analysts. Databricks Genie Spaces offered exactly that 'talk to your data' interface, and turning it on was almost suspiciously easy.

Because the department already operated Databricks, the hard enablement work was effectively done: identity and access controls, platform governance and auditability were already in place, so there was no new hosting decision, no vendor onboarding and no extended approval chain just to test the capability.

The real problem surfaced once testing began: even clean, well-structured data produced confident nonsense. Out of the box, Genie made incorrect assumptions about meaning and logic: using the wrong aggregation, joining tables incorrectly, applying inconsistent time windows, or comparing incomplete periods against complete ones and presenting the result as insight. For an analyst that is manageable; for a policy officer or executive who cannot verify the SQL, the risk is obvious.

We treated the semantic layer as the real deliverable, not a nice-to-have. The goal was to make Genie stop guessing and start behaving, so that answers would be trustworthy for non-technical users rather than merely impressive in a demo.

We used the levers Databricks provides to guide Genie directly in-product: plain-English metadata on tables, views and fields to anchor meaning; space-level instructions setting out how the space should respond and what assumptions to avoid; and explicit join rules for cases where field names differ or joins need multiple conditions to avoid duplication.

We added curated metrics and aggregations (reusable measures, similar to Power BI measures or Tableau calculations) to encode non-trivial business logic and correct aggregation behaviour, and we pinned validated SQL answers to canonical questions so Genie could reuse proven logic rather than improvise.

We tested against real departmental questions and edge cases. When Genie compared a partial year (data only to November) against prior full years, we worked through the correct logic, validated the query and stored it as a canonical pattern so the mistake stopped recurring. For CPI adjustments (where the CPI series can start decades before the analysis period), we implemented a semantic measure that calculates the deflation factor dynamically from the first year referenced in the question.

The most important outcome was not 'cool AI'. It was trust. With the semantic layer in place, Genie's answers became substantially more consistent, explainable and safe for non-technical users.

Curated joins and metrics reduced guesswork and prevented common failure modes, cutting the number of confidently wrong answers.

Once a tricky scenario such as time windows or CPI logic was solved, it could be encoded and reused, and Genie could handle a long tail of questions that a fixed dashboard would never have the bandwidth to cover.

Policy officers and decision-makers gained stronger, self-serve access to data: they could ask the right questions and get reliable answers without becoming SQL specialists. The clear lesson: Genie does not remove the need for modelling. It amplifies its value, and the semantic layer is the product.