Semanticus

Benchmarks

Numbers we publish are reproducible or they don't ship. Raw data lives on this page.

1. How AI-ready are real semantic models?

We scanned 40 public semantic models from GitHub with the Semanticus AI-readiness analyzer: Microsoft's own samples, well-known community tools, real production repos and training material. Scans are offline and read metadata only. Every repo is pinned by commit, and the raw per-model scores are downloadable below, so anyone can re-run the scan and check us.

Median: 54.1 out of 100. That's an F.
28grade F
9grade D
2grade C
1grade B

Every official Microsoft sample in the corpus graded F. These are not broken models; they render their reports fine. They fail the things an AI consumer needs: descriptions, unambiguous names, synonyms, linguistic schema, Prep-for-AI settings, documented relationships.

Disclosed bias: public, source-controlled models are the polished end of the ecosystem. Authors published them for an audience, and teams using git are the engineering-mature minority. Typical private client models are rougher than this sample, so the median here is plausibly an upper bound.

Download the raw scores (JSON) · scanned 4 July 2026 · scanner and corpus are in the Semanticus repository under tools/readiness-corpus.

Model (repo)KindGradeScore
DaniBunny/Fabric-DE-CICDtrainingF39
CSCfi/anterocommunityF39.1
microsoft/fabric-samplesmicrosoft sampleF41.4
tomatminceddata/PBIR_XRAYcommunityF43.1
microsoft/Analysis-Servicesmicrosoft sampleF43.3
Cyberlorians/nistframeworkcommunityF43.3
MeteoWatch/MeteoWatchcommunityF45.5
microsoft/fabric-racing-simmicrosoft sampleF46.2
Azure/tech-debt-analyticsmicrosoft sampleF46.6
kevchant/GitHub-FUAM-DeploymenatorcommunityF46.9
miguelASL/Eurocopa_EspanacommunityF47.2
Mike-Honey/covid-19-au-vaccinationscommunityF47.6
ecotte/Fabric-Monitoring-RTIcommunityF48.3
bcgov/moh-APO-ReportingcommunityF49.5
ayodejiayodele/github-developer-metricscommunityF51.1
kerski/fabric-dataops-patternstrainingF51.2
Open-Education-AI/OEAIcommunityF51.3
sonbaoharryson/Data_Engineer_JobPulse_ProjectcommunityF52.5
djouallah/aemo_fabriccommunityF53.4
microsoft/PowerBI-LogAnalytics-Template-Reportsmicrosoft sampleF53.6
FHaurum/FHSQLMonitorcommunityF54.1
PacktPublishing/Microsoft-Power-BI-CookbooktrainingF54.9
alisonpezzott/reactor-pbi-maio-25trainingF55.3
aditiv101/Youtube_analytics_dashboardcommunityF56.2
ProdataSQL/FinancialModellingcommunityF56.4
CareTogether/CareTogether-PowerBIcommunityF57.6
PBI-DataVizzle/pbi_contentcommunityF58.6
DataChant/Trello-Power-BIcommunityF59.9
alisonpezzott/pbi-docscommunityD60
jurgenfolz/WorldDataReportcommunityD61.4
NelsonNeba/Workforce-Hiring-Optimization-Dashboard-communityD61.8
stephbruno/Power-BI-Accessibility-CheckercommunityD63.1
vlpatkosdani/powerbi-cicd-with-githubactions-demoscommunityD63.5
jurgenfolz/Stock-IntelligencecommunityD67
RuiRomano/fabric-cli-powerbi-cicd-samplecommunityD68.2
pbi-tools/sales-samplecommunityD69
jeremypj/budget-intelligence-ynabcommunityD69
Rede-DSBR/DocPBI2communityC70.3
jeremypj/Power-BI-for-BigTimecommunityC72.6
data-goblin/power-bi-visual-templatescommunityB81.5

2. Does the Pro referee actually help? (in progress)

The claim behind Semanticus Pro is that an AI assistant wrapped in the engine's verify/probe/benchmark loop writes more-correct DAX than the same assistant retrying on raw error text. We are measuring exactly that, and we pre-registered the method before running it:

Results, raw run logs and full session transcripts will be published on this page.