Defensible consultation reports, not just plausible summaries.
Communiti produces consultation reports with a visible evidence trail: citations that resolve, decision-critical counts that do not flip across runs, and minority voices that remain visible instead of disappearing into summary language.
Headline results
- 0%
- of Communiti citations resolved across the three evidence runs
- 0
- decision-changing flips across repeated Communiti reports
- 0/3
- runs preserved the benchmark's minority viewpoint
- 0
- report modules rendered per Communiti run
Proof point № 1
A report is only defensible when every important claim can be checked.
Consultation teams do not just need a polished summary. They need a report they can take to leadership, legal review, and the community: one where citations resolve, headline counts stay stable across repeated runs, and small but material voices are not lost in broad narrative language. The report-showdown benchmark tests that evidence trail end to end.
Evidence trail demo
From source documents to reviewable report claims.
The benchmark checks the chain that matters in public reporting: documents are ingested, report modules are generated, citations are resolved, and reviewers can trace claims back to source evidence.
- 1 Synthetic source set 18 documents Survey rows, PDFs, transcripts, workshop boards, and image-based feedback.
- 2 Report structure 23 modules Each Communiti run rendered the same report module set for review.
- 3 Citation check 372 resolved Across three runs: 142, 96, and 134 citations, with zero unresolved.
Evidence snapshot
372/372
All Communiti citations resolved across three evidence runs.
Claim 1
ResolvedSurvey responses recorded 53.8% favourable and 34.2% unfavourable support-level scores.
Source evidence
survey_responses.xlsx support_level distribution: 1=304, 2=380, 3=240, 4=646, 5=430.
The headline support count is grounded in the structured survey column, not inferred from narrative tone.
Claim 2
ResolvedA small heritage concern remained visible even though only seven survey rows raised it.
Source evidence
Gold register: heritage_minority_rows=7, including KP-0157, KP-0490, KP-0903, KP-1204, KP-1491, KP-1756, and KP-1969.
Minority-view survival is treated as a decision-safety check, not a nice-to-have reporting detail.
Claim 3
ResolvedEvery Communiti report citation in the evidence snapshot resolved back to source material.
Source evidence
raw_communiti_run1.json, raw_communiti_run2.json, raw_communiti_run3.json: 372 total citations, 372 resolved, 0 unresolved.
A reviewer can test the report instead of trusting a polished narrative on sight.
The example uses synthetic benchmark data only. It is designed to show the audit behavior, not a real community consultation.
372 resolved citations
Three Communiti report runs, 18 synthetic documents, zero unresolved citations
The public claim is simple: every Communiti citation resolved in the evidence snapshot.
0 decision-changing flips
Repeated report runs kept the benchmark's headline support direction stable
A report can vary in wording and still be safe. What cannot vary is which way the evidence points for decision-makers.
Minority viewpoint survived every run
Small heritage concern intentionally placed where shallow summaries can drop it
The benchmark checks whether a small but material viewpoint survives the report instead of being averaged away.
The publication gates that mattered
The page leads with the checks that make a report safer to publish
The strongest story is not that every generic proxy metric is highest. It is that the report clears the checks that make public conclusions reviewable.
At a glance
What changes when a report is built to be audited
| Capability | Manual report Analyst + spreadsheet | Assistant draft General-purpose summary | Communiti Intelligence Evidence-first report |
|---|---|---|---|
| Citation trail | By hand Reviewers paste quotes and maintain source notes separately. | Fragile A polished answer can still leave reviewers chasing where each claim came from. | Resolved 372 of 372 citations resolved in the evidence snapshot. |
| Headline decision direction | Reviewable Stable when reviewers reconcile counts carefully. | Variable Assistant workflows had decision-changing flips in repeated runs. | Stable 0 of 3 Communiti runs flipped the benchmark's headline support direction. |
| Minority voices | Possible Small signals survive when reviewers know to look for them. | At risk Small but material viewpoints can disappear into broad summary language. | Preserved The benchmark's minority heritage concern survived in 3 of 3 runs. |
| Repeat runs | Slow A second report pass means re-reading and re-reconciling. | Drifts Repeated drafts may be plausible but not equivalent. | Checked Repeated evidence runs can be compared against fixed publication gates. |
| Governance review | Labour-heavy IT, procurement, and governance teams need a separate evidence pack. | Thin A chat transcript is not a reporting control. | Packaged Raw outputs, scoring files, charts, and methodology notes are available for review. |
For your technical reviewers
The scores behind the headlines
Headline figures are rounded for readability. These are the underlying benchmark results and technical context behind the public claims on this page.
The technical context
The assistant workflow comparison is included for reviewers, not as the headline claim. Some generic proxy metrics are mixed, so the page leads only with the evidence checks Communiti clearly cleared.
| Metric | Communiti | General-purpose assistant workflows |
|---|---|---|
| Citations resolved Communiti runs captured citation snapshots with zero unresolved citations | 372/372 across three runs | Not structured for automatic citation validation |
| Decision-changing flips Whether the report changes the headline support direction | 0/3 runs | 1/3 to 2/3 runs |
| Minority viewpoint survival Whether the benchmark's small heritage concern survived the report | 3/3 runs | 0/3 to 1/3 runs |
| Claim grounding proxy Secondary judge-scored proxy, not used as a headline claim | 75.1% | 75.8% best observed assistant workflow |
| Evidence density proxy Secondary proxy; the public claim is citation resolution, not raw density | 0.194 | 0.204 best observed assistant workflow |
| Numeric exactness proxy Mixed result; retained here for transparency | 34.0% | 55.0% best observed assistant workflow |
This is why the page is framed around defensibility: resolved citations, stable decision direction, and minority-view survival are the claims the benchmark evidence supports cleanly.
Methodology
How we measured
Test corpus
Synthetic consultation material only - no resident data - spanning 18 documents, three Communiti evidence runs, report modules, raw outputs, citations, charts, and scoring notebooks.
- Source documents
- 18
- Survey responses, written submissions, transcripts, workshop boards, and image-based feedback
- Communiti runs
- 3
- Three report evidence runs captured with raw output and citation snapshots
- Resolved citations
- 372
- 142 + 96 + 134 citations resolved across the three Communiti runs
Processed in Australia
Analysis runs on AWS in Sydney and Melbourne using Australia-geographic infrastructure. Feedback is not processed offshore.
Never used to train AI
Your community's feedback is not used to train any AI model, and the model provider has no access to it - contractually guaranteed by AWS.
Evidence on request
The benchmark pack includes synthetic data, raw outputs, scoring code, charts, and methodology notes for technical review.
Every report claim reviewable
The reporting workflow is designed around reviewer checks: source documents, citations, headline counts, and minority viewpoints all stay available for audit.
The fine print we think you should read
- Test data. The benchmark uses synthetic consultation material written for testing. No resident data was used.
- Evidence source. The public claims are drawn from the report-showdown evidence snapshot generated on 15 June 2026: raw Communiti captures, metrics tables, summary report, consequence ledger, and charts.
- Assistant context. General-purpose assistant workflows were evaluated to reflect how practitioners may draft reports today. They are included as technical context, not as the page's primary comparison claim.
- Metric interpretation. Some generic proxy metrics were mixed. The page therefore leads only with claims the evidence supports cleanly: resolved citations, no decision flips, and minority-view survival.
- Reproducibility. The benchmark pack includes raw outputs, scoring code, generated charts, methodology notes, and cached verification artifacts for technical review.
Check our work
See your own consultation benchmarked this way
Bring one real, de-identified feedback export to a 30-minute walkthrough - or request the benchmark pack and have your technical team verify every number on this page.
