← Steinbach City Council Reporting · Council deliberation report

1. Purpose and limits

Assess observable public deliberation: whether councillors state a position, explain it, independently evaluate relevant considerations, engage opposing arguments, and remain open to revision. The instrument does not measure whether a policy choice is correct, whether a councillor is personally courageous, or whether a town is successful. Those require separate evidence.

2. Established foundations and Canadian fit

Canadian governance reference. The Canadian Association of Municipal Administrators (CAMA) Council Orientation Toolkit (June 2026), Appendix C, provides a Council Governance Self-Check addressing collective leadership, procedural fairness, respectful disagreement, and decisions based on information and process. It is explicitly designed for collective council reflection, not individual councillor evaluation. Therefore, use it to guide council-level contextual assessment, not as a validated individual rating scale [1].

Deliberation research. Steenbergen et al. (2003) developed the Discourse Quality Index (DQI), an observational coding approach for public discourse. Its categories include justification, respect, common-good orientation, and constructive politics. The present instrument adapts these concepts rather than claiming to reproduce the original DQI [2, 3].

Canadian public engagement reference. IAP2 Canada describes standards for transparency, public input, and explaining how public contributions affect decisions. These are useful contextual indicators, not a substitute for individual transcript coding [4].

3. Unit of analysis and sampling

Primary unit: councillor-by-substantive-agenda-item. Within each item, code the councillor's complete relevant public contribution(s), including questions and replies. Record each distinct speech act and evidence excerpt; aggregate to one item-level score without rewarding repetition or speaking length.

Pre-register a sampling period (e.g., 12 months), include all regular and special open council meetings with usable records, and classify substantive items using a written rule: discretionary policy, budget, land use, infrastructure, service levels, or major governance decisions. Exclude purely ceremonial and routine procedural items. Log exclusions, recusals, absences, closed-session limitations, and missing audio. Minutes-only analysis is lower confidence because minutes may omit speech.

4. Proposed observational rubric

Score each dimension 0–4 when there is a genuine opportunity to observe it. N/A means no fair opportunity or insufficient source material; it is not a zero. Weights are provisional and require pilot validation.

Intermediate anchors: 1 = minimal/unsupported demonstration; 3 = clear, substantive demonstration short of the 4 anchor. Do not infer private motives. Agreement with a majority is not evidence of dependence; disagreement is not automatically a virtue. For dimensions requiring a challenge or new evidence, mark N/A when no such opportunity exists.

5. Two required headline measures

Public reasoned-position rate: number of eligible councillor-item opportunities with an identifiable public stance and at least one substantive reason / total eligible councillor-item opportunities. Report separately the position-only rate and the reasons-conditional-on-position rate. For abstentions, recusals, or absence, follow pre-registered denominator rules and publish counts.

Deliberation quality: for each scored item, 100 × [sum(score/4 × applicable weight)] / [sum(applicable weights)]. Report mean and median across items, the number scored, dimension coverage, and percentage N/A. Do not rank councillors with sparse comparable observations.

6. Evidence, coding and safeguards

Capture municipality, meeting date, agenda identifier, item type, councillor, exact quotation, recording timestamp/page, stated position, final vote, each dimension score, N/A reason, confidence level, and coder ID. Quote evidence for each nonzero rating; use full recordings when available to check context. Separate transcript errors from genuine ambiguity. Record when an argument is made by staff rather than a councillor.

Use AI to segment transcripts, extract passages and suggest provisional codes; require a human reviewer for disputed, low-confidence, or publication-bound ratings. Never invent missing statements or infer beliefs from voting alone. Protect personal information and observe applicable municipal records and privacy rules.

7. Pilot and validation plan

Pilot on 10–20 meetings across multiple agenda types and at least two municipalities where feasible. Train two coders on a common codebook; independently double-code at least 20–30% of councillor-items. Examine weighted Cohen's kappa or Krippendorff's alpha for ordinal scores, alongside raw agreement and disagreement patterns. Revise anchors and re-test; do not assert reliability before measuring it. Check whether transcript completeness, meeting chairing, and issue complexity systematically change scores.

For external validity, compare aggregated council-level indicators with pre-specified municipal outcomes over multiple years (e.g., financial resilience, infrastructure condition, service performance). Control for population, fiscal capacity, provincial transfers and other confounders. Treat associations as non-causal absent stronger research design.

8. Reporting rules

Publish separate indicators before any composite score. Show source coverage, denominator, uncertainty, issue mix, and at least one representative quotation. Do not label individuals as “good” or “bad” councillors. Describe findings as recorded public deliberative behaviour, not character or competence. Allow factual corrections and independent audit.

9. Reusable coding record

References and provenance

1. CAMA, Council Orientation Toolkit (June 2026), Appendix C. camacam.ca
2. Steenbergen, M. R., Bächtiger, A., Spörndli, M., & Steiner, J. (2003). Measuring Political Deliberation: A Discourse Quality Index. Comparative European Politics, 1, 21–48. DOI: 10.1057/palgrave.cep.6110002
3. Research Methods in Deliberative Democracy, chapter “Discourse Quality Index”. academic.oup.com
4. IAP2 Canada, “Pillars of P2”. iap2canada.ca (copyright-protected framework; concepts summarized, not reproduced).