Organisation type

Civil Society Operations

The problem

The tool writes the donor report fast and well, with a sentence on beneficiaries reached across several municipalities that the activity register does not show. The same number differs in the narrative, the table and the financial report, and the donor notices.

27 courses · 2 specific to this profession

A process from your work

How long does it take you to write the report for a donor?

Step 1From the monitoring data, the tool drafts the report for the donor.
Step 2You compare every reported result with the indicator and the project's evidence.
Step 3The organisation signs what is reported to the donor.
Where the tool goes wrongA change seen at some hearings is declared an achieved outcome, caused by the project.
The verification stepNext to each outcome, write how many sites showed the change, out of how many.

10 minutes less on a 27-minute writing task, in a randomised experiment. Source: MIT, working paper (later published in Science), March 2023.

More: what was measured and the full check

What changed, as measured

Time on the writing task dropped by 10 minutes, 37%, against a comparison group that took 27 minutes on average.

Who was measured: College-educated professionals, among them grant writers, randomised to write texts typical of their own occupation, with or without ChatGPT.

The caveat: Short, self-contained tasks, not full reports; grant writers were one of several occupations measured together.

The check, step by step

For every outcome the draft declares "achieved", open the indicator table and write next to the sentence at how many sites the change was observed, out of how many in total.
Mark the sites with no data (no attendance lists, no questionnaires) separately and do not let them be counted as success.
Check the verbs: "increased", "improved", "ensured" claim that the project caused the change. Without a comparison with the baseline year or with sites outside the project, write "was observed", not "the project increased".
Before sending, compare every figure in the narrative with the indicator table and with the source documents the donor may ask for in verification.

The course covers proposals, monitoring, donor reports and beneficiary data, with AI used for drafts and summaries, never to invent results.

We can help you with this process: “Civil society AI: Programme evidence and reporting”, 8 hours.

See the course A workshop for your team

A documented failure pattern

Two hearings out of five, reported as an outcome achieved

A paragraph of the narrative report to the donor, drafted by an AI tool from the monitoring table for the programme officer of a civil society organisation.

Source: Royal Society Open Science, 2025, and 1 more.

Why it is easy to miss

A donor who checks the report against the attendance lists finds a reported outcome that the records do not support. The tool has dropped the three limits that define the result: only 2 of 5 hearings, 3 with no attendance list at all, and no comparison showing that the change comes from the project. The sentence moves from the part to the whole and from "it happened" to "the project caused it", in the usual wording of a logical framework. A reader without the indicator table has nothing to stumble on, because the facts themselves are true: the hearings took place and at two of them more women came.

Where the failure pattern is documented: A peer-reviewed study in Royal Society Open Science on overgeneralisation in language-model summaries (April 2025); guidance from the Charity Commission for England and Wales on the use of AI (April 2024).Sources: Royal Society Open Science, 2025; The Charity Commission for England and Wales, 2024.The case above is ours, written for work in Albania; the laws it names were checked against the official text.

What it looks like

The monitoring dataPublic hearings supported: 5. At 2 of them more women attended than at the hearings of the baseline year. For the other 3 there are no attendance lists.
The AI tool's draftOutcome 1 achieved: the project increased women's participation in the public hearings it supported.

Translated from the Albanian original.

Courses for this type of organisation

What you leave with

Before the report goes to the donor, the evidence pack links every claim to the activity register, the results evidence or the budget, shows that each number matches in the narrative, the table and the financial report, and explains every gap.