Evaluation by professional field, documented as evidence.
betaPlan defines the professional field, prepares and assesses the people who will do the work, where a route to qualification exists, runs the work, keeps independent review apart from that work, and preserves a record a buyer can check.
Why expert judgement in Albanian, from betaPlan
- A low-resource language
- Albanian has little training data and few annotators who know both the language and a professional field. Someone who only knows the language will not correct a model that gets accounting or pharmacy wrong in Albanian.
- Professionals, not a crowd
- The work is done by accountants, pharmacists, lawyers and other professionals qualified in their own field, not by casual workers from platforms where anyone can pick up the work.
- Two assessors, a published rubric
- Two independent assessors decide whether each annotator qualifies, on a work sample and against a rubric published with a version number. No decision is taken by AI and none comes from self-assessment.
- A record for every piece of work
- Every task comes with a record of where it came from: who did it, who reviewed it, under which guideline and with what decision, sealed so you can check it yourself without asking us.
Four kinds of work
In professional fields, work with AI can depend on safety, regulation, source quality and judgement, none of which a generic label captures. So the work begins by defining the task and the professional competence it requires.
Capacity for that field is commissioned only once we have confirmed that it can be provided. No one is assigned just because they took a course, and a qualification does not guarantee assignment. Where qualification is used, the human decision and the rule version are recorded.
Work in regulated professions requires professional review, confirmed separately: for a commissioned task, we confirm that the reviewer has the relevant authority. Working in a public-sector field never implies public-sector authority.
Labelling and structuring
Specialist material is structured to a schema, access rules and an evidence standard, all agreed in advance. The professional field required is agreed before capacity is commissioned.Output evaluation
A reviewer from the relevant field evaluates the AI output against observable criteria and records the reason, the evidence and any escalation, rather than returning an unexplained label.Domain red-teaming
Tests target failures that matter in the domain, including fabricated sources, unsafe recommendations, privacy breaches and missed escalation. Claims in regulated fields require the relevant professional review.Agent workflow review
A step-by-step review finds gaps in the evidence, unsafe hand-offs from one step to the next, and the points where human judgement is needed. Automation is not declared safe without the evidence the field requires.
What a procurement team asks now
- Who did the work, and were they paid properly?
- The project contract sets out the worker category, access rights, pay and task records. Personal identity is disclosed only under the lawful arrangement agreed; the evidence given to the buyer can use pseudonymous identifiers.
- How do you know they were competent?
- The competence required, the assessment route and the state of professional ratification are declared for the project. Completing the learning is, on its own, never evidence of qualification, and critical failures cannot be averaged away.
- What jurisdiction is my data in?
- Jurisdiction, hosting, subprocessors, retention and access controls are confirmed in the project agreement. This public product description does not replace the data-processing review carried out under that agreement.
- Can I check any of that without trusting you?
- Yes. In the delivery model, an approved delivery comes with a sealed record that you verify offline with a published key. The public checker shows this with a synthetic record, not a client delivery.
Request a sample
Describe the professional field, the risk in the decision, the constraints on the data and the evidence you need. We then assess whether an evidence-backed pilot is feasible, and what professional ratification or capacity must be secured first.