Label QA Calculator
Inter-annotator agreement, major-error rate and the sample size you actually need — before a client asks.
Statistical measures computed from counts you enter. It is not a certification, an audit opinion, or a guarantee of client acceptance.
The thresholds we publish
Below this, the guideline is ambiguous — fix the guideline before blaming the annotators.
An error a client would reject the batch for.
Safety, personal data, or policy failures. One is a stop-and-fix, every time.
Log a batch
Record the audit you actually did, not the one you wish you had done. Small samples are fine — the point is to know what they can and cannot tell you.
Rolling QA position
Loading your local batches…
How much should you audit?
The honest answer is a number, not a percentage. Set the precision you need and this works out the sample size for a proportion, corrected for the size of each batch.
For a large population this is 139 tasks (z = 1.96, p = 10.0%, ±5.0pp). Small batches need fewer, which is what the correction below works out:
Log a batch above and its required sample will appear here.
Percent agreement on a small sample is a weak signal: at 10 audited tasks, one disagreement moves the number ten points. That is why the sample size is shown next to the score, not hidden behind it.
Your data, your call
Batch numbers stay in this browser. Export a CSV for your own QA records; nothing is transmitted to Mokoro or to any client.
How we handle data
- Local-first: every tool writes to your own browser storage. No account is required to use it, and no entry is uploaded by default.
- Export is a first-class button, not a support ticket: JSON for re-import, CSV for a spreadsheet.
- Delete means delete: one button clears the tool’s store on this device, immediately, with no shadow copy on any server.
- If you later switch on cloud sync, data is encrypted in transit and at rest, and you can withdraw and erase it from the same screen.
- We follow the Botswana Data Protection Act, South Africa POPIA, and GDPR for EU data subjects. No sale of personal data, ever.