Canonset: The datasheet that comes with every delivery

Every delivery comes with a datasheet: a Markdown document that says what the data is, where it came from, how it was made and checked, its quality figures, the rights in it and how long we keep it. It is generated at the moment you ask for it, from the delivery as it stands then.

Fetching it

• In your client account, Datasheet sits beside the exports on the project page.
• By API, GET /api/v1/projects/{id}/datasheet returns it as text/markdown.
• Every export response, in the app and by API, carries a Link header with rel="describedby" pointing at it.

The figures are the quality report's at the moment of the request, so a datasheet fetched after new approvals, a return or a replacement reflects them.

What it contains

The heading names the project and the date it was generated; then six sections.

What it is. The task type and field, with a one-line summary of the work; how many approved answers were delivered, on how many tasks, and how many experts answer each task; and the format: JSONL, one approved answer a line, with preference pairs for preference projects and a Harbor package for coding-task projects. It says that known-answer tasks, rejected answers and answers still in review aren't included.

Where it came from. Who uploaded the tasks and when. How many experts wrote the answers, all living and working in the United States, under per-project pseudonyms; that each passed a screening test and a person's review, and verified their ID, before paid work; how many of them accepted the Canonset expert agreement (their own work only, no AI tools unless the project allows one, confidentiality, and assignment of their rights); and whether any AI model read the answers, where you allowed a provider.

How it was checked. That a person reviewed every answer before delivery: an expert reviewer in the same field, never their own work, or our team. How many known-answer tasks were mixed in unannounced to measure accuracy, or that none were. For coding tasks, that every delivered answer's tests ran in an isolated sandbox with no network, failing on the starting code and passing on the reference solution twice. That submissions were flagged for reviewers when unusually fast or short, mostly pasted, close to another expert's answer, or written in an AI style.

Quality figures. A table: the average review score out of 5; accuracy on known-answer checks, with their number; agreement between experts; the shares returned for changes at least once, rejected, flagged for a reviewer and, for coding tasks, verified by the sandbox; the median time on a task; and the median time from submission to decision. A figure that can't be measured yet says "not measured yet".

Rights and licences. That the experts assigned all their rights in the work to Canonset (expert agreement 4.1) and waived moral rights (4.2), and that the deliverables become yours as your agreement with Canonset says. That no copyleft material is allowed in answers, and that where a task allows outside material the expert names its source and licence in the answer's notes. What the upload check found in the tasks (email addresses, phone numbers, Social Security or card numbers), or that it found none, and the date your team confirmed any was made up or covered by a data processing agreement.

Retention. That we keep the project's data until 30 days after it ends, or 7 days after you ask us to delete it, and that backups age out 7 days later.

See Export formats and The Canonset API.

Questions: support@canonset.com.