AI wrote the first draft. What's still up to me?

A complete draft isn't the end of the work. Sources, context, and editing time still deserve a closer look.

Suppose AI helps you summarize an article

Imagine a workflow that reads source material, creates a summary, and turns it into a post. Every step could return success while the result still contains inaccurate summaries, unsupported claims, or a draft that needs substantial rewriting.

I'd separate two questions: did the system work as intended, and is the final content useful? This article proposes a method; it is not a report of measured results from an existing project.

What makes a draft usable?

Start with a specific task to make quality easier to judge. For example, summarizing a sourced article gives us something concrete to review using the following criteria.

CriterionReview questionIf it falls short
Faithful to the sourceCan each important claim be traced to the original text?Flag the claim and return it for revision
Context preservedAre any conditions or limitations missing?Restore the relevant context
Useful to the readerCan the reader tell when this applies?Clarify the use case

Track your own editing time

A quick generation step may offer less benefit than expected if every draft takes a long time to edit. Keep the original process as a baseline and include review, revision, and exception handling in the measurements.

Count all of it, from gathering the material to the final edit.

If the draft isn't ready, send it back

A few simple rules make it clear what should happen when a problem appears. The example below illustrates a data structure; it is not a complete, executable review system.

review-policy.js · Structure example
const review = {
  task: "summarize_a_document",
  checks: ["source_accuracy", "missing_context"],
  humanReview: "required",
  onFailure: "return_to_draft"
};

Returning a piece to draft, keeping its sources, and allowing human edits is often easier to understand and manage than unlimited retries. The right approach still depends on the task and its risks.

A place to start

  1. Choose a recurring task with a clear scope.
  2. Prepare comparable inputs and document the original process.
  3. Write down the criteria for an acceptable result.
  4. Record human edits and how failures are handled.
  5. Use the results to decide whether to keep, change, or stop the workflow.
Read next: how I'd test local AI ← Back to all notes