Name the task precisely
“Use AI in marketing” is too broad to evaluate. “Draft a first response from an approved knowledge base for a person to review” gives the team something concrete to test.
Describe the current input, the work performed, the output and the person responsible. Record the difficult exceptions as well as the typical cases. This often reveals that part of the problem is an unclear process rather than a missing tool.
Describe the current input, the work performed, the output and the person responsible. Record the difficult exceptions as well as the typical cases. This often reveals that part of the problem is an unclear process rather than a missing tool.
Establish a baseline
Before building, agree what improvement would matter. It might be less repetitive preparation, faster retrieval or a more consistent draft. Keep quality and review effort in the picture: a quick output that takes longer to correct is not an obvious improvement.
Use representative examples when evaluating the prototype. Include incomplete inputs and edge cases, not only the clean demonstrations that make a tool look convincing.
Use representative examples when evaluating the prototype. Include incomplete inputs and edge cases, not only the clean demonstrations that make a tool look convincing.
Put the boundaries in the design
Decide which information the system may access, what it may produce and what it must leave to a person. Review data handling with the people responsible for it. Access, retention and vendor choices should be deliberate.
Define a fallback for unavailable services, low confidence or an output that fails a check. A workflow that depends on a perfect response has not yet accounted for everyday operation.
Define a fallback for unavailable services, low confidence or an output that fails a check. A workflow that depends on a perfect response has not yet accounted for everyday operation.
Earn the next stage
Compare the prototype with the baseline and include the time spent reviewing it. Expand only when the evidence supports the next use case. Keep the implementation small enough to revise.
The first useful AI project should teach the business where the technology helps, where a simpler automation works and where human judgement remains essential.
The first useful AI project should teach the business where the technology helps, where a simpler automation works and where human judgement remains essential.
A better question is often the first useful deliverable.
All field notes