How to review a new AI tool before people use it

A request arrives on a Tuesday afternoon: the marketing lead has found an AI tool that writes product descriptions and wants the team using it by Friday. You do not have a security team, a procurement department or a lawyer on staff. You have this week and a browser.

That is enough for most tools, if you ask the right questions in the right order. This post gives a review that fits in an hour or two, not a month.

Begin with the need

Before reading any terms, ask what the tool is for and what information will go into it. Writing product descriptions from public specifications is a very different review from summarising customer complaints. The answer decides how deep you need to go.

Also ask whether a tool you have already approved can do the job. Every new tool adds an account to manage and a set of terms to track.

Ten questions

1. What data will people put in, and does any of it include personal data, client material, code or credentials?
2. Which plan will we use? Free, individual paid, team or business plans often come with different terms.
3. Do the terms for that plan say whether inputs are used to train or improve the provider's models, and can that be switched off?
4. How long does the provider keep inputs and outputs, and can we delete them?
5. Where is the data processed and stored, and does the provider offer a data processing agreement?
6. Who at the provider can see our data, and under what circumstances?
7. Can we sign in with company accounts and remove a person's access when they leave?
8. Does the tool connect to other systems, such as email, calendars or file storage, and what can it read there?
9. Does the provider publish security information, such as an independent audit report or a security page?
10. What happens to our data if we stop using the tool or the provider closes?

You will not get a clear answer to every question from public pages. Where the answer is unclear, write that down. Unclear is itself a finding, and it often leads to a restricted decision rather than a refusal.

Who should answer

At a small company, one person usually does the review and one other person agrees the decision. A sensible pairing is the person who will own the tool day to day and whoever looks after IT or data protection. For tools that will handle personal data at scale, or client data under contract, it is worth asking your legal adviser to read the data processing terms.

Making the decision

Most reviews end in one of three places:

• Approved for named kinds of data, because the terms and controls fit what people will do with it.
• Restricted: allowed only for some data, some people or some purposes. For example, public product information only, by the marketing team.
• Not approved, with a sentence on why and, ideally, an approved alternative.

At Saffron Street Foods, a food distributor of 60, the product description tool was approved for public product data on the team plan, and restricted from any customer or supplier pricing. The review took ninety minutes. Writing the decision down took five.

Record it where people will look

A decision that lives in an email thread is lost within a month. Put it in your AI tool register with the status, the allowed data, the owner, the plan, a link to the terms you read, and a review date six or twelve months out. When the review date arrives, check whether anything changed: the terms, the plan, or the way people use the tool. If you have not set up a register yet, the register post explains the columns.

In Openn Shield

In the Openn Shield tool register, a requested tool can sit as under review while you work through the questions, then move to approved or restricted with its allowed data classes, owner and review date. The change from one status to another is recorded with who made it, and the owner is alerted when the next review is due. See how the register works.