Programming in Plain English

The expression “prompt engineering” has always sounded rather grand for what is, in most cases, the business of giving a machine clear instructions. There is nevertheless an important idea underneath the fashionable terminology. For most of the history of computing, anybody wishing to tell a computer what to do had to learn a language designed for the computer. One of the remarkable things about the current generation of AI is that the language of instruction can simply be English.

That does not mean that any English will do. Lawyers should understand this instinctively. A set of instructions which says “look into the limitation point” is not equivalent to one which identifies the issue, supplies the relevant papers, specifies the jurisdiction, explains what question needs answering and tells the recipient what form the answer should take. Giving poor instructions to a human being and receiving poor work in return is regarded as a failure of supervision. Giving poor instructions to a machine and complaining that AI is useless is much the same exercise conducted through a browser.

A useful instruction begins with the task. “Summarise this judgment” may produce a respectable general account. It may not produce the thing counsel actually wants, which is perhaps a short note identifying the ratio, the treatment of a particular authority and anything which can properly be used in oral submissions tomorrow morning. Those are different tasks. The machine cannot know which one matters unless somebody tells it.

The next question is the source material. There is an important difference between asking a system to answer from its general training and asking it to work only from material which has been supplied. If I am using AI to work on a witness statement, contract, judgment or bundle, I increasingly prefer the latter where the task permits it. The instruction can then be simple: work from these materials, identify anything which is missing, and do not improve the evidential record by inventing what ought conveniently to have happened.

The required output matters just as much. A note for a solicitor, a draft skeleton, a chronology, a client explanation and a list of issues are not interchangeable merely because all of them consist of words. The audience, length, purpose and degree of formality should be specified. Lawyers spend a surprising amount of time complaining that machines are verbose after asking questions which place no limit at all on what the machine is supposed to produce.

Constraints are equally useful. If the task concerns English law, say so. If no authority is to be added unless it appears in supplied material, say so. If assumptions are to remain visible, say so. If disputed facts must not be converted into facts merely because one version is easier to write, say so. These instructions do not make the machine incapable of error. They reduce the amount of territory in which it is invited to improvise.

The last stage is checking, and this is the one which no amount of clever prompting removes. A good instruction can reduce hallucination; it cannot abolish it. If an authority matters, open it. If a quotation matters, find it. If the system says that a witness said something, identify where the witness said it. If it performs a calculation, make sure the calculation can be reproduced. Asking the same machine whether its previous answer was correct may produce a warmer feeling, but it is not independent verification.

There is a tendency to collect elaborate prompts as though they were spells. I am sceptical. The best instructions are usually produced by understanding the task rather than by discovering an incantation which happens to work with this month’s model. The technology will change. The disciplines will not. Define the job, control the material, specify the output, identify the limits and check what comes back.

Lawyers have been giving instructions for centuries. We should be able to survive the discovery that computers require them as well.