When AI Turns Back into API...

Rian Schmidt

October 17, 2023

Table of Contents:

Prompt "engineering"

Output format

So, problem solved, right?

Everything old is new again

If you want to interface AI with... well, much of anything, you need to know what it's going to say back to you.

Prompt "engineering"

You'll hear a lot about prompt engineering (which, I'm sure takes just as much training as learning fast fourier transforms and the like, respect), but what it amounts to is asking a naive genius space alien a question that his stunted emotional state can support.

You can't just say "write me a job description". Well, you can, but if you want anything specific, you need to say "You're a friendly recruiter. Write a job description for a position that's remote, but works in Pacific timezone. The position requires... A degree is preferred but not required. Refer to the hiring company as 'we'. The output should be formatted in Markdown." etc.

Output format

Even so, you'll find yourself getting a variety of things back. The format will change from request to request, depending on the mood of the synthetic brain in a jar. If you're trying to DO something with that information other than just echo it back out, that can be difficult.

Recently I discovered that ChatGPT understands the https://pydantic.dev/ format. Rather, our robot overlord told me that "yeah, I know about that, but I won't execute code" but if I were to supply the definition of the Pydantic object (basically, just a data validation schema), it'd be generous enough to adhere to it.

So, problem solved, right?

Not always. Mostly. Such is the nature of these things. You write your question as nicely as possible on a scrap of silk, pass it through the opening, and hope that the machine doesn't eject it tied to a rock at your forehead. Mostly, it won't. But sometimes. It's better than trying to talk about your feelings about how you would really like the response, but it's definitely not perfect.

Everything old is new again

In a way, what you have there is... an API, but in this case, it's one that YOU define. "Hey, I'm going to ask you a question. When you answer, it should look like this." That's kind of turning that paradigm on its head a bit. Normally, the API tells you "you can ask this question, and this is how I'm going to answer." I'm starting to see the future as some standard that merges those ideas. Submit a schema of some sort before you ask anything, and the A(P)I will reliably answer you in that format.

That would be a huge improvement in efficiency in having to build every possible guardrail in anticipation of Gort deciding to answer you in iambic pentameter every third time.