Skip to main content
NS
AI21 Nov 2026·By Nazmun Sakib·5 min read

LLM API patterns for business workflows: validation, retries, logging and approval

Calling a model is the easy part. This guide covers the engineering patterns that make an LLM step dependable inside a real business workflow.

Getting a model to answer a question takes an afternoon. Getting it to behave inside a business process, every day, on inputs nobody planned for, is an engineering task. The model is one component. The reliability comes from what surrounds it. These are the patterns I use when connecting an LLM API to a real workflow.

Treat the output as untrusted input

A model can produce something fluent and wrong, in a format you did not ask for. Handle its output the way you would handle data from a stranger.

  • Ask for a structured response, such as JSON with named fields, instead of free text you have to parse.
  • Validate it against a schema: required fields, allowed values, length limits.
  • If validation fails, retry with the error message, then fall back to a safe default or a human.
  • Never run an action just because the model said to. Check that the action is allowed for that user and context.

Keep prompts and logic separate and versioned

A prompt is part of your code. Store it in version control, give each change a version and record which version produced each result. Keep business rules in code, not buried in the prompt, so they can be tested and cannot be talked around by a clever input.

Design for failure and slowness

Model APIs time out, rate-limit and occasionally return errors.

  • Set timeouts so one slow call does not hang the workflow.
  • Retry temporary failures with growing delays and a cap on attempts.
  • Run model calls in background jobs, not while a visitor waits, unless the interaction demands it.
  • Make the step idempotent so a retry cannot send two emails or create two records.
  • Define what happens when the call fails for good, such as routing to a person with context.

Put a human in the loop where it counts

The right amount of automation depends on the cost of a mistake.

  • Low stakes and easy to undo, such as tagging, summarising or drafting for internal use: automate.
  • Customer-facing or hard to reverse, such as sending replies, issuing refunds or changing prices: draft and queue for approval.
  • Start with approval for everything, measure how often people accept the output unchanged and relax the approvals only for the categories that have earned it.

An approval queue also gives you a stream of corrections to learn from.

Give the model only what it needs

Less context is cheaper, faster and safer.

  • Send the fields the task requires, not whole records.
  • Remove or mask personal data that is not needed for the task.
  • Keep secrets, tokens and internal notes out of prompts.
  • Be aware of prompt injection: text from emails, web pages or users can contain instructions, so never let untrusted text decide what actions the system takes.

Test with real examples

Build a set of real inputs, including awkward and adversarial ones, with the outputs you consider acceptable. Run it whenever you change the prompt, the model or the logic. This is how you notice that an improvement for one case broke another. Pin the model version where the provider allows, and re-test when you move to a newer one.

Log enough to explain what happened

When something goes wrong, you need to answer why. Record the input, the prompt version, the model, the raw output, the validation result and the action taken, with a reference that ties it to the business record. Mask personal data you do not need to keep and set a retention period. Good logs turn a vague complaint into something you can trace in minutes.

Control cost and usage

  • Cap input and output length.
  • Cache results for repeated or identical requests.
  • Use a smaller model for simple tasks and keep the larger one for hard ones.
  • Set budgets and alerts so a loop or a spike cannot produce a surprise bill.
  • Track cost per workflow run so you know whether the automation is worth it.

Plan for change

Models and APIs change, get deprecated and behave differently between versions. Keep the integration behind a small interface of your own so you can swap or upgrade the model without rewriting the workflow, and monitor quality over time instead of assuming it stays constant.

How I approach it

I build automation where the model is one step in a reliable system: validated outputs, queued processing, retries, logs and approvals where they matter, connected to the CRM, email and store. If you have a repetitive process you suspect could be automated, describe it and I can say what is realistic and where a person should stay involved.

Frequently asked questions

How do I make an LLM step reliable in a workflow?

Treat its output as untrusted input: ask for a structured format, validate it against a schema, retry or fall back when it fails and keep a person in the loop for actions with real consequences.

Should AI-generated actions be approved by a human?

For anything costly, public or hard to reverse, yes. Start with approval for every action and relax it only for categories where the output has been reliable and mistakes are cheap to fix.

How do I control the cost of LLM API calls?

Limit input and output length, avoid sending unnecessary context, cache repeated requests, choose a model sized to the task and set budgets and alerts on usage.

What should I log when using an LLM in a workflow?

Log the request, the model and prompt version, the response, the validation result and what the workflow did with it, while masking personal data you do not need to keep.

ShareX / TwitterLinkedIn