Class: AIGateway
Defined in: AIGateway.ts:157
AIGateway -- the facade of the gateway and its composition root
About
One object the application talks to; every child of the gateway behind it.
Creates every child once from IAIGatewayConfig (through createGatewayDeps), injects the shared instances, and runs a
call as a sequence of child calls: no prompt text, no token math, no SDK of its own.
- Three calls:
ask(one grounded answer, no tools)run(an agent run with tools, may pause for a human),resume(continues a paused run with the human's decisions). The three return at once with anIGatewayCall:eventsto stream,responseto await (D38).
- Thread, file and memory management for the UI (
listThreads,getTurns,uploadFile,getUserFacts, …), the erasure of a user (deleteUserData,anonymizeUserData) and of conversations (delete), the scheduled jobs (expirePausedRuns,recoverStuckRuns) and a gracefulshutdown. - The admin surfaces as getters (
modelRegistry,aiProfileStore,promptTemplateStore,toolRegistry,trace,usageGuard) and the second root,kbIngestor, for the knowledge-base write endpoints.
The host application constructs one per process and keeps it: the children hold caches, the background work and the calls in flight.
Who needs it
The host application: its HTTP handlers for the chat, the approval screens and the admin pages; its scheduled jobs; its SIGTERM handler. Nothing inside the gateway calls the facade.
What needs
IAIGatewayConfig: one config per child, with the repositories and adapters the host implements (or the in-memory ones,
for tests and the first run). Optionally IAIGatewayOverrides: a factory per child the host replaces (tests: a fake
llmClient).
Usage
When to use it
ask for a question an answer settles: a summary, a classification, a grounded reply over the knowledge base, a
structured extraction (outputSchema). run when the model may act through tools, pause for an approval or a question
to the user, and the answer may be reviewed. resume from the request that carries the human's decision, often from
another user than the asker.
When shouldn't use it
Not for writing documents into the knowledge base: that is kbIngestor. Not from inside a tool: a tool receives the
user context and the thread id and talks to the application's own services.
Pitfalls
One call at a time on a thread (D77): a second ask / run on a thread with a call in flight is a 409; keep every call's
timeoutMs at or under IThreadStoreConfig.leaseMs. response resolves FAILED for what the call could not do (a
rejected prompt, a blocked answer, a refused model call, an abort, a timeout, D90) and rejects for what the caller got
wrong (an unknown profile or thread, another user's thread, a thread that waits for a human) or a provider failure.
Call shutdown before the process ends, or whenIdle in tests: the memory updates run in the background after an answer.
Constructors
Constructor
new AIGateway(
config,overrides?):AIGateway
Defined in: AIGateway.ts:164
Parameters
config
overrides?
IAIGatewayOverrides = {}
Factories for the deps the host builds itself; each gets its dep's config and deps (tests: {llmClient: () => fake}).
Returns
AIGateway
Accessors
aiProfileStore
Get Signature
get aiProfileStore():
IAIProfileStore
Defined in: AIGateway.ts:1985
aiProfileStore -- the admin surface of the profiles
About
Create, change and read the profiles of a tenant.
Raw ids, no user check: the host checks the owner permission before it calls.
Who needs it
The host's admin pages and seed scripts.
Returns
kbIngestor
Get Signature
get kbIngestor():
IKbIngestor
Defined in: AIGateway.ts:1951
kbIngestor -- the second root: the write side of the knowledge base
About
Documents go in here; the calls read them through
askandrun.
Who needs it
The host's knowledge-base write endpoints and import jobs.
Returns
modelRegistry
Get Signature
get modelRegistry():
IModelRegistry
Defined in: AIGateway.ts:1968
modelRegistry -- the admin surface of the models
About
Which models exist, with their caps and prices.
Raw ids, no user check: the host checks the owner permission before it calls, as for every admin getter.
Who needs it
The host's admin pages and seed scripts.
Returns
promptTemplateStore
Get Signature
get promptTemplateStore():
IPromptTemplateStore
Defined in: AIGateway.ts:2002
promptTemplateStore -- the admin surface of the prompt templates
About
The versions of every template: read, add one, set the active one.
Raw ids, no user check: the host checks the owner permission before it calls.
Who needs it
The host's prompt editor and seed scripts.
Returns
toolRegistry
Get Signature
get toolRegistry():
IToolRegistry
Defined in: AIGateway.ts:2017
toolRegistry -- the admin surface of the tools
About
The tools as registered, the built-in ones included, for a page that lists them.
Who needs it
The host's admin pages; a profile editor that offers the tool names for allowedTools.
Returns
trace
Get Signature
get trace():
ITrace
Defined in: AIGateway.ts:2035
trace -- the admin surface of the model calls
About
Every model call of the tenant, for the audit page and for debugging.
Raw ids, no user check: it holds system prompts and tool outputs, so the host checks the owner permission before it calls.
Who needs it
The host's audit and debugging pages.
Returns
usageGuard
Get Signature
get usageGuard():
IUsageGuard
Defined in: AIGateway.ts:2050
usageGuard -- the gateway's side of the usage adapter
About
The checks and the reports the gateway's model calls go through, over the adapter the host passed.
Who needs it
A host that checks or reports usage outside a call, with the gateway's own logic.
Returns
Methods
anonymizeUserData()
anonymizeUserData(
__namedParameters):Promise<IForgetUserResult>
Defined in: AIGateway.ts:1389
anonymizeUserData -- "forget this user" by pseudonymisation
About
The person goes, the numbers stay.
The threads, runs and trace entries stay as skeletons for the statistics and the audit -- status, counts, usage,
cost, dates -- under one random pseudonym @anon-<uuid>, the same on every row of this call (D54). What identifies
the person goes, as in deleteUserData: files, turns, remembered facts, prompts, answers, system prompts. An open run
is CANCELLED. Returns the counts per place and the pseudonym.
- Waits for the user's own background work first, so nothing is written after the erasure.
- While it runs, the user's new calls (
ask,run,resume,uploadFile,upsertUserFact) are a 409, and it waits for the calls of the user already running (D114).
Who needs it
The application, on a data-subject erasure request (GDPR) where the books must keep the usage and the cost. Raw ids, no user check: an admin surface, the host decides who may call it.
Usage
When to use it
The person must become unidentifiable but the tenant's statistics, billing and audit trail must keep their rows.
When shouldn't use it
deleteUserData when nothing has to stay; delete when only some conversations go and the user stays.
Pitfalls
No way back: the mapping user -> pseudonym exists only in the returned value. Keep it only where the law asks for it.
A second call for the same user finds nothing: the rows already carry a pseudonym. "Busy" is the thread's lease, the
lock ask / run / resume hold for the duration of a call (a PAUSED run holds none: its thread is kept as a skeleton
and the run cancelled). A thread of the user with a live lease (a call on another instance of the gateway) is refused with a dynaError of status 409, code 5377210486241,
and data.busyThreadIds naming the threads, before anything is touched, so the user's rows never end up under two
pseudonyms: the host retries when the call ended.
Parameters
__namedParameters
tenantId
string
userId
string
Returns
Promise<IForgetUserResult>
ask()
ask<
TStructured>(input):IGatewayCall<TStructured>
Defined in: AIGateway.ts:225
ask -- one grounded answer, no tools
About
A question in, an answer out, streamed while it is written.
Returns at once (D38): events streams the answer, response is the final ANSWER or FAILED. The call screens the
prompt (guardrails), stores the user turn, searches the knowledge base within kbFilters, builds the request with the
thread's memory and the user's facts, calls the model once, checks the answer, stores it and annotates the trace; the
memory updates follow in the background.
responseis FAILED when the prompt is rejected, the answer does not pass the checks (a failed answer is not stored), the usage adapter refuses the model call, orabortSignal/timeoutMsstop the call (D90). Nothing is thrown for those.responserejects with thedynaErrorof a child for what the caller got wrong or the provider broke: an unknown profile, thread or model, another user's thread (404), a thread that waits for a human or has a call in flight (409), a provider failure.
The path: makeCallSignal → getOwnThread → the thread's lease (withThreadLease) → prepareCall →
LLMClient.complete → checkOutput → the assistant turn → the response → scheduleMemory.
Who needs it
The host's chat endpoint for a conversation without tools, and every feature that needs one answer from the model over
the knowledge base: a summary button, an extraction with outputSchema, a classification.
What needs
A profile the tenant has (profileId, required, D103), a kbFilters scope (an empty libraryIds = no search), and for
files the pointers uploadFile returned to the calling user.
Usage
When to use it
The answer is the whole job: nothing has to be looked up through a tool, nobody has to approve anything.
When shouldn't use it
run when the model should act (tools), when a human may have to approve a step or review the answer, or when
"remember this" must be acted on at once: ask has no tools, so the user memory grows only from the background
extraction (after the first answer, then every extractEveryTurns turns).
Pitfalls
Read events before the first await when you stream (const {events, response} = gateway.ask(input)): the stream is
collected from its first access. The user turn is stored even when the answer fails. The thread is locked for the
duration of the call (D77).
Type Parameters
TStructured
TStructured = unknown
Parameters
input
IGatewayAskInput<TStructured>
Returns
IGatewayCall<TStructured>
cancelRun()
cancelRun(
__namedParameters):Promise<void>
Defined in: AIGateway.ts:1119
cancelRun -- drops a paused run without a decision
About
The run becomes CANCELLED, its pending tool call is closed as rejected in the thread, the thread is open again.
HumanGate.cancel marks the run, closeRun does the thread work ("Cancelled by the user"). A run that does not exist
is the RunStore's 404.
Who needs it
The host's "dismiss" action next to an approval card or a question. Open to every user of the tenant, like resume
(D32): the run id is the key, a random UUID nobody can guess (D115).
Parameters
__namedParameters
runId
string
userContext
Returns
Promise<void>
clearUserFacts()
clearUserFacts(
__namedParameters):Promise<void>
Defined in: AIGateway.ts:1638
clearUserFacts -- forgets every fact about the calling user
About
"Forget what you know about me": the facts only, threads and files stay.
Parameters
__namedParameters
userContext
Returns
Promise<void>
delete()
delete(
args):Promise<IDeleteResult>
Defined in: AIGateway.ts:1443
delete -- removes threads with everything they left behind
About
The threads of a tenant, of one user, or a chosen list, gone from every store in one call.
Thread, turns, working memory, files, runs and trace entries of every matched thread are deleted; the count of threads
comes back. A thread with a call in flight (its lease is live: the lock ask / run / resume hold for the duration of
a call; a PAUSED run holds none and goes with its thread) is skipped and not counted -- no error; an unknown id, or one of
another tenant or user, is ignored. The background memory updates of the owners are awaited first, so nothing is
written after the delete. User facts are not touched: they belong to the user, not to a thread.
Who needs it
The application: the user's "delete conversation(s)", a tenant's retention job, a test's teardown. Raw ids, no user check: an admin surface, the host decides who may call it.
Usage
When to use it
{tenantId, threadIds: ["t-1", "t-2"], userId} for the user's own delete; {tenantId, threadIds: "all"} for a tenant's
reset; {tenantId, threadIds: "all", userId} to wipe one user's conversations while keeping the user.
When shouldn't use it
"Forget this user" is deleteUserData or anonymizeUserData: they also remove the files and facts of the user.
Pitfalls
Hard delete, no way back. A second call with the same ids returns 0.
Parameters
args
Returns
Promise<IDeleteResult>
deleteFile()
deleteFile(
__namedParameters):Promise<void>
Defined in: AIGateway.ts:1206
deleteFile -- removes one uploaded file of the calling user
About
For a file that is not referenced yet, or no longer needed.
Another user's file is not touched.
Who needs it
The host's "remove attachment" before the message is sent.
Parameters
__namedParameters
fileId
string
userContext
Returns
Promise<void>
deleteUserData()
deleteUserData(
__namedParameters):Promise<IForgetUserResult>
Defined in: AIGateway.ts:1328
deleteUserData -- "forget this user" by erasure
About
Everything the gateway holds about one person in a tenant, gone from every store in one call.
Deletes the files the user uploaded, the facts remembered about them, their threads with every turn and working
memory, their runs and their trace entries (D54). What the app's usage adapter stored is the app's to erase. Returns the counts per
place; pseudonym is null.
- Waits for the user's own background work first, so nothing is written after the erasure.
- While it runs, the user's new calls (
ask,run,resume,uploadFile,upsertUserFact) are a 409, and it waits for the calls of the user already running (D114).
Who needs it
The application, on a data-subject erasure request (GDPR). Raw ids, no user check: an admin surface, the host decides who may call it.
Usage
When to use it
The law or the user asks that the person's data is removed and nothing has to stay for statistics or audit.
When shouldn't use it
anonymizeUserData when the counts, usage and cost must stay for the books; delete when only some conversations go
and the user stays.
Pitfalls
Hard delete, no way back. A second call finds nothing and returns zeros. "Busy" is the thread's lease, the lock ask /
run / resume hold for the duration of a call (a PAUSED run holds none: its thread is erased and the run with it).
A thread of the user with a live lease (a call on another instance of the gateway) is refused with a dynaError of status 409, code 5377210486241, and
data.busyThreadIds naming the threads, before anything is erased: the host retries when the call ended -- the gateway
waits only for the calls of this instance.
Parameters
__namedParameters
tenantId
string
userId
string
Returns
Promise<IForgetUserResult>
expirePausedRuns()
expirePausedRuns(
__namedParameters):Promise<number>
Defined in: AIGateway.ts:1774
expirePausedRuns -- the scheduled job's entry: closes the pauses nobody answered in time
About
Each expired pause ends like a cancelled one: run CANCELLED, its pending tool call closed as rejected in the thread, the thread open again. Returns how many were closed; a thread that could not be closed is logged and skipped.
HumanGate.expirePaused picks and cancels the due runs (D75); closeRun does the thread work for each.
Who needs it
A scheduled job of the application, once per tenant and interval; the interval decides how long a thread stays blocked
past the pause's expiresAt.
Parameters
__namedParameters
tenantId
string
Returns
Promise<number>
getFileUrl()
getFileUrl(
__namedParameters):Promise<string>
Defined in: AIGateway.ts:1236
getFileUrl -- a short-lived URL of one uploaded file of the calling user, for a screen
About
The browser fetches the bytes from the storage; they never pass through the server.
The adapter signs a URL for the file, valid for ttlSec seconds (default 900). The file must be the calling user's:
another user's, or an unknown id, is the adapter's 404.
Who needs it
The host's chat screen, to show an uploaded image or open a PDF; the files.toUrl hook of the Vercel UI module.
Parameters
__namedParameters
fileId
string
ttlSec?
number
How long the URL stays valid, in seconds.
Default
900
userContext
Returns
Promise<string>
getPausedRun()
getPausedRun(
__namedParameters):Promise<IRun>
Defined in: AIGateway.ts:1925
getPausedRun -- the stored run, for the reviewer's or the approver's screen
About
The whole pause: the draft answer of a review, the tool calls, the request the run was built with.
Raw ids, no user check: an admin surface like trace, the host decides who may read it (a reviewer of the tenant). The
envelopes carry the draft answer (IResultReviewEnvelope.result), and the run its system prompt and messages.
Who needs it
The reviewer's UI of a profile with reviewAnswers (D99): the asker's PAUSED response and listPausedRuns leave the
draft out.
Pitfalls
Never hand it to the end user who asked: it holds the system prompt (memory notes, KB excerpts) and every tool output. A run that does not exist is the RunStore's 404.
Parameters
__namedParameters
runId
string
tenantId
string
Returns
Promise<IRun>
getThreadMemory()
getThreadMemory(
__namedParameters):Promise<IWorkingMemoryState>
Defined in: AIGateway.ts:1661
getThreadMemory -- what the gateway remembers about one conversation
About
The summary and the notes of a thread, for a "memory of this chat" panel.
Under SLIDING_WINDOW nothing is written, so the state is empty, or what an earlier SUMMARY_BUFFER period left
(ignored, never wiped). The thread must be the calling user's; another user's thread is a 404.
Who needs it
The host's chat screen, next to resetThreadMemory.
Parameters
__namedParameters
threadId
string
userContext
Returns
Promise<IWorkingMemoryState>
getTurns()
getTurns(
__namedParameters):Promise<ITurn[]>
Defined in: AIGateway.ts:1493
getTurns -- the history of a thread, for a chat screen that opens it
About
Every turn of the thread, oldest first, in one read.
No paging: a chat screen draws the whole conversation, as the Vercel chat client loads a chat. The thread must be the calling user's; another user's thread is a 404, like an unknown one.
Parameters
__namedParameters
threadId
string
userContext
Returns
Promise<ITurn[]>
getUserFacts()
getUserFacts(
__namedParameters):Promise<IUserMemoryFact[]>
Defined in: AIGateway.ts:1561
getUserFacts -- what the gateway remembers about the calling user
About
The user's facts, across threads, for a "what you know about me" screen.
The facts the UserMemoryStore extracted in the background and the ones set by hand (upsertUserFact); every one of
them goes into every prompt of the user.
Who needs it
The host's memory settings screen, next to upsertUserFact, removeUserFact and clearUserFacts.
Parameters
__namedParameters
userContext
Returns
Promise<IUserMemoryFact[]>
listPausedRuns()
listPausedRuns(
__namedParameters):Promise<IPausedRunSummary[]>
Defined in: AIGateway.ts:1883
listPausedRuns -- the runs that wait for a human, of the calling user
About
What the user's UI lists as "waiting for approval" or "in review".
A summary per run (IPausedRunSummary), never the stored run: no system prompt, no messages, and a review's envelope
without the draft answer (D99, D111); the reviewer reads the whole run with getPausedRun.
Who needs it
The host's inbox of pending approvals and questions, for the user who asked.
Parameters
__namedParameters
userContext
Returns
Promise<IPausedRunSummary[]>
listThreads()
Defined in: AIGateway.ts:1467
listThreads -- the conversations of the calling user, for the sidebar of a chat screen
About
One page of the user's threads, newest first.
Pages as the Vercel chat client does: the first read passes startingAfter: null, endingBefore: null and gets the newest
limit threads; endingBefore the last id shown reads the next, older page; startingAfter an id reads the newer ones.
Only the calling user's threads of the tenant are listed and accepted as cursors.
Pitfalls
limit outside 1–200 or both cursors at once is a 400; a cursor that is not one of the user's threads is a 404.
The order is by creation: a thread used again keeps its place, it does not move to the top.
Parameters
__namedParameters
page
userContext
Returns
recoverStuckRuns()
recoverStuckRuns(
__namedParameters):Promise<number>
Defined in: AIGateway.ts:1821
recoverStuckRuns -- the scheduled job's entry: fails the runs an instance that died left RUNNING
About
A run nobody works on any more is closed like a broken one: run FAILED, its cut tool call answered with an error in the thread, the thread open again. Returns how many were recovered.
An instance that dies inside a call leaves its run RUNNING and, after a resume, its thread waiting for a human:
no door reopens it (D97). The job lists the RUNNING runs of the tenant and, for each, takes the thread's lease first:
a live call holds it, so a lease obtained proves nobody works on the thread. Then it moves the run RUNNING to FAILED
atomically (D69), closes it with closeRun and releases the lease.
Who needs it
A scheduled job of the application, once per tenant and interval, next to expirePausedRuns.
Pitfalls
A thread that is busy (409) is skipped, so is one that cannot be closed (logged, D76); both are found again by the
next job. The proof holds only while a call's timeoutMs (D90) is at most the thread's leaseMs (D77): a longer live
call loses its lease and can be failed by the job while it runs.
Parameters
__namedParameters
tenantId
string
Returns
Promise<number>
removeFilesByUser()
removeFilesByUser(
__namedParameters):Promise<number>
Defined in: AIGateway.ts:1274
removeFilesByUser -- deletes every file a user uploaded
About
The files only;
deleteUserDatais the whole "forget this user".
Returns the number of files deleted. Raw ids, no user check: an admin surface.
Who needs it
The host, when only the uploads of a user must go (GDPR); deleteUserData and anonymizeUserData remove the files as
one of their steps.
Parameters
__namedParameters
tenantId
string
userId
string
Returns
Promise<number>
removeUserFact()
removeUserFact(
__namedParameters):Promise<void>
Defined in: AIGateway.ts:1579
removeUserFact -- forgets one fact about the calling user
About
One entry of the memory screen, gone.
A key that is not there changes nothing.
Parameters
__namedParameters
key
string
userContext
Returns
Promise<void>
renameThread()
renameThread(
__namedParameters):Promise<void>
Defined in: AIGateway.ts:1524
renameThread -- gives a thread the title the user typed
About
The user's own name for a conversation.
Replaces the title, also the one the working memory wrote. The thread must be the calling user's; another user's thread is a 404, like an unknown one.
Parameters
__namedParameters
threadId
string
title
string
userContext
Returns
Promise<void>
resetThreadMemory()
resetThreadMemory(
__namedParameters):Promise<void>
Defined in: AIGateway.ts:1691
resetThreadMemory -- "forget this thread": clears its memory, the turns stay
About
The conversation is kept; what the model remembers of it is dropped.
Runs after the memory updates of the thread that are already queued, so a late update does not bring the memory back. Another user's thread is a 404.
Parameters
__namedParameters
threadId
string
userContext
Returns
Promise<void>
resume()
resume<
TStructured>(__namedParameters):IGatewayCall<TStructured>
Defined in: AIGateway.ts:706
resume -- continues a paused run with the human's decisions
About
The human has decided; the run goes on from where it stopped.
One decision per envelope of the pause (D102), each named by the envelope's callId (a pause of one envelope takes one
decision without it). On a tool approval the step continues: APPROVE or EDIT executes the call (an edited input goes
through the tool's policy again, D109), REJECT hands the rejection text to the model. On a user question the answers
reach the model. On a reviewed answer (D99) no model is called: APPROVE delivers it, EDIT delivers editedInput as the
new text (parsed through the run's outputSchema again), REJECT drops it and the call is FAILED with the reviewer's
text as the reason. The run may pause again when a later step needs an approval or its answer is reviewed.
- The tools run with the context of the user who resumes (the approver is accountable for them); the run stays the asker's (D32): the memory extracted after it, its trace entries and the screens of the answer are under the user who started it.
- The run continues on its profile as it is now, or on
profileId(D65): the model, the output cap, the tools and the limits come from that profile; the system prompt and the messages so far stay. The review of the answer and the approvals are those of both profiles (D113): a switch never skips a guard the run started under. The skills the run kept (D124) and the profile's are offered again through a rebuiltdyna_aigw_load_skill; the system prompt keeps the list of the start. - The free text of an ANSWER goes through the input rules like a prompt (D71): an ERROR returns FAILED and the run stays paused, so the user can answer again; WARNINGs travel on the response.
- A resumed run answers like one that never paused (D36): its answer cites the knowledge-base chunks of its first step, its trace entries carry the template version, the user's facts are extracted from the whole run.
abortSignal/timeoutMsstop the resume as they stop a call (D90). An abort during the step closes the run as FAILED (its open tool call gets an error result, the thread is open again); an abort before the step, while the answer is screened, leaves the run paused. When the continuation throws, the run is marked FAILED, its pending tool call is closed in the thread and the thread is opened again.
The path: makeCallSignal → RunStore.load → the thread's lease → continueRun (the gate's matchDecisions, the
profile, the tools, the answer screen, HumanGate.resume, LLMClient.run, settleRun), or continueReview for a
reviewed answer.
Who needs it
The host's endpoint behind the approval card, the user-question form and the reviewer's screen; called from another
request than the run that paused, and often by another user.
What needs
The runId and pauseId of the PAUSED response (a decision for an earlier pause of the run is a 409, D110), and the
same additionalTools (D66) and outputSchema (D36) the run started with: a stored run keeps neither (a Zod schema is
code).
Usage
When to use it
A run or an earlier resume returned PAUSED and the human has decided.
When shouldn't use it
cancelRun drops a paused run without a decision. A new run on the thread is refused while the run waits.
Pitfalls
A decision that names no envelope, an envelope left without one, an action its envelope does not allow, a pending tool
the profile does not allow, a skill name in both the run and the profile, or an EDIT that misses the schema is a 400,
and the run stays paused. Without the outputSchema a structured run answers in plain text, and an EDIT of a
structured reviewed answer is a 400.
Type Parameters
TStructured
TStructured = unknown
Parameters
__namedParameters
abortSignal?
AbortSignal
Stops the resume when aborted, as IGatewayAskInput.abortSignal.
additionalTools?
IToolDef<unknown, unknown>[]
The tools the run started with (IGatewayRunInput.additionalTools); a run tool that is in neither the registry nor here is a 400.
debug?
boolean
Attach the transcript to the answer, as IGatewayAskInput.debug.
decisions
One per envelope of the pause (IPausedResponse.envelopes), named by its callId; one envelope takes one decision without it.
outputSchema?
ZodType<TStructured, unknown, $ZodTypeInternals<TStructured, unknown>>
The schema the run started with (IGatewayRunInput.outputSchema), handed in again: a stored run keeps none (D36).
Types
structuredof the resumed answer, as onrun().
Omit it for a run started without one. Without it a structured run answers in plain text, and an EDIT of a structured reviewed answer is a 400.
pauseId
string
The pause the decisions are for: IPausedResponse.pauseId (each envelope carries it too). A run can pause again under
the same runId; a decision for an earlier pause is a 409, so a retried approval never decides the next pause.
profileId?
string
Continue on another profile (its model, tools, limits); omit for the profile the run started with.
runId
string
timeoutMs?
number
Deadline of the resume in milliseconds, as IGatewayAskInput.timeoutMs.
userContext
Returns
IGatewayCall<TStructured>
run()
run<
TStructured>(input):IGatewayCall<TStructured>
Defined in: AIGateway.ts:497
run -- an agent run with tools; may pause for a human
About
The model works the task through the tools until it answers, a tool ends the run, or a human has to step in.
Returns at once (D38): events streams the run (text, tool requests and results, step ends, a pause), response is the
final ANSWER, PAUSED or FAILED. The model sees the tools the ToolPolicy resolves for the profile, the user's
permissions and the call (activeTools, additionalTools); every step goes through the usage adapter. A tool whose
policy says NEEDS_APPROVAL, a question the model asks the user and, on a profile with reviewAnswers (D99), the
finished answer pause the run: it is stored, the thread waits for a human, and resume continues it.
- The tool turns a run produced (tool requests, tool results) are stored even when the final answer fails the checks, because the tools did run; the failed answer itself is not stored (D70).
abortSignal/timeoutMsstop the call (D90): the model call and the tool in flight are aborted, the response is FAILED with the reason, the steps that completed are stored, a tool call the abort cut gets an error result, and a paused run is never created.responserejects as inaskfor what the caller got wrong; a tool name outside what the profile and the permissions allow is a 400 before anything is stored (D48).
The path is ask's with LLMClient.run in place of complete, the hooks of createRunHooks (execute a tool, decide
a call, narrow the tools of a step) and settleRun for the turns, the pause and the response.
Who needs it
The host's chat endpoint for an assistant that acts: a support agent with a refund tool, an accounting assistant that posts a credit note after an approval, a form the model fills by asking the user.
What needs
As ask, plus the tools: registered in IToolRegistryConfig.tools or passed as additionalTools, allowed by the
profile's allowedTools and the user's permissions.
Usage
When to use it
The model needs tools, a step may need a human, or the answer must be reviewed before the user sees it.
When shouldn't use it
ask for a plain answer: one model call, no tool list in the prompt, and it never pauses.
Pitfalls
Keep the PAUSED response's pauseId and envelopes: resume needs one decision per envelope. A paused run holds the
thread AWAITING_HUMAN until it is resumed, cancelled or expired (expirePausedRuns). additionalTools and
outputSchema are not stored with a paused run: resume must receive them again.
Type Parameters
TStructured
TStructured = unknown
Parameters
input
IGatewayRunInput<TStructured>
Returns
IGatewayCall<TStructured>
shutdown()
shutdown(
__namedParameters):Promise<void>
Defined in: AIGateway.ts:1747
shutdown -- stops taking calls, drains the ones in flight, aborts the rest (D96)
About
A server that is stopped finishes what it is doing instead of cutting calls in half.
From the first call on, every new ask, run and resume is refused with a 503 (the client did nothing wrong and
may retry on another instance). The calls in flight get timeoutMs to end; the ones still running after it are
aborted through their call signal (D90), so each closes its run, releases its thread lease and keeps its turns, and
answers FAILED with call_aborted. Then the background memory updates land (whenIdle). Paused runs are untouched:
they are stored. A second call returns the same promise.
Who needs it
The application's SIGTERM handler, before it closes its HTTP server and its database connections.
Pitfalls
A crash (kill -9, out of memory) skips all of it. There is no way back: a gateway that was shut down takes no call again; construct a new one.
Parameters
__namedParameters
timeoutMs
number
The grace period of the calls in flight, in milliseconds; a positive integer, as IGatewayAskInput.timeoutMs.
Returns
Promise<void>
uploadFile()
uploadFile(
__namedParameters):Promise<IFileRef>
Defined in: AIGateway.ts:1168
uploadFile -- stores an upload and returns its pointer
About
The bytes go to the
FileStore; the application keeps the pointer.
The file is the calling user's. The pointer goes into IGatewayAskInput.files of a later call, where the model reads
the image or the PDF. Refused with a 400: a type outside fileStore.allowedMime (default PNG, JPEG, WebP, GIF, PDF; no
SVG), an empty file, a name over 255 characters or with control characters; with a 413: a file over fileStore.maxBytes;
with a 409 while the user is being erased (D114).
Who needs it
The host's upload endpoint, before the chat call that sends the file.
Pitfalls
A pointer of another user in a call is a 400. A file nobody references stays until deleteFile, the delete of its
thread or the erasure of the user.
Parameters
__namedParameters
bytes
Uint8Array
mime
string
name
string
userContext
Returns
Promise<IFileRef>
upsertUserFact()
upsertUserFact(
__namedParameters):Promise<void>
Defined in: AIGateway.ts:1609
upsertUserFact -- sets one remembered fact of the calling user by hand
About
The user tells the assistant what to remember, without a conversation.
Same key = replace. Throws a 400 for an empty key or value, and for a key or value over userMemoryStore.maxFactChars
(default 500): every fact goes into every prompt of the user. A 409 while the user is being erased (D114).
Who needs it
The host's memory settings screen ("add", "edit").
Parameters
__namedParameters
key
string
userContext
value
string
Returns
Promise<void>
whenIdle()
whenIdle():
Promise<void>
Defined in: AIGateway.ts:1721
whenIdle -- resolves when the background work is done
About
The memory updates after an answer run in the background; this is how to wait for them.
Never rejects: a background task that fails is reported, not thrown.
Who needs it
Tests, before they read the memory; shutdown, as its last step. A server calls shutdown.
Returns
Promise<void>