Interface: LLMClient
Defined in: internal/llm-client/LLMClient.ts:154
LLMClient -- the one door to every model provider
About
Three doors; the decisions live in hooks.
The only class that imports the AI SDK: complete makes one model call and may return tool requests it never
executes, run is the agent loop (runAgentLoop) with one complete per step and every decision outsourced to
ILLMClientHooks, embed turns texts into vectors. The modelId is resolved through the ModelRegistry, the credentials
come from the config, the provider adapters (createLanguageModel, createEmbeddingModel, one per EModelProvider, D84)
and the mappers to the SDK's shapes live in utils/. Because every model call passes here, the client also asks the
UsageGuard before and reports to it after every call (D121), and records every call in the Trace under the call's
ILLMClientCallMeta; the gateway only annotates the entry afterwards. The system prompt arrives as blocks (D123): a
provider with a cache marker gets one message per block, the others the texts joined.
Sits between the orchestration and the helpers that call a model (Guardrails' judge, QueryRewriter, the memories,
Embedder) and the providers behind the SDK.
Who needs it
AIGateway for the answer and the run; createJudge, QueryRewriter, UserMemoryStore, updateSummary for their
completions; Embedder for embed. Constructed once from IAIGatewayConfig.llmClient; the tests inject a
FakeLLMClient through IAIGatewayDeps.llmClient.
What needs
ILLMClientConfig: the credentials per provider, the timeout and the retries. ILLMClientDeps: the ModelRegistry,
the UsageGuard and the Trace.
Pitfalls
The SDK's own loop (stopWhen, ToolLoopAgent) is not used: a run must pause mid-loop, be stored and continue in another
process, so the loop is ours and the SDK sees one call per step. A provider failure is a 502 with the provider's error as
parentError, stripped of the request body (withoutRequestBody): a logger never writes the prompt. An aborted call
rejects with the signal's reason, is reported with zero usage and is not traced (D90).
Implements
Accessors
providers
Get Signature
get providers():
EModelProvider[]
Defined in: internal/llm-client/LLMClient.ts:524
providers -- the providers with credentials
About
Which of
EModelProviderthis client can call.
The providers that have an entry in ILLMClientConfig.providers, in enum order. A model of any other provider is a 400
when it is called.
Who needs it
The real-provider tests, to skip a provider without a key; a host that lists what its gateway can reach.
Returns
Implementation of
ILLMClient.providers
Methods
complete()
complete(
__namedParameters):Promise<IModelResult>
Defined in: internal/llm-client/LLMClient.ts:181
complete -- one model call
About
Checked before, reported and traced after; tool requests come back unexecuted.
Resolves the model, refuses a temperature on a model without caps.temperature (400, D82), asks the UsageGuard
(a no rejects with the usage-denied error, 429, and the provider is not called), then makes the SDK call: streamed
through hooks.onEvent (TEXT_DELTA, then DONE) when hooks are given, generated otherwise. Returns the text, the parsed
structured output when the request had a schema, the tool requests as the model made them, the finish reason, the usage
and the trace id. After the call the app is told what it used and the entry is recorded, also for a failed or aborted
call (then with zero usage and no entry). A provider failure is a 502; an aborted signal rejects with its reason, the
partial stream is neither returned nor traced (D90). An answer that does not parse against the schema is returned as
text, for the Guardrails to report.
Who needs it
AIGateway for an ask; run, once per step; createJudge, QueryRewriter, UserMemoryStore.extract and
updateSummary for their one completion.
Parameters
__namedParameters
abortSignal?
AbortSignal
Aborts the provider call; the caller's deadline, next to the client's own timeout. An aborted call rejects with the signal's reason, not with a 502; it is reported to the usage adapter with zero usage and not traced (the provider bills its own count).
hooks?
Pick<ILLMClientHooks, "onEvent">
Only onEvent is used.
meta
request
tools?
IToolDef<unknown, unknown>[] = []
Tools the model may request (forced-choice routing).
Returns
Promise<IModelResult>
Implementation of
ILLMClient.complete
embed()
embed(
__namedParameters):Promise<number[][]>
Defined in: internal/llm-client/LLMClient.ts:441
embed -- texts to vectors
About
One vector per text, in order; checked, reported and traced like a completion.
Refuses a model without caps.embeddings (400), asks the UsageGuard with the token count of the texts, embeds them in
one SDK call under the client's timeout and retries, then reports the usage and records the entry (kind EMBED, no
request and no result: the usage is what matters). A provider failure is a 502, after the app was told the call used
nothing. An empty list returns an empty list without a call.
Who needs it
Embedder, for the KB ingestion and the KB search.
Parameters
__namedParameters
meta
modelId
string
texts
string[]
Returns
Promise<number[][]>
Implementation of
ILLMClient.embed
run()
run(
__namedParameters):Promise<IRunOutcome>
Defined in: internal/llm-client/LLMClient.ts:352
run -- the agent loop
About
One
completeper step; every decision is a hook the gateway implements.
Runs runAgentLoop with complete as its model call. Each step offers the tools hooks.prepareStep returns; a tool
request is decided by hooks.decideToolCall (ALLOW runs it through hooks.executeTool, DENY writes a denied result the
model reads, NEEDS_APPROVAL pauses the run, D87), hooks.endsRun may end the run after the step with one final step
without tools (D86), and limits.maxSteps ends it with finishReason LENGTH. A resume continues a paused run from its
stored messages with the human's decisions. The steps stream through hooks.onEvent with their stepIndex; DONE is sent
once, at the end. The signal ends the loop between steps, in a model call or in a tool: the outcome is ABORTED with the
steps that completed (D90).
Who needs it
AIGateway for a run and for a resume; tools is the set the run starts with, as prepareCall resolved it.
Parameters
__namedParameters
abortSignal?
AbortSignal
The call's signal: aborts the model call in flight and the tool in flight. The loop then returns ABORTED with the steps that completed.
hooks
limits
meta
parallelToolCalls?
boolean = true
The tool calls of one step run concurrently (D101); false runs them one after the other.
The profile's
parallelToolCalls, resolved by the gateway.
Default
true
request
resume?
{ decisions: IResumeDecision[]; messages: IMessage[]; rejectionTexts?: Record<string, string>; }
Messages of a paused run plus the human's decisions; omit for a fresh run.
resume.decisions
One per pending call the human decided on; a single decision without callId is for the first unresolved call.
resume.messages
IMessage[]
resume.rejectionTexts?
Record<string, string>
The policy's text for a rejection, per callId (D87), as the pause carried it; a call without one gets DEFAULT_TOOL_REJECTION_TEXT.
tools
IToolDef<unknown, unknown>[]
Returns
Promise<IRunOutcome>
Implementation of
ILLMClient.run