Interface: ILLMClient
Defined in: internal/llm-client/types.ts:376
Public surface of the LLMClient. Every deps.llmClient is typed with it; tests inject a FakeLLMClient that implements it.
Properties
providers
readonlyproviders:EModelProvider[]
Defined in: internal/llm-client/types.ts:497
providers -- the providers with credentials
About
Which of
EModelProviderthis client can call.
The providers that have an entry in ILLMClientConfig.providers, in enum order. A model of any other provider is a 400
when it is called.
Who needs it
The real-provider tests, to skip a provider without a key; a host that lists what its gateway can reach.
Methods
complete()
complete(
args):Promise<IModelResult>
Defined in: internal/llm-client/types.ts:398
complete -- one model call
About
Checked before, reported and traced after; tool requests come back unexecuted.
Resolves the model, refuses a temperature on a model without caps.temperature (400, D82), asks the UsageGuard
(a no rejects with the usage-denied error, 429, and the provider is not called), then makes the SDK call: streamed
through hooks.onEvent (TEXT_DELTA, then DONE) when hooks are given, generated otherwise. Returns the text, the parsed
structured output when the request had a schema, the tool requests as the model made them, the finish reason, the usage
and the trace id. After the call the app is told what it used and the entry is recorded, also for a failed or aborted
call (then with zero usage and no entry). A provider failure is a 502; an aborted signal rejects with its reason, the
partial stream is neither returned nor traced (D90). An answer that does not parse against the schema is returned as
text, for the Guardrails to report.
Who needs it
AIGateway for an ask; run, once per step; createJudge, QueryRewriter, UserMemoryStore.extract and
updateSummary for their one completion.
Parameters
args
abortSignal?
AbortSignal
Aborts the provider call; the caller's deadline, next to the client's own timeout. An aborted call rejects with the signal's reason, not with a 502; it is reported to the usage adapter with zero usage and not traced (the provider bills its own count).
hooks?
Pick<ILLMClientHooks, "onEvent">
Only onEvent is used.
meta
request
tools?
IToolDef<unknown, unknown>[]
Tools the model may request (forced-choice routing).
Returns
Promise<IModelResult>
embed()
embed(
args):Promise<number[][]>
Defined in: internal/llm-client/types.ts:477
embed -- texts to vectors
About
One vector per text, in order; checked, reported and traced like a completion.
Refuses a model without caps.embeddings (400), asks the UsageGuard with the token count of the texts, embeds them in
one SDK call under the client's timeout and retries, then reports the usage and records the entry (kind EMBED, no
request and no result: the usage is what matters). A provider failure is a 502, after the app was told the call used
nothing. An empty list returns an empty list without a call.
Who needs it
Embedder, for the KB ingestion and the KB search.
Parameters
args
meta
modelId
string
texts
string[]
Returns
Promise<number[][]>
run()
run(
args):Promise<IRunOutcome>
Defined in: internal/llm-client/types.ts:432
run -- the agent loop
About
One
completeper step; every decision is a hook the gateway implements.
Runs runAgentLoop with complete as its model call. Each step offers the tools hooks.prepareStep returns; a tool
request is decided by hooks.decideToolCall (ALLOW runs it through hooks.executeTool, DENY writes a denied result the
model reads, NEEDS_APPROVAL pauses the run, D87), hooks.endsRun may end the run after the step with one final step
without tools (D86), and limits.maxSteps ends it with finishReason LENGTH. A resume continues a paused run from its
stored messages with the human's decisions. The steps stream through hooks.onEvent with their stepIndex; DONE is sent
once, at the end. The signal ends the loop between steps, in a model call or in a tool: the outcome is ABORTED with the
steps that completed (D90).
Who needs it
AIGateway for a run and for a resume; tools is the set the run starts with, as prepareCall resolved it.
Parameters
args
abortSignal?
AbortSignal
The call's signal: aborts the model call in flight and the tool in flight. The loop then returns ABORTED with the steps that completed.
hooks
limits
meta
parallelToolCalls?
boolean
The tool calls of one step run concurrently (D101); false runs them one after the other.
The profile's
parallelToolCalls, resolved by the gateway.
Default
true
request
resume?
{ decisions: IResumeDecision[]; messages: IMessage[]; rejectionTexts?: Record<string, string>; }
Messages of a paused run plus the human's decisions; omit for a fresh run.
resume.decisions
One per pending call the human decided on; a single decision without callId is for the first unresolved call.
resume.messages
IMessage[]
resume.rejectionTexts?
Record<string, string>
The policy's text for a rejection, per callId (D87), as the pause carried it; a call without one gets DEFAULT_TOOL_REJECTION_TEXT.
tools
IToolDef<unknown, unknown>[]
Returns
Promise<IRunOutcome>