Class: InMemoryUsageAdapter
Defined in: adapters/in-memory/InMemoryUsageAdapter.ts:50
InMemoryUsageAdapter -- the usage adapter of the tests and the sandbox
About
A test double, not a limit.
Records every check and every addUsage it receives, so a test can read what the gateway asked and reported, and denies
when its deny option says so. It keeps no totals and enforces nothing: a daily limit shipped by the gateway would be token
logic in the gateway, which is exactly what the app owns.
Who needs it
The test configs (gatewayConfigBase) and src/dev-sandbox/sandbox.config.ts. An application writes its own adapter.
Usage
When shouldn't use it
In an application: it approves everything and forgets everything at restart.
Implements
Constructors
Constructor
new InMemoryUsageAdapter(
config?):InMemoryUsageAdapter
Defined in: adapters/in-memory/InMemoryUsageAdapter.ts:57
Parameters
config?
IInMemoryUsageAdapterConfig = {}
Returns
InMemoryUsageAdapter
Properties
addUsageCalls
readonlyaddUsageCalls:IUsageAddArgs[] =[]
Defined in: adapters/in-memory/InMemoryUsageAdapter.ts:55
Every addUsage received, in order.
checkCalls
readonlycheckCalls:IUsageCheckArgs[] =[]
Defined in: adapters/in-memory/InMemoryUsageAdapter.ts:52
Every check received, in order.
Methods
addUsage()
addUsage(
args):Promise<void>
Defined in: adapters/in-memory/InMemoryUsageAdapter.ts:100
addUsage -- what the call used
About
Told once after every approved
check, also for a failed or aborted call.
Then with zero usage, so an adapter that reserves in check can always settle by traceId. A throw is logged; the call's
answer is still returned.
Who needs it
UsageGuard.addUsage, after every provider call.
Parameters
args
Returns
Promise<void>
Implementation of
check()
check(
args):Promise<IUsageCheckResult>
Defined in: adapters/in-memory/InMemoryUsageAdapter.ts:73
check -- may this model call run?
About
The app's answer, before every model call the gateway makes.
Asked for the answer, every run step, every guardrail judge, query rewrite, memory update and embedding, with the call's factors and its rough size. A throw is treated as a denial (fail closed) and logged.
Who needs it
UsageGuard.check, right before the provider is called.
Parameters
args
Returns
Promise<IUsageCheckResult>