Skip to main content

Interface: ModelRegistry

Defined in: internal/model-registry/ModelRegistry.ts:73

ModelRegistry -- the global catalog of the models the platform may use

About​

Facts about a model, never a tenant's choice.

One place that answers "what can this model do and what does it cost": provider and provider id, capabilities, prices, the enabled flag. No provider knowledge leaks into the rest of the gateway: the LLMClient reads provider and providerModelId, the call path reads caps, the UsageGuard and the trace get estimateCostUsd.

  • Global by design, no tenantId: a tenant picks a model through its profile and never edits here.
  • The read path is hot (every call reads caps), so the catalog is cached with a TTL; the write path is rare and drops the cache.
  • A disabled model stays in the catalog: it is refused by every call that names it, and a call in flight is still priced.

Created first by createGatewayDeps: the LLMClient, the AIProfileStore and the guardrails depend on it.

Who needs it​

prepareCall and AIGateway.resume (the profile's model), AIProfileStore.upsertProfile (validation), LLMClient (every call), the guardrails' judge; the host's administration screen through AIGateway.modelRegistry.

What needs​

IModelRegistryConfig: the IModelRegistryRepository and the optional cacheTtlMs.

Usage​

When to use it​

Any question about a model given its id; any admin write to the catalog.

When shouldn't use it​

Which model a tenant uses: that is the profile (AIProfileStore). What a call may spend: that is the UsageGuard.

Pitfalls​

An admin write is visible to other instances only after their cache TTL. A disabled or removed model fails the profiles that name it at their next call: switch the profiles first.

Implements​

Methods​

estimateCostUsd()​

estimateCostUsd(__namedParameters): Promise<number>

Defined in: internal/model-registry/ModelRegistry.ts:155

estimateCostUsd -- the USD estimate of one call, from its token counts

About​

uncached × costIn + cache reads × costCacheRead + cache writes × costCacheWrite + output × costOut, per million tokens.

The uncached input is inputTokens minus the cached and the cache-write tokens, never below zero (a provider that reports more cached tokens than the total yields no negative count). The cache rates are the caps' (costCacheReadPerMTokUsd, costCacheWritePerMTokUsd) or their provider defaults: Anthropic 0.1 × and 1.25 × the input price, any other provider the input price. Works for a disabled model too: a call that was in flight when the model got disabled is still reported.

Who needs it​

LLMClient, after every call, for the UsageGuard and the trace.

Parameters​

__namedParameters​
cachedInputTokens?​

number = 0

Of inputTokens, the ones read from the provider's cache.

cacheWriteInputTokens?​

number = 0

Of inputTokens, the ones written to the provider's cache.

inputTokens​

number

Prompt tokens, the cached ones included.

modelId​

string

Our model id.

outputTokens​

number

Completion tokens.

Returns​

Promise<number>

Implementation of​

IModelRegistry.estimateCostUsd

getModel()​

getModel(__namedParameters): Promise<IModelInfo>

Defined in: internal/model-registry/ModelRegistry.ts:94

getModel -- one model entry, enabled

About​

The facts of a model, for a call that is about to use it.

Reads the cached catalog and returns a copy of the entry. An unknown id is a 404, a disabled one a 400: a call never reaches a model the platform switched off.

Who needs it​

prepareCall (the profile's model, step 1), AIProfileStore.upsertProfile (the profile's models must exist and be enabled), LLMClient (provider and provider id of every call), the guardrails' judge, AIGateway.resume.

Parameters​

__namedParameters​
modelId​

string

Returns​

Promise<IModelInfo>

Implementation of​

IModelRegistry.getModel

listModels()​

listModels(__namedParameters?): Promise<IModelInfo[]>

Defined in: internal/model-registry/ModelRegistry.ts:122

listModels -- the catalog, for an administration screen or a profile editor

About​

Every model the platform knows, enabled ones by default, one provider on request.

Read from the cache; copies, in the repository's order.

Who needs it​

The host's administration screen and profile editor; the dev sandbox.

Parameters​

__namedParameters?​
enabledOnly?​

boolean = true

Skip disabled entries. Default true.

provider?​

EModelProvider

Filter by provider.

Returns​

Promise<IModelInfo[]>

Implementation of​

IModelRegistry.listModels

refresh()​

refresh(): Promise<void>

Defined in: internal/model-registry/ModelRegistry.ts:273

refresh -- drops the cache and reloads the catalog from the repository

About​

For a host that changed the repository behind the registry's back.

The next reads are served from the reloaded catalog; nothing else changes.

Returns​

Promise<void>

Implementation of​

IModelRegistry.refresh

removeModel()​

removeModel(__namedParameters): Promise<void>

Defined in: internal/model-registry/ModelRegistry.ts:258

removeModel -- deletes a model entry

About​

Gone for good: prefer setEnabled to keep the entry and its history.

An unknown model is the 404 of getModel. The registry is global and knows no profiles: a profile that names the removed model is refused at its next call with the same 404, so switch the profiles first.

Parameters​

__namedParameters​
modelId​

string

Returns​

Promise<void>

Implementation of​

IModelRegistry.removeModel

setEnabled()​

setEnabled(__namedParameters): Promise<void>

Defined in: internal/model-registry/ModelRegistry.ts:227

setEnabled -- switches a model off, or on again, without deleting it

About​

A disabled model is refused by every call that names it; its entry, its profiles and its history stay.

Saves the entry with the new flag and drops the cache, so the next call reads it. An unknown model is the 404 of getModel. A call in flight on the model finishes and is reported (estimateCostUsd reads disabled entries too).

Who needs it​

The administration screen of the platform; a profile that names the model is not touched and fails at its next call.

Parameters​

__namedParameters​
enabled​

boolean

New state.

modelId​

string

Our model id.

Returns​

Promise<void>

Implementation of​

IModelRegistry.setEnabled

upsertModel()​

upsertModel(__namedParameters): Promise<void>

Defined in: internal/model-registry/ModelRegistry.ts:206

upsertModel -- inserts or replaces a model entry

About​

The administration write: the entry as given, whole, and the cache dropped.

Saves through the repository and drops this instance's cache, so the next read sees the entry; other instances see it after their TTL. No validation of the entry beyond its type: the host's screen is the editor.

Who needs it​

The host's administration screen; the tests and the first run, to seed the catalog.

Parameters​

__namedParameters​
model​

IModelInfo

Returns​

Promise<void>

Implementation of​

IModelRegistry.upsertModel