Skip to main content

Interface: WorkingMemory

Defined in: internal/working-memory/WorkingMemory.ts:78

WorkingMemory -- the built-in thread memory, two strategies behind one interface

About​

The gateway asks "what should the model know from this thread?"; the strategy answers.

Implements IWorkingMemory for the two built-in strategies of EWorkingMemoryStrategy, chosen per call from the profile's workingMemory, else the config's strategy, else SUMMARY_BUFFER:

  • SLIDING_WINDOW: load gives the newest turns that fit chatHistoryRatio of the history budget, counted with the model's tokenizer; saveContext does nothing, nothing is stored.
  • SUMMARY_BUFFER: load gives the stored summary + facts as the notes and the last recentTurns turns; saveContext has the cheap model rewrite the summary + facts once the thread grew by updateEveryTurns turns.
  • Switched off (enabled: false): load gives the last 20 turns and no notes; saveContext does nothing.

Every window opens on a user turn: a leading assistant or tool turn is dropped, the providers reject a tool result without its call.

It sits between the ThreadStore (the turns and the stored state) and the ContextAssembler (the request).

Who needs it​

ContextAssembler.build calls load for every request; the background memory update calls saveContext after every answer; AIGateway.getThreadMemory and AIGateway.resetThreadMemory call getState and reset. A host replaces it with its own IWorkingMemory in overrides.workingMemory.

What needs​

threadStore for the turns and the state, turnBuilder for turns → messages, promptTemplateStore for the MEMORY_WORKING template and llmClient for the cheap-model call of SUMMARY_BUFFER.

Usage​

When to use it​

Always, through the gateway: it is created by createGatewayDeps from IAIGatewayConfig.workingMemory.

When shouldn't use it​

A host with its own memory model (a vector memory, a graph, a different summary) implements IWorkingMemory instead and passes it in overrides.

Pitfalls​

A thread keeps its stored summary when its profile switches to SLIDING_WINDOW: the summary is ignored, not wiped, so a switch back resumes from it. getState shows it meanwhile.

Implements​

Methods​

getState()​

getState(__namedParameters): Promise<IWorkingMemoryState>

Defined in: internal/working-memory/WorkingMemory.ts:266

getState -- what the memory has stored about one thread

About​

The memory, as the user may see it.

Returns the stored summary, facts and the turn count they were written at; the empty state for a thread the memory never wrote. Under SLIDING_WINDOW the state is whatever an earlier SUMMARY_BUFFER period left: the window ignores it, it is not wiped.

  • Never calls a model.
  • Never writes.

Who needs it​

AIGateway.getThreadMemory, for a "what do you remember" UI.

Pitfalls​

An unknown thread, or one of another tenant, is a 404.

Parameters​

__namedParameters​
tenantId​

string

threadId​

string

Returns​

Promise<IWorkingMemoryState>

Implementation of​

IWorkingMemory.getState

load()​

load(__namedParameters): Promise<IWorkingMemoryLoad>

Defined in: internal/working-memory/WorkingMemory.ts:107

load -- the history part of one model request

About​

Everything the model should know from this thread, sized for this call.

Returns the message window (the turns the model reads verbatim, the current user turn last) and the notes (the text of the "About this conversation" part of the {memory} slot; empty when there is nothing). Which turns, how many and whether there are notes is the implementation's business: the strategy, the profile's override, the budget.

  • Never calls a model.
  • Never writes.

Who needs it​

ContextAssembler.build, once per request, after it rendered the system prompt, so budgetTokens is known.

Pitfalls​

budgetTokens is what is left of the window after the system prompt and the output reserve; an implementation that returns more is trimmed by the assembler, oldest message first, with a turns_trimmed warning.

Parameters​

__namedParameters​
budgetTokens​

number

Tokens left for the history after the system prompt and the output reserve.

model​

IModelInfo

profile​

IAIProfile

The call's profile; profile.workingMemory may override the gateway's strategy.

tenantId​

string

threadId​

string

Returns​

Promise<IWorkingMemoryLoad>

Implementation of​

IWorkingMemory.load

reset()​

reset(__namedParameters): Promise<void>

Defined in: internal/working-memory/WorkingMemory.ts:300

reset -- forgets what the memory stored about one thread

About​

"Forget this conversation": the notes go, the turns stay.

Clears the stored summary and facts and marks the turns so far as seen, so the next saveContext does not bring them back. The thread and its turns are untouched; the model still reads the window.

Who needs it​

AIGateway.resetThreadMemory, for a "forget" action of the UI.

Pitfalls​

An unknown thread, or one of another tenant, is a 404.

Parameters​

__namedParameters​
tenantId​

string

threadId​

string

Returns​

Promise<void>

Implementation of​

IWorkingMemory.reset

saveContext()​

saveContext(__namedParameters): Promise<void>

Defined in: internal/working-memory/WorkingMemory.ts:183

saveContext -- learns from one answered call

About​

The answer is in; remember what must be remembered.

Called in the background after every answer. The new turns are already stored on the thread (ThreadStore); an implementation reads them from there, or from request and result when the gateway has them (an answer a human edited in a review has no model result). The built-in SUMMARY_BUFFER updates the summary + facts when updateEveryTurns turns are due; SLIDING_WINDOW does nothing.

Who needs it​

The gateway's background memory update, after ask and after every closed run.

Pitfalls​

Runs on the gateway's background queue (whenIdle): it must not throw; it reports and returns.

Parameters​

__namedParameters​
cheapModelId​

string

Cheap model for helper calls, resolved by the gateway (the profile's or the platform default).

meta​

ILLMClientCallMeta

For the usage guard and the trace; kind = MEMORY_WORKING.

profile​

IAIProfile

request?​

IModelRequest

The request the answer was produced for; absent when the gateway has none at hand.

result?​

IModelResult

The model's answer; absent when a human wrote or edited it (a reviewed run).

tenantId​

string

threadId​

string

userId​

string

Returns​

Promise<void>

Implementation of​

IWorkingMemory.saveContext