Terminology
The documentation of dyna-ai-gateway uses the vocabulary of the field as it stands. No term is invented here. This
page lists the terms that appear in the docs, what each one means in this library, and where the term comes from when its
origin is known. Where the library uses a plain word for a standard concept, the row says so.
Names in backticks are the code's names. A (D…) points to the entry of the same id in
Decisions.
1. Models and calls
| Term | In this library | Origin |
|---|---|---|
| LLM, model | A large language model behind a provider API. The gateway calls one model per step through the LLMClient. | "Large language model" comes from NLP research; it became the common name with GPT-2 and GPT-3 (2019–2020). |
| Provider | The company and API that serve a model: EModelProvider (OPENAI, ANTHROPIC, GOOGLE, MISTRAL, BEDROCK). Each has an adapter (D84). | The Vercel AI SDK's name for its per-vendor packages (@ai-sdk/openai, …). |
| Model registry, capabilities | The ModelRegistry lists the models the application allows, with their caps: tools, reasoning, temperature (D82). | "Model registry" is MLOps vocabulary (MLflow, 2018); "capabilities" is the generic term of SDKs for what a model supports. |
| Token | The unit a model reads and writes; usage is counted in tokens (IUsage, IUsageAddArgs.usage). | Tokenization is as old as NLP; the subword tokens of today's models go back to byte-pair encoding (Sennrich et al., 2016). |
| Context window | The maximum number of tokens one call may carry: the prompt, the history, the tool results and the answer. | Transformer-era term (Vaswani et al., 2017) for the attention span of the model. |
| Temperature | The sampling randomness of a call, 0 for deterministic. Per call, only on a model whose caps allow it (D58, D82). | Borrowed from statistical mechanics (Boltzmann distribution); entered neural networks through the softmax temperature. |
| Abort signal, deadline | abortSignal and timeoutMs of a call: one AbortSignal inside that stops the model call, the loop, the tool and the rules; the response is FAILED with call_aborted or call_timed_out (D90). | AbortController / AbortSignal are the WHATWG DOM standard for cancelling fetch (2017), adopted by Node; AbortSignal.timeout and AbortSignal.any came with Node 17 and 20. |
| Reasoning, reasoning effort | A model that "thinks" before it answers; the cap reasoning marks it. | OpenAI o1 introduced reasoning_effort (September 2024); Anthropic calls it "extended thinking" (2025). |
| Structured output | An answer that is an object validated by a zod schema instead of text: the TStructured type parameter of ask and run. | OpenAI "JSON mode" (November 2023) and "Structured Outputs" (August 2024); the AI SDK calls it generateObject. |
| Output format | The format of a text answer, EOutputFormat (MARKDOWN, PLAIN), chosen by the caller (D42). | Plain term. |
| Streaming | The answer arrives as events while the model writes; a call returns its stream and its response (D38). | Server-sent events over HTTP; every provider API streams this way. |
| Finish reason | Why the model stopped: EFinishReason. | OpenAI's finish_reason field (2020). |
| Usage, cost | Tokens in and out per call, priced through the model registry, written to the trace. | Provider APIs return a usage object; the name followed. |
| Fallback model | A second model tried when the first fails. Not in the library today. | Resilience vocabulary (circuit breaker, fallback; Nygard, Release It!, 2007). |
2. Conversation
| Term | In this library | Origin |
|---|---|---|
| Thread | One conversation of one user: IThread, kept by the ThreadStore; one call at a time per thread (D32, D77). | Forum and e-mail vocabulary; the OpenAI Assistants API (November 2023) made it the name for a chat history. |
| Turn | One message of the thread, ITurn with an ETurnRole (user, assistant, tool). | Conversation analysis: "turn-taking" (Sacks, Schegloff and Jefferson, 1974). |
| Part | A piece of a turn: text, a file, a tool call or a tool result (EPartKind). | The multimodal message format of the provider APIs ("content parts"). |
| Prompt | What the application sends for one call: the user's text and files. | Came with GPT-2 and GPT-3 (2019–2020); "prompt engineering" followed in 2021. |
| System prompt | The instructions the model receives before the conversation, produced from one template with layers (D20, D42, D68). | The system role of the OpenAI Chat Completions API (March 2023). |
| Prompt template | A versioned text with slots ({memory}, {kb}, …) that the ContextAssembler fills; kept by the PromptTemplateStore (D69). | LangChain's PromptTemplate (October 2022) made the name standard. |
| Addendum | The profile's text appended to the template: promptAddendum. | Plain word. Other libraries say "instructions" (OpenAI Assistants, Agents SDK). |
| Stable block, cache breakpoint | The part of the system prompt that is the same on every call of a profile (template text, output format, addendum, skills), ended by the {cacheBreakpoint} placeholder; sent as its own block so the provider can cache it (D123). | Anthropic's cache_control breakpoints (prompt caching, August 2024): a marker on the last block of the prefix to cache. |
| Prompt caching | A provider's reuse of a prompt prefix it has already processed, billed at a cache-read rate; contextAssembler.promptCaching switches it on, sharePromptCacheAcrossTenants decides whether tenants share entries (D123). | The providers' own name: Anthropic "prompt caching" (2024), OpenAI "prompt caching" (October 2024). |
| Instructions, context | Per-call text: instructions for how to answer, context for what the user is looking at (D68). | "Instructions" is the OpenAI Agents SDK's name for the system prompt; "context" is the generic term. |
| Skill | Instructions the model loads only when a task needs them: ISkill {name, description, body} on a profile (IAIProfile.skills) or on a call (skills). A run() lists name and description and offers the tool dyna_aigw_load_skill; an ask() inlines the body (D124). | Anthropic's Agent Skills and the skills of Claude Code (2025): a folder of instructions the model reads on demand, "progressive disclosure" of context. Here a skill is data on the profile or the call, not a file. |
| Context engineering | Deciding what the model sees on each call: template, addendum, memory, retrieved knowledge, per-call context. | Coined in June 2025 (Tobi Lütke, Andrej Karpathy) as the successor of "prompt engineering". |
| Profile | One assistant configuration: model, allowed tools, approvals, addendum, limits (IAIProfile), chosen per call (D46). | Generic configuration term. The equivalent is an "assistant" (OpenAI Assistants API) or an "agent" (OpenAI Agents SDK). |
| Tenant | The customer whose data and profiles are isolated from every other customer's: IUserContext.tenantId. | Multi-tenancy, SaaS vocabulary of the 2000s (Salesforce era). |
| User context | Who is asking: tenant, user, locale, timezone, permissions (IUserContext). | Plain term. |
| Debug transcript | The exact messages a call sent and received, attached with debug: true (D63). | Plain term. |
3. Agent loop and tools
| Term | In this library | Origin |
|---|---|---|
| Agent | An assistant that can call the application's functions in a loop until it has an answer. In this library an agent is a profile run through run(). | "Rational agent" is classic AI (Russell and Norvig, 1995); the tool-calling LLM agent follows the ReAct paper (Yao et al., October 2022). |
| Agent loop | Call the model; execute the tools it asks for; feed the results back; repeat until the model answers or a limit is reached. runAgentLoop is the library's own loop. | ReAct (2022); every agent SDK has one. |
| Hook | A callback the agent loop calls at a fixed point: the loop owns the flow, the gateway owns the decision (ILLMClientHooks, wired by createRunHooks). | Inversion of control (Johnson and Foote, 1988; Fowler, 2004); the AI SDK's prepareStep, LangChain callbacks. |
| Run | One execution of the agent loop on a thread, from the user's message to the answer, with its pauses: IRun, ERunStatus (RUNNING, PAUSED, DONE, FAILED, CANCELLED), kept by the RunStore. | The OpenAI Assistants API (November 2023); the OpenAI Agents SDK kept the name. |
| Step | One model call plus the execution of the tools it asked for. maxStepsPerRun bounds a run. | The Vercel AI SDK's maxSteps (2024). |
| Tool, function calling | A function of the application the model may ask for, with a name, a description and a zod input schema (IToolDef, defineTool). | OpenAI "function calling" (June 2023); Anthropic and the AI SDK say "tool"; the name stuck. |
| Tool call, tool result | The model's request to run a tool (IToolCallPart) and what came back (IToolResultPart). | Provider API vocabulary (tool_calls, tool_result). |
| Parallel tool calls | A model asking for several tools in one step. The library decides every call first, then runs the allowed ones concurrently and stores the results in request order; parallelToolCalls: false on the profile runs them one after the other (D101). | OpenAI (November 2023); Anthropic disable_parallel_tool_use. |
| Tool choice | Forcing the model to call a given tool, or none. Not in the library today. | OpenAI tool_choice. |
| Tool registry | The fixed set of tools of a gateway instance (ToolRegistry); a call can narrow it, never widen it (D48). | Registry pattern, general software term. |
| Active tools | The subset of the resolved tools offered on one call: activeTools (D48, D66). | The AI SDK's activeTools. |
| Tool timeout, output cap | A tool runs under toolTimeoutMs and its result is cut at maxOutputChars (D60). | Plain terms. |
| Built-in tool | A tool the gateway offers itself, such as the one that asks the user a question; its name starts with dyna_aigw_ (D61, D62, D125). | Plain term; provider APIs use it for their own tools (web search, code execution). |
| Ends run | A tool whose result is the final answer: endsRun (D86). | The AI SDK stops its loop with a stopWhen condition; plain phrasing here. |
| Visible when | A tool offered only in some steps: visibleWhen (D89). | Pydantic AI prepare, OpenAI Agents SDK is_enabled; plain phrasing here. |
| Guarded workflow | A tool hidden until the tools it requires have run in this run (D15, D76, D88). | Plain phrasing for tool dependencies; no single industry name. |
| Agents as tools, handoff | Two ways to let agents cooperate: a tool whose execution runs another profile, or passing the conversation to another agent. Neither is in the library today. | OpenAI Swarm (October 2024) and the Agents SDK (2025) named both. |
4. Tool control and the human
| Term | In this library | Origin |
|---|---|---|
| Policy | The decision on one tool call before it runs: IToolDef.policy, static or a predicate on the input (D87). | Access-control vocabulary (XACML, 2003; Open Policy Agent, 2016). |
| Verdict | The result of a policy: EToolCallVerdict (ALLOW, DENY, NEEDS_APPROVAL). | Plain word, common in moderation and judging APIs. |
| Approval | A tool call that waits for a human's yes: the verdict NEEDS_APPROVAL or the profile's approvals. | Plain word. |
| Human in the loop | A run that stops for a person: approval, review of a result, a question to the user. The HumanGate produces the card for the UI (D40). | Control systems and supervised machine learning; LangGraph made "human-in-the-loop" the agent term (2024). |
| Pause, resume | A run stops with a pending call (ERunOutcomeStatus.PAUSED) and continues later with the human's decision through resume (D36, D69, D78). | LangGraph interrupt (2024); durable-execution vocabulary (Temporal). |
| Pause envelope, pause kind | One card of a pause: a discriminated union by EPauseKind (TOOL_APPROVAL, RESULT_REVIEW, USER_QUESTION) (D51, D52). A pause carries a list of them, envelopes: one per tool call of the step that waits for a human, or the one review (D102). | Plain terms. |
| Resume action, decision | The human's answer to one envelope: IResumeDecision with an EResumeAction (APPROVE, EDIT, REJECT, ANSWER) and the envelope's callId; resume takes one per envelope (D102). | Plain terms. |
| Result review, reviewed answer | The final answer of a run() waits for a human before the user sees it, per profile (reviewAnswers); approved, edited or rejected through resume (D99). | Plain terms; "review the output" in LangGraph's human-in-the-loop vocabulary. |
| User question | The model asks the user instead of guessing, through a built-in tool, in a mode EUserQuestionMode (RADIO, CHECK, FREE_TEXT) (D62, D85). | Also called "clarification" in the literature. |
| Rejection text, denial text | The text the model receives when a call is rejected by the human or denied by the policy (DEFAULT_TOOL_REJECTION_TEXT, DEFAULT_TOOL_DENIAL_TEXT). | Plain terms. |
| Expiry | A pause that is not answered in time ends the run (D75, D80). | Plain term. |
5. Safety
| Term | In this library | Origin |
|---|---|---|
| Guardrails | The checks on the input and on the output of a call, built-in and the application's own (Guardrails, D47). | NVIDIA NeMo Guardrails and Guardrails AI (both 2023); the OpenAI Agents SDK uses the same word (2025). |
| Rule | One guardrail check: inputRules, outputRules. "Rule" means a guardrail; a tool is controlled by a policy, not a rule. | NeMo Guardrails vocabulary. |
| Issue, severity | What a rule reports: IIssue with EIssueSeverity (INFO, WARNING, ERROR); the severity is the only verdict (D49). | Static-analysis vocabulary (linters). |
| Judge, LLM as a judge | A rule that asks a model to classify the text (EJudgeModel), checked, reported and traced (D57, D59, D73). | "LLM-as-a-judge" (Zheng et al., Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena, 2023). |
| Moderation | A judge rule for harmful content. | The OpenAI Moderation API (2022). |
| Prompt injection | Text in the input that tries to override the instructions; a built-in rule rejects known markers. | Named by Simon Willison in September 2022, after Riley Goodside's demonstrations. |
| PII | Personally identifiable information, the subject of a judge rule and of the forgetting methods. | US privacy law and NIST vocabulary (1970s onward). |
| Blocked answer | An answer the output rules rejected; it is not stored and the stream learns it was withdrawn (D70, D74). | Plain term. |
| Erasure, anonymization | Forgetting a user is two methods: delete the data, or keep it without the identity (D54). | GDPR, Article 17 "right to erasure" (2016, in force 2018). |
6. Memory and knowledge
| Term | In this library | Origin |
|---|---|---|
| Working memory | What the model gets from the thread's history, per request: the message window and, with a summary strategy, the notes (IWorkingMemory, D122). | Cognitive psychology (Baddeley and Hitch, 1974); agent frameworks reuse it for the short-term store. |
| Sliding window | The newest turns that fit the budget, verbatim; nothing stored, no helper call (EWorkingMemoryStrategy.SLIDING_WINDOW). | LangChain ConversationBufferWindowMemory; what ChatGPT and Claude do with a conversation. |
| Summary buffer | A rolling summary and facts written by the cheap model, then the last N turns verbatim (EWorkingMemoryStrategy.SUMMARY_BUFFER). | LangChain ConversationSummaryBufferMemory (2023). |
| User memory, user facts | Facts about the user kept across threads (UserMemoryStore, EUserMemoryFactOp) (D56). | ChatGPT "Memory" (February 2024); MemGPT (2023) for the long-term/short-term split. |
| Memory switch | Memory is injected only when the call asks for it (D16, D19, D61, D18). | Plain term. |
| Knowledge base, KB | The application's documents, split and embedded, searched per call (KbRetriever, D21). | Knowledge-management vocabulary of the 1990s. |
| RAG | Retrieval-augmented generation: retrieve chunks, put them in the prompt, generate. | Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020). |
| Chunk, chunking | A piece of a document sized for retrieval (IChunk); the Markdown splitter is the gateway's own (D23). | RAG vocabulary. |
| Embedding | The vector of a text, used for similarity search (D27). | Word embeddings (Bengio et al., 2003; word2vec, 2013). |
| Library, book | The two scope levels of the knowledge base; a book moves without re-embedding (D22). | Plain words. |
| Query rewrite | A model call that turns the user's message into a search query, behind a required switch (QueryRewriter, D24). | Information-retrieval vocabulary; HyDE (2022) is the LLM form. |
| Top-k, minimum score | How many chunks are retrieved and the similarity below which a chunk is dropped (minScore). | Information-retrieval vocabulary. |
| Reranker, hybrid search | A second ranking of the retrieved chunks; keyword plus vector search. Neither is in the library today. | Information-retrieval vocabulary. |
| File pointer | A file is stored by the application and referenced by the gateway (IFileRef, D30, D31). | Plain term. |
7. Operations
| Term | In this library | Origin |
|---|---|---|
| Trace | The record of every model call: request, result, usage, cost, template version, chunk ids, issues (Trace, ETraceKind). | Distributed tracing (Google Dapper, 2010; OpenTelemetry); LLM tracing tools (LangSmith, Langfuse, 2023). |
| Observability | Seeing what the system did from its traces. | Control theory (Kalman, 1960); operations vocabulary since the 2010s. |
| Eval | A fixed set of inputs with expected outputs, re-run after a change. Not in the library today. | Machine-learning evaluation; OpenAI Evals (2023). |
| Background work | Memory and trace writes that run after the answer; whenIdle() waits for them (D17). | Plain term. |
| Lease | One call at a time on a thread (D77). | Distributed systems (Gray and Cheriton, 1989). |
| Cursor pagination | Lists page by an opaque cursor (IPageArgs, IPage) (D37). | Database and API vocabulary. |
8. Software design
| Term | In this library | Origin |
|---|---|---|
| Gateway | One entry point in front of several backends that adds policy, accounting and safety: AIGateway. | The API gateway pattern of microservices (Netflix Zuul, 2013); "AI gateway" products followed in 2023. |
| Facade | The one class the application talks to: AIGateway. | Design Patterns (Gamma et al., 1994). |
| Adapter | A class that fits an external API to an internal interface: the provider adapters, the in-memory stores. | Design Patterns (1994). |
| Dependency injection | The stores and clients are passed in, not constructed inside (IAIGatewayDeps). | Martin Fowler, Inversion of Control Containers and the Dependency Injection pattern (2004). |
| Seam | A place where a behaviour can be replaced without editing the code around it: every store has one (D28, D29). | Michael Feathers, Working Effectively with Legacy Code (2004). |
| Composition root | The one place where the dependencies are wired. | Mark Seemann, Dependency Injection in .NET (2011). |
| Discriminated union | A TypeScript union told apart by one literal field, such as the pause envelope (D51). | Type theory ("tagged union"); TypeScript documentation. |
| Peer dependency | zod is installed by the application, not by the library (D43). | npm vocabulary. |
| Barrel | An index.ts that re-exports a folder's public API. | TypeScript community term. |