mirror of
https://github.com/openclaw/openclaw.git
synced 2026-10-26 10:13:27 +08:00
fix: stop repetitive output from small models (#173844)
* fix: stop repetitive output from small models Send resolved output caps to native Ollama and quarantine long repetitive assistant fragments at the stream owner. Enable detection only for known models up to 32B total parameters, preserve safe history, and stop retries without canceling admitted tool work. Refs #138245, #44965. * fix: recheck replaced assistant snapshots for repetition * fix: classify repetitive terminal snapshots before canceling tools * test(ollama): parse captured request bodies safely * fix(agents): preserve distinct records and cloud output budgets * fix(ollama): preserve cloud output budgets during config defaults * fix(ollama): retain cloud alias output budgets during setup * refactor(ollama): keep config normalization immutable and typed * fix(models): retain total size beside MoE expert tags * fix(ollama): qualify cloud output budgets by route * refactor(plugin-sdk): keep cloud model checks lightweight * fix(ollama): align output budgets with provider route precedence * fix(ollama): preserve remote aliases in app-guided setup * fix(models): preserve parameter counts across catalog projections * fix(models): ignore active-only parameter counts * test(models): build complete catalog overlay fixtures
This commit is contained in:
1 parent
fd2d5d0ae8
commit
b14f705382
76 files changed
+2074
-226
No files matched your search
@@ -76,3 +76,36 @@ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
|
||||
## Hermes Agent
|
||||
|
||||
OpenClaw's live repetition guard adapts the bounded streaming checks, repeated
|
||||
text detection, and sampled exact-period expansion from Hermes Agent.
|
||||
|
||||
- Upstream: https://github.com/NousResearch/hermes-agent
|
||||
- Source: https://github.com/NousResearch/hermes-agent/blob/acc3cada0da86c81efc8aa6c5a291989549ca900/agent/repetition_guard.py
|
||||
- OpenClaw source: `packages/agent-core/src/repetition-guard.ts`
|
||||
- License: MIT
|
||||
- Copyright: Copyright (c) 2025 Nous Research
|
||||
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2025 Nous Research
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -298,6 +298,39 @@ If the model loads cleanly but full agent turns misbehave, check transport first
|
||||
|
||||
5. **Inspect the failing request and server logs.** Check the chat template, context window, memory pressure, and server errors. A successful text-only check does not prove that the model can reliably complete a multi-step agent task.
|
||||
|
||||
## Output budgets and repetition stops
|
||||
|
||||
OpenClaw sends the resolved model output limit to local model servers. Native
|
||||
Ollama receives it as `num_predict`; an explicit request `maxTokens` or authored
|
||||
`params.num_predict` takes precedence. Set realistic `maxTokens` and context
|
||||
limits for custom model entries. Discovered local Ollama models default to 8,192
|
||||
output tokens. Ollama Cloud models use their documented output limit, or their
|
||||
context window when a separate output limit is unavailable. Set
|
||||
`params: { num_predict: -1 }` to restore Ollama's unlimited default; an explicit
|
||||
request output cap still wins.
|
||||
|
||||
Embedded agent runs also watch long text and reasoning from models with a known
|
||||
total parameter count in the **32B class or smaller** (less than 34 billion, to
|
||||
allow for rounded model names). Discovery/catalog metadata takes precedence over
|
||||
a size inferred from the model name. Active-parameter-only MoE names do not establish
|
||||
the total size. The detector stays off for larger models and models whose size is
|
||||
unknown. There is no separate setting.
|
||||
|
||||
The first check starts around 16,000 characters, with increasing intervals and a
|
||||
bounded recent tail. A single repeated line, repeated windows, or a repeated period
|
||||
must dominate the tail; a line-novelty check also protects varied output. Repeated
|
||||
boilerplate lines in distinct multi-line records are not added together to trigger
|
||||
the line check. This heuristic can miss loops and can stop legitimate output with
|
||||
sufficiently dominant repetition. Output limits still bound requests that the
|
||||
detector misses.
|
||||
|
||||
When it triggers, OpenClaw stops the model request and reports a repetition
|
||||
failure. The affected unfinished block is replaced with
|
||||
`[Repetitive output stopped]` in session history, so later turns do not imitate
|
||||
the discarded text. Earlier unaffected blocks and completed tool work remain.
|
||||
Side effects are not retried automatically. Choose a stronger model or request a
|
||||
smaller output, then start a new turn.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **Gateway can't reach the proxy?** `curl http://127.0.0.1:1234/v1/models`.
|
||||
|
||||
@@ -282,7 +282,7 @@ runtime splinters have been removed; bundled-only helpers are private-local.
|
||||
| `plugin-sdk/provider-auth-result` | Private-local after July 2026; Standard OAuth auth-result builder |
|
||||
| `plugin-sdk/provider-env-vars` | Private-local after July 2026; Provider auth env-var lookup helpers |
|
||||
| `plugin-sdk/provider-auth` | `createProviderApiKeyAuthMethod`, `ensureApiKeyFromOptionEnvOrPrompt`, `upsertAuthProfile`, `upsertApiKeyProfile`, `writeOAuthCredentials`, OpenAI Codex auth-import helpers, deprecated `resolveOpenClawAgentDir` compatibility export |
|
||||
| `plugin-sdk/model-ref-parse` | Private-local after July 2026; Lightweight model-reference parsing, provider-id normalization, and `findNormalizedProviderValue` for configured provider lookup without the broad auth runtime |
|
||||
| `plugin-sdk/model-ref-parse` | Private-local after July 2026; Lightweight model-reference parsing, Cloud-source classification with `isCloudModelRef`, provider-id normalization, and `findNormalizedProviderValue` for configured provider lookup without the broad auth runtime |
|
||||
| `plugin-sdk/provider-model-shared` | Private-local after July 2026; `ProviderReplayFamily`, `buildProviderReplayFamilyHooks`, `resolveFamilyForwardCompatModel`, `selectPreferredLocalModelId`, `normalizeModelCompat`, `parseModelRef`, shared replay-policy builders, provider-endpoint helpers, and shared model-id normalization helpers |
|
||||
| `plugin-sdk/provider-catalog-live-runtime` | Private-local after July 2026; Live provider model catalog helpers for guarded `/models`-style discovery: `buildLiveModelProviderConfig`, provider-owned `projectRows`, `buildOpenAICompatibleLiveModels` (the default projection, for provider projections that refine its rows), `fetchLiveProviderModelRows`, `getCachedLiveProviderModelRows`, `fetchLiveProviderModelIds`, `LiveModelCatalogHttpError`, `clearLiveCatalogCacheForTests`, TTL cache, and static fallback |
|
||||
| `plugin-sdk/provider-catalog-runtime` | Provider catalog augmentation runtime hook and plugin-provider registry interfaces for contract tests |
|
||||
|
||||
@@ -164,6 +164,28 @@ sidebarTitle: "Troubleshooting"
|
||||
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Repetitive output stopped">
|
||||
OpenClaw stopped a long repetition-dominated response from a model with a
|
||||
known total size in the 32B class or smaller. It replaced the affected unfinished
|
||||
block in session history with `[Repetitive output stopped]`. Earlier blocks
|
||||
and completed tool work remain; side effects are not retried automatically.
|
||||
|
||||
Try a stronger model, request less output, or split the task into smaller
|
||||
turns. Native Ollama requests also send the resolved model output cap as
|
||||
`num_predict`; lower the model's `maxTokens` if generation runs too long.
|
||||
Discovered local models default to 8,192 output tokens. Ollama Cloud models
|
||||
use their documented output limits, or their context window when a separate
|
||||
output limit is unavailable. Authored `maxTokens` and `params.num_predict`
|
||||
override those defaults. Set `params: { num_predict: -1 }` on a model entry
|
||||
to restore Ollama's unlimited generation default; an explicit request output
|
||||
cap still wins.
|
||||
The detector starts around 16,000 characters and can stop legitimate output
|
||||
with sufficiently dominant repetition. It applies to the 32B class (less than
|
||||
34 billion total parameters) and stays off when total model size is unknown.
|
||||
See [Output budgets and repetition stops](/gateway/local-models#output-budgets-and-repetition-stops).
|
||||
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Kimi or GLM returns garbled symbols">
|
||||
Hosted Kimi/GLM responses that are long, non-linguistic symbol runs are
|
||||
treated as a failed provider call rather than a successful reply, so
|
||||
|
||||
@@ -574,8 +574,9 @@ describe("lmstudio plugin", () => {
|
||||
lmstudio: {
|
||||
models: [
|
||||
{
|
||||
id: "qwen3-8b-instruct",
|
||||
name: "Qwen 3 8B Instruct",
|
||||
id: "compact-chat",
|
||||
name: "Compact Chat",
|
||||
parameterCount: 4_000_000_000,
|
||||
contextWindow: 32768,
|
||||
contextTokens: 8192,
|
||||
reasoning: true,
|
||||
@@ -593,6 +594,7 @@ describe("lmstudio plugin", () => {
|
||||
},
|
||||
{
|
||||
id: "bad-window",
|
||||
parameterCount: -1,
|
||||
contextWindow: Number.POSITIVE_INFINITY,
|
||||
contextTokens: 4096.5,
|
||||
},
|
||||
@@ -621,8 +623,9 @@ describe("lmstudio plugin", () => {
|
||||
).toEqual([
|
||||
{
|
||||
provider: "lmstudio",
|
||||
id: "qwen3-8b-instruct",
|
||||
name: "Qwen 3 8B Instruct",
|
||||
id: "compact-chat",
|
||||
name: "Compact Chat",
|
||||
parameterCount: 4_000_000_000,
|
||||
compat: {
|
||||
supportsUsageInStreaming: true,
|
||||
codeMode: "preferred",
|
||||
|
||||
@@ -52,6 +52,7 @@ type LmstudioReasoningCapabilityWire = {
|
||||
type LmstudioConfiguredCatalogEntry = {
|
||||
id: string;
|
||||
name?: string;
|
||||
parameterCount?: number;
|
||||
contextWindow?: number;
|
||||
contextTokens?: number;
|
||||
reasoning?: boolean;
|
||||
@@ -362,6 +363,7 @@ export function normalizeLmstudioConfiguredCatalogEntry(
|
||||
const name = typeof record.name === "string" && record.name.trim().length > 0 ? record.name : id;
|
||||
const contextWindow = asPositiveSafeInteger(record.contextWindow);
|
||||
const contextTokens = asPositiveSafeInteger(record.contextTokens);
|
||||
const parameterCount = asPositiveSafeInteger(record.parameterCount);
|
||||
const reasoning = typeof record.reasoning === "boolean" ? record.reasoning : undefined;
|
||||
const input = Array.isArray(record.input)
|
||||
? record.input.filter(
|
||||
@@ -373,6 +375,7 @@ export function normalizeLmstudioConfiguredCatalogEntry(
|
||||
return {
|
||||
id,
|
||||
name,
|
||||
...(parameterCount !== undefined ? { parameterCount } : {}),
|
||||
contextWindow,
|
||||
contextTokens,
|
||||
reasoning,
|
||||
|
||||
@@ -3,6 +3,7 @@ import type { ModelDefinitionConfig } from "openclaw/plugin-sdk/provider-model-s
|
||||
import { withServer } from "openclaw/plugin-sdk/test-env";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import plugin from "./index.js";
|
||||
import type { OllamaTagModel } from "./src/provider-models.js";
|
||||
|
||||
const selected = "setup-tools:latest";
|
||||
const configuredModel: ModelDefinitionConfig = {
|
||||
@@ -18,7 +19,7 @@ const configuredModel: ModelDefinitionConfig = {
|
||||
|
||||
async function withOllamaServer(
|
||||
options: {
|
||||
models?: string[];
|
||||
models?: Array<string | OllamaTagModel>;
|
||||
loaded?: string[];
|
||||
show?: Record<string, unknown>;
|
||||
showStatus?: number;
|
||||
@@ -42,7 +43,11 @@ async function withOllamaServer(
|
||||
);
|
||||
} else if (request.url === "/api/tags") {
|
||||
response.end(
|
||||
JSON.stringify({ models: (options.models ?? [selected]).map((name) => ({ name })) }),
|
||||
JSON.stringify({
|
||||
models: (options.models ?? [selected]).map((model) =>
|
||||
typeof model === "string" ? { name: model } : model,
|
||||
),
|
||||
}),
|
||||
);
|
||||
} else if (request.url === "/api/show") {
|
||||
inspected.push((JSON.parse(body) as { model: string }).model);
|
||||
@@ -77,6 +82,35 @@ function guidedSetup() {
|
||||
}
|
||||
|
||||
describe("Ollama app-guided setup HTTP discovery", () => {
|
||||
it.each([
|
||||
{ tag: { remote_host: "https://ollama.com" }, expected: 262_144 },
|
||||
{ tag: { remote_model: "upstream-model" }, expected: 262_144 },
|
||||
{ tag: {}, expected: 8192 },
|
||||
])("preserves alias output budgets for $tag", async ({ tag, expected }) => {
|
||||
const name = "private-alias";
|
||||
await withOllamaServer(
|
||||
{
|
||||
models: [{ name, ...tag }],
|
||||
loaded: [name],
|
||||
show: {
|
||||
model_info: { "model.context_length": 262_144 },
|
||||
capabilities: ["completion", "tools"],
|
||||
},
|
||||
},
|
||||
async (baseUrl) => {
|
||||
const prepared = await guidedSetup().prepare({
|
||||
config: { models: { providers: { ollama: { baseUrl, models: [] } } } },
|
||||
env: {},
|
||||
modelRef: `ollama/${name}`,
|
||||
});
|
||||
expect(prepared?.configPatch?.models?.providers?.ollama?.models?.[0]).toMatchObject({
|
||||
id: name,
|
||||
maxTokens: expected,
|
||||
});
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
it.each(["detect", "prepare"] as const)(
|
||||
"%s finds a loaded model beyond the general catalog cap without inspecting idle models",
|
||||
async (operation) => {
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
import { expectDefined } from "@openclaw/normalization-core";
|
||||
import type { ProviderPlugin } from "openclaw/plugin-sdk/plugin-entry";
|
||||
import { createTestPluginApi } from "openclaw/plugin-sdk/plugin-test-api";
|
||||
import { createPluginRuntimeMock } from "openclaw/plugin-sdk/plugin-test-runtime";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import plugin from "./index.js";
|
||||
|
||||
function registerProvider(): ProviderPlugin {
|
||||
const providers: ProviderPlugin[] = [];
|
||||
plugin.register(
|
||||
createTestPluginApi({
|
||||
id: "ollama",
|
||||
runtime: createPluginRuntimeMock(),
|
||||
registerProvider: (provider) => providers.push(provider),
|
||||
}),
|
||||
);
|
||||
return expectDefined(
|
||||
providers.find((provider) => provider.id === "ollama"),
|
||||
"Ollama provider",
|
||||
);
|
||||
}
|
||||
|
||||
describe("Ollama provider policies", () => {
|
||||
it("classifies incomplete ollama streams as provider failures", () => {
|
||||
const provider = registerProvider();
|
||||
expect(
|
||||
provider?.classifyFailoverReason?.({
|
||||
provider: "ollama",
|
||||
errorMessage: "Ollama API stream ended without a final response",
|
||||
}),
|
||||
).toBe("server_error");
|
||||
expect(
|
||||
provider?.classifyFailoverReason?.({
|
||||
provider: "ollama",
|
||||
errorMessage: "Ollama returned malformed tool arguments",
|
||||
}),
|
||||
).toBeUndefined();
|
||||
});
|
||||
|
||||
it.each(["ollama", "openai-completions"] as const)(
|
||||
"owns replay policy for %s routes",
|
||||
(modelApi) => {
|
||||
const provider = registerProvider();
|
||||
const policy = provider.buildReplayPolicy?.({
|
||||
provider: "ollama",
|
||||
modelApi,
|
||||
modelId: "qwen3:32b",
|
||||
});
|
||||
expect(policy?.sanitizeToolCallIds).toBe(modelApi !== "ollama");
|
||||
expect(policy?.toolCallIdMode).toBe(modelApi === "ollama" ? undefined : "strict");
|
||||
expect(policy?.dropReasoningFromHistory ?? false).toBe(modelApi !== "ollama");
|
||||
},
|
||||
);
|
||||
});
|
||||
@@ -194,7 +194,7 @@ function mockShow(capabilities: string[], contextWindow = 1_048_576) {
|
||||
showMock.mockResolvedValueOnce({ contextWindow, capabilities });
|
||||
}
|
||||
|
||||
function mockInstalledModels(models: Record<string, boolean | undefined>) {
|
||||
function mockInstalledModels(models: Record<string, boolean | undefined>, parameterCount?: number) {
|
||||
modelsMock.mockResolvedValue({
|
||||
reachable: true,
|
||||
models: Object.keys(models).map((name) => ({ name })),
|
||||
@@ -202,6 +202,7 @@ function mockInstalledModels(models: Record<string, boolean | undefined>) {
|
||||
showMock.mockImplementation(async (_baseUrl: string, modelId: string) => ({
|
||||
contextWindow: 32_768,
|
||||
capabilities: models[modelId] ? ["completion", "tools"] : ["completion"],
|
||||
...(parameterCount !== undefined ? { parameterCount } : {}),
|
||||
}));
|
||||
}
|
||||
|
||||
@@ -384,22 +385,6 @@ describe("ollama plugin", () => {
|
||||
expect(modelsMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("classifies incomplete ollama streams as provider failures", () => {
|
||||
const provider = registerProvider();
|
||||
expect(
|
||||
provider?.classifyFailoverReason?.({
|
||||
provider: "ollama",
|
||||
errorMessage: "Ollama API stream ended without a final response",
|
||||
}),
|
||||
).toBe("server_error");
|
||||
expect(
|
||||
provider?.classifyFailoverReason?.({
|
||||
provider: "ollama",
|
||||
errorMessage: "Ollama returned malformed tool arguments",
|
||||
}),
|
||||
).toBeUndefined();
|
||||
});
|
||||
|
||||
it("keeps the agent tool but does not advertise node inference when disabled locally", () => {
|
||||
const registerNodeHostCommand = vi.fn();
|
||||
const registerTool = vi.fn();
|
||||
@@ -450,7 +435,10 @@ describe("ollama plugin", () => {
|
||||
it("discovers and prepares a loaded tool-capable model without pulling it", async () => {
|
||||
const provider = registerProvider();
|
||||
const guided = provider.auth[0].appGuidedSetup;
|
||||
mockInstalledModels({ "embed-only": false, "unknown-tools": undefined, "qwen-tool": true });
|
||||
mockInstalledModels(
|
||||
{ "embed-only": false, "unknown-tools": undefined, "qwen-tool": true },
|
||||
4_000_000_000,
|
||||
);
|
||||
await expect(guided?.detect({ config: {}, env: {} })).resolves.toEqual({
|
||||
modelRef: "ollama/qwen-tool",
|
||||
detail: "qwen-tool at http://127.0.0.1:11434",
|
||||
@@ -470,6 +458,9 @@ describe("ollama plugin", () => {
|
||||
}),
|
||||
});
|
||||
expect(prepared?.configPatch?.models?.mode).toBe("merge");
|
||||
expect(prepared?.configPatch?.models?.providers?.ollama?.models?.[0]).not.toHaveProperty(
|
||||
"parameterCount",
|
||||
);
|
||||
expect(prepared?.configPatch?.models?.providers?.ollama?.apiKey).toBe(OLLAMA_DEFAULT_API_KEY);
|
||||
await expect(
|
||||
guided?.prepare({ config: {}, env: {}, modelRef: "ollama/unknown-tools" }),
|
||||
@@ -778,6 +769,7 @@ describe("ollama plugin", () => {
|
||||
it("reconciles configured refs and incomplete catalog rows without probing complete rows", async () => {
|
||||
showMock.mockResolvedValue({
|
||||
contextWindow: 1_048_576,
|
||||
parameterCount: 4_000_000_000,
|
||||
capabilities: ["completion", "tools", "thinking", "vision"],
|
||||
});
|
||||
const rows = await augmentCatalog(registerProvider(), {
|
||||
@@ -802,6 +794,7 @@ describe("ollama plugin", () => {
|
||||
provider: "ollama",
|
||||
id: "fully-configured",
|
||||
name: "Fully configured",
|
||||
parameterCount: 70_000_000_000,
|
||||
contextWindow: 128_000,
|
||||
reasoning: false,
|
||||
input: ["text"],
|
||||
@@ -826,6 +819,7 @@ describe("ollama plugin", () => {
|
||||
reasoning: true,
|
||||
input: ["text", "image"],
|
||||
contextWindow: 1_048_576,
|
||||
parameterCount: 4_000_000_000,
|
||||
compat: {
|
||||
supportsTools: true,
|
||||
supportsUsageInStreaming: true,
|
||||
@@ -951,6 +945,16 @@ describe("ollama plugin", () => {
|
||||
await expect(provider.prepareDynamicModel?.(context as never)).resolves.toBeUndefined();
|
||||
});
|
||||
|
||||
it("retains a requested model's declared size when show has no other metadata", async () => {
|
||||
const provider = registerProvider();
|
||||
vi.stubEnv("OLLAMA_API_KEY", "ollama-local");
|
||||
mockDiscovery([]);
|
||||
showMock.mockResolvedValueOnce({ parameterCount: 70_000_000_000 });
|
||||
await expect(
|
||||
provider.prepareDynamicModel?.(dynamicContext("custom-4b") as never),
|
||||
).resolves.toMatchObject({ id: "custom-4b", parameterCount: 70_000_000_000 });
|
||||
});
|
||||
|
||||
it.each([
|
||||
{
|
||||
name: "remote",
|
||||
@@ -1062,7 +1066,7 @@ describe("ollama plugin", () => {
|
||||
expect.objectContaining({
|
||||
id: "glm-5.2",
|
||||
contextWindow: 1_000_000,
|
||||
maxTokens: 8192,
|
||||
maxTokens: 131_072,
|
||||
reasoning: true,
|
||||
}),
|
||||
);
|
||||
@@ -1081,7 +1085,7 @@ describe("ollama plugin", () => {
|
||||
provider: "ollama-cloud",
|
||||
id: "glm-5.2",
|
||||
contextWindow: 1_000_000,
|
||||
maxTokens: 8192,
|
||||
maxTokens: 131_072,
|
||||
reasoning: true,
|
||||
});
|
||||
expect(model?.contextTokens).toBeUndefined();
|
||||
@@ -1163,21 +1167,6 @@ describe("ollama plugin", () => {
|
||||
expect(streamFn).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it.each(["ollama", "openai-completions"] as const)(
|
||||
"owns replay policy for %s routes",
|
||||
(modelApi) => {
|
||||
const provider = registerProvider();
|
||||
const policy = provider.buildReplayPolicy?.({
|
||||
provider: "ollama",
|
||||
modelApi,
|
||||
modelId: "qwen3:32b",
|
||||
});
|
||||
expect(policy?.sanitizeToolCallIds).toBe(modelApi !== "ollama");
|
||||
expect(policy?.toolCallIdMode).toBe(modelApi === "ollama" ? undefined : "strict");
|
||||
expect(policy?.dropReasoningFromHistory ?? false).toBe(modelApi !== "ollama");
|
||||
},
|
||||
);
|
||||
|
||||
it("selects cloud native transport with the default URL only for api=ollama", () => {
|
||||
const provider = registerOllamaCloudProvider();
|
||||
const createStream = (api: "ollama" | "openai-completions") =>
|
||||
|
||||
+23
-41
@@ -33,6 +33,7 @@ import {
|
||||
import { createOllamaCloudAuthMethod } from "./src/cloud-auth.js";
|
||||
import {
|
||||
DEFAULT_OLLAMA_EMBEDDING_MODEL,
|
||||
isHostedOllamaCloud,
|
||||
OLLAMA_CLOUD_BASE_URL,
|
||||
OLLAMA_CLOUD_DEFAULT_MODELS,
|
||||
OLLAMA_CLOUD_PROVIDER_ID,
|
||||
@@ -66,14 +67,15 @@ import {
|
||||
capLocalOllamaProviderContext,
|
||||
fetchOllamaModels,
|
||||
fetchLoadedOllamaModelNames,
|
||||
hasOllamaModelShowInfo,
|
||||
isOllamaCloudModel,
|
||||
queryOllamaModelShowInfo,
|
||||
resolveOllamaApiBase,
|
||||
toDynamicOllamaModel,
|
||||
} from "./src/provider-models.js";
|
||||
import {
|
||||
buildAppGuidedOllamaModel,
|
||||
findAvailableOllamaModelName,
|
||||
OLLAMA_APP_GUIDED_MIN_CONTEXT_TOKENS,
|
||||
orderPreferredOllamaModelIds,
|
||||
} from "./src/setup-model-selection.js";
|
||||
import {
|
||||
@@ -235,33 +237,17 @@ async function discoverAppGuidedOllamaModel(
|
||||
candidateId,
|
||||
requestOptions,
|
||||
);
|
||||
const contextWindow = showInfo.contextWindow ?? configuredCloudModel?.contextWindow;
|
||||
const supportsTools =
|
||||
showInfo.capabilities?.includes("tools") ??
|
||||
configuredCloudModel?.compat?.supportsTools === true;
|
||||
const supportsCompletion =
|
||||
showInfo.capabilities?.includes("completion") ?? Boolean(configuredCloudModel);
|
||||
if (
|
||||
!supportsTools ||
|
||||
!supportsCompletion ||
|
||||
contextWindow === undefined ||
|
||||
contextWindow < OLLAMA_APP_GUIDED_MIN_CONTEXT_TOKENS
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
const definition =
|
||||
configuredCloudModel ??
|
||||
buildOllamaModelDefinition(candidateId, contextWindow, showInfo.capabilities, showInfo);
|
||||
model = capLocalOllamaModelContext(
|
||||
{
|
||||
...definition,
|
||||
thinkingLevelMap: definition.thinkingLevelMap ?? showInfo.thinkingLevelMap,
|
||||
contextWindow,
|
||||
compat: { ...definition.compat, supportsTools: true },
|
||||
model = buildAppGuidedOllamaModel({
|
||||
baseUrl: connection.baseUrl,
|
||||
model: installedModels.find((candidate) => candidate.name === candidateId) ?? {
|
||||
name: candidateId,
|
||||
},
|
||||
connection.baseUrl,
|
||||
);
|
||||
break;
|
||||
showInfo,
|
||||
configuredCloudModel,
|
||||
});
|
||||
if (model) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (!model) {
|
||||
return null;
|
||||
@@ -298,7 +284,8 @@ function needsOllamaCatalogMetadata(entry: ProviderAugmentModelCatalogContext["e
|
||||
!hasContextLimit ||
|
||||
entry.reasoning === undefined ||
|
||||
entry.input === undefined ||
|
||||
entry.compat === undefined
|
||||
entry.compat === undefined ||
|
||||
entry.parameterCount === undefined
|
||||
);
|
||||
}
|
||||
|
||||
@@ -480,11 +467,7 @@ async function buildOllamaCloudProvider(apiKey?: string): Promise<ModelProviderC
|
||||
OLLAMA_GLM52_CLOUD_MODEL_ID,
|
||||
{ apiKey },
|
||||
);
|
||||
if (
|
||||
typeof showInfo.contextWindow !== "number" &&
|
||||
(showInfo.capabilities?.length ?? 0) === 0 &&
|
||||
!showInfo.thinkingLevelMap
|
||||
) {
|
||||
if (!hasOllamaModelShowInfo(showInfo)) {
|
||||
return discovered;
|
||||
}
|
||||
const defaultModel = OLLAMA_CLOUD_DEFAULT_MODELS.find(
|
||||
@@ -501,7 +484,7 @@ async function buildOllamaCloudProvider(apiKey?: string): Promise<ModelProviderC
|
||||
defaultModel.id,
|
||||
showInfo.contextWindow ?? defaultModel.contextWindow,
|
||||
showInfo.capabilities ?? [...defaultModel.capabilities],
|
||||
showInfo,
|
||||
{ ...showInfo, remote: true },
|
||||
),
|
||||
],
|
||||
};
|
||||
@@ -518,18 +501,14 @@ async function resolveRequestedDynamicOllamaModel(params: {
|
||||
const showInfo = params.showApiKey
|
||||
? await queryOllamaModelShowInfo(showBaseUrl, params.modelId, { apiKey: params.showApiKey })
|
||||
: await queryOllamaModelShowInfo(showBaseUrl, params.modelId);
|
||||
if (
|
||||
typeof showInfo.contextWindow !== "number" &&
|
||||
(showInfo.capabilities?.length ?? 0) === 0 &&
|
||||
!showInfo.thinkingLevelMap
|
||||
) {
|
||||
if (!hasOllamaModelShowInfo(showInfo)) {
|
||||
return undefined;
|
||||
}
|
||||
const definition = buildOllamaModelDefinition(
|
||||
params.modelId,
|
||||
showInfo.contextWindow,
|
||||
showInfo.capabilities,
|
||||
showInfo,
|
||||
{ ...showInfo, remote: isHostedOllamaCloud(showBaseUrl) },
|
||||
);
|
||||
const model = params.capContextTokens
|
||||
? capLocalOllamaModelContext(definition, showBaseUrl)
|
||||
@@ -602,6 +581,7 @@ async function augmentConfiguredOllamaCatalogModels(params: {
|
||||
input: requested.input,
|
||||
contextWindow: requested.contextWindow,
|
||||
contextTokens: requested.contextTokens,
|
||||
parameterCount: requested.parameterCount,
|
||||
thinkingLevelMap: requested.thinkingLevelMap,
|
||||
compat: requested.compat,
|
||||
}
|
||||
@@ -793,7 +773,9 @@ export default definePluginEntry({
|
||||
...discovered.existing,
|
||||
...discovered.provider,
|
||||
...(discovered.ownerValue ? { apiKey: discovered.ownerValue } : {}),
|
||||
models: discovered.provider.models,
|
||||
models: discovered.provider.models.map(
|
||||
({ parameterCount: _parameterCount, ...model }) => model,
|
||||
),
|
||||
},
|
||||
},
|
||||
},
|
||||
|
||||
@@ -92,7 +92,7 @@
|
||||
"reasoning": true,
|
||||
"input": ["text", "image"],
|
||||
"contextWindow": 262144,
|
||||
"maxTokens": 8192,
|
||||
"maxTokens": 262144,
|
||||
"compat": {
|
||||
"supportsTools": true,
|
||||
"supportsUsageInStreaming": true
|
||||
@@ -103,7 +103,7 @@
|
||||
"reasoning": true,
|
||||
"input": ["text", "image"],
|
||||
"contextWindow": 262144,
|
||||
"maxTokens": 8192,
|
||||
"maxTokens": 262144,
|
||||
"compat": {
|
||||
"supportsTools": true,
|
||||
"supportsUsageInStreaming": true
|
||||
@@ -119,7 +119,7 @@
|
||||
"cacheRead": 0.3
|
||||
},
|
||||
"contextWindow": 1048576,
|
||||
"maxTokens": 8192,
|
||||
"maxTokens": 1048576,
|
||||
"compat": {
|
||||
"supportsTools": true,
|
||||
"supportsUsageInStreaming": true,
|
||||
@@ -131,7 +131,7 @@
|
||||
"reasoning": true,
|
||||
"input": ["text"],
|
||||
"contextWindow": 1048576,
|
||||
"maxTokens": 8192,
|
||||
"maxTokens": 1048576,
|
||||
"compat": {
|
||||
"supportsTools": true,
|
||||
"supportsUsageInStreaming": true,
|
||||
@@ -143,7 +143,7 @@
|
||||
"reasoning": true,
|
||||
"input": ["text"],
|
||||
"contextWindow": 1048576,
|
||||
"maxTokens": 8192,
|
||||
"maxTokens": 1048576,
|
||||
"compat": {
|
||||
"supportsTools": true,
|
||||
"supportsUsageInStreaming": true
|
||||
@@ -154,7 +154,7 @@
|
||||
"reasoning": true,
|
||||
"input": ["text", "image"],
|
||||
"contextWindow": 262144,
|
||||
"maxTokens": 8192,
|
||||
"maxTokens": 262144,
|
||||
"compat": {
|
||||
"supportsTools": true,
|
||||
"supportsUsageInStreaming": true
|
||||
@@ -165,7 +165,7 @@
|
||||
"reasoning": true,
|
||||
"input": ["text", "image"],
|
||||
"contextWindow": 262144,
|
||||
"maxTokens": 8192,
|
||||
"maxTokens": 262144,
|
||||
"compat": {
|
||||
"supportsTools": true,
|
||||
"supportsUsageInStreaming": true
|
||||
@@ -176,7 +176,7 @@
|
||||
"reasoning": true,
|
||||
"input": ["text"],
|
||||
"contextWindow": 1000000,
|
||||
"maxTokens": 8192,
|
||||
"maxTokens": 131072,
|
||||
"compat": {
|
||||
"supportsTools": true,
|
||||
"supportsUsageInStreaming": true,
|
||||
@@ -188,7 +188,7 @@
|
||||
"reasoning": true,
|
||||
"input": ["text"],
|
||||
"contextWindow": 131072,
|
||||
"maxTokens": 8192,
|
||||
"maxTokens": 131072,
|
||||
"compat": {
|
||||
"supportsTools": true,
|
||||
"supportsUsageInStreaming": true
|
||||
@@ -199,7 +199,7 @@
|
||||
"reasoning": true,
|
||||
"input": ["text"],
|
||||
"contextWindow": 131072,
|
||||
"maxTokens": 8192,
|
||||
"maxTokens": 131072,
|
||||
"compat": {
|
||||
"supportsTools": true,
|
||||
"supportsUsageInStreaming": true
|
||||
@@ -210,7 +210,7 @@
|
||||
"reasoning": true,
|
||||
"input": ["text"],
|
||||
"contextWindow": 196608,
|
||||
"maxTokens": 8192,
|
||||
"maxTokens": 204800,
|
||||
"compat": {
|
||||
"supportsTools": true,
|
||||
"supportsUsageInStreaming": true
|
||||
@@ -221,7 +221,7 @@
|
||||
"reasoning": true,
|
||||
"input": ["text", "image"],
|
||||
"contextWindow": 524288,
|
||||
"maxTokens": 8192,
|
||||
"maxTokens": 524288,
|
||||
"compat": {
|
||||
"supportsTools": true,
|
||||
"supportsUsageInStreaming": true
|
||||
@@ -232,7 +232,7 @@
|
||||
"reasoning": false,
|
||||
"input": ["text", "image"],
|
||||
"contextWindow": 262144,
|
||||
"maxTokens": 8192,
|
||||
"maxTokens": 262144,
|
||||
"compat": {
|
||||
"supportsTools": true,
|
||||
"supportsUsageInStreaming": true
|
||||
@@ -243,7 +243,7 @@
|
||||
"reasoning": true,
|
||||
"input": ["text"],
|
||||
"contextWindow": 262144,
|
||||
"maxTokens": 8192,
|
||||
"maxTokens": 262144,
|
||||
"compat": {
|
||||
"supportsTools": true,
|
||||
"supportsUsageInStreaming": true
|
||||
@@ -254,7 +254,7 @@
|
||||
"reasoning": true,
|
||||
"input": ["text"],
|
||||
"contextWindow": 262144,
|
||||
"maxTokens": 8192,
|
||||
"maxTokens": 262144,
|
||||
"compat": {
|
||||
"supportsTools": true,
|
||||
"supportsUsageInStreaming": true
|
||||
@@ -265,7 +265,7 @@
|
||||
"reasoning": true,
|
||||
"input": ["text"],
|
||||
"contextWindow": 262144,
|
||||
"maxTokens": 8192,
|
||||
"maxTokens": 262144,
|
||||
"compat": {
|
||||
"supportsTools": true,
|
||||
"supportsUsageInStreaming": true
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
import { isCloudModelRef } from "@openclaw/model-catalog-core/model-catalog-refs";
|
||||
import {
|
||||
findNormalizedProviderValue,
|
||||
isCloudModelRef,
|
||||
normalizeProviderId,
|
||||
} from "@openclaw/model-catalog-core/provider-id";
|
||||
} from "openclaw/plugin-sdk/model-ref-parse";
|
||||
import type {
|
||||
ProviderDefaultThinkingPolicyContext,
|
||||
ProviderNormalizeResolvedModelContext,
|
||||
@@ -14,8 +14,12 @@ import type {
|
||||
} from "openclaw/plugin-sdk/provider-model-types";
|
||||
import {
|
||||
OLLAMA_CLOUD_PROVIDER_ID,
|
||||
OLLAMA_CLOUD_BASE_URL,
|
||||
isOllamaCloudOrigin,
|
||||
isHostedOllamaCloud,
|
||||
OLLAMA_DEFAULT_BASE_URL,
|
||||
resolveOllamaCloudDefaultModel,
|
||||
resolveOllamaOutputBudget,
|
||||
} from "./src/defaults.js";
|
||||
import { supportsOllamaCloudFullThinkingEffort } from "./src/model-reasoning.js";
|
||||
import { readProviderBaseUrl, resolveOllamaBaseUrlForRun } from "./src/provider-base-url.js";
|
||||
@@ -90,15 +94,39 @@ export function normalizeConfig({
|
||||
providerConfig: OllamaProviderConfigDraft;
|
||||
}): OllamaProviderConfigDraft {
|
||||
const normalizedProviderId = provider.trim().toLowerCase();
|
||||
if (normalizedProviderId !== "ollama") {
|
||||
if (normalizedProviderId !== "ollama" && normalizedProviderId !== OLLAMA_CLOUD_PROVIDER_ID) {
|
||||
return providerConfig;
|
||||
}
|
||||
|
||||
return {
|
||||
...providerConfig,
|
||||
baseUrl: providerConfig.baseUrl?.trim() ? providerConfig.baseUrl : OLLAMA_DEFAULT_BASE_URL,
|
||||
models: providerConfig.models ?? [],
|
||||
};
|
||||
const baseUrl = providerConfig.baseUrl?.trim()
|
||||
? providerConfig.baseUrl
|
||||
: normalizedProviderId === OLLAMA_CLOUD_PROVIDER_ID
|
||||
? OLLAMA_CLOUD_BASE_URL
|
||||
: OLLAMA_DEFAULT_BASE_URL;
|
||||
const models: ModelProviderConfig["models"] = [];
|
||||
for (const model of providerConfig.models ?? []) {
|
||||
const cloudModel = resolveOllamaCloudDefaultModel(model.id);
|
||||
const remote =
|
||||
isHostedOllamaCloud(baseUrl) || normalizedProviderId === OLLAMA_CLOUD_PROVIDER_ID;
|
||||
if (
|
||||
model.maxTokens !== undefined ||
|
||||
(model.contextWindow === undefined && !cloudModel) ||
|
||||
(!remote && !isCloudModelRef(model.id) && !cloudModel)
|
||||
) {
|
||||
models.push(model);
|
||||
continue;
|
||||
}
|
||||
models.push({
|
||||
...model,
|
||||
maxTokens:
|
||||
providerConfig.maxTokens ??
|
||||
resolveOllamaOutputBudget({
|
||||
modelId: model.id,
|
||||
contextWindow: model.contextWindow,
|
||||
remote,
|
||||
}),
|
||||
});
|
||||
}
|
||||
return { ...providerConfig, baseUrl, models };
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import { isCloudModelRef } from "openclaw/plugin-sdk/model-ref-parse";
|
||||
|
||||
export const OLLAMA_DEFAULT_BASE_URL = "http://127.0.0.1:11434";
|
||||
export const OLLAMA_DEFAULT_API_KEY = "ollama-local";
|
||||
const OLLAMA_DOCKER_HOST_BASE_URL = "http://host.docker.internal:11434";
|
||||
@@ -24,21 +26,28 @@ export const OLLAMA_CLOUD_DEFAULT_MODELS = [
|
||||
{
|
||||
id: "minimax-m2.7",
|
||||
contextWindow: 196_608,
|
||||
// https://platform.minimax.io/docs/api-reference/text-chat
|
||||
maxTokens: 204_800,
|
||||
capabilities: ["completion", "thinking", "tools"],
|
||||
},
|
||||
{
|
||||
id: "minimax-m3",
|
||||
contextWindow: 524_288,
|
||||
maxTokens: 524_288,
|
||||
capabilities: ["completion", "thinking", "tools", "vision"],
|
||||
},
|
||||
{
|
||||
id: "kimi-k3",
|
||||
contextWindow: 1_048_576,
|
||||
// No separate output limit: https://ollama.com/library/kimi-k3
|
||||
maxTokens: 1_048_576,
|
||||
capabilities: ["completion", "thinking", "tools", "vision"],
|
||||
},
|
||||
{
|
||||
id: OLLAMA_GLM52_CLOUD_MODEL_ID,
|
||||
contextWindow: 1_000_000,
|
||||
// https://docs.z.ai/guides/llm/glm-5.2
|
||||
maxTokens: 131_072,
|
||||
capabilities: ["completion", "thinking", "tools"],
|
||||
},
|
||||
] as const;
|
||||
@@ -51,6 +60,23 @@ export function normalizeOllamaCloudModelId(modelId: string): string {
|
||||
.replace(/(?::cloud|-cloud)$/, "");
|
||||
}
|
||||
|
||||
export function resolveOllamaCloudDefaultModel(modelId: string) {
|
||||
const normalized = normalizeOllamaCloudModelId(modelId);
|
||||
return OLLAMA_CLOUD_DEFAULT_MODELS.find((model) => model.id === normalized);
|
||||
}
|
||||
|
||||
export function resolveOllamaOutputBudget(params: {
|
||||
modelId: string;
|
||||
contextWindow?: number;
|
||||
remote?: boolean;
|
||||
}): number {
|
||||
return params.remote || isCloudModelRef(params.modelId)
|
||||
? (resolveOllamaCloudDefaultModel(params.modelId)?.maxTokens ??
|
||||
params.contextWindow ??
|
||||
OLLAMA_DEFAULT_CONTEXT_WINDOW)
|
||||
: OLLAMA_DEFAULT_MAX_TOKENS;
|
||||
}
|
||||
|
||||
export const OLLAMA_DEFAULT_CONTEXT_WINDOW = 128000;
|
||||
export const OLLAMA_LOCAL_CONTEXT_TOKENS = 32_768;
|
||||
export const OLLAMA_DEFAULT_MAX_TOKENS = 8192;
|
||||
|
||||
@@ -315,8 +315,7 @@ export async function resolveOllamaDiscoveryResult(params: {
|
||||
...(discoveryApiKey ? { apiKey: discoveryApiKey } : {}),
|
||||
});
|
||||
const api = explicit?.api ?? provider.api;
|
||||
// Keep configured settings, including tool workarounds, while filling missing native
|
||||
// thinking metadata so mandatory-thinking models keep reasoning out of answer text.
|
||||
// Keep configured settings while filling provider-owned model size and native thinking metadata.
|
||||
const configuredModels = new Map(explicit?.models?.map((model) => [model.id, model]));
|
||||
return {
|
||||
...(modelIds
|
||||
@@ -339,11 +338,23 @@ export async function resolveOllamaDiscoveryResult(params: {
|
||||
const sameEndpoint =
|
||||
!configured.baseUrl ||
|
||||
resolveOllamaApiBase(configured.baseUrl) === resolveOllamaApiBase(provider.baseUrl);
|
||||
return (configured.api ?? api) === "ollama" &&
|
||||
sameEndpoint &&
|
||||
configured.thinkingLevelMap === undefined &&
|
||||
model.thinkingLevelMap
|
||||
? [{ ...configured, thinkingLevelMap: model.thinkingLevelMap }]
|
||||
if (!sameEndpoint) {
|
||||
return [];
|
||||
}
|
||||
const thinkingLevelMap =
|
||||
(configured.api ?? api) === "ollama" && configured.thinkingLevelMap === undefined
|
||||
? model.thinkingLevelMap
|
||||
: undefined;
|
||||
return thinkingLevelMap || model.parameterCount !== undefined
|
||||
? [
|
||||
{
|
||||
...configured,
|
||||
...(thinkingLevelMap ? { thinkingLevelMap } : {}),
|
||||
...(model.parameterCount !== undefined
|
||||
? { parameterCount: model.parameterCount }
|
||||
: {}),
|
||||
},
|
||||
]
|
||||
: [];
|
||||
}
|
||||
return [
|
||||
|
||||
@@ -14,10 +14,45 @@ import {
|
||||
fetchLoadedOllamaModelNames,
|
||||
fetchOllamaModels,
|
||||
queryOllamaModelShowInfo,
|
||||
toDynamicOllamaModel,
|
||||
type OllamaTagModel,
|
||||
} from "./provider-models.js";
|
||||
|
||||
describe("ollama provider models", () => {
|
||||
it.each([
|
||||
{ tagSize: "1.5B", expected: 1_500_000_000 },
|
||||
{ tagSize: "494M", expected: 494_000_000 },
|
||||
{ tagSize: "1.2T", expected: 1_200_000_000_000 },
|
||||
{ tagSize: "4B", showCount: 70_000_000_000, expected: 70_000_000_000 },
|
||||
{ showSize: "32.8B", expected: 32_800_000_000 },
|
||||
{ tagSize: "unknown", expected: undefined },
|
||||
])(
|
||||
"keeps total parameter metadata through discovery: %j",
|
||||
async ({ tagSize, showCount, showSize, expected }) => {
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
vi.fn(async (input: string | URL | Request) =>
|
||||
requestUrl(input).endsWith("/api/tags")
|
||||
? jsonResponse({
|
||||
models: [{ name: "custom-model", details: { parameter_size: tagSize } }],
|
||||
})
|
||||
: jsonResponse({
|
||||
capabilities: ["completion"],
|
||||
model_info: { "general.parameter_count": showCount },
|
||||
details: { parameter_size: showSize },
|
||||
}),
|
||||
),
|
||||
);
|
||||
const provider = await buildOllamaProvider("http://127.0.0.1:11434");
|
||||
const model = expectDefined(provider.models[0], "discovered model");
|
||||
expect(model.parameterCount).toBe(expected);
|
||||
expect(
|
||||
toDynamicOllamaModel({ provider: "ollama", providerConfig: provider, model })
|
||||
.parameterCount,
|
||||
).toBe(expected);
|
||||
},
|
||||
);
|
||||
|
||||
afterEach(() => {
|
||||
vi.unstubAllGlobals();
|
||||
});
|
||||
@@ -121,16 +156,69 @@ describe("ollama provider models", () => {
|
||||
expect(fetchMock).toHaveBeenCalledTimes(4);
|
||||
});
|
||||
|
||||
it("resolves known cloud context windows for bare and :cloud model refs", () => {
|
||||
// A suffixed ref must not silently drop to the generic default when live
|
||||
// inspection is unavailable; both spellings name the same cloud model.
|
||||
for (const modelId of ["kimi-k3", "kimi-k3:cloud"]) {
|
||||
expect(buildOllamaModelDefinition(modelId)).toEqual(
|
||||
expect.objectContaining({ id: modelId, contextWindow: 1_048_576 }),
|
||||
);
|
||||
it.each([
|
||||
{ id: "minimax-m2.7", contextWindow: 196_608, maxTokens: 204_800 },
|
||||
{ id: "minimax-m3", contextWindow: 524_288, maxTokens: 524_288 },
|
||||
{ id: "kimi-k3", contextWindow: 1_048_576, maxTokens: 1_048_576 },
|
||||
{ id: "glm-5.2", contextWindow: 1_000_000, maxTokens: 131_072 },
|
||||
])("keeps the cloud output budget for $id aliases", ({ id, contextWindow, maxTokens }) => {
|
||||
for (const modelId of [id, `${id}:cloud`, `${id}-cloud`]) {
|
||||
expect(
|
||||
buildOllamaModelDefinition(modelId, undefined, undefined, { remote: true }),
|
||||
).toMatchObject({
|
||||
id: modelId,
|
||||
contextWindow,
|
||||
maxTokens,
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
it.each([
|
||||
{ name: "local-model", baseUrl: "http://127.0.0.1:11434", maxTokens: 8192 },
|
||||
{ name: "minimax-m3", baseUrl: "http://127.0.0.1:11434", maxTokens: 8192 },
|
||||
{ name: "kimi-k3", baseUrl: "http://127.0.0.1:11434", maxTokens: 8192 },
|
||||
{ name: "minimax-m3", baseUrl: "https://ollama.com", maxTokens: 524_288 },
|
||||
{ name: "kimi-k3", baseUrl: "https://ollama.com", maxTokens: 1_048_576 },
|
||||
{ name: "glm-5.2:cloud", baseUrl: "http://127.0.0.1:11434", maxTokens: 131_072 },
|
||||
{ name: "unknown:cloud", baseUrl: "http://127.0.0.1:11434", maxTokens: 262_144 },
|
||||
{ name: "unknown", baseUrl: "https://ollama.com", maxTokens: 262_144 },
|
||||
{
|
||||
name: "remote-stub",
|
||||
baseUrl: "http://127.0.0.1:11434",
|
||||
remote_host: "https://ollama.com",
|
||||
maxTokens: 262_144,
|
||||
},
|
||||
{
|
||||
name: "remote-model-stub",
|
||||
baseUrl: "http://127.0.0.1:11434",
|
||||
remote_model: "upstream-model",
|
||||
maxTokens: 262_144,
|
||||
},
|
||||
])(
|
||||
"discovers $name output limits at $baseUrl",
|
||||
async ({ name, baseUrl, remote_host, remote_model, maxTokens }) => {
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
vi.fn(async (input: string | URL | Request) =>
|
||||
requestUrl(input).endsWith("/api/tags")
|
||||
? jsonResponse({ models: [{ name, remote_host, remote_model }] })
|
||||
: jsonResponse({
|
||||
capabilities: ["completion"],
|
||||
model_info: { "test.context_length": 262_144 },
|
||||
}),
|
||||
),
|
||||
);
|
||||
const provider = await buildOllamaProvider(baseUrl);
|
||||
const model = expectDefined(provider.models[0], "discovered model");
|
||||
expect(
|
||||
toDynamicOllamaModel({ provider: "ollama", providerConfig: provider, model }),
|
||||
).toMatchObject({
|
||||
id: name,
|
||||
maxTokens,
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("uses Modelfile num_ctx when it expands the discovered context window", async () => {
|
||||
const models: OllamaTagModel[] = [{ name: "llama3-32k:latest" }];
|
||||
const fetchMock = vi.fn(async () =>
|
||||
|
||||
@@ -17,9 +17,9 @@ import {
|
||||
OLLAMA_DEFAULT_BASE_URL,
|
||||
OLLAMA_DEFAULT_CONTEXT_WINDOW,
|
||||
OLLAMA_DEFAULT_COST,
|
||||
OLLAMA_DEFAULT_MAX_TOKENS,
|
||||
OLLAMA_LOCAL_CONTEXT_TOKENS,
|
||||
normalizeOllamaCloudModelId,
|
||||
resolveOllamaCloudDefaultModel,
|
||||
resolveOllamaOutputBudget,
|
||||
} from "./defaults.js";
|
||||
import {
|
||||
buildOllamaThinkingLevelMap,
|
||||
@@ -98,6 +98,7 @@ export type OllamaModelShowInfo = {
|
||||
contextWindow?: number;
|
||||
capabilities?: string[];
|
||||
thinkingLevelMap?: ModelDefinitionConfig["thinkingLevelMap"];
|
||||
parameterCount?: number;
|
||||
/** Distinguishes a failed request from a successful response that omitted capabilities. */
|
||||
showInspectionFailed?: boolean;
|
||||
};
|
||||
@@ -111,9 +112,14 @@ export const mergeOllamaModelShowInfo = (
|
||||
model.capabilities !== undefined &&
|
||||
isOllamaRemoteModel(model) &&
|
||||
!isOllamaEmbeddingOnlyModel(model);
|
||||
const parameterCount =
|
||||
info.parameterCount ??
|
||||
model.parameterCount ??
|
||||
parseOllamaParameterSize(model.details?.parameter_size);
|
||||
return {
|
||||
...model,
|
||||
...info,
|
||||
...(parameterCount !== undefined ? { parameterCount } : {}),
|
||||
contextWindow: info.contextWindow ?? model.contextWindow ?? model.details?.context_length,
|
||||
capabilities: incomplete
|
||||
? [
|
||||
@@ -167,14 +173,28 @@ function setOllamaModelShowCacheEntry(key: string, value: Promise<OllamaModelSho
|
||||
ollamaModelShowInfoCache.set(key, value);
|
||||
}
|
||||
|
||||
function hasCachedOllamaModelShowInfo(info: OllamaModelShowInfo): boolean {
|
||||
export function hasOllamaModelShowInfo(info: OllamaModelShowInfo): boolean {
|
||||
return (
|
||||
info.parameterCount !== undefined ||
|
||||
info.contextWindow !== undefined ||
|
||||
(info.capabilities?.length ?? 0) > 0 ||
|
||||
info.thinkingLevelMap !== undefined
|
||||
);
|
||||
}
|
||||
|
||||
function parseOllamaParameterSize(value: unknown): number | undefined {
|
||||
const match =
|
||||
typeof value === "string" ? value.trim().match(/^(\d+(?:\.\d+)?)\s*([KMBT])$/i) : undefined;
|
||||
const magnitude = match?.[1];
|
||||
const unit = match?.[2];
|
||||
if (!magnitude || !unit) {
|
||||
return undefined;
|
||||
}
|
||||
const exponent = "KMBT".indexOf(unit.toUpperCase()) + 1;
|
||||
const count = Math.round(Number(magnitude) * 1000 ** exponent);
|
||||
return Number.isSafeInteger(count) && count > 0 ? count : undefined;
|
||||
}
|
||||
|
||||
function parseOllamaNumCtxParameter(parameters: unknown): number | undefined {
|
||||
if (typeof parameters !== "string" || !parameters.trim()) {
|
||||
return undefined;
|
||||
@@ -231,6 +251,7 @@ export async function readOllamaModelShowInfo(
|
||||
}
|
||||
const data = await readProviderJsonResponse<{
|
||||
model_info?: Record<string, unknown>;
|
||||
details?: OllamaTagModel["details"];
|
||||
capabilities?: unknown;
|
||||
parameters?: unknown;
|
||||
thinking?: unknown;
|
||||
@@ -264,8 +285,20 @@ export async function readOllamaModelShowInfo(
|
||||
)
|
||||
: undefined;
|
||||
|
||||
const rawParameterCount = data.model_info?.["general.parameter_count"];
|
||||
const parameterCount =
|
||||
typeof rawParameterCount === "number" &&
|
||||
Number.isSafeInteger(rawParameterCount) &&
|
||||
rawParameterCount > 0
|
||||
? rawParameterCount
|
||||
: parseOllamaParameterSize(data.details?.parameter_size);
|
||||
const thinkingLevelMap = await buildOllamaThinkingLevelMap(modelName, data.thinking);
|
||||
return { contextWindow, capabilities, ...(thinkingLevelMap ? { thinkingLevelMap } : {}) };
|
||||
return {
|
||||
contextWindow,
|
||||
capabilities,
|
||||
...(parameterCount !== undefined ? { parameterCount } : {}),
|
||||
...(thinkingLevelMap ? { thinkingLevelMap } : {}),
|
||||
};
|
||||
} finally {
|
||||
await release();
|
||||
}
|
||||
@@ -302,7 +335,7 @@ async function queryOllamaModelShowInfoCached(
|
||||
}
|
||||
|
||||
const pending = queryOllamaModelShowInfo(normalizedApiBase, model.name, opts).then((result) => {
|
||||
if (!hasCachedOllamaModelShowInfo(result)) {
|
||||
if (!hasOllamaModelShowInfo(result)) {
|
||||
ollamaModelShowInfoCache.delete(cacheKey);
|
||||
}
|
||||
return result;
|
||||
@@ -399,18 +432,6 @@ export function isOllamaRemoteModel(model: OllamaTagModel): boolean {
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Cloud models are referenced both bare (`kimi-k3`) and suffixed (`kimi-k3:cloud`).
|
||||
* Both spellings must reach the same known context window, or a suffixed ref silently
|
||||
* falls back to the generic default whenever live inspection is unavailable.
|
||||
*/
|
||||
function resolveOllamaCloudDefaultModel(
|
||||
modelId: string,
|
||||
): (typeof OLLAMA_CLOUD_DEFAULT_MODELS)[number] | undefined {
|
||||
const normalized = normalizeOllamaCloudModelId(modelId);
|
||||
return OLLAMA_CLOUD_DEFAULT_MODELS.find((model) => model.id === normalized);
|
||||
}
|
||||
|
||||
export function isReasoningModelHeuristic(modelId: string): boolean {
|
||||
return /r1|reasoning|think|reason/i.test(modelId);
|
||||
}
|
||||
@@ -419,8 +440,11 @@ export function buildOllamaModelDefinition(
|
||||
modelId: string,
|
||||
contextWindow?: number,
|
||||
capabilities?: string[],
|
||||
opts?: OllamaModelShowInfo,
|
||||
opts?: OllamaModelShowInfo & { remote?: boolean },
|
||||
): ModelDefinitionConfig {
|
||||
const cloudModel = resolveOllamaCloudDefaultModel(modelId);
|
||||
const resolvedContext =
|
||||
contextWindow ?? cloudModel?.contextWindow ?? OLLAMA_DEFAULT_CONTEXT_WINDOW;
|
||||
return {
|
||||
id: modelId,
|
||||
name: modelId,
|
||||
@@ -434,11 +458,13 @@ export function buildOllamaModelDefinition(
|
||||
: capabilities.includes("thinking")),
|
||||
input: capabilities?.includes("vision") ? ["text", "image"] : ["text"],
|
||||
cost: OLLAMA_DEFAULT_COST,
|
||||
contextWindow:
|
||||
contextWindow ??
|
||||
resolveOllamaCloudDefaultModel(modelId)?.contextWindow ??
|
||||
OLLAMA_DEFAULT_CONTEXT_WINDOW,
|
||||
maxTokens: OLLAMA_DEFAULT_MAX_TOKENS,
|
||||
contextWindow: resolvedContext,
|
||||
maxTokens: resolveOllamaOutputBudget({
|
||||
modelId,
|
||||
contextWindow: resolvedContext,
|
||||
remote: opts?.remote,
|
||||
}),
|
||||
...(opts?.parameterCount !== undefined ? { parameterCount: opts.parameterCount } : {}),
|
||||
...(opts?.thinkingLevelMap ? { thinkingLevelMap: opts.thinkingLevelMap } : {}),
|
||||
compat: {
|
||||
supportsTools: capabilities?.includes("tools") ?? opts?.showInspectionFailed !== true,
|
||||
@@ -452,7 +478,9 @@ export function buildDefaultOllamaCloudModelDefinition(
|
||||
model: (typeof OLLAMA_CLOUD_DEFAULT_MODELS)[number],
|
||||
): ModelDefinitionConfig {
|
||||
return {
|
||||
...buildOllamaModelDefinition(model.id, model.contextWindow, [...model.capabilities]),
|
||||
...buildOllamaModelDefinition(model.id, model.contextWindow, [...model.capabilities], {
|
||||
remote: true,
|
||||
}),
|
||||
compat: {
|
||||
supportsTools: true,
|
||||
supportsUsageInStreaming: true,
|
||||
@@ -469,6 +497,8 @@ export function toDynamicOllamaModel(params: {
|
||||
(value): value is "text" | "image" => value === "text" || value === "image",
|
||||
);
|
||||
const api = params.providerConfig.api ?? "ollama";
|
||||
const cloudModel = resolveOllamaCloudDefaultModel(params.model.id);
|
||||
const contextWindow = params.model.contextWindow ?? cloudModel?.contextWindow ?? 8192;
|
||||
return {
|
||||
id: params.model.id,
|
||||
name: params.model.name ?? params.model.id,
|
||||
@@ -478,11 +508,20 @@ export function toDynamicOllamaModel(params: {
|
||||
reasoning: params.model.reasoning ?? false,
|
||||
input: input.length > 0 ? input : ["text"],
|
||||
cost: params.model.cost ?? { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
|
||||
contextWindow: params.model.contextWindow ?? 8192,
|
||||
contextWindow,
|
||||
...(params.model.contextTokens !== undefined
|
||||
? { contextTokens: params.model.contextTokens }
|
||||
: {}),
|
||||
maxTokens: params.model.maxTokens ?? 8192,
|
||||
maxTokens:
|
||||
params.model.maxTokens ??
|
||||
resolveOllamaOutputBudget({
|
||||
modelId: params.model.id,
|
||||
contextWindow,
|
||||
remote: isHostedOllamaCloud(params.providerConfig.baseUrl),
|
||||
}),
|
||||
...(params.model.parameterCount !== undefined
|
||||
? { parameterCount: params.model.parameterCount }
|
||||
: {}),
|
||||
...(api === "ollama" && params.model.thinkingLevelMap
|
||||
? { thinkingLevelMap: params.model.thinkingLevelMap }
|
||||
: {}),
|
||||
@@ -625,7 +664,10 @@ export async function buildOllamaProvider(
|
||||
baseUrl: apiBase,
|
||||
api: "ollama",
|
||||
models: discovered.map((model) =>
|
||||
buildOllamaModelDefinition(model.name, model.contextWindow, model.capabilities, model),
|
||||
buildOllamaModelDefinition(model.name, model.contextWindow, model.capabilities, {
|
||||
...model,
|
||||
remote: isHostedOllamaCloud(apiBase) || isOllamaRemoteModel(model),
|
||||
}),
|
||||
),
|
||||
};
|
||||
}
|
||||
@@ -2,7 +2,10 @@ import type { OpenClawConfig } from "openclaw/plugin-sdk/provider-auth";
|
||||
import type { WizardPrompter } from "openclaw/plugin-sdk/setup";
|
||||
import { requestBodyText, requestUrl } from "openclaw/plugin-sdk/test-env";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import type { OllamaModelWithContext } from "./provider-models.js";
|
||||
import {
|
||||
buildAppGuidedOllamaModel,
|
||||
buildOllamaModelsConfig,
|
||||
discoverOllamaModelsForSetup,
|
||||
selectAppGuidedOllamaModelFromDiscovery,
|
||||
} from "./setup-model-selection.js";
|
||||
@@ -21,6 +24,53 @@ function pendingAbortableResponse(signal: AbortSignal | null | undefined): Promi
|
||||
}
|
||||
|
||||
describe("Ollama onboarding model selection", () => {
|
||||
it("retains freshly inspected size when using a configured cloud definition", () => {
|
||||
const model = buildAppGuidedOllamaModel({
|
||||
baseUrl: "http://127.0.0.1:11434",
|
||||
model: { name: "compact-chat:cloud" },
|
||||
showInfo: {
|
||||
contextWindow: 32_768,
|
||||
capabilities: ["completion", "tools"],
|
||||
parameterCount: 4_000_000_000,
|
||||
},
|
||||
configuredCloudModel: {
|
||||
id: "compact-chat:cloud",
|
||||
name: "Compact Chat",
|
||||
reasoning: false,
|
||||
input: ["text"],
|
||||
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
|
||||
contextWindow: 32_768,
|
||||
maxTokens: 8192,
|
||||
},
|
||||
});
|
||||
expect(model?.parameterCount).toBe(4_000_000_000);
|
||||
});
|
||||
|
||||
it.each([{ remote_host: "https://ollama.com" }, { remote_model: "upstream-cloud-model" }])(
|
||||
"keeps a remote alias output budget during setup: %j",
|
||||
(remote) => {
|
||||
const models = buildOllamaModelsConfig(
|
||||
["private-alias", "local-model"],
|
||||
new Map<string, OllamaModelWithContext>([
|
||||
["private-alias", { name: "private-alias", contextWindow: 262_144, ...remote }],
|
||||
["local-model", { name: "local-model", contextWindow: 262_144 }],
|
||||
]),
|
||||
);
|
||||
expect(models.map(({ id, maxTokens }) => ({ id, maxTokens }))).toEqual([
|
||||
{ id: "private-alias", maxTokens: 262_144 },
|
||||
{ id: "local-model", maxTokens: 8192 },
|
||||
]);
|
||||
},
|
||||
);
|
||||
|
||||
it("keeps discovery-only parameter metadata out of user config", () => {
|
||||
const models = buildOllamaModelsConfig(
|
||||
["model"],
|
||||
new Map([["model", { name: "model", parameterCount: 4_000_000_000 }]]),
|
||||
);
|
||||
expect(models[0]).not.toHaveProperty("parameterCount");
|
||||
});
|
||||
|
||||
it("selects a deterministic tools-capable model with enough context", () => {
|
||||
expect(
|
||||
selectAppGuidedOllamaModelFromDiscovery([
|
||||
|
||||
@@ -1,8 +1,13 @@
|
||||
import { formatErrorMessage } from "openclaw/plugin-sdk/error-runtime";
|
||||
import { selectPreferredLocalModelId } from "openclaw/plugin-sdk/provider-model-shared";
|
||||
import {
|
||||
selectPreferredLocalModelId,
|
||||
type ModelDefinitionConfig,
|
||||
} from "openclaw/plugin-sdk/provider-model-shared";
|
||||
import { isHostedOllamaCloud } from "./defaults.js";
|
||||
import { getOllamaLatestDedupeKey } from "./model-id.js";
|
||||
import {
|
||||
buildOllamaModelDefinition,
|
||||
capLocalOllamaModelContext,
|
||||
enrichOllamaModelsWithContext,
|
||||
fetchOllamaModels,
|
||||
isOllamaEmbeddingOnlyModel,
|
||||
@@ -12,6 +17,8 @@ import {
|
||||
readOllamaModelShowInfo,
|
||||
resolveOllamaApiBase,
|
||||
type OllamaModelWithContext,
|
||||
type OllamaModelShowInfo,
|
||||
type OllamaTagModel,
|
||||
} from "./provider-models.js";
|
||||
|
||||
export { findAvailableOllamaModelName } from "./model-id.js";
|
||||
@@ -88,6 +95,44 @@ export function selectAppGuidedOllamaModelFromDiscovery(
|
||||
return orderPreferredOllamaModelIds(fastest.map((model) => model.name))[0];
|
||||
}
|
||||
|
||||
export function buildAppGuidedOllamaModel(params: {
|
||||
baseUrl: string;
|
||||
model: OllamaTagModel;
|
||||
showInfo: OllamaModelShowInfo;
|
||||
configuredCloudModel?: ModelDefinitionConfig;
|
||||
}): ModelDefinitionConfig | undefined {
|
||||
const { baseUrl, model, showInfo, configuredCloudModel } = params;
|
||||
const contextWindow = showInfo.contextWindow ?? configuredCloudModel?.contextWindow;
|
||||
const supportsTools =
|
||||
showInfo.capabilities?.includes("tools") ?? configuredCloudModel?.compat?.supportsTools;
|
||||
const supportsCompletion =
|
||||
showInfo.capabilities?.includes("completion") ?? Boolean(configuredCloudModel);
|
||||
if (
|
||||
!supportsTools ||
|
||||
!supportsCompletion ||
|
||||
contextWindow === undefined ||
|
||||
contextWindow < OLLAMA_APP_GUIDED_MIN_CONTEXT_TOKENS
|
||||
) {
|
||||
return undefined;
|
||||
}
|
||||
const definition =
|
||||
configuredCloudModel ??
|
||||
buildOllamaModelDefinition(model.name, contextWindow, showInfo.capabilities, {
|
||||
...showInfo,
|
||||
remote: isHostedOllamaCloud(baseUrl) || isOllamaRemoteModel(model),
|
||||
});
|
||||
return capLocalOllamaModelContext(
|
||||
{
|
||||
...definition,
|
||||
...(showInfo.parameterCount !== undefined ? { parameterCount: showInfo.parameterCount } : {}),
|
||||
thinkingLevelMap: definition.thinkingLevelMap ?? showInfo.thinkingLevelMap,
|
||||
contextWindow,
|
||||
compat: { ...definition.compat, supportsTools: true },
|
||||
},
|
||||
baseUrl,
|
||||
);
|
||||
}
|
||||
|
||||
export function buildOllamaModelsConfig(
|
||||
modelNames: string[],
|
||||
discoveredModelsByName?: Map<string, OllamaModelWithContext>,
|
||||
@@ -97,14 +142,13 @@ export function buildOllamaModelsConfig(
|
||||
if (discovered && isOllamaEmbeddingOnlyModel(discovered)) {
|
||||
return [];
|
||||
}
|
||||
return [
|
||||
buildOllamaModelDefinition(
|
||||
name,
|
||||
discovered?.contextWindow,
|
||||
discovered?.capabilities,
|
||||
discovered,
|
||||
),
|
||||
];
|
||||
const { parameterCount: _parameterCount, ...model } = buildOllamaModelDefinition(
|
||||
name,
|
||||
discovered?.contextWindow,
|
||||
discovered?.capabilities,
|
||||
discovered ? { ...discovered, remote: isOllamaRemoteModel(discovered) } : undefined,
|
||||
);
|
||||
return [model];
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,139 @@
|
||||
import type { Model } from "openclaw/plugin-sdk/llm";
|
||||
import type { fetchWithSsrFGuard } from "openclaw/plugin-sdk/ssrf-runtime";
|
||||
import { requestBodyText } from "openclaw/plugin-sdk/test-env";
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
const { fetchMock } = vi.hoisted(() => ({ fetchMock: vi.fn<typeof fetchWithSsrFGuard>() }));
|
||||
// mock-isolation: Capture native HTTP payloads without DNS or network activity.
|
||||
vi.mock("openclaw/plugin-sdk/ssrf-runtime", () => ({ fetchWithSsrFGuard: fetchMock }));
|
||||
|
||||
import { buildOllamaModelDefinition } from "./provider-models.js";
|
||||
import { createOllamaStreamFn } from "./stream.runtime.js";
|
||||
|
||||
const baseModel: Model = {
|
||||
id: "local-model",
|
||||
name: "Local model",
|
||||
provider: "custom-ollama",
|
||||
api: "ollama",
|
||||
baseUrl: "http://127.0.0.1:11434",
|
||||
reasoning: false,
|
||||
input: ["text"],
|
||||
contextWindow: 200_000,
|
||||
maxTokens: 8192,
|
||||
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
|
||||
};
|
||||
|
||||
describe("native Ollama output guards", () => {
|
||||
beforeEach(() => {
|
||||
fetchMock.mockReset();
|
||||
fetchMock.mockResolvedValue({
|
||||
finalUrl: "http://127.0.0.1:11434/api/chat",
|
||||
response: new Response(
|
||||
JSON.stringify({
|
||||
model: "local-model",
|
||||
message: { role: "assistant", content: "ok" },
|
||||
done: true,
|
||||
}) + "\n",
|
||||
),
|
||||
release: async () => {},
|
||||
});
|
||||
});
|
||||
|
||||
it.each<{
|
||||
name: string;
|
||||
model?: Partial<Model>;
|
||||
maxTokens?: number;
|
||||
expected: number;
|
||||
}>([
|
||||
{ name: "resolved model cap", model: { maxTokens: 3072 }, expected: 3072 },
|
||||
{ name: "incomplete model", model: { maxTokens: undefined }, expected: 8192 },
|
||||
{
|
||||
name: "incomplete model with a smaller context",
|
||||
model: { maxTokens: undefined, contextWindow: 4096 },
|
||||
expected: 4096,
|
||||
},
|
||||
{
|
||||
name: "incomplete model with an effective context cap",
|
||||
model: { maxTokens: undefined, contextTokens: 2048 },
|
||||
expected: 2048,
|
||||
},
|
||||
{ name: "authored native cap", model: { params: { num_predict: 123 } }, expected: 123 },
|
||||
{ name: "authored unlimited sentinel", model: { params: { num_predict: -1 } }, expected: -1 },
|
||||
{
|
||||
name: "explicit runtime cap over authored defaults",
|
||||
model: { maxTokens: 3072, params: { num_predict: 123 } },
|
||||
maxTokens: 55,
|
||||
expected: 55,
|
||||
},
|
||||
{
|
||||
name: "explicit runtime cap above model metadata",
|
||||
model: { maxTokens: 3072 },
|
||||
maxTokens: 4096,
|
||||
expected: 4096,
|
||||
},
|
||||
])("sends $name as num_predict", async ({ model: overrides, maxTokens, expected }) => {
|
||||
const model = { ...baseModel, ...overrides };
|
||||
const stream = await createOllamaStreamFn(model.baseUrl)(
|
||||
model,
|
||||
{ messages: [{ role: "user", content: "hello", timestamp: 0 }] },
|
||||
{ maxTokens },
|
||||
);
|
||||
await expect(stream.result()).resolves.toMatchObject({ stopReason: "stop" });
|
||||
const request = fetchMock.mock.calls[0]?.[0];
|
||||
expect(request?.url).toBe("http://127.0.0.1:11434/api/chat");
|
||||
expect(JSON.parse(requestBodyText(request?.init?.body))).toMatchObject({
|
||||
options: { num_predict: expected },
|
||||
});
|
||||
});
|
||||
|
||||
it.each(["minimax-m3", "kimi-k3"])(
|
||||
"sends the local default for a bare %s catalog collision",
|
||||
async (id) => {
|
||||
const definition = buildOllamaModelDefinition(id);
|
||||
const model = { ...baseModel, id, maxTokens: definition.maxTokens };
|
||||
const stream = await createOllamaStreamFn(model.baseUrl)(model, {
|
||||
messages: [{ role: "user", content: "hello", timestamp: 0 }],
|
||||
});
|
||||
await stream.result();
|
||||
expect(JSON.parse(requestBodyText(fetchMock.mock.calls[0]?.[0].init?.body))).toMatchObject({
|
||||
options: { num_predict: 8192 },
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("replays preserved blocks and the repetition marker through the native request", async () => {
|
||||
const stream = await createOllamaStreamFn(baseModel.baseUrl)(baseModel, {
|
||||
messages: [
|
||||
{ role: "user", content: "start", timestamp: 1 },
|
||||
{
|
||||
role: "assistant",
|
||||
api: baseModel.api,
|
||||
provider: baseModel.provider,
|
||||
model: baseModel.id,
|
||||
content: [
|
||||
{ type: "text", text: "Earlier answer stays intact." },
|
||||
{ type: "text", text: "[Repetitive output stopped]" },
|
||||
],
|
||||
stopReason: "error",
|
||||
errorCode: "openclaw_repetitive_output",
|
||||
timestamp: 2,
|
||||
usage: {
|
||||
input: 0,
|
||||
output: 0,
|
||||
cacheRead: 0,
|
||||
cacheWrite: 0,
|
||||
totalTokens: 0,
|
||||
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
|
||||
},
|
||||
},
|
||||
{ role: "user", content: "next", timestamp: 3 },
|
||||
],
|
||||
});
|
||||
await stream.result();
|
||||
expect(JSON.parse(requestBodyText(fetchMock.mock.calls[0]?.[0].init?.body)).messages).toEqual([
|
||||
{ role: "user", content: "start" },
|
||||
{ role: "assistant", content: "Earlier answer stays intact.[Repetitive output stopped]" },
|
||||
{ role: "user", content: "next" },
|
||||
]);
|
||||
});
|
||||
});
|
||||
@@ -39,6 +39,7 @@ import {
|
||||
isOllamaCloudOrigin,
|
||||
OLLAMA_CLOUD_PROVIDER_ID,
|
||||
OLLAMA_DEFAULT_BASE_URL,
|
||||
OLLAMA_DEFAULT_MAX_TOKENS,
|
||||
} from "./defaults.js";
|
||||
import { normalizeOllamaWireModelId } from "./model-id.js";
|
||||
import { applyOllamaThinkingFloor } from "./model-reasoning.js";
|
||||
@@ -146,19 +147,21 @@ function resolveOllamaNativeNumCtx(model: ProviderRuntimeModel): number | undefi
|
||||
return configured;
|
||||
}
|
||||
const effective = model.contextTokens;
|
||||
if (effective === undefined || effective <= 0) {
|
||||
return undefined;
|
||||
}
|
||||
return Math.floor(effective);
|
||||
return effective !== undefined && effective > 0 ? Math.floor(effective) : undefined;
|
||||
}
|
||||
|
||||
function resolveOllamaModelOptions(model: ProviderRuntimeModel): Record<string, unknown> {
|
||||
const options = pickOllamaParams(model.params, OLLAMA_OPTION_PARAM_KEYS);
|
||||
const numCtx = resolveOllamaNativeNumCtx(model);
|
||||
if (numCtx !== undefined) {
|
||||
options.num_ctx = numCtx;
|
||||
}
|
||||
return options;
|
||||
return {
|
||||
num_predict:
|
||||
model.maxTokens ??
|
||||
Math.min(
|
||||
OLLAMA_DEFAULT_MAX_TOKENS,
|
||||
model.contextTokens ?? model.contextWindow ?? OLLAMA_DEFAULT_MAX_TOKENS,
|
||||
),
|
||||
...pickOllamaParams(model.params, OLLAMA_OPTION_PARAM_KEYS),
|
||||
...(numCtx !== undefined ? { num_ctx: numCtx } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
function pickOllamaParams(
|
||||
@@ -183,12 +186,9 @@ function normalizeOllamaGreedySamplingOptions(options: Record<string, unknown>):
|
||||
}
|
||||
|
||||
function resolveStreamingTextDelta(previousText: string, nextText: string): string {
|
||||
if (nextText.startsWith(previousText)) {
|
||||
return nextText.slice(previousText.length);
|
||||
}
|
||||
// Sanitizers may rewrite previously accumulated content. Fall back to
|
||||
// re-emitting the latest complete text so downstream partial state converges.
|
||||
return nextText;
|
||||
return nextText.startsWith(previousText) ? nextText.slice(previousText.length) : nextText;
|
||||
}
|
||||
|
||||
export function buildOllamaChatRequest(params: {
|
||||
|
||||
@@ -22,7 +22,10 @@ type ThinkingLevel = NonNullable<ProviderWrapStreamFnContext["thinkingLevel"]>;
|
||||
afterEach(() => vi.unstubAllGlobals());
|
||||
|
||||
function mockOllama(modelId: string, thinking: unknown) {
|
||||
const metadata = { digest: "first", thinking };
|
||||
const metadata: { digest: string; thinking: unknown; parameterCount?: number } = {
|
||||
digest: "first",
|
||||
thinking,
|
||||
};
|
||||
const requests: string[] = [];
|
||||
const chatBodies: Record<string, unknown>[] = [];
|
||||
vi.stubGlobal(
|
||||
@@ -35,7 +38,10 @@ function mockOllama(modelId: string, thinking: unknown) {
|
||||
}
|
||||
if (pathname === "/api/show") {
|
||||
return jsonResponse({
|
||||
model_info: { "test.context_length": 32768 },
|
||||
model_info: {
|
||||
"test.context_length": 32768,
|
||||
"general.parameter_count": metadata.parameterCount,
|
||||
},
|
||||
capabilities: ["completion", "thinking"],
|
||||
...(metadata.thinking === undefined ? {} : { thinking: metadata.thinking }),
|
||||
});
|
||||
@@ -125,6 +131,7 @@ describe("Ollama discovered thinking contracts", () => {
|
||||
values: [true],
|
||||
default: true,
|
||||
});
|
||||
fixture.metadata.parameterCount = 70_000_000_000;
|
||||
const model = await fixture.discover("ollama", {
|
||||
id: "manual-mandatory-thinking:latest",
|
||||
name: "Manual thinking model",
|
||||
@@ -137,6 +144,7 @@ describe("Ollama discovered thinking contracts", () => {
|
||||
});
|
||||
expect(model).toMatchObject({
|
||||
name: "Manual thinking model",
|
||||
parameterCount: 70_000_000_000,
|
||||
contextWindow: 8192,
|
||||
maxTokens: 2048,
|
||||
compat: { supportsTools: false },
|
||||
|
||||
@@ -0,0 +1,300 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { createDeferred } from "../../../test/helpers/promise.js";
|
||||
import { runAgentLoop } from "./agent-loop.js";
|
||||
import {
|
||||
config,
|
||||
createTurnSequenceStream,
|
||||
makeAssistantMessage,
|
||||
makeCall,
|
||||
makeTool,
|
||||
reply,
|
||||
user,
|
||||
} from "./agent-loop.test-support.js";
|
||||
import { createAssistantMessageEventStream, type AssistantMessage, type Message } from "./llm.js";
|
||||
import { distinctRecordOutputs, variedRows } from "./repetition-guard.test-fixtures.js";
|
||||
import type { AgentEvent, AgentToolCall, StreamFn } from "./types.js";
|
||||
|
||||
const loop = "Please provide more input. ".repeat(800);
|
||||
const marker = "[Repetitive output stopped]";
|
||||
|
||||
async function runResponse(
|
||||
content: AssistantMessage["content"],
|
||||
enabled = true,
|
||||
terminalOnly = false,
|
||||
initialText?: string,
|
||||
) {
|
||||
const events: AgentEvent[] = [];
|
||||
let aborted = false;
|
||||
let requests = 0;
|
||||
const streamFn: StreamFn = (_model, _context, options) => {
|
||||
requests += 1;
|
||||
options?.signal?.addEventListener(
|
||||
"abort",
|
||||
() => {
|
||||
aborted = true;
|
||||
},
|
||||
{ once: true },
|
||||
);
|
||||
const response = createAssistantMessageEventStream();
|
||||
const message = makeAssistantMessage(content);
|
||||
message.providerReplay = {
|
||||
v: 1,
|
||||
type: "test-checkpoint",
|
||||
data: "opaque state containing discarded output",
|
||||
api: message.api,
|
||||
provider: message.provider,
|
||||
model: message.model,
|
||||
};
|
||||
queueMicrotask(() => {
|
||||
if (!terminalOnly || initialText !== undefined) {
|
||||
response.push({ type: "start", partial: makeAssistantMessage([]) });
|
||||
}
|
||||
if (initialText !== undefined) {
|
||||
response.push({
|
||||
type: "text_delta",
|
||||
contentIndex: 0,
|
||||
delta: initialText,
|
||||
partial: makeAssistantMessage([{ type: "text", text: initialText }]),
|
||||
});
|
||||
}
|
||||
if (!terminalOnly) {
|
||||
content.forEach((block, contentIndex) => {
|
||||
if (block.type === "text") {
|
||||
response.push({
|
||||
type: "text_delta",
|
||||
contentIndex,
|
||||
delta: block.text,
|
||||
partial: message,
|
||||
});
|
||||
} else if (block.type === "thinking") {
|
||||
response.push({
|
||||
type: "thinking_delta",
|
||||
contentIndex,
|
||||
delta: block.thinking,
|
||||
partial: message,
|
||||
});
|
||||
}
|
||||
});
|
||||
}
|
||||
response.push({ type: "done", reason: "stop", message });
|
||||
response.end();
|
||||
});
|
||||
return response;
|
||||
};
|
||||
const loopConfig = { ...config, repetitionGuard: () => enabled };
|
||||
const result = await runAgentLoop(
|
||||
[user()],
|
||||
{ systemPrompt: "", messages: [], tools: [] },
|
||||
loopConfig,
|
||||
(event) => {
|
||||
events.push(event);
|
||||
},
|
||||
undefined,
|
||||
streamFn,
|
||||
);
|
||||
return { result, events, aborted, requests };
|
||||
}
|
||||
|
||||
describe("repetition stop at the stream owner", () => {
|
||||
it.each(distinctRecordOutputs)("delivers $name without truncation", async ({ text }) => {
|
||||
const content: AssistantMessage["content"] = [{ type: "text", text }];
|
||||
const { result, events, requests } = await runResponse(content);
|
||||
expect(requests).toBe(1);
|
||||
expect(result.at(-1)).toMatchObject({ stopReason: "stop", content });
|
||||
expect(events.findLast((event) => event.type === "message_end")).toMatchObject({
|
||||
message: { content },
|
||||
});
|
||||
});
|
||||
|
||||
it.each(["text", "thinking", "terminal"])(
|
||||
"quarantines a repetitive %s block before persistence and replay",
|
||||
async (kind) => {
|
||||
const content: AssistantMessage["content"] = [
|
||||
{ type: "text", text: "Earlier answer stays intact." },
|
||||
kind === "thinking" ? { type: "thinking", thinking: loop } : { type: "text", text: loop },
|
||||
];
|
||||
const { result, events, requests } = await runResponse(content, true, kind === "terminal");
|
||||
const expected = [content[0], { type: "text", text: marker }];
|
||||
expect(result.at(-1)).toMatchObject({
|
||||
stopReason: "error",
|
||||
errorCode: "openclaw_repetitive_output",
|
||||
content: expected,
|
||||
});
|
||||
expect(result.at(-1)).toHaveProperty("providerReplay", undefined);
|
||||
expect(events.findLast((event) => event.type === "message_end")).toMatchObject({
|
||||
message: { content: expected },
|
||||
});
|
||||
expect(requests).toBe(1);
|
||||
const replay: Message[][] = [];
|
||||
await runAgentLoop(
|
||||
[user("try a shorter request")],
|
||||
{ systemPrompt: "", messages: result, tools: [] },
|
||||
config,
|
||||
() => {},
|
||||
undefined,
|
||||
createTurnSequenceStream([[{ type: "text", text: "Recovered." }]], replay),
|
||||
);
|
||||
expect(JSON.stringify(replay)).not.toContain("Please provide more input.");
|
||||
expect(JSON.stringify(replay)).toContain(marker);
|
||||
},
|
||||
);
|
||||
|
||||
it("leaves the same response untouched when the policy is off", async () => {
|
||||
const content: AssistantMessage["content"] = [{ type: "text", text: loop }];
|
||||
const { result } = await runResponse(content, false);
|
||||
expect(result.at(-1)).toMatchObject({ stopReason: "stop", content });
|
||||
});
|
||||
|
||||
it.each([
|
||||
{ kind: "shorter final", initialLength: loop.length + 1, terminalOnly: true },
|
||||
{ kind: "same-length final", initialLength: loop.length, terminalOnly: true },
|
||||
{ kind: "longer final", initialLength: 16_000, terminalOnly: true },
|
||||
{ kind: "streamed", initialLength: loop.length, terminalOnly: false },
|
||||
])("checks a $kind replacement snapshot", async ({ initialLength, terminalOnly }) => {
|
||||
const { result, events, requests } = await runResponse(
|
||||
[{ type: "text", text: loop }],
|
||||
true,
|
||||
terminalOnly,
|
||||
variedRows.slice(0, initialLength),
|
||||
);
|
||||
expect(result.at(-1)).toMatchObject({
|
||||
stopReason: "error",
|
||||
errorCode: "openclaw_repetitive_output",
|
||||
content: [{ type: "text", text: marker }],
|
||||
});
|
||||
expect(events.findLast((event) => event.type === "message_end")).toMatchObject({
|
||||
message: { content: [{ type: "text", text: marker }], providerReplay: undefined },
|
||||
});
|
||||
expect(requests).toBe(1);
|
||||
});
|
||||
|
||||
it("checks snapshot-only text before persisting and admitting an async tool", async () => {
|
||||
const executed: string[] = [];
|
||||
const toolCall: AgentToolCall = { ...makeCall("write"), async: true };
|
||||
const message = makeAssistantMessage([{ type: "text", text: loop }, toolCall]);
|
||||
let requests = 0;
|
||||
const result = await runAgentLoop(
|
||||
[user()],
|
||||
{
|
||||
systemPrompt: "",
|
||||
messages: [],
|
||||
tools: [makeTool("write", executed)],
|
||||
},
|
||||
{ ...config, repetitionGuard: () => true },
|
||||
() => {},
|
||||
undefined,
|
||||
() => {
|
||||
if (++requests > 1) {
|
||||
return reply(makeAssistantMessage([{ type: "text", text: "Unexpected retry." }]));
|
||||
}
|
||||
const response = createAssistantMessageEventStream();
|
||||
response.push({ type: "start", partial: makeAssistantMessage([]) });
|
||||
response.push({ type: "toolcall_end", contentIndex: 1, toolCall, partial: message });
|
||||
response.push({ type: "done", reason: "toolUse", message });
|
||||
response.end();
|
||||
return response;
|
||||
},
|
||||
);
|
||||
expect(executed).toEqual([]);
|
||||
expect(requests).toBe(1);
|
||||
expect(result.at(-1)).toMatchObject({
|
||||
stopReason: "error",
|
||||
content: [{ type: "text", text: marker }],
|
||||
});
|
||||
expect(JSON.stringify(result)).not.toContain("Please provide more input.");
|
||||
});
|
||||
|
||||
it.each(["delta", "error snapshot"])(
|
||||
"settles an admitted tool once without tail admission or retries after a repetitive %s",
|
||||
async (source) => {
|
||||
const response = createAssistantMessageEventStream();
|
||||
const started = createDeferred();
|
||||
const finish = createDeferred();
|
||||
const toolFinished = createDeferred();
|
||||
let requestAborted = false;
|
||||
let requests = 0;
|
||||
let toolAborted = false;
|
||||
const executed: string[] = [];
|
||||
const kept: AgentToolCall = { ...makeCall("kept"), async: true };
|
||||
const discarded = makeCall("discarded");
|
||||
const streamFn: StreamFn = (_model, _context, options) => {
|
||||
requests++;
|
||||
if (requests > 1) {
|
||||
return reply(makeAssistantMessage([{ type: "text", text: "Unexpected retry." }]));
|
||||
}
|
||||
options?.signal?.addEventListener(
|
||||
"abort",
|
||||
() => {
|
||||
requestAborted = true;
|
||||
finish.resolve();
|
||||
},
|
||||
{ once: true },
|
||||
);
|
||||
return response;
|
||||
};
|
||||
const run = runAgentLoop(
|
||||
[user()],
|
||||
{
|
||||
systemPrompt: "",
|
||||
messages: [],
|
||||
tools: [
|
||||
{
|
||||
...makeTool("kept"),
|
||||
execute: async (_id, _args, signal) => {
|
||||
executed.push("kept");
|
||||
started.resolve();
|
||||
await finish.promise;
|
||||
toolAborted = signal?.aborted ?? false;
|
||||
toolFinished.resolve();
|
||||
return { content: [{ type: "text", text: "Saved." }], details: {} };
|
||||
},
|
||||
},
|
||||
makeTool("discarded", executed),
|
||||
],
|
||||
},
|
||||
{ ...config, repetitionGuard: () => true },
|
||||
(event) => {
|
||||
if (
|
||||
event.type === "message_end" &&
|
||||
event.message.role === "assistant" &&
|
||||
event.message.content.some((item) => item.type === "text")
|
||||
) {
|
||||
finish.resolve();
|
||||
}
|
||||
},
|
||||
undefined,
|
||||
streamFn,
|
||||
);
|
||||
response.push({ type: "start", partial: makeAssistantMessage([]) });
|
||||
response.push({
|
||||
type: "toolcall_end",
|
||||
contentIndex: 0,
|
||||
toolCall: kept,
|
||||
partial: makeAssistantMessage([kept]),
|
||||
});
|
||||
await started.promise;
|
||||
const bad = makeAssistantMessage([kept, { type: "text", text: loop }, discarded]);
|
||||
if (source === "delta") {
|
||||
response.push({ type: "text_delta", contentIndex: 1, delta: loop, partial: bad });
|
||||
response.push({ type: "done", reason: "toolUse", message: bad });
|
||||
} else {
|
||||
response.push({ type: "error", reason: "error", error: { ...bad, stopReason: "error" } });
|
||||
}
|
||||
response.end();
|
||||
const result = await run;
|
||||
expect(requests).toBe(1);
|
||||
expect(requestAborted).toBe(true);
|
||||
expect(toolAborted).toBe(false);
|
||||
await toolFinished.promise;
|
||||
expect(executed).toEqual(["kept"]);
|
||||
expect(result.filter((message) => message.role === "toolResult")).toMatchObject([
|
||||
{ toolCallId: "kept", content: [{ type: "text", text: "Saved." }] },
|
||||
]);
|
||||
expect(result.filter((message) => message.role === "assistant")).toMatchObject([
|
||||
{ content: [kept], stopReason: "toolUse" },
|
||||
{ content: [{ type: "text", text: marker }], stopReason: "error" },
|
||||
]);
|
||||
expect(JSON.stringify(result)).not.toContain("Please provide more input.");
|
||||
},
|
||||
);
|
||||
});
|
||||
@@ -6,6 +6,7 @@ import {
|
||||
import {
|
||||
appendTextDeltaToAssistantMessage,
|
||||
PROVIDER_FAILURE_WITH_OUTPUT_ERROR_CODE,
|
||||
REPETITIVE_OUTPUT_ERROR_CODE,
|
||||
} from "@openclaw/llm-core";
|
||||
import type {
|
||||
AssistantMessage,
|
||||
@@ -13,6 +14,8 @@ import type {
|
||||
Context,
|
||||
ToolResultMessage,
|
||||
} from "@openclaw/llm-core";
|
||||
import { AssistantRepetitionGuard } from "./assistant-repetition-guard.js";
|
||||
import { REPETITIVE_OUTPUT_ERROR_MESSAGE } from "./errors.js";
|
||||
import { uuidv7 } from "./harness/session/uuid.js";
|
||||
import { copyInternalToolResultState } from "./internal-hooks.js";
|
||||
import {
|
||||
@@ -172,10 +175,13 @@ export async function streamAgentResponse(
|
||||
const executionSignal = signal
|
||||
? AbortSignal.any([signal, executionAbort.signal])
|
||||
: executionAbort.signal;
|
||||
const modelAbort = new AbortController();
|
||||
const requestSignal = AbortSignal.any([executionSignal, modelAbort.signal]);
|
||||
const abortFailedResponse = (message?: AssistantMessage) => {
|
||||
if (
|
||||
message &&
|
||||
(message.stopReason === "error" || message.stopReason === "aborted") &&
|
||||
message.errorCode !== REPETITIVE_OUTPUT_ERROR_CODE &&
|
||||
!isResponsesOutputLimitToolCallError(message)
|
||||
) {
|
||||
executionAbort.abort(new Error(message.errorMessage ?? "Model response interrupted"));
|
||||
@@ -264,7 +270,7 @@ export async function streamAgentResponse(
|
||||
const stream = streamFunction(config.model, llmContext, {
|
||||
...config,
|
||||
apiKey: resolvedApiKey,
|
||||
signal: executionSignal,
|
||||
signal: requestSignal,
|
||||
onActiveResponse: steering.onActiveResponse,
|
||||
asyncToolExecution: true,
|
||||
});
|
||||
@@ -273,6 +279,9 @@ export async function streamAgentResponse(
|
||||
stream,
|
||||
async () => {
|
||||
const response = await stream;
|
||||
const repetitionGuard = config.repetitionGuard?.(config.model)
|
||||
? new AssistantRepetitionGuard()
|
||||
: undefined;
|
||||
let partialMessage: AssistantMessage | null = null;
|
||||
let partialIndex: number | undefined;
|
||||
let committedContentCount = 0;
|
||||
@@ -325,6 +334,15 @@ export async function streamAgentResponse(
|
||||
if (event.contentIndex < committedContentCount) {
|
||||
break;
|
||||
}
|
||||
const stopped = repetitionGuard?.check(
|
||||
message,
|
||||
committedContentCount,
|
||||
event.type === "toolcall_end" ? undefined : event.contentIndex,
|
||||
);
|
||||
if (stopped) {
|
||||
modelAbort.abort(new Error(REPETITIVE_OUTPUT_ERROR_MESSAGE));
|
||||
return await finalizeAssistantMessage(stopped);
|
||||
}
|
||||
const fragment = remainingFragment(message);
|
||||
if (partialIndex === undefined) {
|
||||
partialIndex = context.messages.length;
|
||||
@@ -392,9 +410,22 @@ export async function streamAgentResponse(
|
||||
return await finalizeAssistantMessage();
|
||||
|
||||
async function finalizeAssistantMessage(terminal?: AssistantMessage) {
|
||||
// Output-limit recovery drains admitted tools; other failures fence queued starts.
|
||||
abortFailedResponse(terminal);
|
||||
const result = await response.result();
|
||||
const stoppedTerminal =
|
||||
terminal?.errorCode === REPETITIVE_OUTPUT_ERROR_CODE
|
||||
? terminal
|
||||
: terminal && repetitionGuard?.check(terminal, committedContentCount);
|
||||
if (stoppedTerminal) {
|
||||
modelAbort.abort(new Error(REPETITIVE_OUTPUT_ERROR_MESSAGE));
|
||||
}
|
||||
// Classify first: repetition drains admitted tools even in a provider error snapshot.
|
||||
abortFailedResponse(stoppedTerminal ?? terminal);
|
||||
const providerResult = await response.result();
|
||||
const stopped =
|
||||
stoppedTerminal ?? repetitionGuard?.check(providerResult, committedContentCount);
|
||||
if (stopped) {
|
||||
modelAbort.abort(new Error(REPETITIVE_OUTPUT_ERROR_MESSAGE));
|
||||
}
|
||||
const result = stopped ? { ...stopped, usage: providerResult.usage } : providerResult;
|
||||
abortFailedResponse(result);
|
||||
const outputLimit = isResponsesOutputLimitToolCallError(result);
|
||||
if (outputLimit) {
|
||||
|
||||
@@ -12,6 +12,7 @@ import { TranscriptNotContinuableError } from "./errors.js";
|
||||
import {
|
||||
attachInternalSyncSteeringGetter,
|
||||
getInternalBeforeToolBatch,
|
||||
getInternalRepetitionGuard,
|
||||
getInternalToolTurnCompletion,
|
||||
} from "./internal-hooks.js";
|
||||
import { isOpenClawSystemUpdateMessage } from "./operator-messages.js";
|
||||
@@ -593,6 +594,7 @@ export class Agent {
|
||||
);
|
||||
return {
|
||||
model: this.mutableState.model,
|
||||
repetitionGuard: getInternalRepetitionGuard(this),
|
||||
thinkingLevel: this.mutableState.thinkingLevel,
|
||||
reasoning: resolveAgentReasoningOption(
|
||||
this.mutableState.model,
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
import { REPETITIVE_OUTPUT_ERROR_CODE, type AssistantMessage } from "@openclaw/llm-core";
|
||||
import { REPETITIVE_OUTPUT_ERROR_MESSAGE } from "./errors.js";
|
||||
import { RepetitionWatch } from "./repetition-guard.js";
|
||||
|
||||
/** Tracks each text/reasoning block once, including providers that only emit final snapshots. */
|
||||
export class AssistantRepetitionGuard {
|
||||
private blocks = new Map<number, { watch: RepetitionWatch; text: string }>();
|
||||
|
||||
check(
|
||||
message: AssistantMessage,
|
||||
committed: number,
|
||||
contentIndex?: number,
|
||||
): AssistantMessage | undefined {
|
||||
const end = contentIndex === undefined ? message.content.length : contentIndex + 1;
|
||||
for (let index = contentIndex ?? committed; index < end; index++) {
|
||||
const block = message.content[index];
|
||||
const text =
|
||||
block?.type === "text"
|
||||
? block.text
|
||||
: block?.type === "thinking"
|
||||
? block.thinking
|
||||
: undefined;
|
||||
if (text === undefined || index < committed) {
|
||||
continue;
|
||||
}
|
||||
let state = this.blocks.get(index);
|
||||
if (!state || !text.startsWith(state.text)) {
|
||||
state = { watch: new RepetitionWatch(), text: "" };
|
||||
this.blocks.set(index, state);
|
||||
}
|
||||
const stopped = state.watch.feed(text.slice(state.text.length));
|
||||
state.text = text;
|
||||
if (stopped) {
|
||||
// A server-side checkpoint could replay the discarded block even with clean visible text.
|
||||
return {
|
||||
...message,
|
||||
providerReplay: undefined,
|
||||
content: [
|
||||
...message.content.slice(0, committed),
|
||||
...message.content.slice(committed, index).filter((item) => item.type !== "toolCall"),
|
||||
{ type: "text", text: "[Repetitive output stopped]" },
|
||||
],
|
||||
stopReason: "error",
|
||||
endTurn: true,
|
||||
errorCode: REPETITIVE_OUTPUT_ERROR_CODE,
|
||||
errorMessage: REPETITIVE_OUTPUT_ERROR_MESSAGE,
|
||||
};
|
||||
}
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
@@ -1,6 +1,8 @@
|
||||
import type { AgentMessage } from "./types.js";
|
||||
|
||||
export const TRANSCRIPT_NOT_CONTINUABLE_ERROR_CODE = "openclaw_transcript_not_continuable";
|
||||
export const REPETITIVE_OUTPUT_ERROR_MESSAGE =
|
||||
"Repetitive output stopped. Try a larger model or shorten your request before retrying.";
|
||||
export const REPEATED_TOOL_ERROR_CODE = "openclaw_repeated_tool_error";
|
||||
export const REPEATED_TOOL_ERROR_MESSAGE =
|
||||
"OpenClaw stopped this turn after 3 consecutive identical tool failures. Check the tool arguments or switch to a model with native tool calling before retrying.";
|
||||
|
||||
@@ -11,6 +11,18 @@ import type {
|
||||
ToolResultContentSource,
|
||||
} from "./types.js";
|
||||
|
||||
type RepetitionGuardPolicy = (model: AgentLoopConfig["model"]) => boolean;
|
||||
const repetitionGuardByAgent = new WeakMap<object, RepetitionGuardPolicy>();
|
||||
|
||||
/** Host policy only: no plugin-facing option or user setting. */
|
||||
export function setInternalRepetitionGuard(agent: object, policy: RepetitionGuardPolicy): void {
|
||||
repetitionGuardByAgent.set(agent, policy);
|
||||
}
|
||||
|
||||
export function getInternalRepetitionGuard(agent: object): RepetitionGuardPolicy | undefined {
|
||||
return repetitionGuardByAgent.get(agent);
|
||||
}
|
||||
|
||||
export type InternalBeforeToolBatchHook = NonNullable<AgentLoopConfig["beforeToolBatch"]>;
|
||||
|
||||
const beforeToolBatchByAgent = new WeakMap<object, InternalBeforeToolBatchHook>();
|
||||
|
||||
@@ -0,0 +1,175 @@
|
||||
// Captured seeded Ollama runs and the varied-row control from the local-model research corpus.
|
||||
export const recordedPhraseLoop = " Done, I will output it.".repeat(877) + " Done, I will output";
|
||||
export const recordedVariedLoopTail = [
|
||||
"",
|
||||
"Okay, I think that's it.",
|
||||
"\\boxed{There is no input to check}",
|
||||
"The answer is: \\boxed{There is no input to check}",
|
||||
"",
|
||||
"Yes, the answer is \\boxed{There is no input to check}.",
|
||||
"\\boxed{There is no input to check}",
|
||||
"The answer is: \\boxed{There is no input to check}",
|
||||
"",
|
||||
"I think that's the answer.",
|
||||
"\\boxed{There is no input to check}",
|
||||
"The answer is: \\boxed{There is no input to check}",
|
||||
"",
|
||||
"Yes, I'll go with that.",
|
||||
"\\boxed{There is no input to check}",
|
||||
"The answer is: \\boxed{There is no input to check}",
|
||||
"",
|
||||
"Okay, done.",
|
||||
"\\boxed{There is no input to check}",
|
||||
"The answer is: \\boxed{There is no input to check}",
|
||||
"",
|
||||
"Yes, the answer is \\boxed{There is no input to check}.",
|
||||
"\\boxed{There is no input to check}",
|
||||
"The answer is: \\boxed{There is no input to check}",
|
||||
"",
|
||||
"I think that's it.",
|
||||
"\\boxed{There is no input to check}",
|
||||
"The answer is: \\boxed{There is no input to check}",
|
||||
"",
|
||||
"Yes, I'm confident.",
|
||||
"\\boxed{There is no input to check}",
|
||||
"The answer is: \\boxed{There is no input to check}",
|
||||
"",
|
||||
"Okay, final answer.",
|
||||
"\\boxed{There is no input to check}",
|
||||
"The answer is: \\boxed{There is no input to check}",
|
||||
"",
|
||||
"Yes, that's it.",
|
||||
"\\boxed{There is no input to check}",
|
||||
"The answer is: \\boxed{There is no input to check}",
|
||||
"",
|
||||
"I think that's the answer.",
|
||||
"\\boxed{There is no input to check}",
|
||||
"The answer is: \\boxed{There is no input to check}",
|
||||
"",
|
||||
"Yes, I'll stop here.",
|
||||
"\\boxed{There is no input to check}",
|
||||
"The answer is: \\boxed{There is no input to check}",
|
||||
"",
|
||||
"Okay, the answer is \\boxed{There is no input to check}.",
|
||||
"\\boxed{There is no input to check}",
|
||||
"The answer is: \\boxed{There is no input to check}",
|
||||
"",
|
||||
"Yes, that's it.",
|
||||
"\\boxed{There is no input to check}",
|
||||
"The answer is: \\boxed{There is no input to check}",
|
||||
"",
|
||||
"I think that's the final answer.",
|
||||
"\\boxed{There is no input to check}",
|
||||
"The answer is: \\boxed{There is no input to check}",
|
||||
"",
|
||||
"Yes, I'm done.",
|
||||
"\\boxed{There is no input to check}",
|
||||
"The answer is: \\boxed{There is no input to check}",
|
||||
"",
|
||||
"Okay, I think that's it.",
|
||||
"\\boxed{There is no input to check}",
|
||||
"The answer is: \\boxed{There is no input to check}",
|
||||
"",
|
||||
"Yes, the answer is \\boxed{There is no input to check}.",
|
||||
"\\boxed{There is no input to check}",
|
||||
"The answer is: \\boxed{There is no input to check}",
|
||||
"",
|
||||
"I think that's the answer.",
|
||||
"\\boxed{There is no input to check}",
|
||||
"",
|
||||
].join("\n");
|
||||
export const variedRows = Array.from(
|
||||
{ length: 600 },
|
||||
(_, index) => `${index + 1}. Verify service health and report status.`,
|
||||
).join("\n");
|
||||
|
||||
// Distinct records can share most of their markup without being a generation loop.
|
||||
export const distinctRecordOutputs = [
|
||||
{
|
||||
name: "HTML table with 150 distinct seven-line rows",
|
||||
text: `<table>\n${Array.from({ length: 150 }, (_, index) =>
|
||||
[
|
||||
" <tr>",
|
||||
` <td>${index}</td>`,
|
||||
` <td>User ${index}</td>`,
|
||||
` <td>user${index}@example.com</td>`,
|
||||
' <td><span class="badge badge-success">Active</span></td>',
|
||||
' <td><button class="btn btn-primary">View details</button></td>',
|
||||
" </tr>",
|
||||
].join("\n"),
|
||||
).join("\n")}\n</table>`,
|
||||
},
|
||||
{
|
||||
name: "Kubernetes manifest with 100 distinct Services",
|
||||
text: Array.from({ length: 100 }, (_, index) =>
|
||||
[
|
||||
"---",
|
||||
"apiVersion: v1",
|
||||
"kind: Service",
|
||||
"metadata:",
|
||||
` name: application-${index}`,
|
||||
" namespace: production",
|
||||
"spec:",
|
||||
" type: ClusterIP",
|
||||
" selector:",
|
||||
` app: application-${index}`,
|
||||
" ports:",
|
||||
" - protocol: TCP",
|
||||
" port: 80",
|
||||
" targetPort: 8080",
|
||||
].join("\n"),
|
||||
).join("\n"),
|
||||
},
|
||||
{
|
||||
name: "pretty JSON with 150 distinct users",
|
||||
text: JSON.stringify(
|
||||
Array.from({ length: 150 }, (_, index) => ({
|
||||
id: index,
|
||||
name: `User ${index}`,
|
||||
email: `user${index}@example.com`,
|
||||
active: true,
|
||||
role: "member",
|
||||
preferences: { theme: "dark", language: "en" },
|
||||
})),
|
||||
null,
|
||||
2,
|
||||
),
|
||||
},
|
||||
{
|
||||
name: "Jest suite with 90 distinct cases",
|
||||
text: Array.from({ length: 90 }, (_, index) =>
|
||||
[
|
||||
`describe("user ${index}", () => {`,
|
||||
' it("returns the requested user", async () => {',
|
||||
` const response = await request(app).get("/users/${index}");`,
|
||||
" expect(response.status).toBe(200);",
|
||||
` expect(response.body.id).toBe(${index});`,
|
||||
" });",
|
||||
"});",
|
||||
].join("\n"),
|
||||
).join("\n"),
|
||||
},
|
||||
{
|
||||
name: "Terraform with 120 distinct buckets",
|
||||
text: Array.from({ length: 120 }, (_, index) =>
|
||||
[
|
||||
`resource "aws_s3_bucket" "bucket_${index}" {`,
|
||||
` bucket = "application-assets-${index}"`,
|
||||
" force_destroy = false",
|
||||
" tags = {",
|
||||
' Environment = "production"',
|
||||
` Name = "application-assets-${index}"`,
|
||||
' ManagedBy = "terraform"',
|
||||
" }",
|
||||
"}",
|
||||
].join("\n"),
|
||||
).join("\n"),
|
||||
},
|
||||
{
|
||||
name: "CSV with 600 distinct users",
|
||||
text: `id,name,email,status\n${Array.from(
|
||||
{ length: 600 },
|
||||
(_, index) => `${index},User ${index},user${index}@example.com,active`,
|
||||
).join("\n")}`,
|
||||
},
|
||||
];
|
||||
@@ -0,0 +1,59 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { RepetitionWatch } from "./repetition-guard.js";
|
||||
import {
|
||||
recordedPhraseLoop,
|
||||
recordedVariedLoopTail,
|
||||
variedRows,
|
||||
} from "./repetition-guard.test-fixtures.js";
|
||||
|
||||
function firstStop(text: string, width = 32): number | undefined {
|
||||
const watch = new RepetitionWatch();
|
||||
for (let offset = 0; offset < text.length; offset += width) {
|
||||
if (watch.feed(text.slice(offset, offset + width))) {
|
||||
return Math.min(offset + width, text.length);
|
||||
}
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
describe("RepetitionWatch", () => {
|
||||
it("stops the recorded phrase loop only after substantial output", () => {
|
||||
const watch = new RepetitionWatch();
|
||||
expect(watch.feed(recordedPhraseLoop.slice(0, 15_999))).toBe(false);
|
||||
expect(watch.feed(recordedPhraseLoop.slice(15_999, 16_000))).toBe(true);
|
||||
expect(watch.feed("")).toBe(true);
|
||||
});
|
||||
|
||||
it.each([1, 7, 32, 4096, 100_000])("finds the loop with %i-character chunks", (width) => {
|
||||
expect(firstStop(recordedPhraseLoop, width)).toBe(
|
||||
Math.min(Math.ceil(16_000 / width) * width, recordedPhraseLoop.length),
|
||||
);
|
||||
});
|
||||
|
||||
it("detects repeated varying lines without requiring one line to dominate", () => {
|
||||
expect(firstStop(recordedVariedLoopTail.repeat(12))).toBe(16_000);
|
||||
});
|
||||
|
||||
it("detects long exact periods with few copies of each short window", () => {
|
||||
const period = Array.from({ length: 500 }, (_, index) => `${index.toString(36)}:`).join("");
|
||||
expect(firstStop(period.repeat(20))).toBe(16_000);
|
||||
});
|
||||
|
||||
it("keeps similar but distinct numbered rows", () => {
|
||||
expect(firstStop(variedRows)).toBeUndefined();
|
||||
});
|
||||
|
||||
it("does not treat overlapping windows as majority coverage", () => {
|
||||
const unique = Array.from({ length: 5000 }, (_, index) => `${index.toString(36)}:`)
|
||||
.join("")
|
||||
.slice(0, 12_000);
|
||||
expect(firstStop(unique + " Done, I will output it.".repeat(170))).toBeUndefined();
|
||||
});
|
||||
|
||||
it("eventually detects a loop after a long varied prefix", () => {
|
||||
const prefix = variedRows.repeat(8);
|
||||
const watch = new RepetitionWatch();
|
||||
expect(watch.feed(prefix)).toBe(false);
|
||||
expect(watch.feed(recordedPhraseLoop.repeat(10))).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,109 @@
|
||||
// Adapted from Hermes Agent; see THIRD_PARTY_NOTICES.md for the pinned source and MIT license.
|
||||
const FIRST_CHECK_CHARS = 16_000;
|
||||
const TAIL_CHARS = 64_000;
|
||||
const WINDOW_CHARS = 60;
|
||||
const MIN_REPEATS = 5;
|
||||
|
||||
function isRepetitionDominated(text: string): boolean {
|
||||
const counts = new Map<string, number>();
|
||||
let lineCount = 0;
|
||||
for (const line of text.split(/\r\n|[\r\n]/)) {
|
||||
const normalized = line.trim();
|
||||
if (normalized) {
|
||||
counts.set(normalized, (counts.get(normalized) ?? 0) + 1);
|
||||
lineCount++;
|
||||
}
|
||||
}
|
||||
// Similar prefixes in distinct records are not a loop, even when their windows repeat.
|
||||
if (lineCount >= MIN_REPEATS && counts.size > lineCount / 2) {
|
||||
return false;
|
||||
}
|
||||
const dominantChars = text.length / 2;
|
||||
let repeatedLineChars = 0;
|
||||
for (const [line, count] of counts) {
|
||||
if (count >= MIN_REPEATS) {
|
||||
repeatedLineChars = Math.max(repeatedLineChars, line.length * count);
|
||||
}
|
||||
}
|
||||
if (repeatedLineChars >= dominantChars) {
|
||||
return true;
|
||||
}
|
||||
|
||||
const windows = new Map<string, { count: number; end: number }>();
|
||||
const needed = Math.max(MIN_REPEATS, Math.ceil(dominantChars / WINDOW_CHARS));
|
||||
for (let start = 0; start <= text.length - WINDOW_CHARS; start++) {
|
||||
const window = text.slice(start, start + WINDOW_CHARS);
|
||||
const previous = windows.get(window);
|
||||
if (previous && start < previous.end) {
|
||||
continue;
|
||||
}
|
||||
const count = (previous?.count ?? 0) + 1;
|
||||
if (count >= needed) {
|
||||
return true;
|
||||
}
|
||||
windows.set(window, { count, end: start + WINDOW_CHARS });
|
||||
}
|
||||
|
||||
// Sample exact periods as well: a long repeated unit can have few equal 60-char windows.
|
||||
const lastStart = text.length - WINDOW_CHARS;
|
||||
const step = Math.max(1, Math.ceil(lastStart / 31));
|
||||
for (let sample = 0; sample <= 31; sample++) {
|
||||
const start = Math.min(sample * step, lastStart);
|
||||
const anchor = text.slice(start, start + WINDOW_CHARS);
|
||||
let match = start;
|
||||
for (let attempt = 0; attempt < 8; attempt++) {
|
||||
match = text.indexOf(anchor, match + 1);
|
||||
if (match < 0) {
|
||||
break;
|
||||
}
|
||||
const period = match - start;
|
||||
let left = start;
|
||||
let right = start + WINDOW_CHARS;
|
||||
while (left > 0 && text[left - 1] === text[left - 1 + period]) {
|
||||
left--;
|
||||
}
|
||||
while (right + period < text.length && text[right] === text[right + period]) {
|
||||
right++;
|
||||
}
|
||||
const coverage = right + period - left;
|
||||
if (coverage >= MIN_REPEATS * period && coverage >= dominantChars) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
if (start === lastStart) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/** Hermes-inspired long-output guard, scoped to one visible-text or reasoning block. */
|
||||
export class RepetitionWatch {
|
||||
private parts: string[] = [];
|
||||
private chars = 0;
|
||||
private nextCheck = FIRST_CHECK_CHARS;
|
||||
private stopped = false;
|
||||
|
||||
feed(delta: string): boolean {
|
||||
if (this.stopped) {
|
||||
return true;
|
||||
}
|
||||
let offset = 0;
|
||||
while (offset < delta.length) {
|
||||
const end = Math.min(delta.length, offset + this.nextCheck - this.chars);
|
||||
this.parts.push(delta.slice(offset, end));
|
||||
this.chars += end - offset;
|
||||
offset = end;
|
||||
if (this.chars === this.nextCheck) {
|
||||
const tail = this.parts.join("").slice(-TAIL_CHARS);
|
||||
this.parts = [tail];
|
||||
this.nextCheck += Math.min(this.chars, TAIL_CHARS);
|
||||
if (isRepetitionDominated(tail)) {
|
||||
this.stopped = true;
|
||||
return true;
|
||||
}
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
}
|
||||
@@ -191,6 +191,8 @@ export type ToolLoopRecoveryState = {
|
||||
};
|
||||
|
||||
export interface AgentLoopConfig extends SimpleStreamOptions {
|
||||
/** @internal Host-resolved small-model repetition policy. */
|
||||
repetitionGuard?: (model: Model) => boolean;
|
||||
model: Model;
|
||||
/** Logical thinking level retained across model changes before provider mapping. */
|
||||
thinkingLevel?: ThinkingLevel;
|
||||
|
||||
@@ -38,6 +38,20 @@ function makeAssistant(
|
||||
};
|
||||
}
|
||||
|
||||
it("keeps host-sanitized repetition history through provider replay", () => {
|
||||
const message = {
|
||||
...makeAssistant(
|
||||
[
|
||||
{ type: "text", text: "Earlier answer stays intact." },
|
||||
{ type: "text", text: "[Repetitive output stopped]" },
|
||||
],
|
||||
"error",
|
||||
),
|
||||
errorCode: "openclaw_repetitive_output",
|
||||
};
|
||||
expect(transformMessages([message], model)).toEqual([message]);
|
||||
});
|
||||
|
||||
describe.each(["aborted"] as const)(
|
||||
"%s assistant replay at the provider boundary",
|
||||
(stopReason) => {
|
||||
|
||||
@@ -1,8 +1,10 @@
|
||||
import { REPETITIVE_OUTPUT_ERROR_CODE } from "@openclaw/llm-core/types";
|
||||
import { asOptionalRecord } from "@openclaw/normalization-core/record-coerce";
|
||||
|
||||
type AssistantTurnLike = {
|
||||
role?: unknown;
|
||||
stopReason?: unknown;
|
||||
errorCode?: unknown;
|
||||
content?: unknown;
|
||||
};
|
||||
|
||||
@@ -87,6 +89,10 @@ export function resolveFailedAssistantReplay(
|
||||
) {
|
||||
return "keep";
|
||||
}
|
||||
// The stream owner already removed the bad block and retained safe history.
|
||||
if (message.errorCode === REPETITIVE_OUTPUT_ERROR_CODE) {
|
||||
return "keep";
|
||||
}
|
||||
const content = Array.isArray(message.content) ? message.content : [];
|
||||
if (content.some((block) => asOptionalRecord(block)?.type === "toolCall")) {
|
||||
// Pairing repair needs the failed call and its owned results together.
|
||||
|
||||
@@ -359,6 +359,8 @@ export type StopReason = "stop" | "length" | "toolUse" | "error" | "aborted";
|
||||
/** Stable error codes for provider outcomes that cannot be replayed safely. */
|
||||
export const PROVIDER_POST_DISPATCH_AMBIGUITY_ERROR_CODE = "PROVIDER_POST_DISPATCH_AMBIGUITY";
|
||||
export const PROVIDER_FAILURE_WITH_OUTPUT_ERROR_CODE = "PROVIDER_FAILURE_WITH_OUTPUT";
|
||||
/** The stream owner removed repetitive content before persistence and replay. */
|
||||
export const REPETITIVE_OUTPUT_ERROR_CODE = "openclaw_repetitive_output";
|
||||
/** Pre-dispatch argument rejection; callers still enforce output and effect guards. */
|
||||
export const MALFORMED_TOOL_CALL_ARGUMENTS_ERROR_CODE = "malformed_tool_call_arguments";
|
||||
|
||||
@@ -823,6 +825,8 @@ export interface Model<TApi extends Api = Api> {
|
||||
* smaller practical window.
|
||||
*/
|
||||
contextTokens?: number;
|
||||
/** Provider/catalog-declared total parameter count, including inactive experts. */
|
||||
parameterCount?: number;
|
||||
maxTokens: number;
|
||||
/** Provider-specific request/runtime parameters passed through to provider plugins. */
|
||||
params?: Record<string, unknown>;
|
||||
|
||||
@@ -21,6 +21,7 @@ describe("model catalog normalization", () => {
|
||||
],
|
||||
contextWindowDefault: "1m",
|
||||
contextTokens: 200000,
|
||||
parameterCount: 70_000_000_000,
|
||||
maxTokens: 128000,
|
||||
thinkingLevelMap: { off: null, minimal: "low", max: "max" },
|
||||
cost: {
|
||||
|
||||
@@ -421,6 +421,7 @@ function normalizeModelCatalogModel(value: unknown): ModelCatalogModel | undefin
|
||||
const contextWindow = normalizePositiveNumber(value.contextWindow);
|
||||
const contextWindowSelection = normalizeModelCatalogContextWindowSelection(value);
|
||||
const contextTokens = normalizePositiveInteger(value.contextTokens);
|
||||
const parameterCount = normalizePositiveInteger(value.parameterCount);
|
||||
const maxTokens = normalizePositiveNumber(value.maxTokens);
|
||||
const thinkingLevelMap = normalizeModelCatalogThinkingLevelMap(value.thinkingLevelMap);
|
||||
const cost = normalizeModelCatalogCost(value.cost);
|
||||
@@ -443,6 +444,7 @@ function normalizeModelCatalogModel(value: unknown): ModelCatalogModel | undefin
|
||||
...(contextWindow !== undefined ? { contextWindow } : {}),
|
||||
...contextWindowSelection,
|
||||
...(contextTokens !== undefined ? { contextTokens } : {}),
|
||||
...(parameterCount !== undefined ? { parameterCount } : {}),
|
||||
...(maxTokens !== undefined ? { maxTokens } : {}),
|
||||
...(thinkingLevelMap ? { thinkingLevelMap } : {}),
|
||||
...(cost ? { cost } : {}),
|
||||
|
||||
@@ -233,6 +233,8 @@ export type ModelCatalogModel = {
|
||||
contextWindows?: ModelCatalogContextWindowOption[];
|
||||
contextWindowDefault?: string;
|
||||
contextTokens?: number;
|
||||
/** Total model parameters, including inactive experts. */
|
||||
parameterCount?: number;
|
||||
maxTokens?: number;
|
||||
thinkingLevelMap?: ModelCatalogThinkingLevelMap;
|
||||
cost?: ModelCatalogCost;
|
||||
|
||||
@@ -61,6 +61,7 @@ const modelFieldsSchema = z.object({
|
||||
.optional(),
|
||||
contextWindowDefault: z.string().trim().min(1).optional(),
|
||||
contextTokens: z.number().int().positive().optional(),
|
||||
parameterCount: z.number().int().positive().optional(),
|
||||
maxTokens: z.number().finite().positive().optional(),
|
||||
thinkingLevelMap: z
|
||||
.partialRecord(z.enum(MODEL_CATALOG_THINKING_LEVELS), z.string().nullable())
|
||||
|
||||
@@ -52,6 +52,17 @@ describe("formatAssistantErrorText streaming JSON parse classification", () => {
|
||||
},
|
||||
);
|
||||
|
||||
it("preserves repetition-stop guidance instead of provider diagnostics", () => {
|
||||
expect(
|
||||
formatUserFacingAssistantErrorText({
|
||||
...makeAssistantError("HTTP 503 temporary failure"),
|
||||
errorCode: "openclaw_repetitive_output",
|
||||
}),
|
||||
).toBe(
|
||||
"Repetitive output stopped. Try a larger model or shorten your request before retrying.",
|
||||
);
|
||||
});
|
||||
|
||||
it.each(["request timed out", "LLM request timed out."])(
|
||||
"preserves safe user-facing timeout copy for provider error %j",
|
||||
(errorMessage) => {
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
import { truncateUtf16Safe } from "@openclaw/normalization-core/utf16-slice";
|
||||
import {
|
||||
REPETITIVE_OUTPUT_ERROR_MESSAGE,
|
||||
REPEATED_TOOL_ERROR_CODE,
|
||||
REPEATED_TOOL_ERROR_MESSAGE,
|
||||
} from "../../../packages/agent-core/src/errors.js";
|
||||
import type { OpenClawConfig } from "../../config/types.openclaw.js";
|
||||
import { classifyGatewayStorageFailure } from "../../infra/sqlite-error-diagnostics.js";
|
||||
import type { AssistantMessage } from "../../llm/types.js";
|
||||
import { REPETITIVE_OUTPUT_ERROR_CODE, type AssistantMessage } from "../../llm/types.js";
|
||||
import { createSubsystemLogger } from "../../logging/subsystem.js";
|
||||
import {
|
||||
CONTEXT_OVERFLOW_ERROR_MESSAGE,
|
||||
@@ -313,6 +314,9 @@ export function formatUserFacingAssistantErrorText(
|
||||
msg: AssistantMessage,
|
||||
opts?: AssistantErrorTextOptions,
|
||||
): string {
|
||||
if (msg.errorCode === REPETITIVE_OUTPUT_ERROR_CODE) {
|
||||
return REPETITIVE_OUTPUT_ERROR_MESSAGE;
|
||||
}
|
||||
if (msg.errorCode === REPEATED_TOOL_ERROR_CODE) {
|
||||
return REPEATED_TOOL_ERROR_MESSAGE;
|
||||
}
|
||||
|
||||
@@ -7,7 +7,10 @@ import type { PluginMetadataSnapshotOwnerMaps } from "../../plugins/plugin-metad
|
||||
import { createProviderModelCatalogIdNormalizer } from "../../plugins/provider-model-routes.js";
|
||||
import type { ProviderRuntimeModel } from "../../plugins/provider-runtime-model.types.js";
|
||||
import { DEFAULT_CONTEXT_TOKENS } from "../defaults.js";
|
||||
import { resolveCatalogOwnedModelCompat } from "../model-compat-catalog.js";
|
||||
import {
|
||||
modelTransportRoutesMatch,
|
||||
resolveCatalogOwnedModelCompat,
|
||||
} from "../model-compat-catalog.js";
|
||||
import { attachModelProviderLocalService } from "../provider-local-service.js";
|
||||
import {
|
||||
attachModelProviderRequestRouteFacts,
|
||||
@@ -180,6 +183,12 @@ export function buildConfiguredFallbackModel(params: {
|
||||
}),
|
||||
contextWindow: resolvedFallbackContextWindow,
|
||||
contextTokens: configuredModel?.contextTokens ?? staticCatalogModel?.contextTokens,
|
||||
parameterCount:
|
||||
configuredModel?.parameterCount ??
|
||||
(staticCatalogModel &&
|
||||
modelTransportRoutesMatch(staticCatalogModel, fallbackTransport)
|
||||
? staticCatalogModel.parameterCount
|
||||
: undefined),
|
||||
// maxTokens is a wire-level output cap, not a context-budget fallback.
|
||||
// Omit an unknown cap so strict providers can apply their own limit.
|
||||
...(normalizedResolvedFallbackMaxTokens !== undefined
|
||||
|
||||
@@ -18,6 +18,7 @@ function resolve(configuredModel?: Partial<ModelDefinitionConfig>) {
|
||||
baseUrl,
|
||||
contextWindow: 128_000,
|
||||
contextWindowSource: "synthetic",
|
||||
parameterCount: 70_000_000_000,
|
||||
};
|
||||
const providerConfig = {
|
||||
baseUrl,
|
||||
@@ -40,6 +41,13 @@ function resolve(configuredModel?: Partial<ModelDefinitionConfig>) {
|
||||
}
|
||||
|
||||
describe("configured overrides and synthetic context provenance", () => {
|
||||
it.each([
|
||||
[baseUrl, 70_000_000_000],
|
||||
["https://different.example/v1", undefined],
|
||||
])("binds total parameter metadata to the selected route %s", (configuredBaseUrl, expected) => {
|
||||
expect(resolve({ baseUrl: configuredBaseUrl })?.parameterCount).toBe(expected);
|
||||
});
|
||||
|
||||
it("drops the synthetic marker when config authors the context window", () => {
|
||||
const model = resolve({ contextWindow: 64_000 });
|
||||
expect(model?.contextWindow).toBe(64_000);
|
||||
|
||||
@@ -260,6 +260,11 @@ export function mergeCatalogInlineModel(
|
||||
headers: inlineModel.headers ?? catalogModel.headers,
|
||||
contextWindow: inlineModel.contextWindow ?? catalogModel.contextWindow,
|
||||
contextTokens: inlineModel.contextTokens ?? catalogModel.contextTokens,
|
||||
parameterCount:
|
||||
inlineModel.parameterCount ??
|
||||
(modelTransportRoutesMatch(catalogModel, inlineModel)
|
||||
? catalogModel.parameterCount
|
||||
: undefined),
|
||||
compat,
|
||||
...(mediaInput ? { mediaInput } : {}),
|
||||
...(params ? { params } : {}),
|
||||
@@ -608,6 +613,14 @@ export function applyConfiguredProviderOverrides(params: {
|
||||
}),
|
||||
contextWindow,
|
||||
contextTokens: metadataOverrideModel?.contextTokens ?? discoveredModel.contextTokens,
|
||||
parameterCount:
|
||||
metadataOverrideModel?.parameterCount ??
|
||||
(modelTransportRoutesMatch(discoveredModel, resolvedTransport)
|
||||
? discoveredModel.parameterCount
|
||||
: undefined) ??
|
||||
(modelTransportRoutesMatch(catalogModel, resolvedTransport)
|
||||
? catalogModel.parameterCount
|
||||
: undefined),
|
||||
...(normalizedResolvedMaxTokens !== undefined
|
||||
? {
|
||||
maxTokens: normalizedResolvedMaxTokens,
|
||||
|
||||
@@ -18,6 +18,7 @@ export function modelFromStaticCatalogRow(row: NormalizedModelCatalogRow): Provi
|
||||
contextWindows: row.contextWindows?.map((option) => ({ ...option })),
|
||||
contextWindowDefault: row.contextWindowDefault,
|
||||
contextTokens: row.contextTokens,
|
||||
...(row.parameterCount !== undefined ? { parameterCount: row.parameterCount } : {}),
|
||||
maxTokens: row.maxTokens ?? DEFAULT_CONTEXT_TOKENS,
|
||||
thinkingLevelMap: row.thinkingLevelMap ? { ...row.thinkingLevelMap } : undefined,
|
||||
headers: row.headers,
|
||||
|
||||
@@ -655,21 +655,29 @@ describe("resolveModel", () => {
|
||||
expect(discoverModelsAsync).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("leaves maxTokens undefined when no configured or catalog value is available (regression: #98295)", async () => {
|
||||
// Strict providers reject synthesized caps above their own completion-token ceiling.
|
||||
resolveBundledStaticCatalogModelMock.mockReturnValueOnce(undefined);
|
||||
const cfg = makeProviderConfig("xiaomi", {
|
||||
baseUrl: "https://api.xiaomimimo.com/v1",
|
||||
models: [{ id: "mimo-v2.5-pro", name: "mimo-v2.5-pro" }],
|
||||
});
|
||||
it.each([
|
||||
["xiaomi", "https://api.xiaomimimo.com/v1", "mimo-v2.5-pro", undefined],
|
||||
["local-custom", "http://127.0.0.1:1234/v1", "local-model", "openai-completions"],
|
||||
["local-ollama", "http://127.0.0.1:11434", "local-model", "ollama"],
|
||||
] as const)(
|
||||
"leaves unknown output caps unset for %s (regression: #98295)",
|
||||
async (provider, baseUrl, modelId, api) => {
|
||||
// Strict providers reject synthesized caps above their own completion-token ceiling.
|
||||
resolveBundledStaticCatalogModelMock.mockReturnValueOnce(undefined);
|
||||
const cfg = makeProviderConfig(provider, {
|
||||
baseUrl,
|
||||
...(api ? { api } : {}),
|
||||
models: [{ id: modelId, name: modelId }],
|
||||
});
|
||||
|
||||
const result = await resolveModelForTest("xiaomi", "mimo-v2.5-pro", cfg);
|
||||
const model = expectResolvedModel(result);
|
||||
const result = await resolveModelForTest(provider, modelId, cfg);
|
||||
const model = expectResolvedModel(result);
|
||||
|
||||
expect(model.id).toBe("mimo-v2.5-pro");
|
||||
expect(model.baseUrl).toBe("https://api.xiaomimimo.com/v1");
|
||||
expect(model.maxTokens).toBeUndefined();
|
||||
});
|
||||
expect(model.id).toBe(modelId);
|
||||
expect(model.baseUrl).toBe(baseUrl);
|
||||
expect(model.maxTokens).toBeUndefined();
|
||||
},
|
||||
);
|
||||
|
||||
it("inherits bundled static transport for configured provider fallback models", async () => {
|
||||
resolveBundledStaticCatalogModelMock.mockReturnValueOnce(
|
||||
|
||||
@@ -6,6 +6,7 @@ import type { AssistantMessage } from "../../../llm/types.js";
|
||||
import type { DiagnosticEmbeddedRunOwner } from "../../../logging/diagnostic-run-activity.js";
|
||||
import { resolveToolCallArgumentsEncoding } from "../../../plugins/provider-model-compat.js";
|
||||
import { captureAsyncWorkTracker } from "../../../shared/async-work-scope.js";
|
||||
import { isSmallModel } from "../../../shared/model-param-b.js";
|
||||
import {
|
||||
assertOperatorModelAllowed,
|
||||
readRunOperatorAuthority,
|
||||
@@ -13,6 +14,7 @@ import {
|
||||
import { shouldAllowProviderOwnedThinkingReplay } from "../../embedded-agent-helpers/turns.js";
|
||||
import { wrapStreamFnTextTransforms } from "../../plugin-text-transforms.js";
|
||||
import type { StreamFn } from "../../runtime/index.js";
|
||||
import { setInternalRepetitionGuard } from "../../runtime/internal-hooks.js";
|
||||
import { withSessionManagerWrite } from "../../sessions/session-manager-write-admission.js";
|
||||
import { resolveAgentTimeoutMs } from "../../timeout.js";
|
||||
import { UNKNOWN_TOOL_THRESHOLD } from "../../tool-loop-detection.js";
|
||||
@@ -126,6 +128,7 @@ export function installEmbeddedAttemptStreamGuards(
|
||||
providerTextTransforms,
|
||||
},
|
||||
} = input.prepared.sessionRuntime;
|
||||
setInternalRepetitionGuard(session.agent, isSmallModel);
|
||||
const { liveAllowedToolNames, replayAllowedToolNames } =
|
||||
input.prepared.toolCatalog.toolSearchRunPlan;
|
||||
const installStreamWrapper = <Args extends unknown[]>(
|
||||
|
||||
@@ -17,6 +17,7 @@ describe("model catalog metadata projection", () => {
|
||||
maxTokens: 4096,
|
||||
contextWindow: 96_000,
|
||||
contextTokens: 48_000,
|
||||
parameterCount: 4_000_000_000,
|
||||
contextWindows: [{ id: "large", label: "Large", contextWindow: 96_000 }],
|
||||
contextWindowDefault: "large",
|
||||
thinkingLevelMap: { off: null, high: "high" },
|
||||
@@ -36,6 +37,7 @@ describe("model catalog metadata projection", () => {
|
||||
api: "openai-responses",
|
||||
contextWindow: 96_000,
|
||||
contextTokens: 48_000,
|
||||
parameterCount: 4_000_000_000,
|
||||
contextWindows: [{ id: "large", label: "Large", contextWindow: 96_000 }],
|
||||
contextWindowDefault: "large",
|
||||
reasoning: true,
|
||||
@@ -74,6 +76,7 @@ describe("model catalog metadata projection", () => {
|
||||
baseUrl: "https://first.example/v1",
|
||||
contextWindow: 128_000,
|
||||
contextWindowSource: "synthetic" as const,
|
||||
parameterCount: 4_000_000_000,
|
||||
};
|
||||
expect(
|
||||
overlayCatalogMetadata(base, {
|
||||
@@ -82,14 +85,14 @@ describe("model catalog metadata projection", () => {
|
||||
contextWindowSource: undefined,
|
||||
}).contextWindowSource,
|
||||
).toBeUndefined();
|
||||
expect(
|
||||
overlayCatalogMetadata(base, {
|
||||
provider: "fixture",
|
||||
id: "m",
|
||||
name: "M",
|
||||
baseUrl: "https://second.example/v1",
|
||||
}).contextWindowSource,
|
||||
).toBeUndefined();
|
||||
const moved = overlayCatalogMetadata(base, {
|
||||
provider: "fixture",
|
||||
id: "m",
|
||||
name: "M",
|
||||
baseUrl: "https://second.example/v1",
|
||||
});
|
||||
expect(moved.contextWindowSource).toBeUndefined();
|
||||
expect(moved.parameterCount).toBeUndefined();
|
||||
expect(
|
||||
overlayCatalogMetadata(base, { provider: "fixture", id: "m", name: "M", reasoning: true })
|
||||
.contextWindowSource,
|
||||
|
||||
@@ -41,6 +41,7 @@ export function modelCatalogRowToEntry(
|
||||
: {}),
|
||||
...(row.contextWindowDefault ? { contextWindowDefault: row.contextWindowDefault } : {}),
|
||||
...(contextTokens !== undefined ? { contextTokens } : {}),
|
||||
...(row.parameterCount !== undefined ? { parameterCount: row.parameterCount } : {}),
|
||||
reasoning: row.reasoning,
|
||||
...(row.thinkingLevelMap ? { thinkingLevelMap: { ...row.thinkingLevelMap } } : {}),
|
||||
...(row.input ? { input: [...row.input] } : {}),
|
||||
|
||||
@@ -38,6 +38,7 @@ function clearRouteBoundCatalogMetadata(
|
||||
contextWindows: _contextWindows,
|
||||
contextWindowDefault: _contextWindowDefault,
|
||||
contextTokens: _contextTokens,
|
||||
parameterCount: _parameterCount,
|
||||
reasoning: _reasoning,
|
||||
configuredReasoning: _configuredReasoning,
|
||||
thinkingPolicyProvider: _thinkingPolicyProvider,
|
||||
@@ -108,6 +109,7 @@ export function overlayCatalogMetadata(
|
||||
? { contextWindowSource: overlay.contextWindowSource }
|
||||
: {}),
|
||||
...(overlay.contextTokens !== undefined ? { contextTokens: overlay.contextTokens } : {}),
|
||||
...(overlay.parameterCount !== undefined ? { parameterCount: overlay.parameterCount } : {}),
|
||||
...(overlay.contextCapacitySource
|
||||
? { contextCapacitySource: overlay.contextCapacitySource }
|
||||
: {}),
|
||||
|
||||
@@ -200,6 +200,7 @@ export function projectModelCatalogEntryForRoute(params: {
|
||||
? { contextWindow: donor.contextWindow, contextWindowSource: donor.contextWindowSource }
|
||||
: {}),
|
||||
...(donor?.contextTokens !== undefined ? { contextTokens: donor.contextTokens } : {}),
|
||||
...(donor?.parameterCount !== undefined ? { parameterCount: donor.parameterCount } : {}),
|
||||
...(donor?.contextCapacitySource
|
||||
? { contextCapacitySource: donor.contextCapacitySource }
|
||||
: {}),
|
||||
|
||||
@@ -27,6 +27,8 @@ export type ModelCatalogEntry = {
|
||||
contextWindows?: ModelCatalogContextWindowOption[];
|
||||
contextWindowDefault?: string;
|
||||
contextTokens?: number;
|
||||
/** Provider/catalog-declared total parameter count. */
|
||||
parameterCount?: number;
|
||||
/** Visible failed-acquisition starter; sizing comes from current static or accepted owner facts. */
|
||||
contextCapacitySource?: "unaccepted-starter";
|
||||
/** Provider-declared unknown-model estimate; not an authored or curated static limit. */
|
||||
|
||||
@@ -83,6 +83,29 @@ describe("scanOpenRouterModels", () => {
|
||||
expect(byPricing.image.skipped).toBe(true);
|
||||
});
|
||||
|
||||
it("keeps independently sized MoE models when filtering by minimum parameters", async () => {
|
||||
const fetchImpl = createFetchFixture({
|
||||
data: [
|
||||
{
|
||||
id: "meta-llama/llama4-maverick-17b-128e:free",
|
||||
name: "Llama 4 Maverick 400B",
|
||||
},
|
||||
{
|
||||
id: "meta-llama/llama4-scout-17b-16e:free",
|
||||
name: "Llama 4 Scout",
|
||||
},
|
||||
{ id: "qwen/qwen3-30b-a3b:free", name: "Qwen3 30B A3B" },
|
||||
{ id: "meta-llama/llama-70b:free", name: "Llama 70B" },
|
||||
],
|
||||
});
|
||||
|
||||
const results = await scanOpenRouterModels({ fetchImpl, probe: false, minParamB: 100 });
|
||||
|
||||
expect(results.map(({ id, inferredParamB }) => ({ id, inferredParamB }))).toEqual([
|
||||
{ id: "meta-llama/llama4-maverick-17b-128e:free", inferredParamB: 400 },
|
||||
]);
|
||||
});
|
||||
|
||||
it("uses OpenRouter top-provider limits for scan metadata", async () => {
|
||||
const fetchImpl = createFetchFixture({
|
||||
data: [
|
||||
|
||||
@@ -27,7 +27,7 @@ import { DEFAULT_PROVIDER } from "./defaults.js";
|
||||
import { findModelCatalogEntry } from "./model-catalog-lookup.js";
|
||||
import { overlayCatalogMetadata } from "./model-catalog-metadata.js";
|
||||
import type { ModelCatalogEntry } from "./model-catalog.types.js";
|
||||
import { isVllmQwenThinkingCompat } from "./model-compat-catalog.js";
|
||||
import { isVllmQwenThinkingCompat, modelTransportRoutesMatch } from "./model-compat-catalog.js";
|
||||
import type { ModelFallbackRouteResolution } from "./model-fallback.types.js";
|
||||
import { splitTrailingAuthProfile } from "./model-ref-profile.js";
|
||||
import {
|
||||
@@ -1193,6 +1193,11 @@ export function buildConfiguredModelCatalog(params: {
|
||||
);
|
||||
const api = model.api ?? accepted?.api ?? provider.api;
|
||||
const baseUrl = model.baseUrl ?? accepted?.baseUrl ?? provider.baseUrl;
|
||||
const parameterCount =
|
||||
model.parameterCount ??
|
||||
(accepted && modelTransportRoutesMatch(accepted, { api, baseUrl })
|
||||
? accepted.parameterCount
|
||||
: undefined);
|
||||
const name = normalizeOptionalString(model?.name) || id;
|
||||
const contextWindow =
|
||||
typeof model?.contextWindow === "number" && model.contextWindow > 0
|
||||
@@ -1203,15 +1208,8 @@ export function buildConfiguredModelCatalog(params: {
|
||||
? model.contextTokens
|
||||
: undefined;
|
||||
const input = Array.isArray(model?.input) ? model.input : undefined;
|
||||
const modelParams =
|
||||
model?.params && typeof model.params === "object" ? model.params : undefined;
|
||||
const compat = model?.compat && typeof model.compat === "object" ? model.compat : undefined;
|
||||
const reasoning =
|
||||
typeof model?.reasoning === "boolean"
|
||||
? model.reasoning
|
||||
: isVllmQwenThinkingCompat(providerId, compat)
|
||||
? true
|
||||
: undefined;
|
||||
model.reasoning ?? (isVllmQwenThinkingCompat(providerId, model.compat) ? true : undefined);
|
||||
catalog.push({
|
||||
provider: providerId,
|
||||
id,
|
||||
@@ -1220,12 +1218,13 @@ export function buildConfiguredModelCatalog(params: {
|
||||
...(baseUrl ? { baseUrl } : {}),
|
||||
contextWindow,
|
||||
contextTokens,
|
||||
...(parameterCount !== undefined ? { parameterCount } : {}),
|
||||
reasoning,
|
||||
...(typeof model?.reasoning === "boolean" ? { configuredReasoning: model.reasoning } : {}),
|
||||
...(model.thinkingLevelMap ? { thinkingLevelMap: model.thinkingLevelMap } : {}),
|
||||
input,
|
||||
...(modelParams ? { params: modelParams } : {}),
|
||||
compat,
|
||||
...(model.params ? { params: model.params } : {}),
|
||||
compat: model.compat,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
@@ -9,6 +9,8 @@ import { createPluginMetadataSnapshotFixture } from "../plugins/plugin-metadata.
|
||||
import * as providerPolicy from "../plugins/provider-policy-surface.js";
|
||||
import { prepareModelCatalogThinkingPolicies } from "../plugins/provider-thinking.js";
|
||||
import { createEmptyPluginRegistry } from "../plugins/registry-empty.js";
|
||||
import { isSmallModel } from "../shared/model-param-b.js";
|
||||
import { makeModel } from "./embedded-agent-runner/model.test-harness.js";
|
||||
import {
|
||||
type ModelCatalogRoutePolicy,
|
||||
projectModelCatalogEntryForRoute,
|
||||
@@ -38,6 +40,60 @@ const routePolicy: ModelCatalogRoutePolicy = {
|
||||
};
|
||||
|
||||
describe("configured catalog route overlays", () => {
|
||||
it.each([undefined, 4_000_000_000])(
|
||||
"keeps discovered parameter counts with their configured physical route (override %s)",
|
||||
(parameterCount) => {
|
||||
const cfg: OpenClawConfig = {
|
||||
models: {
|
||||
providers: {
|
||||
"route-fixture": {
|
||||
...capturedRoute,
|
||||
models: [{ ...makeModel("compact-chat"), name: "Compact Chat", parameterCount }],
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
const catalog = buildConfiguredModelCatalog({
|
||||
cfg,
|
||||
catalog: [
|
||||
{
|
||||
...capturedRoute,
|
||||
provider: "route-fixture",
|
||||
id: "compact-chat",
|
||||
name: "Compact Chat",
|
||||
parameterCount: parameterCount === undefined ? 4_000_000_000 : 70_000_000_000,
|
||||
},
|
||||
],
|
||||
manifestPlugins: [],
|
||||
});
|
||||
const entry = expectDefined(catalog[0], "configured parameter metadata");
|
||||
expect(entry.parameterCount).toBe(4_000_000_000);
|
||||
for (const baseUrl of [capturedRoute.baseUrl, "https://different.example/v1"]) {
|
||||
const { entry: projected, runtimeEntry } = projectModelCatalogEntryForRoute({
|
||||
entry,
|
||||
catalog,
|
||||
projection: {
|
||||
kind: "selected",
|
||||
route: {
|
||||
...capturedRoute,
|
||||
baseUrl,
|
||||
authRequirement: "api-key",
|
||||
requestTransportOverrides: "none",
|
||||
},
|
||||
policy: routePolicy,
|
||||
},
|
||||
overrides: createConfiguredModelCatalogOverridesResolver({ cfg, policy: routePolicy })(
|
||||
entry,
|
||||
),
|
||||
});
|
||||
const matchesConfiguredRoute = baseUrl === capturedRoute.baseUrl;
|
||||
expect(projected.parameterCount).toBe(matchesConfiguredRoute ? 4_000_000_000 : undefined);
|
||||
expect(runtimeEntry.parameterCount).toBe(projected.parameterCount);
|
||||
expect(isSmallModel(runtimeEntry)).toBe(matchesConfiguredRoute);
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it.each<OverlayCase>([
|
||||
{
|
||||
name: "same route with explicit disabled reasoning",
|
||||
|
||||
@@ -190,6 +190,11 @@ export function mergeProviderModels(
|
||||
configuredRoute,
|
||||
configuredCompat: explicitModel.compat,
|
||||
});
|
||||
const parameterCount =
|
||||
explicitModel.parameterCount ??
|
||||
(modelTransportRoutesMatch(catalogRoute, configuredRoute)
|
||||
? implicitModel.parameterCount
|
||||
: undefined);
|
||||
const contextSelection = explicitModel.contextWindows
|
||||
? explicitModel
|
||||
: modelTransportRoutesMatch(catalogRoute, configuredRoute)
|
||||
@@ -201,6 +206,7 @@ export function mergeProviderModels(
|
||||
baseUrl: _baseUrl,
|
||||
headers: _headers,
|
||||
maxTokensSource: _maxTokensSource,
|
||||
parameterCount: _parameterCount,
|
||||
...implicitMetadata
|
||||
} = implicitModel;
|
||||
return Object.assign(
|
||||
@@ -214,6 +220,7 @@ export function mergeProviderModels(
|
||||
},
|
||||
contextWindow === undefined ? {} : { contextWindow },
|
||||
contextTokens === undefined ? {} : { contextTokens },
|
||||
parameterCount === undefined ? {} : { parameterCount },
|
||||
maxTokens === undefined ? {} : { maxTokens },
|
||||
maxTokensSource === undefined ? {} : { maxTokensSource },
|
||||
{ compat },
|
||||
|
||||
@@ -12,6 +12,7 @@ export {
|
||||
getInternalToolExecutionPreparer,
|
||||
getInternalToolTurnCompletion,
|
||||
setInternalBeforeToolBatch,
|
||||
setInternalRepetitionGuard,
|
||||
setInternalToolTurnCompletion,
|
||||
type InternalBeforeToolBatchHook,
|
||||
type InternalToolExecutionPreparer,
|
||||
|
||||
@@ -145,6 +145,7 @@ const ModelDefinitionSchema = Type.Object({
|
||||
),
|
||||
contextWindow: Type.Optional(Type.Number()),
|
||||
contextTokens: Type.Optional(Type.Number()),
|
||||
parameterCount: Type.Optional(Type.Integer({ minimum: 1 })),
|
||||
maxTokens: Type.Optional(Type.Number()),
|
||||
params: Type.Optional(Type.Record(Type.String(), Type.Unknown())),
|
||||
headers: Type.Optional(Type.Record(Type.String(), Type.String())),
|
||||
|
||||
@@ -187,7 +187,7 @@ describe("ModelRegistry models.json auth", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it.each(["persisted", "registered"] as const)("preserves %s prompt budgets", (source) => {
|
||||
it.each(["persisted", "registered"] as const)("preserves %s model metadata", (source) => {
|
||||
// Synthetic providers deliberately share an id but not a prompt budget.
|
||||
// Native window and output capacity must remain separate from that budget.
|
||||
const providers: Record<string, ProviderConfigInput> = Object.fromEntries(
|
||||
@@ -210,6 +210,7 @@ describe("ModelRegistry models.json auth", () => {
|
||||
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
|
||||
contextWindow,
|
||||
contextTokens,
|
||||
parameterCount: 4_000_000_000,
|
||||
maxTokens: 128_000,
|
||||
},
|
||||
],
|
||||
|
||||
@@ -656,6 +656,7 @@ export class ModelRegistry {
|
||||
contextWindow: model.contextWindow ?? Math.max(model.contextTokens ?? 0, 128000),
|
||||
...(model.contextWindow === undefined ? { contextWindowSource: "synthetic" } : {}),
|
||||
contextTokens: model.contextTokens,
|
||||
...(model.parameterCount !== undefined ? { parameterCount: model.parameterCount } : {}),
|
||||
contextWindows: model.contextWindows,
|
||||
contextWindowDefault: model.contextWindowDefault,
|
||||
maxTokens: catalog ? (model.maxTokens ?? 16384) : model.maxTokens,
|
||||
@@ -709,10 +710,6 @@ export class ModelRegistry {
|
||||
return config;
|
||||
}
|
||||
|
||||
private getModelRequestKey(provider: string, modelId: string): string {
|
||||
return JSON.stringify([provider, modelId]);
|
||||
}
|
||||
|
||||
private storeProviderRequestConfig(
|
||||
providerName: string,
|
||||
config: {
|
||||
@@ -746,7 +743,7 @@ export class ModelRegistry {
|
||||
modelId: string,
|
||||
headers?: Record<string, string>,
|
||||
): void {
|
||||
const key = this.getModelRequestKey(providerName, modelId);
|
||||
const key = JSON.stringify([providerName, modelId]);
|
||||
if (!headers || Object.keys(headers).length === 0) {
|
||||
this.modelRequestHeaders.delete(key);
|
||||
return;
|
||||
@@ -758,7 +755,7 @@ export class ModelRegistry {
|
||||
return resolveModelRegistryRequestAuth({
|
||||
model,
|
||||
providerConfig: this.getModelProviderRequestConfig(model),
|
||||
modelHeaders: this.modelRequestHeaders.get(this.getModelRequestKey(model.provider, model.id)),
|
||||
modelHeaders: this.modelRequestHeaders.get(JSON.stringify([model.provider, model.id])),
|
||||
authStorage: this.authStorage,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -42,6 +42,7 @@ interface ProviderModelConfig
|
||||
| "thinkingLevelMap"
|
||||
| "input"
|
||||
| "maxTokens"
|
||||
| "parameterCount"
|
||||
| "headers"
|
||||
| "compat"
|
||||
>,
|
||||
|
||||
@@ -0,0 +1,111 @@
|
||||
import path from "node:path";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { loadPluginManifest } from "../plugins/manifest.js";
|
||||
import { createPluginManifestRecordFixture } from "../plugins/plugin-metadata.test-support.js";
|
||||
import { applyModelDefaults } from "./defaults.js";
|
||||
import { validateConfigObjectRaw } from "./validation-core.js";
|
||||
|
||||
const rootDir = path.resolve("extensions/ollama");
|
||||
const loaded = loadPluginManifest(rootDir);
|
||||
if (!loaded.ok) {
|
||||
throw new Error(loaded.error);
|
||||
}
|
||||
const manifestRegistry = {
|
||||
plugins: [createPluginManifestRecordFixture({ ...loaded.manifest, rootDir })],
|
||||
};
|
||||
|
||||
describe("Ollama configured output budgets", () => {
|
||||
it.each([
|
||||
["kimi-k2.6", 262_144],
|
||||
["kimi-k2.7-code", 262_144],
|
||||
["kimi-k3", 1_048_576],
|
||||
["deepseek-v4-pro", 1_048_576],
|
||||
["deepseek-v4-pro:0813", 1_048_576],
|
||||
["gemma4", 262_144],
|
||||
["gemma4:31b", 262_144],
|
||||
["glm-5.2", 131_072],
|
||||
["gpt-oss:120b", 131_072],
|
||||
["gpt-oss:20b", 131_072],
|
||||
["minimax-m2.7", 196_608],
|
||||
["minimax-m3", 524_288],
|
||||
["mistral-large-3:675b", 262_144],
|
||||
["nemotron-3-nano:30b", 262_144],
|
||||
["nemotron-3-super", 262_144],
|
||||
["nemotron-3-ultra", 262_144],
|
||||
])("materializes %s from the authored cloud catalog", (id, expected) => {
|
||||
const parsed = validateConfigObjectRaw({
|
||||
models: {
|
||||
providers: {
|
||||
"ollama-cloud": {
|
||||
baseUrl: "https://ollama.com",
|
||||
api: "ollama",
|
||||
models: [{ id, name: id }],
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
if (!parsed.ok) {
|
||||
throw new Error(JSON.stringify(parsed.issues));
|
||||
}
|
||||
const materialized = applyModelDefaults(parsed.config, { manifestRegistry });
|
||||
expect(materialized.models?.providers?.["ollama-cloud"]?.models[0]?.maxTokens).toBe(expected);
|
||||
});
|
||||
|
||||
it.each([
|
||||
{ id: "custom", baseUrl: "https://ollama.com", expected: 262_144 },
|
||||
{ id: "custom:cloud", baseUrl: "http://127.0.0.1:11434", expected: 262_144 },
|
||||
{ id: "custom", baseUrl: "http://127.0.0.1:11434", expected: 8192 },
|
||||
{ id: "minimax-m3", baseUrl: "http://127.0.0.1:11434", expected: 8192 },
|
||||
{ id: "kimi-k3", baseUrl: "http://127.0.0.1:11434", expected: 8192 },
|
||||
{
|
||||
id: "minimax-m3",
|
||||
baseUrl: "http://127.0.0.1:11434",
|
||||
modelBaseUrl: "https://ollama.com",
|
||||
expected: 8192,
|
||||
},
|
||||
{
|
||||
id: "minimax-m3",
|
||||
baseUrl: "https://ollama.com",
|
||||
modelBaseUrl: "http://127.0.0.1:11434",
|
||||
expected: 262_144,
|
||||
},
|
||||
{ id: "minimax-m3", baseUrl: "https://ollama.com", expected: 262_144 },
|
||||
{ id: "kimi-k3:cloud", baseUrl: "http://127.0.0.1:11434", expected: 262_144 },
|
||||
{ id: "minimax-m3", baseUrl: "http://127.0.0.1:11434", maxTokens: 1024, expected: 1024 },
|
||||
{ id: "custom", baseUrl: "https://ollama.com", maxTokens: 1024, expected: 1024 },
|
||||
{ id: "custom", baseUrl: "https://ollama.com", providerMaxTokens: 2048, expected: 2048 },
|
||||
])(
|
||||
"materializes $id at $baseUrl (model URL $modelBaseUrl) with authored cap $maxTokens",
|
||||
({ id, baseUrl, modelBaseUrl, maxTokens, providerMaxTokens, expected }) => {
|
||||
const parsed = validateConfigObjectRaw({
|
||||
models: {
|
||||
providers: {
|
||||
ollama: {
|
||||
baseUrl,
|
||||
api: "ollama",
|
||||
maxTokens: providerMaxTokens,
|
||||
models: [
|
||||
{
|
||||
id,
|
||||
name: id,
|
||||
baseUrl: modelBaseUrl,
|
||||
contextWindow: 262_144,
|
||||
maxTokens,
|
||||
params: { num_predict: -1 },
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
if (!parsed.ok) {
|
||||
throw new Error(JSON.stringify(parsed.issues));
|
||||
}
|
||||
const materialized = applyModelDefaults(parsed.config, { manifestRegistry });
|
||||
expect(materialized.models?.providers?.ollama?.models[0]).toMatchObject({
|
||||
maxTokens: expected,
|
||||
params: { num_predict: -1 },
|
||||
});
|
||||
},
|
||||
);
|
||||
});
|
||||
@@ -38,6 +38,8 @@ export type ModelDefinitionConfig = Omit<
|
||||
ModelDefinitionSchemaInput,
|
||||
"reasoning" | "input" | "cost" | "maxTokens" | "agentRuntime" | "mediaInput" | "compat"
|
||||
> & {
|
||||
/** Internal discovery metadata; not a user config option. */
|
||||
parameterCount?: number;
|
||||
/** Whether the model supports reasoning/thinking controls. */
|
||||
reasoning: boolean;
|
||||
/** Supported input modalities for routing and media-tool selection. */
|
||||
|
||||
@@ -81,6 +81,7 @@ describe("isRetryableAssistantError", () => {
|
||||
PROVIDER_FAILURE_WITH_OUTPUT_ERROR_CODE,
|
||||
PROVIDER_POST_DISPATCH_AMBIGUITY_ERROR_CODE,
|
||||
"openclaw_repeated_tool_error",
|
||||
"openclaw_repetitive_output",
|
||||
])("does not retry terminal outcome %s", (errorCode) => {
|
||||
const message = {
|
||||
...errorMessage("The WebSocket closed after dispatch"),
|
||||
|
||||
@@ -12,12 +12,14 @@ import {
|
||||
import { isSessionTranscriptTurnMismatchErrorMessage } from "../../agents/sessions/transcript-turn-error.js";
|
||||
import {
|
||||
PROVIDER_FAILURE_WITH_OUTPUT_ERROR_CODE,
|
||||
REPETITIVE_OUTPUT_ERROR_CODE,
|
||||
PROVIDER_POST_DISPATCH_AMBIGUITY_ERROR_CODE,
|
||||
type AssistantMessage,
|
||||
} from "../types.js";
|
||||
|
||||
const TERMINAL_ASSISTANT_ERROR_CODES = new Set([
|
||||
REPEATED_TOOL_ERROR_CODE,
|
||||
REPETITIVE_OUTPUT_ERROR_CODE,
|
||||
PROVIDER_FAILURE_WITH_OUTPUT_ERROR_CODE,
|
||||
PROVIDER_POST_DISPATCH_AMBIGUITY_ERROR_CODE,
|
||||
WEBSOCKET_NON_RETRYABLE_CLOSE_ERROR_CODE,
|
||||
|
||||
@@ -9,6 +9,7 @@ export {
|
||||
findNormalizedProviderValue,
|
||||
normalizeProviderId,
|
||||
} from "@openclaw/model-catalog-core/provider-id";
|
||||
export { isCloudModelRef } from "@openclaw/model-catalog-core/model-catalog-refs";
|
||||
export {
|
||||
normalizeAntigravityPreviewModelId,
|
||||
normalizeGooglePreviewModelId,
|
||||
|
||||
@@ -265,6 +265,7 @@ function buildOpenAICompatibleLiveModel(
|
||||
const topProvider = asOptionalRecord(record.top_provider);
|
||||
const modelInfo = asOptionalRecord(record.model_info);
|
||||
const nestedRecords = [capabilities, architecture, topProvider, modelInfo];
|
||||
const parameterCount = readLiveModelCatalogPositiveSafeIntegerField(record, "parameterCount");
|
||||
const advertisedChatCapability = rowAdvertisesChatModel(record, nestedRecords);
|
||||
if (
|
||||
advertisedChatCapability === false ||
|
||||
@@ -276,9 +277,17 @@ function buildOpenAICompatibleLiveModel(
|
||||
|
||||
const exact = fallback.models.find((model) => model.id === id);
|
||||
if (exact) {
|
||||
const liveContextWindow = readLiveModelContextWindow([record, ...nestedRecords]);
|
||||
return exact.contextWindow === undefined && liveContextWindow !== undefined
|
||||
? { ...exact, contextWindow: liveContextWindow }
|
||||
const contextWindow =
|
||||
exact.contextWindow ?? readLiveModelContextWindow([record, ...nestedRecords]);
|
||||
const resolvedParameterCount = exact.parameterCount ?? parameterCount;
|
||||
return contextWindow !== exact.contextWindow || resolvedParameterCount !== exact.parameterCount
|
||||
? {
|
||||
...exact,
|
||||
...(contextWindow !== undefined ? { contextWindow } : {}),
|
||||
...(resolvedParameterCount !== undefined
|
||||
? { parameterCount: resolvedParameterCount }
|
||||
: {}),
|
||||
}
|
||||
: exact;
|
||||
}
|
||||
// Only unknown ids need the provider's request-shaping gate.
|
||||
@@ -334,6 +343,7 @@ function buildOpenAICompatibleLiveModel(
|
||||
return {
|
||||
id,
|
||||
name: readLiveModelCatalogStringField(record, ["display_name", "displayName", "name"]) ?? id,
|
||||
...(parameterCount !== undefined ? { parameterCount } : {}),
|
||||
...(template?.api ? { api: template.api } : {}),
|
||||
reasoning,
|
||||
input,
|
||||
@@ -442,9 +452,11 @@ export function projectUpstreamProviderCatalogModel(params: {
|
||||
]
|
||||
: undefined;
|
||||
const contextTokens = readLiveModelCatalogPositiveSafeIntegerField(limit, "input");
|
||||
const parameterCount = readLiveModelCatalogPositiveSafeIntegerField(model, "parameterCount");
|
||||
return {
|
||||
id,
|
||||
name: readLiveModelCatalogStringField(model, "name") ?? id,
|
||||
...(parameterCount !== undefined ? { parameterCount } : {}),
|
||||
provider: params.providerId,
|
||||
api,
|
||||
baseUrl,
|
||||
|
||||
@@ -21,6 +21,42 @@ function buildModel(id: string): ModelDefinitionConfig {
|
||||
describe("live provider catalog projection", () => {
|
||||
beforeEach(() => clearLiveCatalogCacheForTests());
|
||||
|
||||
it("preserves identity-specific size facts through live catalog projection", async () => {
|
||||
const fetchGuard: LiveModelCatalogFetchGuard = async () => ({
|
||||
response: Response.json({
|
||||
data: [
|
||||
{ id: "known-chat", parameterCount: 4_000_000_000 },
|
||||
{ id: "enriched-chat", parameterCount: 4_000_000_000 },
|
||||
{ id: "novel-chat", parameterCount: 4_000_000_000 },
|
||||
{ id: "template-next" },
|
||||
{ id: "invalid-chat", parameterCount: 0 },
|
||||
],
|
||||
}),
|
||||
finalUrl: "https://provider.example.test/v1/models",
|
||||
release: async () => undefined,
|
||||
});
|
||||
const provider = await buildLiveModelProviderConfig({
|
||||
providerId: "provider",
|
||||
endpoint: "https://provider.example.test/v1/models",
|
||||
providerConfig: { api: "openai-completions", baseUrl: "https://provider.example.test/v1" },
|
||||
models: [
|
||||
{ ...buildModel("known-chat"), parameterCount: 70_000_000_000 },
|
||||
buildModel("enriched-chat"),
|
||||
{ ...buildModel("template-base"), parameterCount: 7_000_000_000 },
|
||||
],
|
||||
fetchGuard,
|
||||
});
|
||||
expect(
|
||||
Object.fromEntries(provider.models.map((model) => [model.id, model.parameterCount])),
|
||||
).toEqual({
|
||||
"known-chat": 70_000_000_000,
|
||||
"enriched-chat": 4_000_000_000,
|
||||
"novel-chat": 4_000_000_000,
|
||||
"template-next": undefined,
|
||||
"invalid-chat": undefined,
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps cache admission and fallback shared", async () => {
|
||||
const release = vi.fn(async () => undefined);
|
||||
const fetchGuard: MockedFunction<LiveModelCatalogFetchGuard> = vi
|
||||
|
||||
@@ -6,6 +6,7 @@ import {
|
||||
type LiveModelCatalogFetchGuard,
|
||||
type UpstreamProviderCatalog,
|
||||
} from "./provider-catalog-live-runtime.js";
|
||||
import { listProviderCatalogSnapshotEntries } from "./provider-catalog-snapshot.internal.js";
|
||||
|
||||
function buildFetchGuard(body: unknown): {
|
||||
fetchGuard: MockedFunction<LiveModelCatalogFetchGuard>;
|
||||
@@ -98,6 +99,7 @@ describe("shared upstream provider metadata catalogs", () => {
|
||||
"frontier-model": {
|
||||
id: "frontier-model",
|
||||
name: "Frontier Model",
|
||||
parameterCount: 4_000_000_000,
|
||||
reasoning: true,
|
||||
tool_call: true,
|
||||
reasoning_options: [{ type: "effort", values: ["low", "high", "high", false] }],
|
||||
@@ -127,6 +129,7 @@ describe("shared upstream provider metadata catalogs", () => {
|
||||
expect(model).toEqual({
|
||||
id: "frontier-model",
|
||||
name: "Frontier Model",
|
||||
parameterCount: 4_000_000_000,
|
||||
provider: "opencode-go",
|
||||
api: "openai-responses",
|
||||
baseUrl: "https://opencode.ai/zen/go/v1",
|
||||
@@ -158,6 +161,11 @@ describe("shared upstream provider metadata catalogs", () => {
|
||||
supportedReasoningEfforts: ["low", "high"],
|
||||
},
|
||||
});
|
||||
if (!model) {
|
||||
throw new Error("expected projected upstream model");
|
||||
}
|
||||
const snapshot = new Map([[model.id, { model }]]);
|
||||
expect(listProviderCatalogSnapshotEntries(snapshot)[0]?.parameterCount).toBe(4_000_000_000);
|
||||
},
|
||||
);
|
||||
|
||||
|
||||
@@ -79,6 +79,7 @@ export function listProviderCatalogSnapshotEntries(
|
||||
provider: model.provider,
|
||||
id: model.id,
|
||||
name: model.name,
|
||||
...(model.parameterCount !== undefined ? { parameterCount: model.parameterCount } : {}),
|
||||
api: model.api,
|
||||
baseUrl: model.baseUrl,
|
||||
reasoning: model.reasoning,
|
||||
|
||||
@@ -1,10 +1,61 @@
|
||||
// Covers provider catalog entries derived from plugin metadata.
|
||||
import { normalizeModelCatalogProviderRows } from "@openclaw/model-catalog-core/model-catalog-normalize";
|
||||
import type { ModelCatalogProvider } from "@openclaw/model-catalog-core/model-catalog-types";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import type { OpenClawConfig } from "../config/config.js";
|
||||
import type { ModelProviderConfig } from "../config/types.models.js";
|
||||
import { buildSingleProviderApiKeyCatalog, findCatalogTemplate } from "./provider-catalog.js";
|
||||
import {
|
||||
buildEffectiveManifestProviderConfig,
|
||||
buildManifestModelProviderConfig,
|
||||
buildManifestProviderCatalogFamily,
|
||||
buildSingleProviderApiKeyCatalog,
|
||||
findCatalogTemplate,
|
||||
} from "./provider-catalog.js";
|
||||
import type { ProviderCatalogContext } from "./types.js";
|
||||
|
||||
describe("manifest model size projection", () => {
|
||||
const providerId = "local-models";
|
||||
const catalog: ModelCatalogProvider = {
|
||||
baseUrl: "http://127.0.0.1:1234/v1",
|
||||
api: "openai-completions",
|
||||
models: [
|
||||
{
|
||||
id: "compact-chat",
|
||||
name: "Compact Chat",
|
||||
contextWindow: 32_768,
|
||||
maxTokens: 8192,
|
||||
parameterCount: 4_000_000_000,
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
it.each([
|
||||
["raw manifest", () => buildManifestModelProviderConfig({ providerId, catalog }).models],
|
||||
[
|
||||
"effective manifest",
|
||||
() =>
|
||||
buildEffectiveManifestProviderConfig(
|
||||
normalizeModelCatalogProviderRows({
|
||||
provider: providerId,
|
||||
providerCatalog: catalog,
|
||||
source: "manifest",
|
||||
}),
|
||||
)?.models,
|
||||
],
|
||||
[
|
||||
"family augmentation",
|
||||
() =>
|
||||
buildManifestProviderCatalogFamily({
|
||||
surfaces: [{ id: providerId, label: "Local models", catalog }],
|
||||
}).augmentModelCatalog(),
|
||||
],
|
||||
])("keeps total parameters through %s", (_name, project) => {
|
||||
expect(project()).toEqual([
|
||||
expect.objectContaining({ id: "compact-chat", parameterCount: 4_000_000_000 }),
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
function createProviderConfig(overrides: Partial<ModelProviderConfig> = {}): ModelProviderConfig {
|
||||
return {
|
||||
api: "openai-completions",
|
||||
|
||||
@@ -168,6 +168,7 @@ function buildManifestCatalogModel(
|
||||
: {}),
|
||||
...(model.contextWindowDefault ? { contextWindowDefault: model.contextWindowDefault } : {}),
|
||||
...(model.contextTokens !== undefined ? { contextTokens: model.contextTokens } : {}),
|
||||
...(model.parameterCount !== undefined ? { parameterCount: model.parameterCount } : {}),
|
||||
maxTokens: model.maxTokens,
|
||||
...(model.thinkingLevelMap ? { thinkingLevelMap: { ...model.thinkingLevelMap } } : {}),
|
||||
...(model.headers ? { headers: { ...model.headers } } : {}),
|
||||
@@ -289,6 +290,7 @@ export function buildManifestProviderCatalogFamily(params: {
|
||||
reasoning: entry.reasoning,
|
||||
input: [...entry.input],
|
||||
contextWindow: entry.contextWindow,
|
||||
...(entry.parameterCount !== undefined ? { parameterCount: entry.parameterCount } : {}),
|
||||
})),
|
||||
),
|
||||
};
|
||||
|
||||
@@ -38,6 +38,31 @@ async function discoverModelContexts(
|
||||
}
|
||||
|
||||
describe("discoverOpenAICompatibleLocalModels raw discovery", () => {
|
||||
it("retains discovered total parameter counts and display names", async () => {
|
||||
fetchWithSsrFGuardMock.mockResolvedValueOnce(
|
||||
guarded(
|
||||
Response.json({
|
||||
data: [
|
||||
{ id: "compact-chat", name: "Compact Chat", parameterCount: 4_000_000_000 },
|
||||
{ id: "alias", name: "Qwen3 4B" },
|
||||
{ id: "invalid", parameterCount: 0 },
|
||||
],
|
||||
}),
|
||||
),
|
||||
);
|
||||
const models = await discoverOpenAICompatibleLocalModels({
|
||||
baseUrl: "http://127.0.0.1:8080/v1",
|
||||
label: "self-hosted",
|
||||
discoverRuntimeContext: false,
|
||||
env: {},
|
||||
});
|
||||
expect(models.map(({ id, name, parameterCount }) => ({ id, name, parameterCount }))).toEqual([
|
||||
{ id: "compact-chat", name: "Compact Chat", parameterCount: 4_000_000_000 },
|
||||
{ id: "alias", name: "Qwen3 4B", parameterCount: undefined },
|
||||
{ id: "invalid", name: "invalid", parameterCount: undefined },
|
||||
]);
|
||||
});
|
||||
|
||||
it("recognizes reasoning aliases without enabling legacy chat models", async () => {
|
||||
const expected = [
|
||||
{ id: "yk_nemotron-3-super", reasoning: true },
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { asPositiveSafeInteger } from "@openclaw/normalization-core/number-coercion";
|
||||
import { asOptionalRecord, isRecord } from "@openclaw/normalization-core/record-coerce";
|
||||
import { normalizeOptionalString } from "@openclaw/normalization-core/string-coerce";
|
||||
import { isNonSecretApiKeyMarker } from "../agents/model-auth-markers.js";
|
||||
@@ -386,9 +387,15 @@ export async function discoverOpenAICompatibleLocalModels(
|
||||
const generationSettings = asOptionalRecord(props?.default_generation_settings);
|
||||
const runtimeContextTokens =
|
||||
readPositiveInteger(generationSettings?.n_ctx) ?? readPositiveInteger(props?.n_ctx);
|
||||
const parameterCount = asPositiveSafeInteger(model.parameterCount);
|
||||
const modelConfig: ModelDefinitionConfig = {
|
||||
id: modelId,
|
||||
name: modelId,
|
||||
name:
|
||||
normalizeOptionalString(model.display_name) ??
|
||||
normalizeOptionalString(model.displayName) ??
|
||||
normalizeOptionalString(model.name) ??
|
||||
modelId,
|
||||
...(parameterCount !== undefined ? { parameterCount } : {}),
|
||||
reasoning: /r1|reason|think|deepseek[-_]?v4|nemotron-3[-_](?:nano|super|ultra)\b/i.test(
|
||||
modelId,
|
||||
),
|
||||
|
||||
@@ -1,8 +1,28 @@
|
||||
// Model parameter B tests cover reasoning budget normalization and aliases.
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { inferParamBFromIdOrName } from "./model-param-b.js";
|
||||
import { inferParamBFromIdOrName, isSmallModel } from "./model-param-b.js";
|
||||
|
||||
describe("shared/model-param-b", () => {
|
||||
it.each([
|
||||
[{ id: "qwen2.5-coder:1.5b-instruct" }, true],
|
||||
[{ id: "qwen3:4b" }, true],
|
||||
[{ id: "model", name: "Dense 32B" }, true],
|
||||
[{ id: "model", parameterCount: 32_000_000_000 }, true],
|
||||
[{ id: "qwen:32b", parameterCount: 32_800_000_000 }, true],
|
||||
[{ id: "model", parameterCount: 33_999_999_999 }, true],
|
||||
[{ id: "model", parameterCount: 34_000_000_000 }, false],
|
||||
[{ id: "model-4b", parameterCount: 70_000_000_000 }, false],
|
||||
[{ id: "model-70b", parameterCount: 4_000_000_000 }, true],
|
||||
[{ id: "llama4-maverick-17b-128e" }, false],
|
||||
[{ id: "llama4-maverick-17b-128e", name: "Llama 4 Maverick 400B" }, false],
|
||||
[{ id: "custom-a3b" }, false],
|
||||
[{ id: "custom", name: "A3B" }, false],
|
||||
[{ id: "qwen3-30b-a3b" }, true],
|
||||
[{ id: "qwen3-235b-a22b" }, false],
|
||||
[{ id: "model", name: "Unknown size" }, false],
|
||||
] as const)("classifies total model size for %j", (model, expected) => {
|
||||
expect(isSmallModel(model)).toBe(expected);
|
||||
});
|
||||
|
||||
it("extracts the largest valid b-sized parameter token", () => {
|
||||
expect(inferParamBFromIdOrName("llama-8b mixtral-22b")).toBe(22);
|
||||
expect(inferParamBFromIdOrName("Qwen 0.5B Instruct")).toBe(0.5);
|
||||
@@ -16,6 +36,30 @@ describe("shared/model-param-b", () => {
|
||||
expect(inferParamBFromIdOrName("7b-13b")).toBe(13);
|
||||
});
|
||||
|
||||
it.each([
|
||||
"llama4-maverick-17b-128e",
|
||||
"llama4-scout-17b-16e-instruct",
|
||||
"Llama 4 Maverick 17B 128E",
|
||||
"custom-a3b",
|
||||
"Custom A3B",
|
||||
])("does not infer total parameters from an active-size expert tag: %s", (id) => {
|
||||
expect(inferParamBFromIdOrName(id)).toBeNull();
|
||||
});
|
||||
|
||||
it("preserves total parameter counts when active counts are also present", () => {
|
||||
expect(inferParamBFromIdOrName("qwen3-30b-a3b")).toBe(30);
|
||||
expect(inferParamBFromIdOrName("qwen3-235b-a22b")).toBe(235);
|
||||
expect(inferParamBFromIdOrName("custom-a3b Total 70B")).toBe(70);
|
||||
expect(inferParamBFromIdOrName("Total 70B custom-a3b")).toBe(70);
|
||||
});
|
||||
|
||||
it.each([
|
||||
"llama4-maverick-17b-128e Llama 4 Maverick 400B",
|
||||
"Llama 4 Maverick 400B llama4-maverick-17b-128e",
|
||||
])("preserves independent total size beside expert tags: %s", (text) => {
|
||||
expect(inferParamBFromIdOrName(text)).toBe(400);
|
||||
});
|
||||
|
||||
it("ignores malformed, zero, and non-delimited matches", () => {
|
||||
expect(inferParamBFromIdOrName("abc70beta 0b x70b2")).toBeNull();
|
||||
expect(inferParamBFromIdOrName("model 0b")).toBeNull();
|
||||
|
||||
@@ -5,9 +5,16 @@ export function inferParamBFromIdOrName(text: string): number | null {
|
||||
const raw = normalizeLowercaseStringOrEmpty(text);
|
||||
// Trailing boundary is a lookahead so two adjacent `<num>b` tokens sharing one delimiter (e.g.
|
||||
// "8b 70b" / "8b-70b") both match; a consuming boundary ate the delimiter and skipped the second.
|
||||
const matches = raw.matchAll(/(?:^|[^a-z0-9])[a-z]?(\d+(?:\.\d+)?)b(?=[^a-z0-9]|$)/g);
|
||||
// A<size>B denotes active parameters, so it cannot establish the total size.
|
||||
const matches = raw.matchAll(
|
||||
/(?:^|[^a-z0-9])[b-z]?(\d+(?:\.\d+)?)b(?:[-_\s]+(\d+)e)?(?=[^a-z0-9]|$)/g,
|
||||
);
|
||||
let best: number | null = null;
|
||||
for (const match of matches) {
|
||||
// Skip active-size expert tags, retaining any independently stated total size.
|
||||
if (match[2]) {
|
||||
continue;
|
||||
}
|
||||
const numRaw = match[1];
|
||||
const value = Number(numRaw);
|
||||
if (!Number.isFinite(value) || value <= 0) {
|
||||
@@ -19,3 +26,16 @@ export function inferParamBFromIdOrName(text: string): number | null {
|
||||
}
|
||||
return best;
|
||||
}
|
||||
|
||||
/** Restricts live repetition detection to the 32B class (allowing rounded metadata). */
|
||||
export function isSmallModel(model: {
|
||||
id: string;
|
||||
name?: string;
|
||||
parameterCount?: number;
|
||||
}): boolean {
|
||||
const paramB =
|
||||
model.parameterCount === undefined
|
||||
? inferParamBFromIdOrName(`${model.id} ${model.name ?? ""}`)
|
||||
: model.parameterCount / 1_000_000_000;
|
||||
return paramB !== null && paramB > 0 && paramB < 34;
|
||||
}
|
||||
@@ -522,3 +522,13 @@ it("keeps Git operation workers independent of worktree allocation and registry
|
||||
expect(closure).not.toContain("src/agents/worktrees/service-preparation.ts");
|
||||
expect(closure).not.toContain("src/agents/worktrees/allocation.ts");
|
||||
});
|
||||
|
||||
it("keeps Ollama config metadata independent of provider runtime helpers", () => {
|
||||
const closure = collectRuntimeImportClosure(process.cwd(), [
|
||||
"extensions/ollama/src/defaults.ts",
|
||||
"extensions/ollama/provider-policy-api.ts",
|
||||
]);
|
||||
expect(closure).not.toContain("src/plugin-sdk/provider-model-shared.ts");
|
||||
expect(closure).not.toContain("src/plugin-sdk/plugin-entry.ts");
|
||||
expect(closure).not.toContain("src/plugins/provider-replay-helpers.ts");
|
||||
});
|
||||
Reference in new issue
Block a user