Some vendors keep a task running for more than a day: Volcengine Ark waits
48 hours by default and up to 72 when configured, and Vidu off-peak jobs
can take 48 hours. With the 24-hour default the gateway failed and refunded
those tasks while the vendor was still producing, and charging for, the
result. TASK_TIMEOUT_MINUTES now defaults to 4320. A task that is truly
stuck is refunded after 72 hours instead of 24.
Stop Responses SSE after its completed event has been written, preserving
upstream token and tool usage. Reconcile a concurrent client cancellation
only after all scanner workers have joined and a completed response was
successfully delivered. Bare EOF, failed writes and protocol errors do not
qualify. Propagate SSE write/flush errors and suppress cleanup read errors.
Integrates the completion handling from #6904 and #6951 and narrows the
cancellation reconciliation proposed in #6808. Refs #6649.
Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com>
Co-authored-by: zkasuran <zkasuran@gmail.com>
Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com>
* feat(email): add Resend delivery provider
Add Resend as an alternative to SMTP with separately editable sender address
and name, credential-safe configuration status, and validation before activation.
Preserve the target repository's email notification and balance reminder settings.
Only Resend changes are transplanted from hicoderai/new-api; no landing,
legal-document, model-group, pricing, branding, or dependency changes are included.
Validation: Go common/controller email regression tests; 19 frontend email and
notification tests; frontend production build; tsgo typecheck; touched-file lint.
Frontend validation uses the target lockfile's Axios 1.18.1 with local cached tools.
Security references: OWASP Authentication, Session Management, and Forgot Password
Cheat Sheets. Tests cover credential non-disclosure, invalid activation, sanitized
transport errors, and no automatic retry or fallback; token and session flows
are unchanged.
https://cheatsheetseries.owasp.org/cheatsheets/Authentication_Cheat_Sheet.htmlhttps://cheatsheetseries.owasp.org/cheatsheets/Session_Management_Cheat_Sheet.htmlhttps://cheatsheetseries.owasp.org/cheatsheets/Forgot_Password_Cheat_Sheet.html
* fix(email): log Resend rejections and translate Resend errors
Resend's reason for a rejected or failed send now goes to the server log.
People who asked for the email still see the short error.
Resend and email method errors are translatable messages in all seven
languages.
The email settings hints are shorter and reuse the existing key hint.
The new frontend files carry the license header, the sender name check
passes lint, and test fixtures use neutral names.
* fix(email): check Resend settings while Resend is active
Saving the sender address or API key now checks the whole Resend setup
while Resend sends the mail. A typo the form accepts, such as
notify..team@example.com, is rejected instead of breaking every send.
Resend's error text and network errors are quoted in the server log.
---------
Co-authored-by: CaIon <i@caion.me>
* fix(notify): keep notification limit window from resetting every hour
The rate-limit key included the current clock hour, so the count reset on
every hour boundary and NOTIFICATION_LIMIT_DURATION_MINUTE above 60 had no
effect. Drop the hour suffix so the window runs for the configured duration
from the first notification, and make the Redis counter atomic with
SETNX+INCR in one transaction.
* fix(notify): fall back to default for nonpositive notification window
A zero or negative NOTIFICATION_LIMIT_DURATION_MINUTE left the Redis
counter without an expiry (or failed the SET), which blocked quota
notifications for good once the counter filled.
* test(notify): cover the Redis notification limit window
* fix(model): find channels for @-modified model names without the memory cache
With the channel memory cache off (no Redis and MEMORY_CACHE_ENABLED
unset), channel selection matched abilities by the exact requested name,
so a name such as deepseek-flash@effort:max found no channel. Retry with
the routing name when the exact name has no channel, as the cached path
already does.
* feat(deepseek): support V4.1 Flash, the Search and Fetch APIs, and @ reasoning modifiers
Follows the DeepSeek API update of 2026-10-17.
- Default models: deepseek-flash and deepseek-v4-pro, each with -none,
-low and -max, plus deepseek-search and deepseek-fetch. The retired
deepseek-chat, deepseek-reasoner and deepseek-v4-flash names are no
longer listed.
- Thinking: the -none/-low/-max tails also work on deepseek-flash.
@thinking and @effort now reach DeepSeek's own fields on Chat
(thinking, reasoning_effort), Anthropic (thinking, output_config.effort)
and Responses (reasoning.effort), and take precedence over the tails.
- Built-in prices: peak/off-peak expressions for deepseek-flash (also the
deepseek-v4-flash aliases) and deepseek-v4-pro, and $0.75 per 1,000
requests for deepseek-search and deepseek-fetch.
- Server-side search: searches and page fetches DeepSeek runs for Claude
Code and Codex bill at $0.75 per 1,000 under deepseek_search and
deepseek_fetch, whatever model name the client sent. Searches whose
results already arrived stay billed when a stream ends before its final
usage; this also applies to Alibaba's Anthropic endpoint.
- New endpoints POST /v1/tools/search and /v1/tools/fetch relay the
DeepSeek APIs like /v1/alpha/search. A call is charged only when
DeepSeek returns results or page content.
* fix(deepseek): price /v1/tools/search and /v1/tools/fetch by endpoint
A model named in the request body chose the billing model, so a body such
as {"query":"...","model":"deepseek-flash"} was priced as a chat model with
no tokens and the search or fetch cost nothing. Both endpoints now always
use their routing name, deepseek-search or deepseek-fetch. DeepSeek ignores
the body field, so the body is still forwarded unchanged.
* fix(deepseek): bill Claude-format page fetches at the DeepSeek Fetch price
usage.server_tool_use.web_fetch_requests always went to the web_fetch
tool price, which has no built-in value, while only searches used the
DeepSeek key. A channel can now name its own fetch key next to the search
key, and DeepSeek channels bill reported fetches under deepseek_fetch. A
channel retry clears it like the search key.
SMTP settings now only decide whether mail can be sent. Verification
codes, password reset, email binding and account security mails depend
on SMTP alone, and no notification switch affects them.
Admins get a site-wide low balance reminder switch, on by default, next
to the SMTP settings. The default threshold is entered as an amount.
Users choose which notifications they receive. Low balance reminders are
on for everyone by default. Admins can also turn channel notifications
and upstream model update notices on or off. The root user receives
channel notifications until it turns them off. Other admins receive them
after they turn them on.
A low balance reminder is sent once, when a charge takes the balance
below the threshold. A top-up above the threshold arms it again.
Channel notifications reach every admin who has them on, in that admin's
language. Email binding codes and account security mails follow the
account's language.
Saving notification settings no longer clears the language, the sidebar
layout or the billing preference.
Existing settings keep their behavior, and old clients that do not send
the new fields keep the stored switches. Webhook, Bark and Gotify
payloads are unchanged.
* fix(relay): mask only the upstream address in error messages
MaskSensitiveInfo masked every URL, domain and IPv4 in upstream error
messages, which garbled documentation links, email domains and dotted
identifiers that clients need to read.
It now takes the upstream addresses of the request: the channel base URL,
the origin the gateway actually dialed (recorded in doRequest, DoWssRequest,
the AWS SDK client and the Xunfei and Volcengine TTS dials, which covers
absolute Advanced Custom routes), the channel proxy and the environment
proxy. URLs and bare hostnames on those hosts and all IPv4 addresses are
masked; other URLs and domains stay readable. With no upstream address no
address is masked. Credentials are masked on every URL regardless of host:
api_key values, query values and userinfo.
The URL matcher also keeps quotes, colons and semicolons that follow a URL,
keeps parentheses and apostrophes inside it, masks an IPv4 URL host whole and
keeps a country-code TLD when the host has a port.
* fix(relay): keep preset official endpoints readable in masked errors
A channel on the base URL new-api presets for its type reaches a public
official endpoint, so its address is not masked. Only addresses the admin
configured are: a custom base URL, the origin of an Advanced Custom route,
and the channel and environment proxies. Preset hosts are compared with
their port so a configured localhost upstream stays apart from the Ollama
preset.
The origins recorded for the AWS SDK, Xunfei and Volcengine TTS dials and the
plugin allowedHosts masking are dropped: they are all official endpoints.
* fix(relay): mask IPv4 addresses and plugin allowed hosts in every error
IPv4 addresses in network errors include the gateway's own address, so they
are masked even when the channel uses a preset base URL. Task plugin build
failures mask the plugin's allowedHosts again: an installed plugin may point
them at private hosts.
* fix(relay): mask bracketed IPv6 addresses in errors
Go network errors write IPv6 addresses in brackets, as in
"dial tcp [2001:db8::1]:443". Like IPv4, they are the resolved upstream or
the gateway's own address, so they are masked on every error once the
bracketed content parses as an IP.
Add a per-token "Allow querying user quota" switch. When it is on,
GET /api/usage/token/ also returns user_quota with the account's
remaining and used quota in quota units. New tokens start with the
switch on: the console form, POST /api/token/ without the field, and the
default token created at registration. Tokens created before this change
stay off. PUT /api/token/ keeps the stored value when the field is
absent, and the token audit lists the field when it changes.
The key usage and key log queries now check the key's IP whitelist the
same way API requests do, and answer 403 from an address outside it.
Expired and exhausted keys can still use both queries.
New API channels now query their balance from the upstream's
/api/usage/token/ with the channel key. When the upstream also returns
the owner's quota, the smaller of the key's and the owner's remaining
quota is the balance, and an unlimited key uses the owner's quota alone.
Without it, a limited key reports its own remaining quota, and an
unlimited key asks for the switch to be turned on upstream. Quota
converts to USD with the upstream's quota_per_unit from /api/status.
Refs #3122
Refs #6828
A failed read consumes and closes the request body. Endpoints that read
it a second time then answered "http: invalid Read on closed Body"
instead of the cause: /v1/video/generations, and audio transcriptions
and translations. The first failure is now kept for the rest of the
request, so every later reader reports it.
A body over MAX_REQUEST_BODY_MB now gets 413 instead of 400 on the
endpoints that still answered 400: ordinary relay endpoints such as
/v1/chat/completions, Midjourney, task plugin routes and /v1/tasks.
Task plugin routes also write the cause to the log, and Midjourney
errors no longer repeat "Invalid request:".
utils.fetch is a synchronous, read-only GET or HEAD to the channel Base URL
host or the plugin's allowedHosts, through the channel proxy used for
polling. It is available in buildSubmitRequest, the query and batch build
and parse hooks, and buildContentRequest; every other hook throws. Each hook
call gets at most 4 fetches, 15 s per fetch, 30 s in total and 1 MiB per
body. Redirects are returned, not followed. responseType "bytes" returns an
ArrayBuffer for binary data such as a media file header.
While a fetch waits, the runtime gives up its execution slot and the
watchdog pauses. BuildContentRequest now receives the request context, and
the media proxy shares the plugin request header rules.
* feat(i18n): translate web console messages in the frontend and AI-client errors in the backend
Chinese text was hard-coded in API messages, stored log content, emails and
server logs, so it reached users of every interface language.
Text is now translated where its reader is:
- Web console APIs return the English source text as message_key with
message_params, and the web console translates it in all seven languages.
message keeps the English text for other API callers.
- Errors returned to AI clients, emails and notifications are translated by
the backend (en, zh-CN, zh-TW) in the language saved on the account, then
the language of the request's Accept-Language header.
- A reader who states no language still gets the original Chinese text of the
messages that were Chinese before, word for word. Gateways that use another
new-api as their upstream match that text, for example to disable a channel
whose account ran out of quota.
- Usage, top-up and system log content is stored as content_parts (key and
params) and rendered in the viewer's language. Task fail reasons and refund
reasons written by the gateway are English keys. Rows written earlier are
shown as before.
- Server logs are English.
Also:
- The saved interface language zhTW, and zh-HK, zh-MO and zh-Hant, now select
Traditional Chinese in the backend instead of Simplified Chinese.
- The built-in group descriptions are translated; descriptions an
administrator saved are shown as written.
- .agents/rules/i18n.md records the conventions. Tests in i18n/ fail on a
non-English string literal, on a web console message without translations
and on a locale key without a constant.
* feat(i18n): print server logs in Chinese when LOG_LANGUAGE asks for it
Server log lines are English by default. With LOG_LANGUAGE=zh-CN, the lines
that were Chinese before print their original Chinese text.
common.LogText formats a log line from its English format, which is also the
key of its Chinese text in common/log_text.zh-CN.json. A line without an entry
prints in English. go vet checks the arguments of every call, and
TestServerLogTexts checks that each entry still matches a call and has the
same verbs as its key.
* docs: list LOG_LANGUAGE in the docker compose example
* feat(i18n): translate every server log line and add DEFAULT_LANGUAGE
Every line the gateway prints goes through common.LogText. With
DEFAULT_LANGUAGE=zh-CN, the lines that have a Chinese text in
common/log_text.zh-CN.json print it and the others print English.
DEFAULT_LANGUAGE replaces LOG_LANGUAGE. It also sets the language of
backend messages, emails and notifications for a reader who states no
language: en, zh-CN or zh-TW, English when unset. The list of messages
that stayed Chinese for such readers is removed.
Review fixes:
- Channel test logs store the token name "Model test"; the console shows
it in the interface language.
- Task fail reasons written by the gateway are fixed sentences the console
translates. The HTTP status and the upstream detail go to the server log
when a polled task is failed.
- The auto-disable reason of a slow channel is written in DEFAULT_LANGUAGE.
Also:
- Drop the repository-scanning lint tests, the i18n rules file and its
read gate. AGENTS.md keeps the conventions in four lines.
- Payment error toasts keep their previous text and only add the
translated message key.
- Remove guards and helpers that no caller needs.
* fix(i18n): address review on language lookup, notices and error responses
- Relay requests no longer load the user a second time to find the
language: the settings TokenAuth put into the request are used as they
are, also when they hold no language.
- Upstream model update notices are rendered in each recipient's saved
language, DEFAULT_LANGUAGE when none.
- /api/usage/token and /api/log/token answer with backend-translated
messages again, with the keys and status codes used before this PR.
Their callers are scripts, not the web console.
- Webhook notifications fill {{value}} placeholders like the other
senders. Running the rendered text through Sprintf appended
%!(EXTRA ...) and broke %2F in links.
- RootUserLanguage reads the setting of the root user it already loaded.
- zh_TW and zh_HK are read as Traditional Chinese.
- Add common.ApiErrorStatus for error responses under another HTTP status
and use it where those responses were built by hand.
- Remove middleware.GetLanguage, which had no callers.
Token estimates badly undercounted agent requests. Claude tool
definitions, Responses function calls and their outputs, Chat assistant
tool calls, and Gemini system instructions, function calls, function
responses, and tools never reached the counter; Chat tool schemas were
counted as Go map syntax and Claude tool results as escaped JSON.
Measured against upstream usage on 54 real Claude Code and Codex
conversation windows, native prompt estimates were 31-93% low, and the
Claude and Gemini completion fallbacks about 90% low because streamed
tool calls were not counted.
All of these now reach the counter. The completion fallbacks count tool
names and arguments, plus 40 tokens per Claude tool_use block. The
estimator weights are refitted per tokenizer: Claude before Opus 4.7,
Claude 4.7 and later (about 30% more tokens for the same text), GPT, and
Gemini. Prompt estimates now land within 1% bias and 3-4% mean error for
each family.
Token counting moves into tokenkit, an independent module like
relaykit, so other gateways can use it. It also replaces the flat
520-token image estimate with each provider's sizing rules, checked
against upstream usage on sizes outside the test table: OpenAI patch and
tile rules per model and detail level (gpt-5.5 and later default to
original detail, and their high detail is billed as measured, not as
documented), Claude's two resolution tiers, and Gemini 3 media
resolution and Gemini 2.x tiles. Chat image_url parts without detail
no longer default to high in ParseContent, so they estimate as the
upstream default auto; no converter sent that value upstream.
tiktoken-go moves to v0.8.1, about twice as fast on ordinary text. It
requires Go 1.26, so the root module and tokenkit move to Go 1.26;
relaykit stays on 1.25.1. In every version BPE cost grows with the
square of a pre-tokenized piece, so 100,000 Chinese characters without
punctuation took 21.6 s to count during pre-consume. Count now cuts
single-class runs longer than 1024 bytes, which leaves ordinary text
unchanged and brings that input down to 106 ms. The encoder cache no
longer grows with client-supplied model names.
GLOBAL_WEB_RATE_LIMIT (120 requests per 180 s per IP) counted every file
served from the frontend build. Since provider icons became lazy chunks,
one cold session sends about 90 static requests, so a second user behind
the same IP gets 429 on route chunks and the page fails to load.
Files that exist in the build now count against a separate static
limiter (GLOBAL_STATIC_RATE_LIMIT_ENABLE, GLOBAL_STATIC_RATE_LIMIT,
GLOBAL_STATIC_RATE_LIMIT_DURATION), disabled by default. Pages and
unknown paths still count against the web limit.
The gzip middleware also recompressed every static response, about
120 ms of CPU per request for the 4.8 MB main bundle. Build files are
embedded and never change, so each one is now served from a gzip
encoding computed once per process. Pages and unknown API paths still
go through the gzip middleware.
Add the synchronous openai_image host protocol so a task plugin can claim
POST /v1/images/generations and /v1/images/edits; the host submits the
task, polls it inside the request and lets the plugin render the OpenAI
image response. Unclaimed models still fall back to the HTTP image relay.
- Migrate Ali (Bailian) image models to the alibaba plugin and remove the
ali channel image/wan adaptors and the qwen model setting
- Add Doubao Seedream image generation to the doubao plugin
- Support retainResult on routes and indexed multipart file references
- Settle image counts from upstream usage or image payloads; document the
rule in the billing conventions and plugin API docs
- Show billing source and task artifacts in usage logs; rename the
DoubaoVideo channel display name to Doubao and refresh the favicon
Track in-flight reservations in the memory limiter and count a request as
successful only when its response outcome completed, instead of relying
on the HTTP status code alone.
Uppercase or mixed-case extensions (e.g. .MP3, .WAV) fell through to the
default branch and returned "unsupported audio format", so audio token
usage could not be measured. Lowercase the extension before the switch.
Closes#7319
Unify image count validation, outbound parameters, reservation, retries, and settlement. Reject explicit zero Ali parameters.n and insufficient supplemental reservations. Centralize image model classification, extract deprecated DALL-E compatibility, and record image cache billing inputs through shared audit injection.
TaskPluginOverrideEnabled had no UI since the master switch landed, yet
when left off it marked every third-party plugin "disabled; platform
unavailable" regardless of its own toggle. Drop the option, env var,
registry flag, and the dead branch in ListTaskPlugins; the master switch
and per-plugin toggles are the only two levels now.
`common/json.go` and `relaykit/relayconvert/kitutil/json.go` were two
hard-wired copies of the same encoding/json wrapper, so swapping the JSON
engine required editing both modules.
- kitutil defines a `Codec` interface with a standard-library default and
a `SetCodec` hook, mirroring the existing SetLogging host hook; every
kitutil JSON helper and relaykit DTO (un)marshal method goes through it
- `common/json.go` forwards to kitutil and injects `hostJSONCodec` from
init() so tests run on the same engine as production; swapping the
engine now touches only this type in the root module
- route the remaining direct encoding/json calls inside relaykit
(dto/values.go, responses stream validation) through kitutil
- add a codec routing test and a host codec conformance test locking the
encoding semantics the DTOs depend on
Direct encoding/json call sites in the root module are left for a
separate cleanup.
Require scoped, single-use verification for account bindings and password
operations. Bind OAuth authorization and email confirmations to the initiating
session; preserve the last usable login method and audit operation outcomes.
Apply Unicode-aware password length limits, Argon2id writes with bcrypt
compatibility, and long-password encryption.
Return has_password with the existing profile SELECT without extra queries.
Reuse the existing security dialogs and add all seven locale translations.
Validation:
- Go: go test ./common ./model ./service ./middleware ./controller ./router -count=1
- DB: SQLite 3.50.4, MySQL 8.4.11, PostgreSQL 16.15; separate main/log databases
- MySQL/PostgreSQL: TEST_SECURITY_DIALECT=<dialect> with TEST_<DIALECT>_DSN,
go test ./controller -run '^(TestSecurityAccount|TestSecurityEnrollment|TestGenerateOAuthCode|TestOAuthBind|TestTelegramOAuth)' -count=1 -v
- Web: relevant Vitest suites, bun run typecheck, targeted oxlint/format,
bun run i18n:sync, and bun run build
Roll out dual-format readers to every instance with
ACCOUNT_PASSWORD_HASH_ALGORITHM=bcrypt before enabling Argon2id writes
and the new UI. Rollbacks must retain Argon2id and v2 envelope readers.
Relevant controls: ASVS 5.0.0 6.2.1-6.2.3, 6.2.5-6.2.9, 6.3.7, 7.4.3, 7.5.1;
this change does not assert application-wide ASVS certification.
Require single-use operation proofs for passkey enrollment, two-factor setup, and channel key access. Add password and OAuth verification flows, enforce session-bound enrollment, and redact OAuth callback secrets from logs.
Validation: affected Go packages pass; frontend typecheck, changed-file lint, and 111 tests pass. Security enrollment regressions pass on SQLite 3.50.4, MySQL 8.0.46, and PostgreSQL 16.15. Full frontend lint has pre-existing errors outside the changed files.
Plugin polling hooks previously ran against a hollow context: parseTaskResult
and parseBatchResult received {} / nil, buildQueryRequest received a
{task_id, action} map under the misleading name requestBody, and batch hooks
saw only bare task ids. The per-task poller also never looked at the upstream
HTTP status, and every built-in plugin papered over unrecognized bodies with
`|| "IN_PROGRESS"`, so a 404, a revoked key, or a shape the plugin did not
know would sit in IN_PROGRESS for the full 24h TASK_TIMEOUT_MINUTES while
holding the user's pre-charged quota.
Contract (docs/plugin-api v1.d.ts, v1.md, v1.schema.json):
- TaskQueryContext is declared separately from DriverContext and rebuilt from
the persisted Task row: taskId, publicTaskId, action, model, upstreamModel,
baseUrl, apiKey, authHeader, auth, data, state. Query-side requestBody is
removed; the original request is not persisted and hooks that need a
request-derived value must save it into state at submit time.
- parseTaskResult / parseBatchResult receive a third {status, headers}
argument. Batch hooks receive tasks[] with one TaskQueryContext per task.
- NormalizedTaskResult accepts status "UNKNOWN" meaning "I do not recognize
this body". Falling back to IN_PROGRESS for unknown shapes is forbidden;
`plugin lint` warns on the literal.
- parseSubmitResponse / parseTaskResult / parseBatchResult may return `state`.
Task.Data remains a per-round snapshot overwritten on every valid parse;
state is plugin-owned, persisted in TaskPrivateData.PluginState, preserved
when a hook omits it, byte-capped like taskData, and never exposed through
presenter views.
Host (service/task_polling.go, relay/channel/task/jsplugin/adaptor.go):
- TaskPollingAdaptor / BatchTaskPollingAdaptor take *model.Task and the
*http.Response so the adaptor can build the full context; jsplugin is the
only implementation.
- HTTP classification before the plugin sees the body: 2xx -> plugin;
404/410 -> FAILURE and refund; 401/403 -> poll failure plus a channel-scoped
warning, no auto-disable; 429/5xx/transport -> poll failure; other 4xx ->
plugin with the status visible, counted as unrecognized if the plugin still
reports a non-terminal state.
- TaskPrivateData.PollFailures counts consecutive poll failures (transient
HTTP, auth, transport, hook error, UNKNOWN). It is persisted through the
existing UpdateWithStatus CAS so a concurrent terminal transition on another
instance is never clobbered, and reset on any valid 2xx non-terminal parse.
Reaching TASK_POLL_MAX_FAILURES (default 20, <= 0 disables) fails the task
with the last classification and HTTP code in fail_reason and runs the
existing settle/refund chain exactly once. sweepTimedOutTasks and its
1440-minute default are unchanged as the outer backstop.
- Unrecognized bodies are logged at WARN with a bounded redacted copy since
Task.Data is intentionally not overwritten on that path.
Plugins (all ten bumped one patch version):
- jimeng persists the outbound req_key in state and reads it back in
buildQueryRequest, replacing dead reads of ctx.data / ctx.requestBody that
never resolved.
- sunoapi batch hooks read tasks[] instead of the removed requestBody.
- hailuo treats base_resp.status_code != 0 as FAILURE before the status table.
- kling, vidu, sora, alibaba, doubao, hailuo, jimeng return UNKNOWN with the
raw upstream status in reason on table miss.
- google and vertex-ai treat a missing `done` as in-progress: Google
long-running operations omit proto3 default fields, so a running Veo
operation has no `done` key at all. Only a body without an operation name is
UNKNOWN. plugins/veo_poll_test.go locks this so the poll-failure cutoff can
never fail a rendering Veo task.
Tests cover the classification table end to end against a real DB (404
immediate refund, 429xN refund, 401 increments without status change, 2xx
reset, UNKNOWN increments, state preserved vs replaced, PollFailures survives
the CAS write), the query-context shape, UNKNOWN on unrecognized bodies, and
the absence of PluginState/PollFailures from TaskView. Controller tests derive
the kling factory version from the embedded manifest instead of hardcoding it.
* fix(relay): bound the wait for upstream response headers (fixes unbounded heap growth)
The relay transport sets a dial timeout, a TLS handshake timeout and an expect-continue
timeout, but nothing bounds how long it waits for the upstream *response headers* after
the request has been written. An upstream that accepts the connection and then never
answers -- without sending FIN/RST, which is what happens when a NAT/firewall silently
drops the flow or the provider hangs -- parks the goroutine in
net/http.(*persistConn).roundTrip forever.
That goroutine keeps the whole request alive, which in practice means three copies of the
request body stay reachable for the lifetime of the process: the raw bytes from
io.ReadAll in CreateBodyStorageFromReader, the decoded messages held as json.RawMessage,
and the re-marshalled upstream body from common.Marshal. BodyStorageCleanup cannot help
here: it runs after c.Next() returns, and for these requests c.Next() never returns.
Measured on v1.0.0-rc.23 in production (see #6947 for the full evidence):
- 23 goroutines stuck in persistConn.roundTrip on a single 40h-old instance,
blocked between 353 and 1894 minutes (5.9h to 31.5h)
- 96.9% of the live heap, sampled after a forced GC, attributable to those three
body copies (HeapAlloc 892 MiB surviving three GC cycles; HeapObjects dropping
30x while bytes dropped only 25%)
- the live floor grows with uptime: 33.7 MiB at 0.1h, 89.2 at 13.8h, 510.0 at 40.1h,
955.2 at 146.8h, OOMKilled at 172.9h -- same image, same config, same load
Doubling the memory limit and adding GOMEMLIMIT only moved the OOM from 132h to 172.9h.
RELAY_TIMEOUT (http.Client.Timeout) cannot be used for this: it covers the whole response
read and would cut legitimate long streaming calls, which is why it defaults to 0.
ResponseHeaderTimeout only bounds the wait for the headers; streaming after they arrive is
unaffected.
The default is deliberately generous. Non-streaming upstreams usually send the response
headers only once generation has finished, so the value has to leave room for a long
completion. 1800s is 12x shorter than the shortest hang observed here while leaving
several times the headroom a normal non-streaming request needs; 0 restores the previous
unbounded behaviour.
The assignment goes next to the other transport.* lines rather than inside the else
branch: newRelayHTTPTransport() normally takes the http.DefaultTransport.Clone() path,
and DefaultTransport does not set ResponseHeaderTimeout either.
This repo already sets ResponseHeaderTimeout on its other outbound transports
(controller/model_sync.go, controller/ratio_sync.go); the relay path appears to have
been missed.
Refs #6947. Likely also the root cause of #6731, which reported the same symptom
(production OOM on /v1/responses after ~64h) but was closed for template reasons.
* review: clamp overflowing timeout values and switch the test to testify
Addresses the two CodeRabbit findings on this PR.
Overflow (common/init.go:113): a RELAY_RESPONSE_HEADER_TIMEOUT beyond ~9.2e9 seconds
overflows time.Duration and can wrap into a *tiny positive* timeout, which would cut
every relay request instead of only the stuck ones. The value is now clamped before the
conversion, with regression tests for both the negative and the overflowing input.
I did not add fail-on-startup validation for negative values, for two reasons: the
existing `if seconds > 0` guard already treats them as "disabled", and the neighbouring
env-driven timeouts in this file are less strict still -- RelayIdleConnTimeout is
converted with no guard at all. Failing startup on a bad value would be a behaviour
change out of step with the rest of the file; happy to add it if you'd prefer that
direction repo-wide.
Test style: switched to testify (require.Equal / require.Zero / require.Positive), which
is what every other test under service/ uses.
go build, go vet and go test ./common/... ./service/... pass.
(`go build ./...` fails on the `web/dist` embed both with and without this change -- the
frontend bundle is not checked in.)
* fix(relay): set Request.GetBody so the HTTP/2 transport can transparently retry after an upstream stream reset
The outbound request body is a type-erased io.Reader over BodyStorage, so
net/http cannot derive Request.GetBody (it only does so for *bytes.Reader,
*bytes.Buffer and *strings.Reader). With GetBody nil, the HTTP/2 transport
cannot transparently retry a request once the body has been written and the
upstream resets the stream with a retryable error (REFUSED_STREAM, or a
connection-level GOAWAY); the relay request then fails with:
http2: Transport: cannot retry err [...] after Request.Body was written;
define Request.GetBody to avoid this error
This affects every relay path that goes through DoApiRequest (chat, claude,
gemini, responses, embedding, image, rerank).
BodyStorage (memory and disk) already implements io.Seeker, so replay support
only needed wiring:
- NewOutboundJSONBody additionally returns a getBody that rewinds the storage
and hands out a fresh non-closing reader. The transport only calls GetBody
after the previous attempt's body has been abandoned, so the rewind cannot
race an in-flight read.
- RelayInfo carries it in the new UpstreamRequestGetBody field, set alongside
UpstreamRequestBodySize by the handlers that build storage-backed bodies.
- applyUpstreamGetBody (symmetric with applyUpstreamContentLength) wires it
into DoApiRequest/DoFormRequest/DoTaskApiRequest, only when req.GetBody is
still nil.
Also remove the hand-rolled GetBody override in DoTaskApiRequest: it returned
the same already-consumed reader, so any transport-level replay would have
silently sent an empty body, and it clobbered the correct snapshot-based
GetBody that net/http derives from the *bytes.Reader bodies the task adaptors
pass in. For non-replayable bodies GetBody now stays nil, so a retry fails
loudly instead of corrupting the request.
Covered by unit tests plus an end-to-end raw-frame HTTP/2 test that resets
the first stream with REFUSED_STREAM after the body is written and asserts
the transport transparently retries with the complete body.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(relay): hand out independent readers from GetBody (address review)
Per the http.Request.GetBody contract ("returns a new copy of Body"),
each call must yield a reader with its own cursor. The previous
implementation rewound and reused the shared BodyStorage, so two
consecutive GetBody readers would interfere with each other, and a
replay could disturb the primary body's offset under extreme transport
timing (e.g. attempt N's body write not yet fully abandoned when the
transport builds attempt N+1).
Instead of snapshotting the payload (an extra copy), add
BodyStorage.NewReader, which returns an independent zero-copy reader:
- memory mode: a fresh bytes.Reader over the same immutable backing
array;
- disk mode: a separate file descriptor over the cache file, so the
transport closing a replayed body only closes that descriptor.
NewOutboundJSONBody's getBody now simply hands out storage.NewReader,
and once the handler releases the storage, GetBody fails with
ErrStorageClosed instead of replaying stale data.
Tests: interleaved reads across two replay readers and the primary
body each observe exactly their own byte stream, for both the memory
and the disk-backed storage; the existing GetBody and HTTP/2 retry
suites still pass (h2 e2e tests flake-free with -count=20).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(relay): bind replayable metadata on pass-through requests
* fix(relay): reset upstream body metadata between channels
* test(relay): cover replay across retries and channel attempts
* fix(relay): stop following upstream redirects
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* test(relayconvert): add golden snapshot matrix and relaykit boundary guard
Phase 0 of the relaykit extraction plan: pin byte-level output of every
registered (from,to) request/response/stream conversion route, and
forbid kit-bound packages from growing host-only imports.
* wip(relayconvert): drop gin.Context from converter signatures; add convmeta draft
Phase 1 in progress: relayconvert now takes context.Context; host media
resolver adapts gin.Context back at the service boundary.
* refactor(relayconvert): decouple converters from RelayInfo, gin, and settings
Phase 1 of the relaykit extraction plan:
- converters now depend on convmeta.Meta (implemented by RelayInfo) instead
of *relaycommon.RelayInfo; ClaudeConvertInfo and the format guesser move
to convmeta with aliases left behind
- host settings reach converters via a convmeta.Options snapshot built in
RelayInfo.ConvOptions; no more model_setting/reasoning global reads inside
the conversion layer
- effort-suffix helpers move to service/relayconvert/reasoning (old package
forwards); chat-to-responses upgrade policy moves to service (host routing
logic, not conversion)
- golden conversion matrix unchanged
* test(relayconvert): tighten boundary — kit packages now free of gin/setting imports
* refactor(dto): drop gin and logger dependencies
Phase 2 (part 1): dto.Request.IsStream now takes *http.Request instead of
*gin.Context (Gemini's impl reads query/path off the std request); dto's
three logger calls become common.SysError. Boundary test allowlist is now
empty — kit-bound packages import no gin/setting/logger/model.
* refactor(kit): extract dependency-free kitutil; dto/types/relayconvert stop importing common
Phase 2 of the relaykit extraction plan:
- new service/relayconvert/kitutil holds the pure helpers the kit needs
(JSON wrappers, pointer/string/uuid/timestamp utils, MaskSensitiveInfo,
pluggable LogInfo/LogError hooks, Debug flag)
- dto, types, and all relayconvert packages now use kitutil; their only
remaining internal deps are dto/types/constant
- common keeps every original symbol (MaskSensitiveInfo delegates to
kitutil) so host code is untouched; main.go routes kit logging into
common.SysLog/SysError and mirrors DebugEnabled
- golden conversion matrix unchanged
* refactor(kit): move EndpointType/FinishReason to types; OpenRouter dialect via Options
Kit packages (dto/types/relayconvert/reasonmap) no longer import constant:
- EndpointType and finish-reason values live in types; constant re-exports
- the OpenRouter special-case in claude->openai request conversion reads
Options.OpenRouterDialect, set by the host from the channel type;
InitChannelMeta invalidates the cached snapshot on channel switch
* refactor: extract relaykit submodule (dto/types/relayconvert/reasonmap)
Phase 3 of the relaykit extraction plan:
- new go module github.com/QuantumNous/new-api/relaykit containing dto
(minus task family), types, relayconvert (with convmeta/kitutil/reasoning),
and reasonmap; host consumes it via require + replace, go.work for dev
- task-family dto (task/suno/midjourney/video) stays in the host dto
package; dual-consumer host files alias it as taskdto
- relaykit builds and tests standalone (GOWORK=off): no host imports,
no gin, no DB, no settings
- golden conversion matrix unchanged
* build(docker): copy relaykit/go.mod before go mod download
The local-replace submodule's go.mod must exist inside the build context
for the main module graph to resolve.
* fix: address relaykit extraction regressions
* fix: address relaykit review regressions
* docs: document Meta nil receiver contract
* fix(relaykit): fail OpenAI→Claude conversion without max_tokens; reject negative default_max_tokens
The Claude Messages API requires max_tokens (omitting it is a 400
"Field required"), but with a nil Options.Claude.DefaultMaxTokens hook
the converters silently emitted a request the upstream is guaranteed to
reject. Both OpenAI Chat and Responses → Claude conversions now return
sharedclaude.ErrMissingMaxTokens when no path (client value, default
hook, thinking-adapter floor) supplied one. Unreachable in the host,
which always configures the hook.
Host side, claude.default_max_tokens now rejects negative values at the
option API before persisting — they would wrap into huge unsigned values
during conversion. Zero stays allowed: the current API treats
max_tokens: 0 as cache pre-warming.
* fix: make Gemini safety settings read path race-free
CaIon