Replace wall-clock identifiers with an atomic sequence and test names so repeated Windows runs cannot share usage-cache entries.
Verification: latest main baseline failed with cumulative counts; patched affinity tests passed -count=20; full service suite and go vet passed.
Signed-off-by: sstar16 <3221386464@qq.com>
OpenAI-compatible upstreams with explicit context cache (e.g. Alibaba Bailian)
report cache-write tokens as prompt_tokens_details.cache_creation_input_tokens.
The field was ignored, so cache-write tokens were billed at the plain input
price instead of the cache-creation price.
Parse it into InputTokenDetails.CacheCreationInputTokens and include it in
CacheCreationTokensTotal using max() with the existing cache-write fields, so
the same tokens are never counted twice.
Some vendors keep a task running for more than a day: Volcengine Ark waits
48 hours by default and up to 72 when configured, and Vidu off-peak jobs
can take 48 hours. With the 24-hour default the gateway failed and refunded
those tasks while the vendor was still producing, and charging for, the
result. TASK_TIMEOUT_MINUTES now defaults to 4320. A task that is truly
stuck is refunded after 72 hours instead of 24.
OpenAI added a 12-hour prompt cache to the Responses API on 2026-10-20
(prompt_cache_options.ttl "12h", GPT-5.6 and later). A 12-hour write
costs 2.5x the input price, twice the 30-minute write. We already
forwarded the option but billed every cache write at the 30-minute price.
- Read cache_write_tokens_details from Responses usage and keep the split
through streaming, compaction and format conversion.
- Ratio pricing bills 12-hour writes at twice the cache write ratio, the
same way Claude 1-hour writes use 6/3.75.
- Expressions get a cc12h price. When an expression leaves the long-lived
price out, 12-hour writes cost twice the cc price and Claude 1-hour
writes 6/3.75 times it; without a cc price they bill as input. Claude
cache writes are no longer free when an expression omits cc or cc1h.
- gpt-6-astra prices 12-hour writes at $25 ($50 above 272K tokens).
- GPT models with a cache write ratio convert to expressions with cc12h.
- Chat Completions forwards prompt_cache_options.
- The pricing editor, cost estimator, pricing pages and usage log details
show the 12-hour price and tokens.
Structured outputs fill a schema's properties in the order the schema
lists them. The gateway decoded schemas into maps and encoded them
again, which sorts the keys: a schema that lists reasoning before the
answer reached the upstream with the answer first. Chat Completions
sent through the Responses API lost the order of response_format, and
tool parameters lost it on most paths.
Schemas now reach the upstream as the client wrote them: tool
parameters and Claude input_schema, Chat and Responses format
conversion, the disabled-field filter, Bedrock pass-through, param
override move, copy and append, the output_config edits for DeepSeek
and model modifiers, and the Ollama format.
relaykit: dto.FunctionRequest.Parameters and dto.Tool.InputSchema are
now json.RawMessage.
When a task plugin claimed a model on a shared host endpoint
(/v1/images/generations, /v1/images/edits, /v1/responses, /v1/videos),
every request for that model went to the plugin, even when an ordinary
OpenAI-compatible channel served the model too. Ordinary channels and
the matching plugin channels now compete for the request under the usual
group, priority, weight, affinity and channel-pin rules. When an
ordinary channel wins, the original request reaches the ordinary relay
unchanged: no plugin decoder, mode check or model rewrite runs, and
retries stay on ordinary channels. When a plugin channel wins, the
plugin handles the request as before and retries stay on its accepted
plugin candidates. Native plugin routes still belong to their plugins.
Ordinary and plugin prices are kept apart. Task expressions return USD,
while ordinary expressions keep their USD-per-million result, so an
ordinary request now fails with 400 model_price_error before any quota
is reserved when the model's expression reads task usage (u()) or is
still inherited by a task plugin on that endpoint. The model pricing
editor can set ordinary OpenAI prices, per returned image with
fixed(...) * image_count, while each plugin keeps its price in its own
override. A model served by a single plugin now shows that plugin's
pricing editor as well.
Upstream price sync saved an upstream's price as it was, so a price
quoted in another currency had to be corrected by hand afterwards. Each
selected model now has a price adjustment: keep the source price, or
divide or multiply it by the site exchange rate. "Set selected prices"
applies one choice to every selected model. Fixed prices, ratios and the
prices inside billing expressions are adjusted; an expression that
cannot be adjusted automatically has to keep its source price or be
edited by hand. The confirmation dialog shows the adjustment, the final
saved price and the price in the site currency, with the exchange rate
in use.
"Show models with matching prices" also lists upstream prices that
already match the local ones, so they can be selected and adjusted too.
Order the keys of every web locale file the way localeCompare sorts them,
with numeric keys first as JSON objects keep them. No key or translation
is added, removed or changed.
Before billing, the gateway walked the whole decoded request of a task
plugin and rejected duration, seconds and count members that were out
of range at any depth, including vendor parameters passed through
metadata. A plugin had to declare duration-auto@1 for the gateway to let
a duration of -1 through.
The gateway no longer reads billing quantities from the request. It
still bounds every usage fact a plugin returns before quota is
calculated: second facts at most 3600, count facts at most 128, token and
credit facts saturated at the quota limit. Each plugin validates its own
request and derives its facts from the request it sends upstream.
duration-auto@1 is still accepted and has no effect.
Kling now bills the duration it sends. metadata.duration, which is the
whole body on native routes, overrides the top-level duration and must be
a whole number of seconds from 1 to 3600. Before, a metadata duration of
10 was sent upstream while 5 seconds were billed.
Driver hooks saw only the Content-Type and Accept headers of the client
request. ctx.requestHeaders now carries every client header, with the
first value under its canonical name, except the headers that carry
gateway credentials: Authorization, Proxy-Authorization, Cookie,
X-Api-Key, X-Goog-Api-Key and Mj-Api-Secret. The gateway still forwards
none of them upstream; a driver copies only the headers its vendor
defines into its request.
Native submit and dynamic routes of task plugins accepted only JSON and
answered 415 to a multipart upload before the plugin saw it. They now
pass multipart bodies to the plugin's decoder, which rejects the body
kinds its vendor format does not use; the built-in plugins keep requiring
JSON. A route that declares models still takes JSON only, because the
gateway reads the model from the JSON body. URL-encoded forms are still
refused.
Multipart text fields no longer have their own 1 MiB limit. Like a JSON
body, they are bounded by the request body limit, so a JSON request sent
as a form field can be as large as the same request sent as the body.
Expression billing priced Claude cache writes only from the 5-minute and
1-hour split. When an upstream reported the cache-write total without a
full split, the unlabelled tokens were billed as nothing. The remainder of
the total is now billed as 5-minute cache writes.
Every write to a streaming client had its own fixed 30-second deadline.
A client that downloads a multi-megabyte event, such as a generated
image, at a modest rate could not drain it in time. The write failed and
the request was logged as a client disconnect.
Stream writes now share one budget per stream: the deadline is 30 seconds
past the time a client reading at 8 KiB/s needs for everything written so
far. A client that stops reading is still cut off once the budget runs
out. The budget covers SSE responses, including keepalive pings, error
events and task plugin protocol streams, and the client side of the
Responses WebSocket.
Stop Responses SSE after its completed event has been written, preserving
upstream token and tool usage. Reconcile a concurrent client cancellation
only after all scanner workers have joined and a completed response was
successfully delivered. Bare EOF, failed writes and protocol errors do not
qualify. Propagate SSE write/flush errors and suppress cleanup read errors.
Integrates the completion handling from #6904 and #6951 and narrows the
cancellation reconciliation proposed in #6808. Refs #6649.
Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com>
Co-authored-by: zkasuran <zkasuran@gmail.com>
Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com>
Reuse the channel base host and plugin allowedHosts for redirected query
credentials. Keep anonymous redirects working, prevent credentials from
returning after an untrusted hop, and restrict body replay to trusted hosts.
Preserve SSRF and redirect limits, reject HTTPS downgrades, and redact signed
URLs from redirect-policy errors. Document the existing allowedHosts contract.
Validation: go test ./relay/channel/task/jsplugin -count=1
Passed against clean 6347fb0aa plus this patch, without an overlay.
Covers single/batch polling, 301/302/303/307/308, host and port boundaries,
multi-hop credentials, SSRF, downgrade rejection, and real HTTP receivers.
Related OWASP ASVS v5.0.0 controls: 14.2.3, 1.3.6, 12.3.1, 16.2.5.
https://github.com/OWASP/ASVS/tree/v5.0.0/5.0/en
The email binding audit test failed now and then because it searched the
whole audit JSON for the six-digit code. Audit event and request IDs start
with a long run of time digits, so a random code sometimes appeared inside
one of them. Treat only a code that no other digit touches as a leak.
A task retry that moved to another plugin or channel kept the previous
attempt's usage expression. A retry from an expression-priced plugin to
one priced per request charged, and later settled, at the first plugin's
price. Each attempt now prices itself from scratch.
A retry to a more expensive channel was topped up only after the
upstream had accepted the task. A user who could not cover the
difference went into arrears, or the task was dropped while the upstream
was already running it. The retry now raises the reservation before the
request is sent, with the same balance check as the first pre-consume,
and a channel the balance cannot cover is never called.
A wallet top-up taken before submission replaced the amount held by the
first pre-consume instead of adding to it, so a request that then failed
refunded only the top-up. Image requests whose quantity was raised
before submission were affected too. The full amount is now refunded.
DeepSeek /user/balance returns both USD and CNY wallets. An account
funded in CNY reports USD 0.00, and getDeepSeekBalanceUSD returned that
zero before ever reading the CNY entry.
Only short-circuit on a positive USD balance. When the USD wallet is 0
and CNY holds funds, convert CNY with USDExchangeRate; a lone zero USD
wallet still returns 0 instead of an error.
* fix(relaykit): keep tool calls when merging adjacent assistant messages
Adjacent assistant messages with string content were merged keeping only
the second message's tool calls, so tool results for the first message's
calls had no matching tool_use and Claude rejected the request.
- concatenate both messages' tool calls in input order
- skip the "..." placeholder of a tool-call message in the merged text
* fix(relaykit): preserve literal ellipses in assistant message merges
---------
Co-authored-by: seefs001 <i@seefs.me>
* feat: add ali MultimodalEmbedding qwen3-vl-embedding
* fix: report image/video tokens in qwen3-vl-embedding usage for billing
* fix(ali): serve every Bailian multimodal embedding model natively
Route qwen2.5-vl-embedding, the tongyi-embedding-vision series and
multimodal-embedding-v1, dated snapshots included, to the native
multimodal-embedding endpoint like qwen3-vl-embedding. Text embedding
models stay in compatible mode.
Take tongyi-embedding-vision's text and image tokens from
input_tokens_details so the image ratio applies to its image tokens,
and set total_tokens to the billed input tokens: tongyi counts its
unpriced output_tokens in total_tokens. A response without usage bills
the request estimate, as compatible mode does. Return an error code
carried in a 200 body as a 500, close the response body when reading
it fails, and fold the input conversion into one function.
Count the text, image and video parts of embedding input in the request
estimate the way chat content is counted, so image calls are pre-charged
and a response without usage is not free. With the default settings no
image is downloaded for the estimate.
* fix(gemini): reject embedding input that is not plain text
Gemini embeds text only, so the converter kept the strings and dropped
image, video and text parts. The embedding request estimate now counts
those parts, and Gemini embeddings bill that estimate, so a dropped part
would be charged without ever reaching Gemini. A part now fails the
request with a 400 that skips retries, before anything is sent or charged.
---------
Co-authored-by: CaIon <i@caion.me>
* fix(task): honor channel header overrides for submission and polling
Apply the existing Request Header Override configuration after task plugin
headers, including per-task, batch and realtime queries. Resolve api_key
from the credential used for that request and skip client-only placeholders
when polling has no original client request. Reuse the HTTP header resolver
without changing plugin contracts or adding provider-specific settings.
Keep polling redirects visible, matching submission and utils.fetch, so a
custom credential header cannot be forwarded to a redirect destination.
Document configuration and cover precedence, saved keys, absent client
context, invalid values, redirects and all polling initialization paths.
Validation with Go 1.26.4 and GOWORK=off:
- make test: root, relaykit and tokenkit passed before the realtime follow-up
- go test ./relay ./relay/channel ./relay/channel/task/jsplugin ./service -count=1: passed after the follow-up
- go vet ./... and go build ./...: passed after the follow-up
- Independent read-only review: passed
Scoped security review used OWASP ASVS 5.0.0 and the Authentication,
Session Management and REST Security cheat sheets. Credentials remain in
headers, client headers are not persisted, and redirects are not followed.
No live provider calls or production deployment were performed.
https://owasp.org/www-project-application-security-verification-standard/https://cheatsheetseries.owasp.org/cheatsheets/Authentication_Cheat_Sheet.htmlhttps://cheatsheetseries.owasp.org/cheatsheets/Session_Management_Cheat_Sheet.htmlhttps://cheatsheetseries.owasp.org/cheatsheets/REST_Security_Cheat_Sheet.html
* fix(task): follow poll redirects and override content request headers
Polling follows redirects again through the shared HTTP client, which
checks every hop against the SSRF protection settings. Refusing them
turned each poll of a redirecting upstream into a failure, and after
enough failures the task was failed and refunded.
Credentialed content requests now carry the channel header overrides,
as submission and polling do. Credentialless links, utils.fetch and the
Host header stay as the plugin builds them.
* fix(task): settle tasks that a realtime query sees finish
Fetching a Gemini or Vertex task queries the upstream directly. When
that query saw the task finish, it wrote the final status itself, so
the poller skipped the task and never settled or refunded it: a failed
task kept its charge. Hand that transition to the poller's single-task
update, which records the status and settles exactly once.
---------
Co-authored-by: CaIon <i@caion.me>
* feat(email): add Resend delivery provider
Add Resend as an alternative to SMTP with separately editable sender address
and name, credential-safe configuration status, and validation before activation.
Preserve the target repository's email notification and balance reminder settings.
Only Resend changes are transplanted from hicoderai/new-api; no landing,
legal-document, model-group, pricing, branding, or dependency changes are included.
Validation: Go common/controller email regression tests; 19 frontend email and
notification tests; frontend production build; tsgo typecheck; touched-file lint.
Frontend validation uses the target lockfile's Axios 1.18.1 with local cached tools.
Security references: OWASP Authentication, Session Management, and Forgot Password
Cheat Sheets. Tests cover credential non-disclosure, invalid activation, sanitized
transport errors, and no automatic retry or fallback; token and session flows
are unchanged.
https://cheatsheetseries.owasp.org/cheatsheets/Authentication_Cheat_Sheet.htmlhttps://cheatsheetseries.owasp.org/cheatsheets/Session_Management_Cheat_Sheet.htmlhttps://cheatsheetseries.owasp.org/cheatsheets/Forgot_Password_Cheat_Sheet.html
* fix(email): log Resend rejections and translate Resend errors
Resend's reason for a rejected or failed send now goes to the server log.
People who asked for the email still see the short error.
Resend and email method errors are translatable messages in all seven
languages.
The email settings hints are shorter and reuse the existing key hint.
The new frontend files carry the license header, the sender name check
passes lint, and test fixtures use neutral names.
* fix(email): check Resend settings while Resend is active
Saving the sender address or API key now checks the whole Resend setup
while Resend sends the mail. A typo the form accepts, such as
notify..team@example.com, is rejected instead of breaking every send.
Resend's error text and network errors are quoted in the server log.
---------
Co-authored-by: CaIon <i@caion.me>
* fix(relay): keep Claude streams alive during preparation and retries
* fix(relay): retain failed outcomes for keepalive stream errors
* fix(relay): wait one interval before sending keepalive
* fix(relay): synchronize request keepalive handoff before headers
Stop and join the request keepalive before the stream scanner copies upstream
headers. The scanner then owns subsequent ping and data writes through its
shared write mutex.
Add regression coverage for an immediate stream and for the first keepalive
committing response headers. Before the fix, the latter case reports a data
race between Header.Clone and copyCodexSSEHeaders.
Validation: GOWORK=off GOCACHE=/private/tmp/new-api-pr7857-review-go-cache go test -race -p 2 -parallel 1 ./controller ./relay/channel ./relay/helper -run 'TestRelayKeepAlive|TestRelayStreamErrorAfterKeepAlive|TestStreamKeepAlive|TestDoRequest|TestNewTaskAPIRequest|TestProcessHeaderOverride|TestStreamScannerHandler_' -count=1
Result: all three packages passed on Go 1.26.6, darwin/arm64.
git diff --check passed.
---------
Co-authored-by: Calcium-Ion <i@caion.me>
* fix(notify): keep notification limit window from resetting every hour
The rate-limit key included the current clock hour, so the count reset on
every hour boundary and NOTIFICATION_LIMIT_DURATION_MINUTE above 60 had no
effect. Drop the hour suffix so the window runs for the configured duration
from the first notification, and make the Redis counter atomic with
SETNX+INCR in one transaction.
* fix(notify): fall back to default for nonpositive notification window
A zero or negative NOTIFICATION_LIMIT_DURATION_MINUTE left the Redis
counter without an expiry (or failed the SET), which blocked quota
notifications for good once the counter filled.
* test(notify): cover the Redis notification limit window
* fix(model): find channels for @-modified model names without the memory cache
With the channel memory cache off (no Redis and MEMORY_CACHE_ENABLED
unset), channel selection matched abilities by the exact requested name,
so a name such as deepseek-flash@effort:max found no channel. Retry with
the routing name when the exact name has no channel, as the cached path
already does.
* feat(deepseek): support V4.1 Flash, the Search and Fetch APIs, and @ reasoning modifiers
Follows the DeepSeek API update of 2026-10-17.
- Default models: deepseek-flash and deepseek-v4-pro, each with -none,
-low and -max, plus deepseek-search and deepseek-fetch. The retired
deepseek-chat, deepseek-reasoner and deepseek-v4-flash names are no
longer listed.
- Thinking: the -none/-low/-max tails also work on deepseek-flash.
@thinking and @effort now reach DeepSeek's own fields on Chat
(thinking, reasoning_effort), Anthropic (thinking, output_config.effort)
and Responses (reasoning.effort), and take precedence over the tails.
- Built-in prices: peak/off-peak expressions for deepseek-flash (also the
deepseek-v4-flash aliases) and deepseek-v4-pro, and $0.75 per 1,000
requests for deepseek-search and deepseek-fetch.
- Server-side search: searches and page fetches DeepSeek runs for Claude
Code and Codex bill at $0.75 per 1,000 under deepseek_search and
deepseek_fetch, whatever model name the client sent. Searches whose
results already arrived stay billed when a stream ends before its final
usage; this also applies to Alibaba's Anthropic endpoint.
- New endpoints POST /v1/tools/search and /v1/tools/fetch relay the
DeepSeek APIs like /v1/alpha/search. A call is charged only when
DeepSeek returns results or page content.
* fix(deepseek): price /v1/tools/search and /v1/tools/fetch by endpoint
A model named in the request body chose the billing model, so a body such
as {"query":"...","model":"deepseek-flash"} was priced as a chat model with
no tokens and the search or fetch cost nothing. Both endpoints now always
use their routing name, deepseek-search or deepseek-fetch. DeepSeek ignores
the body field, so the body is still forwarded unchanged.
* fix(deepseek): bill Claude-format page fetches at the DeepSeek Fetch price
usage.server_tool_use.web_fetch_requests always went to the web_fetch
tool price, which has no built-in value, while only searches used the
DeepSeek key. A channel can now name its own fetch key next to the search
key, and DeepSeek channels bill reported fetches under deepseek_fetch. A
channel retry clears it like the search key.
Seefs