mirror of
https://github.com/QuantumNous/new-api.git
synced 2026-10-28 16:46:57 +08:00
CaIon
bb28f2ccad
Token estimates badly undercounted agent requests. Claude tool definitions, Responses function calls and their outputs, Chat assistant tool calls, and Gemini system instructions, function calls, function responses, and tools never reached the counter; Chat tool schemas were counted as Go map syntax and Claude tool results as escaped JSON. Measured against upstream usage on 54 real Claude Code and Codex conversation windows, native prompt estimates were 31-93% low, and the Claude and Gemini completion fallbacks about 90% low because streamed tool calls were not counted. All of these now reach the counter. The completion fallbacks count tool names and arguments, plus 40 tokens per Claude tool_use block. The estimator weights are refitted per tokenizer: Claude before Opus 4.7, Claude 4.7 and later (about 30% more tokens for the same text), GPT, and Gemini. Prompt estimates now land within 1% bias and 3-4% mean error for each family. Token counting moves into tokenkit, an independent module like relaykit, so other gateways can use it. It also replaces the flat 520-token image estimate with each provider's sizing rules, checked against upstream usage on sizes outside the test table: OpenAI patch and tile rules per model and detail level (gpt-5.5 and later default to original detail, and their high detail is billed as measured, not as documented), Claude's two resolution tiers, and Gemini 3 media resolution and Gemini 2.x tiles. Chat image_url parts without detail no longer default to high in ParseContent, so they estimate as the upstream default auto; no converter sent that value upstream. tiktoken-go moves to v0.8.1, about twice as fast on ordinary text. It requires Go 1.26, so the root module and tokenkit move to Go 1.26; relaykit stays on 1.25.1. In every version BPE cost grows with the square of a pre-tokenized piece, so 100,000 Chinese characters without punctuation took 21.6 s to count during pre-consume. Count now cuts single-class runs longer than 1024 bytes, which leaves ordinary text unchanged and brings that input down to 106 ms. The encoder cache no longer grows with client-supplied model names.
40 lines
1.2 KiB
Docker
40 lines
1.2 KiB
Docker
# Backend-only build for frontend development
|
|
# Skips frontend build and uses a placeholder for //go:embed web/dist
|
|
|
|
FROM golang:1.26.1-alpine AS builder
|
|
|
|
ENV GO111MODULE=on CGO_ENABLED=0
|
|
ARG TARGETOS
|
|
ARG TARGETARCH
|
|
ENV GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH:-amd64}
|
|
ENV GOEXPERIMENT=greenteagc
|
|
|
|
WORKDIR /build
|
|
|
|
ADD go.mod go.sum ./
|
|
# relaykit and tokenkit are local submodules referenced via replace; their
|
|
# go.mod files must be present for go mod download to resolve the main module graph.
|
|
ADD relaykit/go.mod ./relaykit/go.mod
|
|
ADD tokenkit/go.mod ./tokenkit/go.mod
|
|
RUN go mod download
|
|
|
|
COPY . .
|
|
|
|
RUN mkdir -p web/dist && \
|
|
echo '<!doctype html><html><head><title>dev</title></head><body>use frontend dev server</body></html>' > web/dist/index.html
|
|
|
|
RUN go build -ldflags "-s -w -X 'github.com/QuantumNous/new-api/common.Version=$(cat VERSION)'" -o new-api
|
|
|
|
FROM debian:bookworm-slim
|
|
|
|
RUN apt-get update \
|
|
&& apt-get install -y --no-install-recommends ca-certificates tzdata wget \
|
|
&& rm -rf /var/lib/apt/lists/* \
|
|
&& update-ca-certificates
|
|
|
|
COPY --from=builder /build/new-api /
|
|
COPY LICENSE NOTICE THIRD-PARTY-LICENSES.md /licenses/
|
|
EXPOSE 3000
|
|
WORKDIR /data
|
|
ENTRYPOINT ["/new-api"]
|