mirror of
https://github.com/QuantumNous/new-api.git
synced 2026-10-28 08:35:33 +08:00
CaIon
bb28f2ccad
Token estimates badly undercounted agent requests. Claude tool definitions, Responses function calls and their outputs, Chat assistant tool calls, and Gemini system instructions, function calls, function responses, and tools never reached the counter; Chat tool schemas were counted as Go map syntax and Claude tool results as escaped JSON. Measured against upstream usage on 54 real Claude Code and Codex conversation windows, native prompt estimates were 31-93% low, and the Claude and Gemini completion fallbacks about 90% low because streamed tool calls were not counted. All of these now reach the counter. The completion fallbacks count tool names and arguments, plus 40 tokens per Claude tool_use block. The estimator weights are refitted per tokenizer: Claude before Opus 4.7, Claude 4.7 and later (about 30% more tokens for the same text), GPT, and Gemini. Prompt estimates now land within 1% bias and 3-4% mean error for each family. Token counting moves into tokenkit, an independent module like relaykit, so other gateways can use it. It also replaces the flat 520-token image estimate with each provider's sizing rules, checked against upstream usage on sizes outside the test table: OpenAI patch and tile rules per model and detail level (gpt-5.5 and later default to original detail, and their high detail is billed as measured, not as documented), Claude's two resolution tiers, and Gemini 3 media resolution and Gemini 2.x tiles. Chat image_url parts without detail no longer default to high in ParseContent, so they estimate as the upstream default auto; no converter sent that value upstream. tiktoken-go moves to v0.8.1, about twice as fast on ordinary text. It requires Go 1.26, so the root module and tokenkit move to Go 1.26; relaykit stays on 1.25.1. In every version BPE cost grows with the square of a pre-tokenized piece, so 100,000 Chinese characters without punctuation took 21.6 s to count during pre-consume. Count now cuts single-class runs longer than 1024 bytes, which leaves ordinary text unchanged and brings that input down to 106 ms. The encoder cache no longer grows with client-supplied model names.
98 lines
2.6 KiB
YAML
98 lines
2.6 KiB
YAML
name: CI
|
|
|
|
on:
|
|
pull_request:
|
|
types: [opened, synchronize, closed]
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
concurrency:
|
|
group: ci-${{ github.workflow }}-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
jobs:
|
|
backend:
|
|
name: Backend vet, build, and test
|
|
if: github.event.action != 'closed' || github.event.pull_request.merged == true
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 15
|
|
env:
|
|
GOWORK: 'off'
|
|
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
|
|
with:
|
|
repository: ${{ github.event.pull_request.base.repo.full_name }}
|
|
ref: ${{ github.event.action == 'closed' && github.event.pull_request.base.ref || github.ref }}
|
|
|
|
- name: Set up Go
|
|
uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6.5.0
|
|
with:
|
|
go-version-file: go.mod
|
|
cache-dependency-path: |
|
|
go.sum
|
|
relaykit/go.sum
|
|
tokenkit/go.sum
|
|
|
|
- name: Create frontend embed placeholder
|
|
run: |
|
|
mkdir -p web/dist
|
|
touch web/dist/index.html
|
|
|
|
- name: Vet root module
|
|
run: go vet ./...
|
|
|
|
- name: Vet relaykit independently
|
|
working-directory: relaykit
|
|
run: go vet ./...
|
|
|
|
- name: Vet tokenkit independently
|
|
working-directory: tokenkit
|
|
run: go vet ./...
|
|
|
|
- name: Build root module
|
|
run: go build ./...
|
|
|
|
- name: Build relaykit independently
|
|
working-directory: relaykit
|
|
run: go build ./...
|
|
|
|
- name: Build tokenkit independently
|
|
working-directory: tokenkit
|
|
run: go build ./...
|
|
|
|
- name: Test root, relaykit, and tokenkit modules
|
|
run: make test
|
|
|
|
frontend:
|
|
name: Frontend typecheck and test
|
|
if: github.event.action != 'closed' || github.event.pull_request.merged == true
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 10
|
|
defaults:
|
|
run:
|
|
working-directory: web
|
|
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
|
|
with:
|
|
repository: ${{ github.event.pull_request.base.repo.full_name }}
|
|
ref: ${{ github.event.action == 'closed' && github.event.pull_request.base.ref || github.ref }}
|
|
|
|
- name: Set up Bun
|
|
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
|
|
with:
|
|
bun-version: '1.4.0'
|
|
|
|
- name: Install dependencies
|
|
run: bun install --frozen-lockfile
|
|
|
|
- name: Typecheck
|
|
run: bun run typecheck
|
|
|
|
- name: Test
|
|
run: bun run test
|