mirror of
https://github.com/apache/kafka.git
synced 2026-10-28 08:24:58 +08:00
David Jacot
c2fbf6f52f
`CachedTopicResolver.id` and `CachedTopicResolver.name` pass a lambda capturing the key to `HashMap.computeIfAbsent`, so every lookup allocates one, cache hits included. The JIT does not eliminate it: `HashMap.computeIfAbsent` is above C2's inlining limit, so the lambda escapes. `TopicIds` calls `id` for every topic it iterates and `name` in `contains`, so an assignor walking the subscriptions of a heterogeneous group allocates one lambda per member and subscribed topic. This patch reads the cache with `get` and only looks up the image and calls `put` on a miss. As before, an unknown topic resolves to `null` and is not cached. It also adds `topicId` and `topicName` to `CoordinatorMetadataImage`, which look up a topic without creating an `Optional` and a `TopicMetadata`, and uses them in both resolvers. Bytes allocated per assignment (`gc.alloc.rate.norm`) with heterogeneous subscriptions, before and after. `ConsumerAssignorBenchmark`, 5000 members, 20000 partitions, `FULL`: | Assignor | Subscription | Topics | Before | After | |----------|------------------------|--------|------------|----------| | UNIFORM | HETEROGENEOUS_DISJOINT | 500 | 47.3 MB | 35.2 MB | | UNIFORM | HETEROGENEOUS_NESTED | 500 | 126.2 MB | 90.1 MB | | UNIFORM | HETEROGENEOUS_DISJOINT | 5000 | 422.7 MB | 302.8 MB | | UNIFORM | HETEROGENEOUS_NESTED | 5000 | 1,211.9 MB | 852.2 MB | | RANGE | HETEROGENEOUS_DISJOINT | 500 | 38.0 MB | 14.4 MB | | RANGE | HETEROGENEOUS_NESTED | 500 | 110.1 MB | 38.4 MB | | RANGE | HETEROGENEOUS_DISJOINT | 5000 | 362.6 MB | 123.2 MB | | RANGE | HETEROGENEOUS_NESTED | 5000 | 1,083.0 MB | 363.3 MB | `TargetAssignmentBuilderBenchmark`, 10 partitions per member, `HETEROGENEOUS`: | Members | Topics | Before | After | |---------|--------|----------|----------| | 1000 | 100 | 2.4 MB | 1.7 MB | | 1000 | 1000 | 17.3 MB | 12.2 MB | | 10000 | 100 | 22.1 MB | 14.9 MB | | 10000 | 1000 | 162.5 MB | 112.0 MB | The `STABLE` event gives similar reductions. Homogeneous subscriptions resolve the topics of a single member, so the saving there is at most one lambda per topic. Reviewers: Sean Quah <squah@confluent.io>, Chia-Ping Tsai <chia7712@gmail.com>