compiler: reduce type and escape-analysis work, preserve PGO (#30991)

* tools: prefer llvm-profdata from the selected Clang toolchain

* docs: distinguish self-build baselines and record matched counts

* docs: clarify historical PGO attribution

* docs: separate self-build baselines and record matched revision check

* types: cache immutable canonical types before recursive interning

* types, flat: reuse fingerprints across temporary type wrappers

* docs: record completed compiler optimization validation

* docs: bisect PGO fallback and self-build instruction regression

* tools: honor equals-form compiler options for PGO

* types, transform: avoid repeated tagged type work
This commit is contained in:
Alexander Medvednikov authored and GitHub committed 2026-10-21 23:56:24 +03:00
1 parent 22b23697e8
commit 5f7d4d0980
22 files changed
+844 -103

No files matched your search

+11 -2
View File
@@ -611,6 +611,12 @@ this is the most reliable local measure of compiler work: it is less sensitive t
CPU frequency and scheduling. Also retain elapsed time, cycles, and peak memory; fewer instructions
do not guarantee lower latency or memory use. Compare counts on the same machine and architecture.
The instruction counter is scoped to the measured process; unrelated processes do not add their
instructions to it. An idle host is not required for instruction comparisons. Counts are dynamic,
not a static property of the binary: synchronization, retries, and runtime paths can vary. Use
serial paired measurements to check that variation. Record host load as context for elapsed time
and cycles; use controlled load when making latency claims.
For instructions per line, divide by physical lines in the `.v` and `.vh` files actually parsed,
including builtins, imports, blank lines, and comments. Collect `parsed .v lines` plus
`parsed .vh lines` in a separate unmeasured `-v` run with the same input flags. Preserve that
@@ -648,13 +654,16 @@ For a baseline/candidate comparison:
1. Build both compilers with the same production build recipe and C toolchain. Verify correctness,
diagnostics, and generated C before measuring. For a change intended to preserve output, require
identical C, allowing only explicitly recorded build metadata differences.
identical C, allowing only explicitly recorded build metadata differences. Validate generated
C when changing the fixture or compiler representation; matching invalid C is not sufficient.
2. Use the same host, OS, fixture, standard library, flags, environment, and filesystem placement.
Keep executable, source, and output paths identical between roles where possible; record any
remaining differences. Set `VJOBS=1` and `SOURCE_DATE_EPOCH=0`, and disable tracing/profiling.
Clear unintended compiler environment overrides, including `VFLAGS` and `VEXE`; record the
effective environment. Measure only successfully rebuilt compilers, never a stale binary.
Run only one benchmark process at a time, with no concurrent builds or tests.
Run only one benchmark process at a time. Other builds or tests on the host do not invalidate
process instruction counts; do not pause unrelated work or wait for an idle host. Record that
elapsed time and cycles collected under uncontrolled load are contextual measurements.
3. Run one warmup per role, then **ten measured pairs**, alternating baseline/candidate order:
AB, BA, AB, BA, and so on. Record all 22 runs, excluding both warmups from statistics.
Use a distinct timing/stdout/stderr log for every run and the same generated C output path.
+5 -1
View File
@@ -382,7 +382,11 @@ hello world
`v self` defaults to `-gc none`. Pass `-gc <mode>` if you need a different GC mode.
On non-Windows hosts, `v -prod self` uses profile-guided optimization (PGO) when the selected C
compiler is Clang and `llvm-profdata` is available. It builds an instrumented compiler, trains it on
compiler is Clang and `llvm-profdata` is available. Clang's reported installation directory and
resolved executable directory are searched before `PATH` and macOS `xcrun` for the profile tool,
keeping LLVM versions together.
Both `-cc clang` and `-cc=clang` select that toolchain; the last compiler option takes precedence.
It builds an instrumented compiler, trains it on
uncached self-compilation, then builds the optimized compiler. This takes longer to build the
production compiler. Other C toolchains and failed PGO attempts use a regular production build.
Use `v -prod -parallel-cc self` to skip PGO and compile the generated C units in parallel.
+47 -25
View File
@@ -456,6 +456,30 @@ fn pgo_compiler_kind(args []string) string {
if has_profile_cflag(args) {
return ''
}
ccompiler := self_build_c_compiler(args)
cc_file_name := os.file_name(ccompiler)
if cc_file_name.contains('clang') || cc_file_name.contains('gcc')
|| cc_file_name.contains('g++') || ccompiler == 'cc' {
cc_ver := os.exec([ccompiler, '--version']).output
if cc_ver.contains('clang') {
_ := find_llvm_profdata(ccompiler) or { return '' }
return 'clang'
}
if cc_ver.contains('Free Software Foundation') || cc_ver.contains('GCC') {
return 'gcc'
}
}
if cc_file_name.contains('clang') {
_ := find_llvm_profdata(ccompiler) or { return '' }
return 'clang'
}
if cc_file_name.contains('gcc') || cc_file_name.contains('g++') {
return 'gcc'
}
return ''
}
fn self_build_c_compiler(args []string) string {
// Environment flags precede CLI flags; the driver uses the last -cc.
mut ccompiler := ''
mut i := 0
@@ -463,6 +487,8 @@ fn pgo_compiler_kind(args []string) string {
arg := args[i]
if arg == '-cc' && i + 1 < args.len {
ccompiler = args[i + 1]
} else if arg.starts_with('-cc=') {
ccompiler = arg['-cc='.len..]
}
if arg in ['-prof', '-profile'] && i + 1 < args.len && args[i + 1].starts_with('-') {
i++
@@ -472,32 +498,28 @@ fn pgo_compiler_kind(args []string) string {
i++
}
}
if ccompiler == '' {
ccompiler = os.getenv_opt('CC') or { 'cc' }
}
cc_file_name := os.file_name(ccompiler)
if cc_file_name.contains('clang') || cc_file_name.contains('gcc')
|| cc_file_name.contains('g++') || ccompiler == 'cc' {
cc_ver := os.exec([ccompiler, '--version']).output
if cc_ver.contains('clang') {
_ := find_llvm_profdata() or { return '' }
return 'clang'
}
if cc_ver.contains('Free Software Foundation') || cc_ver.contains('GCC') {
return 'gcc'
}
}
if cc_file_name.contains('clang') {
_ := find_llvm_profdata() or { return '' }
return 'clang'
}
if cc_file_name.contains('gcc') || cc_file_name.contains('g++') {
return 'gcc'
}
return ''
return if ccompiler != '' { ccompiler } else { os.getenv_opt('CC') or { 'cc' } }
}
fn find_llvm_profdata() !string {
fn find_llvm_profdata(ccompiler string) !string {
// Clang launchers such as /usr/bin/clang can report a different installation
// directory. Use that toolchain before looking beside the launcher itself.
for line in os.exec([ccompiler, '--version']).output.split_into_lines() {
if line.starts_with('InstalledDir:') {
profdata := os.join_path(line.all_after(':').trim_space(), 'llvm-profdata')
if os.is_file(profdata) && os.is_executable(profdata) {
return profdata
}
}
}
if compiler := os.find_abs_path_of_executable(ccompiler) {
// Different LLVM releases can use incompatible raw profile formats.
// Resolve symlinks so a compiler in PATH still finds its own toolchain.
profdata := os.join_path(os.dir(os.real_path(compiler)), 'llvm-profdata')
if os.is_file(profdata) && os.is_executable(profdata) {
return profdata
}
}
if profdata := os.find_abs_path_of_executable('llvm-profdata') {
return profdata
}
@@ -609,7 +631,7 @@ fn compile_with_pgo(vroot string, vexe string, args []string, out_binary string,
mut llvm_profdata := ''
mut profile_data := ''
if cc_kind == 'clang' {
llvm_profdata = find_llvm_profdata() or {
llvm_profdata = find_llvm_profdata(self_build_c_compiler(effective_self_build_args(args))) or {
eprintln('PGO disabled: can not find `llvm-profdata`.')
return false
}
+37
View File
@@ -90,6 +90,41 @@ done
assert os.ls(root)!.filter(it.starts_with('.vself_pgo_')).len == 0
assert os.read_file(caller_profile)! == 'caller profile'
// An LLVM utility in PATH can belong to an older, incompatible compiler.
// Both an explicit compiler path and its symlink must select the sibling tool.
compiler_dir := os.join_path(root, 'compiler')
os.mkdir(compiler_dir)!
os.cp(os.join_path(root, 'bin', 'clang'), os.join_path(compiler_dir, 'clang'))!
os.chmod(os.join_path(compiler_dir, 'clang'), 0o755)!
os.mv(os.join_path(root, 'bin', 'llvm-profdata'), os.join_path(compiler_dir, 'llvm-profdata'))!
os.write_file(os.join_path(root, 'bin', 'llvm-profdata'), '#!/bin/sh\nexit 55\n')!
os.chmod(os.join_path(root, 'bin', 'llvm-profdata'), 0o755)!
compiler_link := os.join_path(root, 'bin', 'clang-linked')
os.symlink(os.join_path(compiler_dir, 'clang'), compiler_link)!
compiler_launcher := os.join_path(root, 'bin', 'clang-launcher')
os.write_file(compiler_launcher, '#!/bin/sh\nprintf "%s\\n" "clang version mock" ' +
os.quoted_path('InstalledDir: ${compiler_dir}') + '\n')!
os.chmod(compiler_launcher, 0o755)!
for compiler in [os.join_path(compiler_dir, 'clang'), compiler_link, compiler_launcher] {
for options in [['-cc', compiler], ['-cc=${compiler}']] {
os.write_file(os.join_path(root, 'calls'), '')!
selected := pgo_test_run(root, tool, options, false, '2', '')
assert selected.exit_code == 0, selected.output
assert !selected.output.contains('falling back'), selected.output
selected_calls := os.read_file(os.join_path(root, 'calls'))!.split_into_lines()
assert selected_calls.filter(it.starts_with('MERGE ')).len == 1, selected_calls.join('\n')
assert selected_calls.filter(it.starts_with('BUILD ')).len == 2, selected_calls.join('\n')
}
}
os.rm(os.join_path(root, 'bin', 'llvm-profdata'))!
os.mv(os.join_path(compiler_dir, 'llvm-profdata'), os.join_path(root, 'bin', 'llvm-profdata'))!
// A compiler without a sibling utility retains the PATH fallback.
os.write_file(os.join_path(root, 'calls'), '')!
path_fallback := pgo_test_run(root, tool, ['-cc', os.join_path(compiler_dir, 'clang')],
false, '2', '')
assert path_fallback.exit_code == 0, path_fallback.output
assert !path_fallback.output.contains('falling back'), path_fallback.output
for extra in [['-parallel-cc'], ['-no-prod-options'], ['-cflags', '-fprofile-use=external']] {
os.write_file(os.join_path(root, 'calls'), '')!
plain := pgo_test_run(root, tool, extra, false, '2', '')
@@ -105,6 +140,8 @@ done
extra := if compiler == 'clang' { []string{} } else { ['-cc', '${root}/bin/gcc'] }
for case_index, options in [extra,
['-cc', '${root}/bin/gcc', '-cc', '${root}/bin/${compiler}'],
['-cc', '${root}/bin/${other}', '-cc=${root}/bin/${compiler}'],
['-cc=${root}/bin/${other}', '-cc', '${root}/bin/${compiler}'],
['-profile', '-cc', '${root}/bin/${compiler}']] {
os.write_file(os.join_path(root, 'calls'), '')!
inherited_flags := if case_index == 0 { flags } else { '' }
+6 -3
View File
@@ -257,10 +257,13 @@ pub fn source_type_id(text string) u32 {
return id
}
@[inline]
@[direct_array_access; inline]
fn source_type_recent_slot(text string) int {
address := usize(text.str)
return int(((address >> 3) ^ (address >> 13) ^ (address >> 21)) & 4095)
// Equal spellings share a slot even when callers borrow different buffers.
// The complete canonical spelling is still checked before accepting a hit.
key := u32(text.len) ^ (u32(text[0]) << 1) ^ (u32(text[text.len >> 1]) << 5) ^
(u32(text[text.len - 1]) << 9)
return int((key ^ (key >> 12)) & 4095)
}
fn source_type_id_uncached(text string) u32 {
+13
View File
@@ -2,6 +2,19 @@ module flat
import hash
fn test_source_type_recent_cache_checks_borrowed_spelling_after_mutation() {
mut bytes := 'source_types.MutableBorrowedInput'.bytes()
text := unsafe { tos(bytes.data, bytes.len) }
id := source_type_id(text)
assert source_type_name(id) == 'source_types.MutableBorrowedInput'
bytes[1] = `X`
changed_id := source_type_id(text)
assert changed_id != id
assert source_type_name(changed_id) == 'sXurce_types.MutableBorrowedInput'
assert source_type_name(id) == 'source_types.MutableBorrowedInput'
assert source_type_id('source_types.MutableBorrowedInput'.clone()) == id
}
fn test_existing_source_type_does_not_take_the_insertion_lock() {
text := 'source_types.Existing'
id := source_type_id(text)
@@ -690,3 +690,41 @@ fn test_fixed_array_reference_sums_own_inline_variant_storage() {
seen.clear()
assert t.escape_value_contains_fixed_array(types.Type(types.Struct{ name_id: flat.source_type_id('FixedPayload') }), mut seen)
}
fn test_fixed_array_reference_nominal_cache_keeps_complete_walk_results() {
mut a := flat.FlatAst.new()
mut tc := types.TypeChecker.new(&a)
fixed := types.Type(types.ArrayFixed{ elem_type: types.Type(types.int_), len: 2 })
tc.structs['FixedPayload'] = [types.StructField{ name: 'values', typ: fixed }]
tc.sum_types.set('Empty', ['int', 'string'])
tc.sum_types.set('Payload', ['Other', 'FixedPayload'])
tc.sum_types.set('Other', ['Payload', 'int'])
tc.sum_types.set('Generic', ['int', 'T'])
tc.sum_generic_params['Generic'] = ['T']
mut t := new_transformer(mut a, &tc, map[string]bool{})
for _ in 0 .. 2 {
for name in ['Payload', 'Other', 'Generic[FixedPayload]', 'Generic[int]', 'Empty'] {
typ := types.Type(types.SumType{ name_id: flat.source_type_id(name) })
expected := name in ['Payload', 'Other', 'Generic[FixedPayload]']
assert t.cached_escape_value_contains_fixed_array(typ) == expected, name
assert t.escape_fixed_array_nominal_cache[typ.source_type_id()] == expected, name
}
}
// Other's first nested walk meets Payload in the active set. Only its later,
// complete root walk can be cached: it reaches FixedPayload through Payload.
assert t.escape_fixed_array_nominal_cache[flat.source_type_id('Other')]
}
fn test_fixed_array_reference_nominal_cache_caches_complete_negative_component() {
mut a := flat.FlatAst.new()
mut tc := types.TypeChecker.new(&a)
tc.sum_types.set('Left', ['Right', 'int'])
tc.sum_types.set('Right', ['Left', 'string'])
mut t := new_transformer(mut a, &tc, map[string]bool{})
left := types.Type(types.SumType{ name_id: flat.source_type_id('Left') })
right := types.Type(types.SumType{ name_id: flat.source_type_id('Right') })
assert !t.cached_escape_value_contains_fixed_array(left)
cached := t.escape_fixed_array_nominal_cache[right.source_type_id()] or { panic('missing complete result') }
assert !cached
assert !t.cached_escape_value_contains_fixed_array(right)
}
+1 -2
View File
@@ -1375,8 +1375,7 @@ fn (mut t Transformer) lower_indexed_for_in(id flat.NodeId, node flat.Node, key_
t.set_node_typ(int(container), flat.source_type_id(actual_iter_type))
} else if elem_needs_ref && actual_iter_type.starts_with('[]') && !isnil(t.tc)
&& (elem_name in t.escaping_amp_sources || elem_name in t.escaping_fixed_array_view_sources) {
mut seen := map[u32]bool{}
if t.escape_value_contains_fixed_array(t.tc.parse_type(elem_type), mut seen) {
if t.cached_escape_value_contains_fixed_array(t.tc.parse_type(elem_type)) {
// The binding hides its index from argument lowering. Retain each current
// buffer before exposing a row, including after growth between iterations.
t.mark_fn_used('array.retain_fixed_array_buffer')
+23 -9
View File
@@ -482,9 +482,9 @@ mut:
// parameter that can retain an inline fixed array as a dynamic view. Promote the root
// before any aliases form, including when a containing struct is forwarded by reference.
escaping_fixed_array_view_sources map[string]bool
// escape_fixed_array_struct_cache memoizes, per struct type, whether its inline
// storage holds a fixed array. The escape precheck asks for every reference argument.
escape_fixed_array_struct_cache map[u32]bool
// Cache complete walks of nominal inline storage. Sum variants can contain fixed
// arrays too; repeated reference arguments must not walk the same variants again.
escape_fixed_array_nominal_cache map[u32]bool
// skip_owned_base_nodes_without_regions: see PreparedSelfhostTransform.keeps_scope_without_regions.
skip_owned_base_nodes_without_regions bool
// heaped_amp_locals records which of those sources were actually moved to the heap, so
@@ -8592,13 +8592,24 @@ fn (mut t Transformer) escape_arg_value_contains_fixed_array(arg_id flat.NodeId)
for value_type is types.Pointer {
value_type = types.unalias_type(value_type.base_type)
}
if value_type is types.Struct {
if cached := t.escape_fixed_array_struct_cache[value_type.source_type_id()] {
return t.cached_escape_value_contains_fixed_array(value_type)
}
fn (mut t Transformer) cached_escape_value_contains_fixed_array(value_type &types.Type) bool {
if value_type is types.Struct || value_type is types.SumType {
if cached := t.escape_fixed_array_nominal_cache[value_type.source_type_id()] {
return cached
}
mut seen := map[u32]bool{}
contains := t.escape_value_contains_fixed_array(value_type, mut seen)
t.escape_fixed_array_struct_cache[value_type.source_type_id()] = contains
if !contains {
// A complete unsuccessful root walk proves every visited nominal type
// lacks inline fixed arrays, including nodes in recursive components.
for name_id, _ in seen {
t.escape_fixed_array_nominal_cache[name_id] = false
}
}
t.escape_fixed_array_nominal_cache[value_type.source_type_id()] = contains
return contains
}
mut seen := map[u32]bool{}
@@ -11051,7 +11062,7 @@ fn escape_type_is_struct_value(typ types.Type) bool {
// escape_value_contains_fixed_array follows inline storage only. A dynamic container or
// pointer field already has separate backing storage, so moving its containing value does
// not extend the lifetime of that backing.
fn (t &Transformer) escape_value_contains_fixed_array(typ types.Type, mut seen map[u32]bool) bool {
fn (mut t Transformer) escape_value_contains_fixed_array(typ types.Type, mut seen map[u32]bool) bool {
match typ {
types.ArrayFixed {
return true
@@ -11060,6 +11071,7 @@ fn (t &Transformer) escape_value_contains_fixed_array(typ types.Type, mut seen m
return t.escape_value_contains_fixed_array(typ.base_type, mut seen)
}
types.SumType {
if cached := t.escape_fixed_array_nominal_cache[typ.name_id] { return cached }
if isnil(t.tc) || typ.name_id in seen {
return false
}
@@ -11067,11 +11079,13 @@ fn (t &Transformer) escape_value_contains_fixed_array(typ types.Type, mut seen m
for variant in t.concrete_sum_variants_for_candidate(typ.name_id) {
value := t.tc.parse_type(variant)
if t.escape_value_contains_fixed_array(*value, mut seen) {
t.escape_fixed_array_nominal_cache[typ.name_id] = true
return true
}
}
}
types.Struct {
if cached := t.escape_fixed_array_nominal_cache[typ.name_id] { return cached }
if isnil(t.tc) || typ.source_type_id() in seen {
return false
}
@@ -11091,6 +11105,7 @@ fn (t &Transformer) escape_value_contains_fixed_array(typ types.Type, mut seen m
field.typ
}
if t.escape_value_contains_fixed_array(field_type, mut seen) {
t.escape_fixed_array_nominal_cache[typ.name_id] = true
return true
}
}
@@ -11269,8 +11284,7 @@ fn (mut t Transformer) mark_fixed_array_reference_argument_escape(arg_id flat.No
for value_type is types.Pointer {
value_type = types.unalias_type(value_type.base_type)
}
mut seen := map[u32]bool{}
if !t.escape_value_contains_fixed_array(value_type, mut seen)
if !t.cached_escape_value_contains_fixed_array(value_type)
|| !t.escape_address_expr_is_stack_local(value_id, local_stack_names, amp_ptrs, ptr_aliases) {
return
}
+21 -10
View File
@@ -10583,8 +10583,8 @@ pub fn (tc &TypeChecker) expr_type(id flat.NodeId) ?Type {
return *tc.parse_type(node.typ)
}
}
if t := tc.cached_expr_type(id) {
return t
if t := tc.cached_expr_type_ref(id) {
return *t
}
return none
}
@@ -10610,9 +10610,9 @@ fn (tc &TypeChecker) resolved_call_type(id flat.NodeId) ?Type {
}
}
}
if t := tc.cached_expr_type(id) {
if t := tc.cached_expr_type_ref(id) {
if t !is Void {
return t
return *t
}
}
if name := tc.cached_resolved_call(id) {
@@ -10660,27 +10660,38 @@ pub fn (tc &TypeChecker) invalidate_checked_expr_type(idx int) {
}
}
@[direct_array_access]
@[inline]
fn (tc &TypeChecker) cached_expr_type(id flat.NodeId) ?Type {
if typ := tc.cached_expr_type_ref(id) {
return *typ
}
return none
}
// Dense slots point at immutable canonical types. Sparse map values need stable
// copies because later insertions can move the map backing storage.
@[direct_array_access]
fn (tc &TypeChecker) cached_expr_type_ref(id flat.NodeId) ?&Type {
idx := int(id)
if !isnil(tc.storage_query_probe) {
if typ := tc.sparse_expr_type_values[idx] { return typ }
if typ := tc.sparse_expr_type_values[idx] { return new_type_reference(typ) }
if !isnil(tc.storage_query_probe.read_base) {
return tc.storage_query_probe.read_base.cached_expr_type(id)
return tc.storage_query_probe.read_base.cached_expr_type_ref(id)
}
return none
}
if tc.parallel_check_sparse {
if tc.in_check_range(idx) {
if idx < tc.expr_type_set.len && tc.expr_type_set[idx] {
return *tc.expr_type_values[idx]
return tc.expr_type_values[idx]
}
return none
}
return tc.sparse_expr_type_values[idx] or { none }
if typ := tc.sparse_expr_type_values[idx] { return new_type_reference(typ) }
return none
}
if idx >= 0 && idx < tc.expr_type_set.len && tc.expr_type_set[idx] {
return *tc.expr_type_values[idx]
return tc.expr_type_values[idx]
}
return none
}
+3 -3
View File
@@ -17737,9 +17737,9 @@ fn (mut tc TypeChecker) restore_return_alias_for_in_bindings(node flat.Node) {
fn (mut tc TypeChecker) return_alias_decl_binding_type(node flat.Node, index int) Type {
lhs_id := tc.multi_assign_lhs_id(node, index)
if typ := tc.cached_expr_type(lhs_id) {
if typ := tc.cached_expr_type_ref(lhs_id) {
if typ !is Unknown && typ !is Void {
return typ
return *typ
}
}
lhs_count := tc.multi_assign_lhs_count(node)
@@ -20906,7 +20906,7 @@ fn (mut tc TypeChecker) pointer_binding_state_after_loop_post(loop_node flat.Nod
if post.kind == .postfix && post.op in [.inc, .dec] && post.children_count == 1 {
target_id := tc.a.child(post, 0)
if tc.a.node(target_id).kind == .ident {
if target_type := tc.cached_expr_type(target_id) {
if target_type := tc.cached_expr_type_ref(target_id) {
// Rechecking a scalar counter cannot change pointer identities. The
// post has already been checked for diagnostics above the loop body.
if unalias_type(target_type) is Primitive && precomputed == base {
+3 -3
View File
@@ -15347,7 +15347,7 @@ fn (mut tc TypeChecker) check_valid_call_arg_types(id flat.NodeId, node flat.Nod
} else {
tc.check_node(check_arg_id)
}
if multi := tc.cached_expr_type(check_arg_id) {
if multi := tc.cached_expr_type_ref(check_arg_id) {
if !info.is_variadic && multi is MultiReturn {
expanded_arg_offset += multi.types.len - 1
}
@@ -21216,8 +21216,8 @@ fn (tc &TypeChecker) selector_fn_base_type(base_id flat.NodeId) ?Type {
}
}
}
if typ := tc.cached_expr_type(base_id) {
return typ
if typ := tc.cached_expr_type_ref(base_id) {
return *typ
}
if int(base_id) < 0 {
return none
+7 -16
View File
@@ -10509,9 +10509,6 @@ fn qualified_type_modifiers(name string) string {
fn (tc &TypeChecker) type_compatible(actual &Type, expected &Type) bool {
actual_raw := actual
expected_raw := expected
if semantic_types_equal(actual, expected) {
return true
}
// Equal semantic values have equal spellings even when their immutable
// payloads were allocated separately. Compare them before interning names.
if semantic_types_equal(actual, expected) {
@@ -11079,9 +11076,6 @@ pub fn (tc &TypeChecker) fn_param_compatible(actual &Type, expected &Type) bool
if semantic_types_equal(actual, expected) {
return true
}
if semantic_types_equal(actual, expected) {
return true
}
// V's platform `int` and a fixed-width integer such as `i64`/`i32` may share an
// emitted C spelling on a given target, but they remain distinct, target-
// independent source types. Function-parameter identity must not depend on the C
@@ -11149,9 +11143,6 @@ fn fn_param_is_platform_int(typ &Type) bool {
}
fn (tc &TypeChecker) fn_return_compatible(actual &Type, expected &Type) bool {
if semantic_types_equal(actual, expected) {
return true
}
if semantic_types_equal(actual, expected) {
return true
}
@@ -17082,10 +17073,10 @@ pub fn (tc &TypeChecker) type_name(t &Type) u32 {
return interner.source_type_id(id)
}
// type_recent_hash_slot chooses a recent-cache slot from semantic identity.
// type_recent_hash_slot uses a cheap fingerprint; callers verify structural equality.
@[inline]
fn type_recent_hash_slot(typ &Type) (u64, int) {
hash := semantic_type_hash(typ)
hash := canonical_intern_cache_key(typ)
return hash, int(hash & u64(type_cache_recent_slots - 1))
}
@@ -18814,7 +18805,7 @@ fn (mut memo BodyResolveMemo) begin(lo int, hi int) {
pub fn (tc &TypeChecker) resolve_type(id flat.NodeId) Type {
if tc.trust_checked_expr_types {
if !isnil(tc.storage_query_probe) {
if typ := tc.cached_expr_type(id) {
if typ := tc.cached_expr_type_ref(id) {
if !type_contains_unknown(typ) { return tc.widen_mixed_integer_expr_type(id, typ) }
}
}
@@ -19076,8 +19067,8 @@ fn (tc &TypeChecker) resolve_type_uncached(id flat.NodeId) Type {
// type its check gave it (see check_generic_method_value), which the
// resolution of an index cannot tell from an index of the method value.
if node.children_count > 0 && tc.a.child_node(node, 0).kind == .selector {
if typ := tc.cached_expr_type(id) {
typ_raw := typ
if typ := tc.cached_expr_type_ref(id) {
typ_raw := *typ
if typ is FnType {
return typ_raw
}
@@ -19089,12 +19080,12 @@ fn (tc &TypeChecker) resolve_type_uncached(id flat.NodeId) Type {
return builtin_bool_type
}
if !(tc.smartcasts.len > 0 && (kind_id == 7 || kind_id == 13)) {
if typ := tc.cached_expr_type(id) {
if typ := tc.cached_expr_type_ref(id) {
// A resolution-only annotation can cache `void` for a method whose
// receiver is a local alias that is only known during body checking.
// Re-resolve calls in that case instead of hiding the real return type.
if node.kind != .call || typ !is Void {
return typ
return *typ
}
}
}
+1 -1
View File
@@ -1013,7 +1013,7 @@ fn (tc &TypeChecker) incremental_touches_generics(f IncrementalFunction, generic
return true
}
}
if typ := tc.cached_expr_type(id) {
if typ := tc.cached_expr_type_ref(id) {
if type_mentions_generic_application(typ) {
return true
}
@@ -3,6 +3,27 @@ module types
import v.flat
fn test_repeated_scalar_interning_after_scope_promotion() {
$if prealloc {
mut interner := new_type_interner()
input := Type(int_)
scope := unsafe { prealloc_scope_begin() }
id, scoped := interner.canonicalize(input)
assert unsafe { prealloc_scope_owns(scope, scoped) }
unsafe { prealloc_scope_leave(scope) }
interner.promote_from(0, scope)
promoted_id, promoted := interner.canonicalize(input)
assert promoted_id == id
assert voidptr(promoted) == voidptr(interner.types[int(id)])
assert !unsafe { prealloc_scope_owns(scope, promoted) }
unsafe { prealloc_scope_free_after(scope) }
again_id, again := interner.canonicalize(input)
assert again_id == id
assert voidptr(again) == voidptr(promoted)
assert semantic_types_equal(again, input)
}
}
fn test_interner_owns_edges_from_a_temporary_arena() {
$if prealloc {
mut interner := new_type_interner()
+1 -1
View File
@@ -682,7 +682,7 @@ pub fn (value SumType) name() string {
}
// source_type_id returns an interned spelling, reusing nominal type identities directly.
pub fn (t Type) source_type_id() u32 {
pub fn (t &Type) source_type_id() u32 {
return match t {
Struct, Alias, Enum, Interface, SumType { t.name_id }
Void { flat.source_type_void }
+34
View File
@@ -928,3 +928,37 @@ fn test_new_type_reference_owns_its_descriptor_copy() {
assert reference is Primitive
assert reference.size == 16
}
fn test_cached_expr_type_references_preserve_storage_and_visibility() {
ast := flat.FlatAst.new()
mut tc := TypeChecker.new(&ast)
tc.reset_node_caches(8)
tc.remember_expr_type(flat.NodeId(1), Type(int_))
tc.remember_expr_type(flat.NodeId(5), Type(string_))
dense := tc.cached_expr_type_ref(flat.NodeId(1)) or { panic('missing dense type') }
assert voidptr(dense) == voidptr(tc.expr_type_values[1])
tc.parallel_check_sparse = true
tc.check_range_lo = 0
tc.check_range_hi = 4
assert tc.cached_expr_type_ref(flat.NodeId(5)) == none
tc.sparse_expr_type_values[5] = Type(bool_)
sparse := tc.cached_expr_type_ref(flat.NodeId(5)) or { panic('missing sparse type') }
assert *sparse == Type(bool_)
for i in 8 .. 4096 {
tc.sparse_expr_type_values[i] = Type(int_)
}
tc.sparse_expr_type_values[5] = Type(string_)
assert *sparse == Type(bool_)
assert *dense == Type(int_)
assert tc.cached_expr_type(flat.NodeId(5))? == Type(string_)
mut probe := tc.fork_storage_observation_view()
assert *probe.cached_expr_type_ref(flat.NodeId(1))? == Type(int_)
assert *probe.cached_expr_type_ref(flat.NodeId(5))? == Type(string_)
probe.sparse_expr_type_values[5] = Type(u64_)
assert *probe.cached_expr_type_ref(flat.NodeId(5))? == Type(u64_)
assert *tc.cached_expr_type_ref(flat.NodeId(5))? == Type(string_)
tc.invalidate_checked_expr_type(1)
assert tc.cached_expr_type_ref(flat.NodeId(1)) == none
assert *dense == Type(int_)
}
+24 -4
View File
@@ -8,6 +8,11 @@ import v.flat
// caches and equality-heavy internals can use this compact identity.
pub type TypeId = u32
struct CanonicalTypeEntry {
id TypeId
typ &Type
}
@[heap]
struct TypeInterner {
mut:
@@ -15,6 +20,8 @@ mut:
types []&Type
names []u32
buckets map[u64]TypeId
entries []&CanonicalTypeEntry
recent [4096]voidptr
}
fn new_type_interner() &TypeInterner {
@@ -48,7 +55,9 @@ fn (mut i TypeInterner) intern_locked(t &Type, hash u64) (TypeId, &Type) {
key = type_hash_tag(key, 0x5bd1e995)
}
id := TypeId(i.types.len)
i.types << new_type_reference(Type(owned))
canonical := new_type_reference(Type(owned))
i.types << canonical
i.entries << &CanonicalTypeEntry{ id: id, typ: canonical }
i.names << u32(0)
i.buckets[key] = id
return id, i.types[int(id)]
@@ -177,12 +186,17 @@ fn (mut i TypeInterner) source_type_id(id TypeId) u32 {
}
fn (mut i TypeInterner) canonicalize(t &Type) (TypeId, &Type) {
if entry := i.recent_canonical_type(t) {
return entry.id, entry.typ
}
hash := semantic_type_hash(t)
i.lock.lock()
defer {
i.lock.unlock()
}
return i.intern_locked(t, hash)
id, canonical := i.intern_locked(t, hash)
i.remember_canonical_type(t, i.entries[int(id)])
return id, canonical
}
fn (mut i TypeInterner) len() int {
@@ -202,6 +216,7 @@ fn (mut i TypeInterner) reserve(headroom int) {
unsafe {
i.types.grow_cap(headroom)
i.names.grow_cap(headroom)
i.entries.grow_cap(headroom)
}
i.buckets.reserve(u32(i.buckets.len + headroom))
}
@@ -217,10 +232,15 @@ fn (mut i TypeInterner) promote_from(start int, scope voidptr) {
// not only additions owned by the outer transform scope. Deep-copy the full
// stable-id table before any of those arenas are released.
previous := i.types
// All readers have joined before promotion replaces their canonical pointers.
i.clear_recent_canonical_types()
i.types = []&Type{cap: previous.len}
for typ in previous {
i.entries = []&CanonicalTypeEntry{cap: previous.len}
for idx, typ in previous {
owned := i.own_value(typ)
i.types << new_type_reference(Type(owned))
canonical := new_type_reference(Type(owned))
i.types << canonical
i.entries << &CanonicalTypeEntry{ id: TypeId(idx), typ: canonical }
}
i.names = i.names.clone()
// A scoped insertion can rehash even after the caller reserves headroom.
+87
View File
@@ -0,0 +1,87 @@
module types
// Entries are immutable after atomic publication. Their storage follows the
// canonical table; promotion clears the cache before releasing a stage arena.
@[direct_array_access; inline]
fn (i &TypeInterner) recent_canonical_type(t &Type) ?&CanonicalTypeEntry {
slot := int(canonical_intern_cache_key(t) & 4095)
entry := unsafe { &CanonicalTypeEntry(C.atomic_load_ptr(voidptr(&i.recent[slot]))) }
if !isnil(entry) && semantic_types_equal(entry.typ, t) {
return entry
}
return none
}
@[direct_array_access; inline]
fn (i &TypeInterner) remember_canonical_type(t &Type, entry &CanonicalTypeEntry) {
slot := int(canonical_intern_cache_key(t) & 4095)
C.atomic_store_ptr(voidptr(&i.recent[slot]), entry)
}
fn (i &TypeInterner) clear_recent_canonical_types() {
for slot in 0 .. i.recent.len {
C.atomic_store_ptr(voidptr(&i.recent[slot]), unsafe { nil })
}
}
// This forms a cache key without recursively hashing children. The owned entry is
// still compared structurally, so borrowed or mutated inputs cannot produce a hit
// merely because their addresses, lengths, or nominal spellings match.
@[inline]
fn canonical_intern_cache_key(t &Type) u64 {
mut key := match t {
Void { u64(1) }
Unknown { u64(2) ^ (u64(t.reason.len) << 8) }
Primitive { u64(3) ^ (u64(t.props) << 16) ^ (u64(t.size) << 8) }
String { u64(4) }
Char { u64(5) }
Rune { u64(6) }
ISize { u64(7) }
USize { u64(8) }
Nil { u64(9) }
None { u64(10) }
Array { u64(11) ^ canonical_intern_child_key(t.elem_type) }
ArrayFixed { u64(12) ^ canonical_intern_child_key(t.elem_type) ^ (u64(t.len) << 8) }
Channel { u64(13) ^ canonical_intern_child_key(t.elem_type) ^ (u64(t.is_mut) << 8) }
Map {
u64(14) ^ canonical_intern_child_key(t.key_type) ^ (canonical_intern_child_key(t.value_type) << 7)
}
Pointer { u64(15) ^ canonical_intern_child_key(t.base_type) }
FnType {
u64(16) ^ (u64(t.params.data) >> 3) ^ (u64(t.params.len) << 8) ^
(u64(voidptr(t.return_type)) << 7) ^ u64(t.is_variadic)
}
OptionType { u64(17) ^ canonical_intern_child_key(t.base_type) }
ResultType { u64(18) ^ canonical_intern_child_key(t.base_type) }
Struct { u64(19) ^ (u64(t.name_id) << 8) }
Interface { u64(20) ^ (u64(t.name_id) << 8) }
Enum { u64(21) ^ (u64(t.name_id) << 8) ^ u64(t.is_flag) }
SumType { u64(22) ^ (u64(t.name_id) << 8) }
Alias { u64(23) ^ (u64(t.name_id) << 8) ^ canonical_intern_child_key(t.base_type) }
MultiReturn { u64(24) ^ (u64(t.types.data) >> 3) ^ (u64(t.types.len) << 8) }
}
key = (key ^ (key >> 32)) * u64(0x9e3779b97f4a7c15)
return key ^ (key >> 32)
}
// Scalar and nominal children have cheap content identities even when their
// temporary Type wrappers occupy different addresses. Other children stay shallow.
@[inline]
fn canonical_intern_child_key(t &Type) u64 {
return match t {
Primitive { u64(3) ^ (u64(t.props) << 16) ^ (u64(t.size) << 8) }
Void { u64(1) }
String { u64(4) }
Char { u64(5) }
Rune { u64(6) }
ISize { u64(7) }
USize { u64(8) }
Nil { u64(9) }
None { u64(10) }
Struct { u64(19) ^ (u64(t.name_id) << 8) }
Interface { u64(20) ^ (u64(t.name_id) << 8) }
Enum { u64(21) ^ (u64(t.name_id) << 8) ^ u64(t.is_flag) }
SumType { u64(22) ^ (u64(t.name_id) << 8) }
else { u64(voidptr(t)) >> 3 }
}
}
@@ -8,9 +8,14 @@ fn test_interner_owns_nested_function_parameters() {
return_type: &Type(string_)
})
id, canonical := interner.canonicalize(original)
canonical_count := interner.len()
params[0] = FnParam{ typ: &Type(bool_), is_mut: true }
assert canonical.name() == 'fn(int) string', canonical.name()
changed_id, changed := interner.canonicalize(original)
assert changed_id != id
assert changed is FnType
assert changed.params[0].is_mut
assert semantic_types_equal(changed.params[0].typ, Type(bool_))
canonical_count := interner.len()
stable := Type(FnType{
params: [FnParam{ typ: &Type(int_) }]
return_type: &Type(string_)
+4 -2
View File
@@ -13,7 +13,7 @@ fn interner_parallel_writer(mut interner TypeInterner, input []Type, ready chan
}
}
fn interner_parallel_reader(interner &TypeInterner, ready chan bool, start chan bool, stop chan bool) {
fn interner_parallel_reader(mut interner TypeInterner, ready chan bool, start chan bool, stop chan bool) {
ready <- true
_ := <-start
// Keep probing until every writer has finished growing the table.
@@ -23,6 +23,8 @@ fn interner_parallel_reader(interner &TypeInterner, ready chan bool, start chan
return
}
assert canonical is String
_, repeated := interner.canonicalize(Type(string_))
assert voidptr(repeated) == voidptr(canonical)
select {
_ := <-stop {
return
@@ -60,7 +62,7 @@ fn test_interner_probe_during_parallel_growth() {
for lane in 0 .. lanes {
writers << spawn interner_parallel_writer(mut interner, input[lane * 3000..(lane + 1) * 3000],
ready, start)
readers << spawn interner_parallel_reader(interner, ready, start, stop)
readers << spawn interner_parallel_reader(mut interner, ready, start, stop)
}
// Release readers and writers together once all of them are running.
for _ in 0 .. 2 * lanes {
+451 -20
View File
@@ -4,6 +4,187 @@ Every compiler fix must preserve or improve self-compilation efficiency. Agents
retired instructions before and after compiler changes and resolve regressions before merging.
Keep correctness tests and generated-output checks alongside performance measurements.
## Why the reported scores differed
The earlier **115,328 -> 95,877 instructions/line** comparison used compiler sources and input
at `257b514a`, Apple Clang 21, and a **560,382-line** fixture. Its two roles were ordinary
production and PGO production builds.
The later **124,706.52** score was an **unprofiled baseline**, not its optimized result.
Homebrew Clang 23 wrote a profile that the selected Apple LLVM 21 merger could not read, so
self-building fell back to an ordinary production build. Fixing that tool lookup gave
**101,502.13** in the same comparison, on the newer **559,990-line** fixture.
Those historical scores changed compiler revision, input, and C toolchain. They must not be
presented as a continuous performance series. A fresh revision check, described below, kept
input and Apple Clang 21 fixed: **97,584.84 -> 105,074.91**, or **7.68% more instructions**.
That comparison confirms a revision-associated increase as well as the reporting error. The
follow-up bisection below locates the main increase at the tagged-union compiler change.
## Recent-commit bisection
The PGO failure was bisected over the eight first-parent commits from `257b514a30` to
`2d1367e589`. The first commit activating the failure is **`2d1367e589` (#30962)**,
which re-enabled automatic Clang PGO for production self-builds. Its parent does not attempt
automatic production PGO; a passing predicate there means no failing attempt, not successful PGO.
The underlying lookup predates that change: `find_llvm_profdata` was introduced in `a75cd076ed`
and searched PATH, then `xcrun`, without consulting the selected C compiler's installation.
With explicit Homebrew Clang 23 and Apple LLVM 21 selected for merging, the producer writes raw
profile format 11 while the merger expects format 10. The merge fails and self-building falls
back to an unprofiled compiler. Apple Clang 21 with its matching merger succeeds.
The bisect predicate compiled each historical, unmodified `vself.v` using one stable compiler.
A small V-driver stand-in built a real Clang 23 instrumented program, and the historical tool
ran it and attempted the real profile merge. This tests tool selection and fallback control flow;
it is not a full compiler self-build at each checkpoint. It reproduces the same format mismatch
as the retained full-build `latest2-before-build.log`. The toolchain lookup fix `8a6e8a32dc`
completed PGO in the same probe, providing a positive control.
Reproduction scripts, the complete Git bisect log, commands, and six checkpoint/control results
are in `/Users/alex/code/v3-self-90k-artifacts/bisect-20261021/`, including
`pgo-bisect.py`, `pgo-predicate.py`, `pgo-bisect.log`, and `pgo-summary.json`.
The instruction-count bisect identifies **`da62dd4bd7` (#29578), "v3: represent sum types as
tagged unions"**, with immediate parent `7940bb1a8e`. This is a separate change from the PGO
activation. Both sides completed PGO with Apple Clang 21 and its matching profile merger.
Each checkpoint was rebuilt twice in place with TCC using a seed compatible with its recursive
type representation. The resulting compiler then built its own instrumented and optimized
production versions with Apple Clang 21 and `-cflags '-mcpu=apple-m1'`. PGO training used the
same unchanged 559,990-line fixture, once with default workers and once with `VJOBS=1`.
This fresh recipe differs from earlier sessions; compare roles within this session rather than
treating their absolute scores as a continuation of earlier tables.
Three exploratory samples per checkpoint located the boundary. A separate confirmation used
one warmup per role and ten alternating measured pairs, with `VJOBS=1` and the same executable,
source, and output paths. The parent was built in a second checkout to overlap C compilation;
its exact build path is recorded. Fixture training and benchmark runs were serialized.
| Median metric | Parent `7940bb1a8e` | Tagged unions `da62dd4bd7` |
| --- | ---: | ---: |
| Retired instructions | 54,270,831,065.5 | 59,248,702,390.5 |
| Instructions/parsed line | 96,913.93 | 105,803.14 |
| CPU cycles, billions | 19.4757 | 21.6487 |
| Peak resident memory, MiB | 1,133.09 | 1,117.53 |
| Elapsed seconds | 5.0598 | 6.1247 |
The tagged-union commit used **9.17% more instructions**, increasing work in **all ten pairs**.
The median paired increase was `4,788,159,580` instructions. All 22 runs succeeded; generated
C was consistent within each role and passed Clang syntax checking. C intentionally differs
between these revisions, which change code generation. Elapsed time and cycles reflect shared
host load. These results attribute the main increase to the commit, not to one isolated hunk.
A separate exploratory control disabled PGO for both production compilers while retaining Apple
Clang 21 and the other build flags. Instructions/line increased from **116,908.51 to 125,334.54**
(*7.21% more*, all three pairs worse). Generated C matched the PGO output for each revision.
This supports a compiler-work increase that persists without PGO; the primary attribution remains
the ten-pair PGO comparison above. Control commands and all eight samples are retained as
`unprofiled-confirmation-*`. A disk-space failure before control measurement was retained and
retried after compressing owned artifacts; see `space-recovery.json`.
A separate verbose hardware-counter diagnostic, after one warmup per role, found approximately
1.97 billion additional instructions in checking, 0.98 billion in transforms, and 1.84 billion
in C generation/finish. Those instrumented runs are excluded from the primary statistics.
The commit changes semantic type handling as well as sum layout. An executable layout probe
using identical fixture definitions measured `sizeof(types.Type)` changing from **16 to 72 bytes**;
`flat.Node` remained 64 bytes. Type interning now canonicalizes child references, and C generation
replaces some raw-word identity checks with structural hashing/equality. The compatibility paths
also retain duplicated structural-equality calls when the first comparison returns false;
disassembly confirms both calls survive optimization in `fn_param_compatible` and
`fn_return_compatible`. These are concrete follow-up targets, not a measured attribution of the
entire increase to larger values or duplicated calls. Overall peak memory decreased in this run.
The bisect log, checkpoint builds/profiles, 22 confirmation samples, generated-output checks,
stage counters, layout probe, and disassembly are retained in the bisection directory above.
See `instruction-bisect.log`, `confirmation-{metadata,samples,summary}.json`,
`stage-breakdown.json`, `layout-probe.json`, and `compatibility-assembly.json`.
`exploratory-log-audit.json` records reused exploratory filenames during the parallel parent
prebuild; the accepted confirmation has distinct raw logs for every sample.
Large generated C artifacts were losslessly compressed to `.c.gz` with SHA-256 roundtrip checks;
`generated-c-compression.json` records their original paths and hashes. No compiler source or
test expectations were changed as part of the bisection.
## Tagged-union regression follow-up
The follow-up patch removes repeated work made more expensive by the tagged representation:
* Cache complete inline-storage walks for sums as well as structs, including concrete generic
sums. Reuse positive nested results immediately. Cache negative results across a recursive
component only after a complete root walk proves that it contains no inline fixed array.
* Read existing immutable canonical type references when checking a cached expression, avoiding
intermediate copies of the 72-byte type value. Preserve worker ranges and probe overlays;
sparse map values use stable copies because map growth can move their backing storage.
* Remove duplicate semantic-equality calls in three compatibility paths. This cleanup alone did
not establish a measurable improvement in exploratory runs.
Both compilers were freshly built from `36232086a5`, with the candidate's exact source diff
retained in the build artifacts. Each used the same seed, two in-place TCC rebuilds, Apple
Clang 21, `-mcpu=apple-m1`, and fresh matching PGO training on the unchanged fixture. Compiler
build paths differed; benchmark executable, input, output, and working-directory paths matched.
The PGO option-selection fixes were present in the candidate and do not change this explicit
Apple Clang build recipe. No compiler layout, language semantics, or generated output changed.
One warmup per role preceded ten alternating pairs in each mode. All 44 runs succeeded with
one positive retired-instruction count per run and byte-identical generated C within each mode.
No builds or tests from this task ran concurrently with these comparisons.
Parsed input remained 559,990 lines.
| Median metric | Before, one worker | Fix, one worker | Before, default | Fix, default |
| --- | ---: | ---: | ---: | ---: |
| Retired instructions, billions | 56.5777 | 55.8777 | 60.8406 | 60.1385 |
| Instructions/parsed line | 101,033.39 | 99,783.40 | 108,645.90 | 107,392.18 |
| CPU cycles, billions | 18.7694 | 18.6136 | 27.4735 | 27.3065 |
| Peak resident memory, MiB | 1,117.98 | 1,118.08 | 1,011.30 | 1,010.96 |
| Elapsed seconds | 4.4335 | 4.3891 | 1.0224 | 1.0229 |
The patch used **1.24% fewer instructions with one worker** and **1.15% fewer with default
workers**, with **0/10 regressing pairs** in each mode. Median paired differences were
`-668,456,564` and `-704,473,390.5` instructions, respectively. This comparison measures the
combined patch; it does not assign the improvement to each edit. Elapsed time and cycles remain
contextual measurements on a shared host, not controlled latency claims.
The **90,000 instructions/line target remains unmet**, and the full tagged-union regression has
not been recovered. Do not add percentages from different sessions or compare their medians as
a single continuous optimization series.
The baseline executable SHA-256 is
`5f8939b958bb5f8c9727202c12d51fad8449ab1fca00346f2cc605e6fbf68c68`;
the candidate executable SHA-256 is
`5e5686330fc08937d3bd8252ba9aacc0dfe75113d1a3a5d388fe2bfa6cf2f9af`.
Serial generated C has SHA-256
`88583a0ca0e4f59877f7c2b7a6cb4260c046c4737a61c665b62af265344bba8d`;
default-worker generated C has SHA-256
`04603de3a32496290c63a567e46069509fd0f985906a40b9696303f14a337158`.
Reproduction scripts, exact build/environment metadata, source diffs, fixture inventories,
per-run timing/stdout/stderr/status logs, binary hashes, and generated-output checks are in
`/Users/alex/code/v3-self-90k-artifacts/fix-tagged-unions-20261021/`.
Use `build.py`, `measure.py`, `source-a/metadata.json`, `combined/metadata.json`,
`pr-source-verification.json`, and `combined-confirmation-{serial,default}*`.
The PR source adds a helper comment and a reference ownership/visibility test to the measured
implementation. Failed and rejected exploratory variants remain separately labeled in this
artifact directory and are excluded from the accepted statistics.
Validation rebuilt the compiler in place and passed the type-cache, storage-query, parallel-check,
fixed-array escape, and loop-lowering tests. Compiler diagnostics passed 1,848 cases with 5 skipped;
PGO tool-selection tests, both generated-C syntax checks, formatting, and Markdown checks passed.
Vet reported the same 92 existing diagnostics as the unchanged baseline.
The broad compiler suite finished with **4,138 passed, 40 skipped, and 30 failed**. Twenty-eight
failing files also failed on the unchanged baseline. The two additional failures were harness
effects: the relocated benchmark compiler retained its build-directory path in REPL output, and
`SOURCE_DATE_EPOCH=0` contradicted a test requiring a recent build timestamp. After rebuilding in
the test checkout, all 47 REPL cases passed. The timestamp test passed with the override unset;
the unchanged baseline reproduced its failure with the override set.
The cache-refresh failure also varied with compiler placement and macOS `/tmp` versus
`/private/tmp` aliases. Both baseline and candidate passed this test in the same checkout with
the same canonical `VTMP`. Other failure assertions and primary diagnostics matched the baseline.
No tests or expectations were changed to suppress failures; the broad run remains a failed run.
Logs and comparisons are retained as `final-*`, including `final-full-audit.json`,
`final-harness-reruns.json`, and `final-cache-canonical-comparison.json` in the artifact directory.
## Metric and scope
Use **retired instructions per parsed source line** to express compiler work; lower is better:
@@ -31,6 +212,15 @@ Compare on the same machine and architecture. Keep compiler build options identi
changing the build recipe is the optimization being evaluated; record that change. Record elapsed
time, CPU cycles, and peak resident memory too. Fewer instructions do not guarantee lower latency.
Retired instructions are counted for the measured process. Other processes do not contribute
their instructions, so an idle host is not required. These are dynamic execution counts: locks,
retries, and other runtime paths can still introduce variation. Use `VJOBS=1` and repeated pairs
to assess consistency. Record competing work as context for elapsed time and cycles; collect
latency claims separately under controlled load. Apple documents the per-thread and per-task
accounting in [XNU Recount][recount].
[recount]: https://github.com/apple-oss-distributions/xnu/blob/main/doc/observability/recount.md
The workload compiles the full `cmd/v` compiler through C generation, matching the V stages of
`v self`. It excludes the external C compiler, linker, replacement of `./v`, and execution of
the resulting compiler. It includes parsing, checking, transforms, and C generation. Use one
@@ -85,8 +275,10 @@ worker for the instruction comparison; validate default parallelism separately.
`V_MACOS_V3_NO_FALLBACK=1` prevents silent fallback to the legacy compiler.
Reuse one output C path across all paired runs and keep distinct timing/stdout/stderr logs.
5. Run one warmup per role, then ten measured pairs, alternating AB, BA, AB, BA, and so on.
Run only one benchmark at a time, without concurrent builds or tests. Keep native support
Run only one benchmark at a time. Do not wait for an idle host or pause unrelated work.
Other builds and tests do not invalidate process instruction counts. Keep native support
objects warm for both roles and record that state. Disable tracing and profiling.
Label elapsed time and cycles collected under uncontrolled load as contextual measurements.
6. Require exit status zero and exactly one positive `instructions retired` counter per sample.
Preserve failed samples but do not count them as a completed comparison. If hardware counters
are missing or zero, report them as unavailable; do not substitute elapsed time.
@@ -101,23 +293,259 @@ worker for the instruction comparison; validate default parallelism separately.
On another platform, use an available hardware retired-instruction counter with the same scope
and protocol; do not compare its absolute counts with this machine's results.
## Benchmark acceptance status
## Current self-compilation fixture
**No accepted regression baseline is recorded yet.** The earlier measurements below ran alongside
unrelated compilations and do not meet the required quiet-host protocol. They are retained as
rejected observations for audit purposes, not as evidence that the 10% target has been met.
Both paired modes must be rerun without concurrent builds or tests before accepting a baseline.
New comparisons use a detached checkout of merged master
`2d1367e589ed30c03325e586a8ddfd3bc96c8c3d`, including its standard library.
It contains **559,990 parsed physical lines in 562 `.v` files**, with no `.vh` files.
The 90,000 instructions/line target is **50,399,100,000 instructions** for this fixture.
Keep this source and denominator fixed across baseline and candidate runs.
A guarded attempt on 2026-10-21 waited 120 seconds for a quiet window and still observed 31
compiler or test processes. It started no warmups or measured runs. The attempt and process
records are retained in `/Users/alex/code/v3-self-speed-artifacts/quiet-20261021-attempt1.log`
and `quiet-20261021-attempt1-guard.json`. A quiet host is required to complete this validation.
The older 257b fixture below is incompatible with the merged compiler's sum-type representation:
it emits incomplete recursive C union fields for its old compiler types. A C-file hash match
alone did not catch this. The new fixture's generated C passes Clang syntax checking. Validate
generated C when changing the fixture or compiler representation; do not accept counts for an
invalid output. This establishes a new baseline; do not compare normalized scores across fixtures.
### Rejected observations from the shared host
An exploratory three-pair run measured 105,611.36 instructions/line for merged master and
105,762.28 for local-storage lookup guards. Generated C matched in every run. The guards showed
no convincing gain and are excluded from the optimization. Three pairs do not establish a small
regression or a completed ten-pair comparison. Logs remain under
`/Users/alex/code/v3-self-90k-artifacts/guard-probe*`.
The new fixture is `/Users/alex/code/v3-self-90k-fixture`. Its 8,498-file source manifest
uses the encoding described below and has SHA-256:
`f53a9b162657ba667fde1d8a00cd6f59c0f7e6c7f847985894d61189122a2966`.
Its source inventory, syntax-check logs, build logs, binaries, and instruction samples remain in
`/Users/alex/code/v3-self-90k-artifacts/`.
## Semantic interning cache comparison
The first source optimization on master `9ca360c4ff` adds an atomic cache of immutable canonical
semantic types. Hits verify complete structural equality before bypassing recursive hashing,
locking, and map lookup. Promotion clears the cache before worker arenas are released. The
checker also uses the cheap fingerprint for its existing type-name cache, retaining its full
structural-equality check.
Both roles use successful Clang 23 PGO builds with `-mcpu=apple-m1`, the same build recipe,
and the unchanged 559,990-line fixture. The candidate is the first cache variant, before the
subsequent source-spelling and child-fingerprint experiments.
| Median metric | Before, one worker | Cache, one worker | Before, default | Cache, default |
| --- | ---: | ---: | ---: | ---: |
| Retired instructions | 57,045,612,737.0 | 55,295,951,631.0 | 61,027,236,671.0 | 59,242,819,343.0 |
| Instructions/parsed line | 101,868.98 | 98,744.53 | 108,979.15 | 105,792.64 |
| CPU cycles, billions | 27.0600 | 26.8994 | 28.7436 | 28.6683 |
| Peak resident memory, MiB | 1,118.17 | 1,118.55 | 1,011.12 | 1,014.97 |
| Elapsed seconds | 10.6821 | 9.4104 | 2.7068 | 2.6945 |
Retired instructions decreased **3.07% with one worker** and **2.92% with default parallelism**.
All ten pairs in each mode improved. Median paired instruction differences were
`-1,799,425,264` and `-1,775,404,206.5`, respectively. All 44 samples succeeded and generated
byte-identical C within each mode. C passed Clang syntax checking. Elapsed time and cycles
were collected under uncontrolled host load. **The 90,000 instructions/line target is not met.**
The 151 type tests passed, including borrowed-payload mutation, concurrent growth, and arena
promotion checks. Validation of the final cache bundle is recorded below.
Commands, environment, raw counters, and output checks are retained as
`interner-cache-matched-*` and `interner-cache-syntax-validation.json` in the artifacts directory.
The baseline executable SHA-256 is
`575ec55f8f553cfea6bff81ad937192687a840d84cc0f7e8832327804be5b8b7`;
the candidate executable SHA-256 is
`22e35698c36f05c7009b2516cda9270985366404230fabdf2399c16f178a3572`.
## Follow-up cache comparison with Apple Clang 21
A fresh comparison on master `9ca360c4ff` used Apple Clang 21 PGO for both roles and the same
559,990-line fixture. The candidate combines the semantic interning cache with source-spelling
content fingerprints, shallow semantic child fingerprints, and a reference receiver for
`Type.source_type_id`. Fingerprints still require complete equality checks before reuse.
| Median metric | Before, one worker | Cache, one worker | Before, default | Cache, default |
| --- | ---: | ---: | ---: | ---: |
| Retired instructions, billions | 59.0643 | 57.5221 | 63.1715 | 61.5222 |
| Instructions/parsed line | 105,473.91 | 102,719.79 | 112,808.22 | 109,862.98 |
| CPU cycles, billions | 29.1419 | 28.4944 | 31.1445 | 30.3943 |
| Peak resident memory, MiB | 1,117.54 | 1,118.23 | 1,007.91 | 1,010.11 |
| Elapsed seconds | 33.9576 | 34.9625 | 6.2985 | 6.6990 |
The combined change used **2.61% fewer instructions in both modes**, with **0/10 regressing
pairs** in each. Median paired differences were `-1,529,766,769` instructions serial and
`-1,646,091,022.5` default. All 44 runs succeeded and emitted byte-identical C within each mode.
Elapsed time and cycles reflect uncontrolled host load. This is a matched source-change
comparison; its absolute scores must not be compared with the Clang 23 sessions.
The 90,000 instructions/line target is still unmet.
Raw samples, binary hashes, commands, environment, and output checks are retained as
`reference21-content-matched-*` in the artifacts directory above. This bundle comparison does
not establish an independent benefit from each additional fingerprint change.
Validation of this exact cache bundle completed with 1,848 compiler-error cases passing and
5 skipped; all 151 type tests, source-spelling tests, and PGO/self-build tool tests passed.
Formatting, Markdown checks, and vet completed; vet reported existing documentation warnings.
The broad compiler suite finished with **4,140 passed, 40 skipped, and 28 failed**. All 28 failing
files also failed with the unchanged `9ca360c4ff` baseline binary. Assertion locations and primary
compilation diagnostics matched; the cache-refresh test required the same checkout path to reach
the same assertions. No tests or expectations were changed to suppress these failures.
This is not a full-suite pass. Raw logs and comparisons are retained as
`content-child-compiler-tests.*`, `baseline-failed-subset-*`, and `baseline-cache-same-root.*`.
## LLVM toolchain lookup comparison on latest master
Measured on 2026-10-21 with compiler sources from master `9ca360c4ff`, the toolchain lookup
fix at `8a6e8a32dc`, and the same 559,990-line fixture. Both compilers were rebuilt in place
with the command above, using Homebrew Clang 23.1.0 and `-cflags '-mcpu=apple-m1'`.
The baseline again fell back after the LLVM 21/23 profile-format mismatch; the candidate
completed PGO. This compares the tool lookup fix on the same compiler sources, so upstream
changes between master versions are not attributed to that fix.
| Median metric | Before, one worker | After, one worker | Before, default | After, default |
| --- | ---: | ---: | ---: | ---: |
| Retired instructions, billions | 68.9401 | 56.3825 | 73.8913 | 60.6981 |
| Instructions/parsed line | 123,109.57 | 100,684.76 | 131,951.04 | 108,391.44 |
| CPU cycles, billions | 23.0552 | 20.1368 | 32.7966 | 29.9206 |
| Peak resident memory, MiB | 1,116.04 | 1,118.15 | 1,009.52 | 1,014.15 |
| Elapsed seconds | 5.4194 | 4.7996 | 1.2632 | 1.1951 |
Instruction reductions were **18.22% serial** and **17.85% default**, with **0/10 regressing
pairs** in each mode. Median paired instruction differences were `-12,579,586,975.5` serial
and `-13,218,242,894.5` default. **The 90,000 instructions/line target remains unmet.**
All 44 runs succeeded and generated byte-identical C within each mode; both outputs passed
Clang syntax checking. Elapsed time and cycles were collected under uncontrolled host load.
The PGO, ordinary self-build, and cache-bypass tool tests passed, as did the hello-world smoke test.
Raw samples, complete environments, commands, build logs, binary hashes, syntax checks, and
validation logs are retained as `latest2-*` in the artifacts directory above. Generated C
hashes match the preceding master comparison below. SHA-256 values:
```text
baseline executable, latest2-before-v
8cdaf2fd18d6abd30e12e67953914abfa1d0b3e733211c4509b316095de71b9b
candidate executable, latest2-after-v
575ec55f8f553cfea6bff81ad937192687a840d84cc0f7e8832327804be5b8b7
merged training profile
93e991a1f4f6952c4645d0dab67a4da1a897fca48c647dea5dcc5c53f9f1db1b
```
## Compiler revision check on identical input
A separate 2026-10-21 comparison held the 559,990-line fixture, source-file list, Apple Clang 21,
production settings, and PGO recipe fixed. The older binary uses compiler sources at `257b514a`;
the newer binary uses merged sources at `2d1367e589`. Each build trained on its own revision's
compiler source, following the same self-build recipe. The older binary successfully compiles
the newer fixture to C, which makes this direction of comparison possible.
| Median metric | Older compiler | Merged compiler |
| --- | ---: | ---: |
| Retired instructions | 54,646,536,102.5 | 58,840,900,876.5 |
| Instructions/parsed line | 97,584.84 | 105,074.91 |
| CPU cycles, billions | 23.5851 | 25.1845 |
| Peak resident memory, MiB | 1133.02 | 1117.52 |
| Elapsed seconds | 7.9881 | 8.9251 |
The merged compiler executed **7.68% more instructions** between medians, with more work in
**10/10 pairs**. The median paired increase was `4,174,866,314` instructions. All 22 runs
succeeded and each role produced repeatable C. Both C outputs passed Clang syntax checking.
They intentionally differ across roles because compiler revisions changed sum-type representation;
this is a revision diagnostic, not an output-preserving internal optimization comparison.
It establishes increased work for these compiler builds without isolating a particular commit.
Elapsed time and cycles were collected under uncontrolled host load.
Thus the earlier reporting mixed an unprofiled baseline with a PGO result, but a compiler-version
increase also exists on matched input. The historical 95,877.04 result still uses the older
560,382-line fixture; attributing the remaining fixture effect requires a separate matched check.
Do not explain the entire discrepancy as denominator changes or assume that no regression exists.
Logs, inventories, counters, commands, environments, and syntax checks are retained as
`revision-diagnostic-*` in the artifacts directory above. The 562-file inventory matches exactly
between roles. SHA-256 values:
```text
older compiler executable
e554d76a9ad101c03c74864b5d428cef6f7b1df29fd11dad8ab43e74e60cbde4
merged compiler executable
7aab1e75525e5966566d7fe30debb543aaaeea68159dcd94047b46b7ada3788f
older compiler generated C on the current fixture
abb6372f6afc7892a15db0843ad4cca9fa2ca638ef331704866d59fb6d72d50d
merged compiler generated C on the current fixture
88583a0ca0e4f59877f7c2b7a6cb4260c046c4737a61c665b62af265344bba8d
```
## Matched Clang 23 comparison before the latest master rebuild
Measured on 2026-10-21 with compiler sources from merged master `2d1367e589` and the fixed
559,990-line fixture above. Both roles selected Homebrew Clang 23.1.0 and `-mcpu=apple-m1`.
The baseline selected Apple's LLVM 21 profile merger, which rejected Clang 23's version-11 raw
profile and triggered an ordinary production build. The candidate selected Clang 23's merger
and completed PGO. Both final binaries contain no profiling runtime instrumentation.
The same production self-build command was used in each checkout, outside measured runs:
```sh
env -u VFLAGS -u VEXE -u LLVM_PROFILE_FILE -u VJOBS SOURCE_DATE_EPOCH=0 \
V_MACOS_V3_NO_FALLBACK=1 ./v -prod -cc /opt/homebrew/opt/llvm/bin/clang \
-cflags '-mcpu=apple-m1' -o /absolute/path/to/role-v self
```
Each mode used one warmup per role and ten alternating measured pairs. All 44 runs succeeded;
generated C matched byte for byte within each mode. Elapsed time and cycles reflect shared-host
load and do not establish a latency improvement.
| Median metric | Before, one worker | After, one worker | Before, default | After, default |
| --- | ---: | ---: | ---: | ---: |
| Retired instructions, billions | 69.8344 | 56.8402 | 74.4558 | 61.1362 |
| Instructions/parsed line | 124,706.52 | 101,502.13 | 132,959.12 | 109,173.73 |
| CPU cycles, billions | 33.1496 | 28.9486 | 33.3547 | 30.3399 |
| Peak resident memory, MiB | 1116.16 | 1118.05 | 1008.88 | 1010.32 |
| Elapsed seconds | 21.2254 | 18.6263 | 5.4424 | 5.5684 |
The median instruction reductions were **18.61% serial** and **17.89% default**, with
**0/10 regressing pairs** in each mode. Median paired differences were `-12,984,267,344.5`
and `-13,337,504,752` instructions, respectively. The 90,000 target was not reached.
Logs and samples are `clang23-matched-{serial,default}*` in the artifacts directory above;
`clang23-before-build.log` records the version mismatch and fallback, while
`clang23-after-build.log` records the successful PGO build.
The optimized count is higher than the historical 95,877.04 result below, but these sessions
do not isolate a compiler regression. The historical result used Apple Clang 21, compiler
sources at `257b514a`, and a different source fixture.
Both compiler implementation and input changed after the tagged-union merge. The fixture has
only 392 fewer lines, so the denominator alone does not explain the difference. Isolating the
causes requires matched comparisons; comparing these two sessions cannot attribute the increase.
In particular, 124,706.52 is the current **unprofiled baseline**, not the optimized result.
SHA-256 values for this comparison:
```text
baseline executable, clang23-before-v
1541c85d77b10337467caacbc6cf67314a1635d78ce5c9e20e1b8b7f1fb6d454
candidate executable, clang23-after-v
c5ad981e88dc2f22fd0bd97042094e89a55ac85ee4dcd75e2ffa2ac86714edaa
generated C, every serial run
88583a0ca0e4f59877f7c2b7a6cb4260c046c4737a61c665b62af265344bba8d
generated C, every default-worker run
04603de3a32496290c63a567e46069509fd0f985906a40b9696303f14a337158
```
## Historical measurement status
The PGO comparison below records process instruction counts under shared-host load on the old
fixture. The earlier quiet-host gate was too strict: unrelated compilations do not invalidate
these instruction measurements. All ten paired differences favored PGO in each mode, with
matching source inventories and generated C. Elapsed times and cycles reflect uncontrolled
host load and do not establish a latency improvement. These results describe the older compiler
and fixture, not the current self-compilation baseline.
The 2026-10-21 attempt that waited 120 seconds for an idle host started no measurements.
Its logs remain in `/Users/alex/code/v3-self-speed-artifacts/quiet-20261021-attempt1.log`
and `quiet-20261021-attempt1-guard.json`; an idle host is no longer an acceptance requirement.
### Historical PGO instruction comparison
Measured on 2026-10-20. Baseline: merged master
`257b514a303f4d302ed23bbac1d51a2ca9624a66`. Candidate: that commit with the Clang production
self-build changes in this PR. The compiler source compiled into both binaries is unchanged;
self-build changes later merged in [#30962](https://github.com/vlang/v/pull/30962).
The compiler source compiled into both binaries is unchanged;
the intentional build difference is profile-guided optimization (PGO).
The candidate was rebuilt and remeasured after correcting atomic profile collection and
isolating the training environment.
@@ -129,8 +557,8 @@ default workers and `VJOBS=1`, merged the profiles, and rebuilt with `-fprofile-
contains no profiling instrumentation. This comparison evaluates the resulting compiler's execution,
not the cost of producing it.
The rejected session observed 16.87% fewer instructions with one worker and 16.40% fewer with
default workers. These percentages are not accepted performance claims under the protocol.
The session measured 16.87% fewer instructions with one worker and 16.40% fewer with default
workers. These are instruction-count reductions, not elapsed-time improvement claims.
All 44 runs, including warmups, exited successfully and generated byte-identical C.
The fixed input contains **560,382 parsed physical lines in 555 `.v` files**, with no `.vh` files.
The baseline and candidate report the same source closure and line count. Normalizing the retained
@@ -150,15 +578,17 @@ Median paired instruction differences, candidate minus baseline:
Instruction regressions: **0/10 pairs** in each mode, for both total instructions and
instructions per source line.
Process snapshots recorded unrelated compilations elsewhere on the shared host. The absence of
builds or tests from this task did not satisfy the requirement to measure without any concurrent
builds or tests. Neither the instruction reductions nor the elapsed times from this session
establish an accepted baseline. Preserve the raw data and replace this record with quiet-host
measurements before making performance or regression claims.
Process snapshots recorded unrelated compilations elsewhere on the shared host. Retain them
as context for the elapsed-time and cycle measurements. Compare subsequent compiler changes
against a freshly built baseline in the same measurement session, preserving the fixed fixture
and all raw data.
### Production self-build behavior
On non-Windows hosts, `./v -prod -cc clang self` uses PGO when `llvm-profdata` is available.
The profile merger is searched in Clang's reported installation directory, then beside its
resolved executable, then in `PATH` and through `xcrun` on macOS. This avoids a fallback when
a different LLVM release cannot read the selected compiler's raw profile format.
This spends more time building the production compiler: two C compilations plus training, rather
than one C compilation. Ordinary `./v self` keeps its existing build path. Explicit `-parallel-cc`,
`-no-prod-options`, explicit `-cflags` profile options, unavailable profiling tools, and other C
@@ -170,7 +600,8 @@ counter updates so worker threads do not lose increments. `VJOBS=1` or `-no-para
the serial path. Workspaces are private to each invocation and removed afterward; existing
workspaces and the caller's profile destination are preserved. Profile paths containing spaces
remain a single C compiler argument. The final effective `-cc` selects the PGO toolchain;
command-line options override compiler choices inherited through `VFLAGS`.
command-line options override compiler choices inherited through `VFLAGS`. Split `-cc clang`
and equals-form `-cc=clang` options have the same last-option-wins behavior.
### Reproduction metadata