compiler: reduce type and escape-analysis work, preserve PGO (#30991)
* tools: prefer llvm-profdata from the selected Clang toolchain * docs: distinguish self-build baselines and record matched counts * docs: clarify historical PGO attribution * docs: separate self-build baselines and record matched revision check * types: cache immutable canonical types before recursive interning * types, flat: reuse fingerprints across temporary type wrappers * docs: record completed compiler optimization validation * docs: bisect PGO fallback and self-build instruction regression * tools: honor equals-form compiler options for PGO * types, transform: avoid repeated tagged type work
This commit is contained in:
1 parent
22b23697e8
commit
5f7d4d0980
22 files changed
+844
-103
No files matched your search
@@ -611,6 +611,12 @@ this is the most reliable local measure of compiler work: it is less sensitive t
|
||||
CPU frequency and scheduling. Also retain elapsed time, cycles, and peak memory; fewer instructions
|
||||
do not guarantee lower latency or memory use. Compare counts on the same machine and architecture.
|
||||
|
||||
The instruction counter is scoped to the measured process; unrelated processes do not add their
|
||||
instructions to it. An idle host is not required for instruction comparisons. Counts are dynamic,
|
||||
not a static property of the binary: synchronization, retries, and runtime paths can vary. Use
|
||||
serial paired measurements to check that variation. Record host load as context for elapsed time
|
||||
and cycles; use controlled load when making latency claims.
|
||||
|
||||
For instructions per line, divide by physical lines in the `.v` and `.vh` files actually parsed,
|
||||
including builtins, imports, blank lines, and comments. Collect `parsed .v lines` plus
|
||||
`parsed .vh lines` in a separate unmeasured `-v` run with the same input flags. Preserve that
|
||||
@@ -648,13 +654,16 @@ For a baseline/candidate comparison:
|
||||
|
||||
1. Build both compilers with the same production build recipe and C toolchain. Verify correctness,
|
||||
diagnostics, and generated C before measuring. For a change intended to preserve output, require
|
||||
identical C, allowing only explicitly recorded build metadata differences.
|
||||
identical C, allowing only explicitly recorded build metadata differences. Validate generated
|
||||
C when changing the fixture or compiler representation; matching invalid C is not sufficient.
|
||||
2. Use the same host, OS, fixture, standard library, flags, environment, and filesystem placement.
|
||||
Keep executable, source, and output paths identical between roles where possible; record any
|
||||
remaining differences. Set `VJOBS=1` and `SOURCE_DATE_EPOCH=0`, and disable tracing/profiling.
|
||||
Clear unintended compiler environment overrides, including `VFLAGS` and `VEXE`; record the
|
||||
effective environment. Measure only successfully rebuilt compilers, never a stale binary.
|
||||
Run only one benchmark process at a time, with no concurrent builds or tests.
|
||||
Run only one benchmark process at a time. Other builds or tests on the host do not invalidate
|
||||
process instruction counts; do not pause unrelated work or wait for an idle host. Record that
|
||||
elapsed time and cycles collected under uncontrolled load are contextual measurements.
|
||||
3. Run one warmup per role, then **ten measured pairs**, alternating baseline/candidate order:
|
||||
AB, BA, AB, BA, and so on. Record all 22 runs, excluding both warmups from statistics.
|
||||
Use a distinct timing/stdout/stderr log for every run and the same generated C output path.
|
||||
|
||||
@@ -382,7 +382,11 @@ hello world
|
||||
|
||||
`v self` defaults to `-gc none`. Pass `-gc <mode>` if you need a different GC mode.
|
||||
On non-Windows hosts, `v -prod self` uses profile-guided optimization (PGO) when the selected C
|
||||
compiler is Clang and `llvm-profdata` is available. It builds an instrumented compiler, trains it on
|
||||
compiler is Clang and `llvm-profdata` is available. Clang's reported installation directory and
|
||||
resolved executable directory are searched before `PATH` and macOS `xcrun` for the profile tool,
|
||||
keeping LLVM versions together.
|
||||
Both `-cc clang` and `-cc=clang` select that toolchain; the last compiler option takes precedence.
|
||||
It builds an instrumented compiler, trains it on
|
||||
uncached self-compilation, then builds the optimized compiler. This takes longer to build the
|
||||
production compiler. Other C toolchains and failed PGO attempts use a regular production build.
|
||||
Use `v -prod -parallel-cc self` to skip PGO and compile the generated C units in parallel.
|
||||
|
||||
+47
-25
@@ -456,6 +456,30 @@ fn pgo_compiler_kind(args []string) string {
|
||||
if has_profile_cflag(args) {
|
||||
return ''
|
||||
}
|
||||
ccompiler := self_build_c_compiler(args)
|
||||
cc_file_name := os.file_name(ccompiler)
|
||||
if cc_file_name.contains('clang') || cc_file_name.contains('gcc')
|
||||
|| cc_file_name.contains('g++') || ccompiler == 'cc' {
|
||||
cc_ver := os.exec([ccompiler, '--version']).output
|
||||
if cc_ver.contains('clang') {
|
||||
_ := find_llvm_profdata(ccompiler) or { return '' }
|
||||
return 'clang'
|
||||
}
|
||||
if cc_ver.contains('Free Software Foundation') || cc_ver.contains('GCC') {
|
||||
return 'gcc'
|
||||
}
|
||||
}
|
||||
if cc_file_name.contains('clang') {
|
||||
_ := find_llvm_profdata(ccompiler) or { return '' }
|
||||
return 'clang'
|
||||
}
|
||||
if cc_file_name.contains('gcc') || cc_file_name.contains('g++') {
|
||||
return 'gcc'
|
||||
}
|
||||
return ''
|
||||
}
|
||||
|
||||
fn self_build_c_compiler(args []string) string {
|
||||
// Environment flags precede CLI flags; the driver uses the last -cc.
|
||||
mut ccompiler := ''
|
||||
mut i := 0
|
||||
@@ -463,6 +487,8 @@ fn pgo_compiler_kind(args []string) string {
|
||||
arg := args[i]
|
||||
if arg == '-cc' && i + 1 < args.len {
|
||||
ccompiler = args[i + 1]
|
||||
} else if arg.starts_with('-cc=') {
|
||||
ccompiler = arg['-cc='.len..]
|
||||
}
|
||||
if arg in ['-prof', '-profile'] && i + 1 < args.len && args[i + 1].starts_with('-') {
|
||||
i++
|
||||
@@ -472,32 +498,28 @@ fn pgo_compiler_kind(args []string) string {
|
||||
i++
|
||||
}
|
||||
}
|
||||
if ccompiler == '' {
|
||||
ccompiler = os.getenv_opt('CC') or { 'cc' }
|
||||
}
|
||||
cc_file_name := os.file_name(ccompiler)
|
||||
if cc_file_name.contains('clang') || cc_file_name.contains('gcc')
|
||||
|| cc_file_name.contains('g++') || ccompiler == 'cc' {
|
||||
cc_ver := os.exec([ccompiler, '--version']).output
|
||||
if cc_ver.contains('clang') {
|
||||
_ := find_llvm_profdata() or { return '' }
|
||||
return 'clang'
|
||||
}
|
||||
if cc_ver.contains('Free Software Foundation') || cc_ver.contains('GCC') {
|
||||
return 'gcc'
|
||||
}
|
||||
}
|
||||
if cc_file_name.contains('clang') {
|
||||
_ := find_llvm_profdata() or { return '' }
|
||||
return 'clang'
|
||||
}
|
||||
if cc_file_name.contains('gcc') || cc_file_name.contains('g++') {
|
||||
return 'gcc'
|
||||
}
|
||||
return ''
|
||||
return if ccompiler != '' { ccompiler } else { os.getenv_opt('CC') or { 'cc' } }
|
||||
}
|
||||
|
||||
fn find_llvm_profdata() !string {
|
||||
fn find_llvm_profdata(ccompiler string) !string {
|
||||
// Clang launchers such as /usr/bin/clang can report a different installation
|
||||
// directory. Use that toolchain before looking beside the launcher itself.
|
||||
for line in os.exec([ccompiler, '--version']).output.split_into_lines() {
|
||||
if line.starts_with('InstalledDir:') {
|
||||
profdata := os.join_path(line.all_after(':').trim_space(), 'llvm-profdata')
|
||||
if os.is_file(profdata) && os.is_executable(profdata) {
|
||||
return profdata
|
||||
}
|
||||
}
|
||||
}
|
||||
if compiler := os.find_abs_path_of_executable(ccompiler) {
|
||||
// Different LLVM releases can use incompatible raw profile formats.
|
||||
// Resolve symlinks so a compiler in PATH still finds its own toolchain.
|
||||
profdata := os.join_path(os.dir(os.real_path(compiler)), 'llvm-profdata')
|
||||
if os.is_file(profdata) && os.is_executable(profdata) {
|
||||
return profdata
|
||||
}
|
||||
}
|
||||
if profdata := os.find_abs_path_of_executable('llvm-profdata') {
|
||||
return profdata
|
||||
}
|
||||
@@ -609,7 +631,7 @@ fn compile_with_pgo(vroot string, vexe string, args []string, out_binary string,
|
||||
mut llvm_profdata := ''
|
||||
mut profile_data := ''
|
||||
if cc_kind == 'clang' {
|
||||
llvm_profdata = find_llvm_profdata() or {
|
||||
llvm_profdata = find_llvm_profdata(self_build_c_compiler(effective_self_build_args(args))) or {
|
||||
eprintln('PGO disabled: can not find `llvm-profdata`.')
|
||||
return false
|
||||
}
|
||||
|
||||
@@ -90,6 +90,41 @@ done
|
||||
assert os.ls(root)!.filter(it.starts_with('.vself_pgo_')).len == 0
|
||||
assert os.read_file(caller_profile)! == 'caller profile'
|
||||
|
||||
// An LLVM utility in PATH can belong to an older, incompatible compiler.
|
||||
// Both an explicit compiler path and its symlink must select the sibling tool.
|
||||
compiler_dir := os.join_path(root, 'compiler')
|
||||
os.mkdir(compiler_dir)!
|
||||
os.cp(os.join_path(root, 'bin', 'clang'), os.join_path(compiler_dir, 'clang'))!
|
||||
os.chmod(os.join_path(compiler_dir, 'clang'), 0o755)!
|
||||
os.mv(os.join_path(root, 'bin', 'llvm-profdata'), os.join_path(compiler_dir, 'llvm-profdata'))!
|
||||
os.write_file(os.join_path(root, 'bin', 'llvm-profdata'), '#!/bin/sh\nexit 55\n')!
|
||||
os.chmod(os.join_path(root, 'bin', 'llvm-profdata'), 0o755)!
|
||||
compiler_link := os.join_path(root, 'bin', 'clang-linked')
|
||||
os.symlink(os.join_path(compiler_dir, 'clang'), compiler_link)!
|
||||
compiler_launcher := os.join_path(root, 'bin', 'clang-launcher')
|
||||
os.write_file(compiler_launcher, '#!/bin/sh\nprintf "%s\\n" "clang version mock" ' +
|
||||
os.quoted_path('InstalledDir: ${compiler_dir}') + '\n')!
|
||||
os.chmod(compiler_launcher, 0o755)!
|
||||
for compiler in [os.join_path(compiler_dir, 'clang'), compiler_link, compiler_launcher] {
|
||||
for options in [['-cc', compiler], ['-cc=${compiler}']] {
|
||||
os.write_file(os.join_path(root, 'calls'), '')!
|
||||
selected := pgo_test_run(root, tool, options, false, '2', '')
|
||||
assert selected.exit_code == 0, selected.output
|
||||
assert !selected.output.contains('falling back'), selected.output
|
||||
selected_calls := os.read_file(os.join_path(root, 'calls'))!.split_into_lines()
|
||||
assert selected_calls.filter(it.starts_with('MERGE ')).len == 1, selected_calls.join('\n')
|
||||
assert selected_calls.filter(it.starts_with('BUILD ')).len == 2, selected_calls.join('\n')
|
||||
}
|
||||
}
|
||||
os.rm(os.join_path(root, 'bin', 'llvm-profdata'))!
|
||||
os.mv(os.join_path(compiler_dir, 'llvm-profdata'), os.join_path(root, 'bin', 'llvm-profdata'))!
|
||||
// A compiler without a sibling utility retains the PATH fallback.
|
||||
os.write_file(os.join_path(root, 'calls'), '')!
|
||||
path_fallback := pgo_test_run(root, tool, ['-cc', os.join_path(compiler_dir, 'clang')],
|
||||
false, '2', '')
|
||||
assert path_fallback.exit_code == 0, path_fallback.output
|
||||
assert !path_fallback.output.contains('falling back'), path_fallback.output
|
||||
|
||||
for extra in [['-parallel-cc'], ['-no-prod-options'], ['-cflags', '-fprofile-use=external']] {
|
||||
os.write_file(os.join_path(root, 'calls'), '')!
|
||||
plain := pgo_test_run(root, tool, extra, false, '2', '')
|
||||
@@ -105,6 +140,8 @@ done
|
||||
extra := if compiler == 'clang' { []string{} } else { ['-cc', '${root}/bin/gcc'] }
|
||||
for case_index, options in [extra,
|
||||
['-cc', '${root}/bin/gcc', '-cc', '${root}/bin/${compiler}'],
|
||||
['-cc', '${root}/bin/${other}', '-cc=${root}/bin/${compiler}'],
|
||||
['-cc=${root}/bin/${other}', '-cc', '${root}/bin/${compiler}'],
|
||||
['-profile', '-cc', '${root}/bin/${compiler}']] {
|
||||
os.write_file(os.join_path(root, 'calls'), '')!
|
||||
inherited_flags := if case_index == 0 { flags } else { '' }
|
||||
|
||||
@@ -257,10 +257,13 @@ pub fn source_type_id(text string) u32 {
|
||||
return id
|
||||
}
|
||||
|
||||
@[inline]
|
||||
@[direct_array_access; inline]
|
||||
fn source_type_recent_slot(text string) int {
|
||||
address := usize(text.str)
|
||||
return int(((address >> 3) ^ (address >> 13) ^ (address >> 21)) & 4095)
|
||||
// Equal spellings share a slot even when callers borrow different buffers.
|
||||
// The complete canonical spelling is still checked before accepting a hit.
|
||||
key := u32(text.len) ^ (u32(text[0]) << 1) ^ (u32(text[text.len >> 1]) << 5) ^
|
||||
(u32(text[text.len - 1]) << 9)
|
||||
return int((key ^ (key >> 12)) & 4095)
|
||||
}
|
||||
|
||||
fn source_type_id_uncached(text string) u32 {
|
||||
|
||||
@@ -2,6 +2,19 @@ module flat
|
||||
|
||||
import hash
|
||||
|
||||
fn test_source_type_recent_cache_checks_borrowed_spelling_after_mutation() {
|
||||
mut bytes := 'source_types.MutableBorrowedInput'.bytes()
|
||||
text := unsafe { tos(bytes.data, bytes.len) }
|
||||
id := source_type_id(text)
|
||||
assert source_type_name(id) == 'source_types.MutableBorrowedInput'
|
||||
bytes[1] = `X`
|
||||
changed_id := source_type_id(text)
|
||||
assert changed_id != id
|
||||
assert source_type_name(changed_id) == 'sXurce_types.MutableBorrowedInput'
|
||||
assert source_type_name(id) == 'source_types.MutableBorrowedInput'
|
||||
assert source_type_id('source_types.MutableBorrowedInput'.clone()) == id
|
||||
}
|
||||
|
||||
fn test_existing_source_type_does_not_take_the_insertion_lock() {
|
||||
text := 'source_types.Existing'
|
||||
id := source_type_id(text)
|
||||
|
||||
@@ -690,3 +690,41 @@ fn test_fixed_array_reference_sums_own_inline_variant_storage() {
|
||||
seen.clear()
|
||||
assert t.escape_value_contains_fixed_array(types.Type(types.Struct{ name_id: flat.source_type_id('FixedPayload') }), mut seen)
|
||||
}
|
||||
|
||||
fn test_fixed_array_reference_nominal_cache_keeps_complete_walk_results() {
|
||||
mut a := flat.FlatAst.new()
|
||||
mut tc := types.TypeChecker.new(&a)
|
||||
fixed := types.Type(types.ArrayFixed{ elem_type: types.Type(types.int_), len: 2 })
|
||||
tc.structs['FixedPayload'] = [types.StructField{ name: 'values', typ: fixed }]
|
||||
tc.sum_types.set('Empty', ['int', 'string'])
|
||||
tc.sum_types.set('Payload', ['Other', 'FixedPayload'])
|
||||
tc.sum_types.set('Other', ['Payload', 'int'])
|
||||
tc.sum_types.set('Generic', ['int', 'T'])
|
||||
tc.sum_generic_params['Generic'] = ['T']
|
||||
mut t := new_transformer(mut a, &tc, map[string]bool{})
|
||||
for _ in 0 .. 2 {
|
||||
for name in ['Payload', 'Other', 'Generic[FixedPayload]', 'Generic[int]', 'Empty'] {
|
||||
typ := types.Type(types.SumType{ name_id: flat.source_type_id(name) })
|
||||
expected := name in ['Payload', 'Other', 'Generic[FixedPayload]']
|
||||
assert t.cached_escape_value_contains_fixed_array(typ) == expected, name
|
||||
assert t.escape_fixed_array_nominal_cache[typ.source_type_id()] == expected, name
|
||||
}
|
||||
}
|
||||
// Other's first nested walk meets Payload in the active set. Only its later,
|
||||
// complete root walk can be cached: it reaches FixedPayload through Payload.
|
||||
assert t.escape_fixed_array_nominal_cache[flat.source_type_id('Other')]
|
||||
}
|
||||
|
||||
fn test_fixed_array_reference_nominal_cache_caches_complete_negative_component() {
|
||||
mut a := flat.FlatAst.new()
|
||||
mut tc := types.TypeChecker.new(&a)
|
||||
tc.sum_types.set('Left', ['Right', 'int'])
|
||||
tc.sum_types.set('Right', ['Left', 'string'])
|
||||
mut t := new_transformer(mut a, &tc, map[string]bool{})
|
||||
left := types.Type(types.SumType{ name_id: flat.source_type_id('Left') })
|
||||
right := types.Type(types.SumType{ name_id: flat.source_type_id('Right') })
|
||||
assert !t.cached_escape_value_contains_fixed_array(left)
|
||||
cached := t.escape_fixed_array_nominal_cache[right.source_type_id()] or { panic('missing complete result') }
|
||||
assert !cached
|
||||
assert !t.cached_escape_value_contains_fixed_array(right)
|
||||
}
|
||||
@@ -1375,8 +1375,7 @@ fn (mut t Transformer) lower_indexed_for_in(id flat.NodeId, node flat.Node, key_
|
||||
t.set_node_typ(int(container), flat.source_type_id(actual_iter_type))
|
||||
} else if elem_needs_ref && actual_iter_type.starts_with('[]') && !isnil(t.tc)
|
||||
&& (elem_name in t.escaping_amp_sources || elem_name in t.escaping_fixed_array_view_sources) {
|
||||
mut seen := map[u32]bool{}
|
||||
if t.escape_value_contains_fixed_array(t.tc.parse_type(elem_type), mut seen) {
|
||||
if t.cached_escape_value_contains_fixed_array(t.tc.parse_type(elem_type)) {
|
||||
// The binding hides its index from argument lowering. Retain each current
|
||||
// buffer before exposing a row, including after growth between iterations.
|
||||
t.mark_fn_used('array.retain_fixed_array_buffer')
|
||||
|
||||
@@ -482,9 +482,9 @@ mut:
|
||||
// parameter that can retain an inline fixed array as a dynamic view. Promote the root
|
||||
// before any aliases form, including when a containing struct is forwarded by reference.
|
||||
escaping_fixed_array_view_sources map[string]bool
|
||||
// escape_fixed_array_struct_cache memoizes, per struct type, whether its inline
|
||||
// storage holds a fixed array. The escape precheck asks for every reference argument.
|
||||
escape_fixed_array_struct_cache map[u32]bool
|
||||
// Cache complete walks of nominal inline storage. Sum variants can contain fixed
|
||||
// arrays too; repeated reference arguments must not walk the same variants again.
|
||||
escape_fixed_array_nominal_cache map[u32]bool
|
||||
// skip_owned_base_nodes_without_regions: see PreparedSelfhostTransform.keeps_scope_without_regions.
|
||||
skip_owned_base_nodes_without_regions bool
|
||||
// heaped_amp_locals records which of those sources were actually moved to the heap, so
|
||||
@@ -8592,13 +8592,24 @@ fn (mut t Transformer) escape_arg_value_contains_fixed_array(arg_id flat.NodeId)
|
||||
for value_type is types.Pointer {
|
||||
value_type = types.unalias_type(value_type.base_type)
|
||||
}
|
||||
if value_type is types.Struct {
|
||||
if cached := t.escape_fixed_array_struct_cache[value_type.source_type_id()] {
|
||||
return t.cached_escape_value_contains_fixed_array(value_type)
|
||||
}
|
||||
|
||||
fn (mut t Transformer) cached_escape_value_contains_fixed_array(value_type &types.Type) bool {
|
||||
if value_type is types.Struct || value_type is types.SumType {
|
||||
if cached := t.escape_fixed_array_nominal_cache[value_type.source_type_id()] {
|
||||
return cached
|
||||
}
|
||||
mut seen := map[u32]bool{}
|
||||
contains := t.escape_value_contains_fixed_array(value_type, mut seen)
|
||||
t.escape_fixed_array_struct_cache[value_type.source_type_id()] = contains
|
||||
if !contains {
|
||||
// A complete unsuccessful root walk proves every visited nominal type
|
||||
// lacks inline fixed arrays, including nodes in recursive components.
|
||||
for name_id, _ in seen {
|
||||
t.escape_fixed_array_nominal_cache[name_id] = false
|
||||
}
|
||||
}
|
||||
t.escape_fixed_array_nominal_cache[value_type.source_type_id()] = contains
|
||||
return contains
|
||||
}
|
||||
mut seen := map[u32]bool{}
|
||||
@@ -11051,7 +11062,7 @@ fn escape_type_is_struct_value(typ types.Type) bool {
|
||||
// escape_value_contains_fixed_array follows inline storage only. A dynamic container or
|
||||
// pointer field already has separate backing storage, so moving its containing value does
|
||||
// not extend the lifetime of that backing.
|
||||
fn (t &Transformer) escape_value_contains_fixed_array(typ types.Type, mut seen map[u32]bool) bool {
|
||||
fn (mut t Transformer) escape_value_contains_fixed_array(typ types.Type, mut seen map[u32]bool) bool {
|
||||
match typ {
|
||||
types.ArrayFixed {
|
||||
return true
|
||||
@@ -11060,6 +11071,7 @@ fn (t &Transformer) escape_value_contains_fixed_array(typ types.Type, mut seen m
|
||||
return t.escape_value_contains_fixed_array(typ.base_type, mut seen)
|
||||
}
|
||||
types.SumType {
|
||||
if cached := t.escape_fixed_array_nominal_cache[typ.name_id] { return cached }
|
||||
if isnil(t.tc) || typ.name_id in seen {
|
||||
return false
|
||||
}
|
||||
@@ -11067,11 +11079,13 @@ fn (t &Transformer) escape_value_contains_fixed_array(typ types.Type, mut seen m
|
||||
for variant in t.concrete_sum_variants_for_candidate(typ.name_id) {
|
||||
value := t.tc.parse_type(variant)
|
||||
if t.escape_value_contains_fixed_array(*value, mut seen) {
|
||||
t.escape_fixed_array_nominal_cache[typ.name_id] = true
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
types.Struct {
|
||||
if cached := t.escape_fixed_array_nominal_cache[typ.name_id] { return cached }
|
||||
if isnil(t.tc) || typ.source_type_id() in seen {
|
||||
return false
|
||||
}
|
||||
@@ -11091,6 +11105,7 @@ fn (t &Transformer) escape_value_contains_fixed_array(typ types.Type, mut seen m
|
||||
field.typ
|
||||
}
|
||||
if t.escape_value_contains_fixed_array(field_type, mut seen) {
|
||||
t.escape_fixed_array_nominal_cache[typ.name_id] = true
|
||||
return true
|
||||
}
|
||||
}
|
||||
@@ -11269,8 +11284,7 @@ fn (mut t Transformer) mark_fixed_array_reference_argument_escape(arg_id flat.No
|
||||
for value_type is types.Pointer {
|
||||
value_type = types.unalias_type(value_type.base_type)
|
||||
}
|
||||
mut seen := map[u32]bool{}
|
||||
if !t.escape_value_contains_fixed_array(value_type, mut seen)
|
||||
if !t.cached_escape_value_contains_fixed_array(value_type)
|
||||
|| !t.escape_address_expr_is_stack_local(value_id, local_stack_names, amp_ptrs, ptr_aliases) {
|
||||
return
|
||||
}
|
||||
|
||||
+21
-10
@@ -10583,8 +10583,8 @@ pub fn (tc &TypeChecker) expr_type(id flat.NodeId) ?Type {
|
||||
return *tc.parse_type(node.typ)
|
||||
}
|
||||
}
|
||||
if t := tc.cached_expr_type(id) {
|
||||
return t
|
||||
if t := tc.cached_expr_type_ref(id) {
|
||||
return *t
|
||||
}
|
||||
return none
|
||||
}
|
||||
@@ -10610,9 +10610,9 @@ fn (tc &TypeChecker) resolved_call_type(id flat.NodeId) ?Type {
|
||||
}
|
||||
}
|
||||
}
|
||||
if t := tc.cached_expr_type(id) {
|
||||
if t := tc.cached_expr_type_ref(id) {
|
||||
if t !is Void {
|
||||
return t
|
||||
return *t
|
||||
}
|
||||
}
|
||||
if name := tc.cached_resolved_call(id) {
|
||||
@@ -10660,27 +10660,38 @@ pub fn (tc &TypeChecker) invalidate_checked_expr_type(idx int) {
|
||||
}
|
||||
}
|
||||
|
||||
@[direct_array_access]
|
||||
@[inline]
|
||||
fn (tc &TypeChecker) cached_expr_type(id flat.NodeId) ?Type {
|
||||
if typ := tc.cached_expr_type_ref(id) {
|
||||
return *typ
|
||||
}
|
||||
return none
|
||||
}
|
||||
|
||||
// Dense slots point at immutable canonical types. Sparse map values need stable
|
||||
// copies because later insertions can move the map backing storage.
|
||||
@[direct_array_access]
|
||||
fn (tc &TypeChecker) cached_expr_type_ref(id flat.NodeId) ?&Type {
|
||||
idx := int(id)
|
||||
if !isnil(tc.storage_query_probe) {
|
||||
if typ := tc.sparse_expr_type_values[idx] { return typ }
|
||||
if typ := tc.sparse_expr_type_values[idx] { return new_type_reference(typ) }
|
||||
if !isnil(tc.storage_query_probe.read_base) {
|
||||
return tc.storage_query_probe.read_base.cached_expr_type(id)
|
||||
return tc.storage_query_probe.read_base.cached_expr_type_ref(id)
|
||||
}
|
||||
return none
|
||||
}
|
||||
if tc.parallel_check_sparse {
|
||||
if tc.in_check_range(idx) {
|
||||
if idx < tc.expr_type_set.len && tc.expr_type_set[idx] {
|
||||
return *tc.expr_type_values[idx]
|
||||
return tc.expr_type_values[idx]
|
||||
}
|
||||
return none
|
||||
}
|
||||
return tc.sparse_expr_type_values[idx] or { none }
|
||||
if typ := tc.sparse_expr_type_values[idx] { return new_type_reference(typ) }
|
||||
return none
|
||||
}
|
||||
if idx >= 0 && idx < tc.expr_type_set.len && tc.expr_type_set[idx] {
|
||||
return *tc.expr_type_values[idx]
|
||||
return tc.expr_type_values[idx]
|
||||
}
|
||||
return none
|
||||
}
|
||||
|
||||
@@ -17737,9 +17737,9 @@ fn (mut tc TypeChecker) restore_return_alias_for_in_bindings(node flat.Node) {
|
||||
|
||||
fn (mut tc TypeChecker) return_alias_decl_binding_type(node flat.Node, index int) Type {
|
||||
lhs_id := tc.multi_assign_lhs_id(node, index)
|
||||
if typ := tc.cached_expr_type(lhs_id) {
|
||||
if typ := tc.cached_expr_type_ref(lhs_id) {
|
||||
if typ !is Unknown && typ !is Void {
|
||||
return typ
|
||||
return *typ
|
||||
}
|
||||
}
|
||||
lhs_count := tc.multi_assign_lhs_count(node)
|
||||
@@ -20906,7 +20906,7 @@ fn (mut tc TypeChecker) pointer_binding_state_after_loop_post(loop_node flat.Nod
|
||||
if post.kind == .postfix && post.op in [.inc, .dec] && post.children_count == 1 {
|
||||
target_id := tc.a.child(post, 0)
|
||||
if tc.a.node(target_id).kind == .ident {
|
||||
if target_type := tc.cached_expr_type(target_id) {
|
||||
if target_type := tc.cached_expr_type_ref(target_id) {
|
||||
// Rechecking a scalar counter cannot change pointer identities. The
|
||||
// post has already been checked for diagnostics above the loop body.
|
||||
if unalias_type(target_type) is Primitive && precomputed == base {
|
||||
|
||||
@@ -15347,7 +15347,7 @@ fn (mut tc TypeChecker) check_valid_call_arg_types(id flat.NodeId, node flat.Nod
|
||||
} else {
|
||||
tc.check_node(check_arg_id)
|
||||
}
|
||||
if multi := tc.cached_expr_type(check_arg_id) {
|
||||
if multi := tc.cached_expr_type_ref(check_arg_id) {
|
||||
if !info.is_variadic && multi is MultiReturn {
|
||||
expanded_arg_offset += multi.types.len - 1
|
||||
}
|
||||
@@ -21216,8 +21216,8 @@ fn (tc &TypeChecker) selector_fn_base_type(base_id flat.NodeId) ?Type {
|
||||
}
|
||||
}
|
||||
}
|
||||
if typ := tc.cached_expr_type(base_id) {
|
||||
return typ
|
||||
if typ := tc.cached_expr_type_ref(base_id) {
|
||||
return *typ
|
||||
}
|
||||
if int(base_id) < 0 {
|
||||
return none
|
||||
|
||||
@@ -10509,9 +10509,6 @@ fn qualified_type_modifiers(name string) string {
|
||||
fn (tc &TypeChecker) type_compatible(actual &Type, expected &Type) bool {
|
||||
actual_raw := actual
|
||||
expected_raw := expected
|
||||
if semantic_types_equal(actual, expected) {
|
||||
return true
|
||||
}
|
||||
// Equal semantic values have equal spellings even when their immutable
|
||||
// payloads were allocated separately. Compare them before interning names.
|
||||
if semantic_types_equal(actual, expected) {
|
||||
@@ -11079,9 +11076,6 @@ pub fn (tc &TypeChecker) fn_param_compatible(actual &Type, expected &Type) bool
|
||||
if semantic_types_equal(actual, expected) {
|
||||
return true
|
||||
}
|
||||
if semantic_types_equal(actual, expected) {
|
||||
return true
|
||||
}
|
||||
// V's platform `int` and a fixed-width integer such as `i64`/`i32` may share an
|
||||
// emitted C spelling on a given target, but they remain distinct, target-
|
||||
// independent source types. Function-parameter identity must not depend on the C
|
||||
@@ -11149,9 +11143,6 @@ fn fn_param_is_platform_int(typ &Type) bool {
|
||||
}
|
||||
|
||||
fn (tc &TypeChecker) fn_return_compatible(actual &Type, expected &Type) bool {
|
||||
if semantic_types_equal(actual, expected) {
|
||||
return true
|
||||
}
|
||||
if semantic_types_equal(actual, expected) {
|
||||
return true
|
||||
}
|
||||
@@ -17082,10 +17073,10 @@ pub fn (tc &TypeChecker) type_name(t &Type) u32 {
|
||||
return interner.source_type_id(id)
|
||||
}
|
||||
|
||||
// type_recent_hash_slot chooses a recent-cache slot from semantic identity.
|
||||
// type_recent_hash_slot uses a cheap fingerprint; callers verify structural equality.
|
||||
@[inline]
|
||||
fn type_recent_hash_slot(typ &Type) (u64, int) {
|
||||
hash := semantic_type_hash(typ)
|
||||
hash := canonical_intern_cache_key(typ)
|
||||
return hash, int(hash & u64(type_cache_recent_slots - 1))
|
||||
}
|
||||
|
||||
@@ -18814,7 +18805,7 @@ fn (mut memo BodyResolveMemo) begin(lo int, hi int) {
|
||||
pub fn (tc &TypeChecker) resolve_type(id flat.NodeId) Type {
|
||||
if tc.trust_checked_expr_types {
|
||||
if !isnil(tc.storage_query_probe) {
|
||||
if typ := tc.cached_expr_type(id) {
|
||||
if typ := tc.cached_expr_type_ref(id) {
|
||||
if !type_contains_unknown(typ) { return tc.widen_mixed_integer_expr_type(id, typ) }
|
||||
}
|
||||
}
|
||||
@@ -19076,8 +19067,8 @@ fn (tc &TypeChecker) resolve_type_uncached(id flat.NodeId) Type {
|
||||
// type its check gave it (see check_generic_method_value), which the
|
||||
// resolution of an index cannot tell from an index of the method value.
|
||||
if node.children_count > 0 && tc.a.child_node(node, 0).kind == .selector {
|
||||
if typ := tc.cached_expr_type(id) {
|
||||
typ_raw := typ
|
||||
if typ := tc.cached_expr_type_ref(id) {
|
||||
typ_raw := *typ
|
||||
if typ is FnType {
|
||||
return typ_raw
|
||||
}
|
||||
@@ -19089,12 +19080,12 @@ fn (tc &TypeChecker) resolve_type_uncached(id flat.NodeId) Type {
|
||||
return builtin_bool_type
|
||||
}
|
||||
if !(tc.smartcasts.len > 0 && (kind_id == 7 || kind_id == 13)) {
|
||||
if typ := tc.cached_expr_type(id) {
|
||||
if typ := tc.cached_expr_type_ref(id) {
|
||||
// A resolution-only annotation can cache `void` for a method whose
|
||||
// receiver is a local alias that is only known during body checking.
|
||||
// Re-resolve calls in that case instead of hiding the real return type.
|
||||
if node.kind != .call || typ !is Void {
|
||||
return typ
|
||||
return *typ
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1013,7 +1013,7 @@ fn (tc &TypeChecker) incremental_touches_generics(f IncrementalFunction, generic
|
||||
return true
|
||||
}
|
||||
}
|
||||
if typ := tc.cached_expr_type(id) {
|
||||
if typ := tc.cached_expr_type_ref(id) {
|
||||
if type_mentions_generic_application(typ) {
|
||||
return true
|
||||
}
|
||||
|
||||
@@ -3,6 +3,27 @@ module types
|
||||
|
||||
import v.flat
|
||||
|
||||
fn test_repeated_scalar_interning_after_scope_promotion() {
|
||||
$if prealloc {
|
||||
mut interner := new_type_interner()
|
||||
input := Type(int_)
|
||||
scope := unsafe { prealloc_scope_begin() }
|
||||
id, scoped := interner.canonicalize(input)
|
||||
assert unsafe { prealloc_scope_owns(scope, scoped) }
|
||||
unsafe { prealloc_scope_leave(scope) }
|
||||
interner.promote_from(0, scope)
|
||||
promoted_id, promoted := interner.canonicalize(input)
|
||||
assert promoted_id == id
|
||||
assert voidptr(promoted) == voidptr(interner.types[int(id)])
|
||||
assert !unsafe { prealloc_scope_owns(scope, promoted) }
|
||||
unsafe { prealloc_scope_free_after(scope) }
|
||||
again_id, again := interner.canonicalize(input)
|
||||
assert again_id == id
|
||||
assert voidptr(again) == voidptr(promoted)
|
||||
assert semantic_types_equal(again, input)
|
||||
}
|
||||
}
|
||||
|
||||
fn test_interner_owns_edges_from_a_temporary_arena() {
|
||||
$if prealloc {
|
||||
mut interner := new_type_interner()
|
||||
|
||||
+1
-1
@@ -682,7 +682,7 @@ pub fn (value SumType) name() string {
|
||||
}
|
||||
|
||||
// source_type_id returns an interned spelling, reusing nominal type identities directly.
|
||||
pub fn (t Type) source_type_id() u32 {
|
||||
pub fn (t &Type) source_type_id() u32 {
|
||||
return match t {
|
||||
Struct, Alias, Enum, Interface, SumType { t.name_id }
|
||||
Void { flat.source_type_void }
|
||||
|
||||
@@ -928,3 +928,37 @@ fn test_new_type_reference_owns_its_descriptor_copy() {
|
||||
assert reference is Primitive
|
||||
assert reference.size == 16
|
||||
}
|
||||
|
||||
fn test_cached_expr_type_references_preserve_storage_and_visibility() {
|
||||
ast := flat.FlatAst.new()
|
||||
mut tc := TypeChecker.new(&ast)
|
||||
tc.reset_node_caches(8)
|
||||
tc.remember_expr_type(flat.NodeId(1), Type(int_))
|
||||
tc.remember_expr_type(flat.NodeId(5), Type(string_))
|
||||
dense := tc.cached_expr_type_ref(flat.NodeId(1)) or { panic('missing dense type') }
|
||||
assert voidptr(dense) == voidptr(tc.expr_type_values[1])
|
||||
tc.parallel_check_sparse = true
|
||||
tc.check_range_lo = 0
|
||||
tc.check_range_hi = 4
|
||||
assert tc.cached_expr_type_ref(flat.NodeId(5)) == none
|
||||
tc.sparse_expr_type_values[5] = Type(bool_)
|
||||
sparse := tc.cached_expr_type_ref(flat.NodeId(5)) or { panic('missing sparse type') }
|
||||
assert *sparse == Type(bool_)
|
||||
for i in 8 .. 4096 {
|
||||
tc.sparse_expr_type_values[i] = Type(int_)
|
||||
}
|
||||
tc.sparse_expr_type_values[5] = Type(string_)
|
||||
assert *sparse == Type(bool_)
|
||||
assert *dense == Type(int_)
|
||||
assert tc.cached_expr_type(flat.NodeId(5))? == Type(string_)
|
||||
|
||||
mut probe := tc.fork_storage_observation_view()
|
||||
assert *probe.cached_expr_type_ref(flat.NodeId(1))? == Type(int_)
|
||||
assert *probe.cached_expr_type_ref(flat.NodeId(5))? == Type(string_)
|
||||
probe.sparse_expr_type_values[5] = Type(u64_)
|
||||
assert *probe.cached_expr_type_ref(flat.NodeId(5))? == Type(u64_)
|
||||
assert *tc.cached_expr_type_ref(flat.NodeId(5))? == Type(string_)
|
||||
tc.invalidate_checked_expr_type(1)
|
||||
assert tc.cached_expr_type_ref(flat.NodeId(1)) == none
|
||||
assert *dense == Type(int_)
|
||||
}
|
||||
@@ -8,6 +8,11 @@ import v.flat
|
||||
// caches and equality-heavy internals can use this compact identity.
|
||||
pub type TypeId = u32
|
||||
|
||||
struct CanonicalTypeEntry {
|
||||
id TypeId
|
||||
typ &Type
|
||||
}
|
||||
|
||||
@[heap]
|
||||
struct TypeInterner {
|
||||
mut:
|
||||
@@ -15,6 +20,8 @@ mut:
|
||||
types []&Type
|
||||
names []u32
|
||||
buckets map[u64]TypeId
|
||||
entries []&CanonicalTypeEntry
|
||||
recent [4096]voidptr
|
||||
}
|
||||
|
||||
fn new_type_interner() &TypeInterner {
|
||||
@@ -48,7 +55,9 @@ fn (mut i TypeInterner) intern_locked(t &Type, hash u64) (TypeId, &Type) {
|
||||
key = type_hash_tag(key, 0x5bd1e995)
|
||||
}
|
||||
id := TypeId(i.types.len)
|
||||
i.types << new_type_reference(Type(owned))
|
||||
canonical := new_type_reference(Type(owned))
|
||||
i.types << canonical
|
||||
i.entries << &CanonicalTypeEntry{ id: id, typ: canonical }
|
||||
i.names << u32(0)
|
||||
i.buckets[key] = id
|
||||
return id, i.types[int(id)]
|
||||
@@ -177,12 +186,17 @@ fn (mut i TypeInterner) source_type_id(id TypeId) u32 {
|
||||
}
|
||||
|
||||
fn (mut i TypeInterner) canonicalize(t &Type) (TypeId, &Type) {
|
||||
if entry := i.recent_canonical_type(t) {
|
||||
return entry.id, entry.typ
|
||||
}
|
||||
hash := semantic_type_hash(t)
|
||||
i.lock.lock()
|
||||
defer {
|
||||
i.lock.unlock()
|
||||
}
|
||||
return i.intern_locked(t, hash)
|
||||
id, canonical := i.intern_locked(t, hash)
|
||||
i.remember_canonical_type(t, i.entries[int(id)])
|
||||
return id, canonical
|
||||
}
|
||||
|
||||
fn (mut i TypeInterner) len() int {
|
||||
@@ -202,6 +216,7 @@ fn (mut i TypeInterner) reserve(headroom int) {
|
||||
unsafe {
|
||||
i.types.grow_cap(headroom)
|
||||
i.names.grow_cap(headroom)
|
||||
i.entries.grow_cap(headroom)
|
||||
}
|
||||
i.buckets.reserve(u32(i.buckets.len + headroom))
|
||||
}
|
||||
@@ -217,10 +232,15 @@ fn (mut i TypeInterner) promote_from(start int, scope voidptr) {
|
||||
// not only additions owned by the outer transform scope. Deep-copy the full
|
||||
// stable-id table before any of those arenas are released.
|
||||
previous := i.types
|
||||
// All readers have joined before promotion replaces their canonical pointers.
|
||||
i.clear_recent_canonical_types()
|
||||
i.types = []&Type{cap: previous.len}
|
||||
for typ in previous {
|
||||
i.entries = []&CanonicalTypeEntry{cap: previous.len}
|
||||
for idx, typ in previous {
|
||||
owned := i.own_value(typ)
|
||||
i.types << new_type_reference(Type(owned))
|
||||
canonical := new_type_reference(Type(owned))
|
||||
i.types << canonical
|
||||
i.entries << &CanonicalTypeEntry{ id: TypeId(idx), typ: canonical }
|
||||
}
|
||||
i.names = i.names.clone()
|
||||
// A scoped insertion can rehash even after the caller reserves headroom.
|
||||
|
||||
@@ -0,0 +1,87 @@
|
||||
module types
|
||||
|
||||
// Entries are immutable after atomic publication. Their storage follows the
|
||||
// canonical table; promotion clears the cache before releasing a stage arena.
|
||||
@[direct_array_access; inline]
|
||||
fn (i &TypeInterner) recent_canonical_type(t &Type) ?&CanonicalTypeEntry {
|
||||
slot := int(canonical_intern_cache_key(t) & 4095)
|
||||
entry := unsafe { &CanonicalTypeEntry(C.atomic_load_ptr(voidptr(&i.recent[slot]))) }
|
||||
if !isnil(entry) && semantic_types_equal(entry.typ, t) {
|
||||
return entry
|
||||
}
|
||||
return none
|
||||
}
|
||||
|
||||
@[direct_array_access; inline]
|
||||
fn (i &TypeInterner) remember_canonical_type(t &Type, entry &CanonicalTypeEntry) {
|
||||
slot := int(canonical_intern_cache_key(t) & 4095)
|
||||
C.atomic_store_ptr(voidptr(&i.recent[slot]), entry)
|
||||
}
|
||||
|
||||
fn (i &TypeInterner) clear_recent_canonical_types() {
|
||||
for slot in 0 .. i.recent.len {
|
||||
C.atomic_store_ptr(voidptr(&i.recent[slot]), unsafe { nil })
|
||||
}
|
||||
}
|
||||
|
||||
// This forms a cache key without recursively hashing children. The owned entry is
|
||||
// still compared structurally, so borrowed or mutated inputs cannot produce a hit
|
||||
// merely because their addresses, lengths, or nominal spellings match.
|
||||
@[inline]
|
||||
fn canonical_intern_cache_key(t &Type) u64 {
|
||||
mut key := match t {
|
||||
Void { u64(1) }
|
||||
Unknown { u64(2) ^ (u64(t.reason.len) << 8) }
|
||||
Primitive { u64(3) ^ (u64(t.props) << 16) ^ (u64(t.size) << 8) }
|
||||
String { u64(4) }
|
||||
Char { u64(5) }
|
||||
Rune { u64(6) }
|
||||
ISize { u64(7) }
|
||||
USize { u64(8) }
|
||||
Nil { u64(9) }
|
||||
None { u64(10) }
|
||||
Array { u64(11) ^ canonical_intern_child_key(t.elem_type) }
|
||||
ArrayFixed { u64(12) ^ canonical_intern_child_key(t.elem_type) ^ (u64(t.len) << 8) }
|
||||
Channel { u64(13) ^ canonical_intern_child_key(t.elem_type) ^ (u64(t.is_mut) << 8) }
|
||||
Map {
|
||||
u64(14) ^ canonical_intern_child_key(t.key_type) ^ (canonical_intern_child_key(t.value_type) << 7)
|
||||
}
|
||||
Pointer { u64(15) ^ canonical_intern_child_key(t.base_type) }
|
||||
FnType {
|
||||
u64(16) ^ (u64(t.params.data) >> 3) ^ (u64(t.params.len) << 8) ^
|
||||
(u64(voidptr(t.return_type)) << 7) ^ u64(t.is_variadic)
|
||||
}
|
||||
OptionType { u64(17) ^ canonical_intern_child_key(t.base_type) }
|
||||
ResultType { u64(18) ^ canonical_intern_child_key(t.base_type) }
|
||||
Struct { u64(19) ^ (u64(t.name_id) << 8) }
|
||||
Interface { u64(20) ^ (u64(t.name_id) << 8) }
|
||||
Enum { u64(21) ^ (u64(t.name_id) << 8) ^ u64(t.is_flag) }
|
||||
SumType { u64(22) ^ (u64(t.name_id) << 8) }
|
||||
Alias { u64(23) ^ (u64(t.name_id) << 8) ^ canonical_intern_child_key(t.base_type) }
|
||||
MultiReturn { u64(24) ^ (u64(t.types.data) >> 3) ^ (u64(t.types.len) << 8) }
|
||||
}
|
||||
key = (key ^ (key >> 32)) * u64(0x9e3779b97f4a7c15)
|
||||
return key ^ (key >> 32)
|
||||
}
|
||||
|
||||
// Scalar and nominal children have cheap content identities even when their
|
||||
// temporary Type wrappers occupy different addresses. Other children stay shallow.
|
||||
@[inline]
|
||||
fn canonical_intern_child_key(t &Type) u64 {
|
||||
return match t {
|
||||
Primitive { u64(3) ^ (u64(t.props) << 16) ^ (u64(t.size) << 8) }
|
||||
Void { u64(1) }
|
||||
String { u64(4) }
|
||||
Char { u64(5) }
|
||||
Rune { u64(6) }
|
||||
ISize { u64(7) }
|
||||
USize { u64(8) }
|
||||
Nil { u64(9) }
|
||||
None { u64(10) }
|
||||
Struct { u64(19) ^ (u64(t.name_id) << 8) }
|
||||
Interface { u64(20) ^ (u64(t.name_id) << 8) }
|
||||
Enum { u64(21) ^ (u64(t.name_id) << 8) ^ u64(t.is_flag) }
|
||||
SumType { u64(22) ^ (u64(t.name_id) << 8) }
|
||||
else { u64(voidptr(t)) >> 3 }
|
||||
}
|
||||
}
|
||||
@@ -8,9 +8,14 @@ fn test_interner_owns_nested_function_parameters() {
|
||||
return_type: &Type(string_)
|
||||
})
|
||||
id, canonical := interner.canonicalize(original)
|
||||
canonical_count := interner.len()
|
||||
params[0] = FnParam{ typ: &Type(bool_), is_mut: true }
|
||||
assert canonical.name() == 'fn(int) string', canonical.name()
|
||||
changed_id, changed := interner.canonicalize(original)
|
||||
assert changed_id != id
|
||||
assert changed is FnType
|
||||
assert changed.params[0].is_mut
|
||||
assert semantic_types_equal(changed.params[0].typ, Type(bool_))
|
||||
canonical_count := interner.len()
|
||||
stable := Type(FnType{
|
||||
params: [FnParam{ typ: &Type(int_) }]
|
||||
return_type: &Type(string_)
|
||||
|
||||
@@ -13,7 +13,7 @@ fn interner_parallel_writer(mut interner TypeInterner, input []Type, ready chan
|
||||
}
|
||||
}
|
||||
|
||||
fn interner_parallel_reader(interner &TypeInterner, ready chan bool, start chan bool, stop chan bool) {
|
||||
fn interner_parallel_reader(mut interner TypeInterner, ready chan bool, start chan bool, stop chan bool) {
|
||||
ready <- true
|
||||
_ := <-start
|
||||
// Keep probing until every writer has finished growing the table.
|
||||
@@ -23,6 +23,8 @@ fn interner_parallel_reader(interner &TypeInterner, ready chan bool, start chan
|
||||
return
|
||||
}
|
||||
assert canonical is String
|
||||
_, repeated := interner.canonicalize(Type(string_))
|
||||
assert voidptr(repeated) == voidptr(canonical)
|
||||
select {
|
||||
_ := <-stop {
|
||||
return
|
||||
@@ -60,7 +62,7 @@ fn test_interner_probe_during_parallel_growth() {
|
||||
for lane in 0 .. lanes {
|
||||
writers << spawn interner_parallel_writer(mut interner, input[lane * 3000..(lane + 1) * 3000],
|
||||
ready, start)
|
||||
readers << spawn interner_parallel_reader(interner, ready, start, stop)
|
||||
readers << spawn interner_parallel_reader(mut interner, ready, start, stop)
|
||||
}
|
||||
// Release readers and writers together once all of them are running.
|
||||
for _ in 0 .. 2 * lanes {
|
||||
|
||||
+451
-20
@@ -4,6 +4,187 @@ Every compiler fix must preserve or improve self-compilation efficiency. Agents
|
||||
retired instructions before and after compiler changes and resolve regressions before merging.
|
||||
Keep correctness tests and generated-output checks alongside performance measurements.
|
||||
|
||||
## Why the reported scores differed
|
||||
|
||||
The earlier **115,328 -> 95,877 instructions/line** comparison used compiler sources and input
|
||||
at `257b514a`, Apple Clang 21, and a **560,382-line** fixture. Its two roles were ordinary
|
||||
production and PGO production builds.
|
||||
|
||||
The later **124,706.52** score was an **unprofiled baseline**, not its optimized result.
|
||||
Homebrew Clang 23 wrote a profile that the selected Apple LLVM 21 merger could not read, so
|
||||
self-building fell back to an ordinary production build. Fixing that tool lookup gave
|
||||
**101,502.13** in the same comparison, on the newer **559,990-line** fixture.
|
||||
|
||||
Those historical scores changed compiler revision, input, and C toolchain. They must not be
|
||||
presented as a continuous performance series. A fresh revision check, described below, kept
|
||||
input and Apple Clang 21 fixed: **97,584.84 -> 105,074.91**, or **7.68% more instructions**.
|
||||
That comparison confirms a revision-associated increase as well as the reporting error. The
|
||||
follow-up bisection below locates the main increase at the tagged-union compiler change.
|
||||
|
||||
## Recent-commit bisection
|
||||
|
||||
The PGO failure was bisected over the eight first-parent commits from `257b514a30` to
|
||||
`2d1367e589`. The first commit activating the failure is **`2d1367e589` (#30962)**,
|
||||
which re-enabled automatic Clang PGO for production self-builds. Its parent does not attempt
|
||||
automatic production PGO; a passing predicate there means no failing attempt, not successful PGO.
|
||||
|
||||
The underlying lookup predates that change: `find_llvm_profdata` was introduced in `a75cd076ed`
|
||||
and searched PATH, then `xcrun`, without consulting the selected C compiler's installation.
|
||||
With explicit Homebrew Clang 23 and Apple LLVM 21 selected for merging, the producer writes raw
|
||||
profile format 11 while the merger expects format 10. The merge fails and self-building falls
|
||||
back to an unprofiled compiler. Apple Clang 21 with its matching merger succeeds.
|
||||
|
||||
The bisect predicate compiled each historical, unmodified `vself.v` using one stable compiler.
|
||||
A small V-driver stand-in built a real Clang 23 instrumented program, and the historical tool
|
||||
ran it and attempted the real profile merge. This tests tool selection and fallback control flow;
|
||||
it is not a full compiler self-build at each checkpoint. It reproduces the same format mismatch
|
||||
as the retained full-build `latest2-before-build.log`. The toolchain lookup fix `8a6e8a32dc`
|
||||
completed PGO in the same probe, providing a positive control.
|
||||
|
||||
Reproduction scripts, the complete Git bisect log, commands, and six checkpoint/control results
|
||||
are in `/Users/alex/code/v3-self-90k-artifacts/bisect-20261021/`, including
|
||||
`pgo-bisect.py`, `pgo-predicate.py`, `pgo-bisect.log`, and `pgo-summary.json`.
|
||||
|
||||
The instruction-count bisect identifies **`da62dd4bd7` (#29578), "v3: represent sum types as
|
||||
tagged unions"**, with immediate parent `7940bb1a8e`. This is a separate change from the PGO
|
||||
activation. Both sides completed PGO with Apple Clang 21 and its matching profile merger.
|
||||
|
||||
Each checkpoint was rebuilt twice in place with TCC using a seed compatible with its recursive
|
||||
type representation. The resulting compiler then built its own instrumented and optimized
|
||||
production versions with Apple Clang 21 and `-cflags '-mcpu=apple-m1'`. PGO training used the
|
||||
same unchanged 559,990-line fixture, once with default workers and once with `VJOBS=1`.
|
||||
This fresh recipe differs from earlier sessions; compare roles within this session rather than
|
||||
treating their absolute scores as a continuation of earlier tables.
|
||||
|
||||
Three exploratory samples per checkpoint located the boundary. A separate confirmation used
|
||||
one warmup per role and ten alternating measured pairs, with `VJOBS=1` and the same executable,
|
||||
source, and output paths. The parent was built in a second checkout to overlap C compilation;
|
||||
its exact build path is recorded. Fixture training and benchmark runs were serialized.
|
||||
|
||||
| Median metric | Parent `7940bb1a8e` | Tagged unions `da62dd4bd7` |
|
||||
| --- | ---: | ---: |
|
||||
| Retired instructions | 54,270,831,065.5 | 59,248,702,390.5 |
|
||||
| Instructions/parsed line | 96,913.93 | 105,803.14 |
|
||||
| CPU cycles, billions | 19.4757 | 21.6487 |
|
||||
| Peak resident memory, MiB | 1,133.09 | 1,117.53 |
|
||||
| Elapsed seconds | 5.0598 | 6.1247 |
|
||||
|
||||
The tagged-union commit used **9.17% more instructions**, increasing work in **all ten pairs**.
|
||||
The median paired increase was `4,788,159,580` instructions. All 22 runs succeeded; generated
|
||||
C was consistent within each role and passed Clang syntax checking. C intentionally differs
|
||||
between these revisions, which change code generation. Elapsed time and cycles reflect shared
|
||||
host load. These results attribute the main increase to the commit, not to one isolated hunk.
|
||||
|
||||
A separate exploratory control disabled PGO for both production compilers while retaining Apple
|
||||
Clang 21 and the other build flags. Instructions/line increased from **116,908.51 to 125,334.54**
|
||||
(*7.21% more*, all three pairs worse). Generated C matched the PGO output for each revision.
|
||||
This supports a compiler-work increase that persists without PGO; the primary attribution remains
|
||||
the ten-pair PGO comparison above. Control commands and all eight samples are retained as
|
||||
`unprofiled-confirmation-*`. A disk-space failure before control measurement was retained and
|
||||
retried after compressing owned artifacts; see `space-recovery.json`.
|
||||
|
||||
A separate verbose hardware-counter diagnostic, after one warmup per role, found approximately
|
||||
1.97 billion additional instructions in checking, 0.98 billion in transforms, and 1.84 billion
|
||||
in C generation/finish. Those instrumented runs are excluded from the primary statistics.
|
||||
|
||||
The commit changes semantic type handling as well as sum layout. An executable layout probe
|
||||
using identical fixture definitions measured `sizeof(types.Type)` changing from **16 to 72 bytes**;
|
||||
`flat.Node` remained 64 bytes. Type interning now canonicalizes child references, and C generation
|
||||
replaces some raw-word identity checks with structural hashing/equality. The compatibility paths
|
||||
also retain duplicated structural-equality calls when the first comparison returns false;
|
||||
disassembly confirms both calls survive optimization in `fn_param_compatible` and
|
||||
`fn_return_compatible`. These are concrete follow-up targets, not a measured attribution of the
|
||||
entire increase to larger values or duplicated calls. Overall peak memory decreased in this run.
|
||||
|
||||
The bisect log, checkpoint builds/profiles, 22 confirmation samples, generated-output checks,
|
||||
stage counters, layout probe, and disassembly are retained in the bisection directory above.
|
||||
See `instruction-bisect.log`, `confirmation-{metadata,samples,summary}.json`,
|
||||
`stage-breakdown.json`, `layout-probe.json`, and `compatibility-assembly.json`.
|
||||
`exploratory-log-audit.json` records reused exploratory filenames during the parallel parent
|
||||
prebuild; the accepted confirmation has distinct raw logs for every sample.
|
||||
Large generated C artifacts were losslessly compressed to `.c.gz` with SHA-256 roundtrip checks;
|
||||
`generated-c-compression.json` records their original paths and hashes. No compiler source or
|
||||
test expectations were changed as part of the bisection.
|
||||
|
||||
## Tagged-union regression follow-up
|
||||
|
||||
The follow-up patch removes repeated work made more expensive by the tagged representation:
|
||||
|
||||
* Cache complete inline-storage walks for sums as well as structs, including concrete generic
|
||||
sums. Reuse positive nested results immediately. Cache negative results across a recursive
|
||||
component only after a complete root walk proves that it contains no inline fixed array.
|
||||
* Read existing immutable canonical type references when checking a cached expression, avoiding
|
||||
intermediate copies of the 72-byte type value. Preserve worker ranges and probe overlays;
|
||||
sparse map values use stable copies because map growth can move their backing storage.
|
||||
* Remove duplicate semantic-equality calls in three compatibility paths. This cleanup alone did
|
||||
not establish a measurable improvement in exploratory runs.
|
||||
|
||||
Both compilers were freshly built from `36232086a5`, with the candidate's exact source diff
|
||||
retained in the build artifacts. Each used the same seed, two in-place TCC rebuilds, Apple
|
||||
Clang 21, `-mcpu=apple-m1`, and fresh matching PGO training on the unchanged fixture. Compiler
|
||||
build paths differed; benchmark executable, input, output, and working-directory paths matched.
|
||||
The PGO option-selection fixes were present in the candidate and do not change this explicit
|
||||
Apple Clang build recipe. No compiler layout, language semantics, or generated output changed.
|
||||
|
||||
One warmup per role preceded ten alternating pairs in each mode. All 44 runs succeeded with
|
||||
one positive retired-instruction count per run and byte-identical generated C within each mode.
|
||||
No builds or tests from this task ran concurrently with these comparisons.
|
||||
Parsed input remained 559,990 lines.
|
||||
|
||||
| Median metric | Before, one worker | Fix, one worker | Before, default | Fix, default |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| Retired instructions, billions | 56.5777 | 55.8777 | 60.8406 | 60.1385 |
|
||||
| Instructions/parsed line | 101,033.39 | 99,783.40 | 108,645.90 | 107,392.18 |
|
||||
| CPU cycles, billions | 18.7694 | 18.6136 | 27.4735 | 27.3065 |
|
||||
| Peak resident memory, MiB | 1,117.98 | 1,118.08 | 1,011.30 | 1,010.96 |
|
||||
| Elapsed seconds | 4.4335 | 4.3891 | 1.0224 | 1.0229 |
|
||||
|
||||
The patch used **1.24% fewer instructions with one worker** and **1.15% fewer with default
|
||||
workers**, with **0/10 regressing pairs** in each mode. Median paired differences were
|
||||
`-668,456,564` and `-704,473,390.5` instructions, respectively. This comparison measures the
|
||||
combined patch; it does not assign the improvement to each edit. Elapsed time and cycles remain
|
||||
contextual measurements on a shared host, not controlled latency claims.
|
||||
The **90,000 instructions/line target remains unmet**, and the full tagged-union regression has
|
||||
not been recovered. Do not add percentages from different sessions or compare their medians as
|
||||
a single continuous optimization series.
|
||||
|
||||
The baseline executable SHA-256 is
|
||||
`5f8939b958bb5f8c9727202c12d51fad8449ab1fca00346f2cc605e6fbf68c68`;
|
||||
the candidate executable SHA-256 is
|
||||
`5e5686330fc08937d3bd8252ba9aacc0dfe75113d1a3a5d388fe2bfa6cf2f9af`.
|
||||
Serial generated C has SHA-256
|
||||
`88583a0ca0e4f59877f7c2b7a6cb4260c046c4737a61c665b62af265344bba8d`;
|
||||
default-worker generated C has SHA-256
|
||||
`04603de3a32496290c63a567e46069509fd0f985906a40b9696303f14a337158`.
|
||||
|
||||
Reproduction scripts, exact build/environment metadata, source diffs, fixture inventories,
|
||||
per-run timing/stdout/stderr/status logs, binary hashes, and generated-output checks are in
|
||||
`/Users/alex/code/v3-self-90k-artifacts/fix-tagged-unions-20261021/`.
|
||||
Use `build.py`, `measure.py`, `source-a/metadata.json`, `combined/metadata.json`,
|
||||
`pr-source-verification.json`, and `combined-confirmation-{serial,default}*`.
|
||||
The PR source adds a helper comment and a reference ownership/visibility test to the measured
|
||||
implementation. Failed and rejected exploratory variants remain separately labeled in this
|
||||
artifact directory and are excluded from the accepted statistics.
|
||||
|
||||
Validation rebuilt the compiler in place and passed the type-cache, storage-query, parallel-check,
|
||||
fixed-array escape, and loop-lowering tests. Compiler diagnostics passed 1,848 cases with 5 skipped;
|
||||
PGO tool-selection tests, both generated-C syntax checks, formatting, and Markdown checks passed.
|
||||
Vet reported the same 92 existing diagnostics as the unchanged baseline.
|
||||
|
||||
The broad compiler suite finished with **4,138 passed, 40 skipped, and 30 failed**. Twenty-eight
|
||||
failing files also failed on the unchanged baseline. The two additional failures were harness
|
||||
effects: the relocated benchmark compiler retained its build-directory path in REPL output, and
|
||||
`SOURCE_DATE_EPOCH=0` contradicted a test requiring a recent build timestamp. After rebuilding in
|
||||
the test checkout, all 47 REPL cases passed. The timestamp test passed with the override unset;
|
||||
the unchanged baseline reproduced its failure with the override set.
|
||||
|
||||
The cache-refresh failure also varied with compiler placement and macOS `/tmp` versus
|
||||
`/private/tmp` aliases. Both baseline and candidate passed this test in the same checkout with
|
||||
the same canonical `VTMP`. Other failure assertions and primary diagnostics matched the baseline.
|
||||
No tests or expectations were changed to suppress failures; the broad run remains a failed run.
|
||||
Logs and comparisons are retained as `final-*`, including `final-full-audit.json`,
|
||||
`final-harness-reruns.json`, and `final-cache-canonical-comparison.json` in the artifact directory.
|
||||
|
||||
## Metric and scope
|
||||
|
||||
Use **retired instructions per parsed source line** to express compiler work; lower is better:
|
||||
@@ -31,6 +212,15 @@ Compare on the same machine and architecture. Keep compiler build options identi
|
||||
changing the build recipe is the optimization being evaluated; record that change. Record elapsed
|
||||
time, CPU cycles, and peak resident memory too. Fewer instructions do not guarantee lower latency.
|
||||
|
||||
Retired instructions are counted for the measured process. Other processes do not contribute
|
||||
their instructions, so an idle host is not required. These are dynamic execution counts: locks,
|
||||
retries, and other runtime paths can still introduce variation. Use `VJOBS=1` and repeated pairs
|
||||
to assess consistency. Record competing work as context for elapsed time and cycles; collect
|
||||
latency claims separately under controlled load. Apple documents the per-thread and per-task
|
||||
accounting in [XNU Recount][recount].
|
||||
|
||||
[recount]: https://github.com/apple-oss-distributions/xnu/blob/main/doc/observability/recount.md
|
||||
|
||||
The workload compiles the full `cmd/v` compiler through C generation, matching the V stages of
|
||||
`v self`. It excludes the external C compiler, linker, replacement of `./v`, and execution of
|
||||
the resulting compiler. It includes parsing, checking, transforms, and C generation. Use one
|
||||
@@ -85,8 +275,10 @@ worker for the instruction comparison; validate default parallelism separately.
|
||||
`V_MACOS_V3_NO_FALLBACK=1` prevents silent fallback to the legacy compiler.
|
||||
Reuse one output C path across all paired runs and keep distinct timing/stdout/stderr logs.
|
||||
5. Run one warmup per role, then ten measured pairs, alternating AB, BA, AB, BA, and so on.
|
||||
Run only one benchmark at a time, without concurrent builds or tests. Keep native support
|
||||
Run only one benchmark at a time. Do not wait for an idle host or pause unrelated work.
|
||||
Other builds and tests do not invalidate process instruction counts. Keep native support
|
||||
objects warm for both roles and record that state. Disable tracing and profiling.
|
||||
Label elapsed time and cycles collected under uncontrolled load as contextual measurements.
|
||||
6. Require exit status zero and exactly one positive `instructions retired` counter per sample.
|
||||
Preserve failed samples but do not count them as a completed comparison. If hardware counters
|
||||
are missing or zero, report them as unavailable; do not substitute elapsed time.
|
||||
@@ -101,23 +293,259 @@ worker for the instruction comparison; validate default parallelism separately.
|
||||
On another platform, use an available hardware retired-instruction counter with the same scope
|
||||
and protocol; do not compare its absolute counts with this machine's results.
|
||||
|
||||
## Benchmark acceptance status
|
||||
## Current self-compilation fixture
|
||||
|
||||
**No accepted regression baseline is recorded yet.** The earlier measurements below ran alongside
|
||||
unrelated compilations and do not meet the required quiet-host protocol. They are retained as
|
||||
rejected observations for audit purposes, not as evidence that the 10% target has been met.
|
||||
Both paired modes must be rerun without concurrent builds or tests before accepting a baseline.
|
||||
New comparisons use a detached checkout of merged master
|
||||
`2d1367e589ed30c03325e586a8ddfd3bc96c8c3d`, including its standard library.
|
||||
It contains **559,990 parsed physical lines in 562 `.v` files**, with no `.vh` files.
|
||||
The 90,000 instructions/line target is **50,399,100,000 instructions** for this fixture.
|
||||
Keep this source and denominator fixed across baseline and candidate runs.
|
||||
|
||||
A guarded attempt on 2026-10-21 waited 120 seconds for a quiet window and still observed 31
|
||||
compiler or test processes. It started no warmups or measured runs. The attempt and process
|
||||
records are retained in `/Users/alex/code/v3-self-speed-artifacts/quiet-20261021-attempt1.log`
|
||||
and `quiet-20261021-attempt1-guard.json`. A quiet host is required to complete this validation.
|
||||
The older 257b fixture below is incompatible with the merged compiler's sum-type representation:
|
||||
it emits incomplete recursive C union fields for its old compiler types. A C-file hash match
|
||||
alone did not catch this. The new fixture's generated C passes Clang syntax checking. Validate
|
||||
generated C when changing the fixture or compiler representation; do not accept counts for an
|
||||
invalid output. This establishes a new baseline; do not compare normalized scores across fixtures.
|
||||
|
||||
### Rejected observations from the shared host
|
||||
An exploratory three-pair run measured 105,611.36 instructions/line for merged master and
|
||||
105,762.28 for local-storage lookup guards. Generated C matched in every run. The guards showed
|
||||
no convincing gain and are excluded from the optimization. Three pairs do not establish a small
|
||||
regression or a completed ten-pair comparison. Logs remain under
|
||||
`/Users/alex/code/v3-self-90k-artifacts/guard-probe*`.
|
||||
|
||||
The new fixture is `/Users/alex/code/v3-self-90k-fixture`. Its 8,498-file source manifest
|
||||
uses the encoding described below and has SHA-256:
|
||||
`f53a9b162657ba667fde1d8a00cd6f59c0f7e6c7f847985894d61189122a2966`.
|
||||
Its source inventory, syntax-check logs, build logs, binaries, and instruction samples remain in
|
||||
`/Users/alex/code/v3-self-90k-artifacts/`.
|
||||
|
||||
## Semantic interning cache comparison
|
||||
|
||||
The first source optimization on master `9ca360c4ff` adds an atomic cache of immutable canonical
|
||||
semantic types. Hits verify complete structural equality before bypassing recursive hashing,
|
||||
locking, and map lookup. Promotion clears the cache before worker arenas are released. The
|
||||
checker also uses the cheap fingerprint for its existing type-name cache, retaining its full
|
||||
structural-equality check.
|
||||
|
||||
Both roles use successful Clang 23 PGO builds with `-mcpu=apple-m1`, the same build recipe,
|
||||
and the unchanged 559,990-line fixture. The candidate is the first cache variant, before the
|
||||
subsequent source-spelling and child-fingerprint experiments.
|
||||
|
||||
| Median metric | Before, one worker | Cache, one worker | Before, default | Cache, default |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| Retired instructions | 57,045,612,737.0 | 55,295,951,631.0 | 61,027,236,671.0 | 59,242,819,343.0 |
|
||||
| Instructions/parsed line | 101,868.98 | 98,744.53 | 108,979.15 | 105,792.64 |
|
||||
| CPU cycles, billions | 27.0600 | 26.8994 | 28.7436 | 28.6683 |
|
||||
| Peak resident memory, MiB | 1,118.17 | 1,118.55 | 1,011.12 | 1,014.97 |
|
||||
| Elapsed seconds | 10.6821 | 9.4104 | 2.7068 | 2.6945 |
|
||||
|
||||
Retired instructions decreased **3.07% with one worker** and **2.92% with default parallelism**.
|
||||
All ten pairs in each mode improved. Median paired instruction differences were
|
||||
`-1,799,425,264` and `-1,775,404,206.5`, respectively. All 44 samples succeeded and generated
|
||||
byte-identical C within each mode. C passed Clang syntax checking. Elapsed time and cycles
|
||||
were collected under uncontrolled host load. **The 90,000 instructions/line target is not met.**
|
||||
|
||||
The 151 type tests passed, including borrowed-payload mutation, concurrent growth, and arena
|
||||
promotion checks. Validation of the final cache bundle is recorded below.
|
||||
Commands, environment, raw counters, and output checks are retained as
|
||||
`interner-cache-matched-*` and `interner-cache-syntax-validation.json` in the artifacts directory.
|
||||
The baseline executable SHA-256 is
|
||||
`575ec55f8f553cfea6bff81ad937192687a840d84cc0f7e8832327804be5b8b7`;
|
||||
the candidate executable SHA-256 is
|
||||
`22e35698c36f05c7009b2516cda9270985366404230fabdf2399c16f178a3572`.
|
||||
|
||||
## Follow-up cache comparison with Apple Clang 21
|
||||
|
||||
A fresh comparison on master `9ca360c4ff` used Apple Clang 21 PGO for both roles and the same
|
||||
559,990-line fixture. The candidate combines the semantic interning cache with source-spelling
|
||||
content fingerprints, shallow semantic child fingerprints, and a reference receiver for
|
||||
`Type.source_type_id`. Fingerprints still require complete equality checks before reuse.
|
||||
|
||||
| Median metric | Before, one worker | Cache, one worker | Before, default | Cache, default |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| Retired instructions, billions | 59.0643 | 57.5221 | 63.1715 | 61.5222 |
|
||||
| Instructions/parsed line | 105,473.91 | 102,719.79 | 112,808.22 | 109,862.98 |
|
||||
| CPU cycles, billions | 29.1419 | 28.4944 | 31.1445 | 30.3943 |
|
||||
| Peak resident memory, MiB | 1,117.54 | 1,118.23 | 1,007.91 | 1,010.11 |
|
||||
| Elapsed seconds | 33.9576 | 34.9625 | 6.2985 | 6.6990 |
|
||||
|
||||
The combined change used **2.61% fewer instructions in both modes**, with **0/10 regressing
|
||||
pairs** in each. Median paired differences were `-1,529,766,769` instructions serial and
|
||||
`-1,646,091,022.5` default. All 44 runs succeeded and emitted byte-identical C within each mode.
|
||||
Elapsed time and cycles reflect uncontrolled host load. This is a matched source-change
|
||||
comparison; its absolute scores must not be compared with the Clang 23 sessions.
|
||||
The 90,000 instructions/line target is still unmet.
|
||||
|
||||
Raw samples, binary hashes, commands, environment, and output checks are retained as
|
||||
`reference21-content-matched-*` in the artifacts directory above. This bundle comparison does
|
||||
not establish an independent benefit from each additional fingerprint change.
|
||||
|
||||
Validation of this exact cache bundle completed with 1,848 compiler-error cases passing and
|
||||
5 skipped; all 151 type tests, source-spelling tests, and PGO/self-build tool tests passed.
|
||||
Formatting, Markdown checks, and vet completed; vet reported existing documentation warnings.
|
||||
The broad compiler suite finished with **4,140 passed, 40 skipped, and 28 failed**. All 28 failing
|
||||
files also failed with the unchanged `9ca360c4ff` baseline binary. Assertion locations and primary
|
||||
compilation diagnostics matched; the cache-refresh test required the same checkout path to reach
|
||||
the same assertions. No tests or expectations were changed to suppress these failures.
|
||||
This is not a full-suite pass. Raw logs and comparisons are retained as
|
||||
`content-child-compiler-tests.*`, `baseline-failed-subset-*`, and `baseline-cache-same-root.*`.
|
||||
|
||||
## LLVM toolchain lookup comparison on latest master
|
||||
|
||||
Measured on 2026-10-21 with compiler sources from master `9ca360c4ff`, the toolchain lookup
|
||||
fix at `8a6e8a32dc`, and the same 559,990-line fixture. Both compilers were rebuilt in place
|
||||
with the command above, using Homebrew Clang 23.1.0 and `-cflags '-mcpu=apple-m1'`.
|
||||
The baseline again fell back after the LLVM 21/23 profile-format mismatch; the candidate
|
||||
completed PGO. This compares the tool lookup fix on the same compiler sources, so upstream
|
||||
changes between master versions are not attributed to that fix.
|
||||
|
||||
| Median metric | Before, one worker | After, one worker | Before, default | After, default |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| Retired instructions, billions | 68.9401 | 56.3825 | 73.8913 | 60.6981 |
|
||||
| Instructions/parsed line | 123,109.57 | 100,684.76 | 131,951.04 | 108,391.44 |
|
||||
| CPU cycles, billions | 23.0552 | 20.1368 | 32.7966 | 29.9206 |
|
||||
| Peak resident memory, MiB | 1,116.04 | 1,118.15 | 1,009.52 | 1,014.15 |
|
||||
| Elapsed seconds | 5.4194 | 4.7996 | 1.2632 | 1.1951 |
|
||||
|
||||
Instruction reductions were **18.22% serial** and **17.85% default**, with **0/10 regressing
|
||||
pairs** in each mode. Median paired instruction differences were `-12,579,586,975.5` serial
|
||||
and `-13,218,242,894.5` default. **The 90,000 instructions/line target remains unmet.**
|
||||
All 44 runs succeeded and generated byte-identical C within each mode; both outputs passed
|
||||
Clang syntax checking. Elapsed time and cycles were collected under uncontrolled host load.
|
||||
The PGO, ordinary self-build, and cache-bypass tool tests passed, as did the hello-world smoke test.
|
||||
|
||||
Raw samples, complete environments, commands, build logs, binary hashes, syntax checks, and
|
||||
validation logs are retained as `latest2-*` in the artifacts directory above. Generated C
|
||||
hashes match the preceding master comparison below. SHA-256 values:
|
||||
|
||||
```text
|
||||
baseline executable, latest2-before-v
|
||||
8cdaf2fd18d6abd30e12e67953914abfa1d0b3e733211c4509b316095de71b9b
|
||||
candidate executable, latest2-after-v
|
||||
575ec55f8f553cfea6bff81ad937192687a840d84cc0f7e8832327804be5b8b7
|
||||
merged training profile
|
||||
93e991a1f4f6952c4645d0dab67a4da1a897fca48c647dea5dcc5c53f9f1db1b
|
||||
```
|
||||
|
||||
## Compiler revision check on identical input
|
||||
|
||||
A separate 2026-10-21 comparison held the 559,990-line fixture, source-file list, Apple Clang 21,
|
||||
production settings, and PGO recipe fixed. The older binary uses compiler sources at `257b514a`;
|
||||
the newer binary uses merged sources at `2d1367e589`. Each build trained on its own revision's
|
||||
compiler source, following the same self-build recipe. The older binary successfully compiles
|
||||
the newer fixture to C, which makes this direction of comparison possible.
|
||||
|
||||
| Median metric | Older compiler | Merged compiler |
|
||||
| --- | ---: | ---: |
|
||||
| Retired instructions | 54,646,536,102.5 | 58,840,900,876.5 |
|
||||
| Instructions/parsed line | 97,584.84 | 105,074.91 |
|
||||
| CPU cycles, billions | 23.5851 | 25.1845 |
|
||||
| Peak resident memory, MiB | 1133.02 | 1117.52 |
|
||||
| Elapsed seconds | 7.9881 | 8.9251 |
|
||||
|
||||
The merged compiler executed **7.68% more instructions** between medians, with more work in
|
||||
**10/10 pairs**. The median paired increase was `4,174,866,314` instructions. All 22 runs
|
||||
succeeded and each role produced repeatable C. Both C outputs passed Clang syntax checking.
|
||||
They intentionally differ across roles because compiler revisions changed sum-type representation;
|
||||
this is a revision diagnostic, not an output-preserving internal optimization comparison.
|
||||
It establishes increased work for these compiler builds without isolating a particular commit.
|
||||
Elapsed time and cycles were collected under uncontrolled host load.
|
||||
|
||||
Thus the earlier reporting mixed an unprofiled baseline with a PGO result, but a compiler-version
|
||||
increase also exists on matched input. The historical 95,877.04 result still uses the older
|
||||
560,382-line fixture; attributing the remaining fixture effect requires a separate matched check.
|
||||
Do not explain the entire discrepancy as denominator changes or assume that no regression exists.
|
||||
|
||||
Logs, inventories, counters, commands, environments, and syntax checks are retained as
|
||||
`revision-diagnostic-*` in the artifacts directory above. The 562-file inventory matches exactly
|
||||
between roles. SHA-256 values:
|
||||
|
||||
```text
|
||||
older compiler executable
|
||||
e554d76a9ad101c03c74864b5d428cef6f7b1df29fd11dad8ab43e74e60cbde4
|
||||
merged compiler executable
|
||||
7aab1e75525e5966566d7fe30debb543aaaeea68159dcd94047b46b7ada3788f
|
||||
older compiler generated C on the current fixture
|
||||
abb6372f6afc7892a15db0843ad4cca9fa2ca638ef331704866d59fb6d72d50d
|
||||
merged compiler generated C on the current fixture
|
||||
88583a0ca0e4f59877f7c2b7a6cb4260c046c4737a61c665b62af265344bba8d
|
||||
```
|
||||
|
||||
## Matched Clang 23 comparison before the latest master rebuild
|
||||
|
||||
Measured on 2026-10-21 with compiler sources from merged master `2d1367e589` and the fixed
|
||||
559,990-line fixture above. Both roles selected Homebrew Clang 23.1.0 and `-mcpu=apple-m1`.
|
||||
The baseline selected Apple's LLVM 21 profile merger, which rejected Clang 23's version-11 raw
|
||||
profile and triggered an ordinary production build. The candidate selected Clang 23's merger
|
||||
and completed PGO. Both final binaries contain no profiling runtime instrumentation.
|
||||
|
||||
The same production self-build command was used in each checkout, outside measured runs:
|
||||
|
||||
```sh
|
||||
env -u VFLAGS -u VEXE -u LLVM_PROFILE_FILE -u VJOBS SOURCE_DATE_EPOCH=0 \
|
||||
V_MACOS_V3_NO_FALLBACK=1 ./v -prod -cc /opt/homebrew/opt/llvm/bin/clang \
|
||||
-cflags '-mcpu=apple-m1' -o /absolute/path/to/role-v self
|
||||
```
|
||||
|
||||
Each mode used one warmup per role and ten alternating measured pairs. All 44 runs succeeded;
|
||||
generated C matched byte for byte within each mode. Elapsed time and cycles reflect shared-host
|
||||
load and do not establish a latency improvement.
|
||||
|
||||
| Median metric | Before, one worker | After, one worker | Before, default | After, default |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| Retired instructions, billions | 69.8344 | 56.8402 | 74.4558 | 61.1362 |
|
||||
| Instructions/parsed line | 124,706.52 | 101,502.13 | 132,959.12 | 109,173.73 |
|
||||
| CPU cycles, billions | 33.1496 | 28.9486 | 33.3547 | 30.3399 |
|
||||
| Peak resident memory, MiB | 1116.16 | 1118.05 | 1008.88 | 1010.32 |
|
||||
| Elapsed seconds | 21.2254 | 18.6263 | 5.4424 | 5.5684 |
|
||||
|
||||
The median instruction reductions were **18.61% serial** and **17.89% default**, with
|
||||
**0/10 regressing pairs** in each mode. Median paired differences were `-12,984,267,344.5`
|
||||
and `-13,337,504,752` instructions, respectively. The 90,000 target was not reached.
|
||||
Logs and samples are `clang23-matched-{serial,default}*` in the artifacts directory above;
|
||||
`clang23-before-build.log` records the version mismatch and fallback, while
|
||||
`clang23-after-build.log` records the successful PGO build.
|
||||
|
||||
The optimized count is higher than the historical 95,877.04 result below, but these sessions
|
||||
do not isolate a compiler regression. The historical result used Apple Clang 21, compiler
|
||||
sources at `257b514a`, and a different source fixture.
|
||||
Both compiler implementation and input changed after the tagged-union merge. The fixture has
|
||||
only 392 fewer lines, so the denominator alone does not explain the difference. Isolating the
|
||||
causes requires matched comparisons; comparing these two sessions cannot attribute the increase.
|
||||
In particular, 124,706.52 is the current **unprofiled baseline**, not the optimized result.
|
||||
|
||||
SHA-256 values for this comparison:
|
||||
|
||||
```text
|
||||
baseline executable, clang23-before-v
|
||||
1541c85d77b10337467caacbc6cf67314a1635d78ce5c9e20e1b8b7f1fb6d454
|
||||
candidate executable, clang23-after-v
|
||||
c5ad981e88dc2f22fd0bd97042094e89a55ac85ee4dcd75e2ffa2ac86714edaa
|
||||
generated C, every serial run
|
||||
88583a0ca0e4f59877f7c2b7a6cb4260c046c4737a61c665b62af265344bba8d
|
||||
generated C, every default-worker run
|
||||
04603de3a32496290c63a567e46069509fd0f985906a40b9696303f14a337158
|
||||
```
|
||||
|
||||
## Historical measurement status
|
||||
|
||||
The PGO comparison below records process instruction counts under shared-host load on the old
|
||||
fixture. The earlier quiet-host gate was too strict: unrelated compilations do not invalidate
|
||||
these instruction measurements. All ten paired differences favored PGO in each mode, with
|
||||
matching source inventories and generated C. Elapsed times and cycles reflect uncontrolled
|
||||
host load and do not establish a latency improvement. These results describe the older compiler
|
||||
and fixture, not the current self-compilation baseline.
|
||||
|
||||
The 2026-10-21 attempt that waited 120 seconds for an idle host started no measurements.
|
||||
Its logs remain in `/Users/alex/code/v3-self-speed-artifacts/quiet-20261021-attempt1.log`
|
||||
and `quiet-20261021-attempt1-guard.json`; an idle host is no longer an acceptance requirement.
|
||||
|
||||
### Historical PGO instruction comparison
|
||||
|
||||
Measured on 2026-10-20. Baseline: merged master
|
||||
`257b514a303f4d302ed23bbac1d51a2ca9624a66`. Candidate: that commit with the Clang production
|
||||
self-build changes in this PR. The compiler source compiled into both binaries is unchanged;
|
||||
self-build changes later merged in [#30962](https://github.com/vlang/v/pull/30962).
|
||||
The compiler source compiled into both binaries is unchanged;
|
||||
the intentional build difference is profile-guided optimization (PGO).
|
||||
The candidate was rebuilt and remeasured after correcting atomic profile collection and
|
||||
isolating the training environment.
|
||||
@@ -129,8 +557,8 @@ default workers and `VJOBS=1`, merged the profiles, and rebuilt with `-fprofile-
|
||||
contains no profiling instrumentation. This comparison evaluates the resulting compiler's execution,
|
||||
not the cost of producing it.
|
||||
|
||||
The rejected session observed 16.87% fewer instructions with one worker and 16.40% fewer with
|
||||
default workers. These percentages are not accepted performance claims under the protocol.
|
||||
The session measured 16.87% fewer instructions with one worker and 16.40% fewer with default
|
||||
workers. These are instruction-count reductions, not elapsed-time improvement claims.
|
||||
All 44 runs, including warmups, exited successfully and generated byte-identical C.
|
||||
The fixed input contains **560,382 parsed physical lines in 555 `.v` files**, with no `.vh` files.
|
||||
The baseline and candidate report the same source closure and line count. Normalizing the retained
|
||||
@@ -150,15 +578,17 @@ Median paired instruction differences, candidate minus baseline:
|
||||
Instruction regressions: **0/10 pairs** in each mode, for both total instructions and
|
||||
instructions per source line.
|
||||
|
||||
Process snapshots recorded unrelated compilations elsewhere on the shared host. The absence of
|
||||
builds or tests from this task did not satisfy the requirement to measure without any concurrent
|
||||
builds or tests. Neither the instruction reductions nor the elapsed times from this session
|
||||
establish an accepted baseline. Preserve the raw data and replace this record with quiet-host
|
||||
measurements before making performance or regression claims.
|
||||
Process snapshots recorded unrelated compilations elsewhere on the shared host. Retain them
|
||||
as context for the elapsed-time and cycle measurements. Compare subsequent compiler changes
|
||||
against a freshly built baseline in the same measurement session, preserving the fixed fixture
|
||||
and all raw data.
|
||||
|
||||
### Production self-build behavior
|
||||
|
||||
On non-Windows hosts, `./v -prod -cc clang self` uses PGO when `llvm-profdata` is available.
|
||||
The profile merger is searched in Clang's reported installation directory, then beside its
|
||||
resolved executable, then in `PATH` and through `xcrun` on macOS. This avoids a fallback when
|
||||
a different LLVM release cannot read the selected compiler's raw profile format.
|
||||
This spends more time building the production compiler: two C compilations plus training, rather
|
||||
than one C compilation. Ordinary `./v self` keeps its existing build path. Explicit `-parallel-cc`,
|
||||
`-no-prod-options`, explicit `-cflags` profile options, unavailable profiling tools, and other C
|
||||
@@ -170,7 +600,8 @@ counter updates so worker threads do not lose increments. `VJOBS=1` or `-no-para
|
||||
the serial path. Workspaces are private to each invocation and removed afterward; existing
|
||||
workspaces and the caller's profile destination are preserved. Profile paths containing spaces
|
||||
remain a single C compiler argument. The final effective `-cc` selects the PGO toolchain;
|
||||
command-line options override compiler choices inherited through `VFLAGS`.
|
||||
command-line options override compiler choices inherited through `VFLAGS`. Split `-cc clang`
|
||||
and equals-form `-cc=clang` options have the same last-option-wins behavior.
|
||||
|
||||
### Reproduction metadata
|
||||
|
||||
|
||||
Reference in new issue
Block a user