v3: represent sum types as tagged unions (#29578)

* v3: represent sum types as tagged unions

* v3: fix sum projection lifetimes and parser storage

* sokol: fix sokol.audio on Windows TCC and fix simple_bytebeat example (fixes #30891)

- Fix TCC objbase.h bug where COINIT_MULTITHREADED is defined as 0x3 instead of 0x0, causing CoInitializeEx to fail with E_INVALIDARG and CoCreateInstance for IMMDeviceEnumerator to fail with CO_E_NOTINITIALIZED.
- Fix TCC missing 1-byte struct packing for WAVEFORMATEX / WAVEFORMATEXTENSIBLE, which caused IAudioClient_Initialize to fail with E_INVALIDARG.
- Fix simple_bytebeat.v user_data callback signature and frame count handling.
- Add sokol.audio setup/shutdown test in vlib/sokol/audio/audio_test.c.v.

* cmd/tools, ci: fix four failing master jobs

- cmd/tools/vup.v: build the self-rebuild argument list by appending
  instead of using an array spread. The release compiler that the
  v-up jobs use fails on the spread with "unexpected token `...`".
- cmd/tools/vls_test.v: apply v fmt to the two json2.encode calls.
  This clears the fmt check failure in the clang-macos job.
- vlib/v/gen/arm64/native_allocation_test.v: use tabs instead of four
  spaces for indentation. The change is whitespace-only.
- .github/workflows/v_apps_and_modules_compile_ci.yml: clone the docs
  generator into docs_generator. The repository tracks a docs
  directory, so cloning into docs failed on every run.

Co-Authored-By: WOZCODE <contact@withwoz.com>

* Add files via upload

* time: update bundled zoneinfo.zip to tzdata 2026e

The bundled zoneinfo.zip was built from tzdata 2025b. Windows has no
system zoneinfo, so time.load_location uses this file by default there;
so does any program that imports time.tzdata. Two changes since 2025b
make it wrong:

- tz 2026c: Morocco moved to permanent +00 on 2026-09-20, so
  Africa/Casablanca and Africa/El_Aaiun are already one hour off.
- tz 2026b-2026e: British Columbia, Alberta, the Northwest Territories
  and Manitoba keep their daylight offset after 2026-11-01, so
  America/Vancouver, Edmonton, Inuvik and Winnipeg are one hour off
  from that date.

The file is rebuilt with the tz make line that reproduces the 2025b
file byte for byte (make PACKRATDATA=backzone posix_only), with only
the release changed to 2026e; 29 of the 599 entries change.

Tests: test_bundled_zoneinfo_has_future_posix_rules now uses
America/Los_Angeles, because America/Vancouver no longer observes
PST/PDT in 2030. test_load_location_posix_fixed_future_rule expects
Morocco's new fixed +00. A new test checks the four Canadian zones in
December 2026.

* tools: keep the V3 module cache shared across `v test` sessions

`v test` points VTMP at a new `tsession_<id>` folder, and the V3 module
cache defaults to VTMP. Each session therefore started with an empty
module cache, compiled every cached module (builtin, os, time, ...) with
the C compiler again, and deleted them when the session ended. For a
test of a 69-file program this added about 17s to every run.

setup_new_vtmp_folder now sets V3CACHE to the shared VTMP before it
switches VTMP to the session folder, unless V3CACHE is already set or
V3_TEST_ISOLATE_CACHE=1 requests an isolated cache (V3CACHE would
override that request). Temporary files of the tests still go to the
session folder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* examples: fix cached Collatz chains and guard integer bounds

* sokol: keep bytebeat frame state on the audio callback thread

* testing: cover and document shared V3 cache sessions

* ci: run the current updater with its matching standard library

* cgen: preserve pointer-sum ownership through loops and field replacement

* checker: reject pointer-sum loans escaping through aggregate stores

* sumtypes: preserve projection moves, mixed ownership, and library storage contracts

* ownership: preserve mixed-result loans through sum wrappers and storage

* sumtypes: preserve Result and Option ownership and aligned array references

* compiler: preserve fallback loans and fixed-array pointer sum slots

* compiler: correct type values and ownership flow clones

* compiler: isolate mutable fallback loop bindings

* compiler: preserve wrapped exits and fresh array ownership

* tests: preserve invalid native Type tag checks

* compiler: preserve mut slot contracts and native sum projections

* compiler: repair optional bridges, sum replacement, and native type identity

* compiler: preserve generic optional views and mutable sum storage

* types: preserve original references in lane lifetime fixtures

* fastc: preserve typed references for computed arguments

* fastc: preserve existing sum values in identity casts

* fastc: preserve original storage when reassigning smartcast sum pointers

* compiler: preserve owned sum cleanup and borrowed return lifetimes

* cgen: keep vectored output headers out of headerless compiler C

---------

Co-authored-by: Alexander Medvednikov <alexander@medvednikov.com>
Co-authored-by: Keito Tobichi <keito940@live.jp>
Co-authored-by: Richard Wheeler <18647491+PythonWillRule@users.noreply.github.com>
Co-authored-by: WOZCODE <contact@withwoz.com>
Co-authored-by: Karan Gupta <uft289@gmail.com>
Co-authored-by: CyberNative AI LLC <hello@cybernative.ai>
Co-authored-by: Jakub Jabłoński <jj@simplid.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
295 files changed
+16930 -8375

No files matched your search

+23
View File
@@ -1015,6 +1015,29 @@ fn test_install_external_tool_modules_leaves_a_build_with_a_path_to_the_compiler
assert !os.exists(attempts)
}
fn test_ownership_bootstrap_recognizes_compiler_inputs_only() {
root := os.join_path(os.temp_dir(), 'v3_bootstrap_inputs_${os.getpid()}')
defer { os.rmdir_all(root) or {} }
for entry in ['cmd/v', 'vlib/v'] {
directory := os.join_path(root, entry)
os.mkdir_all(directory)!
os.write_file(os.join_path(directory, 'v.v'), 'module main')!
}
for entry in ['cmd/v', 'cmd/v/v.v', 'vlib/v', 'vlib/v/v.v'] {
input := os.join_path(root, entry)
args := ['-gc', 'none', '-d', 'ownership', '-o', 'compiler', input]
assert compiler_bootstrap_requested(args, root)
}
entry := os.join_path(root, 'vlib/v/v.v')
assert compiler_bootstrap_requested(['run', entry], root)
assert compiler_bootstrap_requested(['crun', entry], root)
input := os.join_path(root, 'v.v')
os.write_file(input, 'module main')!
assert !compiler_bootstrap_requested(['-d', 'ownership', input], root)
assert !compiler_bootstrap_requested(['-o', entry, input], root)
assert !compiler_bootstrap_requested(['-path', entry, input], root)
}
fn test_v1_fallback_rejects_copied_current_launcher() {
root := os.join_path(os.vtmp_dir(), 'v1_fallback_self_${os.getpid()}')
os.mkdir_all(root)!
+28 -6
View File
@@ -307,13 +307,35 @@ fn ownership_compiler_is_required(args []string) bool {
return false
}
fn compiler_bootstrap_requested(args []string, vroot string) bool {
entries := ['cmd/v', 'vlib/v'].map(os.real_path(os.join_path(vroot, it)))
mut option_value_follows := false
for arg in args {
if option_value_follows {
option_value_follows = false
continue
}
if arg == '-cf' || pref.option_may_consume_value(arg) {
option_value_follows = true
continue
}
if arg.starts_with('-') || arg in ['run', 'crun'] {
continue
}
if !os.exists(arg) {
return false
}
input := os.real_path(arg)
if os.is_dir(input) {
return input in entries
}
return os.file_name(input) == 'v.v' && os.dir(input) in entries
}
return false
}
fn ownership_compiler_bootstrap_input(args []string, vroot string) bool {
compiler_sources := [
os.real_path(os.join_path(vroot, 'cmd', 'v')),
os.real_path(os.join_path(vroot, 'cmd', 'v', 'v.v')),
os.real_path(os.join_path(vroot, 'vlib', 'v', 'v.v')),
]
return args.any(os.exists(it) && os.real_path(it) in compiler_sources)
return compiler_bootstrap_requested(args, vroot)
}
// ownership_bootstrap_can_use_current_compiler distinguishes compiling in support from
+13 -6
View File
@@ -5093,7 +5093,12 @@ println(sum)
The built-in method `type_name` returns the name of the currently held
type.
With sum types you could build recursive structures and write concise but powerful code on them.
Copying a sum type copies its held value. Reference variants still refer to the same object.
Sum payloads must have finite layout. A function variant stores a function reference, so its
parameter and return types may refer to the containing sum type.
Use references for recursive fields, as in this binary tree:
```v
// V's binary tree
@@ -5101,8 +5106,8 @@ struct Empty {}
struct Node {
value f64
left Tree
right Tree
left &Tree
right &Tree
}
type Tree = Empty | Node
@@ -5117,9 +5122,11 @@ fn sum(tree Tree) f64 {
}
fn main() {
left := Node{0.2, Empty{}, Empty{}}
right := Node{0.3, Empty{}, Node{0.4, Empty{}, Empty{}}}
tree := Node{0.5, left, right}
empty := Tree(Empty{})
left := Tree(Node{0.2, &empty, &empty})
leaf := Tree(Node{0.4, &empty, &empty})
right := Tree(Node{0.3, &empty, &leaf})
tree := Tree(Node{0.5, &left, &right})
println(sum(tree)) // 0.2 + 0.3 + 0.4 + 0.5 = 1.4
}
```
+4
View File
@@ -6,3 +6,7 @@ retain their fields even when several levels use the same directory type.
When a payload points back to an object already being printed, the cycle is shown as
`<circular>`. Shared objects reached through separate branches are printed in each branch.
Sum payloads use inline storage. Recursive trees need a reference or a dynamic collection
to break the value layout cycle, for example `type Tree = &Directory | File` when
`Directory` contains `Tree` fields. Reference variants also allow deliberate object cycles.
+38
View File
@@ -124,6 +124,44 @@ fn test_free_is_a_noop_for_arena_memory() {
}
}
fn test_noslices_array_growth_and_free_preserve_arena_owned_buffers() {
mut a := arena.new(chunk_size: 4096)
defer {
a.free()
}
a.push()
marker := unsafe { malloc(32) }
unsafe { vmemset(marker, 0x5a, 32) }
mut values := []u8{len: 8, cap: 8, init: u8(index)}
unsafe { values.flags.set(.noslices) }
assert a.owns(values.data)
for i in 8 .. 512 {
values << u8(i % 251)
}
assert a.owns(values.data)
for i in 0 .. values.len {
assert values[i] == u8(i % 251)
}
unsafe { values.free() }
mut offset_values := []u64{len: 4, cap: 4, init: u64(index)}
unsafe { offset_values.flags.set(.noslices) }
assert offset_values.pop_left() == 0
for i in 4 .. 128 {
offset_values << u64(i)
}
assert a.owns(offset_values.data)
assert offset_values[0] == 1
assert offset_values.last() == 127
a.pop()
// Unique buffers still belong to their arena after it is popped.
unsafe { offset_values.free() }
assert a.owns(marker)
// Arena storage remains valid until the explicit bulk reset.
unsafe { assert marker[0] == 0x5a && marker[31] == 0x5a }
a.reset()
assert a.used() == 0
}
fn test_realloc_copies_arena_memory_out_after_pop() {
mut a := arena.new()
defer {
+18
View File
@@ -574,6 +574,24 @@ pub fn free(ptr voidptr) {
}
}
@[unsafe]
fn free_unaliased(ptr voidptr) {
$if prealloc || builtin_free_nop ? {
return
}
$if gcboehm ? {
$if builtin_arena ? {
// Scoped arena allocations are released in bulk, including unique buffers.
if _unlikely_(C.atomic_load_u32(&g_arena_hooks_ready) != 0) && g_arena_owns_hook(ptr) {
return
}
}
C.GC_FREE(ptr)
} $else {
free(ptr)
}
}
// memdup dynamically allocates a `sz` bytes block of memory on the heap
// memdup then copies the contents of `src` into the allocated space and
// returns a pointer to the newly allocated space.
+37 -27
View File
@@ -83,12 +83,16 @@ fn alloc_array_data_uninit(total_size u64) voidptr {
}
@[inline]
fn (a array) uses_noscan_data() bool {
fn (a &array) uses_noscan_data() bool {
return a.flags.has(.noscan_data)
}
@[inline]
fn (a array) alloc_array_data_like(total_size u64) voidptr {
fn (a &array) alloc_array_data_like(total_size u64) voidptr {
alignment := a.data_alignment()
if alignment > 16 {
return alloc_array_data_aligned(total_size, alignment, true)
}
$if gcboehm_opt ? {
if a.uses_noscan_data() {
return alloc_array_data_noscan(total_size)
@@ -98,7 +102,11 @@ fn (a array) alloc_array_data_like(total_size u64) voidptr {
}
@[inline]
fn (a array) alloc_array_data_like_uninit(total_size u64) voidptr {
fn (a &array) alloc_array_data_like_uninit(total_size u64) voidptr {
alignment := a.data_alignment()
if alignment > 16 {
return alloc_array_data_aligned(total_size, alignment, false)
}
$if gcboehm_opt ? {
if a.uses_noscan_data() {
return alloc_array_data_noscan_uninit(total_size)
@@ -108,7 +116,7 @@ fn (a array) alloc_array_data_like_uninit(total_size u64) voidptr {
}
@[inline; unsafe]
fn (a array) data_header() &ArrayDataHeader {
fn (a &array) data_header() &ArrayDataHeader {
if !a.flags.has(.managed) || a.data == unsafe { nil } {
return unsafe { nil }
}
@@ -117,7 +125,7 @@ fn (a array) data_header() &ArrayDataHeader {
}
@[inline]
fn (a array) buffer_has_slices() bool {
fn (a &array) buffer_has_slices() bool {
if a.flags.has(array_flag_retained_aligned_fixed) {
return true
}
@@ -457,9 +465,9 @@ pub fn (mut a array) ensure_cap(required int) {
prealloc_discard_pages(a.data, usize(a.cap) * usize(a.element_size))
}
if a.flags.has(.managed) {
free(a.data_header().allocation)
free_unaliased(a.data_header().allocation)
} else {
free(a.data)
free_unaliased(a.data)
}
}
}
@@ -500,18 +508,13 @@ pub fn (a array) repeat_to_depth(count int, depth int) array {
size = u64(a.element_size)
}
use_noscan_data := depth == 0 && a.uses_noscan_data()
mut data := unsafe { nil }
if use_noscan_data {
data = a.alloc_array_data_like(size)
} else {
data = alloc_array_data(size)
}
data := a.alloc_array_data_like(size)
arr := array{
element_size: a.element_size
data: data
len: count * a.len
cap: count * a.len
flags: if use_noscan_data { .managed | .noscan_data } else { .managed }
flags: a.storage_flags(use_noscan_data)
}
if a.len > 0 {
a_total_size := u64(a.len) * u64(a.element_size)
@@ -535,22 +538,22 @@ pub fn (a array) repeat_to_depth(count int, depth int) array {
}
@[inline]
fn (a array) needs_unique_shift(required int) bool {
fn (a &array) needs_unique_shift(required int) bool {
return required <= a.cap && (a.flags.has(.is_slice) || a.buffer_has_slices())
}
@[inline]
fn (a array) needs_unique_append(required int) bool {
fn (a &array) needs_unique_append(required int) bool {
return required <= a.cap && a.flags.has(.is_slice)
}
@[inline]
fn (a array) needs_unique_shrink() bool {
fn (a &array) needs_unique_shrink() bool {
return a.flags.has(.is_slice) || a.buffer_has_slices()
}
@[inline]
fn (a array) is_slice_view() bool {
fn (a &array) is_slice_view() bool {
return a.flags.has(.is_slice)
}
@@ -1030,7 +1033,8 @@ fn (a array) slice(start int, _end int) array {
offset := u64(start) * u64(a.element_size)
data := unsafe { &u8(a.data) + offset }
l := end - start
mut flags := ArrayFlags.is_slice | (a.flags & ArrayFlags.managed)
mut flags := unsafe { ArrayFlags(u32(a.flags) & (array_alignment_mask | u32(ArrayFlags.managed))) }
unsafe { flags.set(.is_slice) }
if a.uses_noscan_data() {
unsafe { flags.set(.noscan_data) }
}
@@ -1053,7 +1057,8 @@ fn (a array) slice(start int, _end int) array {
// This function always return a valid array.
fn (a array) slice_ni(_start int, _end int) array {
unsafe { a.mark_buffer_has_slices() }
mut flags := ArrayFlags.is_slice | (a.flags & ArrayFlags.managed)
mut flags := unsafe { ArrayFlags(u32(a.flags) & (array_alignment_mask | u32(ArrayFlags.managed))) }
unsafe { flags.set(.is_slice) }
if a.uses_noscan_data() {
unsafe { flags.set(.noscan_data) }
}
@@ -1137,9 +1142,9 @@ pub fn (a &array) clone_to_depth(depth int) array {
data = a.alloc_array_data_like(source_capacity_in_bytes)
}
} else if copies_capacity {
data = alloc_array_data_uninit(source_capacity_in_bytes)
data = a.alloc_array_data_like_uninit(source_capacity_in_bytes)
} else {
data = alloc_array_data(source_capacity_in_bytes)
data = a.alloc_array_data_like(source_capacity_in_bytes)
}
}
mut arr := array{
@@ -1147,7 +1152,7 @@ pub fn (a &array) clone_to_depth(depth int) array {
data: data
len: a.len
cap: a.cap
flags: if use_noscan_data { .managed | .noscan_data } else { .managed }
flags: a.storage_flags(use_noscan_data)
}
// Recursively clone-generated elements if array element is array type
if depth > 0 && a.element_size == sizeof(array) && a.len >= 0 && a.cap >= a.len {
@@ -1373,7 +1378,7 @@ pub fn (a array) reverse() array {
data: a.alloc_array_data_like(u64(a.cap) * u64(a.element_size))
len: a.len
cap: a.cap
flags: if use_noscan_data { .managed | .noscan_data } else { .managed }
flags: a.storage_flags(use_noscan_data)
}
for i in 0 .. a.len {
unsafe { arr.set_unsafe(i, a.get_unsafe(a.len - 1 - i)) }
@@ -1404,10 +1409,15 @@ pub fn (a &array) free() {
mblock_ptr := &u8(usize(a.data) - usize(a.offset))
if mblock_ptr != unsafe { nil } {
unsafe {
if a.flags.has(.managed) {
free(a.data_header().allocation)
allocation := if a.flags.has(.managed) {
a.data_header().allocation
} else {
free(mblock_ptr)
voidptr(mblock_ptr)
}
if a.flags.has(.noslices) {
free_unaliased(allocation)
} else {
free(allocation)
}
}
}
+97
View File
@@ -0,0 +1,97 @@
module builtin
const array_alignment_shift = sizeof(ArrayFlags) * 8 - 8
const array_alignment_mask = u32(63) << array_alignment_shift
@[inline]
fn (a &array) data_alignment() usize {
exponent := (u32(a.flags) & array_alignment_mask) >> array_alignment_shift
return if exponent == 0 { usize(16) } else { usize(1) << exponent }
}
fn data_alignment_exponent(alignment usize) u8 {
if alignment <= 16 {
return 0
}
mut exponent := u8(0)
mut remaining := alignment
for remaining > 1 {
remaining >>= 1
exponent++
}
return exponent
}
fn (mut a array) set_data_alignment(alignment usize) {
exponent := u32(data_alignment_exponent(alignment))
bits := u32(a.flags) & ~array_alignment_mask
a.flags = unsafe { ArrayFlags(bits | (exponent << array_alignment_shift)) }
}
@[inline]
fn (a &array) storage_flags(noscan bool) ArrayFlags {
mut flags := u32(a.flags) & array_alignment_mask
flags |= u32(ArrayFlags.managed)
if noscan {
flags |= u32(ArrayFlags.noscan_data)
}
return unsafe { ArrayFlags(flags) }
}
fn alloc_array_data_aligned(total_size u64, alignment usize, zero bool) voidptr {
size := u64(array_data_header_size()) + u64(alignment - 1)
allocation_size := size + __at_least_one(total_size)
raw := if zero {
vcalloc(allocation_size)
} else {
unsafe { malloc_uninit(allocation_size) }
}
header_size := usize(array_data_header_size())
padding := (alignment - (usize(raw) + header_size) % alignment) % alignment
unsafe {
data := &u8(raw) + header_size + padding
header := &ArrayDataHeader(data - header_size)
header.allocation = raw
header.has_slices = false
header.retained_fixed_views = false
return data
}
}
fn __new_array_aligned(mylen int, capacity int, element_size int, alignment usize) array {
if alignment <= 16 {
return __new_array(mylen, capacity, element_size)
}
panic_on_negative_len(mylen)
panic_on_negative_cap(capacity)
mut result := array{
element_size: element_size
len: mylen
cap: if capacity < mylen { mylen } else { capacity }
flags: .managed
}
result.set_data_alignment(alignment)
if result.cap > 0 {
size := u64(result.cap) * u64(element_size)
result.data = alloc_array_data_aligned(size, alignment, mylen > 0)
}
return result
}
fn new_array_from_c_array_aligned(length int, capacity int, element_size int, source voidptr, alignment usize) array {
panic_on_negative_len(length)
panic_on_negative_cap(capacity)
mut result := __new_array_aligned(0, if capacity < length { length } else { capacity },
element_size, alignment)
result.len = length
if length > 0 {
unsafe { vmemcpy(result.data, source, u64(length) * u64(element_size)) }
}
return result
}
fn new_array_from_c_array_no_alloc_aligned(length int, capacity int, element_size int, source voidptr, alignment usize) array {
mut result := new_array_from_c_array_no_alloc(length, capacity, element_size, source)
result.set_data_alignment(alignment)
return result
}
+22
View File
@@ -0,0 +1,22 @@
module builtin
fn test_array_alignment_does_not_overlap_retained_storage_flags() {
for alignment in [usize(16), 32, 64, 128, 4096] {
mut value := array{}
value.set_data_alignment(alignment)
value.mark_aligned_fixed_array_buffer()
assert value.data_alignment() == alignment
assert value.buffer_has_slices()
value.set_data_alignment(64)
assert value.data_alignment() == 64
assert value.flags.has(array_flag_retained_aligned_fixed)
}
}
fn test_aligned_array_header_initializes_retained_views() {
mut value := __new_array_aligned(0, 4, int(sizeof(u64)), 64)
assert usize(value.data) % 64 == 0
assert !value.buffer_has_retained_fixed_views()
assert !value.buffer_has_slices()
unsafe { value.free() }
}
+2 -1
View File
@@ -285,13 +285,14 @@ fn map_map_eq(a map, b map) bool {
return true
}
@[inline]
@[inline; manualfree]
fn map_clone_string(dest voidptr, pkey voidptr) {
unsafe {
s := *&string(pkey)
cloned := s.clone()
// Use memcpy for native backend compatibility
// (*&string(dest)) = cloned doesn't reliably store full struct
// The destination owns cloned; autofree must not release the transferred key.
vmemcpy(dest, voidptr(&cloned), sizeof(string))
}
}
+103
View File
@@ -0,0 +1,103 @@
module builtin
@[aligned: 64]
struct AlignedCloneCell {
value u64
}
fn clone_aligned_map_key(destination voidptr, source voidptr) {
// The raw map callback copies the complete aligned key, including its padding.
unsafe { vmemcpy(destination, source, sizeof(AlignedCloneCell)) }
}
fn assert_aligned_map_clone_storage(value &map) {
assert value.data.key_values.key_alignment == 6
assert value.data.key_values.value_alignment == 6
if value.data.count == 0 {
return
}
assert usize(value.data.key_values.keys) % 64 == 0
assert usize(value.data.key_values.values) % 64 == 0
keys := value.keys()
values := value.values()
assert keys.data_alignment() == 64
assert values.data_alignment() == 64
assert usize(keys.data) % 64 == 0
assert usize(values.data) % 64 == 0
for i in 0 .. keys.len {
// Raw map exports have the same aligned element layout as the input cells.
key := unsafe { &AlignedCloneCell(&u8(keys.data) + i * int(sizeof(AlignedCloneCell))) }
item := unsafe { &AlignedCloneCell(&u8(values.data) + i * int(sizeof(AlignedCloneCell))) }
assert usize(key) % 64 == 0
assert usize(item) % 64 == 0
assert key.value == u64(i + 1)
assert item.value == key.value * 10
zero := AlignedCloneCell{}
stored := unsafe { &AlignedCloneCell(value.get(key, &zero)) }
assert usize(stored) % 64 == 0
assert stored.value == item.value
}
unsafe {
keys.free()
values.free()
}
}
fn test_map_clones_preserve_key_and_value_alignment_in_every_storage_path() {
assert sizeof(AlignedCloneCell) == 64
for reservation in [u32(0), 256] {
mut original := new_map_aligned(int(sizeof(AlignedCloneCell)),
int(sizeof(AlignedCloneCell)), map_hash_int_8, map_eq_int_8,
clone_aligned_map_key, map_free_nop, 64, 64)
original.reserve(reservation)
mut empty := unsafe { original.clone() }
assert_aligned_map_clone_storage(&empty)
for n in 1 .. 4 {
key := AlignedCloneCell{ value: u64(n) }
item := AlignedCloneCell{ value: u64(n * 10) }
original.set(&key, &item)
}
mut copied := unsafe { original.clone() }
if reservation == 256 {
assert original.data.even_index >= 128
assert copied.data.even_index < original.data.even_index
}
assert_aligned_map_clone_storage(&copied)
mut moved := original.move()
assert_aligned_map_clone_storage(&moved)
assert_aligned_map_clone_storage(&original)
key := AlignedCloneCell{ value: 1 }
item := AlignedCloneCell{ value: 10 }
original.set(&key, &item)
assert_aligned_map_clone_storage(&original)
unsafe {
empty.free()
copied.free()
moved.free()
original.free()
}
}
}
fn test_sparse_typed_map_clone_keeps_aligned_values() {
mut original := map[string]AlignedCloneCell{}
original.reserve(256)
original['only'] = AlignedCloneCell{ value: 42 }
mut copied := original.clone()
assert copied['only'].value == 42
// Inspect each map element's address before any subsequent map mutation.
first_address := unsafe { voidptr(&copied['only']) }
assert usize(first_address) % 64 == 0
values := copied.values()
assert usize(values.data) % 64 == 0
assert values[0].value == 42
copied['next'] = AlignedCloneCell{ value: 99 }
next_address := unsafe { voidptr(&copied['next']) }
assert usize(next_address) % 64 == 0
assert 'next' !in original
unsafe {
values.free()
copied.free()
original.free()
}
}
+107 -54
View File
@@ -78,9 +78,11 @@ struct DenseArray {
key_bytes int
value_bytes int
mut:
cap int
len int
deletes u32 // count
cap int
len int
deletes u32 // count
key_alignment u8
value_alignment u8
// array allocated (with `cap` bytes) on first deletion
// has non-zero element when key deleted
all_deleted &u8 = unsafe { nil }
@@ -89,17 +91,19 @@ mut:
}
@[inline]
fn new_dense_array(key_bytes int, value_bytes int) DenseArray {
fn new_dense_array(key_bytes int, value_bytes int, key_alignment u8, value_alignment u8) DenseArray {
cap := 8
return DenseArray{
key_bytes: key_bytes
value_bytes: value_bytes
cap: cap
len: 0
deletes: 0
all_deleted: unsafe { nil }
keys: unsafe { malloc(__at_least_one(u64(cap) * u64(key_bytes))) }
values: unsafe { malloc(__at_least_one(u64(cap) * u64(value_bytes))) }
key_bytes: key_bytes
value_bytes: value_bytes
key_alignment: key_alignment
value_alignment: value_alignment
cap: cap
len: 0
deletes: 0
all_deleted: unsafe { nil }
keys: dense_array_allocate(u64(cap) * u64(key_bytes), key_alignment)
values: dense_array_allocate(u64(cap) * u64(value_bytes), value_alignment)
}
}
@@ -148,10 +152,10 @@ fn (mut d DenseArray) reserve(n int) {
old_value_size := d.value_bytes * old_cap
d.cap = n
unsafe {
d.keys = realloc_data(d.keys, old_key_size, d.key_bytes * d.cap)
// Zero-sized values keep the initial one-byte allocation across growth.
d.keys = dense_array_resize(d.keys, old_key_size, d.key_bytes * d.cap, d.key_alignment)
// Zero-sized values retain their initial one-byte allocation across growth.
if d.value_bytes != 0 {
d.values = realloc_data(d.values, old_value_size, d.value_bytes * d.cap)
d.values = dense_array_resize(d.values, old_value_size, d.value_bytes * d.cap, d.value_alignment)
}
if d.deletes != 0 {
d.all_deleted = realloc_data(d.all_deleted, old_cap, d.cap)
@@ -165,25 +169,12 @@ fn (mut d DenseArray) reserve(n int) {
// storage and copying. Other allocators use the compact 1.125 growth factor.
@[inline]
fn (mut d DenseArray) expand() int {
old_cap := d.cap
old_key_size := d.key_bytes * old_cap
old_value_size := d.value_bytes * old_cap
if d.cap == d.len {
mut growth := d.cap >> 3
$if prealloc {
d.cap += d.cap
} $else {
d.cap += d.cap >> 3
}
unsafe {
d.keys = realloc_data(d.keys, old_key_size, d.key_bytes * d.cap)
if d.value_bytes != 0 {
d.values = realloc_data(d.values, old_value_size, d.value_bytes * d.cap)
}
if d.deletes != 0 {
d.all_deleted = realloc_data(d.all_deleted, old_cap, d.cap)
vmemset(voidptr(d.all_deleted + d.len), 0, d.cap - d.len)
}
growth = d.cap
}
d.reserve(d.cap + int_max(1, growth))
}
push_index := d.len
unsafe {
@@ -300,13 +291,14 @@ fn map_map_eq(a map, b map) bool {
return true
}
@[inline]
@[inline; manualfree]
fn map_clone_string(dest voidptr, pkey voidptr) {
unsafe {
s := *&string(pkey)
cloned := s.clone()
// Use memcpy for native backend compatibility
// (*&string(dest)) = cloned doesn't reliably store full struct
// The destination owns cloned; autofree must not release the transferred key.
vmemcpy(dest, voidptr(&cloned), sizeof(string))
}
}
@@ -414,6 +406,14 @@ fn new_map(key_bytes int, value_bytes int, hash_fn MapHashFn, key_eq_fn MapEqFn,
}
}
fn new_map_aligned(key_bytes int, value_bytes int, hash_fn MapHashFn, key_eq_fn MapEqFn,
clone_fn MapCloneFn, free_fn MapFreeFn, key_alignment usize, value_alignment usize) map {
mut result := new_map(key_bytes, value_bytes, hash_fn, key_eq_fn, clone_fn, free_fn)
result.data.key_values.key_alignment = data_alignment_exponent(key_alignment)
result.data.key_values.value_alignment = data_alignment_exponent(value_alignment)
return result
}
fn new_map_init(hash_fn MapHashFn, key_eq_fn MapEqFn, clone_fn MapCloneFn, free_fn MapFreeFn, n int, key_bytes int,
value_bytes int, keys voidptr, values voidptr) map {
mut out := new_map(key_bytes, value_bytes, hash_fn, key_eq_fn, clone_fn, free_fn)
@@ -452,6 +452,8 @@ pub fn (mut m map) move() map {
m.data = &map_empty_data
} else {
m.data = new_map_data(m.data.key_bytes, m.data.value_bytes, m.data.hash_fn, m.data.key_eq_fn, m.data.clone_fn, m.data.free_fn)
m.data.key_values.key_alignment = r.data.key_values.key_alignment
m.data.key_values.value_alignment = r.data.key_values.value_alignment
}
return r
}
@@ -596,7 +598,7 @@ fn (mut m VMapData) set(key voidptr, value voidptr) {
if m.metas == unsafe { nil } {
// Most compiler bookkeeping maps remain empty. Allocate backing storage
// only on the first insertion or an explicit reservation.
m.key_values = new_dense_array(m.key_bytes, m.value_bytes)
m.key_values = new_dense_array(m.key_bytes, m.value_bytes, m.key_values.key_alignment, m.key_values.value_alignment)
m.metas = unsafe { &u32(vcalloc_noscan(sizeof(u32) * (m.even_index + 2 + m.extra_metas))) }
}
// Integer-based load factor check: equivalent to (2*len)/even_index > 0.8
@@ -709,12 +711,7 @@ fn (mut m VMapData) reserve(n u32) {
if dense_cap > u64(max_int) {
panic('map.reserve: max_int will be exceeded')
}
// DenseArray's geometric growth starts at eight slots.
if dense_cap > 0 && dense_cap < 8 {
m.key_values.reserve(8)
} else {
m.key_values.reserve(int(dense_cap))
}
m.key_values.reserve(int(dense_cap))
}
// cached_rehashd works like rehash. However, instead of rehashing the
@@ -966,13 +963,55 @@ fn (mut m VMapData) delete(key voidptr) {
}
}
@[inline]
fn dense_array_allocate(size u64, alignment u8) &u8 {
if alignment == 0 {
return unsafe { malloc(__at_least_one(size)) }
}
return alloc_array_data_aligned(size, usize(1) << alignment, false)
}
@[inline; unsafe]
fn dense_array_free(data &u8, alignment u8) {
if alignment == 0 {
unsafe { free(data) }
return
}
unsafe {
header := &ArrayDataHeader(data - array_data_header_size())
free(header.allocation)
}
}
@[inline; unsafe]
fn dense_array_resize(data &u8, previous int, requested int, alignment u8) &u8 {
if alignment == 0 {
return unsafe { realloc_data(data, previous, requested) }
}
result := dense_array_allocate(u64(requested), alignment)
if data == unsafe { nil } {
return result
}
count := if previous < requested { previous } else { requested }
if count > 0 {
unsafe { vmemcpy(result, data, count) }
}
unsafe { dense_array_free(data, alignment) }
return result
}
// keys returns all keys in the map.
pub fn (m &map) keys() array {
return m.data.keys()
}
fn (m &VMapData) keys() array {
mut keys := __new_array(m.count, 0, m.key_bytes)
alignment := m.key_values.key_alignment
mut keys := if alignment == 0 {
__new_array(m.count, 0, m.key_bytes)
} else {
__new_array_aligned(m.count, 0, m.key_bytes, usize(1) << alignment)
}
mut item := unsafe { &u8(keys.data) }
if m.key_values.deletes == 0 {
for i := 0; i < m.key_values.len; i++ {
@@ -1003,7 +1042,12 @@ pub fn (m &map) values() array {
}
fn (m &VMapData) values() array {
mut values := __new_array(m.count, 0, m.value_bytes)
alignment := m.key_values.value_alignment
mut values := if alignment == 0 {
__new_array(m.count, 0, m.value_bytes)
} else {
__new_array_aligned(m.count, 0, m.value_bytes, usize(1) << alignment)
}
if m.count == 0 {
return values
}
@@ -1033,21 +1077,25 @@ fn (m &VMapData) values() array {
@[unsafe]
fn (d &DenseArray) clone() DenseArray {
res := DenseArray{
key_bytes: d.key_bytes
value_bytes: d.value_bytes
cap: d.cap
len: d.len
deletes: d.deletes
all_deleted: unsafe { nil }
values: unsafe { nil }
keys: unsafe { nil }
key_bytes: d.key_bytes
value_bytes: d.value_bytes
key_alignment: d.key_alignment
value_alignment: d.value_alignment
cap: d.cap
len: d.len
deletes: d.deletes
all_deleted: unsafe { nil }
values: unsafe { nil }
keys: unsafe { nil }
}
unsafe {
if d.deletes != 0 {
res.all_deleted = memdup(d.all_deleted, d.cap)
}
res.keys = memdup(d.keys, d.cap * d.key_bytes)
res.values = memdup(d.values, d.cap * d.value_bytes)
res.keys = dense_array_allocate(u64(d.cap) * u64(d.key_bytes), d.key_alignment)
vmemcpy(res.keys, d.keys, d.cap * d.key_bytes)
res.values = dense_array_allocate(u64(d.cap) * u64(d.value_bytes), d.value_alignment)
vmemcpy(res.values, d.values, d.cap * d.value_bytes)
}
return res
}
@@ -1070,7 +1118,10 @@ fn (m &VMapData) clone() &VMapData {
// clear() and deleting every key can leave large backing slabs behind.
// An empty clone needs its own writable header, but none of that capacity.
if m.count == 0 || m.metas == unsafe { nil } {
return new_map_data(m.key_bytes, m.value_bytes, m.hash_fn, m.key_eq_fn, m.clone_fn, m.free_fn)
mut result := new_map_data(m.key_bytes, m.value_bytes, m.hash_fn, m.key_eq_fn, m.clone_fn, m.free_fn)
result.key_values.key_alignment = m.key_values.key_alignment
result.key_values.value_alignment = m.key_values.value_alignment
return result
}
// Reused checker maps can retain a large reservation after their contents
// shrink. Rebuilding a sparse clone copies only live entries, in their
@@ -1078,6 +1129,8 @@ fn (m &VMapData) clone() &VMapData {
if m.even_index >= 128 && u64(m.count) * 8 < u64(m.even_index) {
mut compact := new_map_data(m.key_bytes, m.value_bytes, m.hash_fn, m.key_eq_fn,
m.clone_fn, m.free_fn)
compact.key_values.key_alignment = m.key_values.key_alignment
compact.key_values.value_alignment = m.key_values.value_alignment
compact.reserve(u32(m.count))
for i in 0 .. m.key_values.len {
if m.key_values.has_index(i) {
@@ -1170,14 +1223,14 @@ fn (m &VMapData) free() {
$if prealloc {
prealloc_discard_pages(m.key_values.keys, usize(m.key_values.cap) * usize(m.key_bytes))
}
free(m.key_values.keys)
dense_array_free(m.key_values.keys, m.key_values.key_alignment)
m.key_values.keys = nil
}
if m.key_values.values != nil {
$if prealloc {
prealloc_discard_pages(m.key_values.values, usize(m.key_values.cap) * usize(m.value_bytes))
}
free(m.key_values.values)
dense_array_free(m.key_values.values, m.key_values.value_alignment)
m.key_values.values = nil
}
// TODO: the next lines assume that callback functions are static and independent from each particular
+21
View File
@@ -193,3 +193,24 @@ fn test_empty_map_reserve_after_deletion_and_clear() {
m['next'] = 4
assert m.keys() == ['cleared', 'next']
}
fn test_small_map_reservations_survive_growth_and_deleted_slots() {
for capacity in [1, 2, 3, 5, 7, 8, 9] {
mut values := map[int]int{}
values.reserve(u32(capacity))
for key in 0 .. 32 {
values[key] = key * 3
}
for key in 0 .. 16 {
values.delete(key)
}
values.reserve(64)
for key in 32 .. 80 {
values[key] = key * 3
}
assert values.len == 64
for key in 16 .. 80 {
assert values[key] == key * 3
}
}
}
+9 -1
View File
@@ -38,7 +38,15 @@ fn test_empty_map_values_survive_growth_reserve_and_deletion() {
fn test_dense_array_zero_value_size_with_every_c_compiler() {
// TinyCC assigns empty structs one byte, so exercise zero-sized storage explicitly too.
mut dense := new_dense_array(int(sizeof(int)), 0)
// Use the fields common to both map backends; their private constructors differ.
// The buffers cover all initial keys and one byte for the zero-sized values.
mut dense := DenseArray{
key_bytes: int(sizeof(int))
value_bytes: 0
cap: 8
keys: unsafe { malloc(8 * isize(sizeof(int))) }
values: unsafe { malloc(1) }
}
// DenseArray stores raw key bytes; fill and inspect its integer slots directly.
for i in 0 .. 100 {
index := dense.expand()
+12
View File
@@ -37,6 +37,18 @@
`json2` is an experimental JSON parser written from scratch on V.
## Typed map keys
Integer map keys are parsed as the declared integer type with range checks. Invalid
numeric text and values outside that type's range return errors, including with the
default decoding options. For example, decoding `{"256":1}` as `map[u8]int` fails.
A `rune` map key is a string containing exactly one Unicode character. Encoding and
decoding preserve that character, including non-ASCII keys. Code that used numeric
text such as `"65"` for the code point of `A` should use `"A"` instead. A digit key
such as `"1"` means the character `1`; empty and multi-character keys are rejected.
`json2.decode[map[rune]int](json2.encode(values))` round-trips the rune keys.
## JSON5
Set `json5: true` on the existing encoding or decoding options to use JSON5:
+2 -2
View File
@@ -28,7 +28,7 @@ fn (mut checker Decoder) check_json_format() ! {
start_idx_position := checker.checker_idx
// The values nested in this one are added after it, while it is checked.
value_idx := checker.values_len
value_idx := checker.values_info.len
match checker.json[checker.checker_idx] {
`"` {
@@ -227,7 +227,7 @@ fn (mut checker Decoder) check_null() ! {
@[markused]
fn (mut checker Decoder) check_nested_value(message string) ! {
value_idx := checker.values_len
value_idx := checker.values_info.len
checker.check_json_format()!
value := checker.values_info[value_idx]
// A nested value cannot consume the final byte: its parent still needs a
+173 -143
View File
@@ -47,12 +47,13 @@ fn struct_field_info(field_name string, attrs []string) StructFieldInfo {
mut json_name_str := field_name.str
mut json_name_len := field_name.len
mut is_json_skip := false
for attr in attrs {
for i in 0 .. attrs.len {
attr := &attrs[i]
if start, end := json_attr_value_range(attr) {
if end <= start {
continue
}
if end == start + 1 && attr[start] == `-` {
if end == start + 1 && (*attr)[start] == `-` {
is_json_skip = true
break
}
@@ -123,7 +124,6 @@ mut:
// values. It is one flat array, so nothing is allocated per value, and moving to
// the next value is an index increment.
values_info []ValueInfo
values_len int // values_len is the number of values that the checker has put in values_info.
checker_idx int // checker_idx is the current index of the decoder.
current_idx int // current_idx is the index in values_info of the value that is decoded next.
}
@@ -221,15 +221,19 @@ fn word_has_byte(word u64, b u8) bool {
// values_info. Its length is set once its end is known.
@[direct_array_access; inline; markused]
fn (mut checker Decoder) add_value(value_kind ValueKind) {
if checker.values_len >= checker.values_info.len {
// value_count_bound() leaves room for every value of a valid JSON string. In
// an invalid one the checker can see more values, before it reports the error.
checker.values_info << ValueInfo{}
if checker.values_info.len == checker.values_info.cap
&& !checker.values_info.flags.has(.managed) {
capacity := value_count_bound(checker.json)
unsafe { checker.values_info.grow_cap(capacity - checker.values_info.len) }
}
checker.values_info << ValueInfo{
position: checker.checker_idx
value_kind: value_kind
has_escape: false
}
if checker.values_info.flags.has(.managed) {
unsafe { checker.values_info.flags |= .noslices }
}
checker.values_info[checker.values_len].position = checker.checker_idx
checker.values_info[checker.values_len].value_kind = value_kind
checker.values_info[checker.values_len].has_escape = false
checker.values_len++
}
const max_context_length = 50
@@ -421,13 +425,12 @@ fn decoder_for_text(val string, params DecoderOptions) !Decoder {
mut decoder := Decoder{
json: val
strict: params.strict
values_info: []ValueInfo{len: value_count_bound(val)}
values_info: []ValueInfo{cap: value_count_bound(val)}
}
decoder.check_json_format() or {
unsafe { decoder.values_info.free() }
return err
}
decoder.values_info.trim(decoder.values_len)
return decoder
}
@@ -480,8 +483,8 @@ pub fn decode_reuse[T](val string, mut buffer DecodeBuffer, params DecoderOption
buffer.values_info = unsafe { decoder.values_info }
unsafe { decoder.source_values_info.free() }
}
decoder.values_info.clear()
decoder.check_json_format()!
decoder.values_info.trim(decoder.values_len)
return decoder.decode_root[T]()
}
@@ -494,7 +497,7 @@ fn (mut decoder Decoder) decode_root[T]() !T {
decoder.decode_map(mut result)!
} $else $if T is $pointer {
// `&T`, `&&T` and `&&&T` point to a newly decoded value; `null` keeps `result`.
result = decoder.decode_array_element(result)!
decoder.decode_element[T](mut result)!
} $else {
decoder.decode_value(mut result)!
}
@@ -564,7 +567,7 @@ fn decoder_field_infos[T]() []DecoderFieldInfo {
if end <= start {
continue
}
if end == start + 1 && attr[start] == `-` {
if end == start + 1 && (*attr)[start] == `-` {
is_json_skip = true
break
}
@@ -664,17 +667,46 @@ fn (mut decoder Decoder) json_key_matches(key_info ValueInfo, key_name string) !
// skip_current_value moves past the current value, and past the values nested in it:
// those follow it in values_info, and start before its end.
@[direct_array_access; markused]
fn (decoder &Decoder) next_value(index int) int {
if index >= decoder.values_info.len {
return decoder.values_info.len
}
value := decoder.values_info[index]
end := value.position + value.length
mut next := index + 1
mut limit := decoder.values_info.len
if next == limit || decoder.values_info[next].position >= end {
return next
}
for next < limit {
middle := next + (limit - next) / 2
if decoder.values_info[middle].position < end {
next = middle + 1
} else {
limit = middle
}
}
return next
}
fn (decoder &Decoder) container_len(index int) int {
value := decoder.values_info[index]
end := value.position + value.length
mut next := index + 1
mut count := 0
for next < decoder.values_info.len {
if decoder.values_info[next].position >= end {
break
}
count++
next = decoder.next_value(next)
}
return if value.value_kind == .object { count / 2 } else { count }
}
@[markused]
fn (mut decoder Decoder) skip_current_value() {
if decoder.current_idx >= decoder.values_info.len {
return
}
value_info := decoder.values_info[decoder.current_idx]
value_end := value_info.position + value_info.length
mut next_idx := decoder.current_idx + 1
for next_idx < decoder.values_info.len && decoder.values_info[next_idx].position < value_end {
next_idx++
}
decoder.current_idx = next_idx
decoder.current_idx = decoder.next_value(decoder.current_idx)
}
@[manualfree]
@@ -1295,7 +1327,7 @@ fn (mut decoder Decoder) decode_string_value(string_info ValueInfo) !string {
string_start := string_info.position + 1
string_end := string_info.position + string_info.length - 1
if !string_info.has_escape {
return decoder.retained_text(string_start, string_end)
return decoder.retained_text(string_start, string_end).clone()
}
mut string_buffer := []u8{cap: string_info.length} // might be too long but most json strings don't contain many escape characters anyways
@@ -1397,46 +1429,70 @@ fn (mut decoder Decoder) decode_string_value(string_info ValueInfo) !string {
}
fn (mut decoder Decoder) decode_array[T](mut val []T) ! {
$if T is $interface {
$if T is $interface || T.unaliased_typ is voidptr {
decoder.skip_current_value()
return
} $else $if T.unaliased_typ is voidptr {
decoder.skip_current_value()
}
array_info := decoder.values_info[decoder.current_idx]
if array_info.value_kind == .null && !decoder.strict {
decoder.current_idx++
return
} $else {
array_info := decoder.current_value()
}
if array_info.value_kind != .array {
return decoder.decode_error('Expected array, but got ${array_info.value_kind}')
}
capacity := decoder.container_len(decoder.current_idx)
mut decoded := []T{cap: capacity}
decoder.current_idx++
if array_info.value_kind == .array {
decoder.current_idx++
array_end := array_info.position + array_info.length
array_position := array_info.position
array_end := array_position + array_info.length
for {
if decoder.current_idx >= decoder.values_info.len
|| decoder.values_info[decoder.current_idx].position >= array_end {
break
}
for {
if decoder.current_idx >= decoder.values_info.len
|| decoder.current_value().position >= array_end {
break
}
$if T is $option {
// An option element (`[]?int`): `null` is `none`. Decoded directly,
// since v3 cannot return `!?T` from the element helper.
if decoder.option_element_is_none(T(none)) {
val << T(none)
decoder.skip_current_value()
} else {
val << decoder.decode_option_payload(T(none))!
}
} $else {
val << decoder.decode_array_element(T{})!
}
$if T is $option {
// An option element (`[]?int`): `null` is `none`. Decoded directly,
// since v3 cannot return `!?T` from the element helper.
if decoder.option_element_is_none(T(none)) {
decoded << T(none)
decoder.skip_current_value()
} else {
decoded << decoder.decode_option_payload(T(none))!
}
} else if array_info.value_kind == .null && !decoder.strict {
// Outside of strict mode `null` is an empty array, like in the removed module.
} $else {
mut element := T{}
decoder.decode_element[T](mut element)!
decoded << element
}
}
val = decoded
}
fn (mut decoder Decoder) decode_element[E](mut element E) ! {
$if E is $interface {
decoder.skip_current_value()
} $else $if E.unaliased_typ is voidptr {
decoder.skip_current_value()
} $else $if E.indirections == 1 {
if decoder.values_info[decoder.current_idx].value_kind == .null {
decoder.current_idx++
} else {
decoder.decode_error('Expected array, but got ${array_info.value_kind}')!
mut decoded_ptr := create_decoded_ptr(element)
decoder.decode_value(mut decoded_ptr)!
element = decoded_ptr
}
} $else $if E.indirections > 1 {
// `&&T`, `&&&T`, ... elements point to a newly decoded value, like fields.
if decoder.values_info[decoder.current_idx].value_kind == .null {
decoder.current_idx++
} else {
element = decoder.decode_new_pointer(element)!
}
} $else {
decoder.decode_value(mut element)!
}
}
@@ -1571,96 +1627,70 @@ fn (mut decoder Decoder) decode_enum_map_key[K](key_str string) !K {
return zero
}
fn (mut decoder Decoder) decode_map[K, V](mut val map[K]V) ! {
$if V is $interface {
decoder.skip_current_value()
return
} $else $if V.unaliased_typ is voidptr {
decoder.skip_current_value()
return
fn (mut decoder Decoder) decode_map_key[K](info ValueInfo) !K {
text := if decoder.key_has_escape(info) {
decoder.decode_string_value(info)!
} else {
decoder.json.substr_unsafe(info.position + 1, info.position + info.length - 1)
}
$if K.unaliased_typ is string {
return K(text)
} $else $if K.unaliased_typ is $enum {
return decoder.decode_enum_map_key[K](text)!
} $else $if K.unaliased_typ is rune {
mut runes := text.runes_iterator()
first := runes.next() or { return error('expected one rune for JSON key') }
if _ := runes.next() {
return error('expected one rune for JSON key')
}
return K(first)
} $else $if K.unaliased_typ is $int {
return parse_integer_number[K](text)!
} $else {
map_info := decoder.current_value()
if map_info.value_kind == .object {
map_position := map_info.position
map_end := map_position + map_info.length
decoder.current_idx++
for {
if decoder.current_idx >= decoder.values_info.len
|| decoder.current_value().position >= map_end {
break
}
key_info := decoder.current_value()
if key_info.position >= map_end {
break
}
key_str := if decoder.key_has_escape(key_info) {
decoder.decode_string_value(key_info)!
} else {
// Map insertion clones string keys before the normalized document is freed.
decoder.json[key_info.position + 1..key_info.position + key_info.length - 1]
}
mut key := K{}
$if K is string {
key = K(key_str)
} $else $if K.unaliased_typ is $enum {
key = decoder.decode_enum_map_key[K](key_str)!
} $else $if K is rune {
key = K(key_str.int())
} $else $if K is u8 || K is u16 || K is u32 || K is u64 || K is usize {
key = K(key_str.u64())
} $else $if K is i64 || K is isize {
key = K(key_str.i64())
} $else $if K is $int {
key = K(key_str.int())
} $else {
key = K(key_str)
}
return error('unsupported JSON map key type ${K.name}')
}
}
fn (mut decoder Decoder) decode_map[K, V](mut val map[K]V) ! {
$if V is $interface || V.unaliased_typ is voidptr {
decoder.skip_current_value()
return
}
map_info := decoder.values_info[decoder.current_idx]
if map_info.value_kind == .null && !decoder.strict {
val.clear()
decoder.current_idx++
return
}
if map_info.value_kind != .object {
return decoder.decode_error('Expected object, but got ${map_info.value_kind}')
}
map_end := map_info.position + map_info.length
capacity := val.len + decoder.container_len(decoder.current_idx)
val.reserve(u32(capacity))
decoder.current_idx++
for decoder.current_idx < decoder.values_info.len {
key_info := decoder.values_info[decoder.current_idx]
if key_info.position >= map_end {
break
}
key := decoder.decode_map_key[K](key_info)!
decoder.current_idx++
$if V is $option {
if decoder.values_info[decoder.current_idx].value_kind == .null {
val[key] = V(none)
decoder.current_idx++
value_info := decoder.current_value()
if value_info.position + value_info.length > map_end {
break
}
$if V is $option {
// An option map value: `null` is `none`.
if decoder.current_value().value_kind == .null {
val[key] = V(none)
decoder.current_idx++
} else {
val[key] = decoder.decode_option_payload(V(none))!
}
continue
}
mut map_value := V{}
$if V is $pointer {
map_value = decoder.decode_array_element(map_value)!
} $else {
decoder.decode_value(mut map_value)!
}
// Map alias values (`type Props = map[string]int`) also need to move.
$if V is $map || (V is $alias && V.unaliased_typ is $map) {
val[key] = map_value.move()
} $else {
val[key] = map_value
}
} else {
val[key] = decoder.decode_option_payload(V(none))!
}
} else if map_info.value_kind == .null && !decoder.strict {
// Outside of strict mode `null` is an empty map, like in the removed module.
val.clear()
decoder.current_idx++
} else {
decoder.decode_error('Expected object, but got ${map_info.value_kind}')!
continue
}
mut map_value := V{}
decoder.decode_element[V](mut map_value)!
$if V.unaliased_typ is $map {
val[key] = map_value.move()
} $else {
val[key] = map_value
}
}
}
+207
View File
@@ -0,0 +1,207 @@
module json2
fn test_decode_scratch_storage_spills_and_releases_on_errors() {
for count in [0, 1, 7, 8, 9, 31, 32, 33, 1024] {
values := []string{len: count, init: '"entry"'}
source := '[' + values.join(',') + ']'
decoded := decode[[]string](source)!
assert decoded == []string{len: count, init: 'entry'}
invalid := source[..source.len - 1] + ',]'
if _ := decode[[]string](invalid) {
assert false, invalid
}
assert decode[string]('"after error"')! == 'after error'
}
}
fn test_flat_metadata_survives_growth_and_tracks_nested_spans() {
document := '{"name":"entry","items":[true,false,null]}'
documents := []string{len: 2048, init: document}
source := '[' + documents.join(',') + ']'
mut decoder := Decoder{ json: source }
decoder.check_json_format()!
assert decoder.values_info.len == 1 + 8 * documents.len
assert decoder.values_info[0].length == source.len
assert decoder.values_info[1].length == document.len
assert decoder.next_value(1) == 9
assert decoder.next_value(5) == 9
assert decoder.next_value(0) == decoder.values_info.len
expected := [
ValueKind.array,
.object,
.string,
.string,
.string,
.array,
.boolean,
.boolean,
.null,
]
for index, kind in expected {
assert decoder.values_info[index].value_kind == kind
}
for value in decoder.values_info {
assert value.length > 0
assert value.position + value.length <= source.len
}
decoded := decode[Any](source)!
// Encoding sorts object names lexically; metadata still follows input order.
canonical := []string{len: documents.len, init: '{"items":[true,false,null],"name":"entry"}'}
assert encode(decoded) == '[' + canonical.join(',') + ']'
}
fn test_flat_metadata_discriminator_skips_nested_values() {
source := '{"nested":{"_type":"Wrong"},"_type":"Right"}'
mut decoder := Decoder{ json: source }
decoder.check_json_format()!
index := decoder.get_sumtype_type_field_idx(0)
assert index < decoder.values_info.len
assert decoder.decode_string_value(decoder.values_info[index])! == 'Right'
decoder.skip_current_value()
assert decoder.current_idx == decoder.values_info.len
}
fn test_decoded_values_own_borrowed_input_fragments() {
mut source := '{"key":42,"escaped\\tkey":"a\\tb","plain":"text"}'.clone()
decoded := decode[map[string]Any](source)!
unsafe { source.free() }
assert (decoded['key'] or { panic('missing key') }) == Any(f64(42))
assert (decoded['escaped\tkey'] or { panic('missing key') }) == Any('a\tb')
assert (decoded['plain'] or { panic('missing key') }) == Any('text')
}
fn test_escaped_strings_survive_scratch_release() {
inputs := ['"plain"', '"line\\nend"', '"\\u4e16\\u754c"', '"\\uD83D\\uDE00"']
expected := ['plain', 'line\nend', '世界', '😀']
mut actual := []string{}
for input in inputs {
mut source := input.clone()
actual << decode[string](source)!
unsafe { source.free() }
}
for _ in 0 .. 100 {
assert decode[string]('"replacement\\ntext"')! == 'replacement\ntext'
}
assert actual == expected
}
fn test_encoded_output_survives_builder_release_and_reuse() {
item := 'text\n世界'.repeat(1024)
values := [item, item, item]
first := encode(values)
second := encode(['other'.repeat(8192)])
assert decode[[]string](first)! == values
assert decode[[]string](second)! == ['other'.repeat(8192)]
}
fn test_integer_encoding_preserves_signed_and_unsigned_limits() {
signed := [i64(-9223372036854775807) - 1, -1, 0, 9223372036854775807]
unsigned := [u64(0), 9223372036854775808, 18446744073709551615]
expected_signed := '[-9223372036854775808,-1,0,9223372036854775807]'
expected_unsigned := '[0,9223372036854775808,18446744073709551615]'
assert encode(signed) == expected_signed
assert encode(unsigned) == expected_unsigned
assert decode[[]i64](expected_signed)! == signed
assert decode[[]u64](expected_unsigned)! == unsigned
}
fn test_decoded_array_supports_slices_after_growth() {
source := '[' + []string{len: 4096, init: '42'}.join(',') + ']'
mut decoded := decode[[]Any](source)!
prefix := unsafe { decoded[..2] }
assert prefix.data == decoded.data
for _ in 0 .. 8192 {
decoded << Any('later')
}
assert prefix == [Any(f64(42)), Any(f64(42))]
assert decoded.len == 12288
assert decoded.last() == Any('later')
}
struct ArrayDefault {
mut:
value int = 42
name string = 'default'
items []int = [1]
table map[string]int = {
'count': 1
}
}
fn test_array_storage_preserves_element_defaults() {
mut values := decode[[]ArrayDefault]('[{}, {"value":7}]')!
assert values == [ArrayDefault{}, ArrayDefault{ value: 7 }]
values[0].items << 2
values[0].table['count'] = 2
assert values[1].items == [1]
assert values[1].table['count'] == 1
assert decode[[][]int]('[[],[1,2],[],[3]]')! == [[], [1, 2], [], [3]]
assert decode[[]?int]('[null,7,null]')! == [?int(none), ?int(7), ?int(none)]
assert decode[[]Any]('[]')!.len == 0
}
fn test_container_counts_and_skips_match_nested_boundaries() {
source := '[{},[],{"a":[1,{"b":[]}]},true,null,"[}"]'
mut decoder := Decoder{ json: source }
decoder.check_json_format()!
expected := [1, 2, 3, 10, 11, 12]
mut next := 1
for index in expected {
assert next == index
next = decoder.next_value(next)
}
assert next == decoder.values_info.len
assert decoder.container_len(0) == 6
assert decoder.container_len(1) == 0
assert decoder.container_len(2) == 0
assert decoder.container_len(3) == 1
assert decoder.container_len(5) == 2
assert decoder.next_value(0) == decoder.values_info.len
}
fn test_reserved_maps_keep_duplicates_and_allow_later_mutation() {
mut values := decode[map[string]Any]('{"a":1,"a":2,"b":[3]}')!
assert values.len == 2
assert (values['a'] or { panic('missing key') }) == Any(f64(2))
for key in ['c', 'd', 'e', 'f', 'g', 'h', 'i'] {
values[key] = key
}
values.delete('a')
values['j'] = 'j'
assert values.len == 9
assert (values['j'] or { panic('missing key') }) == Any('j')
assert (values['b'] or { panic('missing key') }) == Any([Any(f64(3))])
}
type StorageIntegerKey = int
type StorageStringKey = string
fn test_decoded_map_keys_follow_the_declared_type() {
integers := decode[map[int]string]('{"42":"answer","-7":"negative"}')!
assert integers[42] == 'answer'
assert integers[-7] == 'negative'
unsigned := decode[map[u64]int]('{"18446744073709551615":7}')!
assert unsigned[u64(18446744073709551615)] == 7
aliases := decode[map[StorageIntegerKey]int]('{"3":5}')!
assert aliases[StorageIntegerKey(3)] == 5
strings := decode[map[StorageStringKey]int]('{"name":9}')!
assert strings[StorageStringKey('name')] == 9
runes := decode[map[rune]int]('{"a":11,"世":13,"🙂":17,"1":19}')!
assert runes[`a`] == 11
assert runes[`世`] == 13
assert runes[`🙂`] == 17
assert runes[`1`] == 19
assert decode[map[rune]int](encode(runes))! == runes
values := decode[map[int]?int]('{"2":null,"3":4}')!
assert values[2] == none
assert values[3] == ?int(4)
}
fn test_decoded_map_keys_reject_out_of_range_and_multiple_runes() {
if _ := decode[map[u8]int]('{"256":1}') {
assert false
}
if _ := decode[map[rune]int]('{"ab":1}') {
assert false
}
}
+17 -82
View File
@@ -23,45 +23,19 @@ fn option_payload_tag[P](_ ?P) string {
return struct_variant_tag[P]()
}
fn (mut decoder Decoder) get_decoded_sumtype_workaround[T](initialized_sumtype T) !T {
$if initialized_sumtype is $sumtype || (T is $alias && T.unaliased_typ is $sumtype) {
resolved_sumtype := initialized_sumtype
// `is` does not tell an alias variant from its base type (`MyString | string`
// matches both), so prefer the variant with the exact type name.
variant_name := initialized_sumtype.type_name()
mut has_exact_variant := false
$for v in T.variants {
if initialized_sumtype is v && variant_name == typeof(v.typ).name {
has_exact_variant = true
}
}
$for v in T.variants {
if initialized_sumtype is v
&& (!has_exact_variant || variant_name == typeof(v.typ).name) {
$if initialized_sumtype is time.Time {
mut val := $zero(v.typ)
decoder.decode_sumtype_time(mut val)!
return T(val)
} $else $if initialized_sumtype !is $option {
mut val := $zero(v.typ)
decoder.decode_value(mut val)!
return T(val)
} $else {
if decoder.current_value().value_kind == .null {
decoder.current_idx++
return resolved_sumtype
} else {
// The payload of an option variant, like in the removed `json` module.
mut option_value := $zero(v.typ)
option_value = decoder.decode_option_payload(option_value)!
return T(option_value)
}
}
fn (mut decoder Decoder) decode_selected_sumtype[T](mut value T) ! {
type_idx := value.type_idx()
$for variant in T.variants {
if mut value is variant && type_idx == variant.typ {
$if variant.typ !is $option && variant.typ is time.Time {
decoder.decode_sumtype_time(mut value)!
} $else {
decoder.decode_value(mut value)!
}
return
}
}
decoder.decode_error('could not decode resolved sumtype (should not happen)')!
return initialized_sumtype // suppress compiler error
}
// check_element_type_valid reports whether the value at `value_idx` in values_info
@@ -164,20 +138,6 @@ fn (mut decoder Decoder) check_array_type_valid[T](arr []T, value_idx int) bool
return decoder.check_element_type_valid(element, value_idx)
}
fn (mut decoder Decoder) get_array_type_workaround[T](initialized_sumtype T) bool {
$if initialized_sumtype is $sumtype || (T is $alias && T.unaliased_typ is $sumtype) {
$for v in T.variants {
if initialized_sumtype is v {
$if initialized_sumtype is $array {
return decoder.check_element_type_valid(initialized_sumtype,
decoder.current_idx)
}
}
}
}
return false
}
fn get_map_element_type[U, V](_m map[U]V) V {
return V{}
}
@@ -187,26 +147,6 @@ fn (mut decoder Decoder) check_map_type_valid[T](m T, value_idx int) bool {
return decoder.check_element_type_valid(element, value_idx)
}
fn (mut decoder Decoder) check_map_empty_valid[T](m T) bool {
element := get_map_element_type(m)
return decoder.check_element_type_valid(element, no_value_idx)
}
fn (mut decoder Decoder) get_map_type_workaround[T](initialized_sumtype T) bool {
$if initialized_sumtype is $sumtype || (T is $alias && T.unaliased_typ is $sumtype) {
$for v in T.variants {
if initialized_sumtype is v {
$if initialized_sumtype is $map {
val := $zero(v.typ)
// The first value of the object, after its first key.
return decoder.check_map_type_valid(val, decoder.current_idx + 2)
}
}
}
}
return false
}
// get_sumtype_type_field_idx returns the index in values_info of the value of the
// `_type` key of the object at `value_idx`, or no_value_idx without such a key.
@[direct_array_access; markused]
@@ -231,12 +171,7 @@ fn (mut decoder Decoder) get_sumtype_type_field_idx(value_idx int) int {
return key_value_idx
}
// Skip the value, with everything nested in it.
value_info := decoder.values_info[key_value_idx]
value_end := value_info.position + value_info.length
key_idx = key_value_idx + 1
for key_idx < values_len && decoder.values_info[key_idx].position < value_end {
key_idx++
}
key_idx = decoder.next_value(key_value_idx)
}
return no_value_idx
}
@@ -422,9 +357,9 @@ fn (mut decoder Decoder) init_sumtype_by_value_kind[T](mut val T, value_info Val
.array {
$for v in T.variants {
$if v.typ is $array {
val = T(v)
if decoder.get_array_type_workaround(val) {
candidate := $zero(v.typ)
if decoder.check_element_type_valid(candidate, decoder.current_idx) {
val = T(candidate)
return
}
}
@@ -436,9 +371,9 @@ fn (mut decoder Decoder) init_sumtype_by_value_kind[T](mut val T, value_info Val
}
$for v in T.variants {
$if v.typ is $map {
val = T(v)
if decoder.get_map_type_workaround(val) {
candidate := $zero(v.typ)
if decoder.check_element_type_valid(candidate, decoder.current_idx) {
val = T(candidate)
return
}
} $else $if v.typ is $struct {
@@ -515,5 +450,5 @@ fn (mut decoder Decoder) decode_sumtype[T](mut val T) ! {
decoder.init_sumtype_by_value_kind(mut val, value_info)!
val = decoder.get_decoded_sumtype_workaround(val)!
decoder.decode_selected_sumtype(mut val)!
}
+8 -10
View File
@@ -5,10 +5,9 @@ module json2
fn checked_decoder(json string, room int) !Decoder {
mut decoder := Decoder{
json: json
values_info: []ValueInfo{len: room}
values_info: []ValueInfo{cap: room}
}
decoder.check_json_format()!
decoder.values_info.trim(decoder.values_len)
return decoder
}
@@ -24,7 +23,6 @@ fn value_texts(decoder Decoder) []string {
fn test_values_are_stored_in_one_array_in_document_order() {
json := '{"a": [1, {"b": null}], "c": "d", "e": true}'
decoder := checked_decoder(json, value_count_bound(json))!
assert decoder.values_info.len == decoder.values_len
assert value_texts(decoder) == [json, '"a"', '[1, {"b": null}]', '1', '{"b": null}', '"b"',
'null', '"c"', '"d"', '"e"', 'true']
assert decoder.values_info.map(it.value_kind) == [ValueKind.object, .string, .array, .number,
@@ -35,7 +33,7 @@ fn test_value_count_bound_is_exact_without_empty_containers() {
for json in ['1', '"text"', 'null', '[1]', '[1, 2, 3]', '{"a": 1}', '{"a": [1, 2], "b": {"c": "d"}}',
'[[[1]]]', ' [ 1 , 2 ] '] {
decoder := checked_decoder(json, value_count_bound(json))!
assert decoder.values_len == value_count_bound(json), json
assert decoder.values_info.len == value_count_bound(json), json
}
}
@@ -44,7 +42,7 @@ fn test_value_count_bound_counts_separators_in_strings_of_short_json() {
r'{"a\",\":": "b\\", "c": ",\\\",:"}', r'["\\\\", 1, "{\"a\":[1,2]}"]'] {
assert json.len < skip_strings_min_json_len
decoder := checked_decoder(json, value_count_bound(json))!
assert decoder.values_len <= value_count_bound(json), json
assert decoder.values_info.len <= value_count_bound(json), json
}
}
@@ -60,9 +58,9 @@ fn test_value_count_bound_skips_strings_of_long_json() {
json := '[' + elements.join(',') + ']'
assert json.len >= skip_strings_min_json_len
decoder := checked_decoder(json, value_count_bound(json))!
assert decoder.values_len == value_count_bound(json)
assert decoder.values_info.len == value_count_bound(json)
// the root array, and the 1, 3, 4, 5 and 4 values of every repetition of `items`
assert decoder.values_len == 1 + (elements.len / items.len) * 17
assert decoder.values_info.len == 1 + (elements.len / items.len) * 17
}
// naive_json_string_end is json_string_end, one byte at a time.
@@ -130,8 +128,8 @@ fn test_value_count_bound_counts_a_value_too_many_for_an_empty_container() {
// number of values.
for json in ['[]', '{}', '[[], {}]', '{"a": {}}'] {
decoder := checked_decoder(json, value_count_bound(json))!
assert decoder.values_len < value_count_bound(json), json
assert decoder.values_len > 0, json
assert decoder.values_info.len < value_count_bound(json), json
assert decoder.values_info.len > 0, json
}
}
@@ -152,7 +150,7 @@ fn test_checker_grows_values_info_when_it_has_no_room() {
expected := checked_decoder(json, value_count_bound(json))!
for room in [0, 1, 2, 5] {
decoder := checked_decoder(json, room)!
assert decoder.values_len == expected.values_len, 'room: ${room}'
assert decoder.values_info.len == expected.values_info.len, 'room: ${room}'
assert value_texts(decoder) == value_texts(expected), 'room: ${room}'
}
}
+77 -148
View File
@@ -1,6 +1,7 @@
module json2
import time
import strings
import sync.stdatomic
// EncoderOptions provides a list of options for encoding
@@ -35,7 +36,7 @@ mut:
level int
prefix string
output []u8 = []u8{cap: 2048}
output strings.Builder = strings.new_builder(2048)
}
// encode encodes a value as JSON, or JSON5 with `json5: true`.
@@ -43,10 +44,11 @@ pub fn encode[T](val T, config EncoderOptions) string {
mut encoder := Encoder{
EncoderOptions: config
}
defer { unsafe { encoder.output.free() } }
encoder.encode_value[T](val)
return encoder.output.bytestr()
return encoder.output.str()
}
// encode_append appends the JSON representation of val to destination, preserving
@@ -170,14 +172,14 @@ fn (mut encoder Encoder) encode_value[T](val T) {
encoder.encode_number(time.Time(val).unix())
} else {
time_val := val.to_json()
unsafe { encoder.output.push_many(time_val.str, time_val.len) }
encoder.output.write_string(time_val)
}
} $else $if T is JsonEncoder { // uses T, because alias could be implementing JsonEncoder, while the base type does not
integer_val := val.to_json()
unsafe { encoder.output.push_many(integer_val.str, integer_val.len) }
encoder.output.write_string(integer_val)
} $else $if T is Encodable { // uses T, because alias could be implementing JsonEncoder, while the base type does not
integer_val := val.json_str()
unsafe { encoder.output.push_many(integer_val.str, integer_val.len) }
encoder.output.write_string(integer_val)
} $else $if T.unaliased_typ is $struct {
unsafe {
$for field in T.fields {
@@ -227,98 +229,59 @@ fn (mut encoder Encoder) encode_string(val string) {
return
}
encoder.output << `"`
mut buffer_start := 0
mut buffer_end := 0
for buffer_end < val.len {
character := val[buffer_end]
match character {
`"`, `\\` {
unsafe { encoder.output.push_many(val.str + buffer_start, buffer_end - buffer_start) }
buffer_end++
buffer_start = buffer_end
encoder.output << `\\`
encoder.output << character
}
`\b` {
unsafe { encoder.output.push_many(val.str + buffer_start, buffer_end - buffer_start) }
buffer_end++
buffer_start = buffer_end
encoder.output << `\\`
encoder.output << `b`
}
`\n` {
unsafe { encoder.output.push_many(val.str + buffer_start, buffer_end - buffer_start) }
buffer_end++
buffer_start = buffer_end
encoder.output << `\\`
encoder.output << `n`
}
`\f` {
unsafe { encoder.output.push_many(val.str + buffer_start, buffer_end - buffer_start) }
buffer_end++
buffer_start = buffer_end
encoder.output << `\\`
encoder.output << `f`
}
`\t` {
unsafe { encoder.output.push_many(val.str + buffer_start, buffer_end - buffer_start) }
buffer_end++
buffer_start = buffer_end
encoder.output << `\\`
encoder.output << `t`
}
`\r` {
unsafe { encoder.output.push_many(val.str + buffer_start, buffer_end - buffer_start) }
buffer_end++
buffer_start = buffer_end
encoder.output << `\\`
encoder.output << `r`
}
else {
if character < 0x20 { // control characters
unsafe {
encoder.output.push_many(val.str + buffer_start, buffer_end - buffer_start)
}
buffer_end++
buffer_start = buffer_end
encoder.encode_unicode_escape(u32(character), false)
continue
}
if encoder.escape_unicode && character >= 0x80 {
unsafe {
encoder.output.push_many(val.str + buffer_start, buffer_end - buffer_start)
}
code_point, width := utf8_rune_at(val, buffer_end)
if code_point > 0xffff {
unicode_point_low := u32(code_point) - 0x10000
encoder.encode_unicode_escape(0xD800 + ((unicode_point_low >> 10) & 0x3FF), true)
encoder.encode_unicode_escape(0xDC00 + (unicode_point_low & 0x3FF), false)
} else {
encoder.encode_unicode_escape(u32(code_point), false)
}
buffer_end += width
buffer_start = buffer_end
continue
}
buffer_end = next_string_escape(val, buffer_end + 1, encoder.escape_unicode)
}
mut start := 0
mut end := 0
for end < val.len {
character := val[end]
escape := character < 0x20 || character in [`"`, `\\`]
unicode := encoder.escape_unicode && character >= 0x80
if !escape && !unicode {
end = next_string_escape(val, end + 1, encoder.escape_unicode)
continue
}
unsafe { encoder.output.push_many(val.str + start, end - start) }
end += encoder.encode_escape(val, end)
start = end
}
unsafe { encoder.output.push_many(val.str + buffer_start, buffer_end - buffer_start) }
unsafe { encoder.output.push_many(val.str + start, end - start) }
encoder.output << `"`
}
fn (mut encoder Encoder) encode_escape(val string, index int) int {
character := val[index]
if encoder.escape_unicode && character >= 0x80 {
code_point, width := utf8_rune_at(val, index)
encoder.encode_unicode(code_point)
return width
}
escaped := match character {
`"`, `\\` { character }
`\b` { u8(`b`) }
`\n` { u8(`n`) }
`\f` { u8(`f`) }
`\t` { u8(`t`) }
`\r` { u8(`r`) }
else { u8(0) }
}
if escaped != 0 {
encoder.output << `\\`
encoder.output << escaped
return 1
}
encoder.encode_unicode(rune(character))
return 1
}
fn (mut encoder Encoder) encode_unicode(code_point rune) {
if code_point <= 0xffff {
encoder.encode_unicode_escape(u32(code_point), false)
return
}
value := u32(code_point) - 0x10000
encoder.encode_unicode_escape(0xD800 + ((value >> 10) & 0x3FF), true)
encoder.encode_unicode_escape(0xDC00 + (value & 0x3FF), false)
}
// encode_unicode_escape writes a UTF-16 code unit without allocating formatted text.
fn (mut encoder Encoder) encode_unicode_escape(code u32, uppercase bool) {
hex := if uppercase { '0123456789ABCDEF' } else { '0123456789abcdef' }
@@ -391,38 +354,24 @@ fn (mut encoder Encoder) encode_enum_number[T](val T) {
fn (mut encoder Encoder) encode_boolean(val bool) {
if val {
unsafe { encoder.output.push_many(true_string.str, true_string.len) }
encoder.output.write_string(true_string)
} else {
unsafe { encoder.output.push_many(false_string.str, false_string.len) }
encoder.output.write_string(false_string)
}
}
@[manualfree]
fn (mut encoder Encoder) encode_number[T](val T) {
$if T is $int {
$if T is u8 || T is u16 || T is u32 || T is u64 || T is usize {
encoder.output.write_u_decimal(u64(val))
} $else {
encoder.output.write_decimal(i64(val))
}
return
}
mut integer_val := ''
$if T is u8 {
integer_val = u8(val).str()
} $else $if T is u16 {
integer_val = u16(val).str()
} $else $if T is u32 {
integer_val = u32(val).str()
} $else $if T is u64 {
integer_val = u64(val).str()
} $else $if T is i8 {
integer_val = i8(val).str()
} $else $if T is i16 {
integer_val = i16(val).str()
} $else $if T is int {
integer_val = int(val).str()
} $else $if T is i32 {
integer_val = i32(val).str()
} $else $if T is i64 {
integer_val = i64(val).str()
} $else $if T is usize {
integer_val = usize(val).str()
} $else $if T is isize {
integer_val = isize(val).str()
} $else $if T is f32 {
$if T is f32 {
$if js {
integer_val = f32(val).str()
} $else {
@@ -477,12 +426,12 @@ fn (mut encoder Encoder) encode_number[T](val T) {
}
}
}
unsafe { encoder.output.push_many(integer_val.str, integer_val.len) }
encoder.output.write_string(integer_val)
}
@[markused]
fn (mut encoder Encoder) encode_null() {
unsafe { encoder.output.push_many(null_string.str, null_string.len) }
encoder.output.write_string(null_string)
}
fn (mut encoder Encoder) encode_array[T](val T) {
@@ -492,15 +441,7 @@ fn (mut encoder Encoder) encode_array[T](val T) {
if i > 0 {
encoder.separate_items(true)
}
$if T is $pointer {
if voidptr(item) == unsafe { nil } {
encoder.encode_null()
} else {
unsafe { encoder.encode_pointer_array_item(item) }
}
} $else {
encoder.encode_value(item)
}
encoder.encode_value(item)
}
encoder.close_items(val.len, true)
encoder.output << `]`
@@ -520,11 +461,7 @@ struct EncoderMapKey[K] {
fn (mut encoder Encoder) encode_map[K, T](val map[K]T) {
source_keys := val.keys()
defer {
$if K.unaliased_typ is string {
for i in 0 .. source_keys.len {
unsafe { source_keys[i].free() }
}
}
// Typed array cleanup also releases the cloned string keys.
unsafe { source_keys.free() }
}
mut keys := []EncoderMapKey[K]{cap: val.len}
@@ -536,7 +473,9 @@ fn (mut encoder Encoder) encode_map[K, T](val map[K]T) {
unsafe { keys[i].name.free() }
}
}
unsafe { keys.free() }
// Entries borrow source keys; only the nonstring names are released above.
raw_keys := unsafe { &array(&keys) }
unsafe { raw_keys.free() }
}
for i in 0 .. source_keys.len {
key := &source_keys[i]
@@ -586,7 +525,7 @@ fn (mut encoder Encoder) encode_enum[T](val T) {
}
}
fn (mut encoder Encoder) encode_sumtype[T](val T) {
fn (mut encoder Encoder) encode_sumtype[T](val &T) {
$if T is $pointer {
// Pointer types are handled by encode_value's $pointer branch;
// this instantiation is generated but never called.
@@ -951,7 +890,7 @@ fn (mut encoder Encoder) encode_struct_field_value[T](val T) {
encoder.encode_null()
} $else $if T is $option {
if val == none {
unsafe { encoder.output.push_many(null_string.str, null_string.len) }
encoder.output.write_string(null_string)
} else {
encoder.encode_value(get_value_from_optional(val))
}
@@ -1152,16 +1091,6 @@ fn (mut encoder Encoder) encode_embedded_struct_fields[T](val T, was_first bool,
return is_first
}
fn (mut encoder Encoder) encode_custom[T](val T) {
integer_val := val.to_json()
unsafe { encoder.output.push_many(integer_val.str, integer_val.len) }
}
fn (mut encoder Encoder) encode_custom2[T](val T) {
integer_val := val.json_str()
unsafe { encoder.output.push_many(integer_val.str, integer_val.len) }
}
fn (mut encoder Encoder) increment_level() {
encoder.level++
encoder.prefix = encoder.line_prefix()
@@ -1222,7 +1151,7 @@ fn (mut encoder Encoder) close_items(count int, is_array bool) {
} else if !is_array && encoder.legacy_layout {
// The removed module wrote an empty object as `{`, a line break and `}`.
prefix := encoder.line_prefix()
unsafe { encoder.output.push_many(prefix.str, prefix.len) }
encoder.output.write_string(prefix)
}
}
@@ -1240,5 +1169,5 @@ fn (mut encoder Encoder) write_key_separator() {
}
fn (mut encoder Encoder) add_indent() {
unsafe { encoder.output.push_many(encoder.prefix.str, encoder.prefix.len) }
encoder.output.write_string(encoder.prefix)
}
-1
View File
@@ -203,7 +203,6 @@ fn decode_json5[T](text string, mut buffer DecodeBuffer, params DecoderOptions)
source_values_info: unsafe { parser.source_values_info }
strict: params.strict
values_info: unsafe { parser.values_info }
values_len: parser.values_info.len
}
return decoder.decode_root[T]()
}
@@ -40,6 +40,6 @@ fn test_decode_wide_integer_map_keys() {
fn test_decode_string_and_rune_map_keys() {
strings := json2.decode[map[string]int]('{"a":1}')!
assert strings['a'] == 1
runes := json2.decode[map[rune]int]('{"97":2}')!
runes := json2.decode[map[rune]int]('{"a":2}')!
assert runes[rune(97)] == 2
}
+20
View File
@@ -21,6 +21,26 @@ and returns an error for `UNIQUE`, `DEFAULT` and `REFERENCES`, which it cannot d
> the removal date; until then the compiler emits a migration notice.
> See https://github.com/vlang/v/issues/27001 for details.
## Constructing infix expressions
`orm.InfixType.right` has type `&orm.Primitive`. When constructing an expression directly,
store the operand as an `orm.Primitive` and pass its address:
```v ignore
import orm
operand := orm.Primitive(2)
expression := orm.InfixType{
name: 'stock'
operator: .add
right: &operand
}
```
Code that previously assigned a `Primitive` value to `right` should use `&operand`.
Read the operand with `*expression.right`. The reference gives recursive expressions
a finite layout; SQL arithmetic and parameter binding keep their existing behavior.
## Nullable
For a nullable column, use an option field. If the field is non-option, the column will be defined
+2 -1
View File
@@ -227,11 +227,12 @@ pub mut:
batch_key string
}
// InfixType describes an arithmetic SQL expression whose right operand is referenced.
pub struct InfixType {
pub:
name string
operator MathOperationKind
right Primitive
right &Primitive
}
pub struct Table {
+6 -2
View File
@@ -727,13 +727,14 @@ fn test_orm_stmt_gen_vsql_update_quotes_infix_column() {
table := orm.Table{
name: 'products'
}
operand := orm.Primitive(2)
data := orm.QueryData{
fields: ['stock']
data: [
orm.Primitive(orm.InfixType{
name: 'stock'
operator: .add
right: orm.Primitive(2)
right: &operand
}),
]
}
@@ -741,9 +742,12 @@ fn test_orm_stmt_gen_vsql_update_quotes_infix_column() {
fields: ['id']
kinds: [.eq]
}
query, _ := orm.orm_stmt_gen(.vsql, table, '"', .update, true, ':p', 1, data, where)
query, returned := orm.orm_stmt_gen(.vsql, table, '"', .update, true, ':p', 1, data, where)
// vsql folds an undelimited `stock` to `STOCK`, which is not the column.
assert query == 'UPDATE "products" SET "stock" = "stock" + :p1 WHERE "id" = :p2;'
expression := returned.data[0] as orm.InfixType
assert expression.right == &operand
assert *expression.right == orm.Primitive(2)
pg_query, _ := orm.orm_stmt_gen(.pg, table, '"', .update, true, '$', 1, data, where)
assert pg_query == 'UPDATE "products" SET "stock" = stock + $1 WHERE "id" = $2;'
}
+20
View File
@@ -0,0 +1,20 @@
module orm
fn infix_value() Primitive {
leaf := Primitive(7)
return InfixType{
name: 'count'
operator: .add
right: &leaf
}
}
fn test_primitive_has_finite_inline_recursive_layout() {
assert sizeof(Primitive) > sizeof(InfixType)
value := infix_value()
copy := value
assert value is InfixType
assert copy is InfixType
assert *(copy as InfixType).right == Primitive(7)
assert (copy as InfixType).name == 'count'
}
+10 -9
View File
@@ -45,7 +45,7 @@ const maxexp32 = 255
// max 46 char
// -3.40282346638528859811704183484516925440e+38
@[direct_array_access]
@[direct_array_access; manualfree]
pub fn (d Dec32) get_string_32(neg bool, i_n_digit int, i_pad_digit int) string {
n_digit := if i_n_digit < 1 { 1 } else { i_n_digit + 1 }
pad_digit := i_pad_digit + 1
@@ -53,14 +53,17 @@ pub fn (d Dec32) get_string_32(neg bool, i_n_digit int, i_pad_digit int) string
mut d_exp := d.e
// mut out_len := decimal_len_32(out)
mut out_len := dec_digits(out)
out_len_original := out_len
mut exponent := d.e + out_len - 1
mut fw_zeros := 0
if pad_digit > out_len {
fw_zeros = pad_digit - out_len
}
mut buf := []u8{len: int(out_len + 5 + 1 + 1 + fw_zeros)} // sign + mant_len + . + e + e_sign + exp_len(2) + \0}
buffer_len := out_len + 7 + fw_zeros
mut buf := []u8{len: buffer_len}
unsafe { buf.flags |= .noslices }
defer { unsafe { buf.free() } }
mut i := 0
if neg {
@@ -82,11 +85,9 @@ pub fn (d Dec32) get_string_32(neg bool, i_n_digit int, i_pad_digit int) string
// println("orig: ${out_len_original}")
out += ten_pow_table_32[out_len - n_digit - 1] * 5 // round to up
out /= ten_pow_table_32[out_len - n_digit]
out_div := d.m / ten_pow_table_32[out_len - n_digit]
if out_div < out && dec_digits(out_div) < dec_digits(out) {
// from `99` to `100`, will need d_exp+1, and the new last digit is dropped
d_exp++
if out >= ten_pow_table_32[n_digit] {
out /= 10
exponent++
}
out_len = n_digit
}
@@ -120,7 +121,7 @@ pub fn (d Dec32) get_string_32(neg bool, i_n_digit int, i_pad_digit int) string
buf[i] = `e`
i++
mut exp := d_exp + out_len_original - 1
mut exp := exponent
if exp < 0 {
buf[i] = `-`
i++
@@ -140,7 +141,7 @@ pub fn (d Dec32) get_string_32(neg bool, i_n_digit int, i_pad_digit int) string
buf[i] = 0
return unsafe {
tos(memdup(&buf[0], i + 1), i)
tos(memdup_noscan(&buf[0], i + 1), i)
}
}
+4 -2
View File
@@ -20,7 +20,7 @@ https://github.com/cespare/ryu/tree/ba56a33f39e3bbbfa409095d0f9ae168a595feea
=============================================================================*/
@[direct_array_access]
@[direct_array_access; manualfree]
fn (d Dec64) get_string_64(neg bool, i_n_digit int, i_pad_digit int) string {
n_digit := if i_n_digit < 1 { 1 } else { i_n_digit + 1 }
pad_digit := i_pad_digit + 1
@@ -36,6 +36,8 @@ fn (d Dec64) get_string_64(neg bool, i_n_digit int, i_pad_digit int) string {
}
mut buf := []u8{len: (out_len + 6 + 1 + 1 + fw_zeros)} // sign + mant_len + . + e + e_sign + exp_len(2) + \0}
unsafe { buf.flags |= .noslices }
defer { unsafe { buf.free() } }
mut i := 0
if neg {
@@ -132,7 +134,7 @@ fn (d Dec64) get_string_64(neg bool, i_n_digit int, i_pad_digit int) string {
buf[i] = 0
return unsafe {
tos(memdup(&buf[0], i + 1), i)
tos(memdup_noscan(&buf[0], i + 1), i)
}
}
+40
View File
@@ -0,0 +1,40 @@
import strconv
fn test_float_strings_keep_independent_storage() {
values := [f64(-0.125), 0.0, 1.5, 123456.75]
expected := ['-0.125', '0.0', '1.5', '123456.75']
mut saved := []string{}
for value in values {
saved << strconv.f64_to_str_l(value)
}
for i in 0 .. 1024 {
assert strconv.f64_to_str_l(f64(i) + 0.5).ends_with('.5')
}
gc_collect()
assert saved == expected
}
fn test_scientific_float_buffers_preserve_large_padding() {
short := strconv.f64_to_str(1.25, 18)
wide := strconv.f64_to_str_pad(1.25, 128)
single := strconv.f32_to_str(f32(-1.25), 8)
integer := strconv.f32_to_str(f32(7), 0)
assert short == '1.25e+00'
assert wide == '1.25' + '0'.repeat(126) + 'e+00'
assert single == '-1.25e+00'
assert integer == '7e+00'
assert strconv.fxx_to_str_l_parse('1.25e+128').len == 131
assert strconv.fxx_to_str_l_parse_with_dot('1.25e-128').len == 132
gc_collect()
assert short == '1.25e+00'
assert wide.ends_with('e+00')
assert single == '-1.25e+00'
assert integer == '7e+00'
}
fn test_f32_rounding_carries_into_the_exponent() {
assert strconv.f32_to_str(f32(9.75), 0) == '1e+01'
assert strconv.f32_to_str(f32(-9.75), 0) == '-1e+01'
assert strconv.f32_to_str(f32(99.75), 1) == '1.0e+02'
assert strconv.f32_to_str_pad(f32(1.25), 128) == '1.25' + '0'.repeat(126) + 'e+00'
}
+8 -2
View File
@@ -201,7 +201,10 @@ pub fn fxx_to_str_l_parse(s string) string {
res[r_i] = 0
tmp_res := unsafe { tos(res.data, r_i).clone() }
unsafe { res.free() }
unsafe {
res.flags |= .noslices
res.free()
}
return tmp_res
}
@@ -338,6 +341,9 @@ pub fn fxx_to_str_l_parse_with_dot(s string) string {
res[r_i] = 0
tmp_res := unsafe { tos(res.data, r_i).clone() }
unsafe { res.free() }
unsafe {
res.flags |= .noslices
res.free()
}
return tmp_res
}
+16
View File
@@ -13,6 +13,22 @@ doc1 := toml.parse_text(<string content>) or { panic(err) }
doc2 := toml.parse_file(<file path>) or { panic(err) }
```
## Parser AST and table views
`toml.ast.DateTime` keeps its public `date` and `time` components, including their
text and source positions, alongside the complete date-time `text`.
The public `Parser.find_table`, `find_sub_table` and `find_in_table` methods return
`!&map[string]ast.Value`. Each reference has a heap header sharing the table's storage.
Insertions, deletions and reservations through a view are visible through other views.
Growing the parent table does not invalidate the reference: it holds no pointer into
the parent's inline values. Keep shared table storage alive until all views are finished;
use `.clone()` when an independent table is needed.
`Parser.double_array_of_tables_contents(target_key)` returns a one-element
`[]ast.Value` containing the parsed table body. It leaves the following table header
available for the caller to continue parsing.
## Example
Here we parse the official [TOML example](https://github.com/toml-lang/toml/blob/3b11f6921da7b6f5db37af039aa021fee450c091/README.md#Example)
+1 -1
View File
@@ -78,7 +78,7 @@ pub fn decode_quoted_escapes(mut q ast.Quoted) ! {
mut eat_whitespace := false
// See https://toml.io/en/v1.0.0#string for more info on string types.
is_basic := q.quote == `\"`
if !is_basic {
if !is_basic || q.text.index_u8(`\\`) < 0 {
return
}
mut decoded_s := strings.new_builder(q.text.len)
+137
View File
@@ -0,0 +1,137 @@
module parser
import hash { sum64_string }
struct IndexedKey {
start int
len int
mut:
next int
count int = 1
}
struct KeyIndex {
mut:
heads map[u64]int
keys []IndexedKey
parts []string
}
fn (key IndexedKey) matches(parts []string, candidate DottedKey) bool {
if key.len != candidate.len {
return false
}
for i, part in candidate {
if parts[key.start + i] != part {
return false
}
}
return true
}
fn (key DottedKey) hash() u64 {
mut code := u64(0)
for part in key {
code = sum64_string(part, code)
}
return code
}
fn (index &KeyIndex) has(key DottedKey) bool {
mut position := index.heads[key.hash()]
for position > 0 {
entry := index.keys[position - 1]
if entry.matches(index.parts, key) {
return entry.count > 0
}
position = entry.next
}
return false
}
fn (mut index KeyIndex) add(key DottedKey) {
code := key.hash()
mut position := index.heads[code]
for position > 0 {
entry := index.keys[position - 1]
if entry.matches(index.parts, key) {
index.keys[position - 1].count++
return
}
position = entry.next
}
entry := IndexedKey{
start: index.parts.len
len: key.len
next: index.heads[code]
}
unsafe {
index.keys.flags |= .noslices
index.parts.flags |= .noslices
}
index.parts << key
index.keys << entry
index.heads[code] = index.keys.len
}
fn (mut index KeyIndex) remove(key DottedKey) bool {
code := key.hash()
mut position := index.heads[code]
mut previous := 0
for position > 0 {
entry := index.keys[position - 1]
if !entry.matches(index.parts, key) {
previous = position
position = entry.next
continue
}
if entry.count > 1 {
index.keys[position - 1].count--
return true
}
if entry.count == 0 {
return false
}
if previous == 0 {
index.heads[code] = entry.next
} else {
index.keys[previous - 1].next = entry.next
}
return true
}
return false
}
fn (mut index KeyIndex) reset(prefix DottedKey) {
for i, entry in index.keys {
if entry.count == 0 || entry.len < prefix.len {
continue
}
parent := IndexedKey{
start: entry.start
len: prefix.len
}
if parent.matches(index.parts, prefix) {
index.keys[i].count = 0
}
}
}
fn (index &KeyIndex) has_parent(key DottedKey, root DottedKey) bool {
for length in 2 .. key.len {
parent := DottedKey(key[..length])
if parent != root && index.has(parent) {
return true
}
}
return false
}
fn (mut index KeyIndex) free() {
index.parts.clear()
unsafe {
index.heads.free()
index.keys.free()
index.parts.free()
}
}
+97
View File
@@ -0,0 +1,97 @@
module parser
fn test_key_index_preserves_segments_and_duplicate_removal() {
mut index := KeyIndex{}
quoted := DottedKey(['a.b', 'c'])
dotted := DottedKey(['a', 'b.c'])
index.add(quoted)
index.add(quoted)
assert index.has(quoted)
assert !index.has(dotted)
assert index.remove(quoted)
assert index.has(quoted)
assert index.remove(quoted)
assert !index.has(quoted)
assert !index.remove(quoted)
}
fn test_key_index_owns_its_path_segments() {
mut index := KeyIndex{}
mut key := DottedKey(['table', 'field'])
index.add(key)
key[1] = 'changed'
assert index.has(DottedKey(['table', 'field']))
assert !index.has(key)
}
fn test_key_index_checks_hash_collisions_structurally() {
key := DottedKey(['table', 'value'])
other := DottedKey(['other', 'value'])
mut index := KeyIndex{}
index.add(other)
index.heads[key.hash()] = index.heads[other.hash()]
assert !index.has(key)
assert !index.remove(key)
index.add(key)
assert index.has(key)
assert index.remove(key)
assert index.heads[key.hash()] == index.heads[other.hash()]
}
fn test_key_index_parent_excludes_root_and_single_segment() {
mut index := KeyIndex{}
root := DottedKey(['table', 'root'])
key := DottedKey(['table', 'root', 'child', 'value'])
index.add(DottedKey(['table']))
index.add(root)
assert !index.has_parent(key, root)
index.add(DottedKey(['table', 'root', 'child']))
assert index.has_parent(key, root)
assert !index.has_parent(root, root)
}
fn test_repeated_paths_share_storage_and_keep_multiplicity() {
mut index := KeyIndex{}
key := DottedKey(['entry', 'name'])
for _ in 0 .. 10000 {
index.add(key)
}
assert index.keys.len == 1
assert index.parts.len == key.len
for _ in 0 .. 10000 {
assert index.has(key)
assert index.remove(key)
}
assert !index.has(key)
assert !index.remove(key)
}
fn test_index_release_preserves_borrowed_text() {
key := DottedKey(['table'.clone(), 'field'.clone()])
mut index := KeyIndex{}
index.add(key)
index.free()
assert key == DottedKey(['table', 'field'])
}
fn test_key_index_reset_preserves_other_entries_and_reuses_storage() {
mut index := KeyIndex{}
first := DottedKey(['a', 'b', 'c'])
second := DottedKey(['a', 'd'])
quoted := DottedKey(['a.b', 'c'])
index.add(first)
index.add(second)
index.add(quoted)
index.reset(DottedKey(['a', 'b']))
assert !index.has(first)
assert !index.remove(first)
assert index.has(second)
assert index.has(quoted)
index.add(first)
assert index.has(first)
assert index.keys.len == 3
index.reset(DottedKey(['a']))
assert !index.has(first)
assert !index.has(second)
assert index.has(quoted)
}
+27
View File
@@ -0,0 +1,27 @@
module parser
import toml.scanner
import toml.token
fn test_lookahead_only_reads_requested_tokens() {
mut source := scanner.new_simple_text('a = 1')!
mut parser := new_parser(Config{ scanner: &source })
parser.init()!
assert source.state().pos == 1
assert parser.peek(2)!.kind == .assign
assert source.state().pos == 3
assert parser.peek(1)!.kind == .whitespace
assert source.state().pos == 3
expected := [
token.Kind.bare,
.whitespace,
.assign,
.whitespace,
.number,
.eof,
]
for kind in expected {
parser.next()!
assert parser.tok.kind == kind
}
}
+275 -427
View File
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,27 @@
module parser
import toml.ast
import toml.scanner
fn test_public_nested_contents_preserves_the_next_header_for_continuation() {
mut source := scanner.new_simple_text('name = "first"\n[[group.items]]\nname = "second"\n[[group.items]]\nname = "third"\n')!
mut p := new_parser(Config{ scanner: &source })
p.init()!
values := p.double_array_of_tables_contents(DottedKey(['group', 'items']))!
assert values.len == 1
first := values[0] as map[string]ast.Value
assert (first['name'] or { panic('missing first name') }).str() == '"first"'
assert p.tok.kind == .lsbr
assert p.peek_tok.kind == .lsbr
// The normal table caller suppresses one next() after an array body returns.
p.skip_next = true
p.root_table()!
assert p.tok.kind == .eof
group := (p.root_map['group'] or { panic('missing group') }) as map[string]ast.Value
remaining := (group['items'] or { panic('missing items') }) as []ast.Value
assert remaining.len == 2
second := remaining[0] as map[string]ast.Value
third := remaining[1] as map[string]ast.Value
assert (second['name'] or { panic('missing second name') }).str() == '"second"'
assert (third['name'] or { panic('missing third name') }).str() == '"third"'
}
+6 -15
View File
@@ -430,19 +430,10 @@ fn (mut s Scanner) extract_string() !string {
is_multiline := s.text[s.pos + 1] == quote && s.text[s.pos + 2] == quote
// Check for escaped multiline quote
if is_multiline {
mls := s.extract_multiline_string()!
return mls
return s.extract_multiline_string()
}
// `has_newline` mirrors what `lit.contains('\n')` used to report, without
// re-scanning the whole literal on every byte: it is set by each append and,
// like `contains`, never goes back to false.
mut has_newline := false
mut lit := strings.new_builder(64)
defer {
unsafe { lit.free() }
}
lit.write_u8(quote)
for {
s.pos++
@@ -460,7 +451,6 @@ fn (mut s Scanner) extract_string() !string {
// Check for escaped chars
if c == u8(92) {
esc, skip := s.handle_escapes(quote, is_multiline)
lit.write_string(esc)
has_newline = has_newline || esc.contains('\n')
if skip > 0 {
s.pos += skip
@@ -477,11 +467,9 @@ fn (mut s Scanner) extract_string() !string {
if c == quote {
s.pos++
s.col++
lit.write_u8(quote)
return lit.str()
return s.text[start..s.pos]
}
lit.write_u8(c)
has_newline = has_newline || c == `\n`
// Don't eat multiple lines in single-line mode
@@ -490,7 +478,7 @@ fn (mut s Scanner) extract_string() !string {
' unfinished single-line string literal `${quote.ascii_str()}` started at ${start} (${s.line_nr},${s.col}) "${u8(s.at()).ascii_str()}" near ...${s.excerpt(s.pos, 5)}...')
}
}
return lit.str()
return s.text[start..s.pos]
}
// extract_multiline_string collects and returns a string containing
@@ -729,6 +717,9 @@ pub fn (s &Scanner) state() State {
}
fn (mut s Scanner) validate_and_skip_headers() ! {
if s.pos != 0 {
return
}
// UTF-16 / UTF-32 headers (BE/LE)
s.check_utf16_or_32_bom()!
@@ -0,0 +1,39 @@
import toml.ast
import toml.parser
import toml.scanner
fn test_datetime_ast_preserves_public_date_and_time_components() {
for text in [
'1979-05-27T07:32:00Z',
'1979-05-27t07:32:00z',
'1979-05-27 07:32:00Z',
'1979-05-27T00:32:00.999999-07:00',
'1979-05-27T00:32:00+07:00',
'1979-05-27T07:32:00',
] {
mut source := scanner.new_simple_text('value = ${text}')!
mut p := parser.new_parser(parser.Config{ scanner: &source })
root := p.parse()!
table := root.table as map[string]ast.Value
parsed := table['value'] or { panic('missing datetime') }
assert parsed is ast.DateTime
dt := parsed as ast.DateTime
assert dt.text == text
assert dt.date.text == text[..10]
assert dt.time.text == text[11..]
assert dt.date.pos.pos == dt.pos.pos
assert dt.time.pos.pos >= dt.date.pos.pos + 10
assert dt.str().contains(" date: '${dt.date}'\n")
assert dt.str().contains(" time: '${dt.time}'\n")
}
}
fn test_datetime_ast_accepts_existing_component_initializers() {
dt := ast.DateTime{
text: '2000-01-02T03:04:05'
date: ast.Date{ text: '2000-01-02' }
time: ast.Time{ text: '03:04:05' }
}
assert dt.date.text == '2000-01-02'
assert dt.time.text == '03:04:05'
}
@@ -0,0 +1,79 @@
import json2
import toml
import toml.to
fn test_path_keys_survive_parser_disposal() {
document := toml.parse_text('root."nested.key".value = 7\n'.clone())!
gc_collect()
expected := '{"root":{"nested.key":{"value":7}}}'
actual := json2.encode(to.json_any(document.to_any()))
assert actual == expected
}
fn test_nested_array_paths_preserve_table_values() {
inputs := [
'[[a]]\n[[a.b]]\nx=1\n[[a]]\n[a.b]\nx=2\n',
'a=[{c={d=1}},{c={d=2}}]\n[[c.e]]\nvalue=3\n',
'[[a]]\nc={d=1}\n[[c.e]]\nvalue=2\n',
'[[a]]\nc={d=1}\n[[a]]\n[a.c]\ne=2\n',
'[[a.b]]\nc={d=1}\n[[c.e]]\nvalue=2\n',
'[[a.b]]\nc={d=1}\n[[a.b]]\n[a.b.c]\ne=2\n',
'[[a]]\nc={d={e=1}}\n[[a]]\n[[a.c.d]]\ne=2\n',
'[[a.b.c]]\nx=1\n[[a.b.c]]\nx=2\n',
'[[a]]\nid=1\n[[a.b.c]]\nx=1\n[[a]]\nid=2\n[[a.b.c]]\nx=2\n',
'[["a.b".c.d]]\nx=1\n',
'[[a.b]]\ntrue=1\nfalse=2\n',
'[[a.b.c]]\nx=1\n[a.b.c.meta]\ny=2\n[[a.b.c]]\nx=3\n',
'[[a.b]]\nid=1\n[[a.b.c]]\nx=1\n[[a.b]]\nid=2\n[[a.b.c]]\nx=2\n',
]
expected := [
'{"a":[{"b":[{"x":1}]},{"b":{"x":2}}]}',
'{"a":[{"c":{"d":1}},{"c":{"d":2}}],"c":{"e":[{"value":3}]}}',
'{"a":[{"c":{"d":1}}],"c":{"e":[{"value":2}]}}',
'{"a":[{"c":{"d":1}},{"c":{"e":2}}]}',
'{"a":{"b":[{"c":{"d":1}}]},"c":{"e":[{"value":2}]}}',
'{"a":{"b":[{"c":{"d":1}},{"c":{"e":2}}]}}',
'{"a":[{"c":{"d":{"e":1}}},{"c":{"d":[{"e":2}]}}]}',
'{"a":{"b":{"c":[{"x":1},{"x":2}]}}}',
'{"a":[{"id":1,"b":{"c":[{"x":1}]}},' +
'{"id":2,"b":{"c":[{"x":2}]}}]}',
'{"a.b":{"c":{"d":[{"x":1}]}}}',
'{"a":{"b":[{"true":1,"false":2}]}}',
'{"a":{"b":{"c":[{"x":1,"meta":{"y":2}},{"x":3}]}}}',
'{"a":{"b":[{"id":1,"c":[{"x":1}]},' +
'{"id":2,"c":[{"x":2}]}]}}',
]
for i, input in inputs {
document := toml.parse_text(input)!
actual := json2.decode[json2.Any](to.json(document))!
want := json2.decode[json2.Any](expected[i])!
assert actual == want, input
}
}
fn test_nested_array_headers_reject_immutable_parents() {
inputs := [
'[[a]]\nc={d=1}\n[[a.c.e]]\nvalue=2\n',
'[[a.b]]\nc={d=1}\n[[a.b.c.e]]\nvalue=2\n',
'[[a.b]]\nc={d=1}\n[a.b.c]\ne=2\n',
'a = {b={}}\n[[a.b.c]]\nx=1\n',
'a.b = {c=[]}\n[[a.b.c]]\nx=1\n',
'a = [{x=1}]\n[[a.b]]\ny=2\n',
'a.b = [{x=1}]\n[a.b.c]\ny=2\n',
]
for input in inputs {
if _ := toml.parse_text(input) {
assert false, input
}
}
}
fn test_array_table_depth_is_not_limited_to_two_components() {
parts := []string{len: 24, init: 'level${index}'}
path := parts.join('.')
document := toml.parse_text('[[${path}]]\nvalue=7\n')!
entries := document.value(path).array()
assert entries.len == 1
value := entries[0].as_map()['value'] or { panic('missing value') }
assert value.int() == 7
}
+82
View File
@@ -0,0 +1,82 @@
import toml.ast
import toml.parser
import toml.scanner
fn root_table_reference(mut p parser.Parser) !&map[string]ast.Value {
return p.find_table()!
}
fn child_table_reference(mut p parser.Parser) !&map[string]ast.Value {
return p.find_sub_table(['child'])!
}
fn nested_table_reference(mut p parser.Parser, mut table map[string]ast.Value) !&map[string]ast.Value {
return p.find_in_table(mut table, ['nested'])!
}
fn set_table_reference_value(table &map[string]ast.Value, key string, value ast.Value) {
// Mutate through the original returned reference, including after its parent rehashes.
unsafe { (*table)[key] = value }
}
fn table_reference_value(table &map[string]ast.Value, key string) ast.Value {
return (*table)[key] or { panic('missing table key `${key}`') }
}
fn test_public_parser_table_references_share_mutations_after_parent_rehash() {
mut source := scanner.new_simple_text('')!
mut p := parser.new_parser(parser.Config{ scanner: &source })
p.init()!
mut root := root_table_reference(mut p)!
mut child := child_table_reference(mut p)!
set_table_reference_value(child, 'before', ast.Number{ text: '1' })
root.reserve(2048)
for i in 0 .. 1024 {
set_table_reference_value(root, 'sibling${i}', ast.Number{ text: i.str() })
}
child.reserve(512)
set_table_reference_value(child, 'after', ast.Number{ text: '2' })
mut again := child_table_reference(mut p)!
assert table_reference_value(again, 'before').str() == '1'
assert table_reference_value(again, 'after').str() == '2'
set_table_reference_value(again, 'from_parent', ast.Number{ text: '3' })
assert table_reference_value(child, 'from_parent').str() == '3'
parent_child := table_reference_value(root, 'child') as map[string]ast.Value
assert (parent_child['after'] or { panic('missing after') }).str() == '2'
assert (parent_child['from_parent'] or { panic('missing from_parent') }).str() == '3'
assert root_table_reference(mut p)!.len == 1025
}
fn test_public_find_in_table_reference_survives_outer_and_inner_rehash() {
mut source := scanner.new_simple_text('')!
mut p := parser.new_parser(parser.Config{ scanner: &source })
p.init()!
mut outer := map[string]ast.Value{}
mut nested := nested_table_reference(mut p, mut outer)!
set_table_reference_value(nested, 'before', ast.Number{ text: '11' })
outer.reserve(2048)
for i in 0 .. 1024 {
outer['sibling${i}'] = ast.Number{ text: i.str() }
}
nested.reserve(512)
set_table_reference_value(nested, 'after', ast.Number{ text: '22' })
mut again := nested_table_reference(mut p, mut outer)!
set_table_reference_value(again, 'from_parent', ast.Number{ text: '33' })
assert table_reference_value(nested, 'from_parent').str() == '33'
assert table_reference_value(again, 'before').str() == '11'
assert table_reference_value(again, 'after').str() == '22'
parent_nested := (outer['nested'] or { panic('missing nested') }) as map[string]ast.Value
assert (parent_nested['after'] or { panic('missing after') }).str() == '22'
}
fn test_public_nested_table_contents_returns_one_table_at_eof() {
mut source := scanner.new_simple_text('name = "first"\ndetails.weight = 3')!
mut p := parser.new_parser(parser.Config{ scanner: &source })
p.init()!
values := p.double_array_of_tables_contents(['group', 'items'])!
assert values.len == 1
entry := values[0] as map[string]ast.Value
assert (entry['name'] or { panic('missing name') }).str() == '"first"'
details := (entry['details'] or { panic('missing details') }) as map[string]ast.Value
assert (details['weight'] or { panic('missing weight') }).str() == '3'
}
-2
View File
@@ -115,8 +115,6 @@ fn test_single_line_strings_reject_raw_newlines() {
}
fn test_long_strings_keep_their_content() {
// Long values are the reason the scanner and the decoder accumulate into a
// `strings.Builder`; check that nothing is truncated, doubled or reordered.
payload := 'aA9+/'.repeat(20_000)
mut toml_doc := toml.parse_text('long = "${payload}"\nescaped = "${payload}\\t${payload}"') or {
panic(err)
+12
View File
@@ -51,3 +51,15 @@ fn test_toml_with_bom() {
}
_ = bad_toml_doc
}
fn test_bom_is_only_a_header_at_document_start() {
bom := '\xef\xbb\xbf'
document := toml.parse_text('${bom}value = "${bom}"')!
assert document.value('value').string() == bom
invalid := ['value=1\n${bom}next=2', ' ${bom}value=1']
for text in invalid {
if _ := toml.parse_text(text) {
assert false, text
}
}
}
+2 -1
View File
@@ -116,13 +116,14 @@ pub fn json_any(a toml.Any) json2.Any {
}
map[string]toml.Any {
mut jmap := map[string]json2.Any{}
jmap.reserve(u32(a.len))
for key, val in a {
jmap[key] = json_any(val)
}
return jmap
}
[]toml.Any {
mut jarr := []json2.Any{}
mut jarr := []json2.Any{cap: a.len}
for val in a {
jarr << json_any(val)
+7 -13
View File
@@ -827,10 +827,6 @@ fn (d Doc) value_(value ast.Value, key []string) Any {
// ast_to_any converts `from` ast.Value to toml.Any value.
pub fn ast_to_any(value ast.Value) Any {
return ast_to_any_(value)
}
fn ast_to_any_(value ast.Value) Any {
match value {
ast.Date {
return Any(Date{value.text.clone()})
@@ -860,7 +856,7 @@ fn ast_to_any_(value ast.Value) Any {
return Any('nan')
}
if !val_text.starts_with('0x')
&& (val_text.contains('.') || val_text.to_lower().contains('e')) {
&& (val_text.contains('.') || val_text.contains_any('eE')) {
return Any(value.f64())
}
return Any(value.i64())
@@ -873,20 +869,18 @@ fn ast_to_any_(value ast.Value) Any {
return Any(false)
}
map[string]ast.Value {
m := (value as map[string]ast.Value)
mut am := map[string]Any{}
for k, v in m {
converted := ast_to_any_(v)
am.reserve(u32(value.len))
for k, v in value {
converted := ast_to_any(v)
am[k] = converted
}
return am
// return d.get_map_value(m, key_split[1..].join('.'))
}
[]ast.Value {
a := (value as []ast.Value)
mut aa := []Any{cap: a.len}
for val in a {
converted := ast_to_any_(val)
mut aa := []Any{cap: value.len}
for val in value {
converted := ast_to_any(val)
aa << converted
}
return aa
+1 -1
View File
@@ -16,7 +16,7 @@ struct BAnd {
type BUnary = BNot | BExpr | BFact | BDefine
struct BNot {
expr BUnary
expr &BUnary
}
type BFact = string
+6 -2
View File
@@ -55,6 +55,10 @@ fn (mut p BParser) and_expr() !BAnd {
}
}
fn unary_reference(value BUnary) &BUnary {
return &value
}
fn (mut p BParser) unary_expr() !BUnary {
t := p.peek(0)
match t.kind {
@@ -72,12 +76,12 @@ fn (mut p BParser) unary_expr() !BUnary {
if nt.kind in [.tfact, .tdefine] {
ident := p.unary_expr()!
return BNot{
expr: ident
expr: unary_reference(BUnary(ident))
}
}
expr := p.expr()!
return BNot{
expr: expr
expr: unary_reference(BUnary(expr))
}
}
.tparen_open {
@@ -43,9 +43,9 @@ fn mutate_bar(mut bar Bar) int {
}
fn main() {
mut bar := Bar{
mut bar := Foobar(Bar{
y: 0
}
})
mutate_foobar1(mut bar)
mutate_foobar2(mut bar)
}
@@ -1,9 +0,0 @@
vlib/v/checker/tests/sum_type_holding_alias_ptr_err.vv:3:12: error: sum type cannot hold a reference type
1 | type Alias1 = &int
2 | type Alias2 = &string
3 | type Qwe = Alias1 | Alias2
| ~~~~~~
4 |
5 | fn main() {
Details: declare alias as non-reference type: `Qwe = Alias1 | ...`
and use a reference to the sum type instead: `var := &Qwe(Alias1(val))`
@@ -1,6 +0,0 @@
type Alias1 = &int
type Alias2 = &string
type Qwe = Alias1 | Alias2
fn main() {
}
@@ -1,33 +0,0 @@
vlib/v/checker/tests/sum_type_ref_variant_err.vv:10:33: error: sum type cannot hold a reference type
8 |
9 | type AliasType = string
10 | type Alphabet1 = Abc | string | &Xyz
| ~~~~
11 | type Alphabet2 = Abc | &Xyz | string
12 | type Alphabet3 = &Xyz | Abc | string
Details: declare the sum type with non-reference types: `Alphabet1 = Xyz | ...`
and use a reference to the sum type instead: `var := &Alphabet1(Xyz{val})`
vlib/v/checker/tests/sum_type_ref_variant_err.vv:11:24: error: sum type cannot hold a reference type
9 | type AliasType = string
10 | type Alphabet1 = Abc | string | &Xyz
11 | type Alphabet2 = Abc | &Xyz | string
| ~~~~
12 | type Alphabet3 = &Xyz | Abc | string
13 | type Alphabet4 = Xyz | Abc | &AliasType
Details: declare the sum type with non-reference types: `Alphabet2 = Xyz | ...`
and use a reference to the sum type instead: `var := &Alphabet2(Xyz{val})`
vlib/v/checker/tests/sum_type_ref_variant_err.vv:12:18: error: sum type cannot hold a reference type
10 | type Alphabet1 = Abc | string | &Xyz
11 | type Alphabet2 = Abc | &Xyz | string
12 | type Alphabet3 = &Xyz | Abc | string
| ~~~~
13 | type Alphabet4 = Xyz | Abc | &AliasType
Details: declare the sum type with non-reference types: `Alphabet3 = Xyz | ...`
and use a reference to the sum type instead: `var := &Alphabet3(Xyz{val})`
vlib/v/checker/tests/sum_type_ref_variant_err.vv:13:30: error: sum type cannot hold a reference type
11 | type Alphabet2 = Abc | &Xyz | string
12 | type Alphabet3 = &Xyz | Abc | string
13 | type Alphabet4 = Xyz | Abc | &AliasType
| ~~~~~~~~~~
Details: declare the sum type with non-reference types: `Alphabet4 = AliasType | ...`
and use a reference to the sum type instead: `var := &Alphabet4(AliasType(val))`
@@ -1,13 +0,0 @@
struct Abc {
val string
}
struct Xyz {
foo string
}
type AliasType = string
type Alphabet1 = Abc | string | &Xyz
type Alphabet2 = Abc | &Xyz | string
type Alphabet3 = &Xyz | Abc | string
type Alphabet4 = Xyz | Abc | &AliasType
@@ -5,8 +5,3 @@ vlib/v/checker/tests/sumtype_define_recursively.vv:3:6: notice: unused parameter
| ~~~~~
4 | return 0
5 | }
vlib/v/checker/tests/sumtype_define_recursively.vv:1:13: error: sum type `Expr` cannot be defined recursively
1 | type Expr = fn ([]Expr) Expr | int
| ~~~~~~~~~~~~~~~~
2 |
3 | fn f(exprs []Expr) Expr {
@@ -12,28 +12,32 @@ import v.cmdexec
const inplace_transform_program = 'module main
struct PtrT {
base_type Ty
base_type &Ty
}
struct OptT {
pad i64
base_type Ty
base_type &Ty
}
struct ResT {
pad1 i64
pad2 i64
base_type Ty
base_type &Ty
}
type Ty = PtrT | OptT | ResT | int
fn ty_reference(value Ty) &Ty {
return &value
}
fn unwrap_stmt(t Ty) int {
mut typ := t
for {
match typ {
PtrT, OptT, ResT {
typ = typ.base_type
typ = *typ.base_type
}
else {
break
@@ -47,7 +51,7 @@ fn unwrap_value(t Ty) int {
mut typ := t
layers := match typ {
PtrT, OptT, ResT {
typ = typ.base_type
typ = *typ.base_type
1
}
else {
@@ -61,7 +65,7 @@ fn unwrap_return(t Ty) int {
mut typ := t
return match typ {
PtrT, OptT, ResT {
typ = typ.base_type
typ = *typ.base_type
if typ is int { typ } else { -1 }
}
else {
@@ -87,14 +91,14 @@ fn collect(m map[string]Local, names []string) []Local {
fn main() {
values := [Ty(PtrT{
base_type: Ty(1)
base_type: ty_reference(Ty(1))
}), Ty(OptT{
pad: 100
base_type: Ty(2)
base_type: ty_reference(Ty(2))
}), Ty(ResT{
pad1: 100
pad2: 200
base_type: Ty(3)
base_type: ty_reference(Ty(3))
})]
for value in values {
println("\${unwrap_stmt(value)} \${unwrap_value(value)} \${unwrap_return(value)}")
+2 -3
View File
@@ -745,11 +745,10 @@ fn run_driver_with_stdin_file(v3_bin string, args []string, stdin_path string) o
}
fn collect_driver_process_result(mut process os.Process) os.Result {
process.set_redirect_stdio()
process.set_redirect_stdio_merged()
process.run()
process.wait()
mut output := process.stdout_slurp()
output += process.stderr_slurp()
process.wait()
if process.err.len > 0 {
output += process.err
}
@@ -18,10 +18,10 @@ fn run_driver_review_process(program string, args []string, environment map[stri
mut process := os.new_process(program)
process.set_args(args)
process.set_environment(environment)
process.set_redirect_stdio()
process.set_redirect_stdio_merged()
process.run()
output := process.stdout_slurp()
process.wait()
output := process.stdout_slurp() + process.stderr_slurp()
result := os.Result{
exit_code: process.code
output: output
@@ -45,6 +45,8 @@ fn generic_sum_type_assert_no_bst_branch_wrap_leaks(c_code string) {
assert c_code.contains('struct main__Node_f64 {'), c_code
assert c_code.contains('struct Tree_int {'), c_code
assert c_code.contains('struct Tree_f64 {'), c_code
assert c_code.contains('main__Node_int* _ptr_Node_int;'), c_code
assert c_code.contains('main__Node_f64* _ptr_Node_f64;'), c_code
assert !c_code.contains('Tree_unknown'), c_code
assert !c_code.contains('Node_unknown'), c_code
assert !c_code.contains('\nNode '), c_code
@@ -70,28 +72,28 @@ struct Node[T] {
right Tree[T]
}
type Tree[T] = Empty | Node[T]
type Tree[T] = Empty | &Node[T]
fn leaf[T](value T, empty Tree[T]) Tree[T] {
return Node[T]{value, empty, empty}
return &Node[T]{value, empty, empty}
}
fn (tree Tree[T]) size[T]() int {
return match tree {
Empty { 0 }
Node[T] { 1 + tree.left.size() + tree.right.size() }
&Node[T] { 1 + tree.left.size() + tree.right.size() }
}
}
fn (tree Tree[T]) add_left[T](value T) Tree[T] {
return match tree {
Empty {
Node[T]{value, tree, tree}
&Node[T]{value, tree, tree}
}
Node[T] {
Node[T]{
&Node[T] {
&Node[T]{
...tree
left: Node[T]{value, Empty{}, Empty{}}
left: &Node[T]{value, Empty{}, Empty{}}
}
}
}
@@ -131,6 +133,8 @@ fn main() {
assert c_code.contains('struct main__Node_f64 {'), c_code
assert c_code.contains('struct Tree_int {'), c_code
assert c_code.contains('struct Tree_f64 {'), c_code
assert c_code.contains('main__Node_int* _ptr_Node_int;'), c_code
assert c_code.contains('main__Node_f64* _ptr_Node_f64;'), c_code
assert !c_code.contains('Array_fixed_Node_T'), c_code
assert !c_code.contains('Node_T'), c_code
assert !c_code.contains('tree.Node_T'), c_code
@@ -148,7 +152,7 @@ struct Node[T] {
right Tree[T]
}
type Tree[T] = Empty | Node[T]
type Tree[T] = Empty | &Node[T]
enum Direction {
left
@@ -156,7 +160,7 @@ enum Direction {
same
}
fn direction[T](tree Node[T], value T) Direction {
fn direction[T](tree &Node[T], value T) Direction {
if value < tree.value {
return .left
}
@@ -169,23 +173,23 @@ fn direction[T](tree Node[T], value T) Direction {
fn (tree Tree[T]) size[T]() int {
return match tree {
Empty { 0 }
Node[T] { 1 + tree.left.size() + tree.right.size() }
&Node[T] { 1 + tree.left.size() + tree.right.size() }
}
}
fn (tree Tree[T]) insert[T](value T) Tree[T] {
return match tree {
Empty {
Node[T]{value, tree, tree}
&Node[T]{value, tree, tree}
}
Node[T] {
&Node[T] {
if value < tree.value {
Node[T]{
&Node[T]{
...tree
left: tree.left.insert(value)
}
} else if tree.value < value {
Node[T]{
&Node[T]{
...tree
right: tree.right.insert(value)
}
@@ -199,18 +203,18 @@ fn (tree Tree[T]) insert[T](value T) Tree[T] {
fn insert_match[T](tree Tree[T], value T) Tree[T] {
return match tree {
Empty {
Node[T]{value, tree, tree}
&Node[T]{value, tree, tree}
}
Node[T] {
&Node[T] {
match direction(tree, value) {
.left {
Node[T]{
&Node[T]{
...tree
left: insert_match(tree.left, value)
}
}
.right {
Node[T]{
&Node[T]{
...tree
right: insert_match(tree.right, value)
}
+6 -8
View File
@@ -2394,10 +2394,8 @@ fn main() {
assert c_code.contains('new_map(sizeof(string), sizeof(i64)')
}
// test_optional_map_or_stages_zeroed_map_after_used_filter_transform
// validates this v3 regression case.
fn test_optional_map_or_stages_zeroed_map_after_used_filter_transform() {
mut a, mut tc := parse_checked_source('option_map_or_new_map_cgen', '
fn test_optional_map_unwrap_does_not_construct_fallback_storage() {
mut a, mut tc := parse_checked_source('option_map_unwrap_no_default_allocation', '
fn maybe_map() ?map[string]int {
return none
}
@@ -2408,16 +2406,16 @@ fn main() {
}
')
mut used := markused.mark_used(a, tc)
assert used['new_map']
used = transform.transform_with_used(mut a, tc, used)
tc.diagnose_unknown_calls = false
tc.reject_unlowered_map_mutation = true
tc.annotate_types()
mut g := cgen.FlatGen.new()
c_code := g.gen_with_used_options(a, used, tc, true)
// The staged map value is zeroed instead of allocated.
assert c_code.contains('\tmap __or_val_'), c_code
assert c_code.all_after('\tmap __or_val_').all_before('\n').ends_with(' = {0};'), c_code
body := c_code.all_after('int main(').all_before('\n}')
assert body.contains('.ok')
assert body.contains('.value')
assert !body.contains('new_map(')
}
// test_string_compound_assign_lowers_to_plus_after_used_filter_transform
@@ -214,7 +214,7 @@ struct Other[T] {
value T
}
type Tree[T] = Empty | Node[T]
type Tree[T] = Empty | &Node[T]
'
fn test_generic_sum_type_checker_accepts_shared_field_on_concrete_sum() {
@@ -247,27 +247,27 @@ fn test_generic_sum_type_checker_accepts_methods_returns_and_smartcasts() {
fn (tree Tree[T]) size[T]() int {
return match tree {
Empty { 0 }
Node[T] { 1 +
&Node[T] { 1 +
tree.left.size() }
}
}
fn make_int_tree() Tree[int] {
return Node[int]{
return &Node[int]{
value: 1
left: Empty{}
}
}
fn make_tree[T](value T) Tree[T] {
return Node[T]{
return &Node[T]{
value: value
left: Empty{}
}
}
fn field_after_is[T](tree Tree[T]) T {
if tree is Node[T] {
if tree is &Node[T] {
return tree.value
}
return T(0)
@@ -287,7 +287,7 @@ fn test_generic_sum_type_checker_rejects_wrong_generic_variant() {
assert_checker_error_contains('generic_sum_wrong_variant_arg', generic_sum_defs +
'
fn bad() Tree[int] {
return Node[string]{
return &Node[string]{
value: "bad"
left: Empty{}
}
@@ -295,7 +295,7 @@ fn bad() Tree[int] {
fn main() {}
',
'cannot use `Node[string]` as type `Tree[int]` in return argument')
'cannot use `&Node[string]` as type `Tree[int]` in return argument')
}
fn test_generic_sum_type_checker_rejects_non_variant_pattern() {
@@ -173,9 +173,12 @@ fn test_imported_module_fn_short_name_does_not_pollute_builtin_return_type() {
assert run.output.trim_space() == 'ok'
generated := os.read_file(out + '.c') or { panic(err) }
assert generated.contains('DenseArray new_dense_array(i64 key_bytes, i64 value_bytes);'), generated
assert generated.contains('.key_values = new_dense_array(key_bytes, value_bytes)')
|| generated.contains('.key_values = (DenseArray){.key_bytes = key_bytes, .value_bytes = value_bytes'), generated
signature := 'DenseArray new_dense_array(i64 key_bytes, i64 value_bytes, ' +
'u8 key_alignment, u8 value_alignment);'
storage := '.key_values = (DenseArray){' +
'.key_bytes = key_bytes, .value_bytes = value_bytes'
assert generated.contains(signature), generated
assert generated.contains(storage), generated
assert generated.contains('collisionmod__new_dense_array_T_v_int'), generated
assert generated.contains('localmod__helper()'), generated
assert generated.contains('localmod__run(command, args)'), generated
@@ -560,8 +560,8 @@ fn main() {
}
')
assert out == "Animal(Cat{\n name: 'Tom'\n})"
assert c_source.contains('replace_T_Animal(value, replacement);')
assert !c_source.contains('replace_T_Animal(&value, replacement);')
assert c_source.contains('replace_T_Animal(value, ')
assert !c_source.contains('replace_T_Animal(&value, ')
}
fn test_mut_param_reassign_keeps_invalid_assignments_rejected() {
@@ -396,23 +396,8 @@ fn main() {
assert out == '7'
}
fn test_sum_type_rejects_pointer_variants() {
fn test_sum_type_checks_pointer_variant_identity() {
v3_bin := build_v3()
run_bad(v3_bin, 'pointer_sum_variant', 'struct Foo {
value int
}
type Item = &Foo | int
fn main() {}
', 'sum type cannot hold a reference type')
run_bad(v3_bin, 'pointer_alias_sum_variant', 'struct Foo {}
type FooPointer = &Foo
type Item = FooPointer | int
fn main() {}
', 'sum type cannot hold a reference type')
run_bad(v3_bin, 'is_pointer_value_variant_rejected', 'struct Foo {}
type Item = Foo | int
@@ -3782,8 +3767,13 @@ fn make_alias() &Bare {
assert c_source.contains('v3_aligned_free(d)')
assert c_source.contains('v3_aligned_free(pa)')
assert c_source.contains('v3_aligned_free(ha)')
// An escaping by-value parameter is copied to the heap at function entry (V1).
assert c_source.contains('v3_aligned_memdup(&__esc_'), c_source
param_signature := 'main__Bare* make_param_alias(main__Bare __heap_value_param_0) {'
param_body := c_fn_body(c_source, param_signature)
assert param_body.contains('main__Bare __esc_1 = __heap_value_param_0;')
assert param_body.contains('v3_aligned_memdup(&__esc_1,')
assert param_body.count('v3_aligned_memdup(') == 1
assert param_body.contains('sizeof(main__Bare)')
assert param_body.contains('__alignof__(*(main__Bare*)(NULL))')
make_direct_body := c_fn_body(c_source, 'main__Bare* make_direct(void) {')
assert make_direct_body.contains('v3_aligned_memdup(&(main__Bare){.x = 5}, sizeof(main__Bare), __alignof__(main__Bare))'), make_direct_body
assert !c_source.contains('__alignof__(Bare)')
@@ -4649,12 +4639,15 @@ fn main() {
}
}
}
println(read(d["a"]))
println(read(Value(d["a"])))
}
'
c_source := gen_c(v3_bin, 'overloaded_index_sum_ref_arg_materializes_getter_result_c', source)
assert c_source.contains('sum_ref_arg'), c_source
assert !c_source.contains('&Dict__index('), c_source
entry_signature := if c_source.contains('int wmain(') { 'int wmain(' } else { 'int main(' }
main_body := c_fn_body(c_source, entry_signature)
assert main_body.contains('main__read(&((Value[])'), main_body
assert main_body.count('Dict__op_index(') == 1, main_body
assert !main_body.contains('&Dict__op_index('), main_body
out := run_good(v3_bin, 'overloaded_index_sum_ref_arg_materializes_getter_result', source)
assert out == 'ok'
}
@@ -7862,8 +7855,8 @@ fn test_nested_generic_receiver_call_waits_for_receiver_type() {
struct Node[T] {
value T
left Tree[T]
right Tree[T]
left &Tree[T]
right &Tree[T]
}
type Tree[T] = Empty | Node[T]
@@ -7882,7 +7875,7 @@ fn (tree Tree[T]) delete[T](value T) Tree[T] {
Node[T]{
...tree
value: tree.right.min()
right: tree.right.delete(tree.right.min())
right: &Tree[T](tree.right.delete(tree.right.min()))
}
}
}
@@ -17,13 +17,14 @@ mut:
}
fn update_loop(mut g Gen, node Stmt) {
node_ref := &node
match node {
ForCStmt {
saved := g.inner_loop
g.inner_loop = unsafe { &node }
g.inner_loop = node_ref
if node.label != '' {
unsafe {
g.labeled_loops[node.label] = &node
g.labeled_loops[node.label] = node_ref
}
}
g.inner_loop = saved
@@ -1940,7 +1940,7 @@ fn test_smartcast_receiver_methods_keep_cast_paths_and_declared_sum_methods() {
out := run_good(v3_bin, 'smartcast_receiver_method_paths', 'struct Leaf {}
struct Container {
left Expr
left &Expr
}
struct Assignment {
@@ -1968,7 +1968,7 @@ fn (leaf &Leaf) is_mut() bool {
fn main() {
container_expr := Expr(Container{
left: Expr(Leaf{})
left: &Expr(Leaf{})
})
if container_expr is Container && (container_expr as Container).left is Leaf {
println((container_expr as Container).left.is_mut())
@@ -13317,7 +13317,7 @@ fn inspect(node &Node, data voidptr) {
fn main() {
item := &Item{}
inspect(item, unsafe { nil })
inspect(Node(*item), unsafe { nil })
}
')
assert out == 'true\ntrue'
@@ -44,7 +44,7 @@ fn test_selfhost_parallel_workers_grow_for_struct_defaults() {
for parallel in [true, false] {
output := os.join_path(root, if parallel { 'parallel' } else { 'serial' })
// The stage table is printed only with `-v` or `-show-timings` (2a45ec1b41).
mut args := ['-nocache', '-building-v', '-show-timings', '-o', output]
mut args := ['-v', '-nocache', '-building-v', '-show-timings', '-o', output]
if !parallel {
args << '-no-parallel'
}
+1 -1
View File
@@ -1,6 +1,6 @@
## This file will be included by all other .expect scripts in this folder.
set timeout 10
set stty_init {rows 24 cols 80}
expect_after timeout { exit 1}
set test_file [lindex $argv 1]
spawn [lindex $argv 0]
expect_after timeout { exit 1}
+2 -2
View File
@@ -4,10 +4,10 @@ source "common.tcl"
expect "Break on * sumtype in ${test_file}:17"
expect "${test_file}:17 vdbg> "
send "p a\n"
expect "a = Test{\r\n a: MySum(false)\r\n} (*Test)"
expect "a = Test{\r\n a: MySum(false)\r\n} (Test)"
send "c\n"
expect "${test_file}:26 vdbg> "
send "p b\n"
expect "b = Test{\r\n a: MySum(1)\r\n} (*Test)"
expect "b = Test{\r\n a: MySum(1)\r\n} (Test)"
send "q\n"
expect eof
+3 -3
View File
@@ -1,5 +1,5 @@
struct Test {
a MySum
a &MySum
}
struct Test2 {
@@ -10,7 +10,7 @@ type MySum = Test | bool | int
fn sumtype() {
a := MySum(Test{
a: false
a: &MySum(false)
})
if a is Test {
dump(a)
@@ -18,7 +18,7 @@ fn sumtype() {
}
b := ?MySum(Test{
a: 1
a: &MySum(1)
})
if b != none {
if b is Test {
+23 -22
View File
@@ -32,24 +32,24 @@ import v.types
// of the compiler spells in a string literal: the functions that generated code
// can call without a call in the program. `cached_declarations_test.v` works the
// list out from the sources again and names what is missing here.
const cached_runtime_function_names = ['__as_cast', '__new_array', '__new_array_noscan', 'arguments',
'array_sort_move', 'atoi', 'autostr_addr_pop', 'autostr_addr_type_in_stack',
'autostr_addr_type_push', 'builtin_init', 'closure_create_with_data', 'closure_data', 'closure_init',
'closure_try_destroy', 'common_parse_int', 'common_parse_uint', 'common_parse_uint2', 'copy',
'cstring_to_vstring', 'data_to_hex_string', 'drop_owned', 'drop_owned_v3_interface', 'eprint',
'eprintln', 'error', 'error_with_code', 'exit', 'fabs', 'fast_string_eq', 'format_int',
'format_uint', 'free', 'gc_runtime_init', 'is_digit', 'is_space', 'isnil', 'malloc', 'malloc_noscan',
'malloc_uncollectable', 'map_clone_int_1', 'map_clone_int_16', 'map_clone_int_2', 'map_clone_int_4',
'map_clone_int_8', 'map_clone_string', 'map_eq_int_1', 'map_eq_int_16', 'map_eq_int_2',
'map_eq_int_4', 'map_eq_int_8', 'map_eq_string', 'map_free_nop', 'map_free_string', 'map_hash_int_1',
'map_hash_int_16', 'map_hash_int_2', 'map_hash_int_4', 'map_hash_int_8', 'map_hash_string',
'map_map_eq', 'memdup', 'memdup_align', 'memdup_noscan', 'min', 'new_array_from_c_array',
'new_array_from_c_array_no_alloc', 'new_array_from_c_array_noscan', 'new_builder', 'new_map',
'panic', 'panic_debug', 'panic_frame_done', 'panic_frame_pop', 'panic_frame_push',
'panic_frame_relink', 'panic_frames_reset', 'panic_recover_frame', 'parse_int', 'parse_uint',
'parser', 'print', 'println', 'ptr_str', 'quote', 'recover', 'repeat', 'string_plus_many',
'tos2', 'v_fixed_index', 'v_fixed_index_i64', 'v_fixed_index_u64', 'v_gettid', 'vcalloc',
'vgc_scan_range']
const cached_runtime_function_names = ['__as_cast', '__new_array', '__new_array_aligned',
'__new_array_noscan', 'arguments', 'array_sort_move', 'atoi', 'autostr_addr_pop',
'autostr_addr_type_in_stack', 'autostr_addr_type_push', 'builtin_init', 'closure_create_with_data',
'closure_data', 'closure_init', 'closure_try_destroy', 'common_parse_int', 'common_parse_uint',
'common_parse_uint2', 'copy', 'cstring_to_vstring', 'data_to_hex_string', 'drop_owned',
'drop_owned_v3_interface', 'eprint', 'eprintln', 'error', 'error_with_code', 'exit', 'fabs',
'fast_string_eq', 'format_int', 'format_uint', 'free', 'gc_runtime_init', 'is_digit', 'is_space',
'isnil', 'malloc', 'malloc_noscan', 'malloc_uncollectable', 'map_clone_int_1', 'map_clone_int_16',
'map_clone_int_2', 'map_clone_int_4', 'map_clone_int_8', 'map_clone_string', 'map_eq_int_1',
'map_eq_int_16', 'map_eq_int_2', 'map_eq_int_4', 'map_eq_int_8', 'map_eq_string', 'map_free_nop',
'map_free_string', 'map_hash_int_1', 'map_hash_int_16', 'map_hash_int_2', 'map_hash_int_4',
'map_hash_int_8', 'map_hash_string', 'map_map_eq', 'memdup', 'memdup_align', 'memdup_noscan',
'min', 'new_array_from_c_array', 'new_array_from_c_array_aligned', 'new_array_from_c_array_no_alloc',
'new_array_from_c_array_noscan', 'new_builder', 'new_map', 'new_map_aligned', 'panic', 'panic_debug',
'panic_frame_done', 'panic_frame_pop', 'panic_frame_push', 'panic_frame_relink',
'panic_frames_reset', 'panic_recover_frame', 'parse_int', 'parse_uint', 'parser', 'print',
'println', 'ptr_str', 'quote', 'recover', 'repeat', 'string_plus_many', 'tos2', 'v_fixed_index',
'v_fixed_index_i64', 'v_fixed_index_u64', 'v_gettid', 'vcalloc', 'vgc_scan_range']
// cached_module_function_names are the plain functions of the other modules of
// vlib that a stage of the compiler spells in a string literal, with the name of
@@ -81,10 +81,11 @@ const cached_module_function_names = ['debug.after_call_hook', 'debug.before_cal
// among them; neither is one that fits no function of vlib, which is the name of
// something else: a temporary, a type, a keyword. `cached_declarations_test.v`
// works this list out again too.
const cached_function_name_patterns = ['*.arg*', '*.free', '*.json', '*.len', '*_break', '*_callback',
'*_clone', '*_continue', '*_free', '*_hash', '*_idx', '*_key_*', '*_map_key', '*_object', '*_source',
'*_str', '*_thread_args', '*_value_*', '*data', '*defined', '*nil', '*void', '.arg*', 'arg*',
'main*', 'map_clone_int_*', 'map_eq_int_*', 'map_hash_int_*', 'ptr_*', 'voidptr*', 'vpc_*_*']
const cached_function_name_patterns = ['*.arg*', '*.free', '*.json', '*.len', '*_aligned', '*_break',
'*_callback', '*_clone', '*_continue', '*_free', '*_hash', '*_idx', '*_key_*', '*_map_key',
'*_object', '*_source', '*_str', '*_thread_args', '*_value_*', '*data', '*defined', '*nil',
'*void', '.arg*', 'arg*', 'main*', 'map_clone_int_*', 'map_eq_int_*', 'map_hash_int_*', 'ptr_*',
'voidptr*', 'vpc_*_*']
// module_lifecycle_function_names are the functions that a program calls for each
// of its modules that has them, without a call in its source.
+17 -21
View File
@@ -7486,11 +7486,11 @@ fn merge_monomorph_cache_specs(cached []transform.MonomorphCacheSpec, generated
return merged
}
// clone_string_bool_map promotes a string-keyed set out of a disposable stage arena.
fn clone_string_bool_map(values map[string]bool) map[string]bool {
mut cloned := map[string]bool{}
cloned.reserve(u32(values.len))
for key, value in values {
cloned[key.clone()] = value
cloned[key] = value
}
return cloned
}
@@ -7498,7 +7498,7 @@ fn clone_string_bool_map(values map[string]bool) map[string]bool {
fn clone_string_string_map(values map[string]string) map[string]string {
mut cloned := map[string]string{}
for key, value in values {
cloned[key.clone()] = value.clone()
cloned[key] = value.clone()
}
return cloned
}
@@ -7506,7 +7506,7 @@ fn clone_string_string_map(values map[string]string) map[string]string {
fn clone_nested_string_bool_map(values map[string]map[string]bool) map[string]map[string]bool {
mut cloned := map[string]map[string]bool{}
for key, value in values {
cloned[key.clone()] = clone_string_bool_map(value)
cloned[key] = clone_string_bool_map(value)
}
return cloned
}
@@ -7883,7 +7883,7 @@ fn clone_struct_field_map(values map[string][]types.StructField) map[string][]ty
is_volatile: field.is_volatile
}
}
cloned[name.clone()] = owned_fields
cloned[name] = owned_fields
}
return cloned
}
@@ -7891,7 +7891,7 @@ fn clone_struct_field_map(values map[string][]types.StructField) map[string][]ty
fn clone_string_list_map(values map[string][]string) map[string][]string {
mut cloned := map[string][]string{}
for name, items in values {
cloned[name.clone()] = clone_string_list(items)
cloned[name] = clone_string_list(items)
}
return cloned
}
@@ -7913,6 +7913,7 @@ fn clone_type_owner_c_abi_fields(values map[u32]map[string]string) map[u32]map[s
}
fn promote_scoped_type_metadata(mut tc types.TypeChecker) {
tc.runtime_type_indexes = tc.runtime_type_indexes.clone()
// Transform and specialization can grow these maps inside a disposable arena,
// so move both their storage and string payloads before releasing that arena.
tc.fn_ret_type_texts = tc.fn_ret_type_texts.clone_owned()
@@ -7962,7 +7963,7 @@ fn promote_scoped_checker_node_caches(mut tc types.TypeChecker, a &flat.FlatAst,
tc.resolved_call_names[idx] = types.promote_cached_name(tc.resolved_call_names[idx], scope)
}
if idx >= generated_start && idx < tc.expr_type_set.len && tc.expr_type_set[idx] {
tc.expr_type_values[idx] = types.clone_owned_type(tc.expr_type_values[idx])
tc.expr_type_values[idx] = types.new_type_reference(types.Type(types.clone_owned_type(tc.expr_type_values[idx])))
}
}
}
@@ -7972,7 +7973,7 @@ fn promote_scoped_checker_node_caches(mut tc types.TypeChecker, a &flat.FlatAst,
for idx in rewritten_base_nodes {
if idx >= 0 && idx < tc.expr_type_set.len && tc.expr_type_set[idx]
&& idx < tc.expr_type_values.len {
tc.expr_type_values[idx] = types.clone_owned_type(tc.expr_type_values[idx])
tc.expr_type_values[idx] = types.new_type_reference(types.Type(types.clone_owned_type(tc.expr_type_values[idx])))
}
}
// The dense caches are reserved in the parent arena, but an unexpectedly
@@ -9171,20 +9172,19 @@ fn ast_contains_sql_expr(a &flat.FlatAst) bool {
}
fn restore_transformed_fn_value_types(mut tc types.TypeChecker, a &flat.FlatAst, used_fns map[string]bool) {
void_type := types.builtin_type_value('void')
for tc.expr_type_values.len < a.nodes.len {
tc.expr_type_values << void_type
tc.expr_type_values << types.empty_type
tc.expr_type_set << false
}
// Signature arrays and payloads stay immutable throughout this restoration.
mut fn_value_types := map[string]types.Type{}
mut fn_value_types := map[string]&types.Type{}
for idx, name in tc.sparse_resolved_fn_values {
if idx < 0 || idx >= a.nodes.len {
continue
}
params := tc.fn_param_types[name] or { continue }
ret := tc.fn_ret_types[name] or { continue }
tc.expr_type_values[idx] = restored_fn_value_type(name, params, ret, tc.fn_variadic[name] or { false }, mut fn_value_types)
tc.expr_type_values[idx] = restored_fn_value_type(tc, name, params, ret, tc.fn_variadic[name] or { false }, mut fn_value_types)
tc.expr_type_set[idx] = true
}
mut cur_module := ''
@@ -9241,7 +9241,7 @@ fn restore_transformed_fn_value_types(mut tc types.TypeChecker, a &flat.FlatAst,
if name.len > 0 {
params := tc.fn_param_types[name] or { []types.Type{} }
if ret := tc.fn_ret_types[name] {
tc.expr_type_values[callee_idx] = restored_fn_value_type(name, params, ret, tc.fn_variadic[name] or { false }, mut fn_value_types)
tc.expr_type_values[callee_idx] = restored_fn_value_type(tc, name, params, ret, tc.fn_variadic[name] or { false }, mut fn_value_types)
tc.expr_type_set[callee_idx] = true
}
}
@@ -9260,7 +9260,7 @@ fn restore_transformed_fn_value_types(mut tc types.TypeChecker, a &flat.FlatAst,
&& (tc.resolved_fn_value_name(base_id) or { '' }) == cname {
params := tc.fn_param_types[cname] or { []types.Type{} }
if ret := tc.fn_ret_types[cname] {
tc.expr_type_values[base_idx] = restored_fn_value_type(cname, params, ret, tc.fn_variadic[cname] or { false }, mut fn_value_types)
tc.expr_type_values[base_idx] = restored_fn_value_type(tc, cname, params, ret, tc.fn_variadic[cname] or { false }, mut fn_value_types)
tc.expr_type_set[base_idx] = true
}
}
@@ -9276,16 +9276,12 @@ fn restore_transformed_fn_value_types(mut tc types.TypeChecker, a &flat.FlatAst,
}
}
// restored_fn_value_type preserves the variadic tail in restored expression metadata.
fn restored_fn_value_type(name string, params []types.Type, ret types.Type, is_variadic bool, mut fn_value_types map[string]types.Type) types.Type {
// restored_fn_value_type preserves canonical references and variadic expression metadata.
fn restored_fn_value_type(tc &types.TypeChecker, name string, params []types.Type, ret types.Type, is_variadic bool, mut fn_value_types map[string]&types.Type) &types.Type {
if cached := fn_value_types[name] {
return cached
}
typ := types.Type(types.FnType{
params: params
is_variadic: is_variadic
return_type: ret
})
typ := tc.intern_type_reference(types.Type(tc.variadic_fn_type(params, ret, []bool{}, is_variadic)))
fn_value_types[name] = typ
return typ
}
@@ -27,7 +27,7 @@ fn test_restored_types_extend_unset_slots_and_preserve_sparse_signature_modes()
params := [types.Type(types.Pointer{ base_type: integer }), types.Type(types.Array{
elem_type: boolean
})]
tc.expr_type_values = [boolean]
tc.expr_type_values = [tc.intern_type_reference(boolean)]
tc.expr_type_set = [true]
tc.fn_param_types['retain'] = params
tc.fn_ret_types['retain'] = integer
@@ -42,24 +42,24 @@ fn test_restored_types_extend_unset_slots_and_preserve_sparse_signature_modes()
restore_transformed_fn_value_types(mut tc, &a, map[string]bool{})
assert tc.expr_type_values.len == a.nodes.len
assert tc.expr_type_set.len == a.nodes.len
assert tc.expr_type_values[int(existing)] == boolean && tc.expr_type_set[int(existing)]
assert *tc.expr_type_values[int(existing)] == boolean && tc.expr_type_set[int(existing)]
assert tc.expr_type_values[int(unset)] is types.Void && !tc.expr_type_set[int(unset)]
assert tc.expr_type_values[int(missing)] is types.Void && !tc.expr_type_set[int(missing)]
expected := types.Type(types.FnType{ params: params, is_variadic: true, return_type: integer })
assert tc.expr_type_values[int(first)] == expected && tc.expr_type_set[int(first)]
assert tc.expr_type_values[int(second)] == expected && tc.expr_type_set[int(second)]
expected := types.Type(tc.variadic_fn_type(params, &integer, []bool{}, true))
assert *tc.expr_type_values[int(first)] == expected && tc.expr_type_set[int(first)]
assert *tc.expr_type_values[int(second)] == expected && tc.expr_type_set[int(second)]
// Replacing one expression slot cannot alter another restored signature.
tc.expr_type_values[int(first)] = boolean
assert tc.expr_type_values[int(second)] == expected
tc.expr_type_values[int(first)] = tc.intern_type_reference(boolean)
assert *tc.expr_type_values[int(second)] == expected
assert tc.fn_param_types['retain'] == params
assert tc.fn_variadic['retain'] && tc.mut_receiver_methods['retain']
assert (tc.expr_type_values[int(second)] as types.FnType).is_variadic
// A later restoration must observe a refreshed fixed-array signature.
tc.fn_variadic['retain'] = false
restore_transformed_fn_value_types(mut tc, &a, map[string]bool{})
fixed := types.Type(types.FnType{ params: params, return_type: integer })
assert tc.expr_type_values[int(first)] == fixed
assert tc.expr_type_values[int(second)] == fixed
fixed := types.Type(tc.fn_type(params, &integer, []bool{}))
assert *tc.expr_type_values[int(first)] == fixed
assert *tc.expr_type_values[int(second)] == fixed
}
fn test_restored_direct_types_keep_module_names_and_c_receiver_identity() {
@@ -105,19 +105,10 @@ fn test_restored_direct_types_keep_module_names_and_c_receiver_identity() {
'first.run': true
'second.run': true
})
assert tc.expr_type_values[int(first_callee)] == types.Type(types.FnType{
params: [integer]
return_type: integer
})
assert tc.expr_type_values[int(second_callee)] == types.Type(types.FnType{
params: [boolean]
return_type: boolean
})
assert tc.expr_type_values[int(native)] == types.Type(types.FnType{
params: [boolean]
return_type: integer
})
assert tc.expr_type_values[int(local)] == types.Type(types.Struct{ name_id: flat.source_type_id('Receiver') })
assert *tc.expr_type_values[int(first_callee)] == types.Type(tc.fn_type([integer], integer, []bool{}))
assert *tc.expr_type_values[int(second_callee)] == types.Type(tc.fn_type([boolean], boolean, []bool{}))
assert *tc.expr_type_values[int(native)] == types.Type(tc.fn_type([boolean], integer, []bool{}))
assert *tc.expr_type_values[int(local)] == types.Type(types.Struct{ name_id: flat.source_type_id('Receiver') })
assert !tc.expr_type_set[int(unused_callee)]
assert_restoration_preserves_canonical_node_texts(&a, canonical_nodes, canonical_text_count)
// A second restoration observes refreshed signatures rather than retaining
@@ -127,10 +118,7 @@ fn test_restored_direct_types_keep_module_names_and_c_receiver_identity() {
'main': true
'first.run': true
})
assert tc.expr_type_values[int(first_callee)] == types.Type(types.FnType{
params: [integer]
return_type: boolean
})
assert *tc.expr_type_values[int(first_callee)] == types.Type(tc.fn_type([integer], boolean, []bool{}))
assert_restoration_preserves_canonical_node_texts(&a, canonical_nodes, canonical_text_count)
}
+29
View File
@@ -0,0 +1,29 @@
module driver
fn test_promoted_set_survives_source_release() {
mut source := map[string]bool{}
for index in 0 .. 1024 {
source[index.str()] = index % 2 == 0
}
for index in 0 .. 1023 {
source.delete(index.str())
}
promoted := clone_string_bool_map(source)
unsafe { source.free() }
assert promoted.len == 1
assert '1023' in promoted
assert !promoted['1023']
}
fn test_promoted_nested_sets_have_independent_storage() {
mut source := {
'entry': {
'field': true
}
}
mut promoted := clone_nested_string_bool_map(source)
promoted['entry']['next'] = true
source['entry']['field'] = false
assert promoted['entry']['field']
assert 'next' !in source['entry']
}
+48 -14
View File
@@ -62,7 +62,7 @@ pub struct MapValue {
pub mut:
key_type_name u32
value_type_name u32
default_value Value
default_value &Value
entries []MapEntry
}
@@ -90,7 +90,7 @@ pub:
type_name u32
variant_name u32
pub mut:
payload Value
payload &Value
}
pub struct TupleValue {
@@ -1923,7 +1923,7 @@ fn (mut e Eval) set_selector_value(container Value, field_name string, value Val
}
SumValue {
mut sv := container
sv.payload = e.set_selector_value(sv.payload, field_name, value)!
sv.payload = &Value(e.set_selector_value(sv.payload, field_name, value)!)
return sv
}
else {
@@ -2058,7 +2058,7 @@ fn (mut e Eval) smartcast_binding_from_condition(cond_id flat.NodeId) ?Smartcast
fn (e &Eval) smartcast_value(value Value, target_type_id u32) ?Value {
if value is SumValue {
if e.type_name_matches(value.variant_name, target_type_id) || e.value_matches_type_name(value.payload, target_type_id) {
return value.payload
return *value.payload
}
}
return none
@@ -4219,7 +4219,7 @@ fn (mut e Eval) eval_map_init_flow(node &flat.Node) !FlowSignal {
mut m := MapValue{
key_type_name: key_type
value_type_name: value_type
default_value: e.zero_value_for_type_name(value_type)
default_value: &Value(e.zero_value_for_type_name(value_type))
}
for j, key in keys {
m = e.map_set_value(m, key, values[j])
@@ -4963,7 +4963,7 @@ fn (mut e Eval) call_value_method(receiver Value, receiver_type_name_id u32, met
receiver: SumValue{
type_name: receiver.type_name
variant_name: receiver.variant_name
payload: result.receiver
payload: &Value(result.receiver)
}
mutated_args: result.mutated_args
}
@@ -5615,7 +5615,7 @@ fn (mut e Eval) zero_value_like(value Value) Value {
return SumValue{
type_name: value.type_name
variant_name: value.variant_name
payload: e.zero_value_like(value.payload)
payload: &Value(e.zero_value_like(value.payload))
}
}
else {
@@ -5661,7 +5661,7 @@ fn (mut e Eval) zero_value_for_type_name_in_module(type_name_id u32, module_name
return MapValue{
key_type_name: qualified_key_type
value_type_name: qualified_value_type
default_value: e.zero_value_for_type_name_in_module(qualified_value_type, module_name)
default_value: &Value(e.zero_value_for_type_name_in_module(qualified_value_type, module_name))
}
}
enum_name := e.qualify_type_name(module_name, name)
@@ -5673,10 +5673,10 @@ fn (mut e Eval) zero_value_for_type_name_in_module(type_name_id u32, module_name
}
sum_name := e.qualify_type_name(module_name, name)
if e.sum_types.contains(flat.source_type_name(sum_name)) {
return SumValue{ type_name: sum_name, payload: void_value() }
return SumValue{ type_name: sum_name, payload: &Value(void_value()) }
}
if e.sum_types.contains(flat.source_type_name(name)) {
return SumValue{ type_name: name, payload: void_value() }
return SumValue{ type_name: name, payload: &Value(void_value()) }
}
struct_name := e.resolve_struct_type_name_in_module(name, module_name)
if flat.source_type_name(struct_name) in e.structs { return e.zero_struct_value(struct_name) }
@@ -5831,8 +5831,8 @@ fn (e &Eval) adapt_value_to_type_name(value Value, type_name_id u32) Value {
mut m := MapValue{
key_type_name: qualified_key_type_name
value_type_name: qualified_value_type_name
default_value: e.adapt_value_to_type_name(value.default_value,
qualified_value_type_name)
default_value: &Value(e.adapt_value_to_type_name(value.default_value,
qualified_value_type_name))
}
for entry in value.entries {
m = e.map_set_value(m, entry.key, entry.value)
@@ -5863,14 +5863,14 @@ fn (e &Eval) is_option_like_value(value Value) bool {
}
fn (e &Eval) wrap_sum_value(type_name_id u32, value Value) Value {
return SumValue{ type_name: type_name_id, variant_name: e.runtime_type_id(value), payload: value }
return SumValue{ type_name: type_name_id, variant_name: e.runtime_type_id(value), payload: &Value(value) }
}
fn (e &Eval) unwrap_sum_cast_value(value Value, type_name_id u32) Value {
if value is SumValue {
if !e.type_name_matches(value.type_name, type_name_id)
&& (e.type_name_matches(value.variant_name, type_name_id) || e.value_matches_type_name(value.payload, type_name_id)) {
return value.payload
return *value.payload
}
}
return value
@@ -6164,6 +6164,9 @@ fn (e &Eval) sizeof_type_layout(raw_name_id u32, module_name string, seen []Type
context := e.declaration_type_context(alias.module_name, alias.file_name)
return context.sizeof_type_layout(alias.target, alias.module_name, visited)
}
if variants := e.sum_types.type_ids(name) {
return e.sizeof_sum_layout(variants, module_name, visited)
}
if name.starts_with('map[') || name.starts_with('fn ') || name.starts_with('chan ')
|| name.starts_with('thread ') {
return TypeLayout{ size: 8, align: 8 }
@@ -6250,6 +6253,37 @@ fn (e &Eval) sizeof_type_layout(raw_name_id u32, module_name string, seen []Type
}
}
fn (e &Eval) sizeof_sum_layout(variants []u32, module_name string, seen []TypeLayoutVisit) TypeLayout {
mut size := i64(1)
mut align := i64(1)
mut owns_pointer := false
for variant in variants {
layout := e.sizeof_type_layout(variant, module_name, seen)
if layout.size > size { size = layout.size }
if layout.align > align { align = layout.align }
owns_pointer = owns_pointer || e.sum_variant_has_pointer_payload(variant, module_name, []u32{})
}
header := if owns_pointer { i64(8) } else { i64(4) }
offset := sizeof_aligned_size(header, align)
if align < 4 { align = 4 }
return TypeLayout{ size: sizeof_aligned_size(offset + size, align), align: align }
}
// sum_variant_has_pointer_payload follows nominal aliases without entering value containers.
fn (e &Eval) sum_variant_has_pointer_payload(raw_name_id u32, module_name string, seen []u32) bool {
name_id := e.qualify_nested_type_name(module_name, raw_name_id)
name := flat.source_type_name(name_id)
if name.starts_with('&') || name in ['voidptr', 'byteptr', 'charptr'] { return true }
if name_id in seen { return false }
if alias := e.type_alias_info_in_module(name_id, module_name) {
mut visited := seen.clone()
visited << name_id
context := e.declaration_type_context(alias.module_name, alias.file_name)
return context.sum_variant_has_pointer_payload(alias.target, alias.module_name, visited)
}
return false
}
fn (e &Eval) sizeof_fixed_array_len(text string, module_name string, seen []flat.NodeId) ?i64 {
if len := fixed_array_len_literal(text) {
return i64(len)
+4
View File
@@ -769,6 +769,10 @@ fn (g &Gen) stack_slot_size(ptr_id int) ?int {
if off_val.kind == .constant {
field_off := int(parse_arm64_int(off_val.name))
for fi in 0 .. base.fields.len {
// Union members overlap; the typed projection selects their extent.
if base.is_union && base.fields[fi] != g.ptr_elem_type(cur) {
continue
}
if g.m.struct_field_offset(base_type, fi) == field_off {
field_size := g.m.struct_field_size(base_type, fi)
if field_size > 0 {
+85
View File
@@ -0,0 +1,85 @@
module arm64
import v.ssa
fn test_inline_sum_member_copy_uses_the_selected_union_member_size() {
for words in [1, 2, 3] {
mut m := ssa.Module.new()
i8_type := m.type_store.get_int(8)
i32_type := m.type_store.get_int(32)
i64_type := m.type_store.get_int(64)
small := m.type_store.register(ssa.Type{ kind: .struct_t, fields: [i8_type], field_names: ['dummy'] })
pair := m.type_store.register(ssa.Type{ kind: .struct_t, fields: [i64_type, i64_type], field_names: [
'name',
'reference',
] })
triple := m.type_store.register(ssa.Type{ kind: .struct_t, fields: [i64_type, i64_type,
i64_type], field_names: ['x', 'y', 'z'] })
members := [small, pair, triple]
member := members[words - 1]
payload := m.type_store.register(ssa.Type{ kind: .struct_t, is_union: true, fields: members, field_names: [
'Small',
'Pair',
'Triple',
] })
choice := m.type_store.register(ssa.Type{ kind: .struct_t, fields: [i32_type, payload], field_names: [
'typ',
'_payload',
] })
function := m.new_function('copy_payload', ssa.TypeID(0))
block := m.add_block(function, 'entry')
storage := m.add_instr(.alloca, block, m.type_store.get_ptr(choice), [])
payload_offset := m.get_or_add_const(i64_type, '8')
zero := m.get_or_add_const(i64_type, '0')
payload_ptr := m.add_instr(.get_element_ptr, block, m.type_store.get_ptr(payload), [
storage,
payload_offset,
])
member_ptr := m.add_instr(.get_element_ptr, block, m.type_store.get_ptr(member), [
payload_ptr,
zero,
])
value := m.add_instr(.struct_init, block, member, [])
store := m.add_instr(.store, block, ssa.TypeID(0), [value, member_ptr])
mut g := Gen.new(m)
g.reset_value_slots(&m.funcs[function])
g.stack_size = 128
g.set_alloca_slot(int(storage), -128, m.type_size(choice), 8)
g.set_stack_slot(int(member_ptr), -80)
g.set_stack_slot(int(value), -64)
assert g.aggregate_store_size(int(member_ptr), member) == m.type_size(member)
assert g.aggregate_load_size(int(member_ptr), member) == m.type_size(member)
g.gen_instr(int(store))
if words == 1 {
assert asm_str_b(Reg(8), Reg(9)) in g.text_words
} else {
// The pointer word in Pair and every word in Triple must reach the payload.
for word in 0 .. words {
assert asm_str_imm(Reg(8), Reg(9), u32(word)) in g.text_words
}
}
}
}
fn test_ordinary_struct_member_copy_keeps_its_storage_bound() {
mut m := ssa.Module.new()
i64_type := m.type_store.get_int(64)
large := m.type_store.register(ssa.Type{ kind: .struct_t, fields: [i64_type, i64_type, i64_type] })
pair := m.type_store.register(ssa.Type{ kind: .struct_t, fields: [i64_type, i64_type], field_names: [
'left',
'right',
] })
function := m.new_function('bounded_copy', ssa.TypeID(0))
block := m.add_block(function, 'entry')
storage := m.add_instr(.alloca, block, m.type_store.get_ptr(pair), [])
zero := m.get_or_add_const(i64_type, '0')
field := m.add_instr(.get_element_ptr, block, m.type_store.get_ptr(i64_type), [
storage,
zero,
])
mut g := Gen.new(m)
g.reset_value_slots(&m.funcs[function])
g.set_alloca_slot(int(storage), -32, m.type_size(pair), 8)
assert g.aggregate_store_size(int(field), large) == 8
assert g.aggregate_load_size(int(field), large) == 8
}
+41 -5
View File
@@ -4,6 +4,7 @@ import os
import v.parser
import v.pref
import v.ssa
import v.transform
import v.types
fn test_native_aligned_memdup_copies_bytes_and_honors_alignment() {
@@ -32,7 +33,7 @@ fn main() {
invalid := v3_aligned_memdup(&source[0], 4, 24)
if usize(invalid) != 0 { C.exit(5) }
}
')
', false)
}
}
@@ -52,7 +53,7 @@ fn main() {
if values.len != 2 || boxed.len != 1 { C.exit(3) }
C.free(boxed)
}
')
', false)
}
}
@@ -72,11 +73,11 @@ fn main() {
}
if usize(values[0]) != 11 || usize(values[1]) != 22 { C.exit(1) }
}
')
', false)
}
}
fn run_native_allocation_fixture(name string, source string) {
fn run_native_allocation_fixture(name string, source string, lower bool) {
path := os.join_path(os.vtmp_dir(), 'arm64_allocation_${name}_${os.getpid()}.v')
output := path.all_before_last('.')
defer {
@@ -85,16 +86,51 @@ fn run_native_allocation_fixture(name string, source string) {
}
os.write_file(path, source) or { panic(err) }
mut p := parser.Parser.new(pref.new_preferences())
a := p.parse_file(path)
mut a := p.parse_file(path)
assert p.diagnostics.len == 0, p.diagnostics.str()
mut tc := types.TypeChecker.new(a)
tc.collect(a)
tc.annotate_types()
assert tc.errors.len == 0, tc.errors.str()
if lower {
transform.transform(mut a, tc)
}
m := ssa.build_with_used(a, map[string]bool{}, tc)
mut g := Gen.new(m)
g.gen()
g.write_and_link(output)
result := os.exec([output])
assert result.exit_code == 0, '${name}: exit ${result.exit_code}: ${result.output}'
if lower {
c_output := output + '_c'
defer { os.rm(c_output) or {} }
compiled := os.exec([@VEXE, '-b', 'c', '-nocache', '-gc', 'none', '-o', c_output, path])
assert compiled.exit_code == 0, compiled.output
c_result := os.exec([c_output])
assert c_result.exit_code == 0, '${name}: C exit ${c_result.exit_code}: ${c_result.output}'
}
}
fn test_native_conditional_void_pointer_values_preserve_addresses() {
$if macos && arm64 {
run_native_allocation_fixture('conditional_pointers', 'module main
fn C.exit(int)
fn receive(value voidptr) voidptr { return value }
fn choose(flag bool, first voidptr, second voidptr) voidptr {
value := if flag { first } else { second }
return receive(value)
}
fn main() {
mut values := [2]int{}
values[0] = 7
values[1] = 11
// These references borrow the fixed array storage only for this function.
first := unsafe { voidptr(&values[0]) }
second := unsafe { voidptr(&values[1]) }
if choose(true, first, second) != first { C.exit(1) }
if choose(false, first, second) != second { C.exit(2) }
if values[0] != 7 || values[1] != 11 { C.exit(3) }
}
', true)
}
}
+6 -1
View File
@@ -13,9 +13,14 @@ fn test_native_returned_registration_fixed_array_fields_preserve_string_elements
cgen := os.read_file(os.join_path(@VMODROOT, 'vlib', 'v', 'gen', 'c', 'cleanc.v'))!
registration := 'struct FnSignatureRegistration {' +
cgen.all_after('struct FnSignatureRegistration {').all_before('\n}') + '\n}\n'
source := definitions.all_before('// clone_owned_type').replace('module types', 'module main') +
source := definitions.all_before('// new_type_reference').replace('module types', 'module main') +
registration.replace('types.Type', 'Type') + r'
fn C.exit(int)
fn C.malloc(usize) voidptr
fn C.memcpy(voidptr, voidptr, usize) voidptr
fn memdup(value voidptr, size isize) voidptr {
return C.memcpy(C.malloc(usize(size)), value, usize(size))
}
fn make_registration() FnSignatureRegistration {
mut aliases := [6]string{}
aliases[0] = "sum"
+19 -14
View File
@@ -15,6 +15,8 @@ fn test_native_negative_sum_guards_preserve_enum_payload_fields() {
// Use the compiler's recursive Type layout and its actual Enum tag.
source := type_source.all_before('// clone_owned_type').replace('module types', 'module main').replace('import v.flat', '') +
r'
const builtin_void_type = Type(Void{})
const empty_type = &builtin_void_type
fn C.exit(int)
fn C.malloc(usize) voidptr
fn C.memcpy(voidptr, voidptr, usize) voidptr
@@ -26,9 +28,10 @@ fn __as_cast(obj voidptr, obj_type int, expected_type int, obj_name string, expe
if obj_type != expected_type { C.exit(80) }
return obj
}
fn unalias_type(value Type) Type {
if value is Alias { return unalias_type(value.base_type) }
return value
fn unalias_type(value &Type) &Type {
mut current := value
for current is Alias { current = current.base_type }
return current
}
fn fixture_type_name(id u32) string {
return match id {
@@ -41,7 +44,7 @@ fn fixture_type_name(id u32) string {
else { "other" }
}
}
fn (value Type) name() string {
fn (value &Type) name() string {
if value is Enum { return fixture_type_name(value.name_id) }
if value is Struct { return fixture_type_name(value.name_id) }
return "other"
@@ -69,17 +72,17 @@ fn whole_sum_receiver(subject Type) string {
fn same_function_shape(expected FnType, actual Type) bool {
if actual !is FnType || expected.params.len != actual.params.len { return false }
for i in 0 .. expected.params.len {
if expected.params[i].name() != actual.params[i].name() { return false }
if expected.params[i].typ.name() != actual.params[i].typ.name() { return false }
}
return actual.return_type.name() == expected.return_type.name()
}
struct TypeChecker {}
fn fn_param_modes_compatible(actual FnType, expected FnType, idx int) bool {
actual_mut := idx < actual.params_mut.len && actual.params_mut[idx]
expected_mut := idx < expected.params_mut.len && expected.params_mut[idx]
actual_mut := idx < actual.params.len && actual.params[idx].is_mut
expected_mut := idx < expected.params.len && expected.params[idx].is_mut
return actual_mut == expected_mut
}
fn fn_compatible_param_type(value FnType, idx int) Type { return value.params[idx] }
fn fn_compatible_param_type(value FnType, idx int) Type { return *value.params[idx].typ }
fn (tc &TypeChecker) types_match_ignoring_module_qualification(expected Type, actual Type) bool {
return expected.name() == actual.name()
}
@@ -122,7 +125,7 @@ fn main() {
flag := Type(Enum{name_id: 2, is_flag: true})
listed := Type(Enum{name_id: 3, is_flag: false})
other := Type(Struct{name_id: 4})
alias := Type(Alias{name_id: 5, base_type: plain})
alias := Type(Alias{name_id: 5, base_type: new_type_reference(plain)})
if needs_else(plain, flags, true) { C.exit(1) }
if !needs_else(flag, flags, true) { C.exit(2) }
if !needs_else(listed, flags, true) { C.exit(3) }
@@ -135,19 +138,21 @@ fn main() {
|| !compound_guard(other, true) || !compound_guard(plain, false) { C.exit(9) }
if whole_sum_receiver(plain) != "Plain" || whole_sum_receiver(flag) != "guarded"
|| whole_sum_receiver(other) != "guarded" { C.exit(10) }
signature := FnType{params: [other], return_type: plain}
signature := FnType{params: [FnParam{typ: new_type_reference(other)}], return_type: new_type_reference(plain)}
if !same_function_shape(signature, Type(signature)) { C.exit(11) }
if same_function_shape(signature, Type(FnType{return_type: plain})) { C.exit(12) }
if same_function_shape(signature, Type(FnType{return_type: new_type_reference(plain)})) { C.exit(12) }
if same_function_shape(signature, other) { C.exit(13) }
if same_function_shape(signature, Type(FnType{params: [plain], return_type: plain})) { C.exit(14) }
if same_function_shape(signature, Type(FnType{params: [FnParam{typ: new_type_reference(plain)}], return_type: new_type_reference(plain)})) { C.exit(14) }
if reassign_after_guard(plain) != "changed" || reassign_after_guard(other) != "other" { C.exit(15) }
if branch_join(plain, true) != "Plain" || branch_join(flag, true) != "flag"
|| branch_join(other, false) != "Other" || branch_join(other, true) != "other" { C.exit(16) }
checker := TypeChecker{}
if !checker.fn_types_match_ignoring_module_qualification(signature, Type(signature)) { C.exit(17) }
if checker.fn_types_match_ignoring_module_qualification(signature, Type(FnType{return_type: plain})) { C.exit(18) }
if checker.fn_types_match_ignoring_module_qualification(signature, Type(FnType{return_type: new_type_reference(plain)})) { C.exit(18) }
if checker.fn_types_match_ignoring_module_qualification(signature, other) { C.exit(19) }
if checker.fn_types_match_ignoring_module_qualification(signature, Type(FnType{params: [other], return_type: other})) { C.exit(20) }
if checker.fn_types_match_ignoring_module_qualification(signature, Type(FnType{params: [FnParam{typ: new_type_reference(other)}], return_type: new_type_reference(other)})) { C.exit(20) }
mutable := FnType{params: [FnParam{typ: new_type_reference(other), is_mut: true}], return_type: new_type_reference(plain)}
if checker.fn_types_match_ignoring_module_qualification(signature, Type(mutable)) { C.exit(21) }
}
'
for building_v in [false, true] {
+52 -27
View File
@@ -7,13 +7,20 @@ import v.ssa
import v.transform
import v.types
fn test_native_compiler_type_identity_uses_the_active_payload_slot() {
fn test_native_compiler_types_preserve_semantic_payload_identity() {
$if macos && arm64 {
type_source := os.read_file(os.join_path(@VMODROOT, 'vlib', 'v', 'types', 'type.v'))!
checker := os.read_file(os.join_path(@VMODROOT, 'vlib', 'v', 'types', 'checker_tail_stmt.v'))!
source := type_source.all_before('// clone_owned_type').replace('module types', 'module main') +
'\nfn type_value_words(' + checker.all_after('fn type_value_words(').all_before('// type_cache_stats') +
r'
identity_source := os.read_file(os.join_path(@VMODROOT, 'vlib', 'v', 'types', 'type_interner.v'))!
assert identity_source.contains('pub fn semantic_type_hash(')
assert identity_source.contains('pub fn semantic_types_equal(')
assert identity_source.contains('pub fn (tc &TypeChecker) fn_type(')
// Run the production semantic hash/equality over the real recursive Type
// layout; copied inline descriptors no longer have boxed payload identity.
identity_functions := 'pub fn semantic_type_hash(' + identity_source.all_after('pub fn semantic_type_hash(').all_before('pub fn (tc &TypeChecker) fn_type(')
source := type_source.all_before('// clone_owned_type').replace('module types', 'module main').replace('import v.flat', '') +
'\n' + identity_functions + r'
const builtin_void_type = Type(Void{})
const empty_type = &builtin_void_type
fn C.exit(int)
fn C.malloc(usize) voidptr
fn C.calloc(usize, usize) voidptr
@@ -22,35 +29,53 @@ fn memdup(value voidptr, size isize) voidptr {
result := C.malloc(usize(size))
return C.memcpy(result, value, usize(size))
}
// SSA sum storage starts with the i32 tag declared in ssa/builder.v.
struct NativeSumHeader { typ i32 }
fn native_sum_tag(value &Type) i32 {
return unsafe { (&NativeSumHeader(value)).typ }
}
fn main() {
integer := Type(Primitive{props: .integer, size: 32})
boolean := Type(Primitive{props: .boolean, size: 8})
copied_integer := integer
integer_tag, integer_payload, integer_slot := type_value_words(&integer)
boolean_tag, boolean_payload, _ := type_value_words(&boolean)
copy_tag, copy_payload, copy_slot := type_value_words(&copied_integer)
if integer_tag != boolean_tag || integer_payload == 0 || boolean_payload == 0
|| integer_payload == boolean_payload { C.exit(1) }
if copy_tag != integer_tag || copy_payload != integer_payload || copy_slot != integer_slot { C.exit(2) }
array := Type(Array{elem_type: integer})
optional_array := Type(OptionType{base_type: array})
if integer !is Primitive || boolean !is Primitive
|| semantic_types_equal(&integer, &boolean)
|| semantic_type_hash(&integer) == semantic_type_hash(&boolean) { C.exit(1) }
if !semantic_types_equal(&copied_integer, &integer)
|| semantic_type_hash(&copied_integer) != semantic_type_hash(&integer) { C.exit(2) }
array := Type(Array{elem_type: new_type_reference(integer)})
optional_array := Type(OptionType{base_type: new_type_reference(array)})
tuple := Type(MultiReturn{types: [integer, array, boolean]})
optional_tuple := Type(OptionType{base_type: tuple})
optional_tuple := Type(OptionType{base_type: new_type_reference(tuple)})
copied_optional := optional_array
array_tag, array_payload, array_slot := type_value_words(&optional_array)
tuple_tag, tuple_payload, _ := type_value_words(&optional_tuple)
copied_tag, copied_payload, copied_slot := type_value_words(&copied_optional)
if array_tag != tuple_tag || array_payload == 0 || tuple_payload == 0
|| array_payload == tuple_payload { C.exit(3) }
if copied_tag != array_tag || copied_payload != array_payload || copied_slot != array_slot { C.exit(4) }
first := Type(FnType{params: [integer], return_type: boolean})
second := Type(FnType{params: [boolean], return_type: boolean})
first_tag, first_payload, _ := type_value_words(&first)
second_tag, second_payload, _ := type_value_words(&second)
if first_tag != second_tag || first_payload == 0 || first_payload == second_payload { C.exit(5) }
if optional_array !is OptionType || optional_tuple !is OptionType
|| semantic_types_equal(&optional_array, &optional_tuple)
|| semantic_type_hash(&optional_array) == semantic_type_hash(&optional_tuple) { C.exit(3) }
if !semantic_types_equal(&copied_optional, &optional_array)
|| semantic_type_hash(&copied_optional) != semantic_type_hash(&optional_array) { C.exit(4) }
first := Type(FnType{params: [FnParam{typ: new_type_reference(integer)}], return_type: new_type_reference(boolean)})
second := Type(FnType{params: [FnParam{typ: new_type_reference(boolean)}], return_type: new_type_reference(boolean)})
if first !is FnType || second !is FnType || semantic_types_equal(&first, &second)
|| semantic_type_hash(&first) == semantic_type_hash(&second) { C.exit(5) }
zero := unsafe { &Type(C.calloc(1, usize(sizeof(Type)))) }
zero_tag, zero_payload, zero_slot := type_value_words(zero)
if zero_tag != 0 || zero_payload != 0 || zero_slot != 0 { C.exit(6) }
if native_sum_tag(zero) != 0 { C.exit(6) }
valid_void := Type(Void{})
if native_sum_tag(&valid_void) != 1 || valid_void !is Void
|| !semantic_types_equal(&valid_void, &builtin_void_type)
|| semantic_type_hash(&valid_void) != semantic_type_hash(&builtin_void_type) { C.exit(6) }
equivalent_integer := Type(Primitive{props: .integer, size: 32})
equivalent_array := Type(Array{elem_type: new_type_reference(equivalent_integer)})
equivalent_optional := Type(OptionType{base_type: new_type_reference(equivalent_array)})
if !semantic_types_equal(&integer, &equivalent_integer)
|| semantic_type_hash(&integer) != semantic_type_hash(&equivalent_integer)
|| !semantic_types_equal(&optional_array, &equivalent_optional)
|| semantic_type_hash(&optional_array) != semantic_type_hash(&equivalent_optional) { C.exit(7) }
mutable := Type(FnType{params: [FnParam{typ: new_type_reference(integer), is_mut: true}], return_type: new_type_reference(boolean)})
if semantic_types_equal(&first, &mutable)
|| semantic_type_hash(&first) == semantic_type_hash(&mutable) { C.exit(8) }
variadic := Type(FnType{params: [FnParam{typ: new_type_reference(integer)}], return_type: new_type_reference(boolean), is_variadic: true})
if semantic_types_equal(&first, &variadic)
|| semantic_type_hash(&first) == semantic_type_hash(&variadic) { C.exit(9) }
}
'
for building_v in [false, true] {
+80 -68
View File
@@ -2,7 +2,7 @@ module c
import v.flat
import v.gen.c.naming
import v.types
import v.types { unalias_type }
import strings
// array_like_type supports array like type handling for c.
@@ -11,7 +11,7 @@ fn array_like_type(t types.Type) ?types.Array {
return t
}
if t is types.Alias {
base := t.base_type
base := *t.base_type
if base is types.Array {
return base
}
@@ -21,7 +21,7 @@ fn array_like_type(t types.Type) ?types.Array {
// array_fixed_type supports array fixed type handling for c.
fn array_fixed_type(t types.Type) ?types.ArrayFixed {
clean := cgen_unalias_type(t)
clean := unalias_type(t)
if clean is types.ArrayFixed {
return clean
}
@@ -29,7 +29,7 @@ fn array_fixed_type(t types.Type) ?types.ArrayFixed {
}
fn fixed_array_pointer_type(t types.Type) ?types.ArrayFixed {
clean := if t is types.Alias { t.base_type } else { t }
clean := if t is types.Alias { *t.base_type } else { t }
if clean is types.Pointer {
return array_fixed_type(clean.base_type)
}
@@ -37,8 +37,8 @@ fn fixed_array_pointer_type(t types.Type) ?types.ArrayFixed {
}
fn (mut g FlatGen) fixed_array_decay_byte_compatible(actual types.Type, expected types.Type) bool {
a := cgen_unalias_type(actual)
e := cgen_unalias_type(expected)
a := unalias_type(actual)
e := unalias_type(expected)
if a is types.ArrayFixed && e is types.ArrayFixed {
return g.fixed_array_len_value(a) == g.fixed_array_len_value(e)
&& g.fixed_array_decay_byte_compatible(a.elem_type, e.elem_type)
@@ -50,8 +50,8 @@ fn (mut g FlatGen) fixed_array_decay_byte_compatible(actual types.Type, expected
}
fn (mut g FlatGen) fixed_array_decay_shape_equal(actual types.Type, expected types.Type, allow_byte_cast bool) bool {
a := cgen_unalias_type(actual)
e := cgen_unalias_type(expected)
a := unalias_type(actual)
e := unalias_type(expected)
if a is types.ArrayFixed {
return e is types.ArrayFixed && g.fixed_array_len_value(a) == g.fixed_array_len_value(e)
&& g.fixed_array_decay_shape_equal(a.elem_type, e.elem_type, allow_byte_cast)
@@ -93,8 +93,9 @@ fn (mut g FlatGen) gen_fixed_array_index(index_id flat.NodeId, len string) {
g.gen_expr(index_id)
return
}
index_type := cgen_unalias_type(g.usable_expr_type(index_id))
helper := match index_type.name() {
index_type := unalias_type(g.usable_expr_type(index_id))
index_type_name := string(index_type.name())
helper := match index_type_name {
'i64' { 'v_fixed_index_i64' }
'u64' { 'v_fixed_index_u64' }
else { 'v_fixed_index' }
@@ -127,7 +128,7 @@ fn (g &FlatGen) fixed_array_type_from_alias_text(type_name_id u32) ?types.ArrayF
return none
}
fn runtime_array_struct_type(t types.Type) bool {
fn runtime_array_struct_type(t &types.Type) bool {
if t is types.Struct {
return t.name() == 'array' || t.name() == 'Array'
}
@@ -149,6 +150,18 @@ fn runtime_array_struct_index_info(t types.Type) (bool, bool) {
return false, false
}
fn (mut g FlatGen) array_allocation_expr(elem_type &types.Type, length string, capacity string) string {
ct := g.value_sizeof_target(elem_type)
layout := g.tc.allocation_layout(elem_type)
if layout.has(.aligned) {
return '__new_array_aligned(${length}, ${capacity}, sizeof(${ct}), __alignof__(${ct}))'
}
if !layout.has(.scanned) {
return '__new_array_noscan(${length}, ${capacity}, sizeof(${ct}))'
}
return 'array_new(sizeof(${ct}), ${length}, ${capacity})'
}
// gen_array_literal_value emits array literal value output for c.
fn (mut g FlatGen) gen_array_literal_value(node flat.Node, elem_type types.Type) {
canonical_elem_type := g.canonical_import_alias_type_for_node(elem_type, &node)
@@ -156,10 +169,13 @@ fn (mut g FlatGen) gen_array_literal_value(node flat.Node, elem_type types.Type)
sizeof_elem := g.value_sizeof_target(canonical_elem_type)
count := node.children_count
if count == 0 {
g.write('array_new(sizeof(${sizeof_elem}), 0, 0)')
g.write(g.array_allocation_expr(canonical_elem_type, '0', '0'))
return
}
new_fn := if count == 1 && array_literal_elem_can_use_noscan(canonical_elem_type) {
aligned := g.tc.requires_aligned_allocation(canonical_elem_type)
new_fn := if aligned {
'new_array_from_c_array_aligned'
} else if count == 1 && !g.tc.allocation_layout(canonical_elem_type).has(.scanned) {
'new_array_from_c_array_noscan'
} else {
'new_array_from_c_array'
@@ -187,7 +203,11 @@ fn (mut g FlatGen) gen_array_literal_value(node flat.Node, elem_type types.Type)
}
g.gen_expr_with_expected_type(child_id, canonical_elem_type)
}
g.write('})')
g.write('}')
if aligned {
g.write(', __alignof__(${sizeof_elem})')
}
g.write(')')
}
// gen_array_init_value emits a dynamic `[]T{len:, cap:, init:}` as a C value.
@@ -199,19 +219,9 @@ fn (mut g FlatGen) gen_array_init_value(node flat.Node, elem_type types.Type) {
init_id := g.array_init_field_value(node, 'init') or { flat.empty_node }
c_elem := g.value_sizeof_target(elem_type)
if int(init_id) < 0 {
g.write('array_new(sizeof(${c_elem}), ')
if int(len_id) >= 0 {
g.gen_expr_with_expected_type(len_id, types.Type(types.int_))
} else {
g.write('0')
}
g.write(', ')
if int(cap_id) >= 0 {
g.gen_expr_with_expected_type(cap_id, types.Type(types.int_))
} else {
g.write('0')
}
g.write(')')
length := if int(len_id) >= 0 { g.expr_to_string(len_id) } else { '0' }
capacity := if int(cap_id) >= 0 { g.expr_to_string(cap_id) } else { '0' }
g.write(g.array_allocation_expr(elem_type, length, capacity))
return
}
len_name := g.tmp_name()
@@ -231,7 +241,8 @@ fn (mut g FlatGen) gen_array_init_value(node flat.Node, elem_type types.Type) {
} else {
g.write('0')
}
g.write('; Array ${array_name} = array_new(sizeof(${c_elem}), ${len_name}, ${cap_name}); ')
allocation := g.array_allocation_expr(elem_type, len_name, cap_name)
g.write('; Array ${array_name} = ${allocation}; ')
g.write('for (${int_type} ${index_name} = 0; ${index_name} < ${array_name}.len; ${index_name}++) { ')
if g.node_contains_ident(init_id, 'index') {
g.write('${int_type} ${g.local_decl_cname('index')} = ${index_name}; ')
@@ -251,12 +262,6 @@ fn (mut g FlatGen) gen_array_init_value(node flat.Node, elem_type types.Type) {
g.write('} ${array_name}; })')
}
fn array_literal_elem_can_use_noscan(elem_type types.Type) bool {
clean := cgen_unalias_type(elem_type)
return clean is types.Primitive || clean is types.Char || clean is types.Rune
|| clean is types.ISize || clean is types.USize || clean is types.Enum
}
fn (mut g FlatGen) gen_array_equality_literal_arg(names []string, arg_idx int, arg_id flat.NodeId, node flat.Node) bool {
if arg_idx !in [0, 1] || node.kind != .array_literal
|| !names.any(it in ['__v3_array_eq_raw', '__v3_array_eq_string', '__v3_array_eq_array']) {
@@ -422,7 +427,7 @@ fn (mut g FlatGen) gen_fixed_array_data_arg(id flat.NodeId, arr types.ArrayFixed
g.write(').data')
return
}
actual := cgen_unalias_type(g.usable_expr_type(id))
actual := unalias_type(g.usable_expr_type(id))
if actual is types.Pointer && fn_type_from(actual.base_type) != none {
// C array parameters decay to pointers. Preserve a callback slot's
// address by supplying its pointer context instead of the array type.
@@ -541,11 +546,11 @@ fn (g &FlatGen) array_init_field_value(node flat.Node, field_name string) ?flat.
}
fn (mut g FlatGen) gen_fixed_array_pointer_lvalue_arg(id flat.NodeId, expected types.Type) bool {
expected_ptr := cgen_unalias_type(expected)
expected_ptr := unalias_type(expected)
if expected_ptr !is types.Pointer {
return false
}
expected_fixed := array_fixed_type(cgen_unalias_type(expected_ptr.base_type))
expected_fixed := array_fixed_type(unalias_type(expected_ptr.base_type))
if int(id) < 0 || int(id) >= g.a.nodes.len {
return false
}
@@ -559,13 +564,13 @@ fn (mut g FlatGen) gen_fixed_array_pointer_lvalue_arg(id flat.NodeId, expected t
if actual is types.Pointer {
return false
}
if actual_fixed := array_fixed_type(cgen_unalias_type(actual)) {
if cgen_unalias_type(expected_ptr.base_type) is types.Void {
if actual_fixed := array_fixed_type(unalias_type(actual)) {
if unalias_type(expected_ptr.base_type) is types.Void {
g.gen_fixed_array_data_arg(id, actual_fixed)
return true
}
if fixed := expected_fixed {
if inner_fixed := array_fixed_type(cgen_unalias_type(actual_fixed.elem_type)) {
if inner_fixed := array_fixed_type(unalias_type(actual_fixed.elem_type)) {
if g.fixed_array_decay_shape_equal(types.Type(inner_fixed), types.Type(fixed),
false) {
if g.expr_is_addressable(id) {
@@ -677,11 +682,19 @@ fn (mut g FlatGen) gen_slice_expr(node flat.Node, base_id flat.NodeId, base_type
if trimmed_space(literal).len > 0 {
data_str = literal
}
aligned := g.tc.requires_aligned_allocation(fixed.elem_type)
allocator := if aligned {
'new_array_from_c_array_aligned'
} else {
'new_array_from_c_array'
}
alignment_arg := if aligned { ', __alignof__(${c_elem})' } else { '' }
len_val := g.fixed_array_len_value(fixed)
copy_expr := '${allocator}(${len_val}, ${len_val}, sizeof(${c_elem}), &(${data_str})[0]${alignment_arg})'
if gated {
// Route gated fixed-array slices through the clamped slice_ni on a
// heap copy of the fixed data, matching dynamic-array semantics.
len_val := g.fixed_array_len_value(fixed)
g.write('array__slice_ni(new_array_from_c_array(${len_val}, ${len_val}, sizeof(${c_elem}), &(${data_str})[0]), ${start_str}, ${end_str})')
g.write('array__slice_ni(${copy_expr}, ${start_str}, ${end_str})')
return
}
// Copy the complete fixed array before slicing so the runtime slice helper
@@ -689,8 +702,7 @@ fn (mut g FlatGen) gen_slice_expr(node flat.Node, base_id flat.NodeId, base_type
array_tmp := g.tmp_name()
start_tmp := g.tmp_name()
end_tmp := g.tmp_name()
len_val := g.fixed_array_len_value(fixed)
g.write('({ Array ${array_tmp} = new_array_from_c_array(${len_val}, ${len_val}, sizeof(${c_elem}), &(${data_str})[0]); int ${start_tmp} = (${start_str}); int ${end_tmp} = (${end_str}); array_slice(${array_tmp}, ${start_tmp}, ${end_tmp}); })')
g.write('({ Array ${array_tmp} = ${copy_expr}; int ${start_tmp} = (${start_str}); int ${end_tmp} = (${end_str}); array_slice(${array_tmp}, ${start_tmp}, ${end_tmp}); })')
} else if is_array {
arr_str := if is_ptr { '*${base_str}' } else { base_str }
if gated {
@@ -731,17 +743,17 @@ fn (g &FlatGen) array_accessor_index_method(node &flat.Node) string {
// gen_array_method_call emits array method call output for c.
fn (mut g FlatGen) gen_array_method_call(node flat.Node, fn_node &flat.Node, arr types.Array) {
base_id := g.a.child(fn_node, 0)
mut elem_type := arr.elem_type
mut elem_type := *arr.elem_type
base_expr_type := g.usable_expr_type(base_id)
receiver_type0 := if g.a.nodes[int(base_id)].kind == .call {
declared := g.declared_call_return_type(base_id)
if declared !is types.Unknown && declared !is types.Void {
declared
} else {
base_expr_type
*base_expr_type
}
} else {
base_expr_type
*base_expr_type
}
receiver_type := types.unwrap_pointer(receiver_type0)
if receiver_arr := array_like_type(receiver_type) {
@@ -965,7 +977,7 @@ fn (mut g FlatGen) gen_array_method_call(node flat.Node, fn_node &flat.Node, arr
g.write('Array_u8__hex(')
if base_node.kind == .array_literal {
g.gen_expr_with_expected_type(base_id, types.Type(types.Array{
elem_type: types.Type(types.u8_)
elem_type: types.new_type_reference(types.Type(types.u8_))
}))
} else {
g.gen_expr(base_id)
@@ -979,7 +991,7 @@ fn (mut g FlatGen) gen_array_method_call(node flat.Node, fn_node &flat.Node, arr
// Any other element type is not a thread, so route it through the normal
// method fallback instead of joining arbitrary array data as pthread_t handles.
mut is_thread := false
elem := cgen_unalias_type(arr.elem_type)
elem := unalias_type(arr.elem_type)
if elem is types.Struct {
tn := trimmed_space(elem.name())
is_thread = tn == 'thread' || tn.starts_with('thread ')
@@ -1141,7 +1153,7 @@ fn (mut g FlatGen) gen_thread_array_wait(base_id flat.NodeId, is_ptr bool, elem_
fn (mut g FlatGen) ensure_thread_arr_wait_fn(ret_name_id u32) string {
is_void := ret_name_id == 0
if !is_void {
ret_type := cgen_unalias_type(g.tc.parse_type(ret_name_id))
ret_type := unalias_type(g.tc.parse_type(ret_name_id))
if ret_type is types.OptionType || ret_type is types.ResultType {
return g.ensure_thread_optional_arr_wait_fn(ret_type)
}
@@ -1180,18 +1192,18 @@ fn (mut g FlatGen) ensure_thread_optional_arr_wait_fn(ret_type types.Type) strin
name := g.cname('__v_thread_arr_wait_${naming.type_name_part(ret_ct)}_array')
g.spawn_wrapper_names[key] = name
base_type := match ret_type {
types.OptionType, types.ResultType { ret_type.base_type }
types.OptionType, types.ResultType { *ret_type.base_type }
else { return name }
}
payload := if base_type is types.Void {
types.Type(base_type)
} else {
types.Type(types.Array{ elem_type: base_type })
types.Type(types.Array{ elem_type: types.new_type_reference(types.Type(base_type)) })
}
array_result_type := if ret_type is types.OptionType {
types.Type(types.OptionType{ base_type: payload })
types.Type(types.OptionType{ base_type: types.new_type_reference(types.Type(payload)) })
} else {
types.Type(types.ResultType{ base_type: payload })
types.Type(types.ResultType{ base_type: types.new_type_reference(types.Type(payload)) })
}
result_ct := g.optional_type_name(array_result_type)
mut def := strings.new_builder(1024)
@@ -1239,7 +1251,7 @@ fn (mut g FlatGen) ensure_thread_optional_arr_wait_fn(ret_type types.Type) strin
}
// array_lookup_suffix supports array lookup suffix handling for c.
fn array_lookup_suffix(elem_type types.Type) string {
fn array_lookup_suffix(elem_type &types.Type) string {
if elem_type is types.String {
return 'string'
}
@@ -1251,7 +1263,7 @@ fn array_lookup_suffix(elem_type types.Type) string {
return 'int'
}
fn array_last_index_suffix(elem_type types.Type) ?string {
fn array_last_index_suffix(elem_type &types.Type) ?string {
elem_name := elem_type.name()
return match elem_name {
'string' { 'string' }
@@ -1328,7 +1340,7 @@ fn (mut g FlatGen) array_method_fallback_for_receiver(method string, base_id fla
return best_mname
}
fn (g &FlatGen) array_elem_type_matches(expected types.Type, actual types.Type) bool {
fn (g &FlatGen) array_elem_type_matches(expected &types.Type, actual &types.Type) bool {
if expected.name() == actual.name() {
return true
}
@@ -1421,7 +1433,7 @@ fn (mut g FlatGen) gen_index_overload_set(node flat.Node, lhs flat.Node, base_id
fn (mut g FlatGen) gen_index_overload_compound_set(lhs flat.Node, base_id flat.NodeId, base_type types.Type, setter types.CallInfo, getter types.CallInfo, assign_op flat.Op, infix_op flat.Op, rhs_id flat.NodeId) {
recv_tmp := g.tmp_name()
index_tmp := g.tmp_name()
recv_type := if base_type is types.Pointer { base_type.base_type } else { base_type }
recv_type := if base_type is types.Pointer { *base_type.base_type } else { base_type }
recv_ct := g.value_c_type(recv_type)
index_ct := g.value_c_type(setter.params[1])
g.write('{ ${recv_ct}* ${recv_tmp} = ')
@@ -1506,7 +1518,7 @@ fn (mut g FlatGen) index_overload_cached_getter_call_string(recv_tmp string, ind
return result
}
fn index_overload_compound_type_is_string(typ types.Type) bool {
fn index_overload_compound_type_is_string(typ &types.Type) bool {
clean := if typ is types.Alias { typ.base_type } else { typ }
return clean is types.String
}
@@ -1638,16 +1650,16 @@ fn (mut g FlatGen) gen_slice_index_value(id flat.NodeId) {
g.write('}')
}
fn cgen_is_builtin_slice_index_type(typ types.Type) bool {
clean0 := select_receive_unalias_type(typ)
fn cgen_is_builtin_slice_index_type(typ &types.Type) bool {
clean0 := unalias_type(typ)
clean := types.unwrap_pointer(clean0)
return clean.name() in ['SliceIndex', 'builtin.SliceIndex']
}
fn cgen_builtin_slice_index_array_elem(typ types.Type) ?types.Type {
clean := select_receive_unalias_type(typ)
clean := unalias_type(typ)
if clean is types.Array && cgen_is_builtin_slice_index_type(clean.elem_type) {
return clean.elem_type
return *clean.elem_type
}
return none
}
@@ -1774,7 +1786,7 @@ fn (mut g FlatGen) gen_index_operator_compound_assign(node flat.Node, lhs flat.N
g.writeln('); }')
}
fn (g &FlatGen) index_operator_type_is_string_like(typ types.Type) bool {
fn (g &FlatGen) index_operator_type_is_string_like(typ &types.Type) bool {
if typ is types.String {
return true
}
@@ -1784,7 +1796,7 @@ fn (g &FlatGen) index_operator_type_is_string_like(typ types.Type) bool {
return false
}
fn (g &FlatGen) index_operator_compound_operator_method(getter_type types.Type, setter_type types.Type, op flat.Op) ?string {
fn (g &FlatGen) index_operator_compound_operator_method(getter_type &types.Type, setter_type &types.Type, op flat.Op) ?string {
if method_name := g.assign_struct_operator_method(getter_type, op) {
return method_name
}
@@ -1824,7 +1836,7 @@ fn (mut g FlatGen) gen_index_operator_tmp_arg(index_tmp string, actual types.Typ
// index expression on base_type calls. A generic specialization can be registered under
// its C name, so the resolved name can already be lowered; c_name would move a name
// that is already in the operator namespace out of it again.
fn (g &FlatGen) index_operator_call_c_name(info types.CallInfo, base_type types.Type, method string) string {
fn (g &FlatGen) index_operator_call_c_name(info types.CallInfo, base_type &types.Type, method string) string {
name := g.index_operator_call_name(info, base_type, method)
if name.starts_with(naming.operator_method_c_prefix) {
return name
@@ -1832,7 +1844,7 @@ fn (g &FlatGen) index_operator_call_c_name(info types.CallInfo, base_type types.
return g.cname(name)
}
fn (g &FlatGen) index_operator_call_name(info types.CallInfo, base_type types.Type, method string) string {
fn (g &FlatGen) index_operator_call_name(info types.CallInfo, base_type &types.Type, method string) string {
if !info.name.contains('[') {
return info.name
}
@@ -2313,7 +2325,7 @@ fn (mut g FlatGen) gen_index_assign(node flat.Node) {
}
if base_type is types.Pointer {
ptr_type := base_type
mut expected_type := ptr_type.base_type
mut expected_type := *ptr_type.base_type
mut fixed_len := ''
_, fixed_is_ptr, fixed := fixed_array_index_info(base_type)
if fixed_is_ptr {
+1 -1
View File
@@ -502,7 +502,7 @@ fn (mut g FlatGen) cache_require_declaration_type(typ types.Type, mut seen map[u
types.FnType {
g.cache_require_declaration_type(typ.return_type, mut seen)
for param in typ.params {
g.cache_require_declaration_type(param, mut seen)
g.cache_require_declaration_type(param.typ, mut seen)
}
}
types.MultiReturn {
+19 -17
View File
@@ -83,21 +83,23 @@ fn test_cache_declarations_include_nested_payloads_and_recursive_fields() {
mut tc := types.TypeChecker.new(&a)
tc.structs['sample.Parent'] = [types.StructField{
name: 'callback'
typ: types.FnType{
params: [types.Type(types.Map{
key_type: types.Type(types.string_)
value_type: types.Type(types.OptionType{
base_type: types.Type(types.Struct{ name_id: flat.source_type_id('sample.Child') })
typ: tc.intern_type_reference(types.Type(types.FnType{
params: [types.FnParam{
typ: &types.Type(types.Map{
key_type: &types.Type(types.string_)
value_type: &types.Type(types.OptionType{
base_type: &types.Type(types.Struct{ name_id: flat.source_type_id('sample.Child') })
})
})
})]
return_type: types.Type(types.void_)
}
}]
return_type: &types.Type(types.void_)
}))
}]
tc.structs['sample.Child'] = [types.StructField{
name: 'parent'
typ: types.Pointer{
base_type: types.Type(types.Struct{ name_id: flat.source_type_id('sample.Parent') })
}
typ: tc.intern_type_reference(types.Type(types.Pointer{
base_type: &types.Type(types.Struct{ name_id: flat.source_type_id('sample.Parent') })
}))
}]
tc.structs['sample.Unused'] = []types.StructField{}
mut g := FlatGen.new()
@@ -182,10 +184,10 @@ fn test_cache_signature_collection_omits_unused_function_payloads() {
tc.structs['sample.Used'] = []types.StructField{}
tc.structs['sample.Unused'] = []types.StructField{}
tc.fn_ret_types['sample.used'] = types.OptionType{
base_type: types.Type(types.Struct{ name_id: flat.source_type_id('sample.Used') })
base_type: &types.Type(types.Struct{ name_id: flat.source_type_id('sample.Used') })
}
tc.fn_ret_types['sample.unused'] = types.OptionType{
base_type: types.Type(types.Struct{ name_id: flat.source_type_id('sample.Unused') })
base_type: &types.Type(types.Struct{ name_id: flat.source_type_id('sample.Unused') })
}
tc.fn_type_modules['sample.used'] = 'sample'
tc.fn_type_modules['sample.unused'] = 'sample'
@@ -247,16 +249,16 @@ fn test_cache_demand_keeps_unused_constant_wrappers_and_their_field_graph_out()
}]
tc.structs['sample.UnusedChild'] = [types.StructField{
name: 'parent'
typ: types.Pointer{ base_type: types.Type(types.Struct{ name_id: flat.source_type_id('sample.Unused') }) }
typ: types.Pointer{ base_type: &types.Type(types.Struct{ name_id: flat.source_type_id('sample.Unused') }) }
}]
tc.const_types['sample.used'] = types.OptionType{
base_type: types.Type(types.Struct{ name_id: flat.source_type_id('sample.Used') })
base_type: &types.Type(types.Struct{ name_id: flat.source_type_id('sample.Used') })
}
tc.const_types['sample.unused'] = types.OptionType{
base_type: types.Type(types.Struct{ name_id: flat.source_type_id('sample.Unused') })
base_type: &types.Type(types.Struct{ name_id: flat.source_type_id('sample.Unused') })
}
tc.const_types['sample.unused_result'] = types.ResultType{
base_type: types.Type(types.Struct{ name_id: flat.source_type_id('sample.Unused') })
base_type: &types.Type(types.Struct{ name_id: flat.source_type_id('sample.Unused') })
}
// A constant and a function can share a V name but use different C symbols.
tc.fn_ret_types['sample.used'] = types.int_
+5 -5
View File
@@ -10,12 +10,12 @@ fn test_void_callback_result_adapter_returns_success() {
g.a = &a
g.tc = &tc
actual := types.FnType{
params: [types.Type(types.int_)]
return_type: types.Type(types.void_)
params: [types.FnParam{ typ: tc.intern_type_reference(types.Type(types.int_)) }]
return_type: types.empty_type
}
expected := types.FnType{
params: actual.params
return_type: types.Type(types.ResultType{ base_type: types.Type(types.void_) })
return_type: tc.intern_type_reference(types.Type(types.ResultType{}))
}
adapter := g.ensure_callback_userdata_wrapper('record', actual, expected, '') or {
assert false, 'a void callback needs a successful Result adapter'
@@ -36,9 +36,9 @@ fn test_void_callback_does_not_supply_nonvoid_result() {
mut g := FlatGen.new()
g.a = &a
g.tc = &tc
actual := types.FnType{ return_type: types.Type(types.void_) }
actual := types.FnType{ return_type: types.empty_type }
expected := types.FnType{
return_type: types.Type(types.ResultType{ base_type: types.Type(types.int_) })
return_type: tc.intern_type_reference(types.Type(types.ResultType{ base_type: tc.intern_type_reference(types.Type(types.int_)) }))
}
assert g.ensure_callback_userdata_wrapper('record', actual, expected, '') == none
}
+375 -1021
View File
File diff suppressed because it is too large. Load diff
+75
View File
@@ -173,3 +173,78 @@ fn test_headerless_cmdexec_poll_uses_bionic_nfds_type() ! {
source])
assert checked.exit_code == 0, checked.output
}
fn test_vectored_output_header_is_redeclared_only_for_the_shipped_binding() {
mut a := flat.FlatAst{}
mut g := FlatGen.new()
g.a = &a
g.compiler_vroot = '/vroot'
g.set_target(pref.target_from('linux', 'amd64')!)
source := '/vroot/vlib/v/gen/c/output_nix.c.v'
assert g.core_native_header_is_redeclared(source, '<sys/uio.h>')
assert !g.core_native_header_is_redeclared(source, '<custom.h>')
assert !g.core_native_header_is_redeclared('/project/v/gen/c/output_nix.c.v', '<sys/uio.h>')
a.cached_header_sources['/cache/cgen.vh'] = source
assert g.core_native_header_is_redeclared('/cache/cgen.vh', '<sys/uio.h>')
// General portable output still uses SDK headers; compiler bootstrap output does not.
g.output_cross_c = true
assert !g.core_native_header_is_redeclared(source, '<sys/uio.h>')
g.set_compile_defines(['v3_headerless_bootstrap'])
assert g.core_native_header_is_redeclared(source, '<sys/uio.h>')
g.output_cross_c = false
g.set_compile_defines([])
g.target_libc_headers = true
assert !g.core_native_header_is_redeclared(source, '<sys/uio.h>')
g.target_libc_headers = false
g.object_file_mode = true
assert !g.core_native_header_is_redeclared(source, '<sys/uio.h>')
g.object_file_mode = false
g.no_builtin = true
assert !g.core_native_header_is_redeclared(source, '<sys/uio.h>')
}
fn test_vectored_output_header_preserves_native_context_and_sdk_ownership() {
mut a := flat.FlatAst.new()
mut tc := types.TypeChecker.new(&a)
mut g := FlatGen.new()
g.a = &a
g.tc = &tc
g.compiler_vroot = '/vroot'
g.set_target(pref.target_from('linux', 'amd64')!)
source := '/vroot/vlib/v/gen/c/output_nix.c.v'
g.register_struct_decl_info('C.iovec', 'C.iovec', 'c', source, flat.Node{})
include_node := flat.Node{
kind: .directive
value: 'include'
typ: flat.source_type_id('<sys/uio.h>')
}
assert g.collect_c_directive('c', include_node, source, false)
assert !g.c_directives_use_system_libc()
assert g.c_directives.len == 0
assert g.core_header_directives.len == 1
assert !g.skip_builtin_struct('C.iovec')
assert g.should_emit_c_extern_decl_from_file('writev', source, 'c')
context := g.native_source_contexts['c'] or { panic('missing native C context') }
assert context.any(it.text == '#include <sys/uio.h>')
// A later application header selects the SDK and restores its iovec/writev ownership.
g.add_c_directive('main', '#include <stdio.h>', false)
g.restore_core_header_directives_for_system_libc()
assert g.c_directives_use_system_libc()
assert g.core_header_directives.len == 0
assert g.c_directives.len == 2
assert g.c_directives[1].text == '#include <sys/uio.h>'
assert g.files_with_c_includes[source]
assert g.skip_builtin_struct('C.iovec')
assert !g.should_emit_c_extern_decl_from_file('writev', source, 'c')
// The same filename outside this compiler's vlib always keeps the user's include.
mut user := FlatGen.new()
user.a = &a
user.tc = &tc
user.compiler_vroot = '/vroot'
user.set_target(pref.target_from('linux', 'amd64')!)
assert user.collect_c_directive('main', include_node, '/project/v/gen/c/output_nix.c.v', false)
assert user.core_header_directives.len == 0
assert user.c_directives.len == 1
assert user.c_directives[0].text == '#include <sys/uio.h>'
assert user.c_directives_use_system_libc()
}
+6 -9
View File
@@ -17,9 +17,9 @@ fn escaped_c_field_test_node(mut a flat.FlatAst, kind flat.NodeKind, value strin
fn test_escaped_c_field_names_follow_the_resolved_owner() {
g := FlatGen.new()
c_struct := types.Type(types.Struct{ name_id: flat.source_type_id('C.EscapedFields') })
alias := types.Type(types.Alias{ name_id: flat.source_type_id('Fields'), base_type: c_struct })
pointer := types.Type(types.Pointer{ base_type: alias })
pointer_alias := types.Type(types.Alias{ name_id: flat.source_type_id('FieldsPtr'), base_type: pointer })
alias := types.Type(types.Alias{ name_id: flat.source_type_id('Fields'), base_type: &types.Type(c_struct) })
pointer := types.Type(types.Pointer{ base_type: &types.Type(alias) })
pointer_alias := types.Type(types.Alias{ name_id: flat.source_type_id('FieldsPtr'), base_type: &types.Type(pointer) })
for owner in [c_struct, alias, pointer, pointer_alias] {
assert g.field_c_name(owner, '@type') == 'type'
assert g.field_c_name(owner, '@module') == 'module'
@@ -35,8 +35,8 @@ fn test_escaped_c_field_names_follow_the_resolved_owner() {
fn test_escaped_v_field_names_keep_their_existing_spelling() {
g := FlatGen.new()
v_struct := types.Type(types.Struct{ name_id: flat.source_type_id('EscapedFields') })
alias := types.Type(types.Alias{ name_id: flat.source_type_id('Fields'), base_type: v_struct })
pointer := types.Type(types.Pointer{ base_type: alias })
alias := types.Type(types.Alias{ name_id: flat.source_type_id('Fields'), base_type: &types.Type(v_struct) })
pointer := types.Type(types.Pointer{ base_type: &types.Type(alias) })
for owner in [v_struct, alias, pointer] {
for field in ['@type', '@struct', '@select', '_v_type', 'type', 'index', 'select', 'malloc',
'exit', 'byte', 'int_str', 'complex'] {
@@ -94,10 +94,7 @@ fn test_escaped_c_callback_selector_assignment_keeps_c_abi_adapter() {
for owner_name in ['C.CallbackHolder', 'CallbackAlias', 'CallbackPointerAlias'] {
mut a := flat.FlatAst.new()
mut tc := types.TypeChecker.new(&a)
callback_type := types.Type(types.FnType{
params: [types.Type(types.int_)]
return_type: types.Type(types.int_)
})
callback_type := types.Type(tc.fn_type([types.Type(types.int_)], &types.Type(types.int_), []bool{}))
tc.cur_module = 'main'
tc.structs['C.CallbackHolder'] = [types.StructField{ name: 'type', typ: callback_type, is_mut: true }]
tc.type_aliases.set_id('CallbackAlias', flat.source_type_id('C.CallbackHolder'))
+286 -218
View File
File diff suppressed because it is too large. Load diff
+3 -6
View File
@@ -14,16 +14,13 @@ fn fn_field_call_test_output(pointer_storage bool, pointer_type bool, pointer_fi
value_type := types.Type(types.Struct{ name_id: flat.source_type_id('FnFieldHolder') })
base_type := if pointer_type {
types.Type(types.Pointer{ base_type: value_type })
types.Type(types.Pointer{ base_type: &types.Type(value_type) })
} else {
value_type
}
callback_type := types.Type(types.FnType{
params: [types.Type(types.int_)]
return_type: types.Type(types.int_)
})
callback_type := types.Type(tc.fn_type([types.Type(types.int_)], &types.Type(types.int_), []bool{}))
field_type := if pointer_field {
types.Type(types.Pointer{ base_type: callback_type })
types.Type(types.Pointer{ base_type: &types.Type(callback_type) })
} else {
callback_type
}
+21 -30
View File
@@ -1399,12 +1399,7 @@ fn (mut g FlatGen) replay_prep_candidates(args []FlatCgenCostArgs) {
g.tc.cur_module = item.module
}
if cand.is_expr {
w0, w1, slot := preseed_type_words(cand.typ)
if !replay_seen.seen[slot] || replay_seen.w0[slot] != w0
|| replay_seen.w1[slot] != w1 {
replay_seen.w0[slot] = w0
replay_seen.w1[slot] = w1
replay_seen.seen[slot] = true
if replay_seen.insert(cand.typ) {
g.preseed_parallel_fn_ptr_type(cand.typ)
}
} else {
@@ -2062,7 +2057,7 @@ fn clone_cgen_fixed_array_typedef_map(values map[string]FixedArrayTypedefInfo) m
for key, info in values {
cloned[key.clone()] = FixedArrayTypedefInfo{
arr: types.ArrayFixed{
elem_type: types.clone_owned_type(info.arr.elem_type)
elem_type: types.new_type_reference(types.Type(types.clone_owned_type(info.arr.elem_type)))
len: info.arr.len
len_expr: info.arr.len_expr.clone()
}
@@ -2784,7 +2779,7 @@ fn (g &FlatGen) parallel_cached_expr_type(id flat.NodeId, node &flat.Node) ?type
return none
}
if idx < g.tc.expr_type_set.len && idx < g.tc.expr_type_values.len && g.tc.expr_type_set[idx] {
return g.tc.expr_type_values[idx]
return *g.tc.expr_type_values[idx]
}
if node.kind == .call && idx < g.tc.resolved_call_set.len && idx < g.tc.resolved_call_names.len
&& g.tc.resolved_call_set[idx] {
@@ -2808,11 +2803,12 @@ struct FlatCgenPrepCandidate {
struct ResolvedCallTypeCache {
mut:
ptrs [4096]voidptr
lens [4096]int
values [4096]types.Type
seen [4096]bool
found [4096]bool
ptrs [4096]voidptr
lens [4096]int
indexes [4096]u32
values []types.Type
seen [4096]bool
found [4096]bool
}
// exact_flat_fn_gen_item_cost_and_prep is exact_flat_fn_gen_item_cost plus the
@@ -2864,11 +2860,7 @@ fn exact_flat_fn_gen_item_cost_and_prep(g &FlatGen, node_id flat.NodeId, item_id
}
}
if expr_type := g.parallel_cached_expr_type_with_cache(id, node, mut resolved_call_cache) {
w0, w1, slot := preseed_type_words(expr_type)
if !type_seen.seen[slot] || type_seen.w0[slot] != w0 || type_seen.w1[slot] != w1 {
type_seen.w0[slot] = w0
type_seen.w1[slot] = w1
type_seen.seen[slot] = true
if type_seen.insert(expr_type) {
cands << FlatCgenPrepCandidate{
is_expr: true
typ: expr_type
@@ -2896,7 +2888,7 @@ fn (g &FlatGen) parallel_cached_expr_type_with_cache(id flat.NodeId, node &flat.
return g.parallel_cached_expr_type(id, node)
}
if idx < g.tc.expr_type_set.len && idx < g.tc.expr_type_values.len && g.tc.expr_type_set[idx] {
return g.tc.expr_type_values[idx]
return *g.tc.expr_type_values[idx]
}
if node.kind != .call || idx >= g.tc.resolved_call_set.len
|| idx >= g.tc.resolved_call_names.len || !g.tc.resolved_call_set[idx] {
@@ -2906,7 +2898,7 @@ fn (g &FlatGen) parallel_cached_expr_type_with_cache(id flat.NodeId, node &flat.
slot := int((u64(voidptr(name.str)) >> 4 ^ u64(name.len)) & 4095)
if cache.seen[slot] && cache.ptrs[slot] == voidptr(name.str) && cache.lens[slot] == name.len {
if cache.found[slot] {
return cache.values[slot]
return cache.values[int(cache.indexes[slot] - 1)]
}
return none
}
@@ -2914,7 +2906,13 @@ fn (g &FlatGen) parallel_cached_expr_type_with_cache(id flat.NodeId, node &flat.
cache.lens[slot] = name.len
cache.seen[slot] = true
if typ := g.tc.fn_ret_types[name] {
cache.values[slot] = typ
index := cache.indexes[slot]
if index == 0 {
cache.values << typ
cache.indexes[slot] = u32(cache.values.len)
} else {
cache.values[int(index - 1)] = typ
}
cache.found[slot] = true
return typ
}
@@ -2983,14 +2981,6 @@ fn parallel_type_text_may_preseed(g &FlatGen, typ_id u32) bool {
return short in g.prep_alias_short_names
}
@[inline]
fn preseed_type_words(typ &types.Type) (u64, u64, int) {
words := unsafe { &u64(voidptr(typ)) }
w0 := unsafe { words[0] }
w1 := unsafe { words[1] }
return w0, w1, int((w0 >> 4 ^ w1) & 4095)
}
// par_cgen_prep_enabled gates the parallel fused item prep, so a single binary
// can A/B or disable it (`V3_NO_PAR_CGEN_PREP=1`).
fn par_cgen_prep_enabled() bool {
@@ -3065,7 +3055,8 @@ fn (g &FlatGen) parallel_base_type_text(typ_id u32) u32 {
fn (mut g FlatGen) preseed_parallel_fn_ptr_type(typ types.Type) {
if typ is types.FnType {
g.register_fn_ptr_type(g.fn_ptr_type_key(typ))
for param in typ.params {
for parameter in typ.params {
param := parameter.typ
g.preseed_parallel_fn_ptr_type(param)
}
g.preseed_parallel_fn_ptr_type(typ.return_type)
+10 -10
View File
@@ -1,7 +1,7 @@
module c
import v.flat
import v.types
import v.types { unalias_type }
struct LoopBodyNodeResult {
emitted_continue_label bool
@@ -297,9 +297,9 @@ fn (mut g FlatGen) gen_for_in(node flat.Node) {
c_val := g.value_c_type(clean_container_type.value_type)
container_str := g.expr_to_string(g.a.child(&node, 2))
storage_container_type := g.usable_expr_type(g.a.child(&node, 2))
container_storage_is_pointer := cgen_unalias_type(storage_container_type) is types.Pointer
container_storage_is_pointer := unalias_type(storage_container_type) is types.Pointer
container_is_mutable_value_storage := g.for_in_mutable_value_storage(container_id)
mut clean_value_type := clean_container_type.value_type
mut clean_value_type := *clean_container_type.value_type
for clean_value_type is types.Alias {
clean_value_type = clean_value_type.base_type
}
@@ -391,7 +391,7 @@ fn (mut g FlatGen) gen_for_in(node flat.Node) {
base_type: clean_container_type.value_type
})
} else {
clean_container_type.value_type
*clean_container_type.value_type
}
val_owner := g.tc.cur_scope.insert_with_owner(elem_binding_name, val_scope_type)
g.track_shadowed_global_local(elem_binding_name, val_owner)
@@ -466,12 +466,12 @@ fn (mut g FlatGen) gen_for_in(node flat.Node) {
base_type: container_type.elem_type
})
} else {
container_type.elem_type
*container_type.elem_type
}
elem_owner := g.tc.cur_scope.insert_with_owner(elem_binding_name, elem_scope_type)
g.track_shadowed_global_local(elem_binding_name, elem_owner)
g.declare_local_mutability(elem_owner, node.op == .amp
&& (cgen_unalias_type(g.usable_expr_type(container_id)) !is types.Pointer
&& (unalias_type(g.usable_expr_type(container_id)) !is types.Pointer
|| container_is_mutable_value_storage))
if node.op == .amp {
g.declare_local_indirect_value_type(elem_owner, container_type.elem_type)
@@ -506,12 +506,12 @@ fn (mut g FlatGen) gen_for_in(node flat.Node) {
base_type: af.elem_type
})
} else {
af.elem_type
*af.elem_type
}
elem_owner := g.tc.cur_scope.insert_with_owner(elem_binding_name, elem_scope_type)
g.track_shadowed_global_local(elem_binding_name, elem_owner)
g.declare_local_mutability(elem_owner, node.op == .amp
&& (cgen_unalias_type(g.usable_expr_type(container_id)) !is types.Pointer
&& (unalias_type(g.usable_expr_type(container_id)) !is types.Pointer
|| container_is_mutable_value_storage))
if node.op == .amp {
g.declare_local_indirect_value_type(elem_owner, af.elem_type)
@@ -614,7 +614,7 @@ fn (g &FlatGen) for_in_container_type(node flat.Node, container_id flat.NodeId)
return typ
}
}
return g.usable_expr_type(container_id)
return *g.usable_expr_type(container_id)
}
fn (g &FlatGen) for_in_array_literal_element_needs_ierror_copy(container flat.Node) bool {
@@ -645,7 +645,7 @@ fn (mut g FlatGen) gen_range_for_in(node flat.Node, key_id flat.NodeId, low_id f
|| low_type is types.USize {
low_type
} else {
types.Type(types.int_)
g.tc.parse_type(flat.source_type_id('int'))
}
ct := g.value_c_type(range_type)
low_name := '__range_low_${g.tmp_count}'
+30
View File
@@ -0,0 +1,30 @@
module c
import os
fn test_literal_iteration_uses_fixed_storage() {
path := os.join_path(os.vtmp_dir(), 'literal_loop_${os.getpid()}.v')
output := path + '.c'
defer {
os.rm(path) or {}
os.rm(output) or {}
}
source := '@[noinline]
fn sum_pair(first int, second int) int {
mut sum := 0
for value in [first, second] { sum += value }
return sum
}
fn main() { assert sum_pair(12, 30) == 42 }'
os.write_file(path, source)!
command := '${os.quoted_path(@VEXE)} -new-compiler -o ${os.quoted_path(output)} ${os.quoted_path(path)}'
result := os.execute(command)
assert result.exit_code == 0, result.output
generated := os.read_file(output)!
body := generated.all_after('__attribute__((noinline)) i64 sum_pair(')
.all_before('\n}')
assert body.contains('first') && body.contains('second')
assert !body.contains('new_array')
assert !body.contains('array_get')
assert !body.contains('malloc')
}
+9 -12
View File
@@ -1,5 +1,6 @@
module c
import v.gen.c.naming { sum_field_name }
import v.flat
import v.types
@@ -193,15 +194,11 @@ fn (mut g FlatGen) smartcast_is_expr(cond &flat.Node) {
return
}
variant_ct := g.tc.c_type(variant_type)
field_name := g.sum_field_name(variant_name)
is_ptr_variant := g.variant_references_sum(flat.source_type_id(variant_name), clean_sum.source_type_id())
field_name := sum_field_name(variant_name)
var_name := g.local_cname(expr_node.value)
tmp := g.tmp_name()
if is_ptr_variant {
g.writeln('${variant_ct} ${tmp} = *${var_name}.${field_name};')
} else {
g.writeln('${variant_ct} ${tmp} = ${var_name}.${field_name};')
}
payload := g.sum_pointer_payload_expr(variant_type, '${var_name}.${field_name}')
g.writeln('${variant_ct} ${tmp} = ${payload};')
g.writeln('${variant_ct} ${var_name} = ${tmp};')
g.tc.cur_scope.insert(expr_node.value, variant_type)
}
@@ -745,22 +742,22 @@ fn (mut g FlatGen) if_expr_block_tail_type(block &flat.Node) types.Type {
fn (mut g FlatGen) if_expr_tail_value_type(id flat.NodeId) types.Type {
t := g.usable_expr_type(id)
if int(id) < 0 || int(id) >= g.a.nodes.len {
return t
return *t
}
node := g.a.nodes[int(id)]
if node.kind != .or_expr || node.children_count == 0 {
return t
return *t
}
source_id := g.a.child(&node, 0)
source_node := g.a.nodes[int(source_id)]
source_type := g.optional_source_type_for_expr(source_id, g.or_expr_source_type(source_id, source_node))
if source_type is types.OptionType {
return source_type.base_type
return *source_type.base_type
}
if source_type is types.ResultType {
return source_type.base_type
return *source_type.base_type
}
return t
return *t
}
// if_expr_type supports if expr type handling for FlatGen.
+8 -8
View File
@@ -1,7 +1,7 @@
module c
import v.flat
import v.types
import v.types { unalias_type }
// Preamble and expression lowering for the `u128` / `i128` primitives.
//
@@ -41,8 +41,8 @@ fn (mut g FlatGen) emit_int128_preamble() {
// int128_signedness returns none unless t is one of the 128-bit integer
// primitives. `true` means signed (`i128`), `false` means `u128`.
fn int128_signedness(t types.Type) ?bool {
clean := cgen_unalias_type(t)
fn int128_signedness(t &types.Type) ?bool {
clean := unalias_type(t)
if clean is types.Primitive {
if clean.props.has(.integer) && clean.size == 128 {
return !clean.props.has(.unsigned)
@@ -423,7 +423,7 @@ fn (mut g FlatGen) gen_int128_cast(target_type types.Type, source_id flat.NodeId
// are types of their own in V rather than primitives, so they need naming here
// too, or the cast falls through to a C cast the struct representation rejects.
fn (mut g FlatGen) int128_narrow_c_type(t types.Type) ?string {
clean := cgen_unalias_type(t)
clean := types.Type(*unalias_type(t))
match clean {
types.Primitive {
if clean.props.has(.integer) && clean.size > 0 && clean.size < 128 {
@@ -584,8 +584,8 @@ fn (g &FlatGen) int128_source_is_plain_literal(id flat.NodeId) bool {
// source_signedness_known reports whether the cast source is an integer whose
// signedness can be read, which is what decides between the from_i64 and
// from_u64 widening helpers.
fn source_signedness_known(t types.Type) bool {
clean := cgen_unalias_type(t)
fn source_signedness_known(t &types.Type) bool {
clean := unalias_type(t)
if clean is types.Primitive {
return clean.props.has(.integer)
}
@@ -594,8 +594,8 @@ fn source_signedness_known(t types.Type) bool {
// int128_source_is_signed reports whether an integer source casts with a sign
// extension. `isize` is signed, `usize` and the unsigned primitives are not.
fn int128_source_is_signed(t types.Type) bool {
clean := cgen_unalias_type(t)
fn int128_source_is_signed(t &types.Type) bool {
clean := unalias_type(t)
if clean is types.Primitive {
return !clean.props.has(.unsigned)
}
+101 -319
View File
@@ -1,43 +1,13 @@
module c
import strings
import v.gen.c.naming
import v.types
import v.gen.c.naming { sum_field_name }
import v.types { unalias_type }
import v.flat
const cgen_ierror_type_name_id = flat.source_type_id('IError')
const cgen_builtin_ierror_type_name_id = flat.source_type_id('builtin.IError')
// emit_sum_type emits emit sum type output for c.
fn (mut g FlatGen) emit_sum_type(name string) {
variants := g.tc.sum_types.type_ids(name) or { []u32{} }
g.writeln('struct ${g.cname(name)} {')
g.writeln('\tint typ;')
g.writeln('\tu32 _pointer_variant_is_owned;')
g.writeln('\tunion {')
for v in variants {
variant_type := select_receive_unalias_type(g.tc.parse_canonical_type(v))
ct := if variant_type is types.Pointer {
g.value_c_type(variant_type.base_type)
} else {
g.value_c_type(variant_type)
}
field := g.sum_field_name(flat.source_type_name(v))
g.writeln('\t\t${ct}* ${field};')
}
g.writeln('\t};')
g.writeln('};')
g.writeln('')
}
// sum_type_contains_struct reports whether sum type contains struct applies in c.
fn (g &FlatGen) sum_type_contains_struct(sum_name_id u32, struct_name_id u32) bool {
for variant in g.tc.sum_types.type_ids(flat.source_type_name(sum_name_id)) or { return false } {
if variant == struct_name_id { return true }
}
return false
}
// sum_type_index supports sum type index handling for FlatGen.
fn (g &FlatGen) sum_type_index(sum_name_id u32, variant_id u32) int {
mut resolved_sum := sum_name_id
@@ -95,30 +65,27 @@ fn (g &FlatGen) sum_type_index(sum_name_id u32, variant_id u32) int {
fn (g &FlatGen) sum_type_index_resolved(sum_name_id u32, variant_id u32) int {
variants := g.tc.sum_types.type_ids(flat.source_type_name(sum_name_id)) or { return 0 }
for i, v in variants { if v == variant_id { return i + 1 } }
exact := variants.index(variant_id)
if exact >= 0 {
return exact + 1
}
variant := flat.source_type_name(variant_id)
resolved_variant := flat.source_type_id(g.tc.qualify_name(variant))
if resolved_variant != variant_id {
for i, v in variants { if v == resolved_variant { return i + 1 } }
qualified_id := flat.source_type_id(g.tc.qualify_name(variant))
qualified := variants.index(qualified_id)
if qualified >= 0 {
return qualified + 1
}
for i, v in variants {
if flat.source_type_name(v).all_after_last('.') == variant { return i + 1 }
}
// Container variants written through an import alias (`map[string]ast.Value`
// vs the registered `map[string]toml.ast.Value`) match on the
// module-stripped spelling only when that spelling is unambiguous.
shape := flat.source_type_shape(variant_id)
if shape.has(.dotted) || shape.has(.bracketed) {
mut short_match := 0
for i, v in variants {
if short_module_type_ids_equal(v, variant_id) {
if short_match != 0 { return 0 }
short_match = i + 1
mut found := 0
for i, candidate in variants {
if flat.source_type_name(candidate).all_after_last('.') == variant
|| short_module_type_ids_equal(candidate, variant_id) {
if found != 0 {
return 0
}
found = i + 1
}
return short_match
}
return 0
return found
}
fn (mut g FlatGen) interface_tmp(prefix string) string {
@@ -208,7 +175,7 @@ fn (g &FlatGen) interface_dispatch_receiver_expr(concrete_id u32, concrete_param
}
}
fn (g &FlatGen) interface_arg_conversion_expr(name string, source_type types.Type, target_type types.Type) ?string {
fn (g &FlatGen) interface_arg_conversion_expr(name string, source_type &types.Type, target_type &types.Type) ?string {
if source_type is types.Pointer || target_type is types.Pointer {
return none
}
@@ -345,96 +312,23 @@ fn (g &FlatGen) interface_dispatch_target_short_name_is_unambiguous(short_name s
return matches == 1
}
// variant_references_sum supports variant references sum handling for FlatGen.
fn (g &FlatGen) variant_references_sum(variant_id u32, sum_name_id u32) bool {
_ = variant_id
_ = sum_name_id
return true
}
// variant_refs_sum_inner supports variant refs sum inner handling for FlatGen.
fn (g &FlatGen) variant_refs_sum_inner(variant_id u32, sum_name_id u32, mut visited map[u32]bool) bool {
normalized_id := g.normalize_variant_type_id(variant_id)
normalized := flat.source_type_name(normalized_id)
sum_name := flat.source_type_name(sum_name_id)
if normalized_id == sum_name_id || normalized.all_after_last('.') == sum_name.all_after_last('.') {
return true
}
if normalized_id in visited { return false }
visited[normalized_id] = true
mut lookup := normalized
if lookup !in g.tc.structs && !lookup.contains('.') && sum_name.contains('.') {
qualified := '${sum_name.all_before_last('.')}.${lookup}'
if qualified in g.tc.structs { lookup = qualified }
}
if lookup !in g.tc.structs && !lookup.contains('.') {
for struct_name, _ in g.tc.structs {
if struct_name.all_after_last('.') == lookup {
lookup = struct_name
break
}
}
}
if lookup in g.tc.structs {
for f in g.tc.structs[lookup] {
if g.type_references_sum(f.typ, sum_name_id, mut visited) { return true }
}
}
return false
}
// normalize_variant_name transforms normalize variant name data for c.
fn (g &FlatGen) normalize_variant_name(name string) string {
_ = g
mut res := name
if res.starts_with('&') {
res = res[1..]
}
if res.starts_with('ptr') && res.len > 3 {
res = res[3..]
}
if res.contains('__') && !res.contains('.') {
res = res.replace('__', '.')
}
return res
}
// type_references_sum returns type references sum data for FlatGen.
fn (g &FlatGen) type_references_sum(typ types.Type, sum_name_id u32, mut visited map[u32]bool) bool {
resolved_sum := g.resolve_sum_name(sum_name_id)
clean := types.unwrap_pointer(typ)
if clean is types.Struct && g.resolve_sum_name(clean.source_type_id()) == resolved_sum {
return true
}
if clean is types.SumType && g.resolve_sum_name(clean.source_type_id()) == resolved_sum {
return true
}
if clean is types.SumType { return true }
if clean is types.Struct {
if g.variant_refs_sum_inner(clean.source_type_id(), resolved_sum, mut visited) {
return true
}
}
if clean is types.Array {
return g.type_references_sum(clean.elem_type, resolved_sum, mut visited)
}
return false
}
// resolve_sum_name resolves a canonical or unqualified sum name without
// interpreting its first component as a source-file import alias.
fn (g &FlatGen) resolve_sum_name(sum_name_id u32) u32 {
if resolved := g.sum_name_lookup[sum_name_id] { return resolved }
name := flat.source_type_name(sum_name_id)
if name.len == 0 || name[0] in [`&`, `?`, `!`, `[`] || name.starts_with('map[') {
return sum_name_id
}
if resolved := g.sum_name_lookup[sum_name_id] {
return if resolved != 0 { resolved } else { sum_name_id }
}
if flat.source_type_shape(sum_name_id).has(.dotted) {
sum_name := flat.source_type_name(sum_name_id)
// A qualified name that already names a concrete type is that type. The
// short-name fallback below would otherwise hand `one.Any` the unrelated
// `two.Any` sum type and box the value into the wrong C struct.
if sum_name in g.tc.structs || sum_name in g.tc.interface_names || sum_name in g.tc.enum_names || g.tc.type_aliases.contains(sum_name) {
if name in g.tc.structs || name in g.tc.interface_names || name in g.tc.enum_names
|| g.tc.type_aliases.contains(name) {
return sum_name_id
}
if resolved := g.sum_name_lookup[flat.source_type_id(c_short_name_view(sum_name))] {
return resolved
if resolved := g.sum_name_lookup[flat.source_type_id(c_short_name_view(name))] {
return if resolved != 0 { resolved } else { sum_name_id }
}
}
return sum_name_id
@@ -461,169 +355,61 @@ fn (mut g FlatGen) precompute_sum_name_lookup() {
g.sum_variant_lookup = map[u32]map[u32]u32{}
for name, variants in g.tc.sum_types.ids {
name_id := flat.source_type_id(name)
g.sum_name_lookup[name_id] = name_id
short := c_short_name_view(name)
if short.len > 0 {
if short.len > 0 && !g.tc.sum_types.contains(short) {
short_id := flat.source_type_id(short)
if short_id !in g.sum_name_lookup { g.sum_name_lookup[short_id] = name_id }
g.sum_name_lookup[short_id] = if short_id in g.sum_name_lookup {
u32(0)
} else {
name_id
}
}
mut lookup := map[u32]u32{}
for variant in variants { lookup[variant] = variant }
for variant in variants {
short_variant := c_short_name_view(flat.source_type_name(variant))
if short_variant.len > 0 {
short_id := flat.source_type_id(short_variant)
if short_id !in lookup { lookup[short_id] = variant }
lookup[variant] = variant
}
for variant in variants {
variant_short := c_short_name_view(flat.source_type_name(variant))
if variant_short.len == 0 {
continue
}
short_id := flat.source_type_id(variant_short)
if short_id !in variants {
lookup[short_id] = if short_id in lookup { u32(0) } else { variant }
}
}
g.sum_variant_lookup[name_id] = lookup.move()
}
// Exact canonical and emitted C spellings always supersede short aliases.
for name, _ in g.tc.sum_types.ids {
name_id := flat.source_type_id(name)
g.sum_name_lookup[flat.source_type_id(g.cname(name))] = name_id
g.sum_name_lookup[name_id] = name_id
}
}
// resolve_variant resolves resolve variant information for c.
fn (g &FlatGen) resolve_variant(sum_name_id u32, variant_id u32) u32 {
resolved_sum := g.resolve_sum_name(sum_name_id)
normalized := g.normalize_variant_type_id(variant_id)
if lookup := g.sum_variant_lookup[resolved_sum] {
if resolved := lookup[normalized] { return resolved }
}
return normalized
}
// normalize_variant_type_id renders encoded variants only at the spelling boundary.
fn (g &FlatGen) normalize_variant_type_id(variant_id u32) u32 {
name := flat.source_type_name(variant_id)
normalized := g.normalize_variant_name(name)
return if normalized == name { variant_id } else { flat.source_type_id(normalized) }
}
// sum_field_name supports sum field name handling for FlatGen.
fn (g &FlatGen) sum_field_name(variant string) string {
if variant.starts_with('&') {
return g.sum_field_name(variant[1..])
}
if variant.starts_with('?') {
return '_Option_${g.cname(variant[1..])}'
}
if variant.starts_with('!') {
return '_Result_${g.cname(variant[1..])}'
}
if variant.starts_with('ptr') && variant.len > 3 && variant[3..].contains('.') {
return g.sum_field_name(variant[3..])
}
if variant.starts_with('ptr') && variant.len > 3 && variant[3..].contains('__') {
return g.sum_field_name(variant[3..].replace('__', '.'))
}
if variant.starts_with('[]') {
return '_Array_${g.cname(variant[2..])}'
}
if variant.starts_with('map[') {
return '_Map_${g.cname(variant[4..].replace(']', '_'))}'
}
if variant.starts_with('fn(') || variant.starts_with('fn (') {
return '_Fn_${callback_stable_key_hash(sum_fn_variant_key(variant))}'
}
if sum_variant_needs_type_name_field(variant) {
return '_${naming.type_name_part(variant)}'
}
return match variant {
'int' { '_int' }
'i8' { '_i8' }
'i16' { '_i16' }
'i64' { '_i64' }
'u8' { '_u8' }
'u16' { '_u16' }
'u32' { '_u32' }
'u64' { '_u64' }
'f32' { '_f32' }
'f64' { '_f64' }
'bool' { '_bool' }
'string' { '_string' }
else { g.cname(variant) }
}
}
fn sum_variant_needs_type_name_field(variant string) bool {
return variant.contains('(') || variant.contains(')') || variant.contains(' ')
}
fn sum_fn_variant_key(variant string) string {
clean := variant.trim_space()
open := clean.index('(') or { return clean.replace(' ', '') }
close := clean.last_index(')') or { return clean.replace(' ', '') }
params := clean[open + 1..close]
ret := clean[close + 1..].trim_space().replace(' ', '')
mut parts := []string{}
for part in sum_fn_split_top_level_commas(params) {
ptyp := sum_fn_param_type(part)
if ptyp.len > 0 {
parts << ptyp
if resolved := lookup[variant_id] {
return if resolved != 0 { resolved } else { variant_id }
}
}
return 'fn(${parts.join(',')})${ret}'
}
fn sum_fn_split_top_level_commas(params string) []string {
mut parts := []string{}
mut depth := 0
mut start := 0
for i := 0; i < params.len; i++ {
ch := params[i]
if ch == `(` || ch == `[` || ch == `{` {
depth++
} else if ch == `)` || ch == `]` || ch == `}` {
if depth > 0 {
depth--
variant := flat.source_type_name(variant_id)
if variant.contains('__') && !variant.contains('.') {
normalized := flat.source_type_id(variant.replace('__', '.'))
if resolved := lookup[normalized] {
return if resolved != 0 { resolved } else { normalized }
}
} else if ch == `,` && depth == 0 {
parts << params[start..i].trim_space()
start = i + 1
return normalized
}
}
parts << params[start..].trim_space()
return parts
return variant_id
}
fn sum_fn_param_type(param string) string {
clean := param.trim_space()
if clean.len == 0 {
return ''
}
if clean.starts_with('fn(') || clean.starts_with('fn (') {
return sum_fn_variant_key(clean)
}
space := clean.index(' ') or { return clean }
first := clean[..space]
if sum_fn_is_ident(first) && first !in ['fn', 'mut', 'shared'] {
return clean[space + 1..].trim_space().replace(' ', '')
}
if first in ['mut', 'shared'] {
rest := clean[space + 1..].trim_space()
second_space := rest.index(' ') or { return clean.replace(' ', '') }
second := rest[..second_space]
if sum_fn_is_ident(second) {
return '${first}${rest[second_space + 1..].trim_space().replace(' ', '')}'
}
}
return clean.replace(' ', '')
}
fn sum_fn_is_ident(s string) bool {
if s.len == 0 {
return false
}
first := s[0]
if !((first >= `a` && first <= `z`) || (first >= `A` && first <= `Z`) || first == `_`) {
return false
}
for i := 1; i < s.len; i++ {
ch := s[i]
if !((ch >= `a` && ch <= `z`) || (ch >= `A` && ch <= `Z`)
|| (ch >= `0` && ch <= `9`) || ch == `_`) {
return false
}
}
return true
// sum_field_name uses the shared C/SSA naming for the exact variant spelling.
fn (g &FlatGen) sum_field_name(variant string) string {
return sum_field_name(variant)
}
// register_interface_strings updates register interface strings state for c.
@@ -1087,7 +873,7 @@ fn (g &FlatGen) ierror_method_signature_matches(name string, concrete_id u32, me
return g.type_names_match(receiver, expected)
}
fn (g &FlatGen) ierror_method_return_matches(method string, ret types.Type) bool {
fn (g &FlatGen) ierror_method_return_matches(method string, ret &types.Type) bool {
clean := if ret is types.Alias { ret.base_type } else { ret }
return match method {
'msg' { clean is types.String }
@@ -1098,7 +884,7 @@ fn (g &FlatGen) ierror_method_return_matches(method string, ret types.Type) bool
fn (g &FlatGen) ierror_clean_type(typ types.Type) types.Type {
clean0 := types.unwrap_pointer(typ)
return if clean0 is types.Alias { clean0.base_type } else { clean0 }
return if clean0 is types.Alias { *clean0.base_type } else { clean0 }
}
struct IErrorMethodCall {
@@ -1167,7 +953,7 @@ fn (g &FlatGen) type_can_box_as_ierror(concrete_id u32) bool {
return g.tc.named_type_compatible_with_ierror(concrete_id)
}
fn (g &FlatGen) ierror_concrete_name(t types.Type) ?u32 {
fn (g &FlatGen) ierror_concrete_name(t &types.Type) ?u32 {
clean := g.ierror_payload_concrete_type(t)
if clean !is types.Struct {
return none
@@ -1234,7 +1020,7 @@ fn (mut g FlatGen) ierror_none_literal_string() string {
fn (mut g FlatGen) ierror_from_expr_string(id flat.NodeId) ?string {
node := g.a.nodes[int(id)]
mut actual := g.usable_expr_type(id)
mut actual := *g.usable_expr_type(id)
if node.kind == .struct_init && node.value.len > 0 {
// A concrete error returned from a result function can carry the surrounding
// result type as its node annotation. The literal name still identifies the
@@ -1458,7 +1244,7 @@ fn (g &FlatGen) iface_type_id_for_concrete(iface_id u32, concrete types.Type) in
}
fn (mut g FlatGen) gen_interface_value_expr(id flat.NodeId, expected types.Type) bool {
iface_type := cgen_unalias_type(expected)
iface_type := unalias_type(expected)
if iface_type !is types.Interface {
return false
}
@@ -1467,7 +1253,7 @@ fn (mut g FlatGen) gen_interface_value_expr(id flat.NodeId, expected types.Type)
// Initializing the bitfields alone leaves the upper bytes of the pointer-sized
// metadata union unspecified. Keep the nil expression's block effects while
// clearing the entire metadata word.
ct := g.tc.c_type(iface)
ct := g.tc.c_type(iface_type)
g.write('(${ct}){._object = ')
g.gen_expr(id)
g.write(', ._interface_meta = NULL}')
@@ -1486,14 +1272,14 @@ fn (mut g FlatGen) gen_interface_value_expr(id flat.NodeId, expected types.Type)
// A `mut p &T` parameter uses `&&T` storage, but reading `p` yields
// the semantic `&T` value that is being boxed into the interface.
actual = if g.current_param_is_mut_pointer(node.value) && param_type is types.Pointer {
param_type.base_type
*param_type.base_type
} else {
param_type
*param_type
}
}
}
actual_clean := if actual is types.Pointer { actual.base_type } else { actual }
actual_base := cgen_unalias_type(actual_clean)
actual_clean := if actual is types.Pointer { *actual.base_type } else { actual }
actual_base := unalias_type(actual_clean)
actual_name := actual_base.name()
if actual_base is types.Interface || actual_name == iface.name()
|| (actual_name.starts_with('main.') && actual_name['main.'.len..] == iface.name())
@@ -1512,7 +1298,7 @@ fn (mut g FlatGen) gen_interface_value_expr(id flat.NodeId, expected types.Type)
return false
}
type_id := g.iface_type_id_for_concrete(iface.source_type_id(), actual_clean)
ct := g.tc.c_type(iface)
ct := g.tc.c_type(types.Type(iface))
fields := g.interface_cached_fields(iface.source_type_id())
concrete_ct := g.tc.c_type(actual_base)
if concrete_ct == ct {
@@ -1588,9 +1374,9 @@ fn (mut g FlatGen) gen_interface_pointer_value_expr(id flat.NodeId, expected typ
} else {
return false
}
mut iface_type := cgen_unalias_type(ptr_type.base_type)
mut iface_type := unalias_type(ptr_type.base_type)
if iface_type is types.Alias {
iface_type = cgen_unalias_type(iface_type.base_type)
iface_type = unalias_type(iface_type.base_type)
}
if iface_type !is types.Interface {
return false
@@ -1601,14 +1387,14 @@ fn (mut g FlatGen) gen_interface_pointer_value_expr(id flat.NodeId, expected typ
g.gen_expr(id)
return true
}
actual := cgen_unalias_type(g.interface_source_type(id))
actual := unalias_type(g.interface_source_type(id))
if actual is types.Nil || actual is types.Void
|| (actual is types.Pointer && cgen_unalias_type(actual.base_type) is types.Void) {
|| (actual is types.Pointer && unalias_type(actual.base_type) is types.Void) {
g.write('(${ct}*)')
g.gen_expr(id)
return true
}
if actual is types.Pointer && cgen_unalias_type(actual.base_type) is types.Interface {
if actual is types.Pointer && unalias_type(actual.base_type) is types.Interface {
return false
}
iface_value := g.interface_value_to_string(id, iface_type)
@@ -1628,7 +1414,7 @@ fn (mut g FlatGen) interface_source_type(id flat.NodeId) types.Type {
return local_type
}
if param_type := g.current_param_type(node.value) {
return param_type
return *param_type
}
}
if node.kind == .ident && g.current_param_type(node.value) == none
@@ -1644,7 +1430,7 @@ fn (mut g FlatGen) interface_source_type(id flat.NodeId) types.Type {
}
}
}
return g.usable_expr_type(id)
return *g.usable_expr_type(id)
}
fn (g &FlatGen) interface_unknown_qualified_name_matches(actual_name_id u32, iface_name_id u32) bool {
@@ -2313,18 +2099,18 @@ fn (mut g FlatGen) gen_interface_dispatch_wrapped_return(call string, expected t
}
expected_base := interface_dispatch_wrapped_base_type(expected) or { return false }
actual_base := interface_dispatch_wrapped_base_type(actual) or { return false }
expected_iface_type := cgen_unalias_type(expected_base)
expected_iface_type := unalias_type(expected_base)
if expected_iface_type !is types.Interface {
return false
}
expected_iface := expected_iface_type as types.Interface
actual_clean := cgen_unalias_type(actual_base)
actual_clean := unalias_type(actual_base)
actual_value := if actual_clean is types.Pointer {
actual_clean.base_type
} else {
actual_clean
}
if cgen_unalias_type(actual_value) is types.Interface {
if unalias_type(actual_value) is types.Interface {
return false
}
type_id := g.iface_type_id_for_concrete(expected_iface.source_type_id(), actual_value)
@@ -2363,21 +2149,21 @@ fn (mut g FlatGen) gen_interface_dispatch_wrapped_return(call string, expected t
return true
}
fn (g &FlatGen) interface_dispatch_wrapped_return_can_adapt(expected types.Type, actual types.Type) bool {
fn (g &FlatGen) interface_dispatch_wrapped_return_can_adapt(expected &types.Type, actual &types.Type) bool {
expected_base := interface_dispatch_wrapped_base_type(expected) or { return false }
actual_base := interface_dispatch_wrapped_base_type(actual) or { return false }
expected_iface_type := cgen_unalias_type(expected_base)
expected_iface_type := unalias_type(expected_base)
if expected_iface_type !is types.Interface {
return false
}
expected_iface := expected_iface_type as types.Interface
actual_clean := cgen_unalias_type(actual_base)
actual_clean := unalias_type(actual_base)
actual_value := if actual_clean is types.Pointer {
actual_clean.base_type
} else {
actual_clean
}
if cgen_unalias_type(actual_value) is types.Interface {
if unalias_type(actual_value) is types.Interface {
return false
}
return g.iface_type_id_for_concrete(expected_iface.source_type_id(), actual_value) != 0
@@ -2386,7 +2172,7 @@ fn (g &FlatGen) interface_dispatch_wrapped_return_can_adapt(expected types.Type,
fn interface_dispatch_wrapped_base_type(typ types.Type) ?types.Type {
match typ {
types.OptionType, types.ResultType {
return typ.base_type
return *typ.base_type
}
else {
return none
@@ -2401,7 +2187,7 @@ fn (mut g FlatGen) interface_dispatch_param_c_type(typ types.Type) string {
mut ct := if typ is types.OptionType || typ is types.ResultType {
g.optional_type_name(typ)
} else {
g.tc.c_type(typ)
g.parameter_c_type(typ)
}
if ct.starts_with('fn_ptr:') {
ct = g.resolve_fn_ptr_type(ct)
@@ -2465,7 +2251,7 @@ fn (mut g FlatGen) collect_interface_boxed_types_for_dispatch() {
continue
}
mut obj_type := if field.typ != 0 {
g.tc.parse_type(field.typ)
*g.tc.parse_type(field.typ)
} else {
g.tc.resolve_type(g.a.child(field, 0))
}
@@ -2574,7 +2360,7 @@ fn (g &FlatGen) interface_concrete_storage_c_type(concrete_id u32) string {
return g.interface_storage_c_type(concrete_type)
}
fn (g &FlatGen) interface_storage_c_type(typ types.Type) string {
fn (g &FlatGen) interface_storage_c_type(typ &types.Type) string {
ct := g.tc.c_type(typ)
if ct.starts_with('fn_ptr:') {
return naming.fn_ptr_type_name(ct)
@@ -2588,7 +2374,7 @@ fn (g &FlatGen) interface_storage_c_type(typ types.Type) string {
fn (g &FlatGen) interface_concrete_type(concrete_id u32) types.Type {
concrete := flat.source_type_name(concrete_id)
if types.is_builtin_type_name(concrete) {
return g.tc.parse_type(concrete_id)
return *g.tc.parse_type(concrete_id)
}
for candidate_id in [concrete_id, flat.source_type_id(g.tc.qualify_name(concrete))] {
candidate := flat.source_type_name(candidate_id)
@@ -2619,7 +2405,7 @@ fn (g &FlatGen) interface_concrete_type(concrete_id u32) types.Type {
})
}
}
return g.tc.parse_type(concrete_id)
return *g.tc.parse_type(concrete_id)
}
// short_module_type_ids_equal compares the module-stripped spellings without
@@ -2857,7 +2643,7 @@ fn (mut g FlatGen) interface_custom_str_expr(type_name_id u32, typ types.Type, e
fn (mut g FlatGen) interface_pointer_str_expr(base_type types.Type, expr string, prefix_pointer bool, mut stack []InterfaceStringifyVisit) ?string {
ptr_type := types.Type(types.Pointer{
base_type: base_type
base_type: types.new_type_reference(types.Type(base_type))
})
ptr_ct := g.value_c_type(ptr_type)
tmp := g.interface_tmp('iface_str_ptr')
@@ -3021,7 +2807,7 @@ fn (mut g FlatGen) interface_sum_str_expr(sum_type types.SumType, expr string, m
stack.delete_last()
}
variants := g.tc.sum_types.type_ids(sum_name) or { return none }
ct := g.tc.c_type(sum_type)
ct := g.tc.c_type(types.Type(sum_type))
tmp := g.interface_tmp('iface_str_sum')
out := g.interface_tmp('iface_str_out')
display_name := sum_name.all_after_last('.')
@@ -3031,18 +2817,14 @@ fn (mut g FlatGen) interface_sum_str_expr(sum_type types.SumType, expr string, m
resolved := g.resolve_variant(sum_name_id, variant)
variant_type := g.tc.parse_type(resolved)
idx := g.sum_type_index(sum_name_id, resolved)
field := g.sum_field_name(flat.source_type_name(resolved))
value_expr := if variant_type is types.Pointer {
'${tmp}.${field}'
} else {
'*${tmp}.${field}'
}
field := sum_field_name(flat.source_type_name(resolved))
value_expr := g.sum_pointer_payload_expr(variant_type, '${tmp}.${field}')
inner := g.interface_implicit_str_expr(variant_type, value_expr, false, mut stack) or {
g.interface_str_lit('<value>')
}
wrapped := g.interface_str_plus(g.interface_str_plus(g.interface_str_lit('${display_name}('),
inner), g.interface_str_lit(')'))
body += ' case ${idx}: if (${tmp}.${field} != 0) ${out} = ${wrapped}; break;'
body += ' case ${idx}: ${out} = ${wrapped}; break;'
}
body += ' default: break; } ${out};'
return '({ ${body} })'
@@ -3062,7 +2844,7 @@ fn (mut g FlatGen) interface_dynamic_str_expr(iface types.Interface, expr string
qualified := g.tc.qualify_name(iface_name)
g.iface_impls[qualified] or { return none }
}
ct := g.tc.c_type(iface)
ct := g.tc.c_type(types.Type(iface))
tmp := g.interface_tmp('iface_str_dynamic')
out := g.interface_tmp('iface_str_out')
fallback := g.interface_str_lit('${iface_name.all_after_last('.')}{}')
+1 -1
View File
@@ -113,7 +113,7 @@ fn (mut g FlatGen) gen_race_blank_read(rhs_id flat.NodeId) bool {
return false
}
rhs := g.a.nodes[int(rhs_id)]
mut typ := g.usable_expr_type(rhs_id)
mut typ := *g.usable_expr_type(rhs_id)
// The value of a `mut` parameter, also in parentheses, is behind the pointer that C passes.
mut inner := rhs
for inner.kind == .paren && inner.children_count > 0 {
+3 -3
View File
@@ -41,14 +41,14 @@ fn test_map_codegen_callbacks_unwrap_alias_keys() {
for key in [types.Type(types.string_), types.Type(types.int_), types.Type(types.u8_),
types.Type(types.u16_), types.Type(types.u32_), types.Type(types.u64_),
types.Type(types.isize_), types.Type(types.usize_), types.Type(types.voidptr_),
types.Type(types.ArrayFixed{ elem_type: types.Type(types.u64_), len: 2 })] {
types.Type(types.ArrayFixed{ elem_type: &types.Type(types.u64_), len: 2 })] {
alias := types.Type(types.Alias{
name_id: flat.source_type_id('MapKey')
base_type: key
base_type: &types.Type(key)
})
nested := types.Type(types.Alias{
name_id: flat.source_type_id('NestedMapKey')
base_type: alias
base_type: &types.Type(alias)
})
expected := map_codegen_callback_test_names(g, key)
assert map_codegen_callback_test_names(g, alias) == expected, '${bits}: ${key.name()}'
+40 -12
View File
@@ -319,7 +319,7 @@ fn test_main_function_is_prefixed_when_declared_c_function_owns_name() {
g.a = &a
g.tc = &tc
tc.fn_ret_types['C.get_value'] = types.Type(types.int_)
g.fn_decl_ret_types[fn_decl_module_key('main', 'get_value')] = types.Type(types.int_)
g.fn_decl_ret_types[fn_decl_module_key('main', 'get_value')] = tc.parse_type(flat.source_type_id('int'))
assert g.fn_c_name_in_module('main', 'get_value') == 'main__get_value'
assert g.main_runtime_shadow_fn_c_name('main', 'get_value') or { '' } == 'main__get_value'
@@ -472,8 +472,8 @@ fn test_voidptr_method_value_arg_does_not_panic_for_alias_to_voidptr() {
g.tc = &tc
alias_to_voidptr := types.Type(types.Alias{
name_id: flat.source_type_id('Data')
base_type: types.Type(types.Pointer{
base_type: types.Type(types.void_)
base_type: &types.Type(types.Pointer{
base_type: &types.Type(types.void_)
})
})
assert !g.voidptr_method_value_arg(flat.empty_node, alias_to_voidptr)
@@ -577,9 +577,9 @@ fn test_cgen_typeof_display_canonicalizes_fixed_array_generic_args() {
assert typeof_display_type_name('Pair[int[3], Box[string[2]]]') == 'Pair[[3]int, Box[[2]string]]'
assert typeof_display_type_name('Box[int][3]') == '[3]Box[int]'
fixed_maps := types.Type(types.ArrayFixed{
elem_type: types.Type(types.Map{
key_type: types.Type(types.String{})
value_type: types.Type(types.int_)
elem_type: &types.Type(types.Map{
key_type: &types.Type(types.String{})
value_type: &types.Type(types.int_)
})
len: 3
})
@@ -596,7 +596,7 @@ fn test_fixed_array_typedef_allows_opaque_pointer_elements() {
})
assert g.fixed_array_type_has_unknown_struct(opaque)
assert !g.fixed_array_type_has_unknown_struct(types.Type(types.Pointer{
base_type: opaque
base_type: &types.Type(opaque)
}))
}
@@ -610,6 +610,29 @@ fn test_sum_type_index_rejects_ambiguous_qualified_suffix() {
assert g.sum_type_index(flat.source_type_id('tast.Value'), flat.source_type_id('b.tast.Target')) == 0
}
fn test_sum_name_lookup_preserves_qualified_type_identity() {
mut ast := flat.FlatAst.new()
mut tc := types.TypeChecker.new(&ast)
tc.sum_types.set('left.Value', ['left.Item', 'right.Item'])
tc.sum_types.set('right.Value', ['int', 'string'])
mut g := FlatGen.new()
g.tc = &tc
g.precompute_sum_name_lookup()
assert g.resolve_sum_name(flat.source_type_id('Value')) == flat.source_type_id('Value')
assert g.resolve_sum_name(flat.source_type_id('left.Value')) == flat.source_type_id('left.Value')
assert g.resolve_sum_name(flat.source_type_id('right.Value')) == flat.source_type_id('right.Value')
assert g.resolve_variant(flat.source_type_id('left.Value'), flat.source_type_id('Item')) == flat.source_type_id('Item')
assert g.sum_type_index(flat.source_type_id('left.Value'), flat.source_type_id('Item')) == 0
assert g.sum_type_index(flat.source_type_id('left.Value'), flat.source_type_id('left.Item')) == 1
assert g.sum_type_index(flat.source_type_id('left.Value'), flat.source_type_id('right.Item')) == 2
assert g.resolve_variant(flat.source_type_id('left.Value'), flat.source_type_id('left.Item')) == flat.source_type_id('left.Item')
assert g.resolve_variant(flat.source_type_id('left.Value'), flat.source_type_id('right.Item')) == flat.source_type_id('right.Item')
tc.sum_types.set('extra.left__Value', ['Raw__Name', 'Raw.Name'])
g.precompute_sum_name_lookup()
assert g.resolve_sum_name(flat.source_type_id('left__Value')) == flat.source_type_id('left.Value')
assert g.resolve_variant(flat.source_type_id('extra.left__Value'), flat.source_type_id('Raw__Name')) == flat.source_type_id('Raw__Name')
}
fn test_sum_type_index_emission_override_is_limited_to_flatgen() {
mut a := flat.FlatAst.new()
mut tc := types.TypeChecker.new(&a)
@@ -630,17 +653,22 @@ fn test_sum_type_index_emission_override_is_limited_to_flatgen() {
}, -1, 'transform', 'sum.v', 'transform__Transformer__sum_type_index', false)
}
fn test_typeof_type_index_fallback_uses_matching_sum_variant() {
fn test_typeof_type_index_is_independent_of_sum_variant_order() {
mut a := flat.FlatAst.new()
mut tc := types.TypeChecker.new(&a)
tc.cur_module = 'main'
tc.sum_types.set('main.Value', ['main.Foo', 'main.Bar'])
tc.sum_types.set('main.Other', ['main.Baz', 'main.Qux'])
names := ['main.Foo', 'main.Bar', 'main.Baz', 'main.Qux']
tc.runtime_type_indexes = types.stable_type_indexes(names.map(flat.source_type_id(it)))
mut g := FlatGen.new()
g.tc = &tc
assert g.type_index_for_type_name(flat.source_type_id('Foo')) == 1
assert g.type_index_for_type_name(flat.source_type_id('Bar')) == 2
assert g.type_index_for_type_name(flat.source_type_id('Qux')) == 2
assert g.type_index_for_type_name(flat.source_type_id('NotVariant')) == 0
for name in names {
short := name.all_after_last('.')
assert g.type_index_for_type_name(flat.source_type_id(short)) == tc.runtime_type_indexes[flat.source_type_id(name)]
}
assert g.type_index_for_type_name(flat.source_type_id('int')) == 8
assert g.type_index_for_type_name(flat.source_type_id('&int')) == 8 | (1 << 16)
}
fn test_fn_decl_variadic_resolves_alias_before_short_fallback() {
+124
View File
@@ -0,0 +1,124 @@
module naming
pub fn sum_field_name(variant string) string {
if variant.starts_with('&') {
return '_ptr_${sum_field_name(variant[1..])}'
}
if variant.starts_with('?') {
return '_Option_${c_name(variant[1..])}'
}
if variant.starts_with('!') {
return '_Result_${c_name(variant[1..])}'
}
if variant.starts_with('[]') {
return '_Array_${c_name(variant[2..])}'
}
if variant.starts_with('map[') {
return '_Map_${c_name(variant[4..].replace(']', '_'))}'
}
if variant.starts_with('fn(') || variant.starts_with('fn (') {
key := fn_signature_key(variant)
mut hash := u64(1469598103934665603)
for b in key {
hash = (hash ^ u64(b)) * u64(1099511628211)
}
return '_Fn_${hash}'
}
if variant.contains('(') || variant.contains(')') || variant.contains(' ') {
return '_${type_name_part(variant)}'
}
return match variant {
'int' { '_int' }
'i8' { '_i8' }
'i16' { '_i16' }
'i64' { '_i64' }
'u8' { '_u8' }
'u16' { '_u16' }
'u32' { '_u32' }
'u64' { '_u64' }
'f32' { '_f32' }
'f64' { '_f64' }
'bool' { '_bool' }
'string' { '_string' }
else { c_name(variant) }
}
}
pub fn fn_signature_key(variant string) string {
clean := variant.trim_space()
open := clean.index('(') or { return clean.replace(' ', '') }
close := clean.last_index(')') or { return clean.replace(' ', '') }
params := clean[open + 1..close]
ret := clean[close + 1..].trim_space().replace(' ', '')
mut parts := []string{}
for part in sum_fn_split_top_level_commas(params) {
ptyp := sum_fn_param_type(part)
if ptyp.len > 0 {
parts << ptyp
}
}
return 'fn(${parts.join(',')})${ret}'
}
fn sum_fn_split_top_level_commas(params string) []string {
mut parts := []string{}
mut depth := 0
mut start := 0
for i := 0; i < params.len; i++ {
ch := params[i]
if ch == `(` || ch == `[` || ch == `{` {
depth++
} else if ch == `)` || ch == `]` || ch == `}` {
if depth > 0 {
depth--
}
} else if ch == `,` && depth == 0 {
parts << params[start..i].trim_space()
start = i + 1
}
}
parts << params[start..].trim_space()
return parts
}
fn sum_fn_param_type(param string) string {
clean := param.trim_space()
if clean.len == 0 {
return ''
}
if clean.starts_with('fn(') || clean.starts_with('fn (') {
return fn_signature_key(clean)
}
space := clean.index(' ') or { return clean }
first := clean[..space]
if sum_fn_is_ident(first) && first !in ['fn', 'mut', 'shared'] {
return clean[space + 1..].trim_space().replace(' ', '')
}
if first in ['mut', 'shared'] {
rest := clean[space + 1..].trim_space()
second_space := rest.index(' ') or { return clean.replace(' ', '') }
second := rest[..second_space]
if sum_fn_is_ident(second) {
return '${first}${rest[second_space + 1..].trim_space().replace(' ', '')}'
}
}
return clean.replace(' ', '')
}
fn sum_fn_is_ident(s string) bool {
if s.len == 0 {
return false
}
first := s[0]
if !((first >= `a` && first <= `z`) || (first >= `A` && first <= `Z`) || first == `_`) {
return false
}
for i := 1; i < s.len; i++ {
ch := s[i]
if !((ch >= `a` && ch <= `z`) || (ch >= `A` && ch <= `Z`)
|| (ch >= `0` && ch <= `9`) || ch == `_`) {
return false
}
}
return true
}
Loaded 100 of 295 files, more files were not shown because too many files have changed in this diff. Show more