d26355a6a5
* wasm,ssa: lower dynamic arrays of primitive elements
Arrays of ints/floats/bools/bytes/chars now compile and run on wasm: literals, []T{} init, index load/store, push, slice and for-in. The driver gate admits them; maps, structs, fixed and nested arrays stay rejected, so an unsupported construct still fails loudly rather than miscompiling.
Four pieces: the gate accepts primitive element types; a single for-in variable binds the element (an .empty placeholder previously bound the index); array literals take their element width from the array type, so the header stride matches every reader; and the allocator supplies realloc plus __new_array_noscan, which the transformer emits for scalar rows.
* wasm: lower structs, options and results; add memcmp
Three driver gates are lifted. Struct literals are admitted when every field has a lowering -- primitives, strings, pointers, or another admitted struct, walked recursively with a depth bound so a self-referential type cannot loop and a pointer to an unlifted struct is still refused. Option and result return types no longer need their own rejection: both are ordinary struct_t values the backend already passes by pointer and returns through the sret slot. Struct == and != are admitted on the same field rule, since the lowering is field-wise comparisons the backend already emits.
memcmp is added as a wasm intrinsic, because string comparison lowers to it and it was an unhandled C extern. It is registered with an i64 return shadowing C's int whose upper bits are outside the ABI result, so the value is widened before it is stored.
* wasm: gate the string-int regression; name the built compiler correctly on Windows
Two independent fixes.
match_and_wasm_diagnostics_test.v passes a path with no suffix to -o and then executes that same string. V appends .exe on Windows, so the exec found no file. The -o path keeps its bare name and a separate helper supplies the suffixed name for is_executable and exec.
The string-int test is gated behind V3_TEST_WASM, which is the variable the end-to-end wasm tests already use, because it is a known failure rather than a passing case: with a string literal in the module, 'ab'.int() returns 540 on wasm instead of 0. Copying strconv.common_parse_uint2 verbatim into main lowers correctly, and string_int_base replicated verbatim is also correct, so the defect is in the builtin to strconv cross-module chain rather than in strconv. A runtime string is handled correctly, which is why the test drives it through a literal in the module instead of from JS.
* pref: let a wasm build compile the wasm file family
get_v_files_from_dir_for_target is backend-blind, so a wasm build pulled in foo.c.v beside foo.wasm.v and the override grouping had to suppress the duplicate declaration rather than pick the right file. The new get_v_files_from_dir_for_backend_target takes the backend, and the filter runs only for the wasm target: a file qualified with another backend's suffix belongs to that backend. Every existing caller passes the empty backend and keeps exactly the list it had, which the regression test pins.
This takes effect only together with vlib/builtin/printing.wasm.v, which supplies the println declaration the wasm target now needs. Its bodies are never linked, because the SSA builder synthesises print/println/eprint/eprintln from a raw write to fd 1 and fd 2 and skips the source body of each. Printing.c.v supplied the same declarations for the C backend.
* wasm: honour @[wasm_import_namespace] for declared host imports
A body-less fn declared in a .wasm.v file with that attribute now becomes a
real wasm module import instead of the unsupported-extern rejection. The
mechanism is read from the AST directive the parser emits for the
attribute, because TypeChecker.declaration_attributes is private and keyed
by pre-transform ids.
The split is deliberate: SSAGen only receives &ssa.Module, and SSA records
an extern name without its signature or namespace, so the layer that still
sees V types has to collect the imports. Gen already owns the flat AST and
the type checker and already builds SSAConfiguration, so WasmImport is
collected there and carried in that config.
Signature mapping covers i32, i64, f32, f64 and pointers. A parameter with
no wasm value type is refused with an error naming the function rather
than silently mapped to a wrong signature. fd_write and proc_exit stay
synthesised and un-declared, because SSA lowers print bodies to a raw
write on fd 1 and fd 2 and skips their source bodies.
The unsupported-extern guard is extended rather than weakened: a declared
import is skipped there, and every other C extern still hard-errors.
The declarations live in user .wasm.v files rather than vlib/builtin,
because the backend filter only runs for backend == 'wasm', so a builtin
file would also be compiled by C, js, native and eval builds.
* wasm: validate emitted modules end to end in the wasm test suite
The existing suites assert that SSAGen compiles a shape without erroring,
which does not establish that the module it produced is a valid wasm module
that runs. This drives seven small programs through the whole pipeline
(parser, checker, transform, SSA, SSAGen, encode) and asserts on the bytes.
The proofs are values rather than compile success: i64 arithmetic returns
4000000000n through an ->i64 export; u8 and i8 in one expression return
255099, which only holds if the zero- and sign-extensions differ; if/else
returns 106 and 207 for its two arms; a for loop returns 4950; a struct
literal of primitives returns 304. Each runs both unoptimized and under
-prod.
The string constant is checked where it lands: data_base is 64, an 11-byte
literal is written at offset 460 with its NUL terminator, and the module
stores the little-endian 460. A printing module is handed to node with host
fd_write and proc_exit stubs, and the captured bytes are exactly the literal
plus println's newline; a proc_exit(42) arrives as 42. That exercises the
memory layout and iovec path, which byte assertions cannot.
Integer literals here are lowered as i64.const followed by i32.wrap_i64
rather than i32.const, so the assertions are on that encoding; it is the
one version-sensitive number in the file and is labelled as such.
* wasm: generate a real string.int body instead of a digit fold
The wasm string.int stub computed `result = result * 10 + (byte - 48)` with no
digit validation, no base-prefix detection and no + handling, so every byte was
folded in as a digit. Measured against a native build of the same expressions,
"ab" was 540 and not one of sign, prefix or validation survived: "a" 49, "0x"
72, "0b101" 50101, "+9" -41, "9999999999" 1410065407.
The stub now mirrors strconv.common_parse_int over the base
builtin.string_int_base picks. An optional + or - skips a byte before the prefix
scan; a 0x/0b/0o prefix selects 16, 2 or 8 and consumes two bytes, falling back
to base 8 when the input ends at the prefix, because common_parse_uint2 needs
three bytes to read one; a letter folds onto its digit value against the base, so
"a" is 10 only in base 16 and invalid everywhere else; and a byte the base cannot
use stops the parse and keeps the value so far, which is the partial result the
reference returns. common_parse_int's 2^31-1 clamp is applied to the signed
value, so an overflow saturates rather than wrapping.
Every expectation in the new table is a value read off a native build of the same
expression, not recalled. The table covers empty, non-numeric, decimal, leading
zeros, bare and trailing prefixes, the prefix letters in both cases, +9, partial
parses and the overflow clamp, and it runs both unoptimized and under -prod.
The previous string-int regression is ungated now that it passes, and its comment
named a cross-module chain that does not exist: the wasm module holds no
strconv__common_parse function at all, and the defect was entirely in the body
the builder synthesised. It is replaced by the table, which covers the same case
among others.
* wasm: measure the shapes whose wasm values were never executed
Compiling without an error is not evidence a shape is right, and several
had never been run at all: float arithmetic, string comparison and
concatenation, match over a value and over an enum, for and while loops,
and signed division and modulo at both 32 and 64 bits. Each case now
compiles one small program through the pipeline (parser, checker,
transform, SSA, SSAGen, encode), runs the module under node with host
fd_write and proc_exit, and compares against a normal build of the same
source, unoptimized and under -prod.
Eight of the ten shapes agree with native, asserted as values rather
than compile success: f64 arithmetic, both float widths against the
precision each one has, string concatenation and its length, match over
an int and over an enum, both loop forms, and signed division and
modulo where -7/2 is -3 in both directions.
String ordering does not agree and is left red, with the full comparison
in the test comment. Equality is right in every case, so == and != are
sound. Ordering is right only where the two strings differ at index 0.
It is wrong where the difference is further along the strings, and for
every length tiebreak. generate_string_lt_body in the SSA builder is
correct, and the memcmp intrinsic lowering is a correct byte loop, so
the corruption is in how that block structure lowers to wasm; the
likely point is the two-block phi that selects the minimum length, but
that is stated as a hypothesis, not a measurement.
The file also closes a parse defect of its own: its first version held
an apostrophe inside a single-quoted const, which terminated the string
early and cascaded into eighty-eight scanner errors.
* wasm: read the string length from the target layout, not offset 8
generate_string_eq_body, generate_string_lt_body and
generate_builder_write_string_body each took the string's length word at
a hardcoded offset 8. That is where a 64-bit pointer puts it; the wasm
target has a 4-byte pointer, so its string is {str@0, len@4, is_lit@8}
and offset 8 is is_lit, which is 1 for every literal.
Measured on the emitted module: min(len_a, len_b) came out as 1, so
memcmp compared byte 0 only, and the length tiebreak was 1 < 1. Both
channels of test_shape_string_comparisons then disagreed with native in
five of eleven cases -- mid_lt, end_lt, end_gt, long_vs_short and
long_ge_short -- which is exactly the shape of a minimum that is too
small together with a tiebreak that can never fire.
block_struct_field_ptr already resolves this on both layouts, and it is
what generate_print_body and generate_string_plus_body use. On a 64-bit
pointer it yields the same const i64 8 and the same get_ptr(i32) type as
the code it replaces, so the generated SSA is unchanged for every other
consumer of these bodies (the arm64 and native tests, which build at
ptr_size 8).
Equality only looked right because every case in the suite compared two
literals, which the constant folder resolves without calling string__eq.
With a parameter on one side it read the same wrong offset: on the module
built before this change, the wasm values for s < "abcXef" were right but
"abcXef" == "abcYef" was true and "abcYef" != "abcXef" was false.
The third site is the same defect measured the same way -- write_string
appended one byte per call, so a builder produced Xh where native
produces X=hello -- so all three are fixed by the same substitution.
Fixes test_shape_string_comparisons, which was left red as the
specification for this.
* wasm: correct the string-comparison defect note in shapes_test.v
Comment-only change. The two blocks above test_shape_string_comparisons
said the case was deliberately left red, and named gen_memcmp's 0x49 as
the likely cause -- but 0x49 is i32.lt_u, as that comment already said,
and the byte loop is sound. Neither block described what was measured:
the generated bodies read the string length at a hardcoded offset 8,
which on wasm32 is is_lit rather than len.
Recorded what the fix was and why the equality cases looked right, so the
next reader does not go looking in the memcmp lowering. No expectation,
no source and no test function was touched: the whole diff is comment
lines.
* wasm: audit the synthesised runtime functions against native
register_basic_format_stubs replaces a set of builtin functions with
bodies the compiler generates itself, and skip_source_fn_in_module
removes the real ones. string.int turned out to be wrong in every
non-decimal case, which raised the question of what else in that set is
wrong, and nobody had measured any of it.
Eighteen cases, each compiled through the pipeline and run under node
alongside a normal build of the same source. Five are correct and stay
pinned: int_str across every width and both signs, format_uint, bool_str,
string.int on twenty inputs including every base prefix and both overflow
directions, string concatenation at two, three and five operands, string
padding to an explicit width, and the strings module's own to_upper,
to_lower, trim, contains, starts_with, ends_with, replace, indexing and
len.
Eleven are refused, and a refusal is a compiler panic rather than a wrong
value: any float interpolation or .str() at all dies on an unregistered
f64.str, 'strg' and f64__strg; an explicit '+' sign dies on
v3_string_plus_sign; strings split, repeat, reverse and index are
unregistered; and the pointer-string stubs die because the encoder
refuses C.strlen. A program that prints a float in interpolation is
ordinary V, so f64.str is the widest-blast-radius item here.
Two are wrong. Zero padding ignores its width for int and i64, and
returns the constant '0' for u64, which is the only stub that destroys
its input outright: every zero-padded unsigned prints 0. A rune in
interpolation prints its decimal code point, so 'Z' prints 90 rather
than Z.
The test is left red on purpose, and each case names what the wasm value
was and what native printed. Removing a case to make the suite green
would lose the only measurement that exists for this set.
* wasm: zero-pad, u64 and rune stubs print what native prints
The audit left two of the eighteen cases wrong, and the interp_pad case
carried all three of the defects named here: the zero-padding pair, the
u64 stub that returned the constant '0', and the rune that printed 90
for Z.
v3_int_zpad and v3_i64_zpad took the width argument and then discarded
it with `_ :=`, so they returned int_str's text unchanged. v3_u64_zpad
was registered as the constant string "0", which is the only stub in
the set that destroys its input rather than truncating it. Both get a
real body now: strconv__format_uint for the unsigned spelling so u64::MAX
reads as twenty digits rather than a negative, int_str otherwise, then
memset of '0' between the sign and the digits.
The sign split is the part that is easy to get backwards, so it was
measured against native rather than reasoned about: -7 at width 6 is
-00007, not 0000-7 and not 000-07. The zeros go after the sign, so the
sign byte is copied first and the fill length is width - text.len.
For the rune, the registered body was never the problem. v3_char_string
is only reached when the format carries a `c` verb, which the audit's
`[Z]` case never uses -- that value came out as 90 because
build_scalar_str_call routes rune.str through stringify_scalar, which
hands the codepoint to int_str like any other integer. The fix is
there, not in the stub, and the stub body that was already correct for
the `c` path is now the same body rune.str uses.
Measured on the module before and after, run under node beside a normal
build of the same source: `${r}`, `r.str()`, a `Z` literal, `é`, `€` and
an emoji now agree with native, including the string length, which was
2, 3 and 6 bytes for 90, 233 and 8364 and is now 1, 2, 3 and 4.
interp_pad goes from wrong to matching. interp_float_fixed, which needs
a real v3_f64_fixed rather than the constant "0.0", is still wrong and
the suite is still red for it.
* wasm: register v3_string_plus_sign so the + interpolation flag prints
The wasm stub set never registered v3_string_plus_sign, so \\\
panicked the SSA builder. Add generate_plus_sign_body (a self-contained
malloc+copy, since string__plus is registered only afterwards) and register
it in the ptr_size==4 fragment. interp_plus_sign now matches native.
* wasm: give the e2e inspector its own names
e2e_test.v redeclared WasmReader, wasm_imports and wasm_type_at, which
ssa_gen_test.v also declares, so the wasm test directory held the same
three helpers twice. The checker tolerated it, which is worse than
rejecting it: a branch that compiles only because a check happens to be
lenient breaks the day that check tightens. Rename them with an e2e_
prefix so each file owns its helpers.
* wasm: prove the driver path end to end, and declare exit for wasm
Every other suite in this directory drives SSAGen directly. That cannot
catch a regression in the driver path: file selection, the wasm file
family, or the printing declarations. A module can lower perfectly and
still never be emitted by v -b wasm.
A new test writes a V program, runs the real compiler on it with -b wasm,
and executes the module it produces under node with a
wasi_snapshot_preview1 host. It asserts the wasm magic so a C-backend
module cannot pass, the import section for fd_write and proc_exit, and
the printed values: an array with for-in, recursion, a range loop and a
struct literal of primitives, each interpolated into a line of stdout.
The gap it exposed while writing it: exit is declared in builtin.c.v, a
C-backend file, so on wasm the checker could not resolve it at all and a
guest calling exit died with 'unknown function: exit'. The backend has
an exit intrinsic that lowers to proc_exit already; only the declaration
was missing. Added to printing.wasm.v with a body that is never linked.
The third case pins the float gap rather than skipping it. A program
printing a float is expected not to compile, and the assertion names
f64.str so the day it is closed this fails and has to be rewritten.
* wasm: pin the struct-signature gap with the layer that causes it
A struct in a signature is a second hole, distinct from the float one,
and it was only visible as a panic naming the wrong place. Sweeping
ordinary V through v -b wasm isolated it: a scalar signature compiles,
the same function taking a struct parameter panics with unknown
function, and returning a struct does too.
The cause is fn_info in vlib/v/gen/wasm/gen.v, which maps every
parameter and the return type through prim_wtype and returns none for
anything that is not a primitive. collect_user_fns therefore never
records such a function, used_fns has no entry for it, and because the
wasm path builds with exact_used_fns the SSA builder skips its
declaration. The function exists and is reachable; it is filtered out
before the builder ever sees it, which is why the error blames the call
site.
The new case keeps the scalar compile as its control, so a failure
cannot be blamed on the println or the struct definition, and pins the
message so a rewrite that changes the shape is visible. Closing it needs
a wasm representation for an aggregate parameter, which is a pointer,
alongside the sret protocol a struct return already uses. That is a
feature rather than a wiring fix, so the case stays red until someone
does it.
* wasm: pass a struct parameter as the pointer it already is
fn_info maps every parameter through prim_wtype and returned none for
anything that is not a primitive, so collect_user_fns never recorded a
function whose signature held a struct, an enum or an option. The
function was reachable and correct; it was filtered out before the SSA
builder saw it, which is why a call to it died with unknown function
naming the caller instead of the filter.
An aggregate arrives as a pointer, and on a 32-bit target a pointer is
the i32 the signature already described, so a parameter now takes that
slot when every field it transitively holds lowers. The rule is
duplicated rather than shared because the driver already imports this
package, so the reverse import would be a cycle; it matches the
driver's struct-literal rule exactly, including the depth bound and the
pointer-field recursion.
Measured against a normal build of the same source, all four shapes
agree: two integer fields, a string field, a nested struct, and a
pointer field. A map parameter is still refused by the driver gate.
A struct return stays refused and now says why in the test. The ABI
passes it through a hidden sret pointer and the function's own result
type is void, which is not what an i32 slot describes, so describing it
as one would silently miscompile rather than fail. That is the half a
separate change has to explain.
Two expectations moved, and both were documenting the filter rather
than the language. shapes_test asserted that a function taking an enum
was not exported; an enum is an i32, so it is exported now. The
struct-signature case in tour_test asserted a refusal; it now asserts
the value, because a parameter arriving as garbage would compile just
as happily.
* wasm: refuse u128 instead of answering with a truncated value
primitive_type_id has no entry for u128 or i128, so the name falls
through to the alias and struct lookups and resolves to something
narrower. Nothing reports this, and the resulting program answers
wrong: measured against a normal build, u128(6148914691236517205) *
u128(6) gives hi=1 natively and hi=0 on wasm, and u128 >> 64 gives 0
where it should give 1. The low limb is right, so a test that only
checked the value it multiplied would have passed.
The wasm target has no 128-bit integer, so the value cannot be held in
one register at all. Refusing the name at resolution is the honest
outcome: it names the missing type at the point of use, where a
developer can act on it, rather than leaving a wrong number to be found
later.
The check is gated on a 32-bit target because that is how this file
already distinguishes wasm from the 64-bit backends, which either
support u128 themselves or refuse it in their own way. Verified that
native still returns hi=1 and that ordinary u64 programs compile
unchanged.
* wasm: pass a struct, option or string return through the sret slot
fn_info mapped a return type through prim_wtype and returned none for
anything that was not a primitive, so any function returning a struct,
an option, a result or a string was dropped from used_fns and a call to
it died with unknown function naming the caller. A parameter taking one
of those already worked: the difference was only that a return has no
slot of its own.
It does have a slot, but it is the caller's. The ABI passes an aggregate
return through a hidden pointer and the function's own result type is
void, so the signature gains a leading i32 and the result becomes void.
That is exactly what the SSA sret path already describes, which is why
this turned out to be a description problem rather than a lowering one.
Measured against a normal build of the same source, all of these agree:
a struct of two integers, a struct with a string field, a nested struct,
a struct returned into a function that also takes one, an option return,
and a string return. The string case is the one worth calling out: a
string is a three-word struct and it travels the same way, so
'hello world' round-trips rather than arriving as an address.
wasm_param_may_be_aggregate now admits OptionType and ResultType as well
as Struct, because an option is a struct_t in SSA with a leading ok. The
payload still decides, so ?int and !int represent and ?map[string]int
stays refused.
Two expectations moved, and both documented a filter that no longer
exists rather than a property of the language. shapes_test asserted that
a fn f() string was not exported, and tour_test asserted that a struct
return was refused. Both are now value assertions, because a return that
arrived as garbage would compile just as happily.
* wasm: lay out only the constants the emitted functions reach
collect_data walked every value in the SSA module and emitted each string
literal, with no reachability test and no deduplication. On a 32-bit
target the SSA builder registers literals for every stub it creates, so
most of them have no referrer at all, and each cost bytes for its text
plus a whole string struct.
reachable_values is the same worklist shape as reachable_functions, but
over values: a block references its instructions, an instruction
references the operands it reads, and a constant is a leaf. A literal
outside that set is now skipped, and a repeat of a literal already laid
out reuses its address instead of reserving another copy.
Measured on this host, compiling and running each program and comparing
against a normal build of the same source:
program before after
hello world 3131 1267
strings, arrays, loops 7014 5150
struct parameters ~4550 ~2700
The data section of hello world went from 2594 bytes to 730, and its 80
distinct strings went to 19, of which exactly one is program data. Every
program above still prints the value it printed before.
The lookup at codegen now fails loudly rather than defaulting to zero: a
literal that was dropped as unreachable but is referenced is an error,
not a null string pointer. That is the guard that makes the filter safe
to trust, because the failure mode of a reachability bug is otherwise
silent.
* Reject unsupported float formatting and clamp Wasm integer overflow
---------
Co-authored-by: MR <mr@local>
Co-authored-by: Alexander Medvednikov <alexander@medvednikov.com>
3.9 KiB
3.9 KiB
WebAssembly backend
V's WebAssembly backend lowers the checked and transformed program to v.ssa, then emits a
WebAssembly binary directly. It shares SSA construction and optimization with the ARM64 backend.
-prod runs the SSA optimizer, including scalar local promotion and phi elimination, before
WebAssembly generation. No C compiler, Emscripten, or Binaryen is needed to emit the module.
The standard compiler includes the backend. Rebuild it, then compile a program:
./v self
./v -b wasm -o hello.wasm examples/hello_world.v
./v -prod -b wasm -o hello.wasm examples/hello_world.v
A standalone compiler built from vlib/v/v.v needs -compile-backend wasm or -all-backends
when it is compiled. -d skip_wasm excludes WebAssembly from a compiler build.
The output path supplied with -o is used exactly. Programs with main export a WASI _start
entry point and linear memory. Module initialization functions run before main, in dependency
order. Modules without main run their initialization functions when instantiated.
The backend supports scalar integer, boolean, and floating-point functions, calls, recursion,
branches, loops, and numeric globals. print and println support integers, booleans, and string
literals. WASI output uses
wasi_snapshot_preview1.fd_write. Main-module scalar functions are exported using their V names;
imported functions use their qualified module name with dots replaced by double underscores.
An explicit @[export: 'name'] attribute supplies the export name.
Rune literals retain their full Unicode code points in expressions and global initializers. Integer literals retain their width until an operation or comparison selects its operand types, including full-width constants in production optimization. Numeric assignments and arguments convert to their declared types before optimization, for both direct calls and function values. Floating-point unary negation preserves signed zero in both unoptimized and optimized output. Constants and global initializers in moduleless scripts keep the script module scope after imports. Implicit-main scripts retain imported functions called by their top-level statements. User functions retain their bodies when their names overlap synthetic runtime helpers.
WebAssembly uses 32-bit pointers and target-specific SSA memory layouts. Control flow is emitted from SSA basic blocks, with parallel copies on phi edges. Unsupported operations produce a compiler error. Aggregate language features remain experimental. Dynamic arrays of primitive elements, structs whose fields all have a lowering, options and results are supported; maps, fixed and nested arrays, aggregate array elements, and structs holding an unlifted field type are not, and still produce a compiler error rather than a wrong answer.
Floating-point calculations are supported, but formatting a float as text remains unsupported. Fixed float interpolation is rejected instead of returning a placeholder value.
Host functions can be declared in a .wasm.v file with a module import namespace:
@[wasm_import_namespace: 'host']
fn WASM.read_value(address &u8, count i64) int
The host supplies host.read_value when instantiating the module. Scalar parameters and results
map to WebAssembly value types; pointers are 32-bit offsets into the module's linear memory.
Aggregate import parameters and results are rejected. Only called imports are emitted.
WebAssembly builds select .wasm.v sources and exclude other backend file families.
Runtime allocations use a zeroing bump allocator. Heap storage remains allocated for the module's lifetime. Function calls use a separate 1 MiB stack in linear memory and trap if it is exhausted. After a runtime trap, create a fresh instance before continuing execution.
The focused tests instantiate generated modules with Node.js and exercise both unoptimized and optimized SSA:
./v test vlib/v/gen/wasm/ssa_gen_test.v
V3_TEST_WASM=1 ./v vlib/v/compiler_tests/wasm_codegen_test.v