I'm writing a small numeric library whose kernels run on the CPU, on CUDA and on Vulkan through the self-hosted SPIR-V backend, and I was making them generic over the float type. `f64` needs its own `exp` and `log` on the GPU, so I wrote them as polynomial series, and the `f64` Vulkan module stopped validating. This fixes the three backend bugs underneath that, and a crash I ran into after merging master.
The series take floats apart with `@bitCast` on vectors. Between vectors of the same length this reaches the Vulkan/OpenGL fallback in `bitCast`, which stores the source into a function variable of the destination type:
```zig
export fn main() callconv(.{ .spirv_kernel = .{ .x = 1, .y = 1, .z = 1 } }) void {
var x: @Vector(2, f32) = .{ 1, 2 };
_ = &x;
const bits: @Vector(2, i32) = @bitCast(x);
_ = bits;
}
```
```
$ zig build-exe -fno-llvm -target spirv64-vulkan -mcpu vulkan_v1_2 bitcast.zig -femit-bin=bitcast.spv
$ spirv-val --target-env vulkan1.2 bitcast.spv
error: line 45: OpStore Pointer <id> '36[%36]'s type does not match Object <id> '29[%29]'s type.
```
An elementwise vector bitcast is now a scalar `bitCast` per lane, the way `buildConvert` and friends already unroll vectors. Going through the scalar path also normalizes lanes of strange integers: a single `OpBitcast` on the whole vector validates for `@Vector(n, u3)` to `@Vector(n, i3)`, but leaves lanes that don't compare equal to the negative values they should hold.
The series also convert between integer and float lanes. A vector of one lane, or of more than four without `vector16`, is an array in SPIR-V, and `airFloatFromInt` and `airIntFromFloat` emitted a single `OpConvert*` with that array as its result type:
```zig
export fn main() callconv(.{ .spirv_kernel = .{ .x = 1, .y = 1, .z = 1 } }) void {
var x: @Vector(1, i32) = .{1};
_ = &x;
const y: @Vector(1, f32) = @floatFromInt(x);
_ = y;
}
```
```
$ spirv-val --target-env vulkan1.2 convert.spv
error: line 40: Expected float scalar or vector type as Result Type: ConvertSToF
%y = OpConvertSToF %_arr_f32_u32_1 %26
```
`@floatCast` and `@intCast` already go through `buildConvert`, which unrolls the vector; these two now do as well, with `buildConvert` picking the int/float conversion opcodes.
The coefficients sit in a comptime array that the series take as a comptime slice, and that alone fails to compile, although nothing reads the slice at runtime:
```zig
const weights = [_]f32{ 1, 2, 3 };
inline fn first(comptime w: []const f32) f32 {
return w[0];
}
export fn main() callconv(.{ .spirv_kernel = .{ .x = 1, .y = 1, .z = 1 } }) void {
var x: f32 = first(&weights);
_ = &x;
}
```
```
$ zig build-exe -fno-llvm -target spirv64-vulkan -mcpu vulkan_v1_2 dbgvar.zig -femit-bin=dbgvar.spv
dbgvar.zig:7:8: error: TODO (SPIR-V): pointer to constant 'dbgvar.weights'
```
It appears `airDbgVar` lowers the operand only to attach an `OpName` to it, and a pointer to a non-extern constant is still a TODO. A comptime-known value has no runtime variable of its own to name, so `airDbgVar` now leaves interned operands alone. The same thing held back 58 of the behavior tests skipped on SPIR-V, and the test commit un-skips them.
After merging master, four of those tests crashed the compiler in `structFieldPtr`, which has `unreachable` for packed containers since ebb8683ef1 ([#36996](https://codeberg.org/ziglang/zig/pulls/36996)):
```zig
const Flags = packed struct { a: bool, b: bool };
export fn main() callconv(.{ .spirv_kernel = .{ .x = 1, .y = 1, .z = 1 } }) void {
var flags: Flags = .{ .a = true, .b = false };
const b = &flags.b;
b.* = true;
}
```
```
thread 3961624 panic: reached unreachable code
src/codegen/spirv/CodeGen.zig:6906:27: 0x4767157 in structFieldPtr (main.zig)
.@"packed" => unreachable,
```
My reading is that the `unreachable` holds for the pointer derivations it was written for, since `constantPtr` now turns packed pointers into pointers to their host integer first, but `airStructFieldPtr` and `airStructFieldPtrIndex` still pass packed structs through. A pointer to a packed field is a bit-pointer into the struct's own host integer, so it is the struct pointer: that is what the old code computed, and what the C backend now does with `.begin`. Packed unions get back the TODO error they had before. I may be missing a plan for packed field pointers here, and I'm happy to change it if so. The commit also un-skips three packed tests that crash on master the same way. I suppose nothing caught this because those tests are skipped on SPIR-V and the `spirv64-vulkan` behavior target is commented out in `test/tests.zig`.
I ran the behavior tests through `tools/unskip_spirv_tests.zig` and [zig-spirv-test-executor](https://github.com/Snektron/zig-spirv-test-executor) on a Radeon 680M, and the new and un-skipped tests on an RTX 4060 as well. Every fix has tests that fail on master and pass with it; the vector ones compare lane by lane, since `expectEqual` on vectors doesn't compile for SPIR-V yet. Running each behavior file whole, nothing that passes on master fails. Five more skips already pass on master without these changes, so I left them out, but I can add them. The new vector tests copy the aarch64, arm, sparc64 and riscv64 skips of the tests around them, which I couldn't check.
Reviewed-on: https://codeberg.org/ziglang/zig/pulls/37040
Reviewed-by: Ali Cheraghi <alichraghi@noreply.codeberg.org>
The compiler actually doesn't need any functional changes for this: Sema
does reification based on the tag indices of `std.builtin.Type` already!
So, no zig1.wasm update is necessary.
This change is necessary to disallow name clashes between fields and
decls on a type, which is a prerequisite of #9938.
Reorganize how the binOp and genBinOp functions work.
I've spent quite a while here reading exactly through the spec and so many
tests are enabled because of several critical issues the old design had.
There are some regressions that will take a long time to figure out individually
so I will ignore them for now, and pray they get fixed by themselves. When
we're closer to 100% passing is when I will start diving into them one-by-one.
what was happening is that instructions like `lb` were only affecting the lower bytes of the register and leaving the top dirty. this would lead to situtations were `cmp_eq` for example was using `xor`, which was failing because of the left-over stuff in the top of the register.
with this commit, we now zero out or truncate depending on the context, to ensure instructions like xor will provide proper results.
- implements `airSlice`, `airBitAnd`, `airBitOr`, `airShr`.
- got a basic design going for the `airErrorName` but for some reason it simply returns
empty bytes. will investigate further.
- only generating `.got.zig` entries when not compiling an object or shared library
- reduced the total amount of ops a mnemonic can have to 3, simplifying the logic
Besides the Intel OpenCL CPU runtime, we can now run the
behavior tests using the Portable Computing Language. This
implementation is open-source, so it will be easier for us
to patch in updated versions of spirv-llvm-translator that
have bug fixes etc.
We've got a big one here! This commit reworks how we represent pointers
in the InternPool, and rewrites the logic for loading and storing from
them at comptime.
Firstly, the pointer representation. Previously, pointers were
represented in a highly structured manner: pointers to fields, array
elements, etc, were explicitly represented. This works well for simple
cases, but is quite difficult to handle in the cases of unusual
reinterpretations, pointer casts, offsets, etc. Therefore, pointers are
now represented in a more "flat" manner. For types without well-defined
layouts -- such as comptime-only types, automatic-layout aggregates, and
so on -- we still use this "hierarchical" structure. However, for types
with well-defined layouts, we use a byte offset associated with the
pointer. This allows the comptime pointer access logic to deal with
reinterpreted pointers far more gracefully, because the "base address"
of a pointer -- for instance a `field` -- is a single value which
pointer accesses cannot exceed since the parent has undefined layout.
This strategy is also more useful to most backends -- see the updated
logic in `codegen.zig` and `codegen/llvm.zig`. For backends which do
prefer a chain of field and elements accesses for lowering pointer
values, such as SPIR-V, there is a helpful function in `Value` which
creates a strategy to derive a pointer value using ideally only field
and element accesses. This is actually more correct than the previous
logic, since it correctly handles pointer casts which, after the dust
has settled, end up referring exactly to an aggregate field or array
element.
In terms of the pointer access code, it has been rewritten from the
ground up. The old logic had become rather a mess of special cases being
added whenever bugs were hit, and was still riddled with bugs. The new
logic was written to handle the "difficult" cases correctly, the most
notable of which is restructuring of a comptime-only array (for
instance, converting a `[3][2]comptime_int` to a `[2][3]comptime_int`.
Currently, the logic for loading and storing work somewhat differently,
but a future change will likely improve the loading logic to bring it
more in line with the store strategy. As far as I can tell, the rewrite
has fixed all bugs exposed by #19414.
As a part of this, the comptime bitcast logic has also been rewritten.
Previously, bitcasts simply worked by serializing the entire value into
an in-memory buffer, then deserializing it. This strategy has two key
weaknesses: pointers, and undefined values. Representations of these
values at comptime cannot be easily serialized/deserialized whilst
preserving data, which means many bitcasts would become runtime-known if
pointers were involved, or would turn `undefined` values into `0xAA`.
The new logic works by "flattening" the datastructure to be cast into a
sequence of bit-packed atomic values, and then "unflattening" it; using
serialization when necessary, but with special handling for `undefined`
values and for pointers which align in virtual memory. The resulting
code is definitely slower -- more on this later -- but it is correct.
The pointer access and bitcast logic required some helper functions and
types which are not generally useful elsewhere, so I opted to split them
into separate files `Sema/comptime_ptr_access.zig` and
`Sema/bitcast.zig`, with simple re-exports in `Sema.zig` for their small
public APIs.
Whilst working on this branch, I caught various unrelated bugs with
transitive Sema errors, and with the handling of `undefined` values.
These bugs have been fixed, and corresponding behavior test added.
In terms of performance, I do anticipate that this commit will regress
performance somewhat, because the new pointer access and bitcast logic
is necessarily more complex. I have not yet taken performance
measurements, but will do shortly, and post the results in this PR. If
the performance regression is severe, I will do work to to optimize the
new logic before merge.
Resolves: #19452Resolves: #19460
Andrew Kelley