Use -1 element instead of 0, as 0 wouldn't affect sum.
Fix 'reslice' wording as example is an array, not a slice.
Increase slice length to 2 to demonstrate feature better.
Split out "for ranges" test from basics with shorter array.
Use usize sum to avoid cast.
Improve comments.
Reuse sum vars.
Reviewed-on: https://codeberg.org/ziglang/zig/pulls/37181
Reviewed-by: Andrew Kelley <andrew@ziglang.org>
This commit implements the `@backingInt` and `@fromBackingInt` builtins.
These builtins are meant to replace the now-deprecated `@intFromEnum` and
`@enumFromInt` builtins.
`@backingInt` works with all enums and with bitpacks with *explicit* backing
integer types only. It also works with tagged unions, returning the backing
integer of the active tag value.
An `undefined` enum or bitpack yields an `undefined` backing integer.
The builtin is lowered to `bit_cast`.
`@fromBackingInt` infers its result type, which may be any enum or a bitpack
with an *explicit* backing integer type. It takes a parameter of exactly
that backing integer type.
For enums, passing a backing integer that is either `undefined` or would
yield an invalid tag value results in safety-checked Illegal Behavior.
For bitpacks, passing an `undefined` backing integer yields an `undefined`
bitpack.
The builtin is lowered to `bit_cast` or `bit_cast_safe` (more details below).
`@bitCast` now also performs a safety check for invalid tag values if its
destination type is an `enum`.
This commit also introduces a new AIR instruction, `bit_cast_safe`.
Similarly to `int_cast_safe`, it includes a safety check for invalid enum
tag values. When `@enumFromInt` is eventually removed, `int_cast_safe` won't
need to perform this safety check anymore.
Legalize only implements `expand_bit_cast_safe`, and no additional insts
to scalarize `bit_cast_safe` with vector/array operands. That is because
it can just reuse the existing scalarizations for regular `bit_cast`.
The safety check mandated by `bit_cast_safe` is only desired if the result
type is a scalar enum tag value, so the scalarization process is exactly
equivalent. The only difference is that we have to perform a safety check
on the result of the scalarization.
This means that there's no reason for a backend to want scalarization of
regular `bit_cast`, but not of `bit_cast_safe` (or the other way round).
If a backend is having trouble with the `bit_cast_safe` safety check it
should just use `expand_bit_cast_safe` instead.
Also adds a `std.meta.BackingInt()` function to conveniently get the result
type of `@backingInt` and for use in the langref.
Empty enums are uninstantiable, so their backing integer should be too.
This avoids a bunch of weird special cases during semantic analysis.
If you were previously using `enum {}`, your code should continue to work
fine.
If you were previously using `enum(uN) {}` (where `uN` is any integer type),
you'll probably want to use `enum(uN) { _ }` instead.
Currently these definitions do not result in an LL(k) generated parser.
For in depth reasoning, see commit 296b39c323
(grammar: fix ParamDeclList worst-case exponential time)
AFL++ is able to identify "hangs" as well as crashes and has found
several small inputs that take multiple seconds for the generated parser
to parse. This is because our PEG is not yet LL(k) despite recognizing
the same language as our handwritten LL(k) recursive descent parser.
Consider the PEG semantics for the following non-terminal:
ParamDeclList <- (ParamDecl COMMA)* (ParamDecl / DOT3 COMMA?)?
Here, the PEG checks twice if the last ParamDecl matches when
there is no trailing comma. Now, consider that the ParamDecl may
recursively contain a nested ParamDeclList, which in turn has ParamDecls
that contain further nested ParamDeclLists. It should be clear that this
results in worst-case exponential runtime.
This can be avoided by defining ParamDeclList using recursion instead of
the * operator.
This same pattern likely needs to be applied to other instances of this
pattern in the grammar.
The main motivation for fixing this is to improve fuzzer performance.
Additionally, using recursion rather than the * operator makes it easier
to translate from a PEG to a CFG, which I'm quite confident we will want
to do for the language specification eventually. A CFG would also allow
for smith-based fuzz testing to complement the oracle-based approach.
Consider the new test case (found by AFL++):
test{for(0)|t|0,const w=0;}
Currently the grammar backtracks after failing to parse this with the
ForStatement rule and instead finds that the VarAssignStatement rule
is a match. This behavior is not LL(k) and inconsistent with the parser.
Add the required negative lookahead to eliminate this case and similar
cases.
Currently the grammar doesn't function as intended and forbids
addrspace followed by align. Add the required negative lookaheads
to fix this as well as a test case.
This is a case in which the grammar is currently not LL(k) and needs a
negative lookahead to prevent backtracking.
The new grammar is consistent with the Parse.zig implementation.
See the new test case for an example.
The tokenizer allows the unicode byte order mark (U+FEFF) to be present
at the start of the file, but the grammar currently does not.
Fixing the grammar requires some care due to how same-line doc comments
are forbidden in the grammar.
Currently the grammar only allows exactly one pointer modifier order but
the Parse.zig implementation allows any order. However, the current
Parse.zig behavior of allowing any order while forbidding duplicate
modifiers would require a combinatorial explosion of grammar rules to
specify (currently 5! i.e. 120).
The simplest, most permissive grammar would remove the ordering
requirement and allow duplicate pointer modifiers. However, this would
unfortunately require a more complex AST data layout due to the child
nodes of align() and addrspace() modifiers. Since these are the only
pointer modifiers with child nodes and 2! is a perfectly reasonable
number, forbid duplicate align() and addrspace() modifiers to keep the
Ast data structure simple. In order to resolve the difference between
the formal grammar and parser implementation, allow duplicate
single-token modifiers (allowzero, const, volatile) in the grammar and
move the compile error for that case to AstGen.
Note that zig fmt will now silently remove duplicate single-token
modifiers, which I think is a nice little UX improvement.
This is necessary to match the behavior of the parser. This is not a
complete list of of prefixes that cause the PEG to differ in behavior
from the parser, it is however difficult to determine this list by
reading the grammar alone and I plan to let the AFL++ fuzzer find
the missing items and fill it in while adding test cases to prevent
regression.
This is a first pass at consistency with Parse.zig at the expression
level, adding constant lookaheads to the grammar in several places
to match the behavior of Parse.zig.
I also updated the comments in Parse.zig and refactored for clarity,
adding assertions in a few places but not intending any change in
behavior.
This commit also adds comments to the grammar and Parse.zig explaining
what needs to be done in practice to ensure that the grammar is LL(k)
and that the parser has worst-case linear runtime.
This is a first pass at consistency with Parse.zig on the statement
level, adding constant lookaheads to the grammar in several places
to match the behavior of Parse.zig.
It is not clear why this lookahead was added in the first place, but
it does not reflect the behavior of the Parse.zig implementation as
demonstrated with the new test case.
My attempt to handle this case in 6f151109b5fdf88eb8b has proven
insufficient, the test case added by this commit was failing.
Solve this with a "start of file" non-terminal in the PEG grammar,
which unfortunately isn't portable but much better reflects the actual
implementation and is much simpler.
This unbounded lookahead is causing discrepancies between the Parse.zig
implementation and the PEG grammar. Also, unbounded lookahead should
never have been added to the PEG grammar in the first place.
It was added in 785fb1be11 which
demonstrated insufficient understanding of PEG semantics through many
redundant additions.
Whatever problem this unbounded lookahead attempted to solve needs to be
solved differently in any case.
This commit adds one test case found by AFL++ that fails before this
commit and now passes.
Commit 785fb1be11 made many redundant
changes to the grammar due to insufficient understanding of PEG
semantics. This commit reverts the most obviously redundant of those
changes.
The unbounded lookahead through e.g. !ExprSuffix still needs to be
reverted, but I want to do that in a future commit since that change
can't be reasoned about locally.
Currently the grammar fails to enforce matching whitespace or lack of
whitespace around operators that are immediately preceded by a multiline
string literal and terminating newline.
This matches the Parse.zig implementation, which has a special parse
error for this case:
> ambiguous use of '&&'; use 'and' for logical AND, or change whitespace
> to ' & &' for bitwise AND
/// should not be parsed as a division operator followed by a comment
if it appears in a context where a division operator is accepted.
Add the missing negative lookahead to the SLASH token.
The grammar needs to be fixed to match Parse.zig's "operator has
whitespace on one side but not the other" error, which is implemented
in Parse.zig using a constant lookbehind of 1.
However, standard PEG does not support lookbehind, which means our
approach to handling whitespace needs to be reworked.
Moving the `skip` non-terminal to be processed before each "token"
in the grammar rather than afterwards makes it possible to cleanly
match the Parse.zig operator whitespace handling, which will be
implemented in the next commit.
This commit is purely a refactor, no functional changes are intended.
twil3akine