This was included as part of 5d58306162
but there was no mention of it in the commit message and I didn't notice
it in the review.
This is a partial (one-line) revert. Let's discuss this change carefully
and separately.
The existing `lock` and `unlock` methods continue to act exclusively.
They should be used when data may be mutated. New methods, `lockShared`
and `unlockShared`, may be used for shared locking in situations where
multiple independent users are reading but not mutating data.
For generic code, it is far more useful to consider this equivalent to
`unreachable`. Meanwhile, it is difficult to accidentally write concrete
code which performs this operation, so very little is actually lost from
not having a compile error.
If we accept that this operation does not trigger a compile error, then
it already has the potential to invoke Illegal Behavior today, because
uninstantiable types have an unspecified size in memory, so the
dereference always potentially exceeds the pointer's provenance. (That
said, this only means it is *possible* for the operation to invoke IB,
so the langspec should nonetheless explicitly specify that dereferencing
a pointer to an uninstantiable type is itself Illegal Behavior.)
Storing uninstantiable types into memory does not require any specific
language rules, because that operation can never be reached anyway due
to it requiring an operand (the RHS of `a = b`) whose type is the store
type, which is (by definition) impossible for uninstantiable types.
Resolves: https://codeberg.org/ziglang/zig/issues/36247
and remove "release" from the enum tag names.
no functional change, however, despite the addition of
backwards-compatibile declarations in this patch, it is breaking because
expressions that use `==` or `!=` operators will not able to use the
deprecated names.
I predict these names to survive until the 1.0 tag without any more
breakage.
Adds target info for the PSX. Builds fine with the following:
```zig
const target = b.resolveTargetQuery(.{
.os_tag = .psx,
.cpu_arch = .mipsel,
});
```
the only "problem" being that it spits out an error from LLVM even
though generating an object file succeeds:
```sh
❯ zig build
install
└─ install generated to main.o
└─ compile obj obj Debug mipsel-psx failure
error: warning: MIPS-I support is experimental
```
If one overrides `SelfInfo` in their `root.debug`, it is no longer possible
to refer to the default `SelfInfo` that the std.debug uses.
This allows the developer to still refer to the std's default without
copying the SelfInfo zig files to their project.
The intention behind the iterator was to avoid needing to allocate the
symbols, but in practice we need to allocate them anyway since we need
to reverse their order and don't have random access. The alternative
would be an N^2 algorithm.
In practice this isn't that bad, because even if the allocation fails,
we'll still end up printing the address, so the user still ends up with
the necessary information to reconstruct the crash. I don't think it's
worth it to try to set up some kind of ring buffer or return partial
results on failure, but may revisit this.
This will be relevant once #31605 is merged.
In general, stack traces do *not* contain unique addresses for inlined
frames, but for error return traces, they will after the above PR. This
bool indicates that code printing the trace should not try to resolve
inline frames since they're explicitly encoded into the instruction
addresses.
This is set as state on stack trace rather than passed into the
formatting methods as an argument, as it's not really a formatting
option--whether or not it's correct to resolve inlines is decided at the
time of capture!
Modifies the `Allocator` implementation provided by `ArenaAllocator` to be
threadsafe using only atomics and no synchronization primitives locked
behind an `Io` implementation.
At its core this is a lock-free singly linked list which uses CAS loops to
exchange the head node. A nice property of `ArenaAllocator` is that the
only functions that can ever remove nodes from its linked list are `reset`
and `deinit`, both of which are not part of the `Allocator` interface and
thus aren't threadsafe, so node-related ABA problems are impossible.
There *are* some trade-offs: end index tracking is now per node instead of
per allocator instance. It's not possible to publish a head node and its
end index at the same time if the latter isn't part of the former.
Another compromise had to be made in regards to resizing existing nodes.
Annoyingly, `rawResize` of an arbitrary thread-safe child allocator can
of course never be guaranteed to be an atomic operation, so only one
`alloc` call can ever resize at the same time, other threads have to
consider any resizes they attempt during that time failed. This causes
slightly less optimal behavior than what could be achieved with a mutex.
The LSB of `Node.size` is used to signal that a node is being resized.
This means that all nodes have to have an even size.
Calls to `alloc` have to allocate new nodes optimistically as they can
only know whether any CAS on a head node will succeed after attempting it,
and to attempt the CAS they of course already need to know the address of
the freshly allocated node they are trying to make the new head.
The simplest solution to this would be to just free the new node again if
a CAS fails, however this can be expensive and would mean that in practice
arenas could only really be used with a GPA as their child allocator. To
work around this, this implementation keeps its own free list of nodes
which didn't make their CAS to be reused by a later `alloc` invocation.
To keep things simple and avoid ABA problems the free list is only ever
be accessed beyond its head by 'stealing' the head node (and thus the
entire list) with an atomic swap. This makes iteration and removal trivial
since there's only ever one thread doing it at a time which also owns all
nodes it's holding. When the thread is done it can just push its list onto
the free list again.
This implementation offers comparable performance to the previous one when
only being accessed by a single thread and a slight speedup compared to
the previous implementation wrapped into a `ThreadSafeAllocator` up to ~7
threads performing operations on it concurrently.
(measured on a base model MacBook Pro M1)
-- On the standard library side:
The `input: []const u8` parameter of functions passed to `testing.fuzz`
has changed to `smith: *testing.Smith`. `Smith` is used to generate
values from libfuzzer or input bytes generated by libfuzzer.
`Smith` contains the following base methods:
* `value` as a generic method for generating any type
* `eos` for generating end-of-stream markers. Provides the additional
guarantee `true` will eventually by provided.
* `bytes` for filling a byte array.
* `slice` for filling part of a buffer and providing the length.
`Smith.Weight` is used for giving value ranges a higher probability of
being selected. By default, every value has a weight of zero (i.e. they
will not be selected). Weights can only apply to values that fit within
a u64. The above functions have corresponding ones that accept weights.
Additionally, the following functions are provided:
* `baselineWeights` which provides a set of weights containing every
possible value of a type.
* `eosSimpleWeighted` for unique weights for `true` and `false`
* `valueRangeAtMost` and `valueRangeLessThan` for weighing only a range
of values.
-- On the libfuzzer and abi side:
--- Uids
These are u32s which are used to classify requested values. This solves
the problem of a mutation causing a new value to be requested and
shifting all future values; for example:
1. An initial input contains the values 1, 2, 3 which are interpreted
as a, b, and c respectively by the test.
2. The 1 is mutated to a 4 which causes the test to request an extra
value interpreted as d. The input is now 4, 2, 3, 5 (new value) which
the test corresponds to a, d, b, c; however, b and c no longer
correspond to their original values.
Uids contain a hash component and type component. The hash component
is currently determined in `Smith` by taking a hash of the calling
`@returnAddress()` or via an argument in the corresponding `WithHash`
functions. The type component is used extensively in libfuzzer with its
hashmaps.
--- Mutations
At the start of a cycle (a run), a random number of values to mutate is
selected with less being exponentially more likely. The indexes of the
values are selected from a selected uid with a logarithmic bias to uids
with more values.
Mutations may change a single values, several consecutive values in a
uid, or several consecutive values in the uid-independent order they
were requested. They may generate random values, mutate from previous
ones, or copy from other values in the same uid from the same input or
spliced from another.
For integers, mutations from previous ones currently only generates
random values. For bytes, mutations from previous mix new random data
and previous bytes with a set number of mutations.
--- Passive Minimization
A different approach has been taken for minimizing inputs: instead of
trying a fixed set of mutations when a fresh input is found, the input
is instead simply added to the corpus and removed when it is no longer
valuable.
The quality of an input is measured based off how many unique pcs it
hit and how many values it needed from the fuzzer. It is tracked which
inputs hold the best qualities for each pc for hitting the minimum and
maximum unique pcs while needing the least values.
Once all an input's qualities have been superseded for the pcs it hit,
it is removed from the corpus.
-- Comparison to byte-based smith
A byte-based smith would be much more inefficient and complex than this
solution. It would be unable to solve the shifting problem that Uids
do. It is unable to provide values from the fuzzer past end-of-stream.
Even with feedback, it would be unable to act on dynamic weights which
have proven essential with the updated tests (e.g. to constrain values
to a range).
-- Test updates
All the standard library tests have been updated to use the new smith
interface. For `Deque`, an ad hoc allocator was written to improve
performance and remove reliance on heap allocation. `TokenSmith` has
been added to aid in testing Ast and help inform decisions on the smith
interface.
Andrew Kelley