The resize and signal goroutines in runProcess both assigned the err
variable of the enclosing function, so concurrent Resize and Signal
calls on a process raced, and either goroutine could log the other's
error. Use a local error in each goroutine.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Propagate proxy CA cleanup failures from both Linux executors so builds
cannot succeed when the temporary certificate cannot be removed.
Restore certificate bundles byte-for-byte while preserving build-time changes
and report process and cleanup failures together.
Based on: moby/buildkit#7043
Co-authored-by: bbartels <23058572+bbartels@users.noreply.github.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 62a1b69ee38bc68b23e7c096bbb7372ed13af6e1)
The rootless spec conversion removes the /sys mount, so the runtime never
creates its mount point in the rootfs. A rootful build gets that empty
directory, which meant the same LLB left different mount points behind
depending on whether the worker was rootless.
Recreate the mount points of the removed mounts after the container has
exited, rather than cleaning them up from the rootful result, which would
change existing output and need a compatibility version bump. Creating
them up front would instead turn a mount point that the image does not
ship into a directory the build can write to. Nested destinations are
skipped: the runtime creates those inside the parent mount, where they
never reach the rootfs.
Fixes#6686
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
- Updated multiple instances across various files to replace errors.Errorf with errors.New for error message creation.
- This change enhances consistency in error handling throughout the codebase.
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
Delay containerd stdin CloseIO until BuildKit's stdin reader reaches EOF.
This avoids racing the shim FIFO copy path, where closing stdin before
the copy completes can leave execs stuck or missing stdin data.
Add a client gateway regression that runs concurrent execs writing stdin
into files and verifies every payload is present.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Keep exec network modes limited to sandbox, host, and none, and pass proxy
network configuration separately through solve and executor runtime state.
Proxy execs now use bridge-style egress by default, host egress only for host
network mode with entitlement, and no proxy for none mode. Add integration
coverage for bridge, host, and none proxy behavior across OCI and containerd
workers.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Record successful GET responses through the exec proxy as provenance
materials and report incomplete material coverage as a typed solve error.
Thread proxy policy and capture state through typed executor/network options.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Add a build request option that rewrites default exec networking to an
internal proxy network while preserving explicit none networking.
Route HTTP and HTTPS traffic through a BuildKit-owned proxy namespace, enforce
source policy checks for proxied requests, and inject a temporary CA into Linux
rootfs trust bundles for HTTPS interception.
Share namespace pooling between CNI and proxy providers, and cover proxy mode
with unit and integration tests.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Add executor.ValidContainerID and enforce it in runc/containerd Run paths.
Only runc executor used the ID in filesystem operations.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 789df2422341960b7549d14ea475add43e73cd74)
Replace duplicate nopCloser, nopWriteCloser, and bufferCloser
type definitions across test and production code with the
shared iohelper.NopWriteCloser type.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Convert usages of `github.com/docker/docker/pkg/idtools` to
`github.com/moby/sys/user` in order to break the dependency between
buildkit and docker.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
Like CRI, we should support custom shim paths. We can just add this as a
new config field "path" under the runtime object.
While "name" can be either the name of a custom shim (from which a path
is derived), or a path itself, if configured as a path, no options can
be provided - this is because to derive the type for the options struct,
we need the name to be a member of a well-known set (currently runc and
hcs).
With this patch, it's now possible to configure a runtime with a custom
shim at a non-default path, and include it's options (which was
previously not possible to do in buildkit).
Signed-off-by: Justin Chadwell <me@jedevc.com>
This change splits the containerdexecutor.Run() function into smaller
pieces and enables it to run on Windows.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Buildkit code is mostly generic enough to support FreeBSD, however
there are some quirks / infrastructural pieces that need to be
addressed for full support, to name some
- contenthash.NewFromStat attempts to set Devmajor / Devminor for
regular files, assuming that RDev is zero for regular
files. Unlike on Linux, it's not the case for FreeBSD.
- containerdexecutor.Run uses bind mounts for rootfs. Bind mounts
are not supported in FreeBSD and we should use nullfs instead
- There is no CI job to run tests on FreeBSD
- Some dependencies weren't ported
This change ports buildkit to FreeBSD
Signed-off-by: Artem Khramov <akhramov@pm.me>
Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
In order to support identity mapping and user namespaces, the Moby
project needs to defer the creation of a container's network namespace
to the runtime and hook into the container lifecycle to configure the
network namespace before the user binary is started. The standard way to
do so is by configuring a `createRuntime` OCI lifecycle hook, in which
the OCI runtime executes a specified process in the runtime environment
after the container has been created and before it is started. In the
case of Moby the network namespace needs to be configured from the
daemon process, which necessitates that the hook process communicate
with the daemon process. This is complicated and slow. All the hook
process does is inform the daemon of the container's PID and wait until
the daemon has finished applying the network namespace configuration.
There is an alternative to the `createRuntime` OCI hook which containerd
clients can take advantage of. The `container.NewTask` method is
directly analogous to the OCI create operation, and the `task.Start`
method is directly analogous to the OCI start operation. Any operations
performed between the `NewTask` and `Start` calls are therefore directly
analogous to `createRuntime` OCI hooks, without needing to execute any
external processes! Provide a mechanism for network.Namespace instances
to register a callback function which can be used to configure a
container's network namespace instead of, or in addition to,
`createRuntime` OCI hooks.
Signed-off-by: Cory Snider <csnider@mirantis.com>
This allows a frontend to request a specific for stubs removal.
By default, if not specified, this will revert to the previous
behaviour. New gateway clients however will set the property to the
desired recursive removal mode.
This property needs to be set for both components that call the
executor: for ExecOp, as well as for the StartContainer API.
Signed-off-by: Justin Chadwell <me@jedevc.com>
Deleting a containerd task whose status is Created fails with a
"precondition failed" error. This is because (aside from Windows) a
process is spawned when the task is created, and deleting the task while
the process is running would leak the process if it was allowed.
Change the deferred `task.Delete` call to pass the `WithProcessKill`
delete option so the cleanup has a chance to succeed in the event that
the `p.Start` call inside `runProcess` returns an error.
Signed-off-by: Cory Snider <csnider@mirantis.com>
This adds netNSPoolSize pool options which allow setting a target
network namespace pool size. buildkitd will create this number of
network namespaces at startup (without blocking). When a container
execution finishes, the network namespace gets returned to the pool. If
the pool goes above the target size, there is a grace period to allow
network namespaces to be reused, and if this passes without reuse, the
extra namespaces will be released.
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
Tonis Tiigi