* tests/cp/capability.sh: Add a getlimits_ invocation. Allow 'cp' to
fail with ENOTSUP, which occurs when trying to copy the security.selinux
extended attribute.
E.g. with podman we get default SELinux context mount options
for /dev/shm and /, resulting in all files getting context like
the following when created:
security.selinux="system_u:object_r:container_file_t:s0:c458,c677"
However you can't explicitly set that xattr:
$ touch file
$ setfattr -n "security.selinux" \
-v "system_u:object_r:container_file_t:s0:c458,c677" file
setfattr: file: Operation not supported
* tests/mv/part-symlink.sh: Avoid the issue. Note this is new
since v9.12 as mv now warns about xattr ENOTSUP failures.
* tests/misc/xattr.sh: Avoid the issue (which would have
existed before v9.12).
Quoting is currently only enabled by default when writing to terminals,
but it is usually desired to constrain the output format for processing.
E.g. for processing by a shell you can now:
env --quoting-style=shell-escape | grep ^FOO
We use an option rather than reyling on $QUOTING_STYLE so
that $QUOTING_STYLE is restricted to be user config for terminals
and does not impact redirected processing.
Also it avoids an extra getenv() call if not outputting to tty.
* src/env.c (main): Parse --quoting-style and use that for the output
format for all outputs.
* src/printenv.c (main): Likewise.
* doc/coreutils.texi (env invocation): Describe --quoting-style.
(printenv invocation): Likewise.
* tests/env/env.sh: Reinstate testing of various output modes.
* tests/misc/printenv.sh: Likewise.
* NEWS: Mention the new feature. Also mention the change from
the previous commit not done there so that builders would have
a clean patch against the previous release.
csplit always creates the split holding the remainder of the input,
even when the patterns have already consumed everything and that
split is therefore empty. Two ways of reaching that state were not
exercised: a regexp with an offset that lands past the last line,
and --suppress-matched with a mix of a line number and a regexp
matching the last line. Only the all-line-numbers case was covered.
identified here: https://github.com/uutils/coreutils/pull/14565
* tests/csplit/csplit.sh: Test an offset reaching past end of input,
with and without -z.
* tests/csplit/csplit-suppress-matched.pl (mixed-1, mixed-2): Test a
line number combined with a regexp matching the last line.
Link: https://github.com/coreutils/coreutils/pull/353
In a long listing, -u and -c only choose which timestamp is shown;
the entries stay sorted by name, and time order is selected by -t
(or by asking for a format that is not long). This was only ever
exercised for short listings, so the rule went untested for every
long listing, including the ones requested with something other
than -l.
Identified here: https://github.com/uutils/coreutils/pull/14573
* tests/ls/ls-time.sh: Check that -u and -c keep name order for each
of -l, -g, -o, -n, --full-time and --dired, and that -t still gives
time order in a long listing.
Link: https://github.com/coreutils/coreutils/pull/354
* src/paste.c (usage): Simplify the description in --help to make it
easier to understand.
Fixes https://bugs.gnu.org/81859
Copyright-paperwork-exempt: Yes
parse_ls_color stores color_indicator[C_LINK].string as a pointer into
the color_buf buffer, then frees color_buf on the unparsable-value path.
The "ln=target" check after the parse loop still read that freed buffer,
e.g. with LS_COLORS='ln=target:x'. Skip it on the failure path, where
color output is disabled anyway and the buffer is gone.
* src/ls.c (parse_ls_color): Return after freeing color_buf on failure,
so the color_indicator[C_LINK] check only runs when the buffer is live.
* tests/ls/color-symlink-target.sh: New test.
* tests/local.mk (all_tests): Add it.
* NEWS: Mention the fix.
Link: https://github.com/coreutils/coreutils/pull/352
On GNU/Hurd, this test would get stuck in an infinite loop like the
following:
+ kill -s INFO 16535
+ sleep .01
+ kill -s INFO 16535
+ sleep .01
+ kill -s INFO 16535
+ sleep .01
[...]
The signals being sent that frequently seem to prevent 'dd' from
performing writes, since it outputs the following during that time:
0+0 records in
0+0 records out
0 bytes copied, 7161.11 s, 0.0 kB/s
0+0 records in
0+0 records out
0 bytes copied, 7161.88 s, 0.0 kB/s
[...]
* tests/dd/stats.sh: Use a larger interval between signals on GNU/Hurd.
`strace -e tee true` will run the tee command on cygwin,
rather than filtering for the tee syscall.
Reported by Bruno Haible on Cygwin 2.9.0
* init.cfg: Use -e trace=foo where foo might be a command,
so that strace probes will fail rather than invoke foo on Cygwin.
* tests/cat/cat-distinct-err.sh: Likewise.
* tests/cat/splice.sh: Likewise.
* tests/ls/no-cap.sh: Likewise.
* tests/misc/yes.sh: Likewise.
* tests/mv/atomic.sh: Likewise.
* tests/mv/atomic2.sh: Likewise.
* tests/nproc/nproc-quota.sh: Likewise.
* tests/pwd/pwd-long.sh: Likewise.
* tests/stty/stty.sh: Likewise.
* tests/tail/debug.sh: Likewise.
* tests/tail/inotify-only-regular.sh: Likewise.
* tests/sort/sort-NaN-infloop.sh: Up the timeout to about 240x
a "standard" system, as this was seen to timeout on a
mipsel VM using QEMU emulation.
Reported by Bruno Haible.
* tests/dd/no-allocate.sh: Expand memory restriction
(and allocated buffers) by 6MiB, to give more head-room
to the shell.
Reported and tested by Bruno Haible on OpenIndiana (from 2025).
On GNU/Hurd, file descriptor limits set via 'ulimit -n' seem to be
ineffective when set below the default:
$ (ulimit -n; ulimit -n 512; ulimit -n; \
tee $(seq 1024) < /dev/null; echo $?)
1024
512
tee: 1022: Too many open files
tee: 1023: Too many open files
tee: 1024: Too many open files
1
* tests/sort/sort-merge-fdlimit.sh: Don't fail if using 'ulimit -n' to
decrease the file descriptor limit is ineffective.
* tests/stat/stat-mount.sh: LeakSanitizer requires /proc access
for instrumented binaries, so ensure we're checking coreutils
binaries, rather than shell builtins.
* tests/df/skip-rootfs.sh: Test for /init which is listed as rootfs in
/proc/self/mountinfo, but tmpfs according to statfs.
Fixes https://bugs.gnu.org/65331
This adds an ELF note as per the UAPI Group's .note.dlopen spec:
$ readelf -p .note.dlopen src/sort
String dump of section '.note.dlopen':
[ 4] y
[ a] |@FDO
[ 10] [{"feature":"random-sort",
"description":"Support for random sorting",
"priority":"required",
"soname":["libcrypto.so.3"]}]
* bootstrap.conf: Add sd-dlopen.
* src/sort.c (DLOPEN_LIBCRYPTO && HAVE_OPENSSL_MD5): Add an ELF note
with SD_ELF_NOTE_DLOPEN() so that packagers can determine the libcrypto
dependency, needed when --random-sort is specified.
* NEWS: Mention the Build-related change.
Simplify cut_fields_mb_any() used with:
-w in a multi-byte locale, including UTF-8
--w=trimmed in any locale, including C
-f -d CHAR in non utf-8 multi-byte locale
-f -d CHAR in utf-8 with certain invalid CHARs
This is a net reducation of 24 lines,
and is performance neutral.
(cut_fields_mb_any): Use persistent first field and output state.
This allows using the existing field output helpers and
avoiding separate first field and line completion handling.
(cut_fields_bytesearch): Handle blanks explicitly.
(finish_current_line): Remove redundant condition.
There were no significant performance changes from the test cases
in commit a325c9978, but adding additional whitespace delimited cases
shows significant improvement. Below, with -f1, typical space delimited
input shows 4x more throughput, while the long line test shows 17x more.
Performance was tested with GCC 16 on an i7-5600U with
CFLAGS='-march=native -O3 -flto' with the following setup:
$ yes | head -n10M > sl.in
$ yes $(yes eeeaae | head -n10K | paste -s -d' ') | head -n10K > llw.in
$ yes $(yes eeeaae | head -n9 | paste -s -d' ') | head -n1M > asw.in
$ yes $(yes éééááé | head -n9 | paste -s -d$'\xe2\x80\x83') |
head -n1M > mbw.in
for type in sl llw asw mbw; do
cat $type.in >/dev/null;
for imp in '-before' '-after'; do
echo ============ "${imp:-base}" $type ==============;
for d in -w; do
fields='-f1 -f10 -f100'
test "$d" = "-b" && { fields='-b1 -b10 -b100'; d=''; }
test "$d" = "-c" && { fields='-c1 -c10 -c100'; d=''; }
for f in $fields; do
for loc in C.UTF-8; do
# Skip -b for UTF-8 as no different
test "$loc" = C.UTF-8 && echo "$f" | grep -q -- -b \
&& continue
# Skip multi-byte delimiter for C and not allowed
test "$loc" = C && test $(echo -n "$d" | wc -c) -ge 4 \
&& continue
bin=src/cut${imp}
LC_ALL=$loc $bin $f $d /dev/null 2>/dev/null &&
hyperfine --warmup=2 -m2 -M6 \
"LC_ALL=$loc $bin $f $d $type.in >/dev/null" ||
printf 'Benchmark 1: %s\n unsupported\n\n' \
"LC_ALL=$loc $bin $f $d $type.in >/dev/null"
done;
done;
done;
done;
done
After a little post-processing of the results, we get:
-- src/cut-before
| command | sl | llw | asw | mbw |
| --------------- | -------- | -------- | -------- | -------- |
| UTF8 -f1 -w | 96.2 ms | 1.916 s | 190.6 ms | 906.5 ms |
| UTF8 -f10 -w | 104.6 ms | 1.921 s | 187.7 ms | 845.6 ms |
| UTF8 -f100 -w | 104.4 ms | 1.916 s | 187.4 ms | 846.4 ms |
-- src/cut-after
| command | sl | llw | asw | mbw |
| --------------- | -------- | -------- | -------- | -------- |
| UTF8 -f1 -w | 88.7 ms | 112.4 ms | 43.5 ms | 146.3 ms |
| UTF8 -f10 -w | 95.2 ms | 114.4 ms | 193.5 ms | 848.4 ms |
| UTF8 -f100 -w | 94.8 ms | 130.6 ms | 194.4 ms | 848.1 ms |
* src/cut.c (scan_mb_line_end): A new function to skip
to EOL once we have exhausted the field selection.
(cut_fields_mb_any): Only skip after recognizing a field delimiter.
* tests/cut/cut.pl: Add test cases.
* tests/cut/mb-non-utf8.sh: Likewise.
* NEWS: Mention the improvement.
Adding a couple of inlining hints to the compiler
gives a 30% boost in raw character processing performance
in cut_fields_mb_any().
$ yes $(yes eeeaae | head -n10K | paste -s -d,) | head -n10K > ll.in
$ time LC_ALL=C.UTF-8 src/cut-before -f1 -w ll.in >/dev/null
real 0m2.279s
$ time LC_ALL=C.UTF-8 src/cut-after -f1 -w ll.in >/dev/null
real 0m1.619s
* src/cut.c (scan_mb_blank_field): Tag as inline.
(scan_mb_delim_field): Likewise.
Problem reported by Pádraig Brady (bug#81776#16).
* src/copy.c (handle_clone_fail): Be more cautious if
ioctl+FICLONE failed with EIO; do not fall back to read+write,
even if --reflink=auto is in effect.
* src/copy.h: Use errno values for syscall failures, and
negative values otherwise. That way, debugging output can
contain more info when clone or offload ops fail.
All uses changed.
Assume that XFS allocation groups can also fail with EDQUOT etc.,
so treat those failures like ENOSPC.
* src/copy.c (is_terminal_error): Remove; no longer used.
(handle_clone_fail): Remove DEST_DESC arg; no longer needed.
All uses changed. Simplify by acting as if is_terminal_error
always returns false. Change "!= 0" to "< 0" when either will do
for a syscall, as a matter of convention.
reflinking can fail due to failure to allocate metadata,
while a standard copy may succeed.
* src/copy.c (is_terminal_error): Remove ENOSPC from the list.
Fixes https://bugs.gnu.org/81776
It seems that NetBSD requires a fifo to be open, not merely opened,
to ensure a subsequent open proceeds. Immediately closing the opened
fifo caused intermittent failures on NetBSD 10.0. While that seems
like a bug in NetBSD, we can avoid the issue.
* tests/tail/pid-pipe.sh: Hold the fifo open until our writer
has opened it. This also allows specifying a single timeout
(of 12.7s), rather than 2 separate timeouts.
eq_mode(), which is used to determine if tcsetattr() was successful,
failed unnecessarily after speed setting under certain circumstances:
1. the kernel MAY round the speed to the nearest supported value,
so allow a variation of +/- 1/4 bit time per character from
what was originally requested.
2. There MAY be more than one internal binary encoding of a specific
speed (e.g. on Linux). On systems where that information
affects c_cflag, this may introduce inconsistencies;
filter out the known speed-related flags in c_cflag.
3. It is undefined in POSIX if cfsetispeed (..., 0) is retained or
normalized across tcsetattr () ... tcgetattr ().
If cfgetispeed () returns 0, compare the output speed instead.
* src/stty.c (eq_mode): Mask of some of the speed related bits
to avoid checking them for exact equality.
(speed_matches): A new helper to apply the more lenient matching
of requested and resulting speeds.
* NEWS: Mention the bug fix.
* src/printenv.c (main): Make sure quoting is disabled when we have
arguments and standard output is not a terminal.
* tests/misc/printenv.sh: Adjust single argument test cases. Add
multiple argument test cases.
* tests/misc/tty-quoting.sh: Add test cases to check if isatty is used.
* doc/coreutils.texi (printenvAlwaysQuoted): Remove macro.
(printenv invocation): Describe the adjusted quoting behavior.
(env invocation): Move the text from printenvAlwaysQuoted here since it
is no longer used for 'printenv'.
Collin Funk