When the access lists are rebuilt, only regular `<uuid>.sql` files are entities and unreadable or unparsable
files are skipped, like `DiskAccessStorage::reloadAllAndRebuildLists` does.
If no XML users config defines the default user and a storage that is not an XML users config follows
all of them, report that the default user may be defined there instead of reporting it as removed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A bare-relative `users.yaml` / `users.yml` may point to a file in the working
directory of the server; creating it in the config directory would silently
switch the active users storage on the next start.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`ConfigProcessor::getConfigMergeFiles` merges both `<name>.d` and `conf.d` of a users config,
sorted by full paths. For a custom users config like `a.xml`, `conf.d` is applied after `a.d`,
so `default-password.xml` written to `a.d` could be overridden by a `conf.d` fragment.
Choose whichever of the two directories is applied last.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The server checks a user-level OTP with every authentication method of the user, even `no_password` or
an empty password (`checkBasicAuthentication` returns `NeedSecondFactor` without it), so such a `default`
user cannot log in without credentials. `hasCredentials` now says so, so the installer no longer reports it
as accessible without a password and asks the `listen_host` question for it.
- Apply the ownership and permission fixups to every absolute XML users config the server loads
and to its merge directories (`<name>.d` and `conf.d`, like `ConfigProcessor`), not only to the
first users config and the `.d` directory of the one that defines the default user.
- Split `hasCredentials` (the default user cannot log in without credentials, used for the
`listen_host` question) from the new `hasPassword` (used for the "Password for the default user
is already specified" message and the `clickhouse-client --password` hint), so that e.g.
`ssl_certificates`, `ssh_keys` or `kerberos` are not reported as a password.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`ConfigProcessor::parseConfig` treats the extension case-insensitively, so a
relative `users.YAML` / `users.XML` users config must also be treated as the
stock one and resolved against the config directory.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`hasAuthentication` counts any configured method, including `no_password`, and is
right for deciding whether the installer may set up a password, but its result also
drove the "password is already specified" message, the `--password` hint for
`clickhouse-client`, and the prompt to accept connections from the network.
Split the two: `hasAuthentication` now only blocks the password prompt, while the new
`hasCredentials` (the flat fields and every entry of `auth_methods`, any of which
without credentials makes the user accessible without them) decides whether the
default user needs credentials. A default user configured with `no_password` gets
a dedicated message and no `--password` hint or `listen_host` prompt.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When the first users config was missing, `clickhouse install` created it but skipped the scan
for the default user, so existing fragments in the companion `users.d` directory (e.g.
`<default remove="1" />`, or authentication for `default`) were ignored and the installer
still asked for a password. Run the same `ConfigProcessor`-based scan after creating the file.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Like the stock `users.xml`, a bare relative `users.yaml` or `users.yml` as the first users config
is now resolved against the config directory, so `clickhouse install` seeds it from the YAML template
instead of treating it as resolved against the working directory of the server.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- `hasAuthentication` now counts an authentication method by its presence, like
`UsersConfigParser`: any of `no_password`, the hashed password fields, `ldap`,
`kerberos`, `ssl_certificates`, `ssh_keys`, `http_authentication`,
`auth_methods` or `time_based_one_time_password`, or a non-empty `password`.
The installer's override replaces only a flat `password` with
`password_sha256_hex`, so prompting in any other case would make the merged
users config invalid.
- When the `default` user is removed from the first XML users config and defined
in a later one, write `default-password.xml` into the `.d` directory of the
config that defines it, instead of creating a new `default` user in the first
one that would shadow the later definition.
- Compare the users config extension case-insensitively, like
`ConfigProcessor::parseConfig`, when choosing between XML and YAML templates.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Same as `DiskAccessStorage::readLists`, a list file that cannot be read means the lists are rebuilt from the `.sql` files.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`isKnownWithoutDefaultUserDiskAccessStorage` now follows `DiskAccessStorage` startup:
if there is no `need_rebuild_lists.mark` and all `*.list` files can be read, only
`users.list` is consulted and unlisted `<id>.sql` files are ignored, as the server does;
otherwise all `<id>.sql` files are scanned, as in rebuild mode.
A later XML users config with a relative path that does not exist in the config directory
is resolved by the server against its working directory, which is not known here.
Previously the scan skipped it and could conclude that the default user was removed;
now it stops there and does not set up a password, reporting that the default user may be
defined in that file.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`AccessControl` resolves a relative users config path against the working directory
of the server when the file does not exist in the config directory. Rewriting it to the
config directory and creating it there could silently switch the server to a new users
config. Only the stock `users.xml` keeps being created in the config directory; for any
other such path the installer neither creates the users config nor sets up a password.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fixes the `catch_all` style check: the `catch (...)` in
`isKnownWithoutDefaultUserDiskAccessStorage` silently swallowed the exception.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A `DiskAccessStorage` writes empty `*.list` files after the first server start, and may hold only roles or settings profiles, so the existence of `*.list` or `*.sql` files does not mean that it defines the default user. `clickhouse install` now reads the names in `users.list` and deserializes the `<id>.sql` files (the two sources `DiskAccessStorage` loads users from), and treats the storage as shadowing the XML users config only if one of them defines `default`. An unreadable or corrupted directory still counts as possibly shadowing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- When `access_control_path` or a `local_directory` precedes every XML users config, the installer
still creates the users config if it is missing (the server reads it), and only skips the password
setup for the default user. A local directory storage that has no access entities (no `<id>.sql`
or `*.list` files, or the directory does not exist) cannot define the default user, so it does not
shadow the XML users config at all. Errors while inspecting the directory are treated as "not
known empty" instead of being thrown.
- A missing users config with a `.yaml` / `.yml` extension is created from the embedded
`users.yaml.example`, because `ConfigProcessor` parses the file according to its extension.
- A missing first users config with a relative path is created in the config directory again, as
before this PR: once it exists there, the server resolves the relative path to it. Previously
`clickhouse install` did not recreate `users.xml` for the stock `config.xml`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`MultipleAccessStorage::find` looks up the default user across all access storages in order,
so a `local_directory`, `replicated` or `ldap` storage (or `access_control_path`) placed before
the XML users config that defines `default` may hold the effective default user. In that case
`clickhouse install` now neither writes `default-password.xml` nor reports the user as removed,
and prints which storage may define it instead.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`clickhouse install` now treats the default user as secured if it has any authentication method
that `UsersConfigParser` accepts besides an empty password or `no_password`, including
`password_scram_sha256_hex` and entries of `auth_methods`.
When the active users config or its `.d` directory lives outside the config directory,
it is chowned to the server user before its permissions are tightened.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`AccessControl` prepends the config directory to a relative users config path
only if the file exists there; otherwise the server resolves it against its
working directory. The installer now does the same, and if the first XML users
config remains a bare relative path it neither creates nor modifies it and does
not set up a password for the default user.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`AccessControl` reads users from the top-level `users_config` and from every XML
entry in `user_directories`, and `MultipleAccessStorage::find` falls through to
later storages when an earlier one does not define a user. Collect all XML users
configs in the server's order, keep the first one as the write target, and treat
the default user as removed only if none of them defines it; the password check
uses the first definition found.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Recognize every `users_xml` / `users.xml` / `users_config` entry under
`<user_directories>` (including indexed entries such as `users_xml[1]`),
the same way as `AccessControl::addStoragesFromUserDirectoriesConfig`,
and inspect the first one, instead of only `user_directories.users_xml.path`.
Otherwise a supported layout like `<users_config><path>...</path></users_config>`
made `clickhouse install` skip the password setup even though `default`
still existed with an empty password.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Address review: the installer always assumed `<config-path>/users.xml`, but the
server takes the users config from `users_config` or
`user_directories.users_xml.path` (relative to the config directory), falls
back to the main config when neither `users_config` nor `user_directories`
is set, and reads no XML users config when `user_directories` has no
`users_xml`. Mirror `AccessControl::setupFromMainConfig`, derive the
matching `.d` directory, and skip the password setup when there is no XML
users config.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Move the `shellQuote` helper out of `clickhouse install` into
`Common/shellQuote.h` so it can be reused. Apply it to two more
shell-command construction sites:
- `formatWithSudo` on FreeBSD wraps the inner command in `su -c '...'`.
All current callers pass static literals, but an embedded single
quote would break the wrapper. Quote the command via `shellQuote`
so future callers can pass arbitrary strings safely.
- `clickhouse git-import` interpolates a commit hash parsed from
`git log --pretty=%H` into `git show ... {hash}` and runs it via
`/bin/sh -c`. `%H` is hex-only today, but a hand-crafted hash list
or a future format change should not be able to inject shell
syntax through the input.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`changeOwnership`, `createGroup`, `createUser`, and the `setcap`
invocation in `clickhouse install` interpolated CLI-controlled values
(`--user`, `--group`, `--log-path`, `--data-path`, `--pid-path`,
`--config-path`, `--binary-path`, `--prefix`) directly into shell
command strings passed to `/bin/sh -c`. The `shellQuote` helper
already used by the sudo invocation in the same file was not applied
elsewhere. Wrap every interpolated value in `shellQuote` so shell
metacharacters in CLI arguments cannot be interpreted by the shell.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`--group` for `clickhouse start` / `restart` had a default value of
`clickhouse` purely for documentation. The launcher applied it
unconditionally on the `sudo` path, so
clickhouse start --user alice
was silently expanded to `sudo --preserve-env -u alice -g clickhouse ...`.
That fails when `alice` is not a member of the `clickhouse` group,
which is a behavior regression for users who only ever passed `--user`.
Clear `group` in `mainEntryClickHouseStart` and `mainEntryClickHouseRestart`
when `options["group"].defaulted()` is true, so the default is shown in
`--help` but not applied to the spawned server. The `no-sudo` path then
runs `clickhouse su user:`, where `setUserAndGroup` falls back to the
user's primary group.
The `--group` option for `clickhouse start` / `restart` was only honored on
the `--no-sudo` branch (`clickhouse su user:group ...`). On the default path
the command was built as `sudo --preserve-env -u <user> ...`, with no `-g`,
so the group was silently ignored.
Pass `-g <group>` to `sudo` (shell-quoted) when `group` is non-empty so the
advertised CLI option takes effect on both code paths.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous attempt to harden the `--no-sudo` and `sudo` paths in
`clickhouse start` wrapped interpolated `user`, `group`, and binary
paths in single quotes only. This is insufficient: a value containing
a single quote (e.g. `bob';rm -rf /;'`) still breaks out because
single quotes do not nest.
Introduce a POSIX-compliant `shellQuote` helper in `Install.cpp` and
apply it to every user-controlled field interpolated into the `su` /
`sudo` command strings: `executable`, `config`, `pid_file`,
`binary`, `user`, and `group`. The helper replaces any `'` inside
the value with `'\''` before re-wrapping, which is the standard POSIX
idiom and safe against arbitrary byte content.
While CLI arguments to `clickhouse start` are not typically attacker
controlled (the caller already runs the binary), defense-in-depth is
worthwhile: these commands go through `/bin/sh -c` via
`ShellCommand::execute`, and unquoted interpolation has been flagged
as a hazard in code review.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- `printHelp` now takes an `std::ostream &` parameter: explicit `help`
writes to stdout, while `printHelpOnError` (mistyped commands) writes
to stderr, preserving the error-output contract for scripts.
- Shell-quote binary path and user:group in the `--no-sudo` branch of
`clickhouse start` to prevent command injection via crafted CLI args.
- Make the keeper availability check in the help return codes test more
precise: match `clickhouse keeper [args]` exactly instead of bare
`keeper` which could false-match `keeper-converter` etc.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The original PR #96025 was reverted (#98038) because:
1. The install completion message showed wrong relative paths like
`var/run/clickhouse-server` instead of absolute `/var/run/clickhouse-server`.
This happened because `options.contains()` returns true even for options
with default values in boost::program_options. Fix: use `defaulted()`
to only include options that the user explicitly specified.
2. The install completion message did not suggest `sudo` for the
start/restart command. This happened because `getuid() != 0` is false
when install runs as root via sudo, but the user returns to their
unprivileged shell afterward. Fix: always suggest sudo (matching the
original behavior).
Also address review feedback:
- Fix `clickhouse-client` help: remove misleading "initial table definition"
(a `clickhouse-local` concept) from usage line
- Fix typo: "as command line tool" → "as a command line tool"
- Fix `clickhouse-local` help grammar: "After you can execute" → "After that,
you can execute", "in first query" → "in the first query"
- Make keeper `--help` test robust for builds without `ENABLE_CLICKHOUSE_KEEPER`
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add `static` to file-local functions and forward declarations
for cross-TU functions to satisfy `-Wmissing-prototypes`.
This ensures all functions have visible prototypes before their
definitions, improving code quality and catching accidental
linkage issues.
Reference: https://github.com/ClickHouse/ClickHouse/issues/76944
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove "CMake doesn't recognize changes in #embed-ed files" comments
from all files, since this PR now tracks those dependencies.
- Add conditional `OBJECT_DEPENDS` for `FrequencyHolder.cpp` NLP blobs
(guarded by `if (EXISTS ...)` for when the submodule is not checked out).
- Add `OBJECT_DEPENDS` for `attachInformationSchemaTables.cpp` which was
also missing dependency tracking for its embedded SQL files.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extract `ErrnoException` into `Common/ErrnoException.h` to reduce the
include footprint of `Exception.h`. This removes `<base/errnoToString.h>`
and `<cerrno>` from `Exception.h`, which is included by ~95% of all
translation units.
All files referencing `ErrnoException` now include the new header
directly. Files that used `errnoToString` without `ErrnoException`
got an explicit `<base/errnoToString.h>` include.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add --group option to start and restart commands
- Use clickhouse su instead of sudo when --no-sudo is set
- Pass --prefix, --pid-path, --config-path, --binary-path, --user, --group
from install to the suggested start/restart command
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
