mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2026-10-28 16:18:23 +08:00
Move the full page body of each table-function reference page into the
FunctionDocumentation attached at registerFunction time, and convert the
pages to be regenerated from system.table_functions (like table engines,
data types and formats already are).
55 of the 58 table-function pages are now autogenerated. The description of
each function is set to its reference-page body, resolved to the correct
registerFunction<Class>({...}) site by the function name (Class::name),
including the object-storage family (s3/gcs/azureBlobStorage/hdfs/iceberg/
deltaLake/hudi/paimon and their *Cluster variants, via Definitions) and the
templated/conditional-name functions (values, zeros, generateSeries). The
string-name overload functions (remote, cluster) and the doc-variable form
(format) are handled at their own sites. A page->name alias maps the
generate() page to the generateRandom function.
timeSeriesData is intentionally left hand-written: it is a registerAlias of
timeSeriesSamples, so the two share a single description, but their pages
differ (each uses its own name in examples) and an alias cannot carry its
own doc.
Verified against a from-master build: every function's
system.table_functions.description matches its master page body
byte-for-byte, and each regenerated page equals origin/master (body
identical, only the AUTOGENERATED markers added) with zero broken links.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
111 lines
5.0 KiB
Plaintext
111 lines
5.0 KiB
Plaintext
---
|
|
description: 'The `executable` table function creates a table based on the output
|
|
of a user-defined function (UDF) that you define in a script that outputs rows to
|
|
**stdout**.'
|
|
keywords: ['udf', 'user defined function', 'clickhouse', 'executable', 'table', 'function']
|
|
sidebarTitle: 'executable'
|
|
slug: /engines/table-functions/executable
|
|
title: 'executable'
|
|
doc_type: 'reference'
|
|
---
|
|
|
|
{/*AUTOGENERATED_START*/}
|
|
The `executable` table function creates a table based on the output of a user-defined function (UDF) that you define in a script that outputs rows to **stdout**. The executable script is stored in the `users_scripts` directory and can read data from any source. Make sure your ClickHouse server has all the required packages to run the executable script. For example, if it is a Python script, ensure that the server has the necessary Python packages installed.
|
|
|
|
You can optionally include one or more input queries that stream their results to **stdin** for the script to read.
|
|
|
|
<Note>
|
|
A key advantage between ordinary UDF functions and the `executable` table function and `Executable` table engine is that ordinary UDF functions cannot change the row count. For example, if the input is 100 rows, then the result must return 100 rows. When using the `executable` table function or `Executable` table engine, your script can make any data transformations you want, including complex aggregations.
|
|
</Note>
|
|
|
|
## Syntax {#syntax}
|
|
|
|
The `executable` table function requires three parameters and accepts an optional list of input queries:
|
|
|
|
```sql
|
|
executable(script_name, format, structure, [input_query...] [,SETTINGS ...])
|
|
```
|
|
|
|
- `script_name`: the file name of the script. saved in the `user_scripts` folder (the default folder of the `user_scripts_path` setting)
|
|
- `format`: the format of the generated table
|
|
- `structure`: the table schema of the generated table
|
|
- `input_query`: an optional query (or collection or queries) whose results are passed to the script via **stdin**
|
|
|
|
<Note>
|
|
If you are going to invoke the same script repeatedly with the same input queries, consider using the [`Executable` table engine](/reference/engines/table-engines/special/executable).
|
|
</Note>
|
|
|
|
The following Python script is named `generate_random.py` and is saved in the `user_scripts` folder. It reads in a number `i` and prints `i` random strings, with each string preceded by a number that is separated by a tab:
|
|
|
|
```python
|
|
#!/usr/local/bin/python3.9
|
|
|
|
import sys
|
|
import string
|
|
import random
|
|
|
|
def main():
|
|
|
|
# Read input value
|
|
for number in sys.stdin:
|
|
i = int(number)
|
|
|
|
# Generate some random rows
|
|
for id in range(0, i):
|
|
letters = string.ascii_letters
|
|
random_string = ''.join(random.choices(letters ,k=10))
|
|
print(str(id) + '\t' + random_string + '\n', end='')
|
|
|
|
# Flush results to stdout
|
|
sys.stdout.flush()
|
|
|
|
if __name__ == "__main__":
|
|
main()
|
|
```
|
|
|
|
Let's invoke the script and have it generate 10 random strings:
|
|
|
|
```sql
|
|
SELECT * FROM executable('generate_random.py', TabSeparated, 'id UInt32, random String', (SELECT 10))
|
|
```
|
|
|
|
The response looks like:
|
|
|
|
```response
|
|
┌─id─┬─random─────┐
|
|
│ 0 │ xheXXCiSkH │
|
|
│ 1 │ AqxvHAoTrl │
|
|
│ 2 │ JYvPCEbIkY │
|
|
│ 3 │ sWgnqJwGRm │
|
|
│ 4 │ fTZGrjcLon │
|
|
│ 5 │ ZQINGktPnd │
|
|
│ 6 │ YFSvGGoezb │
|
|
│ 7 │ QyMJJZOOia │
|
|
│ 8 │ NfiyDDhmcI │
|
|
│ 9 │ REJRdJpWrg │
|
|
└────┴────────────┘
|
|
```
|
|
|
|
## Settings {#settings}
|
|
|
|
- `send_chunk_header` - controls whether to send row count before sending a chunk of data to process. Default value is `false`.
|
|
- `pool_size` — Size of pool. If 0 is specified as `pool_size` then there is no pool size restrictions. Default value is `16`.
|
|
- `max_command_execution_time` — Maximum executable script command execution time for processing block of data. Specified in seconds. Default value is 10.
|
|
- `command_termination_timeout` — executable script should contain main read-write loop. After table function is destroyed, pipe is closed, and executable file will have `command_termination_timeout` seconds to shutdown, before ClickHouse will send SIGTERM signal to child process. Specified in seconds. Default value is 10.
|
|
- `command_read_timeout` - timeout for reading data from command stdout in milliseconds. Default value 10000.
|
|
- `command_write_timeout` - timeout for writing data to command stdin in milliseconds. Default value 10000.
|
|
|
|
## Passing Query Results to a Script {#passing-query-results-to-a-script}
|
|
|
|
Be sure to check out the example in the `Executable` table engine on [how to pass query results to a script](/reference/engines/table-engines/special/executable#passing-query-results-to-a-script). Here is how you execute the same script in that example using the `executable` table function:
|
|
|
|
```sql
|
|
SELECT * FROM executable(
|
|
'sentiment.py',
|
|
TabSeparated,
|
|
'id UInt64, sentiment Float32',
|
|
(SELECT id, comment FROM hackernews WHERE id > 0 AND comment != '' LIMIT 20)
|
|
);
|
|
```
|
|
{/*AUTOGENERATED_END*/}
|