sql: five more behaviours are main's again, until somebody decides otherwise

None of them was asked for, and main works in each.

- A query that fails with nobody listening does not end the process: execute() alone, a late await or
  catch(), finally(f) with f fine, an unawaited statement on a transaction. What stays: an error that
  the caller's own then / catch / finally handler throws was swallowed and is reported.
- The pool is started until `max` is reached when a caller finds no connection ready, as the docs say.
  What stays: a pool that lost connections grows back, one dial per caller that no dial covers.
- A caller whose dial is refused waits for a connection that is reserved or in a transaction, and is
  rejected only when the pool has none left.
- raw() on PostgreSQL returns the bytes it returned: binary for the eleven types that were asked for in
  binary, text for the rest. A request records what its Bind asked for, so a late raw() or values()
  reads each cell as it came.
- sql.array(values, "JSON" | "JSONB"): a nested array is a dimension and null is the JSON value null.
This commit is contained in:
Jarred Sumner committed 2026-10-16 18:49:06 +00:00
1 parent 6250dc902e
commit 43ad527e44
20 files changed
+554 -279

No files matched your search

+19 -12
View File
@@ -414,11 +414,11 @@ const rows = await sql`SELECT * FROM users`.raw();
console.log(rows); // [[Buffer, Buffer], [Buffer, Buffer], [Buffer, Buffer]]
```
- On PostgreSQL, each `Buffer` holds the text form of the value, such as `1`, `t` or `\x0102`.
- On PostgreSQL, each `Buffer` holds the bytes the server sent. A prepared query gets `bool`, `int4`, `float4`, `float8`, `numeric`, `bytea`, `time`, `timestamp`, `timestamptz`, `int4[]` and `float4[]` in PostgreSQL's binary format, so a `bytea` column is its bytes. Every other type is text. All types are text on the first run of a query without parameters, with `.simple()`, and with `prepare: false`.
- On MySQL, each `Buffer` holds the bytes the server sent. They are binary for a prepared query and text for `.simple()`.
- SQLite returns `Uint8Array` objects.
Call `.raw()` before the query starts, that is before `await`, `.then()` or `.execute()`. After that, a prepared query on PostgreSQL ignores it. On SQLite, `.execute()` has already run the query when it returns, so `.values()` has no effect after it either.
Call `.raw()` before the query starts, that is before `await`, `.then()` or `.execute()`. After that, Bun may already have asked PostgreSQL for the formats that it decodes. Then `float4`, `numeric` and `time` arrive as text, and `int2[]` and `float8[]` in binary. On SQLite, `.execute()` has already run the query when it returns, so `.raw()` and `.values()` have no effect after it.
### Result metadata
@@ -601,7 +601,14 @@ await sql`SELECT ${sql.array([1, 2, 3], "INT")} AS ids`;
// Sends: SELECT $1::INT[] AS ids
```
A `null` element is SQL `NULL`, and nested arrays make a multi-dimensional array. Pass `"JSON"` or `"JSONB"` for a `json[]` or `jsonb[]` column, so that Bun encodes each element as JSON.
A `null` element is SQL `NULL`, and nested arrays make a multi-dimensional array.
Pass `"JSON"` or `"JSONB"` for a `json[]` or `jsonb[]` column, so that Bun encodes each element as JSON. There, `null` is the JSON value `null`, and `undefined` is SQL `NULL`. A nested array is still a dimension. To store a JSON array as one element, wrap it in [`sql.json`](#sql-json):
```ts
await sql`INSERT INTO events (payloads) VALUES (${sql.array([{ a: 1 }, null, sql.json([1, 2])], "JSONB")})`;
// three elements: {"a": 1}, null and [1, 2]
```
<Note>`sql.array` is PostgreSQL-only.</Note>
@@ -711,7 +718,7 @@ What `cancel()` does depends on how far the query got:
### Unhandled rejections
A query is a promise. When it fails and nothing handles the rejection, Bun reports an unhandled rejection, which ends the process unless you listen for [`unhandledRejection`](https://nodejs.org/api/process.html#event-unhandledrejection). Add a `.catch()` to a query that you start and do not await:
`.then()` and `.catch()` return a promise like any other. When it rejects and nothing handles the rejection, Bun reports an unhandled rejection, which ends the process unless you listen for [`unhandledRejection`](https://nodejs.org/api/process.html#event-unhandledrejection). So does an error that your `.finally()` callback throws. A query that you only start, with `.execute()` or `.finally()`, is the exception: once it has a connection, its own failure is not reported, and the error is lost. Add a `.catch()` to a query that you start and do not await:
```ts
sql`UPDATE stats SET hits = hits + 1`.catch(err => console.error(err));
@@ -879,7 +886,7 @@ DATABASE_URL=postgres://user:pass@localhost:5432/db bun --sql-preconnect index.j
bun --sql-preconnect --hot index.js
```
The `--sql-preconnect` flag opens one connection of the default client, `Bun.sql`, using your configured environment variables. It also works when `DATABASE_URL` is a MySQL or MariaDB URL. If the connection fails, Bun handles the error without crashing your application.
The `--sql-preconnect` flag connects the default client, `Bun.sql`, at startup using your configured environment variables. It also works when `DATABASE_URL` is a MySQL or MariaDB URL. If the connection fails, Bun handles the error without crashing your application.
---
@@ -1354,26 +1361,26 @@ const sql = new SQL({
});
```
Bun opens connections on demand. It doesn't open one until you run a query.
Bun doesn't open a connection until you run a query.
```ts
const sql = new SQL(); // no connection yet
await sql`...`; // opens the first connection
await sql`...`; // reuses it
await sql`...`; // pool is started until max is reached (if possible), first available connection is used
await sql`...`; // previous connection is reused
// two queries at the same time: Bun opens a second connection
// two connections are used now at the same time
await Promise.all([
sql`INSERT INTO users ${sql({ name: "Alice" })}`,
sql`UPDATE users SET name = ${user.name} WHERE id = ${user.id}`,
]);
```
- Bun opens a connection when a query finds no idle one, until the pool has `max` connections. After that, queries share the open connections.
- Bun starts the pool when a caller finds no connection ready: at the first query, and again after every connection has closed, for example at `idleTimeout`. It opens connections until the pool has `max`. Queries share the open connections.
- `sql.begin` and `sql.reserve` need a connection of their own, so they wait for an idle one.
- Bun serves the callers that wait for a connection oldest first.
- A pool that loses connections opens new ones for the next queries.
- When Bun cannot open a new connection, queries run on the connections that are open. A caller that none of them can take rejects with the error of that attempt: `sql.reserve`, `sql.begin`, or a query while every connection is reserved.
- A pool that lost some of its connections opens new ones when queries have to wait.
- When Bun cannot open a new connection, callers wait for the connections that are open. They reject with the error of that attempt only when the pool has no connection left.
- Bun closes a connection that goes back to the pool inside a transaction, and the server rolls that transaction back. That covers a reserved connection that you release after a manual `BEGIN`, and a query that leaves a transaction open or failed, such as `SELECT 1; BEGIN`. Queries of other callers that Bun already sent behind such a query are rolled back with it. With `max: 1`, where a manual `BEGIN` is allowed, a query can leave its transaction open.
### Hot reloading
+20 -10
View File
@@ -668,8 +668,7 @@ declare module "bun" {
connection?: Record<string, string | boolean | number> | undefined;
/**
* Maximum number of connections in the pool. Bun opens them on demand:
* one when a query finds no idle connection, until the pool has `max`.
* Maximum number of connections in the pool
* @default 10
*/
max?: number | undefined;
@@ -904,9 +903,13 @@ declare module "bun" {
* {@link ResultMetadata}.
*
* A query is lazy. It starts in the call to `then()`, `catch()`,
* `finally()` or {@link execute}, and `await` calls `then()`. Like any
* promise, a query that fails with nothing to handle the rejection is
* reported as an unhandled rejection.
* `finally()` or {@link execute}, and `await` calls `then()`.
*
* What `then()` and `catch()` return is a promise like any other: when it
* rejects with nothing to handle the rejection, that is reported as an
* unhandled rejection. So is an error that a `finally()` callback throws.
* The failure of a query that was only started, with `execute()` or
* `finally()`, is not reported once it has a connection.
*
* Bun reads the parameters when it sends the query. Leave a `Buffer`,
* `TypedArray`, `Date` or object that you passed alone until the query
@@ -993,11 +996,16 @@ declare module "bun" {
/**
* Returns rows as arrays of `Buffer` objects instead of objects.
*
* PostgreSQL: the text form of each value. MySQL: the bytes the server
* sent, binary for a prepared query. SQLite: `Uint8Array` objects.
* PostgreSQL and MySQL: the bytes the server sent. On PostgreSQL a
* prepared query gets `bool`, `int4`, `float4`, `float8`, `numeric`,
* `bytea`, `time`, `timestamp`, `timestamptz`, `int4[]` and `float4[]` in
* binary and every other type as text; all types are text on the first run
* of a query without parameters, with `.simple()` and with
* `prepare: false`. On MySQL a prepared query is binary. SQLite:
* `Uint8Array` objects.
*
* Call it before the query starts. After that, SQLite and a prepared query
* on PostgreSQL ignore it.
* Call it before the query starts. After that, SQLite ignores it, and on
* PostgreSQL some types can arrive in the other format.
*/
raw(): Query<T>;
@@ -1314,7 +1322,9 @@ declare module "bun" {
* cast and the server infers the type from the column, the compared value
* or a cast in the query. Pass it when the query gives no context: a bare
* `SELECT ${sql.array([1])}` returns the text `{"1"}`. Pass `"JSON"` or
* `"JSONB"` to encode each element as JSON.
* `"JSONB"` to encode each element as JSON: there `null` is the JSON value
* `null`, `undefined` is SQL `NULL`, and `sql.json([1, 2])` stores a JSON
* array as one element.
* @returns The array parameter, ready to interpolate into a query
*
* @example
+25 -34
View File
@@ -647,22 +647,24 @@ abstract class BaseSQLAdapter<ConnectionHandle extends NativeConnection, QueryHa
return connecting;
}
/// For the callers that wait. If no connection is ready, the pool is started: it dials until `max` is reached.
/// Otherwise it grows by one dial for each waiting caller that no dial in flight covers.
#dialForQueued() {
const wanted = this.queue.length;
if (wanted === 0) {
if (this.queue.length === 0) {
return;
}
const start = this.readyConnections.size === 0;
const connections = this.connections;
let connecting = this.#connecting();
while (connecting < wanted && connections.length < this.connectionInfo.max && this.#closing === null) {
if (this.#adopt()) {
// it has served the queue and dialed for the rest
return;
while (
connections.length < this.connectionInfo.max &&
this.#closing === null &&
(start || this.#connecting() < this.queue.length)
) {
if (!this.#adopt()) {
const connection = new PooledConnection(this);
connections.push(connection);
connection.dial();
}
const connection = new PooledConnection(this);
connections.push(connection);
connecting++;
connection.dial();
}
}
@@ -724,17 +726,16 @@ abstract class BaseSQLAdapter<ConnectionHandle extends NativeConnection, QueryHa
}
/// Serves the oldest caller that can be served now. It is booked before it is called: it can re-enter the pool.
/// `size`: how many connections the pool counts on.
#serveNext(size: number): boolean {
#serveNext(): boolean {
const queue = this.queue;
let reservationsAhead = 0;
for (let i = 0; i < queue.length; i++) {
const waiter = queue[i];
if (typeof waiter === "function") {
// An even share of the demand over `size` connections: what is above it gets a new connection.
// An even share of the demand over `max` connections.
// A reservation ahead gets a dial that is on its way, or one connection has to drain for it.
const connection = this.#leastBusy(
Math.ceil((queue.length + this.totalQueries) / size),
Math.ceil((queue.length + this.totalQueries) / this.connectionInfo.max),
reservationsAhead && Math.max(reservationsAhead - this.#connecting(), 0),
);
if (connection === null) {
@@ -762,8 +763,7 @@ abstract class BaseSQLAdapter<ConnectionHandle extends NativeConnection, QueryHa
}
#serve() {
while (this.#serveNext(this.connectionInfo.max));
this.#dialForQueued();
while (this.#serveNext());
}
/// `prepared`: a distributed transaction ended with its PREPARE.
@@ -828,32 +828,22 @@ abstract class BaseSQLAdapter<ConnectionHandle extends NativeConnection, QueryHa
return;
}
} else if (waiting > 0) {
// They never used `connection`. After one that worked they get a new one.
// They never used `connection`. If it was established, they get a new one. A failed dial rejects them only
// when the pool has no connection left.
if (established) {
this.#serve();
} else {
// The callers make do with the connections that are up or on their way: any of them takes every query,
// and a reservation once it is idle. With none, the oldest caller has its answer, and the next one is
// dialed for while the server is known to answer.
const connecting = this.#connecting();
while (this.#serveNext(this.readyConnections.size + connecting));
if (this.readyConnections.size + connecting === 0) {
this.#failQueued(err, 1);
if (this.isConnected()) {
this.#dialForQueued();
}
}
this.#dialForQueued();
}
if (connections.length === 0) {
this.#failQueued(err, this.queue.length);
this.#failQueued(err);
}
this.#closeIfDrained();
}
this.runUserCallback(this.connectionInfo.onclose, err);
}
#failQueued(err: Error, oldest: number) {
const failed = this.queue.splice(0, oldest);
#failQueued(err: Error) {
const failed = this.queue.splice(0);
for (let i = 0; i < failed.length; i++) {
const waiter = failed[i];
(typeof waiter === "function" ? waiter : waiter.onConnected)(err, null);
@@ -895,7 +885,7 @@ abstract class BaseSQLAdapter<ConnectionHandle extends NativeConnection, QueryHa
clearTimeout(timers[i]);
}
timers.length = 0;
this.#failQueued(this.connectionClosedError(), this.queue.length);
this.#failQueued(this.connectionClosedError());
const connections = this.connections.slice();
for (let i = 0; i < connections.length; i++) {
connections[i].destroy();
@@ -939,6 +929,7 @@ abstract class BaseSQLAdapter<ConnectionHandle extends NativeConnection, QueryHa
this.asked = true;
this.queue.push(reserved ? new Reservation(onConnected) : onConnected);
this.#serve();
this.#dialForQueued();
}
}
+3 -3
View File
@@ -138,9 +138,10 @@ function arrayLiteral(values: ArrayLike<unknown>, kind: ArrayKind): string {
for (let i = 0, length = values.length; i < length; i++) {
if (i > 0) literal += delimiter;
const value = values[i];
if (value == null) literal += "NULL";
else if (kind === ArrayKind.json) literal += jsonElement(value);
// null is a JSON value.
if (value === undefined || (value === null && kind !== ArrayKind.json)) literal += "NULL";
else if ($isArray(value)) literal += arrayLiteral(value, kind);
else if (kind === ArrayKind.json) literal += jsonElement(value);
else if (typeof value !== "object") literal += quoteElement(String(value));
else if (value instanceof Date) literal += quoteElement(value.toISOString());
else if (ArrayBuffer.isView(value)) literal += hex(Buffer.from(value.buffer, value.byteOffset, value.byteLength));
@@ -244,7 +245,6 @@ class PostgresAdapter extends BaseSQLAdapter<
readonly codePrefix = "ERR_POSTGRES_";
protected readonly commitPrepared = "COMMIT PREPARED";
protected readonly rollbackPrepared = "ROLLBACK PREPARED";
readonly rawChangesFormat = true;
array(values: any[], typeNameOrID?: number | ArrayType): SQLArrayParameter {
const arrayType = getArrayType(typeNameOrID);
+15 -19
View File
@@ -73,18 +73,8 @@ class Query<T, Handle extends BaseQueryHandle<any>> extends PublicPromise<T> {
}
#setMode(mode: SQLQueryResultMode) {
const flags = this[_flags];
if (this[_queryStatus] & SQLQueryStatus.executed) {
if (
this[_adapter].rawChangesFormat &&
!(flags & SQLQueryFlags.simple) &&
(mode === SQLQueryResultMode.raw || ((flags >> resultModeShift) & 3) === SQLQueryResultMode.raw)
) {
return this;
}
this[_handle]?.setMode(mode);
}
this[_flags] = (flags & ~(3 << resultModeShift)) | (mode << resultModeShift);
this[_flags] = (this[_flags] & ~(3 << resultModeShift)) | (mode << resultModeShift);
this[_handle]?.setMode(mode);
return this;
}
@@ -260,9 +250,7 @@ class Query<T, Handle extends BaseQueryHandle<any>> extends PublicPromise<T> {
}
simple() {
if (!(this[_queryStatus] & SQLQueryStatus.executed)) {
this[_flags] |= SQLQueryFlags.simple;
}
this[_flags] |= SQLQueryFlags.simple;
return this;
}
@@ -309,15 +297,23 @@ class Query<T, Handle extends BaseQueryHandle<any>> extends PublicPromise<T> {
return super.catch.$apply(this, arguments);
}
finally(_onfinally?: (() => void) | undefined | null) {
finally(onFinally?: (() => void) | undefined | null) {
this.#runTagged();
return super.finally.$apply(this, arguments);
if (!$isCallable(onFinally)) {
return super.finally.$apply(this, arguments);
}
// Once `onFinally` and what it returned are through, the failure of the query passes on to `result`: as with
// execute(), nobody has to handle it. What `onFinally` throws is the caller's own, and the mark is not reached.
const result = super.finally(() => PublicPromise.$resolve(onFinally()).$then(() => $pokePromiseAsHandled(result)));
return result;
}
}
/// Calls `callback` in a microtask once the query has settled, before the reactions of the caller. It is not a
/// reaction of the query: that would count as a handler, and a failure that nobody handles would not be reported.
/// Calls `callback` in a microtask once the query has settled, before the reactions of the caller.
function onQuerySettled(query: Query<any, any>, callback: () => void) {
// A query that is only started, sql`...`.execute(), does not end the process when it fails. What the caller makes
// of it with then() or catch() is a promise like any other.
$pokePromiseAsHandled(query);
let settled = query[_settled];
if (settled === null) {
settled = query[_settled] = $newPromise<void>();
-2
View File
@@ -644,8 +644,6 @@ export interface DatabaseAdapter<Connection, ConnectionHandle, QueryHandle> exte
get closed(): boolean;
readonly supportsReservedConnections: boolean;
/** A query that is not simple asks the server for other formats for raw(): neither chosen nor left once it has started. */
readonly rawChangesFormat?: boolean;
/** `true`: `onConnected` was still queued and will never run. */
cancelConnect(onConnected: OnConnected<Connection>): boolean;
/** Throws `connectionClosedError()` unless the connection is open. */
+1 -1
View File
@@ -2,7 +2,7 @@
// `bun_sql_jsc` crate; this base crate intentionally has no jsc dependency.
pub use super::any_postgres_error::AnyPostgresError;
pub use super::types::tag::Tag;
pub use super::types::tag::{ResultFormats, Tag};
pub use int_types::{Int4, int4};
+26 -16
View File
@@ -139,24 +139,34 @@ impl Tag {
if self == Tag::box_array { b';' } else { b',' }
}
/// The result columns `DataCell` decodes from binary. Not read for parameters.
pub fn is_binary_format_supported(self) -> bool {
matches!(
self,
/// Of a result column. Not read for parameters.
pub fn is_binary_in(self, formats: ResultFormats) -> bool {
match self {
Tag::bool
| Tag::int4
| Tag::float8
| Tag::timestamp
| Tag::timestamptz
| Tag::bytea
| Tag::int2_array
| Tag::int4_array
| Tag::float4_array
| Tag::float8_array
)
| Tag::int4
| Tag::float8
| Tag::timestamp
| Tag::timestamptz
| Tag::bytea
| Tag::int4_array
| Tag::float4_array => formats != ResultFormats::Text,
Tag::int2_array | Tag::float8_array => formats == ResultFormats::Decodable,
Tag::float4 | Tag::numeric | Tag::time => formats == ResultFormats::Raw,
_ => false,
}
}
pub fn format_code(self) -> Short {
Short::from(self.is_binary_format_supported())
pub fn format_code(self, formats: ResultFormats) -> Short {
Short::from(self.is_binary_in(formats))
}
}
/// The result columns that a Bind asks for in binary.
#[derive(Clone, Copy, PartialEq, Eq)]
pub enum ResultFormats {
Text,
/// Those that `DataCell` decodes from binary.
Decodable,
/// Those that `.raw()` has always handed out in binary: programs parse these bytes.
Raw,
}
+4 -4
View File
@@ -371,7 +371,7 @@ fn binary_array(element: types::Tag, mut rest: &[u8]) -> Result<SQLDataCell> {
Ok(cell)
}
/// `Tag::is_binary_format_supported`, plus the types whose binary form is their text.
/// `ResultFormats::Decodable`, plus the types whose binary form is their text.
fn from_binary(tag: types::Tag, bytes: &[u8]) -> Result<SQLDataCell> {
use types::Tag as T;
match tag {
@@ -460,8 +460,8 @@ fn from_text(
pub(crate) struct Putter<'a> {
pub list: &'a mut [SQLDataCell],
pub fields: &'a [protocol::FieldDescription],
/// Bind asked for binary; the `fields` of the Describe before it always say text.
pub binary: bool,
/// What Bind asked for; the `fields` of the Describe before it always say text.
pub formats: types::ResultFormats,
pub bigint: bool,
pub global_object: &'a JSGlobalObject,
}
@@ -521,7 +521,7 @@ impl<'a> Putter<'a> {
let tag = types::Tag::from_oid(field.type_oid);
*cell = match optional_bytes {
None => SQLDataCell::null(),
Some(data) if field.binary || (self.binary && tag.is_binary_format_supported()) => {
Some(data) if field.binary || tag.is_binary_in(self.formats) => {
from_binary(tag, data.slice())?
}
Some(data) => from_text(self.bigint, tag, data.slice(), self.global_object)?,
+8 -18
View File
@@ -10,7 +10,6 @@ use bun_sql::postgres::postgres_protocol as protocol;
use bun_sql::postgres::postgres_types as types;
use bun_sql::postgres::postgres_types::{AnyPostgresError, Int4};
use bun_sql::postgres::protocol::{ReaderContext, WriterContext};
use bun_sql::shared::SQLQueryResultMode;
use bun_sql::shared::SSLMode;
use crate::jsc::js_error_to_postgres;
@@ -186,6 +185,7 @@ fn write_bind<Context: WriterContext>(
values_array: JSValue,
parameter_fields: &[Int4],
result_fields: &[protocol::FieldDescription],
formats: types::ResultFormats,
writer: protocol::NewWriter<Context>,
) -> Result<(), AnyPostgresError> {
let len =
@@ -236,11 +236,11 @@ fn write_bind<Context: WriterContext>(
if result_fields
.iter()
.any(|field| field.type_tag().is_binary_format_supported())
.any(|field| field.type_tag().is_binary_in(formats))
{
writer.short(result_fields.len())?;
for field in result_fields {
writer.short(field.type_tag().format_code())?;
writer.short(field.type_tag().format_code(formats))?;
}
} else {
writer.short(0)?;
@@ -277,17 +277,6 @@ fn write_binary_parameter<Context: WriterContext>(
Ok(true)
}
/// `.raw()` hands out the bytes of the text format: no column is asked for in binary.
fn binary_result_fields(
statement: &PostgresSQLStatement,
result_mode: SQLQueryResultMode,
) -> &[protocol::FieldDescription] {
if result_mode == SQLQueryResultMode::Raw {
return &[];
}
&statement.fields
}
fn write_parse<Context: WriterContext>(
query: &[u8],
signature: &Signature,
@@ -328,7 +317,7 @@ pub(crate) fn bind_and_execute<Context: WriterContext>(
global: &JSGlobalObject,
statement: &PostgresSQLStatement,
values: JSValue,
result_mode: SQLQueryResultMode,
formats: types::ResultFormats,
mut writer: protocol::NewWriter<Context>,
) -> Result<(), AnyPostgresError> {
let name = &statement.signature.prepared_statement_name;
@@ -337,7 +326,8 @@ pub(crate) fn bind_and_execute<Context: WriterContext>(
global,
values,
statement.parameter_types(),
binary_result_fields(statement, result_mode),
&statement.fields,
formats,
writer,
)?;
protocol::Execute {
@@ -355,7 +345,7 @@ pub(crate) fn parse_and_bind_and_execute<Context: WriterContext>(
query: &[u8],
statement: &PostgresSQLStatement,
values: JSValue,
result_mode: SQLQueryResultMode,
formats: types::ResultFormats,
include_describe: bool,
writer: protocol::NewWriter<Context>,
) -> Result<(), AnyPostgresError> {
@@ -363,7 +353,7 @@ pub(crate) fn parse_and_bind_and_execute<Context: WriterContext>(
if include_describe {
write_describe(&statement.signature, writer)?;
}
bind_and_execute(global, statement, values, result_mode, writer)
bind_and_execute(global, statement, values, formats, writer)
}
pub(crate) fn write_close_statement<Context: WriterContext>(
+15 -8
View File
@@ -46,7 +46,7 @@ use bun_sql::postgres::PostgresErrorOptions;
use bun_sql::postgres::Status;
use bun_sql::postgres::TLSStatus;
use bun_sql::postgres::postgres_protocol as protocol;
use bun_sql::postgres::postgres_types::Int4;
use bun_sql::postgres::postgres_types::{Int4, ResultFormats};
use bun_sql::postgres::socket_monitor as SocketMonitor;
use bun_sql::shared::ConnectionFlags;
use bun_sql::shared::Data;
@@ -1893,7 +1893,6 @@ impl PostgresSQLConnection {
return self.remove_unwritten(&req);
};
let values = postgres_sql_query::js::binding_get_cached(this_value).unwrap_or_default();
let result_mode = req.flags.get().result_mode;
let unnamed = self
.flags
.get()
@@ -1919,14 +1918,22 @@ impl PostgresSQLConnection {
if !self.stage(&req, values, statement.parameter_types()) {
return true;
}
req.update_flags(|f| f.binary = !statement.fields.is_empty());
// Whether an unnamed statement is described before its Bind depends on the values.
let formats = if unnamed {
ResultFormats::Text
} else if req.flags.get().result_mode == SQLQueryResultMode::Raw {
ResultFormats::Raw
} else {
ResultFormats::Decodable
};
req.update_flags(|f| f.formats = formats);
if let Err(err) = if unnamed {
PostgresRequest::parse_and_bind_and_execute(
self.global(),
req.query.to_utf8().slice(),
statement,
values,
result_mode,
formats,
false,
self.writer(),
)
@@ -1935,7 +1942,7 @@ impl PostgresSQLConnection {
self.global(),
statement,
values,
result_mode,
formats,
self.writer(),
)
} {
@@ -1966,7 +1973,7 @@ impl PostgresSQLConnection {
query.slice(),
statement,
values,
result_mode,
ResultFormats::Text,
true,
self.writer(),
)
@@ -2041,7 +2048,7 @@ impl PostgresSQLConnection {
let mut putter = DataCell::Putter {
list: &mut cells.0,
fields: &statement.fields,
binary: request_flags.binary,
formats: request_flags.formats,
bigint: request_flags.bigint,
global_object: self.global(),
};
@@ -2702,7 +2709,7 @@ impl PostgresSQLConnection {
request.statement.set(Some(replacement));
request.update_flags(|f| {
f.retried = true;
f.binary = false;
f.formats = ResultFormats::Text;
});
// It keeps its place: nothing is written until its ReadyForQuery, then it.
self.count(&request, RequestCounter::Pending);
+4 -2
View File
@@ -18,6 +18,7 @@ use super::postgres_sql_connection;
use super::postgres_sql_statement::{ResultDescription, Status as StatementStatus, describe};
use bun_sql::postgres::CommandTag;
use bun_sql::postgres::any_postgres_error::AnyPostgresError;
use bun_sql::postgres::postgres_types::ResultFormats;
use bun_sql::shared::ConnectionFlags;
use bun_sql::shared::SQLQueryResultMode as PostgresSQLQueryResultMode;
@@ -61,7 +62,8 @@ impl Default for PostgresSQLQuery {
#[derive(Clone, Copy)]
pub struct Flags {
pub(crate) binary: bool,
/// What its Bind asked for.
pub(crate) formats: ResultFormats,
pub(crate) bigint: bool,
pub(crate) simple: bool,
/// Parse + Describe go out first and Bind waits for the server's parameter types.
@@ -113,7 +115,7 @@ pub enum RequestCounter {
impl Default for Flags {
fn default() -> Self {
Self {
binary: false,
formats: ResultFormats::Text,
bigint: false,
simple: false,
describe_first: false,
+45 -2
View File
@@ -62,7 +62,16 @@ const cases: [string, string, any, string][] = [
["bytea, Uint8Array", "bytea[]", [new Uint8Array([1, 2, 44])], raw`{"\\x01022c"}`],
["bytea, DataView", "bytea[]", [new DataView(new Uint8Array([1, 2, 3]).buffer, 1)], raw`{"\\x0203"}`],
["bytea, ArrayBuffer", "bytea[]", [new Uint8Array([7]).buffer], raw`{"\\x07"}`],
["json", "json[]", [{ a: 1 }, [1, 2], "s", 1, true, null], raw`{"{\"a\":1}","[1,2]","\"s\"",1,true,NULL}`],
["json", "json[]", [{ a: 1 }, "s", 1, true, null, undefined], raw`{"{\"a\":1}","\"s\"",1,true,"null",NULL}`],
[
"json, nested",
"json[]",
[
[1, "a"],
[{ b: 2 }, null],
],
raw`{{1,"\"a\""},{"{\"b\":2}","null"}}`,
],
["jsonb", "jsonb[]", [{ a: 'q"' }, "s"], raw`{"{\"a\": \"q\\\"\"}","\"s\""}`],
["Int32Array", "int4[]", new Int32Array([100, 200]), "{100,200}"],
["Float32Array", "float4[]", new Float32Array([1.5, 2.5]), "{1.5,2.5}"],
@@ -123,7 +132,7 @@ describeWithContainer("postgres", { image: "postgres_plain", concurrent: true },
const { sql } = db;
const cast = (type: string, value: any) => sql`${sql.array(value)}::${sql.unsafe(type)}`;
// these three are written differently when the type is known
const needTheType = ["json", "jsonb", "box"];
const needTheType = ["json", "json, nested", "jsonb", "box"];
const others = cases.filter(([label]) => !needTheType.includes(label));
expect(byLabel(await selectEach(sql, cast, others), others)).toEqual(
Object.fromEntries(others.map(([label, , , stored]) => [label, stored])),
@@ -191,6 +200,40 @@ describeWithContainer("postgres", { image: "postgres_plain", concurrent: true },
});
});
describe("an array of JSON", () => {
test.each<[unknown[], string]>([
[[1, null], '{1,"null"}'],
[[undefined], "{NULL}"],
[[[null]], '{{"null"}}'],
[[[1]], "{{1}}"],
[[[1], [2]], "{{1},{2}}"],
[[[{ a: 1 }]], raw`{{"{\"a\": 1}"}}`],
])("%p: null is a JSON value, a nested array a dimension", async (values, stored) => {
await using db = await connect();
expect(await db.sql`select ${db.sql.array(values, "JSONB")}::text as x`).toEqual([{ x: stored }]);
});
test.each<[unknown[]]>([[[[1], [2, 3]]], [[[1], 2]], [[[]]]])("%p has no dimensions", async values => {
await using db = await connect();
expect(await db.sql`select ${db.sql.array(values, "JSONB")} as x`.catch(error => error.errno)).toBe("22P02");
});
test("sql.json() makes one element of an array", async () => {
await using db = await connect();
const { sql } = db;
expect(await sql`select ${sql.array([sql.json([1, 2]), sql.json([]), { a: 1 }], "JSONB")}::text as x`).toEqual([
{ x: raw`{"[1, 2]",[],"{\"a\": 1}"}` },
]);
});
test("a plain array is written the same way", async () => {
await using db = await connect();
expect(
await db.sql`select ${[1, null, undefined]}::jsonb[]::text as flat, ${[[1], [null]]}::jsonb[]::text as nested`,
).toEqual([{ flat: '{1,"null",NULL}', nested: '{{1},{"null"}}' }]);
});
});
test("the type argument", async () => {
await using db = await connect();
const { sql } = db;
+3 -2
View File
@@ -708,8 +708,9 @@ describe.concurrent("scripted server", () => {
test("the pool goes on with the one that logged in", async () => {
await using mock = await pgScramServer({ password: "aIX" });
await using sql = new SQL({ ...client(mock.port), password: "\u00aa\u2168", max: 2 });
using first = await sql.reserve();
await using sql = new SQL({ ...client(mock.port), password: "\u00aa\u2168", max: 1 });
const first = await sql.reserve();
await first.close();
using second = await sql.reserve();
expect(mock.exchanges.map(exchange => exchange.proofOk)).toEqual([false, true, true]);
});
+7 -1
View File
@@ -218,6 +218,11 @@ describe("named statements", () => {
expect(resultFormats(drain())).toEqual([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0]]);
});
test("raw() asks for the columns in binary that it has always handed out in binary", async () => {
await sql.unsafe(`select $1 /* raw formats: params=${OID.int4} cols=${columns.slice(0, -1)} */`, [1]).raw();
expect(resultFormats(drain())).toEqual([[1, 1, 1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 1, 0, 0, 0]]);
});
test("values of an object through the sql() helper", async () => {
// Literal OIDs: an interpolation in a tagged template is a parameter.
await sql`insert into t ${sql({ a: 7, b: "seven", c: null })} /* params=23,25,23 cols=25 */`;
@@ -343,7 +348,8 @@ describe("prepare: false", () => {
statement: "",
paramFormats: [0, 1],
params: [Buffer.from(text), Buffer.from("00000007", "hex")],
resultFormats: [1],
// Text, as in the one batch of every other query without a named statement.
resultFormats: [],
}),
pgExecute(),
pgSync(),
+3 -14
View File
@@ -219,7 +219,8 @@ const matrix: [label: string, value: () => unknown, cells: Cells, unprepared?: C
"[1,null]": "json",
"[1, null]": "jsonb",
ERR_INVALID_ARG_TYPE: "bytea",
"{1,NULL}": "int4_array float4_array text_array jsonb_array",
"{1,NULL}": "int4_array float4_array text_array",
'{1,"null"}': "jsonb_array",
},
],
[
@@ -233,8 +234,7 @@ const matrix: [label: string, value: () => unknown, cells: Cells, unprepared?: C
"[[1,2],[3,4]]": "json",
"[[1, 2], [3, 4]]": "jsonb",
ERR_INVALID_ARG_TYPE: "bytea",
"{{1,2},{3,4}}": "int4_array float4_array text_array",
'{"[1, 2]","[3, 4]"}': "jsonb_array",
"{{1,2},{3,4}}": "int4_array float4_array text_array jsonb_array",
},
],
[
@@ -436,17 +436,6 @@ describeWithContainer("postgres", { image: "postgres_plain" }, container => {
expect(await sql`select ${"next"}::text as x`).toEqual([{ x: "next" }]);
});
test("raw() returns the text format on every run", async () => {
const run = async () => {
const [row] = await sql`select ${1}::int4, ${true}::bool, ${1.5}::float8, ${Buffer.from("hi")}::bytea`.raw();
return row.map((cell: Uint8Array) => Buffer.from(cell).toString());
};
expect([await run(), await run()]).toEqual([
["1", "t", "1.5", "\\x6869"],
["1", "t", "1.5", "\\x6869"],
]);
});
test.each<[string, (sql: SQL) => Promise<unknown>]>([
["sql.unsafe(text)", sql => sql.unsafe("select 1 as x\0")],
["sql.unsafe(text, [parameter])", sql => sql.unsafe("select $1::int as x\0", [1])],
+156 -1
View File
@@ -1,6 +1,6 @@
// What a PostgreSQL value reads as in JS. It has to be the same on every path a result takes, in text or in binary.
import { SQL } from "bun";
import { afterAll, beforeAll, expect, test } from "bun:test";
import { afterAll, beforeAll, describe, expect, test } from "bun:test";
import { bunEnv, bunExe, describeWithContainer } from "harness";
/** One line per value, so that -0, NaN, an Invalid Date, a Buffer and a typed array differ from their look-alikes. */
@@ -277,6 +277,161 @@ describeWithContainer("postgres", { image: "postgres_plain", concurrent: true },
});
}
describe("raw()", () => {
// [type, literal, the text format, the binary format in hex of a column that raw() asks for in binary]
const columns: [type: string, literal: string, text: string, binary?: string][] = [
["bool", "true", "t", "01"],
["int2", "1", "1"],
["int4", "1", "1", "00000001"],
["int8", "1", "1"],
["oid", "1", "1"],
["float4", "1.5", "1.5", "3fc00000"],
["float8", "1.5", "1.5", "3ff8000000000000"],
["numeric", "1.5", "1.5", "000200000000000100011388"],
["money", "1.5", "$1.50"],
["text", "ab", "ab"],
["varchar", "ab", "ab"],
["bpchar", "a", "a"],
["name", "ab", "ab"],
["bytea", "\\x0102ff", "\\x0102ff", "0102ff"],
["date", "2024-01-02", "2024-01-02"],
["time", "03:04:05", "03:04:05", "0000000292555340"],
["timetz", "03:04:05+00", "03:04:05+00"],
["timestamp", "2024-01-02 03:04:05", "2024-01-02 03:04:05", "0002b0ec8515f340"],
["timestamptz", "2024-01-02 03:04:05+00", "2024-01-02 03:04:05+00", "0002b0ec8515f340"],
["interval", "1 day", "1 day"],
["uuid", "a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11", "a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11"],
["json", "[1]", "[1]"],
["jsonb", "[1]", "[1]"],
["inet", "1.2.3.4", "1.2.3.4"],
["point", "(1,2)", "(1,2)"],
["bool[]", "{t}", "{t}"],
["int2[]", "{1,2}", "{1,2}"],
["int4[]", "{1,2}", "{1,2}", "000000010000000000000017000000020000000100000004000000010000000400000002"],
["int8[]", "{1,2}", "{1,2}"],
["float4[]", "{1.5}", "{1.5}", "0000000100000000000002bc0000000100000001000000043fc00000"],
["float8[]", "{1.5}", "{1.5}"],
["numeric[]", "{1.5}", "{1.5}"],
["text[]", "{a}", "{a}"],
["bytea[]", '{"\\\\x01"}', '{"\\\\x01"}'],
["timestamptz[]", '{"2024-01-02 03:04:05+00"}', '{"2024-01-02 03:04:05+00"}'],
["jsonb[]", "{1}", "{1}"],
["uuid[]", "{a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11}", "{a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11}"],
];
// A mode that decodes asks for these the other way, and execute() writes the Bind of a prepared statement.
const otherwiseDecoded: Record<string, string> = {
"float4": "1.5",
"numeric": "1.5",
"time": "03:04:05",
"int2[]": "0x0000000100000000000000150000000200000001000000020001000000020002",
"float8[]": "0x0000000100000000000002bd0000000100000001000000083ff8000000000000",
};
type Columns = typeof columns;
const select = (list: Columns) =>
"select " + list.map(([type, literal], i) => `'${literal}'::${type} as c${i}`).join(", ");
const inText = (list: Columns) => Object.fromEntries(list.map(([type, , text]) => [type, text]));
const inBinary = (list: Columns) =>
Object.fromEntries(list.map(([type, , text, binary]) => [type, binary ? "0x" + binary : text]));
/** Each cell as its text, or as 0x and its bytes where it is not the text format. */
const cells = (list: Columns, rows: Uint8Array[][]) =>
Object.fromEntries(
list.map(([type, , text], i) => {
const cell = Buffer.from(rows[0][i]);
return [type, cell.equals(Buffer.from(text)) ? text : "0x" + cell.toString("hex")];
}),
);
const raw = (query: any, late: boolean): Promise<Uint8Array[][]> => (late ? query.execute().raw() : query.raw());
type Run = (sql: SQL, text: string, late: boolean) => Promise<Uint8Array[][]>;
const runs: [name: string, binary: boolean, prepared: boolean, run: Run][] = [
["the first run", false, false, (sql, text, late) => raw(template(sql, text), late)],
[
"the second run",
true,
true,
async (sql, text, late) => {
await template(sql, text).raw();
return await raw(template(sql, text), late);
},
],
[
"a run after one that decoded",
true,
true,
async (sql, text, late) => {
await template(sql, text);
return await raw(template(sql, text), late);
},
],
// Its Bind waits for the server's parameter types.
["with a parameter", true, false, (sql, text, late) => raw(sql.unsafe(text + " where $1 = 1", [1]), late)],
[
"with a parameter, the second run",
true,
true,
async (sql, text, late) => {
await sql.unsafe(text + " where $1 = 1", [1]);
return await raw(sql.unsafe(text + " where $1 = 1", [1]), late);
},
],
["simple()", false, false, (sql, text, late) => raw(sql.unsafe(text).simple(), late)],
];
async function session(options?: SQL.Options) {
await container.ready;
const sql = connect(options);
await sql.unsafe("set time zone 'UTC'");
return sql;
}
describe.each([
["raw()", false],
["execute().raw()", true],
])("%s", (_, late) => {
test.each(runs)("%s", async (_, binary, prepared, run) => {
await using sql = await session();
const list = late && prepared ? columns.filter(([type]) => !(type in otherwiseDecoded)) : columns;
expect(cells(list, await run(sql, select(list), late))).toEqual((binary ? inBinary : inText)(list));
});
test.each(runs)("prepare: false, %s", async (_, __, ___, run) => {
await using sql = await session({ prepare: false });
expect(cells(columns, await run(sql, select(columns), late))).toEqual(inText(columns));
});
});
test.each(runs.filter(([, , prepared]) => prepared))(
"execute().raw(), %s: the columns that a mode that decodes asks for the other way",
async (_, __, ___, run) => {
await using sql = await session();
const list = columns.filter(([type]) => type in otherwiseDecoded);
expect(cells(list, await run(sql, select(list), true))).toEqual(otherwiseDecoded);
},
);
const undecodable = ["float4", "numeric", "time"];
test("raw().execute().values() decodes what raw() asked for", async () => {
await using sql = await session();
const text = select(columns.filter(([type]) => !undecodable.includes(type)));
await template(sql, text);
const values = await template(sql, text).values();
expect(await template(sql, text).raw().execute().values()).toEqual(values);
});
test.each(undecodable)("raw().execute().values() of a %s in binary has no decoder", async type => {
await using sql = await session();
const text = select(columns.filter(column => column[0] === type));
await template(sql, text);
expect(
await template(sql, text)
.raw()
.execute()
.values()
.catch((error: any) => error.code),
).toBe("ERR_POSTGRES_UNKNOWN_FORMAT_CODE");
});
});
// An element of an array of numbers is read by other code than a column. Three readers have to agree on every
// value: the array's, the column's, and JS's own on the text that the server prints.
test.each(["float4", "float8", "int4", "int2"])(
+24 -3
View File
@@ -140,10 +140,12 @@ for (const server of servers) {
async function start() {
const tag = `bun_hot_${randomBytes(6).toString("hex")}`;
const { host, port } = container;
const defaultClient = new URL(server.url(host, port, tag));
defaultClient.searchParams.set("max", "2");
const program = new HotProgram({
SQL_URL: server.url(host, port, tag),
SQL_OTHER_URL: server.otherDatabase(host, port, tag),
DATABASE_URL: server.url(host, port, tag),
DATABASE_URL: defaultClient.href,
SQL_SESSION_ID: server.sessionId,
SQL_WAIT_FOR_LOCK: server.waitForLock(tag),
SQL_CHANNEL: tag,
@@ -193,14 +195,33 @@ for (const server of servers) {
expect(await program.load(sameSettings)).toEqual(first);
expect(await program.open()).toBe(2);
}
// `max` is the pool's and `bigint` the queries', not the connections'.
// `max` is the pool's and `bigint` the queries', not the connections'. The third one is dialed.
expect(
await program.load(`
const sql = new SQL(url, { max: 3, bigint: true });
report({ ids: await sessions(sql) });
`),
).toEqual({ ids: first.ids });
expect(await program.open()).toBe(2);
expect(await program.settlesAt(3)).toBe(3);
});
test("the first query of a load takes over the whole pool", async () => {
await using program = await start();
const body = /* ts */ `
let connected = 0;
const all = Promise.withResolvers();
const sql = new SQL(url, { max: 3, onconnect: () => void (++connected === 3 && all.resolve()), onclose });
const query = sql.unsafe("select 1").execute();
const inTheCall = connected;
await query;
await all.promise;
report({ inTheCall, closed: globalThis.closed });
`;
expect(await program.load(body)).toEqual({ inTheCall: 0, closed: [] });
for (let i = 0; i < 2; i++) {
expect(await program.load(body)).toEqual({ inTheCall: 3, closed: [] });
expect(await program.open()).toBe(3);
}
});
test("also when the first query of a load comes from a Bun.serve() handler", async () => {
@@ -178,7 +178,7 @@ for (const [adapter, refusedCode] of [
);
}
// One query is one attempt, whatever `max` is.
// The first query starts the pool: `max` attempts.
test.concurrent("postgres: pool calls from onclose are safe when the password function throws", async () => {
const fixture = /* ts */ `
import { SQL } from "bun";
@@ -211,7 +211,7 @@ try {
process.exit(0);
`;
expect(await runFixture(fixture)).toEqual({
stdout: "reentry ok\nquery rejected: password error\n",
stdout: "reentry ok\nreentry ok\nquery rejected: password error\n",
stderr: "",
exitCode: 0,
});
+174 -125
View File
@@ -59,14 +59,19 @@ async function unhandledRejections(url: string) {
async function row(name, start, on) {
reported.length = 0;
start();
// Answered after the query of the row.
await good(on);
// Answered after the query of the row. PostgreSQL refuses it in a transaction that failed.
await good(on).catch(() => {});
await new Promise(setImmediate);
result[name] = reported.join();
}
await row("execute()", () => bad().execute());
await row("then(f)", () => bad().then(() => {}));
await row("finally(f)", () => bad().finally(() => {}));
await row("finally(rethrow)", () => bad().finally(rethrow));
await row("finally(rethrow).catch(g)", () => bad().finally(rethrow).catch(() => {}));
await row("finally(f that returns a rejected promise)", () => bad().finally(() => Promise.reject(new Error("handler"))));
await row("finally(rethrow) of a query that succeeds", () => good().finally(rethrow));
await row("finally(f).then(h)", () => bad().finally(() => {}).then(() => {}));
await row("catch(f)", () => bad().catch(() => {}));
await row("then(f, g)", () => bad().then(() => {}, () => {}));
await row("catch(rethrow)", () => bad().catch(rethrow));
@@ -77,6 +82,7 @@ async function unhandledRejections(url: string) {
await row("execute() on a reservation", () => bad(reserved).execute(), reserved);
reserved.release();
}
await sql.begin(tx => row("execute() in a transaction", () => bad(tx).execute(), tx)).catch(() => {});
await sql.close();
console.log(JSON.stringify(result));
`,
@@ -89,19 +95,70 @@ async function unhandledRejections(url: string) {
return { result: stdout && JSON.parse(stdout), exitCode };
}
const reportedLikeAnyPromise = {
"execute()": "query",
"then(f)": "query",
"finally(f)": "query",
/** Without a listener. Not in a top-level await: while one is pending, Bun reports the rejection and goes on. */
async function outlivesAQueryThatIsOnlyStarted(url: string) {
await using proc = Bun.spawn({
cmd: [
bunExe(),
"-e",
`
const sql = new Bun.SQL(process.env.SQL_URL, { max: 1 });
(async () => {
await sql.unsafe("select 1");
sql.unsafe("select * from bun_pool_no_such_table").execute();
await sql.unsafe("select 1");
await new Promise(setImmediate);
console.log("alive");
await sql.close();
})();
`,
],
env: { ...bunEnv, SQL_URL: url },
stdout: "pipe",
stderr: "inherit",
});
const [stdout, exitCode] = await Promise.all([proc.stdout.text(), proc.exited]);
return { stdout, exitCode };
}
/** Nothing of the caller's threw, and the query had a connection. */
const onlyStarted = {
"execute()": "",
"execute() on a reservation": "",
"execute() in a transaction": "",
"finally(f)": "",
};
const handled = {
"catch(f)": "",
"then(f, g)": "",
"finally(rethrow).catch(g)": "",
};
/** What then() and catch() return, and what a handler of the caller throws. */
const reportedLikeAnyPromise = {
"then(f)": "query",
"finally(f).then(h)": "query",
"finally(rethrow)": "handler",
"finally(f that returns a rejected promise)": "handler",
"finally(rethrow) of a query that succeeds": "handler",
"catch(rethrow)": "handler",
"then(null, rethrow)": "handler",
"then(rethrow, g) of a query that succeeds": "handler",
};
test.concurrent("sqlite: a failure that nobody handles is reported", async () => {
expect(await unhandledRejections(":memory:")).toEqual({ result: reportedLikeAnyPromise, exitCode: 0 });
describe.concurrent("sqlite: a failure that nobody handles", () => {
let rows: ReturnType<typeof unhandledRejections> | undefined;
const reported = () => (rows ??= unhandledRejections(":memory:"));
test("of a query that is only started is reported outside a transaction", async () => {
expect(await reported()).toMatchObject({
result: { ...handled, "execute()": "query", "execute() in a transaction": "", "finally(f)": "" },
exitCode: 0,
});
});
test("of what then() and catch() return, and what a handler throws, is reported as of any promise", async () => {
expect(await reported()).toMatchObject({ result: reportedLikeAnyPromise, exitCode: 0 });
});
});
const mysqlDialect = {
@@ -166,30 +223,33 @@ for (const server of servers) {
}
describe("dials", () => {
test("queries that follow each other share one connection", async () => {
test.each([
["query", (sql: SQL) => sql.unsafe("select 1")],
["reserve()", (sql: SQL) => sql.reserve().then(reserved => reserved.release())],
["begin()", (sql: SQL) => sql.begin(tx => tx.unsafe("select 1"))],
["connect()", (sql: SQL) => sql.connect()],
])("the first %s starts the pool until max is reached", async (_, call) => {
const connected = new Calls();
await using sql = await pool({ max: 5, onconnect: connected.hit });
await call(sql);
await connected.reached(5);
const sessions = new Set([await sessionId(sql), await sessionId(sql), await sessionId(sql)]);
expect({ sessions: sessions.size, dials: connected.count }).toEqual({ sessions: 1, dials: 1 });
expect({ sessions: sessions.size, dials: connected.count }).toEqual({ sessions: 1, dials: 5 });
});
test("queries that run together get a connection each, up to max", async () => {
const connected = new Calls();
await using sql = await pool({ max: 3, onconnect: connected.hit });
await using held = await gate();
const two = [held.wait(sql), held.wait(sql)].map(outcome);
await connected.reached(2);
await held.roundTrip();
expect(connected.count).toBe(2);
expect((await threeSessions(sql, () => connected.reached(3))).size).toBe(3);
const five = Array.from({ length: 5 }, () => outcome(held.wait(sql)));
await connected.reached(3);
await using held = await gate();
const seven = Array.from({ length: 7 }, () => outcome(held.wait(sql)));
await held.open();
expect(await Promise.all([...two, ...five])).toEqual(Array(7).fill("resolved"));
expect(await Promise.all(seven)).toEqual(Array(7).fill("resolved"));
expect(connected.count).toBe(3);
});
test("reserve() dials instead of waiting for a busy connection", async () => {
test("reserve() does not wait for a busy connection", async () => {
await using sql = await pool({ max: 2 });
await using held = await gate();
const busy = outcome(held.wait(sql));
@@ -212,14 +272,14 @@ for (const server of servers) {
expect({ sessions: sessions.size, kept: sessions.has(kept) }).toEqual({ sessions: 3, kept: true });
});
test("a pool that idled out dials once for one query", async () => {
test("a pool that idled out is started again until max is reached", async () => {
const connected = new Calls();
const closed = new Calls();
await using sql = await pool({ max: 3, idleTimeout: 0.05, onconnect: connected.hit, onclose: closed.hit });
expect((await threeSessions(sql, () => connected.reached(3))).size).toBe(3);
await closed.reached(3);
await sessionId(sql);
expect(connected.count).toBe(4);
await connected.reached(6);
});
});
@@ -314,9 +374,15 @@ for (const server of servers) {
const closed = new Calls();
await using sql = await pool({ max: 2, onclose: closed.hit });
const name = `bun_pool_${(Math.random() * 2 ** 31) | 0}`;
await sql.beginDistributed(name, tx => tx.unsafe("select 1"));
// One of them runs on that connection.
await Promise.all([sql.unsafe("select 1"), sql.unsafe("select 2")]);
let prepared: unknown;
await sql.beginDistributed(name, async tx => {
prepared = await sessionId(tx);
});
// MariaDB commits it on that connection only, so the other one is kept away from what follows.
const [first, second] = await Promise.all([sql.reserve(), sql.reserve()]);
using other = (await sessionId(first)) === prepared ? second : first;
(other === first ? second : first).release();
await sql.unsafe("select 1");
expect([await outcome(sql.commitDistributed(name)), closed.count]).toEqual(["resolved", 0]);
});
@@ -565,11 +631,27 @@ for (const server of servers) {
});
});
test("a failure that nobody handles is reported", async () => {
await container.ready;
expect(await unhandledRejections(server.url(container.host, container.port))).toEqual({
result: { ...reportedLikeAnyPromise, "execute() on a reservation": "query" },
exitCode: 0,
describe("a failure that nobody handles", () => {
let rows: ReturnType<typeof unhandledRejections> | undefined;
async function reported() {
await container.ready;
return (rows ??= unhandledRejections(server.url(container.host, container.port)));
}
test("of a query that is only started, by execute() or finally(f), is not reported", async () => {
expect(await reported()).toMatchObject({ result: { ...onlyStarted, ...handled }, exitCode: 0 });
});
test("of a query that is only started does not end the process", async () => {
await container.ready;
expect(await outlivesAQueryThatIsOnlyStarted(server.url(container.host, container.port))).toEqual({
stdout: "alive\n",
exitCode: 0,
});
});
test("of what then() and catch() return, and what a handler throws, is reported as of any promise", async () => {
expect(await reported()).toMatchObject({ result: reportedLikeAnyPromise, exitCode: 0 });
});
});
@@ -579,7 +661,6 @@ for (const server of servers) {
});
describe("once the query has started", () => {
const isPostgres = server.image === "postgres_plain";
const query = (sql: SQL) => sql`select ${7} + 0 as x, ${"a"} as y`;
test("values() counts", async () => {
@@ -592,9 +673,9 @@ for (const server of servers) {
}
});
test(isPostgres ? "raw() is neither chosen nor left" : "raw() counts", async () => {
test("raw() counts", async () => {
await using sql = await pool();
// The second time the statement is prepared, and PostgreSQL is asked for binary results unless it is raw().
// The second time the statement is prepared, and execute() has asked PostgreSQL for the formats.
for (let i = 0; i < 2; i++) {
const raw = await query(sql).raw();
expect({
@@ -602,11 +683,7 @@ for (const server of servers) {
left: await query(sql).raw().execute().values(),
// simple() comes too late to make it a simple query
leftAfterSimple: await query(sql).raw().execute().simple().values(),
}).toEqual(
isPostgres
? { chosen: [{ x: 7, y: "a" }], left: raw, leftAfterSimple: raw }
: { chosen: raw, left: [[7, "a"]], leftAfterSimple: [[7, "a"]] },
);
}).toEqual({ chosen: raw, left: [[7, "a"]], leftAfterSimple: [[7, "a"]] });
}
});
@@ -706,8 +783,11 @@ const scripted = [
adapter: "postgres",
url: (port: number) => `postgres://u@127.0.0.1:${port}/db`,
prefix: "ERR_POSTGRES_",
ready(socket: net.Socket) {
socket.once("data", () => socket.write(Buffer.concat([pgAuthenticationOk(), pgReadyForQuery()])));
ready(socket: net.Socket, answers = false) {
socket.once("data", () => {
socket.write(Buffer.concat([pgAuthenticationOk(), pgReadyForQuery()]));
if (answers) socket.on("data", () => this.answer(socket));
});
},
refuseLogin(socket: net.Socket) {
socket.once("data", () =>
@@ -720,13 +800,13 @@ const scripted = [
adapter: "mysql",
url: (port: number) => `mysql://u@127.0.0.1:${port}/db`,
prefix: "ERR_MYSQL_",
ready(socket: net.Socket) {
ready(socket: net.Socket, answers = false) {
let buffered: Buffer = Buffer.alloc(0);
let authenticated = false;
socket.write(mysqlHandshakeV10());
socket.on("data", chunk => {
buffered = mysqlReadPackets(Buffer.concat([buffered, chunk]), (seq, payload) => {
if (authenticated) return void mysqlAckSessionSetup(socket, payload);
if (authenticated) return void (mysqlAckSessionSetup(socket, payload) || (answers && this.answer(socket)));
authenticated = true;
socket.write(mysqlOkPacket(seq + 1));
});
@@ -740,7 +820,11 @@ const scripted = [
},
];
describe.each(scripted)("$adapter, scripted server", ({ url, prefix, ready, refuseLogin, answer }) => {
describe.each(scripted)("$adapter, scripted server", script => {
const { url, prefix, refuseLogin, answer } = script;
const ready = (socket: net.Socket) => script.ready(socket);
/** It also answers each simple query. */
const serving = (socket: net.Socket) => script.ready(socket, true);
async function serverOf(onSocket: (socket: net.Socket, nth: number) => void) {
const sockets: net.Socket[] = [];
const { port, server } = await listeningServer(socket => {
@@ -761,7 +845,7 @@ describe.each(scripted)("$adapter, scripted server", ({ url, prefix, ready, refu
/** These servers answer no query: a connection that has one stays busy, and close() would wait for it. */
const unanswered = (sql: SQL) => ({ [Symbol.asyncDispose]: () => sql.close({ timeout: 0 }) });
test.concurrent("a query whose login is refused costs one dial", async () => {
test.concurrent("a server that refuses every login is dialed `max` times for each query", async () => {
await using server = await serverOf(refuseLogin);
await using sql = new SQL({ url: url(server.port), max: 5 });
const dials: number[] = [];
@@ -770,7 +854,7 @@ describe.each(scripted)("$adapter, scripted server", ({ url, prefix, ready, refu
expect(await outcome(sql.unsafe("select 1"))).toBe(prefix + "SERVER_ERROR");
dials.push(server.sockets.length - before);
}
expect(dials).toEqual([1, 1, 1]);
expect(dials).toEqual([5, 5, 5]);
});
test.concurrent("a login that is refused once does not stay refused", async () => {
@@ -790,108 +874,73 @@ describe.each(scripted)("$adapter, scripted server", ({ url, prefix, ready, refu
expect(await outcome(reserved.unsafe("select 1"))).toBe(prefix + "CONNECTION_CLOSED");
});
test.concurrent("callers that no connection can take get the error of the dial that was made for them", async () => {
await using server = await serverOf((socket, nth) => (nth === 1 ? ready(socket) : refuseLogin(socket)));
test.concurrent.each([
["query", (sql: SQL) => sql.unsafe("select 1").simple()],
["reserve()", (sql: SQL) => sql.reserve().then(reserved => reserved.release())],
["begin()", (sql: SQL) => sql.begin(tx => tx.unsafe("select 1").simple())],
])("a %s waits for the connection that is reserved when nothing more can be dialed", async (_, call) => {
await using server = await serverOf((socket, nth) => (nth === 1 ? serving(socket) : refuseLogin(socket)));
const closed = new Calls();
const sql = new SQL({ url: url(server.port), max: 2, onclose: closed.hit });
await using _ = unanswered(sql);
using holder = await sql.reserve();
const [first, second] = [sql.unsafe("select 1").execute(), sql.reserve()].map(outcome);
expect(await unlessPending(first, closed.reached(1).then(nextTurn))).toBe(prefix + "SERVER_ERROR");
expect(await second).toBe(prefix + "SERVER_ERROR");
await using sql = new SQL({ url: url(server.port), max: 2, onclose: closed.hit });
const holder = await sql.reserve();
await closed.reached(1);
const waiting = outcome(call(sql));
expect(await unlessPending(waiting, closed.reached(2).then(nextTurn))).toBe("pending");
holder.release();
expect(await waiting).toBe("resolved");
expect(server.sockets.length).toBe(3);
});
test.concurrent("queries on a cold pool wait for the dial on its way when the others are refused", async () => {
test.concurrent("queries on a cold pool are served by the one connection that comes up", async () => {
const closed = new Calls();
// Room for one, which is let in once the client knows of the refusals.
await using server = await serverOf((socket, nth) =>
nth === 1 ? void closed.reached(2).then(() => ready(socket)) : refuseLogin(socket),
nth === 1 ? void closed.reached(2).then(() => serving(socket)) : refuseLogin(socket),
);
const sql = new SQL({ url: url(server.port), max: 3, onclose: closed.hit });
await using _ = unanswered(sql);
const queries = [1, 2, 3].map(() => sql.unsafe("select 1").execute());
const refused = closed.reached(2).then(nextTurn);
expect(await Promise.all(queries.map(query => unlessPending(outcome(query), refused)))).toEqual(
Array(3).fill("pending"),
);
// With a connection up the pool tries to grow once more, and then makes do.
await closed.reached(4);
expect(queries.map(query => query.active)).toEqual([true, true, true]);
expect(server.sockets.length).toBe(5);
await using sql = new SQL({ url: url(server.port), max: 3, onclose: closed.hit });
const queries = [1, 2, 3].map(() => outcome(sql.unsafe("select 1").simple()));
expect(await Promise.all(queries)).toEqual(Array(3).fill("resolved"));
expect(server.sockets.length).toBe(3);
});
test.concurrent("queries make do with a connection that is up", async () => {
await using server = await serverOf((socket, nth) => (nth === 1 ? ready(socket) : refuseLogin(socket)));
const closed = new Calls();
const sql = new SQL({ url: url(server.port), max: 2, onclose: closed.hit });
await using _ = unanswered(sql);
await sql.connect();
const queries = [1, 2].map(() => sql.unsafe("select 1").execute());
queries.forEach(outcome);
await closed.reached(1);
expect(queries.map(query => query.active)).toEqual([true, true]);
});
test.concurrent("they leave its share to a dial that is on its way", async () => {
const closed = new Calls();
test.concurrent("a pool that lost connections grows by one dial for each caller that waits", async () => {
await using server = await serverOf(ready);
const connected = new Calls();
const letIn = Promise.withResolvers<void>();
await using server = await serverOf((socket, nth) => {
if (nth === 2) return refuseLogin(socket);
if (nth === 3) return void letIn.promise.then(() => ready(socket));
ready(socket);
});
const closed = new Calls();
const sql = new SQL({ url: url(server.port), max: 3, onconnect: connected.hit, onclose: closed.hit });
await using _ = unanswered(sql);
await sql.connect();
const queries = [1, 2, 3].map(() => sql.unsafe("select 1").execute());
queries.forEach(outcome);
await closed.reached(1);
expect(queries.map(query => query.active)).toEqual([true, true, false]);
letIn.resolve();
await connected.reached(2);
expect(queries.map(query => query.active)).toEqual([true, true, true]);
});
test.concurrent("a reservation whose dial is refused gets the connection that is up: it drains for it", async () => {
await using server = await serverOf((socket, nth) => (nth === 1 ? ready(socket) : refuseLogin(socket)));
const closed = new Calls();
const sql = new SQL({ url: url(server.port), max: 2, onclose: closed.hit });
await using _ = unanswered(sql);
await sql.connect();
const busy = outcome(sql.unsafe("select 1").simple().execute());
const reservation = outcome(sql.reserve());
expect(await unlessPending(reservation, closed.reached(1).then(nextTurn))).toBe("pending");
// Nobody got an answer, so nothing is dialed for the next one. Accepted after whatever the pool would have dialed:
await connected.reached(3);
server.sockets[1].destroy();
server.sockets[2].destroy();
await closed.reached(2);
[1, 2].forEach(() => outcome(sql.unsafe("select 1").execute()));
await connected.reached(4);
// Accepted after whatever the pool would have dialed.
const probe = net.connect(server.port, "127.0.0.1");
await once(probe, "connect");
probe.destroy();
while (server.sockets.length < 3) await nextTurn();
expect(server.sockets.length).toBe(3);
const later = sql.unsafe("select 2").execute();
outcome(later);
expect(later.active).toBe(false);
answer(server.sockets[0]);
expect([await busy, await reservation]).toEqual(["resolved", "resolved"]);
while (server.sockets.length < 5) await nextTurn();
expect(server.sockets.length).toBe(5);
});
test.concurrent("a caller is left to a dial that is on its way when another one is refused", async () => {
test.concurrent("the connection that is up drains for a reservation that nothing can be dialed for", async () => {
await using server = await serverOf((socket, nth) => (nth === 1 ? ready(socket) : refuseLogin(socket)));
const closed = new Calls();
await using server = await serverOf((socket, nth) => {
if (nth === 2) return refuseLogin(socket);
// Let in once the client knows of the refusal.
if (nth === 3) return void closed.reached(1).then(() => ready(socket));
ready(socket);
});
const sql = new SQL({ url: url(server.port), max: 3, onclose: closed.hit });
await using _ = unanswered(sql);
await using sql = new SQL({ url: url(server.port), max: 2, onclose: closed.hit });
await sql.connect();
const left = [sql.unsafe("select 1").execute(), sql.unsafe("select 2").execute()].map(outcome);
expect(await outcome(sql.reserve())).toBe("resolved");
expect(server.sockets.length).toBe(3);
await sql.close({ timeout: 0 });
await Promise.all(left);
await closed.reached(1);
const busy = outcome(sql.unsafe("select 1").simple().execute());
const reservation = sql.reserve();
expect(await unlessPending(outcome(reservation), closed.reached(2).then(nextTurn))).toBe("pending");
const later = sql.unsafe("select 2").simple().execute();
expect(later.active).toBe(false);
answer(server.sockets[0]);
expect(await busy).toBe("resolved");
(await reservation).release();
await once(server.sockets[0], "data");
answer(server.sockets[0]);
expect(await outcome(later)).toBe("resolved");
});
test.concurrent("a server that refuses everybody is dialed `max` times for callers that wait together", async () => {