Skip to content

Conversation

Mytherin
Copy link
Collaborator

This PR adds support for overloads to scalar and table macros. Since macro arguments are not typed, overloads are exclusively possible on the number of positional parameters, i.e. after this PR it is possible to create macros that accept a variable number of positional arguments through overloading.

It is currently only possible to define all overloads when creating a macro - i.e. there is no support for adding overloads to existing macros. Example syntax:

CREATE MACRO variable_addition
	(a) AS a,
	(a, b) AS a + b,
	(a, b, c) AS a + b + c;

SELECT variable_addition(42) AS single_arg, variable_addition(42, 84) AS two_args;
┌────────────┬──────────┐
│ single_arg │ two_args │
│   int32    │  int32   │
├────────────┼──────────┤
│         42126 │
└────────────┴──────────┘

If an overload is not present, the error message prints all possible overloads:

D SELECT variable_addition();
-- Binder Error: Macro "variable_addition" does not support 0 parameters.
-- Candidate macros:
-- 	variable_addition(a)
-- 	variable_addition(a, b)
-- 	variable_addition(a, b, c)

Similarly, table functions support this behavior as well:

CREATE MACRO variable_addition_range
	(a) AS TABLE (FROM range(a)),
	(a, b) AS TABLE (FROM range(a + b)),
	(a, b, c) AS TABLE (FROM range(a + b + c));

SELECT COUNT(*) FROM variable_addition_range(42, 84);
┌──────────────┐
│ count_star() │
│    int64     │
├──────────────┤
│          126 │
└──────────────┘
Serialization

In order to preserve backwards compatibility, serialization is done in two parts. The first function is serialized in the old manner (as a single function). Any extra overloads, if present, are serialized in a new field extra_functions. As such, macros without overloads are fully backwards and forwards compatible. Macros with overloads cannot be deserialized by older versions of DuckDB and lead to a serialization error.

Postgres Privilege Functions

The macro overloads are used to add support for Postgres privilege functions that we have as part of the Postgres compatibility layer.

select has_table_privilege('table', 'privilege') AS two_args, has_table_privilege(current_user, 'table', 'privilege') AS three_args;
┌──────────┬────────────┐
│ two_args │ three_args │
│ booleanboolean   │
├──────────┼────────────┤
│ true     │ true       │
└──────────┴────────────┘
array_ptr

This PR also adds the array_ptr class. The array_ptr is a non-owning pointer to an array of objects with a count. It is by default bounds-checked, and can be iterated over similar to a vector. There is also an unsafe_array_ptr that is not bounds checked. The idea is that this pointer class can be used instead of raw pointers to represent arrays in a safer manner throughout the system.

@Mytherin Mytherin requested a review from lnkuiper July 18, 2024 13:36
@duckdb-draftbot duckdb-draftbot marked this pull request as draft July 18, 2024 14:01
@Mytherin Mytherin marked this pull request as ready for review July 18, 2024 14:01
@Mytherin Mytherin added the Needs Documentation Use for issues or PRs that require changes in the documentation label Jul 18, 2024
@duckdb-draftbot duckdb-draftbot marked this pull request as draft July 19, 2024 10:27
@Mytherin Mytherin marked this pull request as ready for review July 19, 2024 10:27
@Mytherin Mytherin merged commit 18254ec into duckdb:main Jul 19, 2024
github-actions bot pushed a commit to duckdb/duckdb-r that referenced this pull request Jul 19, 2024
Merge pull request duckdb/duckdb#13062 from Mytherin/macrooverloads
@Mytherin Mytherin deleted the macrooverloads branch August 4, 2024 08:31
Mytherin added a commit that referenced this pull request Sep 1, 2025
This PR implements (optional) types for macro parameters.

#13062 Implemented macro overloads:
```sql
CREATE MACRO m
	(a) AS a,
	(a, b) AS a + b,
	(a, b, c) AS a + b + c;
SELECT m(40, 2);
-- 42
```
Allowing different behavior based on the supplied arguments.

More recently, #18684 improved how
arguments can be supplied to macro's:
```sql
CREATE MACRO m(a, b := '2') AS a || b;
SELECT m(a := '4');
-- 42
```
Allowing parameters without a default to be used by name, and parameters
with a default to be used by position.

This PR implements typed macro parameters, allowing macro's with the
same number of parameters to be overloaded by type:
```sql
CREATE OR REPLACE MACRO m
    (a TINYINT) AS a + 1,
    (a SMALLINT) AS a + 2,
    (a INTEGER) AS a + 3,
    (a BIGINT) AS a + 4,
    (a HUGEINT) AS a + 5,
    (a) AS a + 10;
 SELECT m(39::INTEGER) i;
-- 42
```
The macro overload is selected similar to our regular functions, and
typed parameters can be mixed with untyped parameters:
```sql
CREATE MACRO m
    (a VARCHAR) AS a || '2',
    (a INTEGER, b) AS a + b,
    (a, b INTEGER) AS a - b;
SELECT m(4);
-- Macro m() does not support the supplied arguments. You might need to add explicit type casts.
SELECT m(4::VARCHAR);
-- 42
SELECT m(40, 2);
-- Macro m() has multiple overloads that match the supplied arguments.
SELECT m(40, 2::TINYINT);
-- 42
SELECT m(44::TINYINT, 2);
-- 42
```
Explicit types can also be supplied for parameters with default values:
```sql
CREATE MACRO m(a VARCHAR := 42) AS a;
-- Default value '42' for parameter 'a' cannot be implicitly cast to 'VARCHAR'. Please add an explicit type cast.
CREATE MACRO m(a VARCHAR := '42') AS a;
SELECT m();
-- 42
SELECT m(42);
-- Macro m() does not support the supplied arguments. You might need to add explicit type casts.
```

This is useful for when different types require different behaviour, or
when you don't want to accidentally supply an argument with a certain
type to a function (which happens to me all the time in Python!).

I've also significantly improved the binding process of macro's in
general, written more descriptive error messages, and unified a lot of
the scalar macro binding code with table macro binding code. All of the
tricks above should also work for table macro's.

Note that this is only supported with in-memory databases, temporary
macro's, or with v1.4.0 storage or later:
```sh
# in-memory DB
duckdb -c "CREATE MACRO m(a INTEGER) AS a;"
# works

# disk DB
duckdb my.duckdb -c "CREATE MACRO m(a INTEGER) AS a;"
# Typed macro parameters are only supported for storage versions v1.4.0 and higher.

# temporary macro for disk DB
duckdb my.duckdb -c "CREATE TEMPORARY MACRO m(a INTEGER) AS a;"
# works

# explicit v1.4.0 storage
duckdb -c "ATTACH 'my.duckdb' AS my (STORAGE_VERSION 'v1.4.0'); USE my; CREATE MACRO m(a INTEGER) AS a;"
# works
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Documentation Use for issues or PRs that require changes in the documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants