Investigating new duckplyr #13

Tmonster · 2022-12-15T09:58:24Z

This should address the issue found by Kirill in tidyverse/duckplyr#1. I don't know if it is "logically" correct however. Here I've provided an explanation of my thinking and feedback I'm looking for

To maintain parity with dplyr, we need to return only 1 row when there is no .by term. When duckplyr_summarise has no .by term, there are no elements in the groups list passed to the function rapi_rel_aggregate. This is equivalent to not having a group by clause in a SQL statement.

> df2
# A tibble: 5 × 2
      g     x
  <dbl> <int>
1     1     1
2     1     2
3     2     3
4     2     4
5     2     5

> duckplyr:::duckplyr_summarise(df2, out=sum(x))
# A tibble: 1 × 1
    out
  <dbl>
1     15

> duckplyr:::duckplyr_summarise(df2, out=1)
# A tibble: 5 × 1
    out
  <dbl>
1     1
2     1
3     1
4     1
5     1
# The above statement compared to this one shows where the issues are.
# statement dplyr good
> summarise(df2,  out=1)
# A tibble: 1 × 1
    out
  <dbl>
1     1

> duckplyr:::duckplyr_summarize(df2, .by=g, out=sum(x))
# A tibble: 2 × 2
      g   out
  <dbl> <dbl>
1     1     3
2     2    12

What I am confused about is that dckplyr:::duckplyr_summarise(df2, out=1) translates to Select 1 from df2 which returns 1 for every row in df2. This behavoir is also present in SQLlite.

sqlite> create table t0 (c0 int, c1 varchar default "sfd");
sqlite> insert into t0 VALUES (1, "a"), (2, "b"), (3, "c"), (4,"d");
sqlite> select 1 from t0;
1
1
1
1

The fix I've proposed is that if there are no groups, we can limit the result to one row. If a user still supplies an aggregation function (e.g. sum(), max(), avg()) only one row will be returned and the limit 1 term has no effect.

Looking for feedback for

Why this might not be a good idea and how limit 1 could break some output when no groups are applied
and
If it is indeed a bad idea, what kind of select/project operator should I be wrapping the aggregate operator with?

Switch to using Laurens' radix partitioned column data collections.

Avoid recreating the RowDataCollectionScanner by adding a Reset method. Tidy up some bits of code.

…om_df or something

… dplyr is using this, technically all rows should be returned imo

Tmonster · 2022-12-15T09:58:57Z

tools/rpkg/tests/testthat/test_relational.R

+
+
+test_that("rel aggregate with no groups but a sum over a column, sums the column", {
+skip("duckplyr specific")


need to learn some more R to write these tests. Also if the duckplyr code changes we may need to change the tests, so need to think about that possibility

Add new POSITIONAL JOIN join type and plumb it through to a straw man physical implementation.

Simple benchmark joining two integer columns. This is 2x faster than a hash join on rowid.

Add a second physical operator that manges two table scans.

make format-head doesn't work unless you have added the files...

Special case rendering of physical positional scans to display the internal tables.

krlmlr

Confirming that this fixes the failing tests in tidyverse/duckplyr#1.

I'll propose tests.

Test (and fix) ragged PJ implementation.

Rename files for consistency.

Change positional scanning to have outer join semantics( NULL padding).

Change positional joining to have outer join semantics (NULL padding).

Issue duckdb#3423: Positional Join Operator

docs: add nodejs connection args to docs

[SQL Logic Test] Add support for environment variables

@Mause

Thanks @Mause Co-authored-by: Elliana May <me@mause.me>

This avoids some unnecessary rebuilding time

Ci: scope / remove env variables

Restore node-pre-gyp credentials

* Use partition size computer * Use ColumnDataConsumer to release data faster * Acquiesce to clangd whinging.

Python: Allow replacement scans on pyrelations, move to BoxRenderer and use pending query API for relations

Issue duckdb#5023: Window Radix Partitions

Update copyright year

Fix duckdb#5968: ignore repetition type of root schema

…ction-on-infinity-of-timestamptz Fix duckdb#5971: addition and subtraction on infinity of TIMESTAMPTZ

Regression ci

…_duckplyr

Tmonster · 2023-01-24T16:20:11Z

closing

I was investigating the following crash where a checkpoint task had its underlying resources being destroyed while it was still running. The two interesting threads are the following: ``` thread #1, name = 'duckling', stop reason = signal SIGTRAP frame #0: 0x0000ffff91bb71ec frame #1: 0x0000aaaad73a38e8 duckling`duckdb::InternalException::InternalException(this=<unavailable>, msg=<unavailable>) at exception.cpp:336:2 frame #2: 0x0000aaaad786eb68 duckling`duckdb::unique_ptr<duckdb::RowGroup, std::default_delete<duckdb::RowGroup>, true>::operator*() const [inlined] duckdb::unique_ptr<duckdb::RowGroup, std::default_delete<duckdb::RowGroup>, true>::AssertNotNull(null=<unavailable>) at unique_ptr.hpp:25:10 frame #3: 0x0000aaaad786eaf4 duckling`duckdb::unique_ptr<duckdb::RowGroup, std::default_delete<duckdb::RowGroup>, true>::operator*(this=0x0000aaacbb73e008) const at unique_ptr.hpp:34:4 frame #4: 0x0000aaaad787abbc duckling`duckdb::CheckpointTask::ExecuteTask(this=0x0000aaabec92be60) at row_group_collection.cpp:732:21 frame #5: 0x0000aaaad7756ea4 duckling`duckdb::BaseExecutorTask::Execute(this=0x0000aaabec92be60, mode=<unavailable>) at task_executor.cpp:72:3 frame #6: 0x0000aaaad7757e28 duckling`duckdb::TaskScheduler::ExecuteForever(this=0x0000aaaafda30e10, marker=0x0000aaaafda164a8) at task_scheduler.cpp:189:32 frame #7: 0x0000ffff91a031fc frame #8: 0x0000ffff91c0d5c8 thread #120, stop reason = signal 0 frame #0: 0x0000ffff91c71c24 frame #1: 0x0000ffff91e1264c frame #2: 0x0000ffff91e01888 frame #3: 0x0000ffff91e018f8 frame #4: 0x0000ffff91e01c10 frame #5: 0x0000ffff91e05afc frame #6: 0x0000ffff91e05e70 frame #7: 0x0000aaaad784b63c duckling`duckdb::RowGroup::~RowGroup() [inlined] std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release(this=<unavailable>) at shared_ptr_base.h:184:10 frame #8: 0x0000aaaad784b5b4 duckling`duckdb::RowGroup::~RowGroup() [inlined] std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count(this=<unavailable>) at shared_ptr_base.h:705:11 frame #9: 0x0000aaaad784b5ac duckling`duckdb::RowGroup::~RowGroup() [inlined] std::__shared_ptr<duckdb::ColumnData, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr(this=<unavailable>) at shared_ptr_base.h:1154:31 frame #10: 0x0000aaaad784b5ac duckling`duckdb::RowGroup::~RowGroup() [inlined] duckdb::shared_ptr<duckdb::ColumnData, true>::~shared_ptr(this=<unavailable>) at shared_ptr_ipp.hpp:115:24 frame #11: 0x0000aaaad784b5ac duckling`duckdb::RowGroup::~RowGroup() [inlined] void std::_Destroy<duckdb::shared_ptr<duckdb::ColumnData, true>>(__pointer=<unavailable>) at stl_construct.h:151:19 frame #12: 0x0000aaaad784b5ac duckling`duckdb::RowGroup::~RowGroup() [inlined] void std::_Destroy_aux<false>::__destroy<duckdb::shared_ptr<duckdb::ColumnData, true>*>(__first=<unavailable>, __last=<unavailable>) at stl_construct.h:163:6 frame #13: 0x0000aaaad784b5a0 duckling`duckdb::RowGroup::~RowGroup() [inlined] void std::_Destroy<duckdb::shared_ptr<duckdb::ColumnData, true>*>(__first=<unavailable>, __last=<unavailable>) at stl_construct.h:195:7 frame #14: 0x0000aaaad784b5a0 duckling`duckdb::RowGroup::~RowGroup() [inlined] void std::_Destroy<duckdb::shared_ptr<duckdb::ColumnData, true>*, duckdb::shared_ptr<duckdb::ColumnData, true>>(__first=<unavailable>, __last=<unavailable>, (null)=<unavailable>) at alloc_traits.h:848:7 frame #15: 0x0000aaaad784b5a0 duckling`duckdb::RowGroup::~RowGroup() [inlined] std::vector<duckdb::shared_ptr<duckdb::ColumnData, true>, std::allocator<duckdb::shared_ptr<duckdb::ColumnData, true>>>::~vector(this=<unavailable>) at stl_vector.h:680:2 frame #16: 0x0000aaaad784b5a0 duckling`duckdb::RowGroup::~RowGroup(this=<unavailable>) at row_group.cpp:83:1 frame #17: 0x0000aaaad786ee18 duckling`std::vector<duckdb::SegmentNode<duckdb::RowGroup>, std::allocator<duckdb::SegmentNode<duckdb::RowGroup>>>::~vector() [inlined] std::default_delete<duckdb::RowGroup>::operator()(this=0x0000aaacbb73e1a8, __ptr=0x0000aaab75ae7860) const at unique_ptr.h:85:2 frame #18: 0x0000aaaad786ee10 duckling`std::vector<duckdb::SegmentNode<duckdb::RowGroup>, std::allocator<duckdb::SegmentNode<duckdb::RowGroup>>>::~vector() at unique_ptr.h:361:4 frame #19: 0x0000aaaad786ee08 duckling`std::vector<duckdb::SegmentNode<duckdb::RowGroup>, std::allocator<duckdb::SegmentNode<duckdb::RowGroup>>>::~vector() [inlined] duckdb::SegmentNode<duckdb::RowGroup>::~SegmentNode(this=0x0000aaacbb73e1a0) at segment_tree.hpp:21:8 frame #20: 0x0000aaaad786ee08 duckling`std::vector<duckdb::SegmentNode<duckdb::RowGroup>, std::allocator<duckdb::SegmentNode<duckdb::RowGroup>>>::~vector() [inlined] void std::_Destroy<duckdb::SegmentNode<duckdb::RowGroup>>(__pointer=0x0000aaacbb73e1a0) at stl_construct.h:151:19 frame #21: 0x0000aaaad786ee08 duckling`std::vector<duckdb::SegmentNode<duckdb::RowGroup>, std::allocator<duckdb::SegmentNode<duckdb::RowGroup>>>::~vector() [inlined] void std::_Destroy_aux<false>::__destroy<duckdb::SegmentNode<duckdb::RowGroup>*>(__first=0x0000aaacbb73e1a0, __last=0x0000aaacbb751130) at stl_construct.h:163:6 frame #22: 0x0000aaaad786ede8 duckling`std::vector<duckdb::SegmentNode<duckdb::RowGroup>, std::allocator<duckdb::SegmentNode<duckdb::RowGroup>>>::~vector() [inlined] void std::_Destroy<duckdb::SegmentNode<duckdb::RowGroup>*>(__first=<unavailable>, __last=0x0000aaacbb751130) at stl_construct.h:195:7 frame #23: 0x0000aaaad786ede8 duckling`std::vector<duckdb::SegmentNode<duckdb::RowGroup>, std::allocator<duckdb::SegmentNode<duckdb::RowGroup>>>::~vector() [inlined] void std::_Destroy<duckdb::SegmentNode<duckdb::RowGroup>*, duckdb::SegmentNode<duckdb::RowGroup>>(__first=<unavailable>, __last=0x0000aaacbb751130, (null)=0x0000fffefc81a908) at alloc_traits.h:848:7 frame #24: 0x0000aaaad786ede8 duckling`std::vector<duckdb::SegmentNode<duckdb::RowGroup>, std::allocator<duckdb::SegmentNode<duckdb::RowGroup>>>::~vector(this=size=4883) at stl_vector.h:680:2 frame #25: 0x0000aaaad7857f74 duckling`duckdb::RowGroupCollection::Checkpoint(this=<unavailable>, writer=<unavailable>, global_stats=0x0000fffefc81a9c0) at row_group_collection.cpp:1017:1 frame #26: 0x0000aaaad788f02c duckling`duckdb::DataTable::Checkpoint(this=0x0000aaab04649e70, writer=0x0000aaab6584ea80, serializer=0x0000fffefc81ab38) at data_table.cpp:1427:14 frame #27: 0x0000aaaad7881394 duckling`duckdb::SingleFileCheckpointWriter::WriteTable(this=0x0000fffefc81b128, table=0x0000aaab023b78c0, serializer=0x0000fffefc81ab38) at checkpoint_manager.cpp:528:11 frame #28: 0x0000aaaad787ece4 duckling`duckdb::SingleFileCheckpointWriter::CreateCheckpoint() [inlined] duckdb::SingleFileCheckpointWriter::CreateCheckpoint(this=<unavailable>, obj=0x0000fffefc81ab38)::$_7::operator()(duckdb::Serializer::List&, unsigned long) const::'lambda'(duckdb::Serializer&)::operator()(duckdb::Serializer&) const at checkpoint_manager.cpp:181:43 frame #29: 0x0000aaaad787ecd8 duckling`duckdb::SingleFileCheckpointWriter::CreateCheckpoint() [inlined] void duckdb::Serializer::List::WriteObject<duckdb::SingleFileCheckpointWriter::CreateCheckpoint()::$_7::operator()(duckdb::Serializer::List&, unsigned long) const::'lambda'(duckdb::Serializer&)>(this=<unavailable>, f=(unnamed class) @ 0x0000600002cbd2b0) at serializer.hpp:385:2 frame #30: 0x0000aaaad787ecc4 duckling`duckdb::SingleFileCheckpointWriter::CreateCheckpoint() [inlined] duckdb::SingleFileCheckpointWriter::CreateCheckpoint()::$_7::operator()(this=<unavailable>, list=<unavailable>, i=2) const at checkpoint_manager.cpp:181:8 frame #31: 0x0000aaaad787ecb0 duckling`duckdb::SingleFileCheckpointWriter::CreateCheckpoint() at serializer.hpp:151:4 frame #32: 0x0000aaaad787ec94 duckling`duckdb::SingleFileCheckpointWriter::CreateCheckpoint(this=0x0000fffefc81b128) at checkpoint_manager.cpp:179:13 frame #33: 0x0000aaaad78954a8 duckling`duckdb::SingleFileStorageManager::CreateCheckpoint(this=0x0000aaaafe1de140, options=(wal_action = DONT_DELETE_WAL, action = CHECKPOINT_IF_REQUIRED, type = FULL_CHECKPOINT)) at storage_manager.cpp:365:17 frame #34: 0x0000aaaad78baac0 duckling`duckdb::DuckTransactionManager::Checkpoint(this=0x0000aaaafe167e00, context=<unavailable>, force=<unavailable>) at duck_transaction_manager.cpp:198:18 frame #35: 0x0000aaaad69d02c0 duckling`md::Server::BackgroundCheckpointIfNeeded(this=0x0000aaaafdbfe900) at server.cpp:1983:31 frame #36: 0x0000aaaadac5d3f0 duckling`std::thread::_State_impl<std::thread::_Invoker<std::tuple<md::BackgroundCronTask::Start(unsigned long)::$_0>>>::_M_run() [inlined] std::function<void ()>::operator()(this=<unavailable>) const at std_function.h:590:9 frame #37: 0x0000aaaadac5d3e0 duckling`std::thread::_State_impl<std::thread::_Invoker<std::tuple<md::BackgroundCronTask::Start(unsigned long)::$_0>>>::_M_run() [inlined] md::BackgroundCronTask::Start(unsigned long)::$_0::operator()(this=0x0000aaaafdf169a8) const at background_cron_task.cpp:25:4 frame #38: 0x0000aaaadac5d30c duckling`std::thread::_State_impl<std::thread::_Invoker<std::tuple<md::BackgroundCronTask::Start(unsigned long)::$_0>>>::_M_run() [inlined] void std::__invoke_impl<void, md::BackgroundCronTask::Start(unsigned long)::$_0>((null)=<unavailable>, __f=0x0000aaaafdf169a8) at invoke.h:61:14 frame #39: 0x0000aaaadac5d30c duckling`std::thread::_State_impl<std::thread::_Invoker<std::tuple<md::BackgroundCronTask::Start(unsigned long)::$_0>>>::_M_run() [inlined] std::__invoke_result<md::BackgroundCronTask::Start(unsigned long)::$_0>::type std::__invoke<md::BackgroundCronTask::Start(unsigned long)::$_0>(__fn=0x0000aaaafdf169a8) at invoke.h:96:14 frame #40: 0x0000aaaadac5d30c duckling`std::thread::_State_impl<std::thread::_Invoker<std::tuple<md::BackgroundCronTask::Start(unsigned long)::$_0>>>::_M_run() [inlined] void std::thread::_Invoker<std::tuple<md::BackgroundCronTask::Start(unsigned long)::$_0>>::_M_invoke<0ul>(this=0x0000aaaafdf169a8, (null)=<unavailable>) at std_thread.h:259:13 frame #41: 0x0000aaaadac5d30c duckling`std::thread::_State_impl<std::thread::_Invoker<std::tuple<md::BackgroundCronTask::Start(unsigned long)::$_0>>>::_M_run() [inlined] std::thread::_Invoker<std::tuple<md::BackgroundCronTask::Start(unsigned long)::$_0>>::operator()(this=0x0000aaaafdf169a8) at std_thread.h:266:11 frame #42: 0x0000aaaadac5d30c duckling`std::thread::_State_impl<std::thread::_Invoker<std::tuple<md::BackgroundCronTask::Start(unsigned long)::$_0>>>::_M_run(this=0x0000aaaafdf169a0) at std_thread.h:211:13 frame #43: 0x0000ffff91a031fc frame #44: 0x0000ffff91c0d5c8 ``` The problem is that if there's an IO exception being thrown in `RowGroupCollection::Checkpoint` after some (but not all) checkpoint tasks have been scheduled but before `checkpoint_state.executor.WorkOnTasks();` is called, it results in an InternalException / DuckDB crash as the `Checkpoint ` method does not wait for the scheduled tasks to have completed before destroying the referenced resources.

We had two users crash with the following backtrace: ``` frame #0: 0x0000ffffab2571ec frame #1: 0x0000aaaaac00c5fc duckling`duckdb::InternalException::InternalException(this=<unavailable>, msg=<unavailable>) at exception.cpp:328:2 frame #2: 0x0000aaaaac1ee418 duckling`duckdb::optional_ptr<duckdb::OptimisticDataWriter, true>::CheckValid(this=<unavailable>) const at optional_ptr.hpp:34:11 frame #3: 0x0000aaaaac1eea8c duckling`duckdb::MergeCollectionTask::Execute(duckdb::PhysicalBatchInsert const&, duckdb::ClientContext&, duckdb::GlobalSinkState&, duckdb::LocalSinkState&) [inlined] duckdb::optional_ptr<duckdb::OptimisticDataWriter, true>::operator*(this=<unavailable>) at optional_ptr.hpp:43:3 frame #4: 0x0000aaaaac1eea84 duckling`duckdb::MergeCollectionTask::Execute(this=0x0000aaaaf1b06150, op=<unavailable>, context=0x0000aaaba820d8d0, gstate_p=0x0000aaab06880f00, lstate_p=<unavailable>) at physical_batch_insert.cpp:219:90 frame #5: 0x0000aaaaac1d2e10 duckling`duckdb::PhysicalBatchInsert::Sink(duckdb::ExecutionContext&, duckdb::DataChunk&, duckdb::OperatorSinkInput&) const [inlined] duckdb::PhysicalBatchInsert::ExecuteTask(this=0x0000aaaafa62ab40, context=<unavailable>, gstate_p=0x0000aaab06880f00, lstate_p=0x0000aab12d442960) const at physical_batch_insert.cpp:425:8 frame #6: 0x0000aaaaac1d2dd8 duckling`duckdb::PhysicalBatchInsert::Sink(duckdb::ExecutionContext&, duckdb::DataChunk&, duckdb::OperatorSinkInput&) const [inlined] duckdb::PhysicalBatchInsert::ExecuteTasks(this=0x0000aaaafa62ab40, context=<unavailable>, gstate_p=0x0000aaab06880f00, lstate_p=0x0000aab12d442960) const at physical_batch_insert.cpp:431:9 frame #7: 0x0000aaaaac1d2dd8 duckling`duckdb::PhysicalBatchInsert::Sink(this=0x0000aaaafa62ab40, context=0x0000aab2fffd7cb0, chunk=<unavailable>, input=<unavailable>) const at physical_batch_insert.cpp:494:4 frame #8: 0x0000aaaaac353158 duckling`duckdb::PipelineExecutor::ExecutePushInternal(duckdb::DataChunk&, duckdb::ExecutionBudget&, unsigned long) [inlined] duckdb::PipelineExecutor::Sink(this=0x0000aab2fffd7c00, chunk=0x0000aab2fffd7d30, input=0x0000fffec0aba8d8) at pipeline_executor.cpp:521:24 frame #9: 0x0000aaaaac353130 duckling`duckdb::PipelineExecutor::ExecutePushInternal(this=0x0000aab2fffd7c00, input=0x0000aab2fffd7d30, chunk_budget=0x0000fffec0aba980, initial_idx=0) at pipeline_executor.cpp:332:23 frame #10: 0x0000aaaaac34f7b4 duckling`duckdb::PipelineExecutor::Execute(this=0x0000aab2fffd7c00, max_chunks=<unavailable>) at pipeline_executor.cpp:201:13 frame #11: 0x0000aaaaac34f258 duckling`duckdb::PipelineTask::ExecuteTask(duckdb::TaskExecutionMode) [inlined] duckdb::PipelineExecutor::Execute(this=<unavailable>) at pipeline_executor.cpp:278:9 frame #12: 0x0000aaaaac34f250 duckling`duckdb::PipelineTask::ExecuteTask(this=0x0000aab16dafd630, mode=<unavailable>) at pipeline.cpp:51:33 frame #13: 0x0000aaaaac348298 duckling`duckdb::ExecutorTask::Execute(this=0x0000aab16dafd630, mode=<unavailable>) at executor_task.cpp:49:11 frame #14: 0x0000aaaaac356600 duckling`duckdb::TaskScheduler::ExecuteForever(this=0x0000aaaaf0105560, marker=0x0000aaaaf00ee578) at task_scheduler.cpp:189:32 frame #15: 0x0000ffffab0a31fc frame #16: 0x0000ffffab2ad5c8 ``` Core dump analysis showed that the assertion `D_ASSERT(lstate.writer);` in `MergeCollectionTask::Execute` (i.e. it is crashing because `lstate.writer` is NULLPTR) was not satisfied when `PhysicalBatchInsert::Sink` was processing merge tasks from (other) pipeline executors. My suspicion is that this is only likely to happen for heavily concurrent workloads (applicable to the two users which crashed). The patch submitted as part of this PR has addressed the issue for these users.

Lindsay Wray and others added 9 commits November 7, 2022 13:40

functional setup

52fb339

merge conflict

4a20707

Issue duckdb#5023: Radix Partition Upgrade

53b2e36

Switch to using Laurens' radix partitioned column data collections.

Issue duckdb#5023: Radix Partition Cleanup

1c83398

Avoid recreating the RowDataCollectionScanner by adding a Reset method. Tidy up some bits of code.

Merge branch 'feature' into window-partitions

c397f26

just some std::cout. think this fix will have to take place in rel_fr…

9452fc7

…om_df or something

Seems to fit, but I think we have to add some kind of check to see if…

db04316

… dplyr is using this, technically all rows should be returned imo

add stubs for tests, remove debug statements

58efaaa

remove std::iostream

010d0fa

Tmonster commented Dec 15, 2022

View reviewed changes

Lindsay Wray and others added 11 commits December 22, 2022 15:04

map alterations

161cd1a

rm comments

714df12

docs: add nodejs connection docs

362cb3f

docs: include access_mode

ad237fc

Merge branch 'master' into window-partitions

22c1b22

Issue duckdb#3423: Positional Join Parsing

55bb00d

Add new POSITIONAL JOIN join type and plumb it through to a straw man physical implementation.

Merge branch 'master' into positional-join

481735c

Issue duckdb#3423: Positional Join Benchmark

6f9c655

Simple benchmark joining two integer columns. This is 2x faster than a hash join on rowid.

Issue duckdb#3423: Positional Join Scans

9e945bc

Add a second physical operator that manges two table scans.

Issue duckdb#3423: Positional Scans Formatting

684bddd

make format-head doesn't work unless you have added the files...

Issue duckdb#3423: Positional Scans Rendering

20dc7af

Special case rendering of physical positional scans to display the internal tables.

krlmlr approved these changes Jan 5, 2023

View reviewed changes

krlmlr mentioned this pull request Jan 5, 2023

Implement summarise() tidyverse/duckplyr#1

Merged

Richard Wesley and others added 7 commits January 5, 2023 11:35

Issue duckdb#3423: Ragged Positional Join

451c6d3

Test (and fix) ragged PJ implementation.

Issue duckdb#3423: Positional Join Rename

94af819

Rename files for consistency.

Issue duckdb#3423: Outer Positional Scan

fdd0b75

Change positional scanning to have outer join semantics( NULL padding).

Issue duckdb#3423: Outer Positional Scan

2485d36

Change positional joining to have outer join semantics (NULL padding).

Restore ICU time_bucket

d39c166

Delete non-ICU time_bucket

516c5d2

cast finished

7b1e05a

Mytherin and others added 26 commits January 23, 2023 09:55

Merge pull request duckdb#5867 from hawkfish/positional-join

484c476

Issue duckdb#3423: Positional Join Operator

Merge pull request duckdb#5780 from tshauck/add-config-to-nodejs-docs

ae49ea0

docs: add nodejs connection args to docs

Merge pull request duckdb#5877 from Tishj/sqllogictest_env

2fd5e0d

[SQL Logic Test] Add support for environment variables

Apply suggestions from code review

5085477

Thanks @Mause Co-authored-by: Elliana May <me@mause.me>

CI: scope or remove env variables

2a3f7e0

CI: scope or remove env variables/2

5a12e91

CI: Serialize regression workflow

0ba4d25

This avoids some unnecessary rebuilding time

CI: Fix scripts/node_version.sh to work on forks

6a6f83e

CI: Fix amalgamation.py to avoid failure when tag are not defined

77979b2

CI: scope or remove env variables/3

e3b37c2

CI: scope or remove env variables/4

83e9a45

Merge pull request duckdb#5967 from carlopi/ci

218bae2

Ci: scope / remove env variables

Fix duckdb#5968: ignore repetition type of root schema

0acda57

Restore node-pre-gyp credentials

fd902b4

Merge pull request duckdb#5970 from carlopi/ci_fixup

09f803d

Restore node-pre-gyp credentials

Fix addition and subtraction on infinity of TIMESTAMPTZ

5900117

Merge branch 'master' into window-partitions

c7b86e3

update copyright year

cce08e5

Issue duckdb#5023: Radix Partition Polish

ebea446

* Use partition size computer * Use ColumnDataConsumer to release data faster * Acquiesce to clangd whinging.

Merge pull request duckdb#5962 from Mytherin/pythonimprovements

b7b7ea6

Python: Allow replacement scans on pyrelations, move to BoxRenderer and use pending query API for relations

Merge pull request duckdb#5909 from hawkfish/window-partitions

914dd8b

Issue duckdb#5023: Window Radix Partitions

Merge pull request duckdb#5974 from sjaenick/master

a27022b

Update copyright year

Merge pull request duckdb#5969 from Mytherin/issue5968

6be5116

Fix duckdb#5968: ignore repetition type of root schema

Merge pull request duckdb#5978 from papparapa/fix-addition-and-subtra…

3da63b4

…ction-on-infinity-of-timestamptz Fix duckdb#5971: addition and subtraction on infinity of TIMESTAMPTZ

Merge pull request duckdb#5961 from carlopi/regression_ci

c4290b2

Regression ci

Merge remote-tracking branch 'upstream/master' into investigating_new…

37498c2

…_duckplyr

Tmonster closed this Jan 24, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Investigating new duckplyr #13

Investigating new duckplyr #13

Uh oh!

Tmonster commented Dec 15, 2022

Uh oh!

Tmonster Dec 15, 2022 •

edited

Loading

Uh oh!

krlmlr left a comment

Uh oh!

Tmonster commented Jan 24, 2023

Uh oh!

Uh oh!



		test_that("rel aggregate with no groups but a sum over a column, sums the column", {
		skip("duckplyr specific")

Investigating new duckplyr #13

Investigating new duckplyr #13

Uh oh!

Conversation

Tmonster commented Dec 15, 2022

Uh oh!

Tmonster Dec 15, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

krlmlr left a comment

Choose a reason for hiding this comment

Uh oh!

Tmonster commented Jan 24, 2023

Uh oh!

Uh oh!

Tmonster Dec 15, 2022 •

edited

Loading