Skip to content

Crash in duckdb::IsNullValue - STRING_AGG #425

@wmann-celonis

Description

@wmann-celonis

Hi,

I'm executing the following query on a very large table e (c varchar(128) not null, a varchar(128) not null, ev bigint not null):

SELECT c, STRING_AGG ( a , ',' ) FROM e GROUP BY c;

On my dataset, this query crashes, so I compiles with CMAKE_BUILD_TYPE=Debug and got address sanitizer output:

./duckdb_cli ../build2/bigdata.duck < ../build2/variant_equivalent.sql
=================================================================
==26423==ERROR: AddressSanitizer: heap-use-after-free on address 0x6210000639f7 at pc 0x55dfc5b436ef bp 0x7ffd4f5ab2b0 sp 0x7ffd4f5ab2a0
READ of size 1 at 0x6210000639f7 thread T0
    #0 0x55dfc5b436ee in bool duckdb::IsNullValue<char const*>(char const*) /home/wmann/duckdb/duckdb/src/include/duckdb/common/types/null_value.hpp:39
    #1 0x55dfc628b490 in operator() /home/wmann/duckdb/duckdb/src/function/aggregate/distributive/string_agg.cpp:36
    #2 0x55dfc629070e in Exec<duckdb::string_agg_update(duckdb::Vector*, duckdb::index_t, duckdb::Vector&)::<lambda(duckdb::index_t, duckdb::index_t)>&> /home/wmann/duckdb/duckdb/src/include/duckdb/common/vector_operations/vector_operations.hpp:197
    #3 0x55dfc628dabe in Exec<duckdb::string_agg_update(duckdb::Vector*, duckdb::index_t, duckdb::Vector&)::<lambda(duckdb::index_t, duckdb::index_t)> > /home/wmann/duckdb/duckdb/src/include/duckdb/common/vector_operations/vector_operations.hpp:217
    #4 0x55dfc628bb4b in duckdb::string_agg_update(duckdb::Vector*, unsigned long, duckdb::Vector&) /home/wmann/duckdb/duckdb/src/function/aggregate/distributive/string_agg.cpp:28
    #5 0x55dfc5ee9536 in duckdb::SuperLargeHashTable::AddChunk(duckdb::DataChunk&, duckdb::DataChunk&) /home/wmann/duckdb/duckdb/src/execution/aggregate_hashtable.cpp:217
    #6 0x55dfc65526ab in duckdb::PhysicalHashAggregate::GetChunkInternal(duckdb::ClientContext&, duckdb::DataChunk&, duckdb::PhysicalOperatorState*) /home/wmann/duckdb/duckdb/src/execution/operator/aggregate/physical_hash_aggregate.cpp:95
    #7 0x55dfc63aa34f in duckdb::PhysicalOperator::GetChunk(duckdb::ClientContext&, duckdb::DataChunk&, duckdb::PhysicalOperatorState*) /home/wmann/duckdb/duckdb/src/execution/physical_operator.cpp:45
    #8 0x55dfc6590fd6 in duckdb::PhysicalProjection::GetChunkInternal(duckdb::ClientContext&, duckdb::DataChunk&, duckdb::PhysicalOperatorState*) /home/wmann/duckdb/duckdb/src/execution/operator/projection/physical_projection.cpp:22
    #9 0x55dfc63aa34f in duckdb::PhysicalOperator::GetChunk(duckdb::ClientContext&, duckdb::DataChunk&, duckdb::PhysicalOperatorState*) /home/wmann/duckdb/duckdb/src/execution/physical_operator.cpp:45
    #10 0x55dfc6563cfa in duckdb::PhysicalExecute::GetChunkInternal(duckdb::ClientContext&, duckdb::DataChunk&, duckdb::PhysicalOperatorState*) /home/wmann/duckdb/duckdb/src/execution/operator/helper/physical_execute.cpp:8
    #11 0x55dfc63aa34f in duckdb::PhysicalOperator::GetChunk(duckdb::ClientContext&, duckdb::DataChunk&, duckdb::PhysicalOperatorState*) /home/wmann/duckdb/duckdb/src/execution/physical_operator.cpp:45
    #12 0x55dfc614952d in duckdb::ClientContext::FetchInternal() /home/wmann/duckdb/duckdb/src/main/client_context.cpp:150
    #13 0x55dfc614adbc in duckdb::ClientContext::ExecutePreparedStatement(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, duckdb::PreparedStatementData&, std::vector<duckdb::Value, std::allocator<duckdb::Value> >, bool) /home/wmann/duckdb/duckdb/src/main/client_context.cpp:229
    #14 0x55dfc614d7cd in duckdb::ClientContext::RunStatementInternal(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unique_ptr<duckdb::SQLStatement, std::default_delete<duckdb::SQLStatement> >, bool) /home/wmann/duckdb/duckdb/src/main/client_context.cpp:329
    #15 0x55dfc614e136 in duckdb::ClientContext::RunStatement(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unique_ptr<duckdb::SQLStatement, std::default_delete<duckdb::SQLStatement> >, bool) /home/wmann/duckdb/duckdb/src/main/client_context.cpp:356
    #16 0x55dfc614cb22 in duckdb::ClientContext::Execute(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::vector<duckdb::Value, std::allocator<duckdb::Value> >&, bool) /home/wmann/duckdb/duckdb/src/main/client_context.cpp:300
    #17 0x55dfc5f19184 in duckdb::PreparedStatement::Execute(std::vector<duckdb::Value, std::allocator<duckdb::Value> >&, bool) /home/wmann/duckdb/duckdb/src/main/prepared_statement.cpp:36
    #18 0x55dfc5aaa9e6 in sqlite3_step /home/wmann/duckdb/duckdb/tools/sqlite3_api_wrapper/sqlite3_api_wrapper.cpp:197
    #19 0x55dfc5a84b65 in exec_prepared_stmt /home/wmann/duckdb/duckdb/tools/shell/shell.c:10135
    #20 0x55dfc5a86949 in shell_exec /home/wmann/duckdb/duckdb/tools/shell/shell.c:10434
    #21 0x55dfc5a9fd3e in runOneSqlLine /home/wmann/duckdb/duckdb/tools/shell/shell.c:15341
    #22 0x55dfc5aa0684 in process_input /home/wmann/duckdb/duckdb/tools/shell/shell.c:15442
    #23 0x55dfc5aa2e0d in main /home/wmann/duckdb/duckdb/tools/shell/shell.c:16114
    #24 0x7f87bc650b96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
    #25 0x55dfc5a600c9 in _start (/home/wmann/duckdb/duckdb/build-dbg/duckdb_cli+0x80b0c9)

0x6210000639f7 is located 3319 bytes inside of 4096-byte region [0x621000062d00,0x621000063d00)
freed by thread T0 here:
    #0 0x7f87bd440490 in operator delete[](void*) (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xe1490)
    #1 0x55dfc5ab55fe in std::enable_if<std::is_convertible<char (*) [], char (*) []>::value, void>::type std::default_delete<char []>::operator()<char>(char*) const /usr/include/c++/8/bits/unique_ptr.h:115
    #2 0x55dfc5ab1b4c in std::unique_ptr<char [], std::default_delete<char []> >::~unique_ptr() /usr/include/c++/8/bits/unique_ptr.h:533
    #3 0x55dfc5d73a1b in duckdb::StringHeap::StringChunk::~StringChunk() /home/wmann/duckdb/duckdb/src/include/duckdb/common/types/string_heap.hpp:48
    #4 0x55dfc5de481f in std::default_delete<duckdb::StringHeap::StringChunk>::operator()(duckdb::StringHeap::StringChunk*) const /usr/include/c++/8/bits/unique_ptr.h:81
    #5 0x55dfc5dde1b8 in std::unique_ptr<duckdb::StringHeap::StringChunk, std::default_delete<duckdb::StringHeap::StringChunk> >::~unique_ptr() /usr/include/c++/8/bits/unique_ptr.h:274
    #6 0x55dfc5e127c5 in duckdb::StringHeap::~StringHeap() /home/wmann/duckdb/duckdb/src/include/duckdb/common/types/string_heap.hpp:18
    #7 0x55dfc611b5d2 in duckdb::VectorStringBuffer::~VectorStringBuffer() /home/wmann/duckdb/duckdb/src/include/duckdb/common/types/vector_buffer.hpp:49
    #8 0x55dfc5ae923f in void __gnu_cxx::new_allocator<duckdb::VectorStringBuffer>::destroy<duckdb::VectorStringBuffer>(duckdb::VectorStringBuffer*) (/home/wmann/duckdb/duckdb/build-dbg/duckdb_cli+0x89423f)
    #9 0x55dfc5ae918a in void std::allocator_traits<std::allocator<duckdb::VectorStringBuffer> >::destroy<duckdb::VectorStringBuffer>(std::allocator<duckdb::VectorStringBuffer>&, duckdb::VectorStringBuffer*) (/home/wmann/duckdb/duckdb/build-dbg/duckdb_cli+0x89418a)
    #10 0x55dfc5ae8cbc in std::_Sp_counted_ptr_inplace<duckdb::VectorStringBuffer, std::allocator<duckdb::VectorStringBuffer>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() /usr/include/c++/8/bits/shared_ptr_base.h:554
    #11 0x55dfc5ab58dc in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /usr/include/c++/8/bits/shared_ptr_base.h:155
    #12 0x55dfc5ab1c3d in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /usr/include/c++/8/bits/shared_ptr_base.h:728
    #13 0x55dfc5ab0847 in std::__shared_ptr<duckdb::VectorBuffer, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /usr/include/c++/8/bits/shared_ptr_base.h:1167
    #14 0x55dfc5ae0b42 in std::__shared_ptr<duckdb::VectorBuffer, (__gnu_cxx::_Lock_policy)2>::reset() (/home/wmann/duckdb/duckdb/build-dbg/duckdb_cli+0x88bb42)
    #15 0x55dfc5ad7270 in duckdb::Vector::Initialize(duckdb::TypeId, bool, unsigned long) /home/wmann/duckdb/duckdb/src/common/types/vector.cpp:87
    #16 0x55dfc610ca40 in duckdb::DataChunk::Reset() /home/wmann/duckdb/duckdb/src/common/types/data_chunk.cpp:33
    #17 0x55dfc655245f in duckdb::PhysicalHashAggregate::GetChunkInternal(duckdb::ClientContext&, duckdb::DataChunk&, duckdb::PhysicalOperatorState*) /home/wmann/duckdb/duckdb/src/execution/operator/aggregate/physical_hash_aggregate.cpp:71
    #18 0x55dfc63aa34f in duckdb::PhysicalOperator::GetChunk(duckdb::ClientContext&, duckdb::DataChunk&, duckdb::PhysicalOperatorState*) /home/wmann/duckdb/duckdb/src/execution/physical_operator.cpp:45
    #19 0x55dfc6590fd6 in duckdb::PhysicalProjection::GetChunkInternal(duckdb::ClientContext&, duckdb::DataChunk&, duckdb::PhysicalOperatorState*) /home/wmann/duckdb/duckdb/src/execution/operator/projection/physical_projection.cpp:22
    #20 0x55dfc63aa34f in duckdb::PhysicalOperator::GetChunk(duckdb::ClientContext&, duckdb::DataChunk&, duckdb::PhysicalOperatorState*) /home/wmann/duckdb/duckdb/src/execution/physical_operator.cpp:45
    #21 0x55dfc6563cfa in duckdb::PhysicalExecute::GetChunkInternal(duckdb::ClientContext&, duckdb::DataChunk&, duckdb::PhysicalOperatorState*) /home/wmann/duckdb/duckdb/src/execution/operator/helper/physical_execute.cpp:8
    #22 0x55dfc63aa34f in duckdb::PhysicalOperator::GetChunk(duckdb::ClientContext&, duckdb::DataChunk&, duckdb::PhysicalOperatorState*) /home/wmann/duckdb/duckdb/src/execution/physical_operator.cpp:45
    #23 0x55dfc614952d in duckdb::ClientContext::FetchInternal() /home/wmann/duckdb/duckdb/src/main/client_context.cpp:150
    #24 0x55dfc614adbc in duckdb::ClientContext::ExecutePreparedStatement(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, duckdb::PreparedStatementData&, std::vector<duckdb::Value, std::allocator<duckdb::Value> >, bool) /home/wmann/duckdb/duckdb/src/main/client_context.cpp:229
    #25 0x55dfc614d7cd in duckdb::ClientContext::RunStatementInternal(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unique_ptr<duckdb::SQLStatement, std::default_delete<duckdb::SQLStatement> >, bool) /home/wmann/duckdb/duckdb/src/main/client_context.cpp:329
    #26 0x55dfc614e136 in duckdb::ClientContext::RunStatement(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unique_ptr<duckdb::SQLStatement, std::default_delete<duckdb::SQLStatement> >, bool) /home/wmann/duckdb/duckdb/src/main/client_context.cpp:356
    #27 0x55dfc614cb22 in duckdb::ClientContext::Execute(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::vector<duckdb::Value, std::allocator<duckdb::Value> >&, bool) /home/wmann/duckdb/duckdb/src/main/client_context.cpp:300
    #28 0x55dfc5f19184 in duckdb::PreparedStatement::Execute(std::vector<duckdb::Value, std::allocator<duckdb::Value> >&, bool) /home/wmann/duckdb/duckdb/src/main/prepared_statement.cpp:36
    #29 0x55dfc5aaa9e6 in sqlite3_step /home/wmann/duckdb/duckdb/tools/sqlite3_api_wrapper/sqlite3_api_wrapper.cpp:197

previously allocated by thread T0 here:
    #0 0x7f87bd43f618 in operator new[](unsigned long) (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xe0618)
    #1 0x55dfc611698b in duckdb::StringHeap::StringChunk::StringChunk(unsigned long) /home/wmann/duckdb/duckdb/src/include/duckdb/common/types/string_heap.hpp:46
    #2 0x55dfc6116b7e in std::unique_ptr<duckdb::StringHeap::StringChunk, std::default_delete<duckdb::StringHeap::StringChunk> > duckdb::make_unique<duckdb::StringHeap::StringChunk, unsigned long const&>(unsigned long const&) /home/wmann/duckdb/duckdb/src/include/duckdb/common/helper.hpp:24
    #3 0x55dfc6116302 in duckdb::StringHeap::AddString(char const*, unsigned long) /home/wmann/duckdb/duckdb/src/common/types/string_heap.cpp:22
    #4 0x55dfc5ae02aa in duckdb::VectorStringBuffer::AddString(char const*, unsigned long) (/home/wmann/duckdb/duckdb/build-dbg/duckdb_cli+0x88b2aa)
    #5 0x55dfc5add3a0 in duckdb::Vector::AddString(char const*, unsigned long) /home/wmann/duckdb/duckdb/src/common/types/vector.cpp:413
    #6 0x55dfc5add421 in duckdb::Vector::AddString(char const*) /home/wmann/duckdb/duckdb/src/common/types/vector.cpp:417
    #7 0x55dfc628b674 in operator() /home/wmann/duckdb/duckdb/src/function/aggregate/distributive/string_agg.cpp:42
    #8 0x55dfc629070e in Exec<duckdb::string_agg_update(duckdb::Vector*, duckdb::index_t, duckdb::Vector&)::<lambda(duckdb::index_t, duckdb::index_t)>&> /home/wmann/duckdb/duckdb/src/include/duckdb/common/vector_operations/vector_operations.hpp:197
    #9 0x55dfc628dabe in Exec<duckdb::string_agg_update(duckdb::Vector*, duckdb::index_t, duckdb::Vector&)::<lambda(duckdb::index_t, duckdb::index_t)> > /home/wmann/duckdb/duckdb/src/include/duckdb/common/vector_operations/vector_operations.hpp:217
    #10 0x55dfc628bb4b in duckdb::string_agg_update(duckdb::Vector*, unsigned long, duckdb::Vector&) /home/wmann/duckdb/duckdb/src/function/aggregate/distributive/string_agg.cpp:28
    #11 0x55dfc5ee9536 in duckdb::SuperLargeHashTable::AddChunk(duckdb::DataChunk&, duckdb::DataChunk&) /home/wmann/duckdb/duckdb/src/execution/aggregate_hashtable.cpp:217
    #12 0x55dfc65526ab in duckdb::PhysicalHashAggregate::GetChunkInternal(duckdb::ClientContext&, duckdb::DataChunk&, duckdb::PhysicalOperatorState*) /home/wmann/duckdb/duckdb/src/execution/operator/aggregate/physical_hash_aggregate.cpp:95
    #13 0x55dfc63aa34f in duckdb::PhysicalOperator::GetChunk(duckdb::ClientContext&, duckdb::DataChunk&, duckdb::PhysicalOperatorState*) /home/wmann/duckdb/duckdb/src/execution/physical_operator.cpp:45
    #14 0x55dfc6590fd6 in duckdb::PhysicalProjection::GetChunkInternal(duckdb::ClientContext&, duckdb::DataChunk&, duckdb::PhysicalOperatorState*) /home/wmann/duckdb/duckdb/src/execution/operator/projection/physical_projection.cpp:22
    #15 0x55dfc63aa34f in duckdb::PhysicalOperator::GetChunk(duckdb::ClientContext&, duckdb::DataChunk&, duckdb::PhysicalOperatorState*) /home/wmann/duckdb/duckdb/src/execution/physical_operator.cpp:45
    #16 0x55dfc6563cfa in duckdb::PhysicalExecute::GetChunkInternal(duckdb::ClientContext&, duckdb::DataChunk&, duckdb::PhysicalOperatorState*) /home/wmann/duckdb/duckdb/src/execution/operator/helper/physical_execute.cpp:8
    #17 0x55dfc63aa34f in duckdb::PhysicalOperator::GetChunk(duckdb::ClientContext&, duckdb::DataChunk&, duckdb::PhysicalOperatorState*) /home/wmann/duckdb/duckdb/src/execution/physical_operator.cpp:45
    #18 0x55dfc614952d in duckdb::ClientContext::FetchInternal() /home/wmann/duckdb/duckdb/src/main/client_context.cpp:150
    #19 0x55dfc614adbc in duckdb::ClientContext::ExecutePreparedStatement(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, duckdb::PreparedStatementData&, std::vector<duckdb::Value, std::allocator<duckdb::Value> >, bool) /home/wmann/duckdb/duckdb/src/main/client_context.cpp:229
    #20 0x55dfc614d7cd in duckdb::ClientContext::RunStatementInternal(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unique_ptr<duckdb::SQLStatement, std::default_delete<duckdb::SQLStatement> >, bool) /home/wmann/duckdb/duckdb/src/main/client_context.cpp:329
    #21 0x55dfc614e136 in duckdb::ClientContext::RunStatement(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unique_ptr<duckdb::SQLStatement, std::default_delete<duckdb::SQLStatement> >, bool) /home/wmann/duckdb/duckdb/src/main/client_context.cpp:356
    #22 0x55dfc614cb22 in duckdb::ClientContext::Execute(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::vector<duckdb::Value, std::allocator<duckdb::Value> >&, bool) /home/wmann/duckdb/duckdb/src/main/client_context.cpp:300
    #23 0x55dfc5f19184 in duckdb::PreparedStatement::Execute(std::vector<duckdb::Value, std::allocator<duckdb::Value> >&, bool) /home/wmann/duckdb/duckdb/src/main/prepared_statement.cpp:36
    #24 0x55dfc5aaa9e6 in sqlite3_step /home/wmann/duckdb/duckdb/tools/sqlite3_api_wrapper/sqlite3_api_wrapper.cpp:197
    #25 0x55dfc5a84b65 in exec_prepared_stmt /home/wmann/duckdb/duckdb/tools/shell/shell.c:10135
    #26 0x55dfc5a86949 in shell_exec /home/wmann/duckdb/duckdb/tools/shell/shell.c:10434
    #27 0x55dfc5a9fd3e in runOneSqlLine /home/wmann/duckdb/duckdb/tools/shell/shell.c:15341
    #28 0x55dfc5aa0684 in process_input /home/wmann/duckdb/duckdb/tools/shell/shell.c:15442
    #29 0x55dfc5aa2e0d in main /home/wmann/duckdb/duckdb/tools/shell/shell.c:16114

SUMMARY: AddressSanitizer: heap-use-after-free /home/wmann/duckdb/duckdb/src/include/duckdb/common/types/null_value.hpp:39 in bool duckdb::IsNullValue<char const*>(char const*)
Shadow bytes around the buggy address:
  0x0c42800046e0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c42800046f0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4280004700: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4280004710: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4280004720: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
=>0x0c4280004730: fd fd fd fd fd fd fd fd fd fd fd fd fd fd[fd]fd
  0x0c4280004740: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4280004750: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4280004760: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4280004770: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4280004780: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==26423==ABORTING

Let me know if you need further information.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions