Segmentation fault when inserting and updating data in a transaction #16587
Description
What happens?
I have built an application, which inserts and updates rows in sets of 1000 within a transaction. After the db gets a certain size, it segfaults on additional updates.
One table, two indices:
CREATE TABLE IF NOT EXISTS test (id BIGINT NOT NULL, importId INTEGER NOT NULL);
CREATE INDEX IF NOT EXISTS test_import_id ON test(importId);
CREATE INDEX IF NOT EXISTS test_id ON test(id);Running batches of updates and inserts will trigger the crash after about 50k-100k operations.
for (i = 0; i < 100; i++) {
BEGIN TRANSACTION;
for (k = 0; k< 1000; k++) {
INSERT INTO test (id, importId) VALUES($id, $value);
}
for (m = 0; m < 1000; m++) {
UPDATE test SET importId = $1 WHERE id = $2;
}
COMMIT;
}
- The problem seems not to happen when not both indices are set
- The problem seems not to happen, without a transaction
To Reproduce
Please use the follwoing database to reproduce the problem: https://github.com/linaGirl/duckdb-segfault/blob/main/test.duckdb
Providing raw SQL to create the problematic database won't work since i was not able to trigger the segfault without adding a significant amount of data to the database.
run duckdb using the database ./duckdb test.duckdb
Paste the follwing sql and it will segfault
BEGIN TRANSACTION;
UPDATE test SET importId = 725 WHERE id = 34165;
UPDATE test SET importId = 663 WHERE id = 42638;
UPDATE test SET importId = 210 WHERE id = 11288;
UPDATE test SET importId = 805 WHERE id = 764;
UPDATE test SET importId = 782 WHERE id = 10151;
UPDATE test SET importId = 53 WHERE id = 3229;
UPDATE test SET importId = 725 WHERE id = 34165;
UPDATE test SET importId = 663 WHERE id = 42638;
UPDATE test SET importId = 210 WHERE id = 11288;
UPDATE test SET importId = 805 WHERE id = 764;
UPDATE test SET importId = 782 WHERE id = 10151;
UPDATE test SET importId = 53 WHERE id = 3229;
UPDATE test SET importId = 725 WHERE id = 34165;
UPDATE test SET importId = 663 WHERE id = 42638;
UPDATE test SET importId = 210 WHERE id = 11288;
UPDATE test SET importId = 805 WHERE id = 764;
UPDATE test SET importId = 782 WHERE id = 10151;
UPDATE test SET importId = 53 WHERE id = 3229;
UPDATE test SET importId = 725 WHERE id = 34165;
UPDATE test SET importId = 663 WHERE id = 42638;
UPDATE test SET importId = 210 WHERE id = 11288;
UPDATE test SET importId = 805 WHERE id = 764;
UPDATE test SET importId = 782 WHERE id = 10151;
UPDATE test SET importId = 53 WHERE id = 3229;
UPDATE test SET importId = 725 WHERE id = 34165;
UPDATE test SET importId = 663 WHERE id = 42638;
UPDATE test SET importId = 210 WHERE id = 11288;
UPDATE test SET importId = 805 WHERE id = 764;
UPDATE test SET importId = 782 WHERE id = 10151;
UPDATE test SET importId = 53 WHERE id = 3229;
UPDATE test SET importId = 725 WHERE id = 34165;
UPDATE test SET importId = 663 WHERE id = 42638;
UPDATE test SET importId = 210 WHERE id = 11288;
UPDATE test SET importId = 805 WHERE id = 764;
UPDATE test SET importId = 782 WHERE id = 10151;
UPDATE test SET importId = 53 WHERE id = 3229;
UPDATE test SET importId = 725 WHERE id = 34165;
UPDATE test SET importId = 663 WHERE id = 42638;
UPDATE test SET importId = 210 WHERE id = 11288;
UPDATE test SET importId = 805 WHERE id = 764;
UPDATE test SET importId = 782 WHERE id = 10151;
UPDATE test SET importId = 53 WHERE id = 3229;
UPDATE test SET importId = 725 WHERE id = 34165;
UPDATE test SET importId = 663 WHERE id = 42638;
UPDATE test SET importId = 210 WHERE id = 11288;
UPDATE test SET importId = 805 WHERE id = 764;
UPDATE test SET importId = 782 WHERE id = 10151;
UPDATE test SET importId = 53 WHERE id = 3229;
UPDATE test SET importId = 725 WHERE id = 34165;
UPDATE test SET importId = 663 WHERE id = 42638;
UPDATE test SET importId = 210 WHERE id = 11288;
UPDATE test SET importId = 805 WHERE id = 764;
UPDATE test SET importId = 782 WHERE id = 10151;
UPDATE test SET importId = 53 WHERE id = 3229;Result:
Segmentation fault (core dumped)A full testcase written in typescript and more information on how to reproduce the error can be found here: https://github.com/linaGirl/duckdb-segfault
OS:
Ubuntu 24.04.2 LTS
DuckDB Version:
v1.2.1 8e52ec4
DuckDB Client:
node & node-neo, duckdb stable binary
Hardware:
AMD Ryzen 7 2700X Eight-Core Processor; 64 GB Ram
Full Name:
Lina van der Weg
Affiliation:
SOFThera Ltd.
What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.
I have tested with a stable release
Did you include all relevant data sets for reproducing the issue?
Yes
Did you include all code required to reproduce the issue?
- Yes, I have
Did you include all relevant configuration (e.g., CPU architecture, Python version, Linux distribution) to reproduce the issue?
- Yes, I have