-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Fix issue related to NodeJS UDF not returning constant vectors #5697
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix issue related to NodeJS UDF not returning constant vectors #5697
Conversation
I'll look to see if this still fails after Mytherins PR is merged |
Hey! Thanks for the above work everyone. Is this one gunna be fixed in the next release then? |
The fix itself is already included in the latest dev builds |
I have just tried running the test on: With the latest dev build, which I assume is what is installed via: And it still fails in the same way. Are we sure this issue has been addressed? Could we get this PR merged to confirm it really is fixed on master branch @Mytherin / @Mause? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Somehow the CI never ran here - will try to rerun it
Yea it seems this wasn't fixed by the other PR and the issue still exists |
So the issue seems to be that we use the c_str() of a temporary case duckdb::LogicalTypeId::BLOB:
case duckdb::LogicalTypeId::VARCHAR: {
auto data = ret.Get("data").As<Napi::Array>();
auto out = duckdb::FlatVector::GetData<duckdb::string_t>(*jsargs->result);
for (size_t i = 0; i < data.Length(); ++i) {
out[i] = duckdb::string_t(data.Get(i).ToString());
}
break;
} This ends up in this constructor string_t(const string &value)
: string_t(value.c_str(), value.size()) { // NOLINT: Allow implicit conversion from `const char*`
} And for non-inlined strings we just copy the pointer // large string: store pointer
#ifndef DUCKDB_DEBUG_NO_INLINE
memcpy(value.pointer.prefix, data, PREFIX_LENGTH);
#else
memset(value.pointer.prefix, 0, PREFIX_BYTES);
#endif
value.pointer.ptr = (char *)data;
} |
…rary string that goes out of scope, this memory becomes random. This is what we have a StringHeap for, using AddString we store non-inlined strings in there, so the memory doesn't go out of scope
…om/Tishj/duckdb into nodejs_non_inlined_string_from_udf
This change should fix the issue 👍 StringVector's have a StringHeap exactly for this purpose, we need to make sure the memory of these non-inlined strings stays alive long enough, so we copy over the memory into this heap and the data of the vector contains a string_t that points into this heap. |
Thanks! Looks good to me. |
This PR fixes #5670
As Mytherin noted, the issue itself is fixed in #5691, so this PR only adds a test for the issue mentioned.