Skip to content

Conversation

Tmonster
Copy link
Contributor

Convert Intersect and Except into Semi and Anti Joins in the optimizer. This allows the join order optimizer to better predict the cardinality of the the operators.

Eventually the join order optimizer will be able to better predict the cardinality of the semi join, then the cardinality of the set operation.

For INTERSECT ALL and EXCEPT ALL the cardinality estimator can just use logic used to predict the cardinality of a semi join or of an anti join.

@github-actions github-actions bot marked this pull request as draft November 29, 2023 10:22
@Mytherin Mytherin marked this pull request as ready for review December 5, 2023 20:37
Copy link
Collaborator

@Mytherin Mytherin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! I'm not too sure about doing this replacement in the optimization phase. It seems to me that we now have two places where EXCEPT/INTERSECT are translated into anti/semi joins - one in the physical planner (plan_set_operation.cpp) and one in this optimizer. I would propose replacing the physical planner one with this implementation entirely, and doing this replacement in the logical planning phase instead.

@github-actions github-actions bot marked this pull request as draft December 19, 2023 17:46
@Tmonster
Copy link
Contributor Author

Closing this as it needs more work

@Tmonster Tmonster closed this Jan 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants