Skip to content

Conversation

l2a5b1
Copy link
Contributor

@l2a5b1 l2a5b1 commented Jul 24, 2018

This pull request drops the boost/algorithm/string/split.hpp dependency from the project. It replaces boost::split with a custom template function Split that has a similar API and returns exactly the same results as boost::split.

In addition this pull request refactors an instance of boost::algorithm::trim_right, which was implicitly made available via the boost/algorithm/string.hpp dependency to prevent having to introduce a new dependency boost/algorithm/string/trim.hpp.

The #include <boost/algorithm/string/predicate.hpp> in src/wallet/rpcdump.cpp will be addressed in #13656 if that PR is merged after this one; or in this PR should #13656 be merged first.

The test vectors in the accompanying unit test have been validated against boost::split.

@l2a5b1 l2a5b1 force-pushed the patch/remove_boost_split branch 6 times, most recently from 9e10b8f to 51c2993 Compare July 24, 2018 12:37
@maflcko
Copy link
Member

maflcko commented Jul 24, 2018

Hmm, this is just replacing a boost header with a self-written header that offers the same functionality and comes with unit tests. Do we really want to re-implement all of boost we are currently using? I mean straightforward replacing boost:: with std:: is fine, but re-implementing boost should be one of our low priority goals, no?

@l2a5b1
Copy link
Contributor Author

l2a5b1 commented Jul 24, 2018

That's a fair point @MarcoFalke 👍

The goal was not to re-implement boost. This is an attempt to drop the boost::split dependency with as little changes to the existing codebase as possible.

I have eliminated boost where it was trivial to do: boost::algorithm::is_any_of has been refactored to a std::string and boost::algorithm::trim_right to a std::string:erase.

There are many ways to simplify this function: e.g. no support for both std::vector and std::set; no support for boost::token_compress_on.

I didn't want to go there initially, because I didn't want to break things and ease review by providing something that is straightforward to validate. Right now it is easy to swap the boost::split and Split functions in the unit test to validate if the external behaviour of Split still works as expected.

That said, I don't have strong opinion on this. I am happy to make changes where needed; or close the pull request if this is not what you're looking for.

I just hope I haven't messed up 😅

@l2a5b1 l2a5b1 force-pushed the patch/remove_boost_split branch 3 times, most recently from cd176ce to 8199976 Compare July 24, 2018 20:31
@DrahtBot
Copy link
Contributor

DrahtBot commented Jul 25, 2018

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #17037 (Testschains: Many regtests with different genesis and default datadir by jtimon)
  • #16887 (Abstract out some of the descriptor Span-parsing helpers by sipa)
  • #16411 (Signet support by kallewoof)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@practicalswift
Copy link
Contributor

practicalswift commented Jul 25, 2018

Concept ACK

The short self-contained non-Boost version is easier to reason about.

Every small step counts in our journey to get rid of our Boost dependencies! :-)

@maflcko
Copy link
Member

maflcko commented Jul 25, 2018

@practicalswift Sure, but it has to go through review to verify that it supports everything that was supported by boost the way we used it.

We are stuck with boost until we adopt C++17, since implementing std::optional or std::variant on our own wouldn't be too wise, imo. So at least there is no rush in getting rid of non-trivial boost features that are not already in std::.

@practicalswift
Copy link
Contributor

@MarcoFalke Agreed – no rush in removing non-trivial Boost features. Let’s do it incrementally starting with the low hanging fruit :-)

@l2a5b1
Copy link
Contributor Author

l2a5b1 commented Jul 25, 2018

So at least there is no rush in getting rid of non-trivial boost features that are not already in std::

@MarcoFalke, apologies about this. I have totally misinterpreted the objective of the Boost migration.

@l2a5b1 l2a5b1 closed this Jul 25, 2018
@maflcko
Copy link
Member

maflcko commented Jul 25, 2018

Sorry for being so pessimistic about it. You can definitely leave this open and see what other reviewers think. I was just saying that we have about 3 years time to finish up everything.

@maflcko maflcko reopened this Jul 25, 2018
@l2a5b1
Copy link
Contributor Author

l2a5b1 commented Jul 25, 2018

Thanks @MarcoFalke!

@practicalswift
Copy link
Contributor

@251labs Text splitting should be in the low hanging fruit category I mentioned :-)

@sipa
Copy link
Member

sipa commented Jul 26, 2018

Note there is a function with similar functionality in #13697, but operating on Span<const char>s rather than std::strings.

@practicalswift
Copy link
Contributor

@251labs See if you can base your work in this PR on @sipa:s code in #13697 :-)

@l2a5b1 l2a5b1 force-pushed the patch/remove_boost_split branch from 8199976 to 7d00b41 Compare August 3, 2018 22:39
@l2a5b1
Copy link
Contributor Author

l2a5b1 commented May 7, 2019

rebased 402da1c

Travis build error is unrelated:

Error! Initial build successful, but not enough time remains to run later build stages and tests. 
Please manually re-run this job by using the travis restart button or asking a bitcoin maintainer 
to restart. The next run should not time out because the build cache has been saved.

@promag
Copy link
Contributor

promag commented May 7, 2019

@251labs restarted.

@l2a5b1
Copy link
Contributor Author

l2a5b1 commented May 7, 2019

Thanks @promag!

@l2a5b1 l2a5b1 force-pushed the patch/remove_boost_split branch from 402da1c to 6186091 Compare June 6, 2019 17:28
@l2a5b1
Copy link
Contributor Author

l2a5b1 commented Jun 6, 2019

Rebased 6186091

@l2a5b1
Copy link
Contributor Author

l2a5b1 commented Jun 21, 2019

rebased 47a758b

@l2a5b1 l2a5b1 force-pushed the patch/remove_boost_split branch from 47a758b to 686b32c Compare June 28, 2019 14:30
@l2a5b1 l2a5b1 force-pushed the patch/remove_boost_split branch from 686b32c to 5730465 Compare July 2, 2019 20:16
@l2a5b1
Copy link
Contributor Author

l2a5b1 commented Jul 3, 2019

Rebased 5730465

@l2a5b1 l2a5b1 force-pushed the patch/remove_boost_split branch from 5730465 to af652d9 Compare August 21, 2019 16:48
This commit drops the `boost/algorithm/string/split.hpp` dependency from
the project.

It replaces `boost::split` with a custom function `Split` that has
an identical API and returns exactly the same results as `boost::split`
to ease refactoring.

In addition this commit refactors an instance of `boost::algorithm::trim_right`
which was implicitly made available via the `boost/algorithm/string.hpp`
dependency to prevent having to introduce a new dependency
`boost/algorithm/string/trim.hpp`.
@l2a5b1 l2a5b1 force-pushed the patch/remove_boost_split branch from af652d9 to 7905ea0 Compare August 21, 2019 16:57
@DrahtBot
Copy link
Contributor

Needs rebase

@fanquake
Copy link
Member

Thanks for the work you've done here. However the project has decided that while it remains impossible to remove all of our Boost usage, we are no longer going to remove smaller components by rewriting them ourselves.

@fanquake fanquake closed this Oct 31, 2019
@l2a5b1
Copy link
Contributor Author

l2a5b1 commented Oct 31, 2019

Thanks @fanquake.

Context FWIW:

bitcoin-core-dev/2019-10-31.log

12:08 < wumpus> #topic close Boost -> C++11 migration project for now (wumpus)
12:09 < warren> too much change required?
12:09 < wumpus> so it looks like all the low-hanging fruit for replacing boost has been picked now
12:10 < wumpus> what remains is hard to replace with C++11 (results in very verbose code, locale dependent legacy C, or introduces harder to maintain platform-specific code)
12:10 < jeremyrubin> It's basically just multiindex and boost::thread now right?
12:10 < wumpus> it's kind of a time wasting trap for developers now (regarding PRs like #17245)
12:10 < gribble> #17245 | wallet: Remove Boost from DecodeDumpTime by elichai . Pull Request #17245 . bitcoin/bitcoin . GitHub
12:10 < wumpus> nah, people stumble over the sleep and date/time handling stuff every time
12:11 < wumpus> #17307 might still be worthwhile
12:11 < gribble> #17307 | Stop using boost::thread_group . Issue #17307 . bitcoin/bitcoin . GitHub
12:11 < jeremyrubin> What about just checking in those specific copies of boost library code
12:11 < sipa> after 17307, won't removing boost::threa dbe a lot easier?
12:11 < jeremyrubin> Or are they too large/dependent on the rest of boost types
12:11 < wumpus> but it needs to be focused, we don't want to close the same PRs time after time that don't really make it
12:12 < wumpus> I think having that project open sends the wrong messag
12:12 < wumpus> we can't replace boost right now
12:12 < jeremyrubin> 17307, having worked on replacing thread_group in the checkqueue before, scares me a bit
12:12 < sipa> agree there
12:12 < wumpus> there's no need to replace, say, small string handling functions with our own impelentation, before we have a vision to replace the rest
12:13 < sipa> right; until we have a reasonable way to remove multiindex (which i'm not sure will ever happen), getting rid of boost entirely is not really useful
12:13 < dongcarl> Just so it's out there... we can maybe run bcp at some point when there's only one or two things left https://www.boost.org/doc/libs/1_71_0/tools/bcp/doc/html/index.html
12:13 < jeremyrubin> dongcarl: ++
12:13 < sipa> i do think there are individual improvements possible that dkn't go all the way, like removing boost::thread
12:14 < jnewbery> I agree that if it's not a priority then the project should be closed. It can always be re-opened later if necessary.
12:14 < wumpus> but in any case it doesn't seem like it needs a big coordinated project anymore
12:14 < sipa> how many non-headers-only boost libs are we still using?
12:14 < sipa> wumpus: agree
12:14 < jnewbery> (leaving a comment in the project description explaining why it's been closed)
12:14 < wumpus> when C++17 is allowed, it can be reopened
12:15 < fanquake> Guess #13751 can be closed with the same rationale as well then.
12:15 < gribble> #13751 | Utils and libraries: Drops the boost/algorithm/string/split.hpp dependency by l2a5b1 . Pull Request #13751 . bitcoin/bitcoin . GitHub
12:15 < wumpus> fanquake: yes

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.