Skip to content

Conversation

jonasschnelli
Copy link
Contributor

This PR adds UniValue over a git subtree. Main repository is currently https://github.com/jgarzik/univalue.

Edit:
You need to re-do ./autogen.sh and ./configure after this is merged

@jgarzik
Copy link
Contributor

jgarzik commented Sep 4, 2015

concept ACK, of course :)

@sipa
Copy link
Member

sipa commented Sep 4, 2015

Please use --squash when modifying subtrees. It avoids the confusing incorporation of the other project's commits in our history, and allows the contrib/devtools/git-subtree-check.sh tool to verify that the claimed included commit actually has the intended contents.

@jonasschnelli jonasschnelli force-pushed the 2015/09/univalue_subtree branch from ccf8779 to 8f13ec9 Compare September 4, 2015 14:59
@jonasschnelli
Copy link
Contributor Author

@sipa: done.

@luke-jr
Copy link
Member

luke-jr commented Sep 4, 2015

Better, but this should really be an external shared library... Concept ACK

@dcousens
Copy link
Contributor

dcousens commented Sep 5, 2015

Is there any modern package management solution for C++?

@luke-jr
Copy link
Member

luke-jr commented Sep 5, 2015

pkg-config works great.

@jonasschnelli
Copy link
Contributor Author

Wouldn't dynamic library and or pkg-config require broad support in various distributions over their package management?

Personally I would recommend static linking of the JSON stack. It's a critical library and an a hacked JSON stack could lead to coin stealing.

@luke-jr
Copy link
Member

luke-jr commented Sep 5, 2015

@jonasschnelli It's not a consensus-critical library, which is the only exception where static linking makes sense. A hacked glibc, or even any other software on the PC can lead to coin stealing - there is nothing special about the JSON library. (obviously the gitian binaries and depends system would be used to continue to static link regardless of proper dependency handling)

@jonasschnelli
Copy link
Contributor Author

@luke-jr: ok, fair enough.

@sipa
Copy link
Member

sipa commented Sep 5, 2015 via email

@jonasschnelli
Copy link
Contributor Author

@sipa: Recently we have coupled UniValue with utilstrencodings.cpp/h (see c023092#diff-0f1b401041a14398229cf7e31b6db7eeR13).
To re-acchive independence from other sources, i have added the depending numparse functions in UniValue (see jgarzik/univalue@d0ec6ad and jgarzik/univalue@4b288b3). Not sure if this was a good step. I think copy and namespace the numparse function to UniValue could have made more sense. But also not sure if it would be a good idea either (duplicate code in bitcoin).

For bitcoin-core, we compile univalue without the number parse functions because we have them already over utilstrencodings.cpp/h. This is why univalue gets compiled with --with-numfunc=no.
But without the number parse functions, univalue itself cannot run the tests because the number parse functions are essential.
Maybe there is a way to tell the UniValue subtrees make check to link with utilstrencodings.o. But this would somehow break the independence of the subtree.

@jgarzik
Copy link
Contributor

jgarzik commented Sep 5, 2015

FWIW, the numberparse code is very nicely C/C++ standard and not tied to Univalue or bitcoin at all.

For the univalue tests, we could create an optional libtest for this case, if no better solution appears.

@sipa
Copy link
Member

sipa commented Sep 5, 2015 via email

@jgarzik
Copy link
Contributor

jgarzik commented Sep 5, 2015

@sipa It's not a dependency so much as a commonality. univalue and bitcoin both use ParseInt32

test/util_tests.cpp uses ParseInt32, ParseInt64 and ParseDouble, but that is a circular unit test dependency to be ignored.

@sipa
Copy link
Member

sipa commented Sep 5, 2015 via email

@jonasschnelli
Copy link
Contributor Author

UniValue doesn't depend on any code. It now just have an option (--with-numfunc=no) to avoid linking numfunc.o (consuming application of libunivalue can provide own number parse functions [3 functions as Jeff mentioned above] to avoid duplicated code).

I think it would make sense to just namespace the ParseInt32, ParseInt64 and ParseDouble into UniValue:: and compile/link them always.
Will do a PR upstream.

@sipa
Copy link
Member

sipa commented Sep 5, 2015 via email

@jonasschnelli jonasschnelli force-pushed the 2015/09/univalue_subtree branch 3 times, most recently from 15c92ac to 74f8e69 Compare September 7, 2015 13:54
@jonasschnelli
Copy link
Contributor Author

Updated this PR:

  • Refreshed subtree to jgarzik/univalue@a243b3d (--squashed)
  • Number parse functions are now independently implemented in UniValue over a anonymous namespace
  • Global make check does now also covers the UniValue make check
  • Replaced all #include "univalue/univalue.h" with #include <univalue.h> (was necessary because UniValues uses now univalue/lib/* as folder structure).

@jonasschnelli
Copy link
Contributor Author

Travis issue is because of missing -fPIC support in UniValue. This PR (jgarzik/univalue#6) should fix it (add libtool script).

@jgarzik
Copy link
Contributor

jgarzik commented Sep 7, 2015

@jonasschnelli What build object needs -fPIC? UniValue has been built as a *.a library without problems...

@jonasschnelli
Copy link
Contributor Author

I think it's recommended to build a *.la (libtool) library to gain more flexibility.

Travis complains about missing -fPIC recompile support:
https://travis-ci.org/bitcoin/bitcoin/jobs/79127433#L1480

Secp256k1 also uses libtool and fPIC:
https://github.com/bitcoin/bitcoin/blob/master/configure.ac#L920

@jonasschnelli jonasschnelli force-pushed the 2015/09/univalue_subtree branch from 74f8e69 to 6b148e3 Compare September 8, 2015 15:18
@jonasschnelli jonasschnelli force-pushed the 2015/09/univalue_subtree branch 3 times, most recently from b8d5e45 to 5aa54a9 Compare September 24, 2015 09:24
@sipa
Copy link
Member

sipa commented Sep 25, 2015 via email

@dcousens
Copy link
Contributor

I hadn't used subtrees before, was confused as to the intention.
I had the impression we were just copying the code in.

After some further research, concept ACK 👍

@jonasschnelli jonasschnelli force-pushed the 2015/09/univalue_subtree branch from 5aa54a9 to 2c3e80a Compare September 25, 2015 17:13
@laanwj
Copy link
Member

laanwj commented Oct 1, 2015

@jonasschnelli @theuni @jgarzik Is this ready for merge? If so I think we should do a rebase and then immediately merge, to prevent it getting out of sync due to large number of files touched.

@jonasschnelli jonasschnelli force-pushed the 2015/09/univalue_subtree branch from 2c3e80a to a852943 Compare October 1, 2015 08:38
@jonasschnelli
Copy link
Contributor Author

Rebased and force-pushed, ... waiting for travis now.

@jonasschnelli jonasschnelli force-pushed the 2015/09/univalue_subtree branch from a852943 to 8a046aa Compare October 1, 2015 08:45
similar to secp256k1 include and compile univalue over a subtree
@jonasschnelli jonasschnelli force-pushed the 2015/09/univalue_subtree branch from 8a046aa to 9623e93 Compare October 1, 2015 08:50
@jgarzik
Copy link
Contributor

jgarzik commented Oct 1, 2015

Upstream univalue tree has all the changes from @theuni - should be ready for merge.

@jgarzik
Copy link
Contributor

jgarzik commented Oct 1, 2015

"make[3]: *** No rule to make target `libunivalue.la'. Stop." hrm, might need LT_INIT in bitcoin configure.ac.

@jonasschnelli jonasschnelli force-pushed the 2015/09/univalue_subtree branch 3 times, most recently from db762c6 to d5d195d Compare October 1, 2015 11:58
@jonasschnelli jonasschnelli force-pushed the 2015/09/univalue_subtree branch from d5d195d to 95acf3c Compare October 1, 2015 12:28
@jonasschnelli
Copy link
Contributor Author

Travis is happy now.
IMO this is ready to merge.

@laanwj
Copy link
Member

laanwj commented Oct 1, 2015

Note: As usual after large build system changes, if you get:

Making all in src
make[1]: Entering directory `/store/orion/projects/bitcoin/bitcoin/src'
make[2]: Entering directory `/store/orion/projects/bitcoin/bitcoin/src'
make[3]: Entering directory `/store/orion/projects/bitcoin/bitcoin/src/univalue'
make[3]: *** No targets specified and no makefile found.  Stop.
make[3]: Leaving directory `/store/orion/projects/bitcoin/bitcoin/src/univalue'
make[2]: *** [univalue/lib/libunivalue.la] Error 2
make[2]: Leaving directory `/store/orion/projects/bitcoin/bitcoin/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/store/orion/projects/bitcoin/bitcoin/src'
make: *** [all-recursive] Error 1

You need to re-do ./autogen.sh and ./configure. The automatic triggering of those is not enough.

Edit: ACK

@btcdrak
Copy link
Contributor

btcdrak commented Oct 1, 2015

ACK

@laanwj laanwj merged commit 95acf3c into bitcoin:master Oct 1, 2015
laanwj added a commit that referenced this pull request Oct 1, 2015
95acf3c remove $(@f) and subdirs from univalue make (Jonas Schnelli)
9623e93 [Univalue] add univalue over subtree (Jonas Schnelli)
2f9f082 Squashed 'src/univalue/' content from commit 87d9045 (Jonas Schnelli)
0917306 remove univalue, prepare for subtree (Jonas Schnelli)
@jgarzik
Copy link
Contributor

jgarzik commented Oct 1, 2015

"git clean -dfx" is your friend. :)

zkbot added a commit to zcash/zcash that referenced this pull request Feb 10, 2017
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants