Port for secp256k1 crypto library #2005

sipsorcery · 2017-10-18T04:33:58Z

Port for https://github.com/bitcoin-core/secp256k1 elliptic curve cryptography library.

I wasn't sure of the best way to use msbuild and ended up creating a standalone solution and project file and then copying those into the extracted source tree. I'm happy to adjust there's an alternative standard approach.

msftclas · 2017-10-18T04:34:06Z

All CLA requirements met.

ghost · 2017-10-18T04:37:24Z

ports/secp256k1/portfile.cmake

+set(SOURCE_PATH ${CURRENT_BUILDTREES_DIR}/src/secp256k1-master)
+
+vcpkg_download_distfile(ARCHIVE
+    URLS "https://github.com/bitcoin-core/secp256k1/archive/master.zip"


Two points:

Use vcpkg_from_github.

Don't reference the master branch because the port will break as soon as someone commits. Use a tag.

Actually, there are no tags, so it would be a ref, like 0b7024185045a49a1a6a4c5615bf31c94f63d9c4

Will do.

I did twig to that. There are no tags on that repo at the moment. I've opened an issue to see if major releases can be tagged.

ras0219-msft · 2017-10-18T06:17:49Z

Thanks for the PR!

If you need to make a replacement buildsystem, it's best to use CMake. This will automatically support (if possible) all the different architectures, platforms, and settings such as building the library as static or dynamic.

For an example CMake replacement that's complete, yet still simple, take a look at uwebsockets[1] or aubio[2].

[1] https://github.com/Microsoft/vcpkg/blob/master/ports/uwebsockets/CMakeLists.txt
[2] https://github.com/Microsoft/vcpkg/blob/master/ports/aubio/CMakeLists.txt

sipsorcery · 2017-10-18T10:19:43Z

Port switched from msbuild to cmake.

…cally.

ras0219-msft · 2017-10-20T02:57:11Z

Thanks! I made a few more tweaks while merging, which you can see in 9761952.

sipsorcery · 2017-10-20T05:48:58Z

Great, thanks for the tweak and merge.

ef7beae Visual Studio build configuration for Bitcoin Core (Aaron Clauson) Pull request description: This PR allows Bitcoin Core to be relatively easily built with Visual Studio 2017. It's anticipated that it could be useful for devs familiar with Visual Studio and Microsoft's tooling. In particular the ability to use the VS debugger is a big benefit. ~~Caveats:~~ - ~~There are some minor code changes required on Bitcoin Core in order for msvc to be able to successfully compile. I'll submit them in a separate PR, The code changes are available in bitcoin#11528 bitcoin#11558 and bitcoin#11562~~. - ~~The vcpkg for SECP256K1 has not yet been accepted by Microsoft. The files are available from this [PR](microsoft/vcpkg#2005) and should be copied into a vcpkg/ports/secp256k1 directory prior to vcpkg install steps.~~ **Update:** For anyone wishing to test out the Visual Studio build with the various open pull requests the steps are: - Clone and build [Vcpkg](https://github.com/Microsoft/vcpkg) (Microsoft's new open source C/C++ package manager) - git clone https://github.com/Microsoft/vcpkg - .\bootstrap-vcpkg.bat - Set up Visual Studio to automatically reference vcpkg installs: .\vcpkg integrate install - Install the required packages (replace x86 with x64 as required): - vcpkg install boost:x86-windows-static - vcpkg install libevent:x86-windows-static - vcpkg install openssl:x86-windows-static - vcpkg install zeromq:x86-windows-static - vcpkg install berkeleydb:x86-windows-static - vcpkg install secp256k1:x86-windows-static - vcpkg install leveldb:x86-windows-static - git clone https://github.com/bitcoin/bitcoin.git - git checkout -b testbuild - git pull origin pull/11526/head # Visual Studio build configuration for Bitcoin Core - ~~git pull origin pull/11558/head # Minimal code changes to allow msvc compilation~~ - ~~git pull origin pull/11562/head # bench: use std::chrono rather than gettimeofday~~ - ~~Copy and unzip attached bitcoin-config.h to src/config, edit as required [bitcoin-config.zip](https://github.com/bitcoin/bitcoin/files/1429484/bitcoin-config.zip)~~ - ~~git pull origin pull/13031/head # gmtime fix for msvc~~ - Build the Visual Studio solution which, if successful, will result in all but the Qt dependent libraries/programs being built. If the build fails please add a comment. Tree-SHA512: 5cd17273d33a09c35d8534c9f49123dec60ec05383669c67674b2cac88ada177bf94d7731c2a827759444f18d4b67085b91b02458124d0c32ab3a8f72ba5dac9

sipa · 2018-08-20T15:35:04Z

This seems to always build the 32-bit version, which is about 4x slower than the 64-bit code. Is that intentional?

sipsorcery · 2018-08-20T17:12:29Z

@sipa the cmake build should pick up the platform type from whatever Visual Studio project is attempting to link with the secp256k1 package. If the specific platform type the compiler/linker wants isn’t installed, e.g. secp256k1:windows-static-x64, then the build will fail. I’ve always used the 64 bit version of this package successfully.

Apart from that maybe there’s a pre-processor directive somewhere that doesn’t get set for msvc that prevents some 64 bit compile time optimisations being used?

sipa · 2018-08-21T15:51:29Z

@sipsorcery For example, you're setting USE_FIELD_10X26, which is the 32-bit field implementation. You should at the very least use USE_FIELD_5X52 on 64-bit systems.

sipsorcery · 2018-08-22T08:46:11Z

@sipa what would you recommend as the way to set the values in libsecp256k1-config.h appropriately for a 64 bit Windows build?

Run ./configure on 64 bit Linux and base the settings off the config file it produces?

sipa · 2018-08-23T16:10:11Z

@sipsorcery The performance relevant defines are:

HAVE_BUILTIN_EXPECT if __builtin_expect is available
HAVE_LIBGMP to use libgmp
HAVE___INT128 if the 128-bit type __int128 exists
USE_ASM_X86_64 to use x86_64 inline assembly
USE_FIELD_10X26 to use the 32-bit field code
USE_FIELD_5X52 to use the 64-bit field code (which requires either USE_ASM_X86_64, or __int128 support).
USE_FIELD_INV_BUILTIN and USE_SCALAR_INV_BUILTIN and USE_NUM_NONE if you want the native inversion code (which is slower than GMP's)
USE_FIELD_INV_NUM and USE_SCALAR_INV_NUM and USE_NUM_GMP if you want the GMP based inversion code (faster, but needs libgmp)
USE_SCALAR_8X32 to use the 32-bit scalar code
USE_SCALAR_4X64 to use the 64-bit scalar code (again, either requires USE_ASM_X86_64, or __int128 support)

sipsorcery · 2018-08-30T13:56:44Z

@sipa I've checked each of the options for msvc and put the results below:

HAVE_BUILTIN_EXPECT - not available on msvc
HAVE_LIBGMP - not vcpkg port for GMP
HAVE___INT128 - not available on msvc
USE_ASM_X86_64 - asm not supported by the x64 msvc compiler (it is by the x86 msvc compiler)
USE_FIELD_10X26 - being used as msvc can't support any of the alternative 64 bit options?
USE_FIELD_5X52 - neither x64 asm or int128 supported by msvc
USE_FIELD_INV_BUILTIN and USE_SCALAR_INV_BUILTIN and USE_NUM_NONE - all in use as no gmp package available
USE_FIELD_INV_NUM and USE_SCALAR_INV_NUM and USE_NUM_GMP - no gmp package available
USE_SCALAR_8X32 - being used as msvc can't support any of the alternative 64 bit options?
USE_SCALAR_4X64 - neither x64 asm or int128 supported by msvc

I guess the moral of the story is if performance is a concern don't build secp256k1 with msvc. Conversely it is very useful to have this port available to be available to debug dependent projects such as bitcoin.

ef7beae Visual Studio build configuration for Bitcoin Core (Aaron Clauson) Pull request description: This PR allows Bitcoin Core to be relatively easily built with Visual Studio 2017. It's anticipated that it could be useful for devs familiar with Visual Studio and Microsoft's tooling. In particular the ability to use the VS debugger is a big benefit. ~~Caveats:~~ - ~~There are some minor code changes required on Bitcoin Core in order for msvc to be able to successfully compile. I'll submit them in a separate PR, The code changes are available in bitcoin#11528 bitcoin#11558 and bitcoin#11562~~. - ~~The vcpkg for SECP256K1 has not yet been accepted by Microsoft. The files are available from this [PR](microsoft/vcpkg#2005) and should be copied into a vcpkg/ports/secp256k1 directory prior to vcpkg install steps.~~ **Update:** For anyone wishing to test out the Visual Studio build with the various open pull requests the steps are: - Clone and build [Vcpkg](https://github.com/Microsoft/vcpkg) (Microsoft's new open source C/C++ package manager) - git clone https://github.com/Microsoft/vcpkg - .\bootstrap-vcpkg.bat - Set up Visual Studio to automatically reference vcpkg installs: .\vcpkg integrate install - Install the required packages (replace x86 with x64 as required): - vcpkg install boost:x86-windows-static - vcpkg install libevent:x86-windows-static - vcpkg install openssl:x86-windows-static - vcpkg install zeromq:x86-windows-static - vcpkg install berkeleydb:x86-windows-static - vcpkg install secp256k1:x86-windows-static - vcpkg install leveldb:x86-windows-static - git clone https://github.com/bitcoin/bitcoin.git - git checkout -b testbuild - git pull origin pull/11526/head # Visual Studio build configuration for Bitcoin Core - ~~git pull origin pull/11558/head # Minimal code changes to allow msvc compilation~~ - ~~git pull origin pull/11562/head # bench: use std::chrono rather than gettimeofday~~ - ~~Copy and unzip attached bitcoin-config.h to src/config, edit as required [bitcoin-config.zip](https://github.com/bitcoin/bitcoin/files/1429484/bitcoin-config.zip)~~ - ~~git pull origin pull/13031/head # gmtime fix for msvc~~ - Build the Visual Studio solution which, if successful, will result in all but the Qt dependent libraries/programs being built. If the build fails please add a comment. Tree-SHA512: 5cd17273d33a09c35d8534c9f49123dec60ec05383669c67674b2cac88ada177bf94d7731c2a827759444f18d4b67085b91b02458124d0c32ab3a8f72ba5dac9

ghost suggested changes Oct 18, 2017

View reviewed changes

sipsorcery force-pushed the port_secp256k1 branch from 8d4ecbc to d865640 Compare October 18, 2017 05:40

Port (cmake) for secp256k1 crypto project.

fc80d72

sipsorcery force-pushed the port_secp256k1 branch from d865640 to fc80d72 Compare October 18, 2017 10:18

sipsorcery changed the title ~~Port for secp256k1 crypto project.~~ Port for secp256k1 crypto library Oct 19, 2017

sipsorcery mentioned this pull request Oct 19, 2017

Visual Studio build configuration for Bitcoin Core bitcoin/bitcoin#11526

Merged

[secp256k1] Only install the single public header. Always build stati…

9761952

…cally.

ras0219-msft merged commit c4ceadc into microsoft:master Oct 20, 2017

sipsorcery deleted the port_secp256k1 branch October 20, 2017 05:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Port for secp256k1 crypto library #2005

Port for secp256k1 crypto library #2005

Uh oh!

sipsorcery commented Oct 18, 2017

Uh oh!

msftclas commented Oct 18, 2017 •

edited

Loading

Uh oh!

ghost Oct 18, 2017

Uh oh!

ghost Oct 18, 2017

Uh oh!

sipsorcery Oct 18, 2017

Uh oh!

ras0219-msft commented Oct 18, 2017

Uh oh!

sipsorcery commented Oct 18, 2017

Uh oh!

ras0219-msft commented Oct 20, 2017

Uh oh!

sipsorcery commented Oct 20, 2017

Uh oh!

sipa commented Aug 20, 2018 •

edited

Loading

Uh oh!

sipsorcery commented Aug 20, 2018

Uh oh!

sipa commented Aug 21, 2018 •

edited

Loading

Uh oh!

sipsorcery commented Aug 22, 2018

Uh oh!

sipa commented Aug 23, 2018

Uh oh!

sipsorcery commented Aug 30, 2018

Uh oh!

Uh oh!

Port for secp256k1 crypto library #2005

Port for secp256k1 crypto library #2005

Uh oh!

Conversation

sipsorcery commented Oct 18, 2017

Uh oh!

msftclas commented Oct 18, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost Oct 18, 2017

Choose a reason for hiding this comment

Uh oh!

ghost Oct 18, 2017

Choose a reason for hiding this comment

Uh oh!

sipsorcery Oct 18, 2017

Choose a reason for hiding this comment

Uh oh!

ras0219-msft commented Oct 18, 2017

Uh oh!

sipsorcery commented Oct 18, 2017

Uh oh!

ras0219-msft commented Oct 20, 2017

Uh oh!

sipsorcery commented Oct 20, 2017

Uh oh!

sipa commented Aug 20, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sipsorcery commented Aug 20, 2018

Uh oh!

sipa commented Aug 21, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sipsorcery commented Aug 22, 2018

Uh oh!

sipa commented Aug 23, 2018

Uh oh!

sipsorcery commented Aug 30, 2018

Uh oh!

Uh oh!

msftclas commented Oct 18, 2017 •

edited

Loading

sipa commented Aug 20, 2018 •

edited

Loading

sipa commented Aug 21, 2018 •

edited

Loading