Skip to content

ECDSA verification compatibility issue due to applying different hash truncation #2415

@guidovranken

Description

@guidovranken

Botan uses a slightly different ECDSA input truncation mechanism than OpenSSL/LibreSSL/BoringSSL/wolfCrypt.

https://nvlpubs.nist.gov/nistpubs/fips/nist.fips.186-4.pdf says:

When the length of the output of the hash function is greater than the bit length of n, then the leftmost n bits of the hash function output block shall be used in any calculation using the hash function output during the generation or verification of a digital signature.

This is what Botan does.

OpenSSL and the others do this: https://github.com/openssl/openssl/blob/5357c10624bedaeed984ef4ff370096911ee2ddf/crypto/ec/ecdsa_ossl.c#L245-L258

So I think Botan is actually right and the others are doing it slightly wrong.

In my fuzzer, I apply the same truncation mechanism before calling Botan's verify_message: https://github.com/guidovranken/cryptofuzz/blob/7f9f35e20fcd8886cbcbf167474dd7f9954e5c02/modules/botan/module.cpp#L726-L739

So for me it's not a problem, but in practice this can lead to interoperability issues (Botan rejecting valid signatures).

Here's an example of a valid signature that Botan rejects:

operation name: ECDSA_Verify
ecc curve: x962_p239v1
public key X: 115014969141122710336858256331515905188079709631033705848897690363377891
public key Y: 81333916963110019576228330948951168219884247801435258672405011123948094
cleartext: {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 
 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00} (30 bytes)
signature R: 336870737920796383793526100561731463195539556177859437212670821204992455
signature S: 328556355548848640685841521416759180891403359973782869535609514452377183

Module Botan result:

false

Module OpenSSL result:

true

(numbers are in decimal, cleartext must be passed through SHA256 first)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions