Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

When I use an RSA key to sign something with PSS, the salt length is set to be MAX_LENGTH, which whilst acceptable in the strictest sense of the spec, is not in line with those required by RFC 4055

https://datatracker.ietf.org/doc/html/rfc4055#section-3.3

This makes verification of such signatures difficult across platforms. I have written a Java security provider which uses KMS to perform mtls connections, and while it works perfectly with actual KMS, my tests using localstack do not pass. Although verification of signed materials in the unit tests works, the resultant signatures do not necessarily verify with other libraries as the expected salt length is different to what has actually been used.

There is some mildly related discussion about this on the pycrypto repository

pyca/cryptography#3008

Expected Behavior

The salt length should be set appropriately for the signing algorithm being used. I believe this can be fixed simply by setting salt_length to be padding.PSS.DIGEST_LENGTH in services/kms/models.py but I haven't fully tested this yet, merely modified code locally for it to work in my specific case.

How are you starting LocalStack?

Custom (please describe below)

Steps To Reproduce

How are you starting localstack (e.g., bin/localstack command, arguments, or docker-compose.yml)

I have used both the testcontainers version of localstack, and used source code and `make start` to reproduce this

Client commands (e.g., AWS SDK code snippet, or sequence of "awslocal" commands)

SignRequest signRequest = SignRequest.builder()
.keyId(key.getId())
.messageType(MessageType.DIGEST)
.signingAlgorithm(SigningAlgorithmSpec.RSASSA_PSS_SHA_512)
.message(SdkBytes.fromByteArray(message))
.build();
SignResponse signatureResponse = kmsClient.sign(signRequest);

Produces a signature which verifies fine with localstack's KMS, but not with, say, BouncyCastle, or OpenSSL. An openssl command to verify signatures with the public key is included below

aws kms get-public-key --key-id <keyid>  --output text --query PublicKey --region us-west-2  | base64 -d > pub.der
 openssl pkeyutl -verify -in <digest> -sigfile <signature> -pkeyopt rsa_padding_mode:pss -pubin -inkey  pub.der -pkeyopt rsa_pss_saltlen:64 -pkeyopt digest:sha512 

This verification fails for signatures produced by localstack, but passes fine with real KMS

Environment

- OS: MacOS, Debian 12 slim
- LocalStack: 2.2.0

Anything else?

No response