Certificates issued by vault with isCa: true are missing CA:TRUE in certificate #4654
Description
Describe the bug:
I'm trying to set up linkerd with cert-manager and vault. I've created a root CA and an intermediate CA for my cluster with Terraform and I'm using the intermediate certificate as a trust anchor for linkerd. Linkerd's identity container does not startup with the following message:
time="2021-12-10T09:39:23Z" level=fatal msg="Failed to initialize identity service: failed to verify issuer certificate: it must be an intermediate-CA, but it is not"
After checking the certificates and trying around I found the following behavior:
For testing purposes, I have created two ClusterIssuers,
A self signed cluster issuer
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"cert-manager.io/v1","kind":"ClusterIssuer","metadata":{"annotations":{},"name":"linkerd-self-signed-issuer"},"spec":{"selfSigned":{}}}
creationTimestamp: "2021-12-10T09:32:10Z"
generation: 1
name: linkerd-self-signed-issuer
resourceVersion: "50506"
uid: ab029ca9-d992-4b32-95db-3f910738d363
spec:
selfSigned: {}
status:
conditions:
- lastTransitionTime: "2021-12-10T09:32:10Z"
observedGeneration: 1
reason: IsReady
status: "True"
type: Ready
and one cluster issuer for vault
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
creationTimestamp: "2021-12-08T14:01:00Z"
generation: 1
name: vault-cluster-issuer
resourceVersion: "1731"
uid: 6ef7b5f8-efbe-4b03-a912-7943316abf2c
spec:
vault:
auth:
kubernetes:
mountPath: /v1/auth/kubernetes_cluster1_local
role: cluster1-cert-manager
secretRef:
key: token
name: cert-manager-token-wfg85
path: pki_intermediate_cluster1_local/sign/cluster1-local.mydomain.lo
server: http://192.168.59.110:31891
status:
conditions:
- lastTransitionTime: "2021-12-08T14:01:02Z"
message: Vault verified
observedGeneration: 1
reason: VaultVerified
status: "True"
type: Ready
My certificate resource :
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: asd
namespace: platform-cert-manager
spec:
isCA: true
commonName: asd.platform-linkerd.cluster1-local.mydomain.lo
secretName: asd
privateKey:
algorithm: RSA
size: 2048
issuerRef:
name: linkerd-self-signed-issuer
kind: ClusterIssuer
group: cert-manager.io
When I use the linkerd-self-signed-issuer to create the certificate with isCa: true, the generated certificate looks like the following:
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
8c:aa:11:4f:07:01:1d:dd:12:34:81:db:2c:52:1d:6b
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN=asd.platform-linkerd.cluster1-local.mydomain.lo
Validity
Not Before: Dec 10 09:32:10 2021 GMT
Not After : Mar 10 09:32:10 2022 GMT
Subject: CN=asd.platform-linkerd.cluster1-local.mydomain.lo
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:fc:bb:2a:45:d6:48:92:bb:6e:c0:94:15:c2:8b:
79:d9:47:05:98:f3:53:c8:31:b7:ff:fc:97:2b:9b:
42:9a:78:de:3f:7f:f1:b4:3a:be:62:aa:6e:81:ee:
7a:0c:ad:d5:7a:d7:45:5b:aa:5f:ab:25:17:ef:d3:
c9:e9:ce:d4:82:d8:18:60:53:d6:e8:7d:08:3c:27:
8c:57:24:c2:79:45:54:3b:13:5d:05:c6:5f:54:56:
44:f7:5d:dc:73:c7:a9:9a:97:e5:56:93:54:76:73:
d6:56:3d:18:a0:83:63:f6:92:69:97:7b:75:f7:76:
53:c3:39:7e:74:31:9f:54:83:2d:86:87:04:d3:a9:
40:17:87:6a:da:38:a8:40:7b:c5:df:bd:53:d6:ae:
82:e9:1b:c9:bc:56:cb:6c:5f:82:0a:27:07:ff:d6:
7d:41:38:47:75:34:b9:e7:14:66:f4:fd:13:ec:cc:
f3:d8:69:9a:a0:f3:b4:62:02:43:e9:21:f3:d4:db:
0b:79:56:9a:6d:bc:fc:30:81:5b:25:56:59:63:32:
40:8a:0a:d6:c2:ac:e5:68:c5:23:fe:fb:52:7f:d4:
84:4e:c5:0f:ef:17:df:30:19:1b:14:06:62:01:d0:
70:6b:ef:0e:fd:0c:52:b6:79:8b:d0:48:ca:75:58:
4a:7d
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Key Usage: critical
Digital Signature, Key Encipherment, Certificate Sign
X509v3 Basic Constraints: critical
CA:TRUE
X509v3 Subject Key Identifier:
15:0C:E8:0C:D6:4B:A5:35:70:BC:46:C8:9F:5D:DA:31:25:C5:8D:36
Signature Algorithm: sha256WithRSAEncryption
64:5e:b4:b2:cd:c6:ab:d1:99:23:bf:f8:db:78:5f:b6:0b:11:
fa:85:d3:61:28:bc:4d:f4:9a:dc:ce:33:d5:39:ea:76:f7:49:
be:4e:71:22:12:7e:fa:c9:d3:ab:8e:e1:7d:8b:60:e0:9d:31:
82:06:76:e8:18:89:cd:14:81:29:a4:d2:33:8c:5b:ca:c3:03:
9f:90:21:8a:fa:d5:65:75:60:c6:2b:47:a2:62:4e:18:3a:1f:
0f:b0:27:df:c5:e2:99:77:90:3d:c0:e9:f6:66:42:36:70:cb:
a8:54:86:67:6e:e3:65:6a:da:a2:13:d0:56:45:35:4a:89:ec:
c8:4d:4d:07:63:26:2c:55:11:b1:ec:8b:d3:12:68:b5:4c:76:
12:2e:3a:ca:1d:09:89:25:c6:bf:ce:b4:18:24:e4:aa:4b:93:
85:9f:94:73:fd:4e:19:37:a6:75:f5:f4:69:34:b0:56:e0:8a:
de:4c:59:91:88:ca:5c:d3:d1:9b:d5:f1:c0:85:43:05:63:b7:
8f:2f:b9:05:e7:06:4c:02:5d:ab:05:fd:6b:df:03:cf:96:9a:
c7:fc:f2:5c:c2:33:87:e5:b4:ee:85:30:30:20:22:b3:bb:40:
eb:93:c8:99:ef:1a:c7:c3:f5:7b:07:03:38:6d:93:58:2c:5f:
f1:6d:6d:76
The x509v3 extensions include CA:TRUE
If I change the issuerRef.name to my vault issuer vault-cluster-issuer, the x509v3 extensions looks completely different, and CA:TRUE is missing.
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
27:c3:70:81:90:d6:ef:07:49:cd:67:08:61:fb:eb:a3:b4:02:96:5a
Signature Algorithm: sha256WithRSAEncryption
Issuer: O=cluster1-local.mydomain.lo, OU=cluster1-local.mydomain.lo, CN=cluster1-local.mydomain.lo
Validity
Not Before: Dec 10 09:01:01 2021 GMT
Not After : Mar 10 09:01:31 2022 GMT
Subject: CN=asd.platform-linkerd.cluster1-local.mydomain.lo
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:be:80:63:d9:ef:c9:88:00:bb:61:ea:e9:95:96:
5e:9e:ba:79:0f:4b:d0:a3:3e:7c:d7:b9:7d:da:8c:
9d:77:6c:8b:c4:8c:66:bd:c9:3a:b0:1e:7c:4a:66:
cc:c6:f5:c2:0a:7f:5c:f1:17:3d:8b:71:6b:2e:37:
40:87:5c:b4:f7:1f:07:19:44:ec:b1:d6:71:76:6e:
bf:89:41:98:cb:04:96:bc:49:fc:d0:fe:d4:95:a1:
05:1f:1a:8f:b5:80:47:ba:97:09:31:e2:20:23:22:
37:e4:ef:d7:33:f7:58:1a:2b:0c:3e:f0:31:71:3d:
db:66:f7:d9:01:2c:fe:6f:60:74:8d:39:78:23:04:
82:4b:ba:8f:a7:3e:08:c8:df:7d:20:8e:7f:84:f6:
93:98:29:2b:1f:ff:64:57:fd:37:21:23:53:ea:d8:
43:53:12:85:cd:b1:95:ba:a9:81:88:05:96:0d:3a:
85:f0:33:85:5a:95:9a:31:1d:65:e6:c1:e7:68:1a:
19:47:53:d2:20:cb:0c:3b:a4:20:95:a4:af:98:ef:
33:9e:4a:4a:72:be:27:cf:35:2c:26:52:4d:3e:72:
ee:54:ee:b2:fa:44:f6:68:bf:a1:61:4e:1f:6f:4d:
53:c6:be:0f:03:bd:f9:56:f4:14:a9:04:2a:3f:4c:
b7:73
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Key Usage: critical
Digital Signature, Key Encipherment, Key Agreement
X509v3 Extended Key Usage:
TLS Web Server Authentication, TLS Web Client Authentication
X509v3 Subject Key Identifier:
16:B0:FA:3B:5B:03:AA:FF:F7:7C:73:30:3D:DF:26:9D:3A:F5:13:0C
X509v3 Authority Key Identifier:
keyid:22:5B:0B:83:32:C6:4F:44:00:2B:A9:01:00:D8:B2:55:03:AE:C8:F6
Signature Algorithm: sha256WithRSAEncryption
2c:e9:ab:3e:4d:dd:7a:ab:fd:31:52:09:3b:3e:d9:51:96:ba:
18:8d:1c:db:1e:bf:57:f4:0a:67:9e:f2:17:d9:26:ec:49:93:
cd:67:53:55:12:eb:8d:a5:ba:a3:01:53:ea:1b:ec:03:77:bc:
4d:5b:bd:f3:bd:53:c1:d2:9d:d2:82:57:24:42:5e:11:fb:bb:
12:8c:2c:fc:e4:a7:2d:0f:24:56:30:56:b9:3c:fc:49:51:25:
0c:3e:2b:c4:0d:fa:11:36:90:6f:62:7f:83:13:3a:fd:e2:38:
b8:f1:d5:9e:91:20:6d:71:01:ff:11:e9:c2:9c:9d:10:37:bd:
68:15:93:f8:46:58:89:dc:de:eb:84:e9:5b:ff:bd:76:7b:93:
d0:e6:d8:49:9d:d6:8e:9b:d1:02:d7:9d:89:1b:df:d7:fd:c1:
9e:c1:db:5a:f9:ed:f2:63:32:aa:51:b4:2d:73:b6:a4:3f:9b:
2a:29:b1:99:59:68:bb:1b:9a:1e:04:0a:01:d7:a3:74:6d:92:
29:eb:84:72:0a:f5:16:73:e0:8c:f5:c6:5c:87:bf:a1:f4:7a:
ff:46:70:ed:dc:72:00:b4:af:d6:85:1b:85:5d:c5:58:c1:e4:
55:c4:27:c5:65:f6:42:2a:1c:e7:75:ca:d8:2a:35:ac:9c:25:
8b:48:1e:68
Expected behaviour:
I would expect, that the certificate extensions for the resulting certificate are the same for both ClusterIssuers and that CA:TRUE is set for the vault-generated certificate.
Steps to reproduce the bug:
Anything else we need to know?:
After looking into the code I found that there is a validation function which suggests, that vault is not capable to create the X509v3 Basic Constraints extension. From what I can see in Github, the method is only been referenced inside the tests. Maybe @munnerz can explain why vault does not support this?
Environment details::
- Kubernetes version: 1.22.3
- Cloud-provider/provisioner: vault: 1.9.0
- cert-manager version: 1.6.1
- Install method: e.g. helm
/kind bug