[Bug]: Unable to use "tmpkms" on Windows

[JMyklebust]

Steps to Reproduce

Using the CGO build on Windows is not working with "tpmkms".

I've build step-ca v0.27.4 with CGO flag on Windows with the help of w64devkit.
I want to secure the certificate key using the TPM so i'm using "tpmkms" (though I would have prefered to use CryptoAPI on Windows tbh).

Some simplification of my actual script for this but I've successfully created the key using the kms module for step-cli

$env:STEPPATH = "C:\ACMECA\conf"
$TemplatesDir = (Join-Path $env:STEPPATH "templates")
$CertsDir = (Join-Path $env:STEPPATH "certs")
step kms create --json  --kty RSA --size 2048 'tpmkms:name=acme-ca-ver1'
step certificate create --kms 'tpmkms:' --key 'tpmkms:name=acme-ca-ver1' --csr --template "$TemplatesDir\CATemplate.tpl" "ACME Server" "$CertsDir\intermediate.csr"

The step.json looks like this (I've cut out the providers to keep it short):

{
    "root":  "C:/ACMECA/conf/certs/root_ca.crt",
    "federatedRoots":  null,
    "crt":  "C:/ACMECA/conf/certs/intermediate_ca.crt",
    "key":  "tpmkms:name=acme-ca-ver1",
    "kms":  {
                "type":  "tpmkms",
                "uri":  "tpmkms:"
            },
    "address":  ":9000",
    "insecureAddress":  "",
    "dnsNames":  [
                     "testing.example.com"
                 ],
    "logger":  {
                   "format":  "text"
               },
    "db":  {
               "type":  "badgerv2",
               "dataSource":  "C:/ACMECA/conf/db",
               "badgerFileLoadingMode":  "FileIO"
           },
    "authority":  {
                      "template":  {

                                   },
                      "backdate":  "1m0s"
                  },
    "tls":  {
                "cipherSuites":  [
                                     "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
                                     "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256"
                                 ],
                "minVersion":  1.2,
                "maxVersion":  1.3,
                "renegotiation":  false
            },
    "commonName":  "ACME Server"
}

Your Environment

• OS - Windows 11 23H2 (also intending to run on Windows Server, but testing locally first)
• step-ca Version - v0.27.4 (build with CGO flag)
• PowerShell 5.1 (since I'm gonna run this on a server where we don't have the newer PowerShell versions on)

Expected Behavior

Expected step-ca to start sucessfully.

Actual Behavior

step-ca fails to start saying it's unable to find the key.

PS C:\ACMECA\bin> $env:STEPPATH = "C:\ACMECA\conf"
PS C:\ACMECA\bin> .\step-ca.exe
badger 2024/09/30 13:40:28 INFO: All 0 tables opened in 0s
badger 2024/09/30 13:40:28 INFO: Replaying file id: 0 at offset: 0
badger 2024/09/30 13:40:28 INFO: Replay took: 537.9µs
failed getting key "acme-ca-ver1": not found

Additional Context

Interesting side-note, the PowerShell session also crashes shortly after this.
Not sure it's entirely related, but it is consistent.
After step-ca exists, i can press enter twice and the shell crashes.

I'm able to verify the key is working in the kms plugin by running this (which prints the public key).

PS C:\ACMECA\bin> $env:STEPPATH = "C:\ACMECA\conf"
PS C:\ACMECA\bin> .\step.exe kms key 'tpmkms:name=acme-ca-ver1'

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[hslatman]

@JMyklebust could you try initializing the KMS specifying the storage-directory? Haven't used the TPMKMS with the CA on Windows yet, but I just got a similar error while trying to reproduce your setup. I suspect the CA might be trying to look it up from a different (default) location at the moment. Some additional metadata is stored in the $env:STEPPATH/tpm directory, but in the end Windows will store the key too.

C:\Users\herman\.step\tpm>step kms key "tpmkms:name=acme-ca-ver1"
Error: open tpmkms:name=acme-ca-ver1: failed getting key "acme-ca-ver1": not found
exit status 1

C:\Users\herman\.step\tpm>step kms key "tpmkms:name=acme-ca-ver1;storage-directory=C:\Users\herman\.step\tpm"
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAvWI50jaNFmGn5oYwDMss
C0aoAnFe7ne7VVBjXZoNFmweeBotnN5BjTODbpySn6yZIEEtlZYEDiAPOKVsg4z5
eIidvcCCN3B/JH2n+Vg9Alpcj35N/25Sbq7s+1sqhEAu5CUpqiFjH7ZVaB4EBLkp
dFAR8LdWDjqqeGrcn0FS0hNGex0f/hjC73EjmnYqS0VyQhQlazNznU0zGkRasKiE
4iuUlY01hpuq6on6kwR8uz2iCrzA5xmFZXFDzbA7oRpyzpaiI5agZUPnM8ljTwI+
+/T8UWn2CFR+LScl5otpa1EUtQ44UMD3bv+ttleeHOZSOkemDZf1S8Q0XZi9o1w1
8QIDAQAB
-----END PUBLIC KEY-----

In the ca.json it would be like this:

"kms":  {
                "type":  "tpmkms",
                "uri":  "tpmkms:storage-directory=..."
            },

If that doesn't work, try specifying the full URI in the key.

Not saying it's not (partially) broken, but it may need a little bit of additional configuration and testing on Windows.

That said, if you don't need attestation of the intermediate CA key, you could try our capi KMS instead. If you use the Microsoft Platform Crypto Provider, it will use the TPM if one is available. Also note that keys created through the TPMKMS will in fact be backed by the Microsoft Platform Crypto Provider, and will thus turn up when running something like this: certutil -key -csp "Microsoft Platform Crypto Provider" -user

[hslatman]

Is there a specific reason you need to build with CGo? step-ca should build with CGO_ENABLED=0, and still support TPM interactions on Windows.

[JMyklebust]

Adding "storage-directory" did work, thanks!

    "kms":  {
                "type":  "tpmkms",
                "uri":  "tpmkms:storage-directory=C:/ACMECA/conf/tpm"
            }

As for why I'm using CGO is because i'm following the documentation here which states you need the CGO version to use the tpm.
https://smallstep.com/docs/step-ca/cryptographic-protection/#tpm-20
So if it's not needed on Windows it probably helps to specify that there.

As for using capi sure, but it's not really documented anywhere as far as i can see?
Probably also helpful to specify that tpmkms uses Microsoft Platform Crypto Provider in the background on Windows.
All of this is not really clear unless you try do do a dive into the source code of this and linked projects.

[JMyklebust]

Tested with the regular Windows build of step-ca and tpmkms works fine.
However if i try to use capi i only get the error unsupported kms type 'capi'.
At least with a keyword i could find things in the source so based on this:
https://github.com/smallstep/crypto/blob/cb53a6ab2f95f900714d03c03e9a6457cf0558af/kms/capi/capi.go#L58

Trying something like this does not work.

    "crt":  "C:/ACMECA/conf/certs/intermediate_ca.crt",
    "key":  "capi:store-location=user;store=My;sha1=24E62B993442FBC3BA3C016C215BA4361AAE533F",
    "kms":  {
                "type":  "capi",
                "uri":  "capi:"
            }

Edit: For now though tpmkms works for my usecase.

[hslatman]

Tested with the regular Windows build of step-ca and tpmkms works fine. However if i try to use capi i only get the error unsupported kms type 'capi'. At least with a keyword i could find things in the source so based on this: https://github.com/smallstep/crypto/blob/cb53a6ab2f95f900714d03c03e9a6457cf0558af/kms/capi/capi.go#L58

Trying something like this does not work.

    "crt":  "C:/ACMECA/conf/certs/intermediate_ca.crt",
    "key":  "capi:store-location=user;store=My;sha1=24E62B993442FBC3BA3C016C215BA4361AAE533F",
    "kms":  {
                "type":  "capi",
                "uri":  "capi:"
            }

Edit: For now though tpmkms works for my usecase.

My bad. It looks like we don't include that in the list of KMS's supported in the CA today: https://github.com/smallstep/certificates/blob/master/cmd/step-ca/main.go#L29-L37.

As for the TPMKMS docs: I think that may be a stray copy/paste from the YubiKey and PKCS11 KMS's. Thanks for the pointer 🙂

[JMyklebust]

I found another issue though probably slightly outside the scope of the orignal bug report.
All keys generated ends up in the user store.
However I'm trying to run step-ca under a virtual service account NT Service\<servicename>.
This means that I need to place the key under the machine store for the virtual service being able to see the key.

I'm using shawl as service wrapper and using file based keys works fine as a virtual service account (so i have a working baseline config).

Looking at tpmkms here https://github.com/smallstep/crypto/blob/cb53a6ab2f95f900714d03c03e9a6457cf0558af/kms/tpmkms/tpmkms.go#L175
I should be able to add store-location=machine;store=MY when i generate the key and when try to use it.
However trying to use step kms key "tpmkms:store-location=machine;store=MY;name=acme-ca-ver1" does not create any key in the machine store (certutil -csp "Microsoft Platform Crypto Provider" -key).

Though it's possible that capi might be the best for this use, since if you have a certificate and key in the machine store then you can more easily use certlm to delegate the private key access to the service account.
Or maybe more "correct" in a Windows environment is that then you can create your certificate/request using certreq instead of step-cli.

[hslatman]

I found another issue though probably slightly outside the scope of the orignal bug report. All keys generated ends up in the user store. However I'm trying to run step-ca under a virtual service account NT Service\<servicename>. This means that I need to place the key under the machine store for the virtual service being able to see the key.

I'm using shawl as service wrapper and using file based keys works fine as a virtual service account (so i have a working baseline config).

Looking at tpmkms here https://github.com/smallstep/crypto/blob/cb53a6ab2f95f900714d03c03e9a6457cf0558af/kms/tpmkms/tpmkms.go#L175 I should be able to add store-location=machine;store=MY when i generate the key and when try to use it. However trying to use step kms key "tpmkms:store-location=machine;store=MY;name=acme-ca-ver1" does not create any key in the machine store (certutil -csp "Microsoft Platform Crypto Provider" -key).

At the moment those additional options only work when enable-cng=true is also set. And it'll only affect operations on certificates currently. They were added to support "no key required" use cases, such as installing a trusted root in the machine store without having to create a new (TPM)KMS instance.

Though it's possible that capi might be the best for this use, since if you have a certificate and key in the machine store then you can more easily use certlm to delegate the private key access to the service account. Or maybe more "correct" in a Windows environment is that then you can create your certificate/request using certreq instead of step-cli.

Yes, I think capi would be better suited in this case. It's more aligned with the high level Windows APIs than the TPMKMS is, as the latter was implemented starting with the TPM and as an abstraction suitable for both Windows and Linux.

[JMyklebust]

I built a custom version of step-ca with adding _ "go.step.sm/crypto/kms/capi" to https://github.com/smallstep/certificates/blob/master/cmd/step-ca/main.go#L29-L37
This at least makes step-ca be able to understand capi.
However it's throwing NTE_BAD_KEYSET left and right.

First I've created a CA cert on the side using other tools (XCA in this case for simplicity during testing).
The certificate has been imported into the My store on both my user and the machine.
Tested with also making certificate exportable.

Then I've gone over to trying to create with step kms, and it get's even more confusing.

$CAName = "Test CA"
$KeyName = "acme-ca-ver1"
$TemplatesDir = "C:\ACMECA\conf\templates"
$CertsDir = "C:\ACMECA\conf\certs"
$outputJSON = & C:\ACMECA\bin\step.exe kms create --json  --kty RSA --size 2048 "capi:provider=Microsoft Platform Crypto Provider;key=$KeyName"
$ConvertedJSON = $outputJSON | ConvertFrom-Json
# Show output 
$ConvertedJSON| fl
# try to use key name
& C:\ACMECA\bin\step.exe certificate create --kms 'capi:' --key "capi:provider=Microsoft Platform Crypto Provider;key=$KeyName" --csr --template "$TemplatesDir\BKCATemplate.tpl" $CAName "$CertsDir\intermediate.csr"
# Try to use key path
& C:\ACMECA\bin\step.exe certificate create --kms 'capi:' --key "$($ConvertedJSON.name)" --csr --template "$TemplatesDir\BKCATemplate.tpl" $CAName "$CertsDir\intermediate.csr"

This is the output:

name      : capi:provider=Microsoft Platform Crypto Provider;key=C:\Users\user\AppData\Local\Microsoft\Crypto\PCPKSP\966de3ef3782562dc8705e96483ff69e7bd97ecc\9db81bc59272b9c833bcd5e9db98ea2cee810f10.PCPKEY
publicKey : -----BEGIN PUBLIC KEY-----
            MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAwROwBrn+rApzfhXDM3e2
            UOuFxVZpWEkY9VkT7GCUhgco3uzxwu7kpe3zA+Qd56tUcKqKTTUb1UJSewn/TFWW
            pNqbsqzZxC698uvT/6FMrQpES1odLgwb/oSLJtZf1wQqVn9LV1NomO4TiqqhrN/f
            aUpRlBOcYokpRTWMBMsJjEAC+lBh90eCSOxEicppNQ+2wGVVaUf5ldswhTfyqKCa
            7xBLS5YqwTLYk9dRpdl/8IxQGadBCGCNwhg8jFmPzJcy/VKXrJMkT+6vXhLZT0sm
            cid+/5b6pRmUApcCftwRvRl69iWwm/tEibc8bOI8IFwOgi+F4vooeI1Nz5KRDxdG
            uwIDAQAB
            -----END PUBLIC KEY-----


failed to get public key: command "C:\\ACMECA\\conf\\plugins\\step-kms-plugin.exe key --kms capi: capi:provider=Microsoft Platform Crypto Provider;key=acme-ca-ver2" failed with:
Error: open capi:provider=Microsoft Platform Crypto Provider;key=acme-ca-ver2: unable to open key: NCryptOpenKey for container "acme-ca-ver2" returned NTE_BAD_KEYSET

failed to get public key: command "C:\\ACMECA\\conf\\plugins\\step-kms-plugin.exe key --kms capi: capi:provider=Microsoft Platform Crypto Provider;key=C:\\Users\\user\\AppData\\Local\\Microsoft\\Crypto\\PCPKSP\\966de3ef3782562dc8705e96483ff69e7bd97ecc\\9db81bc59272b9c833bcd5e9db98ea2cee810f10.PCPKEY" failed with:
Error: open capi:provider=Microsoft Platform Crypto Provider;key=C:\Users\user\AppData\Local\Microsoft\Crypto\PCPKSP\966de3ef3782562dc8705e96483ff69e7bd97ecc\9db81bc59272b9c833bcd5e9db98ea2cee810f10.PCPKEY: unable to open key: NCryptOpenKey for container "C:\\Users\\user\\AppData\\Local\\Microsoft\\Crypto\\PCPKSP\\966de3ef3782562dc8705e96483ff69e7bd97ecc\\9db81bc59272b9c833bcd5e9db98ea2cee810f10.PCPKEY" returned NTE_BAD_KEYSET

Though at this point it might be better to open an issue over in https://github.com/smallstep/crypto?