SSH private key credential is corrupted by export + import

[mattiasflodin]

Jenkins and plugins versions report

Environment

Jenkins: 2.426.1
OS: Linux - 6.2.0-1016-azure
---
antisamy-markup-formatter:162.v0e6ec0fcfcf6
apache-httpcomponents-client-4-api:4.5.14-208.v438351942757
azure-ad:442.v355cca_6b_c169
azure-sdk:157.v855da_0b_eb_dc2
bitbucket:223.vd12f2bca5430
bootstrap5-api:5.3.2-2
bouncycastle-api:2.29
branch-api:2.1135.v8de8e7899051
caffeine-api:3.1.8-133.v17b_1ff2e0599
checks-api:2.0.2
cloudbees-folder:6.858.v898218f3609d
commons-lang3-api:3.13.0-62.v7d18e55f51e2
commons-text-api:1.11.0-94.v3e1f4a_926e49
configuration-as-code:1746.vf1673cfe690a
credentials:1309.v8835d63eb_d8a_
credentials-binding:642.v737c34dea_6c2
display-url-api:2.200.vb_9327d658781
durable-task:523.va_a_22cf15d5e0
echarts-api:5.4.3-1
font-awesome-api:6.4.2-1
git:5.2.1
git-client:4.5.0
gradle:2.9
instance-identity:185.v303dc7c645f9
ionicons-api:56.v1b_1c8c49374e
jackson2-api:2.15.3-372.v309620682326
jakarta-activation-api:2.0.1-3
jakarta-mail-api:2.0.1-3
javax-activation-api:1.2.0-6
javax-mail-api:1.6.2-9
jaxb:2.3.9-1
jquery3-api:3.7.1-1
jsch:0.2.8-65.v052c39de79b_2
junit:1240.vf9529b_881428
mailer:463.vedf8358e006b_
matrix-auth:3.2.1
matrix-project:818.v7eb_e657db_924
mercurial:1260.vdfb_723cdcc81
mina-sshd-api-common:2.11.0-86.v836f585d47fa_
mina-sshd-api-core:2.11.0-86.v836f585d47fa_
okhttp-api:4.11.0-157.v6852a_a_fa_ec11
plain-credentials:143.v1b_df8b_d3b_e48
plugin-util-api:3.6.0
prism-api:1.29.0-9
scm-api:683.vb_16722fb_b_80b_
script-security:1294.v99333c047434
snakeyaml-api:2.2-111.vc6598e30cc65
ssh-credentials:308.ve4497b_ccd8f4
ssh-slaves:2.916.vd17b_43357ce4
structs:325.vcb_307d2a_2782
trilead-api:2.84.v72119de229b_7
workflow-api:1283.v99c10937efcb_
workflow-basic-steps:1042.ve7b_140c4a_e0c
workflow-cps:3812.vc3031a_b_a_c955
workflow-durable-task-step:1289.v4d3e7b_01546b_
workflow-job:1360.vc6700e3136f5
workflow-multibranch:756.v891d88f2cd46
workflow-scm-step:415.v434365564324
workflow-step-api:639.v6eca_cd8c04a_a_
workflow-support:865.v43e78cc44e0d

What Operating System are you using (both controller, and any agents involved in the problem)?

Controller: jenkins/jenkins:lts-alpine docker image (Alpine Linux 3.18)
Agent: jenkins/ssh-agent:latest-jdk17 docker image (Debian GNU/Linux 12)

Reproduction steps

1. Follow Jenkins handbook steps to setting up an SSH credential (add SSH username with private key under Manage Jenkins > Credentials, and paste the SSH private key under "Enter directly").
2. Also according to the same handbook steps, start up a docker-ssh-agent instance with the corresponding public key, add a node with "Launch agents via SSH" launch method in Jenkins, and select the credentials just added.
3. Save. The node connects without issues.
4. Go to Manage Jenkins > Configuration as Code and click "View configuration". Copy credentials settings that contains the privateKey value, and paste into the JCasC configuration file.
5. Wipe docker containers and volumes etc to start from a clean slate, then start everything up again to let JCasC configure the credentials from the YAML configuration.
6. After starting up Jenkins from a clean slate, add a node in the same way again using the same credential.

Expected Results

After SSH credential has been set up by JCasC, the SSH agent connects just like it did when the credentials were set up through the GUI .

Actual Results

The node cannot connect. Looking at log, it complains that the PEM for the private key is of an unknown type, which suggests to me that it has been corrupted. Full log:

SSHLauncher{host='jenkins-agent', port=22, credentialsId='jenkins', jvmOptions='', javaPath='', prefixStartSlaveCmd='', suffixStartSlaveCmd='', launchTimeoutSeconds=60, maxNumRetries=10, retryWaitTime=15, sshHostKeyVerificationStrategy=hudson.plugins.sshslaves.verifiers.ManuallyTrustedKeyVerificationStrategy, tcpNoDelay=true, trackCredentials=true}
[11/30/23 06:43:07] [SSH] Opening SSH connection to jenkins-agent:22.
[11/30/23 06:43:07] [SSH] SSH host key matches key seen previously for this host. Connection will be allowed.
ERROR: Server rejected the 1 private key(s) for jenkins (credentialId:jenkins/method:publickey)
ERROR: Failed to authenticate as jenkins with credential=jenkins
java.io.IOException: Publickey authentication failed.
	at com.trilead.ssh2.auth.AuthenticationManager.authenticatePublicKey(AuthenticationManager.java:349)
	at com.trilead.ssh2.Connection.authenticateWithPublicKey(Connection.java:472)
	at com.cloudbees.jenkins.plugins.sshcredentials.impl.TrileadSSHPublicKeyAuthenticator.doAuthenticate(TrileadSSHPublicKeyAuthenticator.java:110)
	at com.cloudbees.jenkins.plugins.sshcredentials.SSHAuthenticator.authenticate(SSHAuthenticator.java:431)
	at com.cloudbees.jenkins.plugins.sshcredentials.SSHAuthenticator.authenticate(SSHAuthenticator.java:468)
	at hudson.plugins.sshslaves.SSHLauncher.openConnection(SSHLauncher.java:878)
	at hudson.plugins.sshslaves.SSHLauncher.lambda$launch$0(SSHLauncher.java:434)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException: PEM problem: it is of unknown type. Supported algorithms are :[ssh-ed25519, ecdsa-sha2-nistp521, ecdsa-sha2-nistp384, ecdsa-sha2-nistp256, rsa-sha2-256, rsa-sha2-512, ssh-rsa, ssh-dss]
	at com.trilead.ssh2.crypto.PEMDecoder.decodeKeyPair(PEMDecoder.java:482)
	at com.trilead.ssh2.auth.AuthenticationManager.authenticatePublicKey(AuthenticationManager.java:290)
	... 10 more
[11/30/23 06:43:07] [SSH] Authentication failed.
Authentication failed.
[11/30/23 06:43:07] Launch failed - cleaning up connection
[11/30/23 06:43:07] [SSH] Connection closed.

Anything else?

I'm aware that there is a warning in the GUI that the exported configuration is not intended to be directly usable. However, this is the only way I know of to find out how to actually configure an SSH private key, as I have not found any documentation on how to do it. As an alternative, I have tried reading the key from a file like so:

credentials:
  system:
    domainCredentials:
    - credentials:
      - basicSSHUserPrivateKey:
          id: "jenkins"
          privateKeySource:
            directEntry:
              privateKey: "${readFileBase64:/usr/share/jenkins/agent_key}"
          scope: GLOBAL
          username: "jenkins"

but this gives me the same error at connection. So if a non-working exported configuration is the intended behavior, please direct me to a description of how to set up an SSH private key credential using JCasC. I would actually very much prefer to read it from a separate file since I don't want to keep secrets in the YAML file.

[Joshua-Gr]

@mattiasflodin you should read the private key file with "readFile" and not with "readFileBase64", it works great for me.

[mattiasflodin]

@mattiasflodin you should read the private key file with "readFile" and not with "readFileBase64", it works great for me.

You're right, thank you! This needs to be documented.