Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add PEM support #587

Merged
merged 51 commits into from
Aug 20, 2024
Merged

Add PEM support #587

merged 51 commits into from
Aug 20, 2024

Conversation

sjaeckel
Copy link
Member

@sjaeckel sjaeckel commented Mar 16, 2022

Checklist

  • documentation is added or updated
  • tests are added or updated

Summary

This adds support to decode most variations of PEM files.

Changes to existing public APIs

The following public APIs have been changed. None of those APIs have been officially released.

  • ed25519_import_pkcs8()
  • ecc_import_pkcs8()
  • rsa_import_pkcs8()
  • x25519_import_pkcs8()

New public APIs and structs

structs

  • typedef struct password_ctx - a struct containing a call-back function that will be called once a password is required and the according opaque userdata pointer usually provided
  • typedef struct ltc_pka_key - a union containing all supported PK keys

PKCS#8 APIs

  • dh_import_pkcs8()
  • dsa_import_pkcs8()

PEM bytewise APIs

  • pem_decode()
  • pem_decode_pkcs()
  • pem_decode_openssh()

PEM FILE-based APIs

  • pem_decode_filehandle()
  • pem_decode_pkcs_filehandle()
  • pem_decode_openssh_filehandle()

New demos

  • openssh-privkey - not really a usable demo, more like a historical artifact of what this started from

Details

It brings support for:

  • OpenSSH style private key storage, both plain and encrypted, and public keys (in PEM format. authorized_keys format not supported yet)
  • PEM style private key storage, both plain and encrypted, and public keys
  • PKCS#8 style private key storage, both plain and encrypted

All supported PK crypto algorithms can be decoded:

  • Curve25519 (Ed25519 & X25519)
  • DH
  • DSA
  • ECC
  • RSA

@sjaeckel
Copy link
Member Author

@karel-m do you maybe have time to review this? otherwise I'll merge it in the next days

@sjaeckel sjaeckel requested review from karel-m and removed request for karel-m October 9, 2023 12:39
@karel-m
Copy link
Member

karel-m commented Oct 9, 2023

@karel-m
Copy link
Member

karel-m commented Oct 9, 2023

I see these warnings

$ make CFLAGS="-O2 -DUSE_LTM -DLTM_DESC -I../libtommath -Wall" -f makefile.unix
cc -Isrc/headers -Itests -DLTC_SOURCE -O2 -DUSE_LTM -DLTM_DESC -I../libtommath -Wall -c src/pk/ecc/ecc_import_openssl.c -o src/pk/ecc/ecc_import_openssl.o
src/pk/ecc/ecc_import_openssl.c: In function ‘s_ecc_import_private_with_oid’:
src/pk/ecc/ecc_import_openssl.c:30:10: warning: implicit declaration of function ‘ecc_import_with_oid’ [-Wimplicit-function-declaration]
   30 |    err = ecc_import_with_oid(bin_k, seq_priv[1].size, curveoid, custom[0].size, PK_PRIVATE, key);
      |          ^~~~~~~~~~~~~~~~~~~
src/pk/ecc/ecc_import_openssl.c:14:25: warning: unused variable ‘curve’ [-Wunused-variable]
   14 |    const ltc_ecc_curve *curve;
      |                         ^~~~~
src/pk/ecc/ecc_import_openssl.c:13:9: warning: unused variable ‘OID’ [-Wunused-variable]
   13 |    char OID[256];
      |         ^~~
src/pk/ecc/ecc_import_openssl.c:12:18: warning: unused variable ‘len’ [-Wunused-variable]
   12 |    unsigned long len, pkver = 0, curveoid[16];
      |                  ^~~
cc -Isrc/headers -Itests -DLTC_SOURCE -O2 -DUSE_LTM -DLTM_DESC -I../libtommath -Wall -c src/pk/ecc/ecc_import_x509.c -o src/pk/ecc/ecc_import_x509.o
src/pk/ecc/ecc_import_x509.c: In function ‘s_ecc_import_x509_with_oid’:
src/pk/ecc/ecc_import_x509.c:21:10: warning: implicit declaration of function ‘ecc_import_with_oid’; did you mean ‘s_ecc_import_x509_with_oid’? [-Wimplicit-function-declaration]
   21 |    err = ecc_import_with_oid(bin_xy, len_xy, curveoid, len_oid, PK_PUBLIC, key);
      |          ^~~~~~~~~~~~~~~~~~~
      |          s_ecc_import_x509_with_oid
src/pk/ecc/ecc_import_x509.c:13:25: warning: unused variable ‘curve’ [-Wunused-variable]
   13 |    const ltc_ecc_curve *curve;
      |                         ^~~~~
src/pk/ecc/ecc_import_x509.c:12:9: warning: unused variable ‘OID’ [-Wunused-variable]
   12 |    char OID[256];
      |         ^~~
src/pk/ecc/ecc_import_x509.c:11:35: warning: unused variable ‘len’ [-Wunused-variable]
   11 |    unsigned long len_xy, len_oid, len;
      |                                   ^~~
src/pk/ecc/ecc_import_x509.c: In function ‘ecc_import_subject_public_key_info’:
src/pk/ecc/ecc_import_x509.c:34:10: warning: implicit declaration of function ‘ecc_import_with_curve’ [-Wimplicit-function-declaration]
   34 |    err = ecc_import_with_curve(in, inlen, PK_PUBLIC, key);
      |          ^~~~~~~~~~~~~~~~~~~~~

@sjaeckel
Copy link
Member Author

sjaeckel commented Oct 9, 2023

Yeah, something is lost somewhere ... currently looking into it.

@karel-m
Copy link
Member

karel-m commented Oct 10, 2023

If we want to support even more ciphers here is how you can generate more test vectors via openssl

openssl genpkey -algorithm rsa -out rsa_priv.pem

openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aes-128-cbc       -out rsa_priv-aes-128-cbc.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aes-128-cfb       -out rsa_priv-aes-128-cfb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aes-128-cfb1      -out rsa_priv-aes-128-cfb1.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aes-128-cfb8      -out rsa_priv-aes-128-cfb8.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aes-128-ctr       -out rsa_priv-aes-128-ctr.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aes-128-ofb       -out rsa_priv-aes-128-ofb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aes-192-cbc       -out rsa_priv-aes-192-cbc.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aes-192-cfb       -out rsa_priv-aes-192-cfb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aes-192-cfb1      -out rsa_priv-aes-192-cfb1.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aes-192-cfb8      -out rsa_priv-aes-192-cfb8.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aes-192-ctr       -out rsa_priv-aes-192-ctr.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aes-192-ofb       -out rsa_priv-aes-192-ofb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aes-256-cbc       -out rsa_priv-aes-256-cbc.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aes-256-cfb       -out rsa_priv-aes-256-cfb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aes-256-cfb1      -out rsa_priv-aes-256-cfb1.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aes-256-cfb8      -out rsa_priv-aes-256-cfb8.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aes-256-ctr       -out rsa_priv-aes-256-ctr.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aes-256-ofb       -out rsa_priv-aes-256-ofb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aria-128-cbc      -out rsa_priv-aria-128-cbc.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aria-128-cfb      -out rsa_priv-aria-128-cfb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aria-128-cfb1     -out rsa_priv-aria-128-cfb1.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aria-128-cfb8     -out rsa_priv-aria-128-cfb8.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aria-128-ctr      -out rsa_priv-aria-128-ctr.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aria-128-ofb      -out rsa_priv-aria-128-ofb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aria-192-cbc      -out rsa_priv-aria-192-cbc.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aria-192-cfb      -out rsa_priv-aria-192-cfb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aria-192-cfb1     -out rsa_priv-aria-192-cfb1.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aria-192-cfb8     -out rsa_priv-aria-192-cfb8.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aria-192-ctr      -out rsa_priv-aria-192-ctr.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aria-192-ofb      -out rsa_priv-aria-192-ofb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aria-256-cbc      -out rsa_priv-aria-256-cbc.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aria-256-cfb      -out rsa_priv-aria-256-cfb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aria-256-cfb1     -out rsa_priv-aria-256-cfb1.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aria-256-cfb8     -out rsa_priv-aria-256-cfb8.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aria-256-ctr      -out rsa_priv-aria-256-ctr.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -aria-256-ofb      -out rsa_priv-aria-256-ofb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -bf-cbc            -out rsa_priv-bf-cbc.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -bf-cfb            -out rsa_priv-bf-cfb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -bf-ofb            -out rsa_priv-bf-ofb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -camellia-128-cbc  -out rsa_priv-camellia-128-cbc.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -camellia-128-cfb  -out rsa_priv-camellia-128-cfb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -camellia-128-cfb1 -out rsa_priv-camellia-128-cfb1.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -camellia-128-cfb8 -out rsa_priv-camellia-128-cfb8.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -camellia-128-ctr  -out rsa_priv-camellia-128-ctr.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -camellia-128-ofb  -out rsa_priv-camellia-128-ofb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -camellia-192-cbc  -out rsa_priv-camellia-192-cbc.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -camellia-192-cfb  -out rsa_priv-camellia-192-cfb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -camellia-192-cfb1 -out rsa_priv-camellia-192-cfb1.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -camellia-192-cfb8 -out rsa_priv-camellia-192-cfb8.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -camellia-192-ctr  -out rsa_priv-camellia-192-ctr.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -camellia-192-ofb  -out rsa_priv-camellia-192-ofb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -camellia-256-cbc  -out rsa_priv-camellia-256-cbc.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -camellia-256-cfb  -out rsa_priv-camellia-256-cfb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -camellia-256-cfb1 -out rsa_priv-camellia-256-cfb1.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -camellia-256-cfb8 -out rsa_priv-camellia-256-cfb8.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -camellia-256-ctr  -out rsa_priv-camellia-256-ctr.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -camellia-256-ofb  -out rsa_priv-camellia-256-ofb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -cast5-cbc         -out rsa_priv-cast5-cbc.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -cast5-cfb         -out rsa_priv-cast5-cfb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -cast5-ofb         -out rsa_priv-cast5-ofb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -chacha20          -out rsa_priv-chacha20.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -des-cbc           -out rsa_priv-des-cbc.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -des-cfb           -out rsa_priv-des-cfb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -des-cfb1          -out rsa_priv-des-cfb1.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -des-cfb8          -out rsa_priv-des-cfb8.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -des-ede-cbc       -out rsa_priv-des-ede-cbc.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -des-ede-cfb       -out rsa_priv-des-ede-cfb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -des-ede-ofb       -out rsa_priv-des-ede-ofb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -des-ede3-cbc      -out rsa_priv-des-ede3-cbc.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -des-ede3-cfb      -out rsa_priv-des-ede3-cfb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -des-ede3-cfb1     -out rsa_priv-des-ede3-cfb1.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -des-ede3-cfb8     -out rsa_priv-des-ede3-cfb8.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -des-ede3-ofb      -out rsa_priv-des-ede3-ofb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -des-ofb           -out rsa_priv-des-ofb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -desx-cbc          -out rsa_priv-desx-cbc.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -idea-cbc          -out rsa_priv-idea-cbc.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -idea-cfb          -out rsa_priv-idea-cfb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -idea-ofb          -out rsa_priv-idea-ofb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -rc2-40-cbc        -out rsa_priv-rc2-40-cbc.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -rc2-64-cbc        -out rsa_priv-rc2-64-cbc.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -rc2-cbc           -out rsa_priv-rc2-cbc.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -rc2-cfb           -out rsa_priv-rc2-cfb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -rc2-ofb           -out rsa_priv-rc2-ofb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -rc5-cbc           -out rsa_priv-rc5-cbc.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -rc5-cfb           -out rsa_priv-rc5-cfb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -rc5-ofb           -out rsa_priv-rc5-ofb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -seed-cbc          -out rsa_priv-seed-cbc.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -seed-cfb          -out rsa_priv-seed-cfb.pem
openssl pkey -in rsa_priv.pem -inform PEM -traditional -outform PEM -passout pass:secret -seed-ofb          -out rsa_priv-seed-ofb.pem

@sjaeckel
Copy link
Member Author

In my perl module test suite I have these (the password is: secret)

Do we want to support them?

I've also added support for seed. I had to create our own keys, since the ones you have don't match the private keys we already use.

FYI seed is now a legacy algorithm in OpenSSL

@sjaeckel
Copy link
Member Author

If we want to support even more ciphers here is how you can generate more test vectors via openssl

hmm, I'll have a look

sjaeckel and others added 25 commits August 20, 2024 13:29
Signed-off-by: Steffen Jaeckel <[email protected]>
Valgrind 3.15.0 on Ubuntu 20.04 reports a false positive [0]

```
==7922== Conditional jump or move depends on uninitialised value(s)
==7922==    at 0x461F0C: s_decode_header (pem_ssh.c:316)
[...]
```

Simply suppress this false positive.

[0] https://github.com/libtom/libtomcrypt/actions/runs/6507805191/job/17676616149?pr=587

Signed-off-by: Steffen Jaeckel <[email protected]>
Signed-off-by: Steffen Jaeckel <[email protected]>
The design before was not completely fine. The user had to allocate the
buffer and passed ownership to the library.
As of [0] this seems to be a problem in some environments.

[0] #587 (comment)

Signed-off-by: Steffen Jaeckel <[email protected]>
Signed-off-by: Steffen Jaeckel <[email protected]>
The user can now pass a `free()` function pointer that will be used to
free the memory that has been allocated by the `callback()`.
If `free()` is NULL, the library will still call `XFREE()`.

Signed-off-by: Steffen Jaeckel <[email protected]>
Signed-off-by: Steffen Jaeckel <[email protected]>
1. ChaCha20, two-key 3DES and DES-X encrypted OpenSSL PEM files

2. AES-GCM and Chacha20+Poly1305 encrypted SSH keys

* OpenSSH uses a slightly different algorithm for its
  `[email protected]` than defined in the RFC.
  Therefore add an `openssh_compat` flag to
  `chacha20poly1305_state`.
* Add the option to give a 16byte IV and no counter, when calling
  `chacha20poly1305_memory()`
* Add support for DES-X

Signed-off-by: Steffen Jaeckel <[email protected]>
This also changes the requirements when calling `ecc_find_curve()` that
the `cu` argument can be NULL.

Signed-off-by: Steffen Jaeckel <[email protected]>
Fixup of 5ad1681

Signed-off-by: Steffen Jaeckel <[email protected]>
Signed-off-by: Steffen Jaeckel <[email protected]>
Signed-off-by: Steffen Jaeckel <[email protected]>
@sjaeckel sjaeckel merged commit 668bd74 into develop Aug 20, 2024
74 checks passed
@sjaeckel sjaeckel deleted the add-pem-support branch August 20, 2024 12:34
@sjaeckel sjaeckel mentioned this pull request Aug 21, 2024
2 tasks
@sjaeckel sjaeckel mentioned this pull request Sep 5, 2024
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cannot compile with msvc2012 x86
3 participants