SSPI WTF

This is a continuation of TLS WTF. I added SSPI / SChannel support to my list of supported TLS backends. It was not a pleasant experience, so I am taking another moment to document my grievances.

SSPI is a shining example of an API that is hard to use correctly.

  1. Microsoft documents that rather than linking against an export library, you must load security.dll dynamically and look up a symbol.
  2. To begin the TLS handshake, you call InitializeSecurityContext or AcceptSecurityContext, depending on whether you are the client or server. The credentials already know which direction we're going at this point, so this seems redundant.
  3. One argument to those functions is a pointer to a handle that is populated when the call succeeds, and a pointer to that handle must be passed back as a different argument on subsequent calls. But not before it succeeds once.
  4. InitializeSecurityContext, AcceptSecurityContext, EncryptMessage, and DecryptMessage all take arrays of input and output SecBuffer structs. After working with two other libraries that had much simpler mechanisms for passing data in and out, this just seemed wrong.
    1. Data is encrypted and decrypted in-place. While it might seem sort of clever, as though it's saving something or destroying no-longer-needed sensitive information, due to the way it works and the lack of any apparent guarantees, I found I needed an extra copy of the data anyway in order to properly restore the remaining unencrypted/decrypted bytes for the next pass.
    2. Each particular type of invocation has special requirements on the number and type of buffers it takes as input and returns as output. It's basically deliberately throwing away all parameter/type safety. Or conversely it's requiring the user to know quite a lot about the specifics of the protocol.
    3. One type of SecBuffer contents that is returned is the number of data bytes passed in that haven't been consumed. But not a pointer to the bytes like every other buffer.
  5. Shutting down a connection is bizarre. Call ApplyControlToken and then proceed as though you're starting a new handshake. I realize only now that I'm doing this incorrectly.
  6. I was pleased that I was pretty quickly able to load a PEM certificate and private key. And then it took me the better part of a day to be able to associate them with each other to be able to use them.
    1. A certificate context (container?) seems to be necessary. There is a way to get an anonymous one, but it doesn't seem to work for this purpose. I haven't found a way to get one if it exists and create it otherwise except by trying one and if it fails, trying the other.
    2. It is necessary to open a certificate store with the name "MY". What?
    3. A property "CERT_KEY_PROV_HANDLE_PROP_ID" on the certificate context looks exactly like what I would want to set to associate a key with a certificate, but it has no apparent effect. "CERT_KEY_PROV_INFO_PROP_ID" is the only one that apparently actually works, but it requires jumping through a bunch more hoops.
    4. All of this leaves some key/certificate data in some system location. I haven't found any way to avoid that.
    5. Most of the API worked with multi-byte or wide characters. One or two functions didn't.
    6. I've completely lost track of which things are opaque data types, handles, pointers to handles, or pointers to pointers to handles. It's out of control.

Again, I just hope someone stumbles on this or browser:projects/sandboxos/trunk/src/Tls.cpp, and is saved some of the hassle I went through.

I realize I am not done, but I really need to put this aside for now. Some future tasks include:

  • Support allowing individual certificates whose signing authority isn't otherwise respected (self-signed certificates).
  • Need to separate the certificate and key setup from the connection. I'm probably adding an awful amount of overhead for each connection by re-importing them each time.
  • Need to expose error messages. I've just been instrumenting the code as I go to discover what I've been doing wrong, but the actual errors are useful, so I need to make them available.
  • Need actual tests.
  • I want to be able to generate self-signed certificates on all platforms.

Comments

No comments.

Add New Comment