TLS WTF

This is my recollection of my misadventures in trying to support TLS for SandboxOS.

The Goal

I have a JavaScript runtime environment. I want it to be able to do these things:

  • Run a web server that supports HTTPS to secure or even just obfuscate my traffic.
  • Connect to web servers over HTTPS to post to Twitter and whatnot.
  • Connect to XMPP and IRC servers requiring secure connections to run chat clients and bots.

I want it to be able to do those things on Linux, OS X, and Windows, and I don't have a very high tolerance for complicated build steps or large dependencies.

Background

"I want to use SSL," I thought to myself. Apparently what I wanted is called TLS these days. I scratched my head and moved on, because this was the least of my problems.

Going in, I knew that OpenSSL had recent vulnerabilities, leading it to be forked into LibreSSL. I knew that GnuTLS was a thing. Outside of that, I did not have much knowledge about what was available.

I did some research and found that I should look into Mozilla's NSS as something that could potentially work on all the platforms I cared about. I also learned that if I wanted to support the most common libraries on each platform, I would need to look into SSPI on Windows. I also saw that node.js uses OpenSSL on all platforms.

I'm using libuv for doing all of my socket I/O, and I'm pretty happy with it. But it poses a challenge here, because these libraries tend to prefer being used as a wrapper on top of native BSD-style sockets and I didn't want TLS interfering with libuv's event loops at all. I chose to try to pass data around myself in order to avoid that scenario. It looks like NSS creates unnecessary intermediate sockets to abstract away that interface.

Getting Started: Build the Libraries

I believe it went like this:

  1. Look at Mozilla NSS (Network Security Services).
    • GPL-compatible license. Good.
    • Supports the platforms I care about. Good.
    • Get it and try to build it. They have a non-trivial custom build harness. Not a great sign, but OK.
    • Try to build it for win32/x64. Discover this note:

      Note: Building for a 64-bit environment/ABI is only supported on Unix/POSIX platforms.

    • Looked at the API a bit. It looks like it really wants to be bound to system sockets. That will incur unnecessary overhead the way I want to use it. Lame.
    • That was enough strikes for me. I wan't about to maintain my own builds of this for three platforms, and I certainly wasn't about to let this constrain me to Win32/x86, if that note was correct.
  2. Look at LibreSSL.
    • GPL-compatible license. Good.
    • It looks like the latest release is expected to be usable. Great.
    • It only builds on Windows through MinGW. Ugg.
    • By this time I had some OpenSSL code. I tried it on OS X and found that the OpenSSL system library on OS X was supported but has deprecated for some time. Crap.

I Wrote a TLS Wrapper

I resolved to support OpenSSL on Linux, the Secure Transport API on OS X, and SSPI on Windows, and all in code that could be easily extracted to use for other projects.

Currently the entirety of the public API looks like this, though I haven't yet tackled SSPI:

class Tls {
public:
    static Tls* create(const char* key, const char* certificate);
    virtual ~Tls() {}

    virtual void startAccept() = 0;
    virtual void startConnect() = 0;
    virtual void shutdown() = 0;

    enum HandshakeResult {
        kDone,
        kMore,
        kFailed,
    };
    virtual HandshakeResult handshake() = 0;

    enum ReadResult {
        kReadZero = -1,
        kReadFailed = -2,
    };
    virtual int readPlain(char* buffer, size_t bytes) = 0;
    virtual int writePlain(const char* buffer, size_t bytes) = 0;

    virtual int readEncrypted(char* buffer, size_t bytes) = 0;
    virtual int writeEncrypted(const char* buffer, size_t bytes) = 0;

    virtual void setHostname(const char* hostname) = 0;
};

Implementation Rants

I don't know where to begin.

  1. There are not enough good examples. If you are an engineer at Apple who worked on the Security framework, where did you put your ~100 line C file test case that fetches https://www.apple.com/ and fails if you try to reach the same server by a different name?
    • This was the best I could find for OpenSSL. It's alright but 12 years old.
    • This is why I haven't attempted SSPI yet. I think SSPI will be manageable, but that example is insane.
  2. Do not pretend TLS connections are BSD-style sockets. TLS is a terribly leaky abstraction. Socket calls that ought not to block might need to block so that TLS handshaking can finish. New errors can occur at every turn. A TLS session can close without the network connection going away. Stop pretending it's not a separate layer.
  3. Verify hostnames. TLS without hostname verification isn't secure. OpenSSL did not make verifying hostnames easy. The headers I have on debian jessie required me to extract the names from the certificate and do my own pattern matching to account for wildcard certificates. I had several implementations where it appeared the default verification would check names, but only testing showed that it wasn't happening. Apple made it a fair bit easier, but it still didn't happen by default.
  4. So far I have not yet found how to load my certificate and private key from PEM files on OS X without calling this private function from the Security framework:
    extern "C" SecIdentityRef SecIdentityCreate(CFAllocatorRef allocator, SecCertificateRef certificate, SecKeyRef privateKey);
    
    All other attempts I've made at getting a SecIdentityRef from my key and certificate have failed. I could keep them in the login keychain, but I want to support PEM on all platforms for consistency.

Results

So SandboxOS does what I need for TLS for now. It took me five times longer to write than I had hoped, and doesn't support TLS on Windows yet.

It can connect to secure servers. It verifies hostnames when you do that. I can run a secure web server with it. It might be fun to support client certificates, but that is about as far as I want this to go, and that can happen later.

I'm hoping somebody searching for some of these words will stumble upon this and either show me how stupid I've been or benefit from browser:projects/sandboxos/trunk/src/Tls.cpp.

Update

I added SSPI support.

  • Posted: 2015-01-02 21:09 (Updated: 2015-01-05 21:41)
  • Categories: code internet

Comments

No comments.

Add New Comment