"Fossies" - the Fresh Open Source Software Archive

Member "memcached-1.6.15/doc/tls.txt" (21 Feb 2022, 7644 Bytes) of package /linux/www/memcached-1.6.15.tar.gz:


As a special service "Fossies" has tried to format the requested text file into HTML format (style: standard) with prefixed line numbers. Alternatively you can here view or download the uninterpreted source code file.

    1 Securing Memcached with TLS
    2 
    3 Requirements
    4 ------------
    5 We are required to encrypt Memcached network traffic as we deploy our servers in public cloud
    6 environments. We decided to implement SSL/TLS for TCP at the network layer of Memcached
    7 using OpenSSL libraries. This provides following benefits with the expense of added latency
    8 and reduced throughput (to be quantified).
    9 
   10 # Encryption :Data is encrypted on the wire between Memcached client and server.
   11 # Authentication : Optionally, both server and client authenticate each other.
   12 # Integrity: Data is not tampered or altered when transmitted between client and server
   13 
   14 Following are a few additional features.
   15 # Certificate refresh: when the server gets a new certificate, new connections
   16 will use new certificates without a need of re-starting the server process.
   17 
   18 # Multiple ports with and without TLS : by default all TCP ports are secured. Optionally we can setup
   19 the server to secure a specific TCP port.
   20 
   21 Note that initial implementation does not support session renegotiation.
   22 
   23 Design
   24 ------
   25 We experimented two options for implementing TLS, with SSL buffered events and directly using
   26 OpenSSL API.
   27 
   28 Bufferevents can use the OpenSSL library to implement SSL/TLS. Our experiment used
   29 a socket-based bufferevent that tells OpenSSL to communicate with the network directly over.
   30 Unlike a worker thread sets callback on the socket, this uses a “bufferevent” object for
   31 callbacks. Memcached still has to setup the SSL Context but SSL handshake and object
   32 management is done via the “bufferevent_” API. While this was fairly easy to implement,
   33 we noticed a higher memory usage as we don’t have much control over allocating evbuffer
   34 objects in bufferevents. More over there is a discussion on removing the libevent dependency
   35 from Memcached; hence this option was not chosen.
   36 
   37 OpenSSL library provides APIs for us to directly read/write from a socket. With this option,
   38 we create an SSL Context and many SSL objects. The SSL Context object, created at the process level,
   39 holds certificates, a private key, and options regarding the TLS protocol and algorithms.
   40 SSL objects, created at the connection level, represents SSL sessions. SSL objects are responsible
   41 for encryption, and session handshake among other things.
   42 
   43 There are two ways to do network IO over TLS, either only use SSL_read/SSL_write with a network socket or
   44 use the API along with an output/input buffer pair. These buffers are referred as BIO
   45 (Basic Input Output) buffers.
   46 
   47 We started with the first option, create SSL objects with the socket and only interact with SSL_read/SSL_write.
   48 
   49   +------+                                    +-----+
   50   |......|--> read(fd) --> BIO_write(rbio) -->|.....|--> SSL_read(ssl)  --> IN
   51   |......|                                    |.....|
   52   |.sock.|                                    |.SSL.|
   53   |......|                                    |.....|
   54   |......|<-- write(fd) <-- BIO_read(wbio) <--|.....|<-- SSL_write(ssl) <-- OUT
   55   +------+                                    +-----+
   56           |                                  |       |                     |
   57           |<-------------------------------->|       |<------------------->|
   58           |         encrypted bytes          |       |  unencrypted bytes  |
   59 
   60                       Figure 1 : Network sockets, BIO buffers and SSL_read/SSL_write
   61 
   62 (reference:  https://gist.github.com/darrenjs/4645f115d10aa4b5cebf57483ec82eca)
   63 
   64 Memcached uses non blocking sockets and implements a rather complex state machine for
   65 network IO. A listener thread does the TCP handshake and initiates the SSL handshake after
   66 creating an SSL object based on the SSL Context object of the server. If there are no
   67 fatal errors, the listener thread hands over the socket to a worker thread. A worker completes
   68 the SSL handshake.
   69 
   70 -----------                       ----------------------
   71           |                       |
   72   Client  |                       |  Memcached Server
   73           |                       |
   74           |                       |---------------------
   75           |                       |   Listener thread  |
   76           |     TCP connect       |                    |
   77           |---------------------> | (accept)           |
   78           |    ClientHello        |                    |
   79           |---------------------> | (SSL_accept)       |
   80           |                       |                    |
   81           |    ServerHello and    |                    |
   82           |    Certificate,       |                    |
   83           |    ServerHelloDone    |                    |
   84           | <---------------------|                    |
   85           |                       |---------------------
   86           |                       |         |
   87           |                       |         V
   88           |                       |-------------------
   89           |                       |  Worker thread   |
   90           | ClientKeyExchange,    |                  |
   91           | ChangeCipherSpec,     |                  |
   92           | Finished              |                  |
   93           |---------------------> | (SSL_read)       |
   94           |                       |                  |
   95           |                       |                  |
   96           | NewSessionTicket,     |                  |
   97           | ChangeCipherSpec,     |                  |
   98           | Finished              |                  |
   99           | <---------------------|                  |
  100           |                       |                  |
  101           | Memcached request/    |                  |
  102           |    response           |                  |
  103           | <-------------------> | (SSL_read/       |
  104           |                       |   SSL_write)     |
  105 -----------                       -------------------------
  106 
  107                       Figure 2 : The initial SSL handshake
  108 
  109 
  110 Setting-up callbacks when the socket is ready for reading/writing is the same
  111 for both TLS and non-TLS connections. When the socket is ready, the state machine kicks off
  112 and issues a SSL_read/ SSL_write. Note that we implement a SSL_sendmsg wrapper on top of
  113 SSL_write to simulate the sendmsg API.
  114 This way we don't explicitly use BIO buffers or do BIO_write/BIO_read, but let OpenSSL
  115 library to do it on our behalf. Existing state machine takes care of reading the correct amount
  116 of bytes and do the error handling when needed.
  117 
  118 As a best practice, server certificates and keys are periodically refreshed by the PKI.
  119 When this happens we want server to use the new certificate without restarting the process.
  120 Memcached is a cache and restarting servers affects the latency of applications. We implement
  121 the automatic certificate refresh through a command. Upon receiving the "refresh_certs" command,
  122 the server reloads the certificates and key to the SSL Context object. Existing connection won't be
  123 interrupted but new connections will use the new certificate.
  124 
  125 We understand not all users want to use TLS or have the OpenSSL dependency. Therefore
  126 it's an optional module at the compile time. We can build a TLS capable Memcached server with
  127 "./configure --enable-tls". Once the server is built with TLS support, we can enabled it with
  128 "-Z" flag or "--enable-ssl". Certificate (-o ssl_chain_cert) and (-o ssl_key) are required
  129 parameters while others are optional. Supported options can be listed through "memcached -h".
  130 
  131 Developers need to have libio-socket-ssl-perl installed for running unit tests. When the server is
  132 built with TLS support, we can use "test_tls" make target to run all existing tests over TLS and some
  133 additional TLS specific tests. The minimum required OpenSSL version is 1.1.0g.