From 3f4434043194de2b88c89fc2b7b1bfc79513af5e Mon Sep 17 00:00:00 2001 From: prototypicalpro Date: Mon, 22 Mar 2021 17:58:18 +0000 Subject: [PATCH] =?UTF-8?q?Deploying=20to=20gh-pages=20from=20=20@=20a2708?= =?UTF-8?q?d0b54d0bcb8877ca05e83f872ad26ab651e=20=F0=9F=9A=80?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- index.html | 88 +++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 61 insertions(+), 27 deletions(-) diff --git a/index.html b/index.html index 1e9db8c..20db02f 100644 --- a/index.html +++ b/index.html @@ -88,12 +88,12 @@ $(document).ready(function(){initNavTree('index.html',''); initResizable(); });

CI

SSLClient adds TLS 1.2 functionality to any network library implementing the Arduino Client interface, including the Arduino EthernetClient and WiFiClient classes. SSLClient was created to integrate TLS seamlessly with the Arduino infrastructure using BearSSL as an underlying TLS engine. Unlike ArduinoBearSSL, SSLClient is completly self-contained, and does not require any additional hardware (other than a network connection).

-

SSLClient officially supports SAMD21, SAM3X, ESP32, TIVA C, STM32F7, and Teensy >= 3.0; but it should work on any board with at least 110kb flash and 7kb RAM. SSClient does not currently support ESP8266 (see this issue) or AVR due to memory constraints on both platforms.

+

SSLClient officially supports SAMD21, SAM3X, ESP32, TIVA C, STM32F7, and Teensy >= 3.0; but it should work on any board with at least 110kB flash and 7kB RAM. SSClient does not currently support ESP8266 (see this issue) or AVR due to memory constraints on both platforms.

You can also view this README in doxygen.

Overview

Using SSLClient is similar to using any other Arduino-based Client class, as this library was developed around compatibility with EthernetClient. There are a few extra things, however, that you will need to get started:

    -
  1. Board and Network Peripheral - Your board should have a lot of resources (>110kb flash and >7kb RAM), and your network peripheral should have a large internal buffer (>7kb). This library was tested with the Adafruit Feather M0 (256K flash, 32K RAM) and the Adafruit Ethernet Featherwing (16kb Buffer), and we still had to modify the Arduino Ethernet library to support larger internal buffers per socket (see the Implementation Gotchas).
    2. +
  2. Board and Network Peripheral - Your board should have a lot of resources (>110kB flash and >7kB RAM), and your network peripheral should have a large internal buffer (>7kB). This library was tested with the Adafruit Feather M0 (256K flash, 32K RAM) and the Adafruit Ethernet Featherwing (16kB Buffer), and we still had to modify the Arduino Ethernet library to support larger internal buffers per socket (see the Implementation Gotchas).
  3. Trust Anchors - You will need a header containing array of trust anchors (example), which are used to verify the SSL connection later on. This file must generated for every project. Check out TrustAnchors.md on how to generate this file for your project, and for more information about what a trust anchor is.
  4. Network Peripheral Driver Implementing Client - Examples include EthernetClient, WiFiClient, and so on—SSLClient will run on top of any network driver exposing the Client interface.
  5. Analog Pin - Used for generating random data at the start of the connection (see the Implementation Gotchas).
    6. @@ -150,7 +150,7 @@ $(document).ready(function(){initNavTree('index.html',''); initResizable(); });
    // wait for response
    while (!client.available()) { /* ... */ }
    // ...
    -

Notice that every single write() call immediately writes to the network, which is fine with most network clients. With SSL, however, if we are encrypting and writing to the network every write() call, this will result in a lot of small encryption tasks. Encryption takes a lot of time and code, so to reduce the overhead of an SSL connection, SSLClient::write implicitly buffers until the developer states that they are waiting for data to be received with SSLClient::available. A simple example can be found below:

+

Notice that every single client.write() call immediately writes to the network. This behavior is fine for most network clients; with SSL, however, it results in many small encryption tasks that consume resources. To reduce the overhead of an SSL connection, SSLClient::write implicitly buffers until the developer states that they are waiting for data to be received with SSLClient::available. A simple example can be found below:

{C++}
EthernetClient baseClient;
SSLClient client(baseClient, TAs, (size_t)2, A7);
@@ -167,15 +167,17 @@ $(document).ready(function(){initNavTree('index.html',''); initResizable(); });
// ...

If you would like to trigger a network write manually without using the SSLClient::available, you can also call SSLClient::flush, which will write all data and return when finished.

Session Caching

-

As detailed in the resources section, SSL handshakes take an extended period (1-4sec) to negotiate. To remedy this problem, BearSSL is able to keep a SSL session cache of the clients it has connected to. If BearSSL successfully resumes an SSL session, it can reduce connection time to 100-500ms.

+

As detailed in the resources section, SSL handshakes take an extended period (1-4sec) to negotiate. BearSSL is able to keep a SSL session cache of the clients it has connected to which can drastically reduce this time: if BearSSL successfully resumes an SSL session, connection time is typically 100-500ms.

In order to use SSL session resumption:

-

SSLClient automatically stores an IP address and hostname in each session, ensuring that if you call connect("www.google.com") SSLClient will use the SSL session with that hostname. However, because some websites have multiple servers on a single IP address (github.com being an example), you may find that even if you are connecting to the same host the connection does not resume. This is a flaw in the SSL session protocol — though it has been resolved in TLS 1.3, the lack of widespread adoption of the new protocol prevents it from being used here. SSL sessions can also expire based on server criteria, which will result in a standard 4-10 second connection.

-

You can test whether or not a website can resume SSL Sessions using the Session Example included with this library. Because of all the confounding factors of SSL Sessions, it is generally prudent while programming to assume the session will always fail to resume.

-

SSL sessions take a lot of memory to store, so by default SSLClient will only store one at a time. You can change this behavior by adding the following to your SSLClient declaration:

{C++}
+
+

NOTE: SSLClient automatically stores an IP address and hostname in each session, ensuring that if you call connect("www.google.com") SSLClient will use the same SSL session for that hostname. Unfortunately some websites have multiple servers on a single IP address (github.com being an example), so you may find that even if you are connecting to the same host the connection will not resume. This is a flaw in the SSL session protocol—though it has been resolved in TLS 1.3, the lack of widespread adoption of the new protocol prevents it from being resolved here.

+

SSL sessions can also expire based on server criteria (ex. timeout), which will result in a standard 4-10 second connection.

+
+

SSL sessions take memory to store, so by default SSLClient will only store one at a time. You can change this behavior by adding the following to your SSLClient declaration:

{C++}
EthernetClient baseClient;
SSLClient client(baseClient, TAs, (size_t)2, A7, SomeNumber);

Where SomeNumber is the number of sessions you would like to store. For example this declaration can store 3 sessions:

{C++}
@@ -184,7 +186,7 @@ $(document).ready(function(){initNavTree('index.html',''); initResizable(); });

Sessions are managed internally using the SSLSession::getSession function. This function will cycle through sessions in a rotating order, allowing the session cache to continually overwrite old sessions. In general, it is a good idea to use a SessionCache size equal to the number of domains you plan on connecting to.

If you need to clear a session, you can do so using the SSLSession::removeSession function.

mTLS

-

As of v1.6.0, SSLClient supports mutual TLS authentication. mTLS is a varient of TLS that verifys both the server and device identities before a connection, and is commonly used in IoT protocols as a secure layer (MQTT over TLS, HTTPS over TLS, etc.).

+

As of v1.6.0, SSLClient supports mutual TLS authentication. mTLS is a varient of TLS that verifies both the server and device identities before a connection, and is commonly used in IoT protocols as a secure layer (MQTT over TLS, HTTP over TLS, etc.).

To use mTLS with SSLClient you will need to a client certificate and client private key associated with the server you are attempting to connect to. Depending on your use case, you will either generate these yourself (ex. Mosquito MQTT setup), or have them generated for you (ex. AWS IoT Certificate Generation). Given this cryptographic information, you can modify the standard SSLClient connection sketch to enable mTLS authentication:

{C++}
...
/* Somewhere above setup() */
@@ -226,7 +228,7 @@ $(document).ready(function(){initNavTree('index.html',''); initResizable(); });

Implementation Gotchas

Some ideas that didn't quite fit in the API documentation.

SSLClient with Ethernet

-

If you are using the Arduino Ethernet library, you will need to modify the library to support the large buffer sizes required by SSL (detailed in resources). You can either modify the library yourself, or use this fork of the Ethernet library with the modification. To use the fork, simply install the library using the "add a .zip library" button in Arduino, and replace #include "Ethernet.h" with #include "EthernetLarge.h" in your sketch. Alternatively if for some reason this solution does not work, you can apply the modification using the instructions below.

+

If you are using the Arduino Ethernet library you will need to modify the library to support the large buffer sizes required by SSL (detailed in resources). You can either modify the library yourself, or use this fork of the Ethernet library with the modification. To use the fork: download a zipped copy of the fork through GiThub, use the "add a .zip library" button in Arduino to install the library, and replace #include "Ethernet.h" with #include "EthernetLarge.h" in your sketch. Alternatively if for some reason this solution does not work, you can apply the modification manually using the instructions below.

Manual Modification

First find the location of the library in the directory where Arduino is installed (C:\Program Files (x86)\Arduino on Windows). Inside of this directory, navigate to libraries\Ethernet\src (C:\Program Files (x86)\Arduino\libraries\Ethernet\src on Windows). Modify Ethernet.h to replace these lines:

{C++}
...
@@ -273,19 +275,52 @@ $(document).ready(function(){initNavTree('index.html',''); initResizable(); });

Time

The minimal x509 verification engine requires an accurate source of time to properly verify the creation and expiration dates of a certificate. As most embedded devices do not have a reliable source of time, by default SSLClient opts to use the compilation timestamp (__DATE__ and __TIME__) as the "current time" during the verification process. While this approach reduces the complexity of using SSLClient, it is inherently insecure, and can cause errors if certificates are redeployed (see #27): to accommodate these edge cases, SSLClient::setVerificationTime can be used to update the timestamp before connecting, resolving the above issues.

Resources

-

The SSL protocol recommends a device support many different encryption algorithms, as well as protocols for SSL itself. The complexity of both of those components results in many medium sized components forming an extremely large whole. Additionally, most embedded processors lack the sophisticated math hardware commonly found in a modern CPU, and as a result require more instructions to create the encryption algorithms SSL requires. This not only increases size but makes the algorithms slow and memory intensive.

-

To illustrate this, I will run some tests on various domains below. I haven't yet, but I will.

-

If flash footprint is becoming a problem, there are numerous debugging strings (~3kb estimated) that can be removed from SSLClient.h, SSLClientImpl.h, and SSLClientImpl.cpp. I have not figured out a way to configure compilation of these strings, so you will need to modify the library to remove them yourself.

+

The SSL/TLS protocol recommends a device support many different encryption and handshake algorithms. The complexity of these components results in many medium-footprint algorithms forming an extremely large whole. Compilation size of the EthernetHTTPS example in SSLClient v1.6.11 for various boards is shown below:

+ + + + + + + + + + + + + + + + + + + + + +
Board Size
Arduino Zero 
`RAM:   [===       ]  33.7% (used 11052 bytes from 32768 bytes)`

+`Flash: [===       ]  34.7% (used 90988 bytes from 262144 bytes)`
Arduino Due 
`RAM:   [=         ]  11.7% (used 11548 bytes from 98304 bytes)`

+`Flash: [==        ]  16.7% (used 87572 bytes from 524288 bytes)`
Adafruit Feather M0 
`RAM:   [====      ]  40.4% (used 13240 bytes from 32768 bytes)`

+`Flash: [====      ]  40.0% (used 104800 bytes from 262144 bytes)`
ESP32 (Lolin32) 
`RAM:   [=         ]   6.9% (used 22476 bytes from 327680 bytes)`

+`Flash: [==        ]  24.0% (used 314956 bytes from 1310720 bytes)`
Teensy 3.0 
`RAM:   [========  ]  78.2% (used 12812 bytes from 16384 bytes)`

+`Flash: [========  ]  79.8% (used 104532 bytes from 131072 bytes)`
Teensy 3.1 
`RAM:   [==        ]  19.9% (used 13020 bytes from 65536 bytes)`

+`Flash: [====      ]  40.6% (used 106332 bytes from 262144 bytes)`
Teensy 3.5 
`RAM:   [          ]   5.0% (used 12996 bytes from 262136 bytes)`

+`Flash: [==        ]  20.1% (used 105476 bytes from 524288 bytes)`
Teensy 3.6 
`RAM:   [          ]   5.0% (used 13060 bytes from 262144 bytes)`

+`Flash: [=         ]  10.2% (used 106828 bytes from 1048576 bytes)`
Teensy 4.0 
`RAM:   [===       ]  25.9% (used 135860 bytes from 524288 bytes)`

+`Flash: [=         ]   5.7% (used 115344 bytes from 2031616 bytes)`
+

In addition to the above, most embedded processors lack the sophisticated math hardware commonly found in a modern CPU, which results in slow and memory intensive execution of these algorithms. Because of this, it is recommended that SSLClient have 8kb of memory available on the stack during a connection, and 4-10 seconds should be allowed for the connection to complete. Note that this requirement is based on the SAMD21—more powerful processors (such as the ESP32) will see faster connection times.

+
+

NOTE: If flash footprint is becoming a problem, there are numerous debugging strings (~3kB estimated) that can be removed from SSLClient.h, SSLClientImpl.h, and SSLClientImpl.cpp. Unfortunately I have not figured out a way to configure compilation of these strings, so you will need to modify the library to remove them yourself.

+

Read Buffer Overflow

-

SSL is a buffered protocol, and since most microcontrollers have limited resources (see Resources), SSLClient is limited in the size of its buffers. A common problem I encountered with SSL connections is buffer overflow, caused by the server sending too much data at once. This problem is caused by the microcontroller being unable to copy and decrypt data faster than it is being received, forcing some data to be discarded. This usually puts BearSSL in an unrecoverable state, forcing SSLClient to close the connection with a write error. If you are experiencing frequent timeout problems, this could be the reason why.

-

In order to remedy this problem, the device must be able to read the data faster than it is being received, or alternatively have a cache large enough to store the entire payload. Since SSL's encryption forces the device to read slowly, this means we must increase the cache size. Depending on your platform, there are a number of ways this can be done:

    -
  • Sometimes your communication shield will have an internal buffer, which can be expanded through the driver code. This is the case with the Arduino Ethernet library (in the form of the MAX_SOCK_NUM and ETHERNET_LARGE_BUFFERS macros), however the library must be modified for the change to take effect.
    • -
  • SSLClient has an internal buffer SSLClient::m_iobuf, which can be expanded. BearSSL limits the amount of data that can be processed based on the stage in the SSL handshake, and so this will change will have limited usefulness.
    • -
  • In some cases, a website will send so much data that even with the above solutions, SSLClient will be unable to keep up (a website with a lot of HTML is an example). In these cases you will have to find another method of retrieving the data you need.
    • -
  • If none of the above are viable, it is possible to implement your own Client class which has an internal buffer much larger than both the driver and BearSSL. This would require in-depth knowledge of programming and the communication shield you are working with, as well as a microcontroller with a significant amount of RAM.
    • +

    SSL is a buffered protocol, and since most microcontrollers have limited resources (see Resources), SSLClient is limited in the size of its buffers. A common problem I encountered with SSL connections is buffer overflow caused by the server sending too much data at once. This problem is caused by the microcontroller being unable to copy and decrypt data faster than it is being received—forcing some data to be discarded. This usually puts BearSSL in an unrecoverable state, forcing SSLClient to close the connection with a write error. If you are experiencing frequent timeout problems this could be the reason why.

    +

    In order to remedy this problem, the device must be able to read the data faster than it is being received or have a cache large enough to store the entire payload. Since the device is typically already reading as fast as it can, we must increase the cache size in order to resolve this issue. Depending on your platform there are a number of ways this can be done:

      +
    • Sometimes your communication shield will have an internal buffer which can be expanded through the driver code: this is the case with the Arduino Ethernet library (in the form of the MAX_SOCK_NUM and ETHERNET_LARGE_BUFFERS macros show here), but mileage may vary with other drivers.
      • +
    • SSLClient has an internal buffer SSLClient::m_iobuf which can be expanded. Unfortunately, BearSSL limits the amount of data that can be put into the buffer based on the stage in the SSL handshake, and so increasing the buffer will have limited usefulness.
      • +
    • In some cases, a website will send so much data that even with the above solutions SSLClient will be unable to keep up. In these cases you will have to find another method of retrieving the data you need.
      • +
    • If none of the above are viable, it is possible to implement your own Client class which has an internal buffer much larger than both the driver and BearSSL. This implementation would require in-depth knowledge of communication shield you are working with and a microcontroller with a significant amount of RAM, but would be the most robust solution available.

    Cipher Support

    -

    By default, SSLClient supports only TLS1.2 and the ciphers listed in this file under suites[], and the list is relatively small to keep the connection secure and the flash footprint down. These ciphers should work for most applications, however if for some reason you would like to use an older version of TLS or a different cipher, you can change the BearSSL profile being used by SSLClient to an alternate one with support for older protocols. To do this, edit SSLClientImpl::SSLClientImpl to change these lines:

    {C++}
    +

    By default, SSLClient supports only TLS1.2 and the ciphers listed in this file under suites[], and the list is relatively small to keep the connection secure and the flash footprint down. These ciphers should work for most applications, however if for some reason you would like to use an older version of TLS or a different cipher you can change the BearSSL profile being used by SSLClient to an alternate one with support for older protocols. To do this, edit SSLClientImpl::SSLClientImpl to change these lines:

    {C++}
    br_client_init_TLS12_only(&m_sslctx, &m_x509ctx, m_trust_anchors, m_trust_anchors_num);
    // comment the above line and uncomment the line below if you're having trouble connecting over SSL
    // br_ssl_client_init_full(&m_sslctx, &m_x509ctx, m_trust_anchors, m_trust_anchors_num);
    @@ -295,15 +330,14 @@ $(document).ready(function(){initNavTree('index.html',''); initResizable(); });
    br_ssl_client_init_full(&m_sslctx, &m_x509ctx, m_trust_anchors, m_trust_anchors_num);

    If for some unfortunate reason you need SSL 3.0 or SSL 2.0, you will need to modify the BearSSL profile to enable support. Check out the BearSSL profiles documentation and I wish you the best of luck.

    Security

    -

    Unlike BearSSL, SSLClient is not rigorously vetted to be secure. If your project has security requirements, I recommend you utilize BearSSL directly.

    +

    Unlike BearSSL, SSLClient is not rigorously vetted to be secure. If your project has security requirements I recommend you utilize BearSSL directly.

    Known Issues

    • In some drivers (Ethernet), calls to Client::flush will hang if internet is available but there is no route to the destination. Unfortunately SSLClient cannot correct for this without modifying the driver itself, and as a result the recommended solution is ensuring you choose a driver with built-in timeouts to prevent freezing. More information here.
      • -
    • When using PubSubClient on the ESP32, a stack overflow will occur if the user does not flush the buffer immediately after writing. The cause of this issue is under active investigation. More information in issue https://github.com/OPEnSLab-OSU/SSLClient/issues/9.
      • -
    • Previous to SSLClient v1.6.7, calls to SSLClient::stop would sometimes hang the device. More information in issue https://github.com/OPEnSLab-OSU/SSLClient/issues/13.
      • -
    • Previous to SSLClient v1.6.6, calls to SSLClient::connect would fail if the driver indicated that a socket was already opened (Client::connected returned true). This behavior created unintentional permanent failures when Client::stop would fail to close the socket, and as a result was downgraded to a warning in v1.6.6.
      • -
    • Previous to SSLClient v1.6.3, calling SSLClient::write with more than 2Kb of total data before flushing the write buffer would cause a buffer overflow.
      • -
    • Previous to SSLClient v1.6.11, SSLClient::write would sometimes call br_ssl_engine_sendapp_ack with zero bytes, which resulted in a variety of issues including (but not limited to) and infinite recursion loop on the esp32 ( #9, #30).
      • +
    • Previous to SSLClient v1.6.7, calls to SSLClient::stop would sometimes hang the device. More information in issue https://github.com/OPEnSLab-OSU/SSLClient/issues/13.
      • +
    • Previous to SSLClient v1.6.6, calls to SSLClient::connect would fail if the driver indicated that a socket was already opened (Client::connected returned true). This behavior created unintentional permanent failures when Client::stop would fail to close the socket, and as a result was downgraded to a warning in v1.6.6.
      • +
    • Previous to SSLClient v1.6.3, calling SSLClient::write with more than 2kB of total data before flushing the write buffer would cause a buffer overflow.
      • +
    • Previous to SSLClient v1.6.11, SSLClient::write would sometimes call br_ssl_engine_sendapp_ack with zero bytes, which resulted in a variety of issues including (but not limited to) and infinite recursion loop on the esp32 (#9, #30).