# Asynchronous DNS

## Introduction

Lws now features optional asynchronous, ie, nonblocking recursive DNS
resolution done on the event loop, enable `-DLWS_WITH_SYS_ASYNC_DNS=1`
at cmake to build it in.

## Description

The default libc name resolution is via libc `getaddrinfo()`, which is
blocking, possibly for quite long periods (seconds).  If you are
taking care about latency, but want to create outgoing connections,
you can't tolerate this exception from the rule that everything in
lws is nonblocking.

Lws' asynchronous DNS resolver creates a caching name resolver
that directly queries the configured nameserver itself over UDP,
from the event loop.

It supports both ipv4 / A records and ipv6 / AAAA records (see later
for a description about how).  One server supported over UDP :53,
and the nameserver is autodicovered on linux, windows, and freertos.
    
Other features

 - lws-style paranoid response parsing
 - random unique tid generation to increase difficulty of poisoning
 - it's really integrated with the lws event loop, it does not spawn
   threads or use the libc resolver, and of course no blocking at all
 - platform-specific server address capturing (from /etc/resolv.conf
   on linux, windows apis on windows)
 - LRU caching
 - piggybacking (multiple requests before the first completes go on
    a list on the first request, not spawn multiple requests)
 - observes TTL in cache
 - TTL and timeout use `lws_sul` timers on the event loop
 - Uses CNAME resolution inside the same response if present, otherwise
   recurses to resolve the CNAME (up to 3 deep)
 - ipv6 pieces only built if cmake `LWS_IPV6` enabled

## Api

If enabled at cmake, the async DNS implementation is used automatically
for lws client connections.  It's also possible to call it directly, see
the api-test-async-dns example for how.

The Api follows that of `getaddrinfo()` but results are not created on
the heap.  Instead a single, const cached copy of the addrinfo struct
chain is reference-counted, with `lws_async_dns_freeaddrinfo()` provided
to deduct from the reference count.  Cached items with a nonzero
reference count can't be destroyed from the cache, so it's safe to keep
a pointer to the results and iterate through them.

## Dealing with IPv4 and IPv6

DNS is a very old standard that has some quirks... one of them is that
multiple queries are not supported in one packet, even though the protocol
suggests it is.  This creates problems on ipv6 enabled systems, where
it may prefer to have AAAA results, but the server may only have A records.

To square the circle, for ipv4 only systems (`LWS_IPV6=0`) the resolver
requests only A records.  For ipv6-capable systems, it always requests
first A and then immediately afterwards AAAA records.

To simplify the implementation, the tid b0 is used to differentiate
between A (b0 = 0) and AAAA (b0 = 1) requests and responses using the
same query body.

The first response to come back is parsed, and a cache entry made...
it leaves a note in the query about the address of the last `struct addrinfo`
record.  When the second response comes, a second allocation is made,
but not added to the logical cache... instead it's chained on to the
first cache entry and the `struct addrinfo` linked-list from the
first cache entry is extended into the second one.  At the time the
second result arrives, the query is destroyed and the cached results
provided on the result callback.

## Recursion

Where CNAMEs are returned, DNS servers may take two approaches... if the
CNAME is also resolved by the same server and so it knows what it should
resolve to, it may provide the CNAME resolution in the same response
packet.

In the case the CNAME is actually resolved by a different name server,
the server with the CNAME does not have the information to hand to also
resolve the CNAME in the same response.  So it just leaves it for the
client to sort out.

The lws implementation can deal with both of these, first it "recurses"
(it does not recurse on the process stack but uses its own manual stack)
to look for results in the same packet that told it about the CNAME.  If
there are no results, it resets the query to look instead for the CNAME,
and restarts it.  It allows this to happen for 3 CNAME deep.

At the end, either way, the cached result is set using the original
query name and the results from the last CNAME in the chain.

# Building for Android NDK

If you have the ndk and prebuilt toolchains with that, you can simply build
lws library for your android app from one cmake and one make command.

However if you want a tls lib, you have to take care of building and pointing
to that first.  But if it's a cmake project like mbedtls, that also is just a
matter of one cmake and one make.

## Installing NDK pieces

There's probably a more direct way but the official way is install the whole
Android Studio and then run `sdkmanager` to install a recent NDK.

I installed the sdk and ndk pieces into /opt/android/ and that's how the
`./contrib/cross-aarch64-android.cmake` toolchain file is shipped.  You can
adapt some settings at the top of that file including the path if needed.

## Fetching lws (needed first for cross toolchain file)

It doesn't care where you put these projects, but for simplicity they should
be in the same parent dir, like

```
 - /home/someone
  - /home/someone/libwebsockets
  - /home/someone/mbedtls
```

The reason is that building mbedtls need the cross toolchain file from
libwebsockets, that's also why we have to get libwebsockets first now but
build it later.

```
$ git clone https://libwebsockets.org/repo/libwebsockets
```

## Building mbedtls

```
$ git clone https://github.com/ARMmbed/mbedtls.git
$ cd mbedtls
$ mkdir build
$ cd build
$ rm -f CMakeCache.txt && \
  cmake .. -DCMAKE_TOOLCHAIN_FILE=../libwebsockets/contrib/cross-aarch64-android.cmake \
  -DUSE_SHARED_MBEDTLS_LIBRARY=1 \
  -DENABLE_PROGRAMS=0 \
  -Wno-dev && \
  make -j && \
  cmake --install .
```

The lws toolchain file sets the path to install into as the cross root path, so
despite it looks like the destination dir is missing for the install, it will
go into, eg `/opt/android/ndk/21.1.6352462/platforms/android-24/arch-arm64/lib/libmbedcrypto.a`
where lws will look for it

## Building lws

You don't need to explain where mbedtls can be found... lws will build with the
same toolchain file that sets the cross root to the same place as mbedtls, it
will easily find them there without any further hints.

```
$ mkdir build
$ cd build
$ rm -f CMakeCache.txt && \
  cmake .. -DCMAKE_TOOLCHAIN_FILE=../libwebsockets/contrib/cross-aarch64-android.cmake \
  -DLWS_WITH_MBEDTLS=1 \
  -DLWS_WITHOUT_TESTAPPS=1 && \
  make && \
  cmake --install .
```

That's it, both mbedtls and lws library and header files are installed into the
ndk cross root.

# Some notes for the windows jungle

This was how I compiled libwebsockets starting from a blank windows install
in March - April 2020.  Doing this on a linux distro is way simpler and quicker
than all this!

## Notes on vm installation

### Disk size

For building you'll need 40GB+ available for the guest storage.

### Required: Windows product key

Assuming like me the first thing you do with a new laptop is install Linux over
the windows it came with, you can recover your 'windows tax' windows product key
from your device typically using `sudo strings /sys/firmware/acpi/tables/MSDM`,
and use that for your VM install.

### Required: Spice guest 

To have shared clipboard, and for windows video driver to match your vm window
resolution, you must install spice guest tools inside the windows VM.  It also
installs some virtio pieces you will want.

https://www.spice-space.org/download/windows/spice-guest-tools/spice-guest-tools-latest.exe

### Blood-pressure reduction: Firefox

https://www.mozilla.org/en-US/exp/firefox/

When it's up, add-ons: ublock origin, privacy badger, noscript, disable search
bar prediction

### Blood-pressure reduction: Clink

This is a hack on cmd.exe that lets it understand Ctrl-R and fixup unix-style
slashes automagically.

https://github.com/mridgers/clink/releases/download/0.4.9/clink_0.4.9_setup.exe

If you're usually using *nix, you definitely need this to keep your sanity.

### Required: cmake

CMake have a windows installer thing downloadable from here

[cmake](https://cmake.org/download/)

after that you can use `cmake` from the terminal OK.

### Required: git

Visit the canonical git site to download their windows installer thing

[git](https://git-scm.com/download/win)

**Select the install option for "extra unix commands"** so you can get `ls -l`,
`cp`, `mv` and suchlike working in cmd.exe... that's awesome, thanks git!

Afterwards you can just use `git` as normal from cmd.exe as well.

### Required: Install the "free" "community" visual studio

You can do this through "windows store" by searching for "visual studio"

I installed as little as possible, we just want the C "C++" tools... 7GB :-)

It still wouldn't link without the "mt" helper tool from the
huge windows SDK, so you have to install GB of that as well.

They don't mention it during the install, but after 30 days this "free"
"community" edition demands you open a microsoft account or it stops working.
In the install they give you the option to add a microsoft account and the
alternative is, "not now, maybe later".  Compare and contrast to gcc or git or
the other FOSS projects.

### Required: OpenSSL

Ugh... I tried using prebuilts but it's unreliable and needs an unfeasible
amount of trust.  So I recommend bite the bullet and build your own... that's
trivial on Linux but of course windows makes everything nasty.

At least hopefully all the "research" is done and listed out here.

#### OpenSSL build Prerequisite: install perl binary

Move the git version of perl out of the way, it won't work for OpenSSL build

```
mv /usr/bin/perl /usr/bin/perl-git
```

For windows, OpenSSL "recommends" ActiveState perl but it doesn't work for me,
complaining about stuff needed from cpan and then dying when it was installed.
"Strawberry Perl" is installed in `C:\Strawberry` and worked out the box.

http://strawberryperl.com/download/5.30.2.1/strawberry-perl-5.30.2.1-64bit.msi

The installer sets up `%PATH%` if you open a new cmd window.  

#### OpenSSL build Prerequisite: NASM

Go here and click on the latest stable, download the win32 .exe

https://nasm.us/

Just install via the defaults.  Then add it to the PATH temporarily...

```
$ set PATH=%PATH%;C:\Program Files (x86)\NASM
```

#### OpenSSL build setup: source VC env vars

These fix up the PATH and include dirs etc necessary for VC build in the cmd
window.

```
$ call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" x86_amd64
```

### OpenSSL build:

Grab openssl from git... assuming the prerequisites above went well it will
just sit there building for 30 minutes or whatever.

```
$ git clone https://github.com/openssl/openssl
$ cd openssl
$ perl Configure VC-WIN64A
$ nmake
```

Afterwards, open an Administrator mode cmd.exe, redo the msvc path and then
install the build.

```
$ cd openssl
$ call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" x86_amd64
$ nmake install
```

Oh another grindingly slow windows build action.  Finally it's in there in
`C:\Program Files\OpenSSL`.

libraries are looking for a cert bundle at "C:\Program Files\Common Files\SSL\cert.pem"...
it's not documented or included in the zip file from the above, so...

#### Installing a cert bundle

You can get a trusted cert bundle from here

[drwetter/testssl cert bundle](https://raw.githubusercontent.com/drwetter/testssl.sh/3.1dev/etc/Microsoft.pem)

Save it into `C:\Program Files\Common Files\SSL\cert.pem` where openssl will be able to see it.

## Required: pthreads

It's amazing but after all these years windows doesn't offer pthreads compatibility
itself.  Just like the many other missing POSIX bits like fork().

I downloaded the latest (2012) zip release of pthreads-win32 from here

ftp://sourceware.org/pub/pthreads-win32

Then I created a dir "C:\Program Files (x86)\pthreads", and copied the `dll`,
`include` and `lib` subdirs from the `prebuilt` folder in the zip there.

The cmake incantation to build against pthreads set up like that is

```
 $ cmake .. -DLWS_HAVE_PTHREAD_H=1 -DLWS_EXT_PTHREAD_INCLUDE_DIR="C:\Program Files (x86)\pthreads\include" -DLWS_EXT_PTHREAD_LIBRARIES="C:\Program Files (x86)\pthreads\lib\x64\libpthreadGC2.a" -DLWS_WITH_MINIMAL_EXAMPLES=1
```

## Building libwebsockets

We'll clone libwebsockets then use cmake to build via vs tools

```
> git clone https://libwebsockets.org/repo/libwebsockets
> cd libwebsockets
> mkdir build
> cd build
> cmake ..
> cmake --build . --config DEBUG
```

Installing requires admin privs, I opened a second cmd window as admin and did it
there.

```
> cmake --install . --config DEBUG
```

### Hack the libs into view

The libs we built against aren't visible in the system, I don't know what
Real Windows Programmers are supposed to do about that, but I used an Admin cmd
prompt to copy them into C:\windows\system32

```
$ cp "C:\Program Files (x86)\pthreads\dll\x64\pthreadGC2.dll" "C:\Program Files\OpenSSL\bin\libcrypto-3.dll"  "C:\Program Files\OpenSSL\bin\libssl-3.dll" C:\Windows\system32
```

After that you can run the test apps OK, eg

```
$ libwebsockets-test-server.exe -s
```

## Note about using paths with spaces in with cmake

Notes about building lws
========================

You can download and install lws using the [vcpkg](https://github.com/Microsoft/vcpkg/) dependency manager:
```
git clone https://github.com/microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg integrate install
vcpkg install libwebsockets
```
The lws port in vcpkg is kept up to date by Microsoft team members and community contributors. If the version is out of date, please [create an issue or pull request](https://github.com/Microsoft/vcpkg/) on the vcpkg repository.

@section cm Introduction to CMake

CMake is a multi-platform build tool that can generate build files for many
different target platforms. See more info at http://www.cmake.org

CMake also allows/recommends you to do "out of source"-builds, that is,
the build files are separated from your sources, so there is no need to
create elaborate clean scripts to get a clean source tree, instead you
simply remove your build directory.

Libwebsockets has been tested to build successfully on the following platforms
with SSL support (for OpenSSL/wolfSSL/BoringSSL):

- Windows (Visual Studio)
- Windows (MinGW)
- Linux (x86 and ARM)
- OSX
- NetBSD


@section build1 Building the library and test apps

The project settings used by CMake to generate the platform specific build
files is called [CMakeLists.txt](../CMakeLists.txt). CMake then uses one of its "Generators" to
output a Visual Studio project or Make file for instance. To see a list of
the available generators for your platform, simply run the "cmake" command.

Note that by default OpenSSL will be linked, if you don't want SSL support
see below on how to toggle compile options.


@section bu Building on Unix:

1. Install CMake 2.8 or greater: http://cmake.org/cmake/resources/software.html
   (Most Unix distributions comes with a packaged version also)

2. Install OpenSSL.

3. Generate the build files (default is Make files):
```
        $ cd /path/to/src
        $ mkdir build
        $ cd build
        $ cmake ..
```

4. Finally you can build using the generated Makefile:
```
    $ make && sudo make install
```
**NOTE**: The `build/`` directory can have any name and be located anywhere
 on your filesystem, and that the argument `..` given to cmake is simply
 the source directory of **libwebsockets** containing the [CMakeLists.txt](../CMakeLists.txt)
 project file. All examples in this file assumes you use ".."

**NOTE2**:
A common option you may want to give is to set the install path, same
as --prefix= with autotools.  It defaults to /usr/local.
You can do this by, eg
```
    $ cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr .
```

**NOTE3**:
On machines that want libraries in lib64, you can also add the
following to the cmake line
```
    -DLIB_SUFFIX=64
```

**NOTE4**:
If you are building against a non-distro OpenSSL (eg, in order to get
access to ALPN support only in newer OpenSSL versions) the nice way to
express that in one cmake command is eg,
```
    $ cmake .. -DOPENSSL_ROOT_DIR=/usr/local/ssl \
         -DCMAKE_INCLUDE_DIRECTORIES_PROJECT_BEFORE=/usr/local/ssl \
         -DLWS_WITH_HTTP2=1
```

When you run the test apps using non-distro SSL, you have to force them
to use your libs, not the distro ones
```
    $ LD_LIBRARY_PATH=/usr/local/ssl/lib libwebsockets-test-server --ssl
```

To get it to build on latest openssl (2016-04-10) it needed this approach
```
    cmake .. -DLWS_WITH_HTTP2=1 -DLWS_OPENSSL_INCLUDE_DIRS=/usr/local/include/openssl -DLWS_OPENSSL_LIBRARIES="/usr/local/lib64/libssl.so;/usr/local/lib64/libcrypto.so"
```

Mac users have reported

```
 $ export OPENSSL_ROOT_DIR=/usr/local/Cellar/openssl/1.0.2k; cmake ..; make -j4
```

worked for them when using "homebrew" OpenSSL

**NOTE5**:
To build with debug info and _DEBUG for lower priority debug messages
compiled in, use
```
    $ cmake .. -DCMAKE_BUILD_TYPE=DEBUG
```

**NOTE6**
To build on Solaris the linker needs to be informed to use lib socket
and libnsl, and only builds in 64bit mode.

```bash
    $ cmake .. -DCMAKE_C_FLAGS=-m64 -DCMAKE_EXE_LINKER_FLAGS="-lsocket -lnsl"
```

**NOTE7**

Build and test flow against boringssl.  Notice `LWS_WITH_GENHASH` is currently
unavailable with boringssl due to their removing the necessary apis.

Build current HEAD boringssl

```
 $ cd /projects
 $ git clone https://boringssl.googlesource.com/boringssl
 $ cd boringssl
 $ mkdir build
 $ cd build
 $ cmake ..  -DBUILD_SHARED_LIBS=1
 $ make -j8
```

Build and test lws against it

```
 $ cd /projects/libwebsockets/build
 $ cmake .. -DOPENSSL_LIBRARIES="/projects/boringssl/build/ssl/libssl.so;\
   /projects/boringssl/build/crypto/libcrypto.so" \
   -DOPENSSL_INCLUDE_DIRS=/projects/boringssl/include \
   -DLWS_WITH_BORINGSSL=1 -DCMAKE_BUILD_TYPE=DEBUG
 $ make -j8 && sudo make install
 $ LD_PRELOAD="/projects/boringssl/build/ssl/libssl.so \
   /projects/boringssl/build/crypto/libcrypto.so" \
   /usr/local/bin/libwebsockets-test-server -s
```

4. Finally you can build using the generated Makefile:

```bash
    $ make
 ```

@section lcap Linux Capabilities

On Linux, lws now lets you retain selected root capabilities when dropping
privileges.  If libcap-dev or similar package is installed providing
sys/capabilities.h, and libcap or similar package is installed providing
libcap.so, CMake will enable the capability features.

The context creation info struct .caps[] and .count_caps members can then
be set by user code to enable selected root capabilities to survive the
transition to running under an unprivileged user.

@section cmq Quirk of cmake

When changing cmake options, for some reason the only way to get it to see the
changes sometimes is delete the contents of your build directory and do the
cmake from scratch.

deleting build/CMakeCache.txt may be enough.


@section cmw Building on Windows (Visual Studio)

1. Install CMake 2.6 or greater: http://cmake.org/cmake/resources/software.html

2. Install OpenSSL binaries. https://wiki.openssl.org/index.php/Binaries

   (**NOTE**: Preferably in the default location to make it easier for CMake to find them)

   **NOTE2**: 
   Be sure that OPENSSL_CONF environment variable is defined and points at 
   <OpenSSL install location>\bin\openssl.cfg

3. Generate the Visual studio project by opening the Visual Studio cmd prompt:

```
    cd <path to src>
    md build
    cd build
    cmake -G "Visual Studio 10" ..
```

   (**NOTE**: There is also a cmake-gui available on Windows if you prefer that)
   
   **NOTE2**:
   See this link to find out the version number corresponding to your Visual Studio edition:
   http://superuser.com/a/194065

4. Now you should have a generated Visual Studio Solution in  your
   `<path to src>/build` directory, which can be used to build.

5. Some additional deps may be needed

 - iphlpapi.lib
 - psapi.lib
 - userenv.lib

6. If you're using libuv, you must make sure to compile libuv with the same multithread-dll / Mtd attributes as libwebsockets itself


@section cmwmgw Building on Windows (MinGW)

1. Install MinGW

    For Fedora, it's, eg, `dnf install mingw64-gcc`

2. Install current CMake package

    For Fedora, it's `dnf install cmake`

3. Instal mingw-built OpenSSL pieces

    For Fedora, it's `mingw64-openssl.noarch mingw64-openssl-static.noarch`

    mingw64-cmake as described below will auto-find the libs and includes
    for build.  But to execute the apps, they either need to go into the same
    `/usr/x86_64-w64-mingw32/sys-root/mingw/bin/` as the dlls are installed to,
    or the dlls have to be copied into the same dir as your app executable.

4. Generate the build files (default is Make files) using MSYS shell.

   For Fedora, they provide a `mingw64-cmake` wrapper in the package
   `mingw64-filesystem`, with this you can run that instead of cmake directly
   and don't have to get involved with setting the cmake generator.

   Otherwise doing it by hand is like this:

```
    $ cd /drive/path/to/src
    $ mkdir build
    $ cd build
    $ cmake -G "MSYS Makefiles" -DCMAKE_INSTALL_PREFIX=C:/MinGW ..
```

   To generate build files allowing to create libwebsockets binaries with debug information
   set the CMAKE_BUILD_TYPE flag to DEBUG:
```
    $ cmake -G "MSYS Makefiles" -DCMAKE_INSTALL_PREFIX=C:/MinGW -DCMAKE_BUILD_TYPE=DEBUG ..
```
5. Finally you can build using the generated Makefile and get the results deployed into your MinGW installation:

```
    $ make && make install
```

@section distro Selecting CMake options useful for distros

Distro packagers should select the CMake option "LWS_WITH_DISTRO_RECOMMENDED",
which selects common additional options like support for various event libraries,
plugins and lwsws.

@section ssllib Choosing Your TLS Poison

 - If you are really restricted on memory, code size, or don't care about TLS
   speed, mbedTLS is a good choice: `cmake .. -DLWS_WITH_MBEDTLS=1`
 
 - If cpu and memory is not super restricted and you care about TLS speed,
   OpenSSL or a directly compatible variant like Boring SSL is a good choice.
 
Just building lws against stock Fedora OpenSSL or stock Fedora mbedTLS, for
SSL handhake mbedTLS takes ~36ms and OpenSSL takes ~1ms on the same x86_64
build machine here, with everything else the same.  Over the 144 connections of
h2spec compliance testing for example, this ends up completing in 400ms for
OpenSSL and 5.5sec for mbedTLS on x86_64.  In other words mbedTLS is very slow
compared to OpenSSL under the (fairly typical) conditions I tested it.

This isn't an inefficiency in the mbedtls interface implementation, it's just
mbedTLS doing the crypto much slower than OpenSSL, which has accelerated
versions of common crypto operations it automatically uses for platforms
supporting it.  As of Oct 2017 mbedTLS itself has no such optimizations for any
platform that I could find.  It's just pure C running on the CPU.

Lws supports both almost the same, so instead of taking my word for it you are
invited to try it both ways and see which the results (including, eg, binary
size and memory usage as well as speed) suggest you use.

NOTE: one major difference with mbedTLS is it does not load the system trust
store by default.  That has advantages and disadvantages, but the disadvantage
is you must provide the CA cert to lws built against mbedTLS for it to be able
to validate it, ie, use -A with the test client.  The minimal test clients
have the CA cert for warmcat.com and libwebsockets.org and use it if they see
they were built with mbedTLS.

@section optee Building for OP-TEE

OP-TEE is a "Secure World" Trusted Execution Environment.

Although lws is only part of the necessary picture to have an https-enabled
TA, it does support OP-TEE as a platform and if you provide the other
pieces, does work very well.

Select it in cmake with `-DLWS_PLAT_OPTEE=1`


@section cmco Setting compile options

To set compile time flags you can either use one of the CMake gui applications
or do it via the command line.

@subsection cmcocl Command line

To list available options (omit the H if you don't want the help text):

    cmake -LH ..

Then to set an option and build (for example turn off SSL support):

    cmake -DLWS_WITH_SSL=0 ..
or
    cmake -DLWS_WITH_SSL:BOOL=OFF ..

@subsection cmcoug Unix GUI

If you have a curses-enabled build you simply type:
(not all packages include this, my debian install does not for example).

    ccmake

@subsection cmcowg Windows GUI

On windows CMake comes with a gui application:
    Start -> Programs -> CMake -> CMake (cmake-gui)


@section wolf wolfSSL/CyaSSL replacement for OpenSSL

wolfSSL/CyaSSL is a lightweight SSL library targeted at embedded systems:
https://www.wolfssl.com/wolfSSL/Products-wolfssl.html

It contains a OpenSSL compatibility layer which makes it possible to pretty
much link to it instead of OpenSSL, giving a much smaller footprint.

**NOTE**: wolfssl needs to be compiled using the `--enable-opensslextra` flag for
this to work.

@section wolf1 Compiling libwebsockets with wolfSSL

```
    cmake .. -DLWS_WITH_WOLFSSL=1 \
         -DLWS_WOLFSSL_INCLUDE_DIRS=/path/to/wolfssl \
         -DLWS_WOLFSSL_LIBRARIES=/path/to/wolfssl/wolfssl.a ..
```

**NOTE**: On windows use the .lib file extension for `LWS_WOLFSSL_LIBRARIES` instead.

@section cya Compiling libwebsockets with CyaSSL

```
    cmake .. -DLWS_WITH_CYASSL=1 \
         -DLWS_CYASSL_INCLUDE_DIRS=/path/to/cyassl \
         -DLWS_CYASSL_LIBRARIES=/path/to/wolfssl/cyassl.a ..
```

**NOTE**: On windows use the .lib file extension for `LWS_CYASSL_LIBRARIES` instead.

@section gzip Selecting GZIP or MINIZ

By default lws supports gzip when compression is needed.  But you can tell it to use
MINIZ instead by using `-DLWS_WITH_MINIZ=1`.

For native build cmake will try to find an existing libminiz.so or .a and build
against that and the found includes automatically.

For cross-build or building against local miniz, you need the following kind of
cmake to tell it where to get miniz

```
cmake .. -DLWS_WITH_MINIZ=1 -DLWS_WITH_ZIP_FOPS=1 -DMINIZ_INCLUDE_DIRS="/projects/miniz;/projects/miniz/build" -DMINIZ_LIBRARIES=/projects/miniz/build/libminiz.so.2.1.0  
```

@section esp32 Building for ESP32

Building for ESP32 requires the ESP-IDF framework. It can be built under Linux, OSX or Windows (MSYS2).

1. Install ESP-IDF, follow the getting started guide here - http://esp-idf.readthedocs.io/en/latest/get-started/
2. Set ESP-IDF to last known working version (assuming ESP-IDF is in `~/esp/esp-idf`) :
```
    cd ~/esp/esp-idf
    git checkout 0c50b65a34cd6b3954f7435193411a88adb49cb0
    git submodule update --recursive
```
3. Add `libwebsockets` as a submodule in the `components` folder of your ESP-IDF project:
```
    git submodule add https://github.com/warmcat/libwebsockets.git components/libwebsockets
```
4. If on Windows (MSYS2) you will need to install CMake in the MSYS2 environment:
```
    pacman -S mingw-w64-i686-cmake
```
If you're on Linux or OSX ensure CMake version is at least 3.7.

@section extplugins Building plugins outside of lws itself

The directory ./plugin-standalone/ shows how easy it is to create plugins
outside of lws itself.  First build lws itself with -DLWS_WITH_PLUGINS,
then use the same flow to build the standalone plugin
```
    cd ./plugin-standalone
    mkdir build
    cd build
    cmake ..
    make && sudo make install
```

if you changed the default plugin directory when you built lws, you must
also give the same arguments to cmake here (eg,
` -DCMAKE_INSTALL_PREFIX:PATH=/usr/something/else...` )

Otherwise if you run lwsws or libwebsockets-test-server-v2.0, it will now
find the additional plugin "libprotocol_example_standalone.so"
```
    lwsts[21257]:   Plugins:
    lwsts[21257]:    libprotocol_dumb_increment.so
    lwsts[21257]:    libprotocol_example_standalone.so
    lwsts[21257]:    libprotocol_lws_mirror.so
    lwsts[21257]:    libprotocol_lws_server_status.so
    lwsts[21257]:    libprotocol_lws_status.so
```
If you have multiple vhosts, you must enable plugins at the vhost
additionally, discovered plugins are not enabled automatically for security
reasons.  You do this using info->pvo or for lwsws, in the JSON config.


@section http2rp Reproducing HTTP/2 tests

Enable `-DLWS_WITH_HTTP2=1` in cmake to build with http/2 support enabled.

You must have built and be running lws against a version of openssl that has
ALPN.  At the time of writing, recent distros have started upgrading to OpenSSL
1.1+ that supports this already.  You'll know it's right by seeing

```
    lwsts[4752]:  Compiled with OpenSSL support
    lwsts[4752]:  Using SSL mode
    lwsts[4752]:  HTTP2 / ALPN enabled
```
at lws startup.

Recent Firefox and Chrome also support HTTP/2 by ALPN, so these should just work
with the test server running in -s / ssl mode.

For testing with nghttp client:

```
    $ nghttp -nvas https://localhost:7681/test.html
```

Testing with h2spec (https://github.com/summerwind/h2spec)

```
        $ h2spec  -h 127.0.0.1 -p 7681 -t -k -v -o 1
```

```
145 tests, 145 passed, 0 skipped, 0 failed

```

@section coverage Automated Coverage Testing

./test-apps/attack.sh contains scripted tests that are the basis
of the automated test coverage assessment available for gcc and clang.

To reproduce

 $ cd build
 $ cmake .. -DLWS_WITH_GCOV=1 -DCMAKE_BUILD_TYPE=DEBUG
 $ ../scripts/build-gcov.sh
 $ ../test-apps/attack.sh
 $ ../scripts/gcov.sh
...
Lines executed:51.24% of 8279

@section windowsprebuilt Using Windows binary builds on Appveyor

The CI builds on Appveyor now produce usable binary outputs.  Visit

[lws on Appveyor](https://ci.appveyor.com/project/lws-team/libwebsockets)

and select one of the builds, then click on ARTIFACTS at the top right.  The zip file
want to be unpacked into `C:\Program Files (x86)/libwebsockets`, after that, you should be able to run the test server, by running it from `bin/Release/libwebsockets-test-server.exe` and opening a browser on http://127.0.0.1:7681

@section cross Cross compiling

To enable cross-compiling **libwebsockets** using CMake you need to create
a "Toolchain file" that you supply to CMake when generating your build files.
CMake will then use the cross compilers and build paths specified in this file
to look for dependencies and such.

**Libwebsockets** includes an example toolchain file [cross-arm-linux-gnueabihf.cmake](../contrib/cross-arm-linux-gnueabihf.cmake)
you can use as a starting point.

The commandline to configure for cross with this would look like
```
    $ cmake .. -DCMAKE_INSTALL_PREFIX:PATH=/usr/lib/my-cross-root \
         -DCMAKE_TOOLCHAIN_FILE=../contrib/cross-arm-linux-gnueabihf.cmake \
         -DLWS_WITHOUT_EXTENSIONS=1 -DLWS_WITH_SSL=0 \
         -DLWS_WITH_ZIP_FOPS=0 -DLWS_WITH_ZLIB=0
```
The example shows how to build with no external cross lib dependencies, you
need to provide the cross libraries otherwise.

**NOTE**: start from an EMPTY build directory if you had a non-cross build in there
    before the settings will be cached and your changes ignored.
    Delete `build/CMakeCache.txt` at least before trying a new cmake config
    to ensure you are really building the options you think you are.

Additional information on cross compilation with CMake:
    http://www.vtk.org/Wiki/CMake_Cross_Compiling

@section cross_example Complex Cross compiling example

Here are step by step instructions for cross-building the external projects needed for lws with lwsws + mbedtls as an example.

In the example, my toolchain lives in `/projects/aist-tb/arm-tc` and is named `arm-linux-gnueabihf`.  So you will need to adapt those to where your toolchain lives and its name where you see them here.

Likewise I do all this in /tmp but it has no special meaning, you can adapt that to somewhere else.

All "foreign" cross-built binaries are sent into `/tmp/cross` so they cannot be confused for 'native' x86_64 stuff on your host machine in /usr/[local/]....

## Prepare the cmake toolchain file

1) `cd /tmp`

2) `wget -O mytoolchainfile https://raw.githubusercontent.com/warmcat/libwebsockets/main/contrib/cross-arm-linux-gnueabihf.cmake` 

3) Edit `/tmp/mytoolchainfile` adapting `CROSS_PATH`, `CMAKE_C_COMPILER` and `CMAKE_CXX_COMPILER` to reflect your toolchain install dir and path to your toolchain C and C++ compilers respectively.  For my case:

```
set(CROSS_PATH /projects/aist-tb/arm-tc/)
set(CMAKE_C_COMPILER "${CROSS_PATH}/bin/arm-linux-gnueabihf-gcc")
set(CMAKE_CXX_COMPILER "${CROSS_PATH}/bin/arm-linux-gnueabihf-g++")
```

## 1/4: Building libuv cross:

1) `export PATH=/projects/aist-tb/arm-tc/bin:$PATH`  Notice there is a **/bin** on the end of the toolchain path

2) `cd /tmp ; mkdir cross` we will put the cross-built libs in /tmp/cross

3) `git clone https://github.com/libuv/libuv.git` get libuv

4) `cd libuv`

5) `./autogen.sh`

```
+ libtoolize --copy
libtoolize: putting auxiliary files in '.'.
libtoolize: copying file './ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'.
libtoolize: copying file 'm4/libtool.m4'
libtoolize: copying file 'm4/ltoptions.m4'
libtoolize: copying file 'm4/ltsugar.m4'
libtoolize: copying file 'm4/ltversion.m4'
libtoolize: copying file 'm4/lt~obsolete.m4'
+ aclocal -I m4
+ autoconf
+ automake --add-missing --copy
configure.ac:38: installing './ar-lib'
configure.ac:25: installing './compile'
configure.ac:22: installing './config.guess'
configure.ac:22: installing './config.sub'
configure.ac:21: installing './install-sh'
configure.ac:21: installing './missing'
Makefile.am: installing './depcomp'
```
If it has problems, you will need to install `automake`, `libtool` etc.

6) `./configure  --host=arm-linux-gnueabihf --prefix=/tmp/cross`

7) `make && make install` this will install to `/tmp/cross/...`

8) `file /tmp/cross/lib/libuv.so.1.0.0`  Check it's really built for ARM
```
/tmp/cross/lib/libuv.so.1.0.0: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=cdde0bc945e51db6001a9485349c035baaec2b46, with debug_info, not stripped
```

## 2/4: Building zlib cross

1) `cd /tmp`

2) `git clone https://github.com/madler/zlib.git`

3) `CC=arm-linux-gnueabihf-gcc ./configure --prefix=/tmp/cross`
```
Checking for shared library support...
Building shared library libz.so.1.2.11 with arm-linux-gnueabihf-gcc.
Checking for size_t... Yes.
Checking for off64_t... Yes.
Checking for fseeko... Yes.
Checking for strerror... Yes.
Checking for unistd.h... Yes.
Checking for stdarg.h... Yes.
Checking whether to use vs[n]printf() or s[n]printf()... using vs[n]printf().
Checking for vsnprintf() in stdio.h... Yes.
Checking for return value of vsnprintf()... Yes.
Checking for attribute(visibility) support... Yes.
```

4)  `make && make install`
```
arm-linux-gnueabihf-gcc -O3 -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN -I. -c -o example.o test/example.c
...
rm -f /tmp/cross/include/zlib.h /tmp/cross/include/zconf.h
cp zlib.h zconf.h /tmp/cross/include
chmod 644 /tmp/cross/include/zlib.h /tmp/cross/include/zconf.h
```

5) `file /tmp/cross/lib/libz.so.1.2.11`  This is just to confirm we built an ARM lib as expected
```
/tmp/cross/lib/libz.so.1.2.11: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=6f8ffef84389b1417d2fd1da1bd0c90f748f300d, with debug_info, not stripped
```

## 3/4: Building mbedtls cross

1) `cd /tmp`

2) `git clone https://github.com/ARMmbed/mbedtls.git`

3) `cd mbedtls ; mkdir build ; cd build`

3) `cmake .. -DCMAKE_TOOLCHAIN_FILE=/tmp/mytoolchainfile -DCMAKE_INSTALL_PREFIX:PATH=/tmp/cross -DCMAKE_BUILD_TYPE=RELEASE -DUSE_SHARED_MBEDTLS_LIBRARY=1`  mbedtls also uses cmake, so you can simply reuse the toolchain file you used for libwebsockets.  That is why you shouldn't put project-specific options in the toolchain file, it should just describe the toolchain.

4) `make && make install`

5) `file /tmp/cross/lib/libmbedcrypto.so.2.6.0`
```
/tmp/cross/lib/libmbedcrypto.so.2.6.0: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=bcca195e78bd4fd2fb37f36ab7d72d477d609d87, with debug_info, not stripped
```

## 4/4: Building libwebsockets with everything

1) `cd /tmp`

2) `git clone ssh://git@github.com/warmcat/libwebsockets`

3) `cd libwebsockets ; mkdir build ; cd build`

4)  (this is all one line on the commandline)
```
cmake .. -DCMAKE_TOOLCHAIN_FILE=/tmp/mytoolchainfile \
-DCMAKE_INSTALL_PREFIX:PATH=/tmp/cross \
-DLWS_WITH_LWSWS=1 \
-DLWS_WITH_MBEDTLS=1 \
-DLWS_MBEDTLS_LIBRARIES="/tmp/cross/lib/libmbedcrypto.so;/tmp/cross/lib/libmbedtls.so;/tmp/cross/lib/libmbedx509.so" \
-DLWS_MBEDTLS_INCLUDE_DIRS=/tmp/cross/include \
-DLWS_LIBUV_LIBRARIES=/tmp/cross/lib/libuv.so \
-DLWS_LIBUV_INCLUDE_DIRS=/tmp/cross/include \
-DLWS_ZLIB_LIBRARIES=/tmp/cross/lib/libz.so \
-DLWS_ZLIB_INCLUDE_DIRS=/tmp/cross/include
```

3) `make && make install`

4) `file /tmp/cross/lib/libwebsockets.so.11`
```
/tmp/cross/lib/libwebsockets.so.11: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=81e59c6534f8e9629a9fc9065c6e955ce96ca690, with debug_info, not stripped
```

5) `arm-linux-gnueabihf-objdump -p /tmp/cross/lib/libwebsockets.so.11 | grep NEEDED`  Confirm that the lws library was linked against everything we expect (libm / libc are provided by your toolchain)
```
  NEEDED               libz.so.1
  NEEDED               libmbedcrypto.so.0
  NEEDED               libmbedtls.so.10
  NEEDED               libmbedx509.so.0
  NEEDED               libuv.so.1
  NEEDED               libm.so.6
  NEEDED               libc.so.6
```

You will also find the lws test apps in `/tmp/cross/bin`... to run lws on the target you will need to copy the related things from /tmp/cross... all the .so from /tmp/cross/lib and anything from /tmp/cross/bin you want.

@section mem Memory efficiency

Embedded server-only configuration without extensions (ie, no compression
on websocket connections), but with full v13 websocket features and http
server, built on ARM Cortex-A9:

Update at 8dac94d (2013-02-18)
```
    $ ./configure --without-client --without-extensions --disable-debug --without-daemonize

    Context Creation, 1024 fd limit[2]:   16720 (includes 12 bytes per fd)
    Per-connection [3]:                      72 bytes, +1328 during headers

    .text	.rodata	.data	.bss
    11512	2784	288	4
```
This shows the impact of the major configuration with/without options at
13ba5bbc633ea962d46d using Ubuntu ARM on a PandaBoard ES.

These are accounting for static allocations from the library elf, there are
additional dynamic allocations via malloc.  These are a bit old now but give
the right idea for relative "expense" of features.

Static allocations, ARM9

|                                | .text   | .rodata | .data | .bss |
|--------------------------------|---------|---------|-------|------|
| All (no without)               | 35024   | 9940    | 336   | 4104 |
| without client                 | 25684   | 7144    | 336   | 4104 |
| without client, exts           | 21652   | 6288    | 288   | 4104 |
| without client, exts, debug[1] | 19756   | 3768    | 288   | 4104 |
| without server                 | 30304   | 8160    | 336   | 4104 |
| without server, exts           | 25382   | 7204    | 288   | 4104 |
| without server, exts, debug[1] | 23712   | 4256    | 288   | 4104 |

[1] `--disable-debug` only removes messages below `lwsl_notice`.  Since that is
the default logging level the impact is not noticeable, error, warn and notice
logs are all still there.

[2] `1024` fd per process is the default limit (set by ulimit) in at least Fedora
and Ubuntu.  You can make significant savings tailoring this to actual expected
peak fds, ie, at a limit of `20`, context creation allocation reduces to `4432 +
240 = 4672`)

[3] known header content is freed after connection establishment

# Captive Portal Detection

## Background

Wifi devices may face some interception of their connection to the
internet, it's very common for, eg, coffee shop wifi to present some
kind of login or other clickthrough before access to the Internet is
granted.  Devices may need to understand that they are in this
situation, and there are several different techniques for trying to
gague it.

Sequence-wise the device has been granted a DHCP lease and has been
configured with DNS, but the DNS may be wrongly resolving everything
to an address on the LAN or a portal on the net.

Whether there is a captive portal active should be a sticky state for a given
connection if there is not going to be any attempt to login or pass the landing
page, it only needs checking for after DHCP acquisition then.  If there will be
an attempt to satisfy the landing page, the test should be repeated after the
attempt.

## Detection schemes

The most popular detection scheme by numbers is Android's method,
which is to make an HTTP client GET to `http://connectivitycheck.android.com/generate_204`
and see if a 204 is coming back... if intercepted, typically there'll be a
3xx redirect to the portal, perhaps on https.  Or, it may reply on http with
a 200 and show the portal directly... either way it won't deliver a 204
like the real remote server does.

Variations include expecting a 200 but with specific http body content, and
doing a DNS lookup for a static IP that the device knows; if it's resolved to
something else, it knows there's monkey business implying a captive portal.

Other schemes involve https connections going out and detecting that the cert
of the server it's actually talking to doesn't check out, although this is
potentially ambiguous.

Yet more methods are possible outside of tcp or http.

## lws captive portal detect support

lws provides a generic api to start captive portal detection...

```
LWS_EXTERN LWS_VISIBLE int
lws_system_cpd_start(struct lws_context *context);
```

and two states in `lws_system` states to trigger it from, either
`LWS_SYSTATE_CPD_PRE_TIME` which happens after DHCP acquisition but before
ntpclient and is suitable for non https-based scheme where the time doesn't
need to be known, or the alternative `LWS_SYSTATE_CPD_POST_TIME` state which
happens after ntpclient has completed and we know the time.

The actual platform implementation is set using `lws_system_ops_t` function
pointer `captive_portal_detect_request`, ie

```
	int (*captive_portal_detect_request)(struct lws_context *context);
	/**< Check if we can go out on the internet cleanly, or if we are being
	 * redirected or intercepted by a captive portal.
	 * Start the check that proceeds asynchronously, and report the results
	 * by calling lws_captive_portal_detect_result() api
	 */
```

User platform code can provide this to implement whatever scheme they want, when
it has arrived at a result, it can call the lws api `lws_system_cpd_result()` to
inform lws.  If there isn't any captive portal, this will also try to advance the
system state towards OPERATIONAL.

```
/**
 * lws_system_cpd_result() - report the result of the captive portal detection
 *
 * \param context: the lws_context
 * \param result: one of the LWS_CPD_ constants representing captive portal state
 * \param redirect_url: NULL, or the url we were redirected to if result is
 *     LWS_CPD_HTTP_REDIRECT
 *
 * Sets the context's captive portal detection state to result.  User captive
 * portal detection code would call this once it had a result from its test.
 */
LWS_EXTERN LWS_VISIBLE int
lws_system_cpd_result(struct lws_context *context, int result, const char *redirect_url);
```

# RFC8152 COSE apis

|||
|---|---|---|
|cmake| `LWS_WITH_COSE`|
|Header| ./include/libwebsockets/lws-cose.h|
|api-test| ./minimal-examples/api-tests/api-test-cose/|
|README| ./READMEs/README.cbor-cose.md

COSE is the CBOR equivalent of the JOSE suite of crypto objects and operations.
You can represent public and private EC, RSA and SYMMETRIC keys, and sets of
keys of various types; import the logical keys to and from CBOR; and sign /
verify and encrypt / decrypt payloads using structured CBOR.  Key generation is
also supported.

|type|operations|algs|
|---|---|---|
|lws_cose_key_t|import, export, generation|EC / RSA / SYMMETRIC|
|cose_sign1|sign, validate|ES256/384/512, RS256/384/512|
|cose_sign|sign, validate|ES256/384/512, RS256/384/512|
|cose_mac0|sign, validate|HS256/HS256_64/384/512|
|cose_mac|validate only|HS256/HS256_64/384/512|

The lws COSE support uses the lws gencrypto layer, which calls through to the
tls crypto library, and so works on both OpenSSL and mbedTLS the same.

An increasing number of higher-level IETF specifications use COSE underneath.

## cose_key and sets

Lws provides an `lws_cose_key_t` object to contain a single key's metadata and
key material for EC, RSA and SYMMETRIC key types.

There is a commandline tool wrapping the key dumping and generation apis
available at `./minimal-examples/crypto/lws-crypto-cose-key`

### cose_key and sets import from CBOR and destroying

```
lws_cose_key_t *
lws_cose_key_import(lws_dll2_owner_t *pkey_set, lws_cose_key_import_callback cb,
		    void *user, const uint8_t *in, size_t len);
void
lws_cose_key_destroy(lws_cose_key_t **ck);

void
lws_cose_key_set_destroy(lws_dll2_owner_t *o);
```

To convert a single key, `pkey_set` should be NULL and the created key will be
returned, for a cose_key set, which is simply a CBOR array of cose_keys, it
should be a prepared (ie, zero'd down if nothing in it) lws_dll2_owner_t that
will contain the resulting list of `lws_cose_key_t` objects that were created.
In both cases the return is NULL if there was a fatal error and anything created
has been cleaned up, the return has no other meaning in the cose_key set case.

`lws_cose_key_destroy()` destroys a single `lws_cose_key_t` and sets the
contents of the pointer to NULL, for cose_key sets you instead pass a pointer to
the owner object to `lws_cose_key_set_destroy()` to destroy all the keys in the
set in one step.

cose_key has some confusions about type, kty and alg may be either ints,
representing well-known standardized key and alg types, or freeform strings.
We convert the well-known ints to their string representations at import, so
there can be no confusion later.

### cose_key generation

```
lws_cose_key_t *
lws_cose_key_generate(struct lws_context *context, int cose_kty, int use_mask,
		       int bits, const char *curve, const char *kid);
```

This creates an `lws_cose_key_t`, generates a key (SYMMETRIC) or keypair into
it and returns a pointer to it.

`cose_kty` is one of `LWSCOSE_WKKTV_OKP`, `LWSCOSE_WKKTV_EC2`, `LWSCOSE_WKKTV_RSA`,
or `LWSCOSE_WKKTV_SYMMETRIC`.  `bits` is valid for RSA keys and for EC keys,
`curve` should be a well-known curve name, one of `P-256`, `P-384` and `P-521`
currently.  `use_mask` is a bitfield made up of  (1 << LWSCOSE_WKKO_...) set to
enable the usage on the key.

### cose_key export to CBOR

The export api uses the same CBOR write context as `lws_lec_printf()` uses to
emit the key into an output buffer.  Like the CBOR output apis, it may return
`LWS_LECPCTX_RET_AGAIN` to indicate it filled the buffer and should be called
again to fill another buffer.  `lws_lec_init()` should be used to prepare the
write context and `lws_lec_setbuf()` to reset the output buffer on subsequent
calls, exactly the same as the CBOR write apis.

```
enum lws_lec_pctx_ret
lws_cose_key_export(lws_cose_key_t *ck, lws_lec_pctx_t *ctx, int flags);
```

`flags` may be 0 to only output the public key pieces, or `LWSJWKF_EXPORT_PRIVATE`
to output everything.

## Signing and signature validation

COSE specifies three kinds of signed object, `cose_sign1` which signs a payload
with a single algorithm and key, `cose_sign` which may sign a payload with
multiple algorithms and keys, and `countersign`.

`cose_sign1` has the advantage it can be validated with a single pass through
the signed object; `cose_sign` unfortunately specifies the parameters of the
signatures after the payload and must be done with multiple passes through the
payload, for inline payloads, by caching it in heap.

`cose_sign` and `cose_sign1` objects are supported by lws, Countersigned
objects are not yet supported.

`cose_mac0` is supported using HMAC for signing and validation, `cose_mac` is
only supported for validation.

There is a commandline tool wrapping the signing and validation apis
available at `./minimal-examples/crypto/lws-crypto-cose-sign`

### Signature validation

Signature validation does not have to be done synchronously, to facilitate this
first you create a validation context specifying the type (eg, `SIGTYPE_SINGLE`)
and a keyset of public keys the signature might use to validate (notice even a
single key is passed in an lws_dll2_owner_t keyset).

Creation uses a public `lws_cose_validate_create_info_t` info struct

```
typedef struct lws_cose_validate_create_info {
	struct lws_context		*cx;
	/**< REQUIRED: the lws context */
	lws_dll2_owner_t		*keyset;
	/**< REQUIRED: one or more cose_keys */

	enum lws_cose_sig_types		sigtype;
	/**<  0 if a CBOR tag is in the sig, else one of SIGTYPE_MULTI,
	 * SIGTYPE_SINGLE, etc*/

	lws_cose_validate_pay_cb_t	pay_cb;
	/**< optional: called back with unvalidated payload pieces */
	void				*pay_opaque;
	/**< optional: passed into pay_cb callback along with payload chunk */

	lws_cose_sign_ext_pay_cb_t	ext_cb;
	/**< optional extra application data provision callback */
	void				*ext_opaque;
	/**< optional extra application data provision callback opaque */
	size_t				ext_len;
	/**< if we have extra app data, this must be set to the length of it */
} lws_cose_validate_create_info_t;
```

```
struct lws_cose_validate_context *
lws_cose_validate_create(const lws_cose_validate_create_info_t *info);

void
lws_cose_validate_destroy(struct lws_cose_validate_context **cps);
```

after that as pieces of the signature CBOR become available, they can be
processed by the validation context

```
int
lws_cose_validate_chunk(struct lws_cose_validate_context *cps,
			const uint8_t *in, size_t in_len, size_t *used_in);
```

The parsing of the signature yields a list of result objects indicating
information about each signature it encountered and whether it was validated or
not.  The parsing itself only fails if there is an unrecoverable error, the
completion of parsing does not indicate validation, it may yield zero or more
result objects indicating the validation failed.

```
lws_dll2_owner_t *
lws_cose_validate_results(struct lws_cose_validate_context *cps);

typedef struct {
	lws_dll2_t		list;

	const lws_cose_key_t	*cose_key;
	cose_param_t		cose_alg;

	int			result; /* 0 = validated */

} lws_cose_validate_res_t;
```

It's like this because for multiple signatures, we may only have keys for some
of them, and we may have different policies for validation that can only be
assessed as a whole, eg, we may inisit that signatures pass with specific
algorithms, or all signatures for specific keys must be present and pass.  This
way user code can assess the situation after the signature parsing and make its
decision about overall validity according to its own policies.

## Signing

Signing is again done by creating a signing context using an info struct to pass
in the paramter (a `lws_cose_sign_create_info_t`).

```
#define LCSC_FL_ADD_CBOR_TAG		(1 << 0)
#define LCSC_FL_ADD_CBOR_PREFER_MAC0	(1 << 1)

typedef struct lws_cose_sign_create_info {
	struct lws_context		*cx;
	/**< REQUIRED: the lws context */
	lws_dll2_owner_t		*keyset;
	/**< REQUIRED: one or more cose_keys */

	lws_lec_pctx_t			*lec;
	/**< REQUIRED: the cbor output context to emit to, user must
	 * initialize with lws_lec_init() beforehand */

	lws_cose_sign_ext_pay_cb_t	ext_cb;
	/**< optional extra application data provision callback */
	void				*ext_opaque;
	/**< optional extra application data provision callback opaque */
	size_t				ext_len;
	/**< if we have extra app data, this must be set to the length of it */

	size_t				inline_payload_len;
	/**< REQUIRED: size of the inline payload we will provide */

	int				flags;
	/**< bitmap of  LCSC_FL_* */
	enum lws_cose_sig_types		sigtype;
	/**< 0, or sign type hint */
} lws_cose_sign_create_info_t;
```

```
struct lws_cose_sign_context *
lws_cose_sign_create(const lws_cose_sign_create_info_t *info);
```

After creating the signing context, you call `lws_cose_sign_add()` one or more
times to add algorithms and keys to sign with (since cose_sign allows multiple
recipients with the same payload signed in different ways).

```
int
lws_cose_sign_add(struct lws_cose_sign_context *csc, cose_param_t alg,
		  const lws_cose_key_t *ck);
```

The payload does not have to be provided all at once and can be passed in chunk
by chunk over time via `lws_cose_sign_payload_chunk()`.

Output is mediated via an lws CBOR output context provided in the info at
creation-time, it's only emitted during the `lws_cose_sign_payload_chunk()`
phase.  If it returns `LWS_LECPCTX_RET_AGAIN`, you must call that api again
after using the CBOR output context data and resetting its buffer by
`lws_lec_setbuf()`, so it can continue to output.

```
enum lws_lec_pctx_ret
lws_cose_sign_payload_chunk(struct lws_cose_sign_context *csc,
			    const uint8_t *in, size_t in_len);
```

Finally the signing context is destroyed.

```
void
lws_cose_sign_destroy(struct lws_cose_sign_context **csc);
```

# RFC8949 CBOR Stream Parsing and Writing

|||
|---|---|---|
|cmake| `LWS_WITH_CBOR`, `LWS_WITH_CBOR_FLOAT`|
|Header| ./include/libwebsockets/lws-lecp.h|
|api-test| ./minimal-examples/api-tests/api-test-lecp/|
|test app| ./test-apps/test-lecp.c -> libwebsockets-test-lecp|

LECP is the RFC8949 CBOR stream parsing counterpart to LEJP for JSON.

## Features

 - Completely immune to input fragmentation, give it any size blocks of CBOR as
   they become available; 1 byte, or 100K at a time give identical parsing
   results
 - Input chunks discarded as they are parsed, whole CBOR never needed in memory
 - Nonrecursive, fixed stack usage of a few dozen bytes
 - No heap allocations at all, just requires ~500 byte context usually on
   caller stack
 - Creates callbacks to a user-provided handler as members are parsed out
 - No payload size limit, supports huge / endless strings or blobs bigger than
   system memory
 - Collates utf-8 text and blob payloads into a 250-byte chunk buffer for ease
   of access
 - Write apis don't use any heap allocations or recursion either
 - Write apis use an explicit context with its own lifecycle, and printf style
   vaargs including sized blobs, C strings, double, int, unsigned long etc
 - Completely immune to output fragmentation, supports huge strings and blobs
   into small buffers, api returns to indicates unfinished if it needs to be
   called again to continue; 1 byte or 100K output buffer give same results
 - Write apis completely fill available buffer and if unfinished, continues
   into same or different buffer when called again with same args; no
   requirement for subsequent calls to be done sequentially or even from same
   function

## Type limits

CBOR allows negative integers of up to 64 bits, these do not fit into a `uint64_t`.
LECP has a union for numbers that includes the types `uint64_t` and `int64_t`,
but it does not separately handle negative integers.  Only -2^63.. 2^64 -1 can
be handled by the C types, the oversize negative numbers wrap and should be
avoided.

## Floating point support

Floats are handled using the IEEE memory format, it means they can be parsed
from the CBOR without needing any floating point support in the build.  If
floating point is available, you can also enable `LWS_WITH_CBOR_FLOAT` and
a `float` and `double` types are available in the number item union.  Otherwise
these are handled as `ctx->item.u.u32` and `ctx->item.u.u64` union members.

Half-float (16-bit) is defined in CBOR and always handled as a `uint16_t`
number union member `ctx->item.u.hf`.

## Callback reasons

The user callback does not have to handle any callbacks, it only needs to
process the data for the ones it is interested in.

|Callback reason|CBOR structure|Associated data|
|---|---|---|
|`LECPCB_CONSTRUCTED`|Created the parse context||
|`LECPCB_DESTRUCTED`|Destroyed the parse context||
|`LECPCB_COMPLETE`|The parsing completed OK||
|`LECPCB_FAILED`|The parsing failed||
|`LECPCB_VAL_TRUE`|boolean true||
|`LECPCB_VAL_FALSE`|boolean false||
|`LECPCB_VAL_NULL`|explicit NULL||
|`LECPCB_VAL_NUM_INT`|signed integer|`ctx->item.u.i64`|
|`LECPCB_VAL_STR_START`|A UTF-8 string is starting||
|`LECPCB_VAL_STR_CHUNK`|The next string chunk|`ctx->npos` bytes in `ctx->buf`|
|`LECPCB_VAL_STR_END`|The last string chunk|`ctx->npos` bytes in `ctx->buf`|
|`LECPCB_ARRAY_START`|An array is starting||
|`LECPCB_ARRAY_END`|An array has ended||
|`LECPCB_OBJECT_START`|A CBOR map is starting||
|`LECPCB_OBJECT_END`|A CBOR map has ended||
|`LECPCB_TAG_START`|The following data has a tag index|`ctx->item.u.u64`|
|`LECPCB_TAG_END`|The end of the data referenced by the last tag||
|`LECPCB_VAL_NUM_UINT`|Unsigned integer|`ctx->item.u.u64`|
|`LECPCB_VAL_UNDEFINED`|CBOR undefined||
|`LECPCB_VAL_FLOAT16`|half-float available as host-endian `uint16_t`|`ctx->item.u.hf`|
|`LECPCB_VAL_FLOAT32`|`float` (`uint32_t` if no float support) available|`ctx->item.u.f`|
|`LECPCB_VAL_FLOAT64`|`double` (`uint64_t` if no float support) available|`ctx->item.u.d`|
|`LECPCB_VAL_SIMPLE`|CBOR simple|`ctx->item.u.u64`|
|`LECPCB_VAL_BLOB_START`|A binary blob is starting||
|`LECPCB_VAL_BLOB_CHUNK`|The next blob chunk|`ctx->npos` bytes in `ctx->buf`|
|`LECPCB_VAL_BLOB_END`|The last blob chunk|`ctx->npos` bytes in `ctx->buf`|
|`LECPCB_ARRAY_ITEM_START`|A logical item in an array is starting|
|`LCEPDB_ARRAY_ITEM_END`|A logical item in an array has completed|

## CBOR indeterminite lengths

Indeterminite lengths are supported, but are concealed in the parser as far as
possible, the CBOR lengths or its indeterminacy are not exposed in the callback
interface at all, just chunks of data that may be the start, the middle, or the
end.

## Handling CBOR UTF-8 strings and blobs

When a string or blob is parsed, an advisory callback of `LECPCB_VAL_STR_START` or
`LECPCB_VAL_BLOB_START` occurs first.  The `_STR_` callbacks indicate the
content is a CBOR UTF-8 string, `_BLOB_` indicates it is binary data.

Strings or blobs may have indeterminite length, but if so, they are composed
of logical chunks which must have known lengths.  When the `_START` callback
occurs, the logical length either of the whole string, or of the sub-chunk if
indeterminite length, can be found in `ctx->item.u.u64`.

Payload is collated into `ctx->buf[]`, the valid length is in `ctx->npos`.

For short strings or blobs where the length is known, the whole payload is
delivered in a single `LECPCB_VAL_STR_END` or `LECPCB_VAL_BLOB_END` callback.

For payloads larger than the size of `ctx->buf[]`, `LECPCB_VAL_STR_CHUNK` or
`LECPCB_VAL_BLOB_CHUNK` callbacks occur delivering each sequential bufferload.
If the CBOR indicates the total length, the last chunk is delievered in a
`LECPCB_VAL_STR_END` or `LECPCB_VAL_BLOB_END`.

If the CBOR indicates the string end after the chunk, a zero-length `..._END`
callback is provided.

## Handling CBOR tags

CBOR tags are exposed as `LECPCB_TAG_START` and `LECPCB_TAG_END` pairs, at
the `_START` callback the tag index is available in `ctx->item.u.u64`.

## CBOR maps

You can check if you are on the "key" part of a map "key:value" pair using the
helper api `lecp_parse_map_is_key(ctx)`.

## Parsing paths

LECP maintains a "parsing path" in `ctx->path` that represents the context of
the callback events.  As a convenience, at LECP context creation time, you can
pass in an array of path strings you want to match on, and have any match
checkable in the callback using `ctx->path_match`, it's 0 if no active match,
or the match index from your path array starting from 1 for the first entry.

|CBOR element|Representation in path|
|---|---|
|CBOR Array|`[]`|
|CBOR Map|`.`|
|CBOR Map entry key string|`keystring`|

## Accessing raw CBOR subtrees

Some CBOR usages like COSE require access to selected raw CBOR from the input
stream.  `lecp_parse_report_raw(ctx, on)` lets you turn on and off buffering of
raw CBOR and reporting it in the parse callback with `LECPCB_LITERAL_CBOR`
callbacks.  The callbacks mean the temp buffer `ctx->cbor[]` has `ctx->cbor_pos`
bytes of raw CBOR available in it.  Callbacks are triggered when the buffer
fills, or reporting is turned off and the buffer has something in it.

By turning the reporting on and off according to the outer CBOR parsing state,
it's possible to get exactly the raw CBOR subtree that's needed.

Capturing and reporting the raw CBOR does not change that the same CBOR is being
passed to the parser as usual as well.

## Comparison with LEJP (JSON parser)

LECP is based on the same principles as LEJP and shares most of the callbacks.
The major differences:

 - LEJP value callbacks all appear in `ctx->buf[]`, ie, floating-point is
   provided to the callback in ascii form like `"1.0"`.  CBOR provides a more
   strict typing system, and the different type values are provided either in
   `ctx->buf[]` for blobs or utf-8 text strtings, or the `item.u` union for
   converted types, with additional callback reasons specific to each type.

 - CBOR "maps" use `_OBJECT_START` and `_END` parsing callbacks around the
   key / value pairs.  LEJP has a special callback type `PAIR_NAME` for the
   key string / integer, but in LECP these are provided as generic callbacks
   dependent on type, ie, generic string callbacks or integer ones, and the
   value part is represented according to whatever comes.


# Writing CBOR

CBOR is written into a `lws_lec_pctx_t` object that has been initialized to
point to an output buffer of a specified size, using printf type formatting.

Output is paused if the buffer fills, and the write api may be called again
later with the same context object, to resume emitting to the same or different
buffer.

This allows bufferloads of encoded CBOR to be produced on demand, it's designed
to fit usage in WRITEABLE callbacks and Secure Streams tx() callbacks where the
buffer size for one packet is already fixed.

CBOR array and map lengths are deduced from the format string, as is whether to
use indeterminite length formatting or not.  For indeterminite text or binary
strings, a container of < > 

|Format|Arg(s)|Meaning|
|---|---|---|
|`123`||unsigned literal number|
|`-123`||signed literal number|
|`%u`|`unsigned int`|number|
|`%lu`|`unsigned long int`|number|
|`%llu`|`unsigned long long int`|number|
|`%d`|`signed int`|number|
|`%ld`|`signed long int`|number|
|`%lld`|`signed long long int`|number|
|`%f`|`double`|floating point number|
|`123(...)`||literal tag and scope|
|`%t(...)`|`unsigned int`|tag and scope|
|`%lt(...)`|`unsigned long int`|tag and scope|
|`%llt(...)`|`unsigned long long int`|tag and scope|
|`[...]`||Array (fixed len if `]` in same format string)|
|`{...}`||Map (fixed len if `}` in same format string)|
|`<t...>`||Container for indeterminite text string frags|
|`<b...>`||Container for indeterminite binary string frags|
|`'string'`||Literal text of known length|
|`%s`|`const char *`|NUL-terminated string|
|`%.*s`|`int`, `const char *`|length-specified string|
|`%.*b`|`int`, `const uint8_t *`|length-specified binary|
|`:`||separator between Map items (a:b)|
|`,`||separator between Map pairs or array items|

Backslash is used as an escape in `'...'` literal strings, so `'\\'` represents
a string consisting of a single backslash, and `'\''` a string consisting of a
single single-quote.

For integers, various natural C types are available, but in all cases, the
number is represented in CBOR using the smallest valid way based on its value,
the long or long-long modifiers just apply to the expected C type in the args.

For floats, the C argument is always expected to be a `double` type following
C type promotion, but again it is represented in CBOR using the smallest valid
way based on value, half-floats are used for NaN / Infinity and where possible
for values like 0.0 and -1.0.

## Examples

### Literal ints

```
	uint8_t buf[128];
	lws_lec_pctx_t cbw;

	lws_lec_init(&cbw, buf, sizeof(buf));
	lws_lec_printf(ctx, "-1");
```
|||
|---|---|
|Return| `LWS_LECPCTX_RET_FINISHED`|
|`ctx->used`|1|
|`buf[]`|20|

### Dynamic ints

```
	uint8_t buf[128];
	lws_lec_pctx_t cbw;
	int n = -1; /* could be long */

	lws_lec_init(&cbw, buf, sizeof(buf));
	lws_lec_printf(ctx, "%d", n); /* use %ld for long */
```
|||
|---|---|
|Return| `LWS_LECPCTX_RET_FINISHED`|
|`ctx->used`|1|
|`buf[]`|20|

### Maps, arrays and dynamic ints

```
	...
	int args[3] = { 1, 2, 3 };

	lws_lec_printf(ctx, "{'a':%d,'b':[%d,%d]}", args[0], args[1], args[2]);
```

|||
|---|---|
|Return| `LWS_LECPCTX_RET_FINISHED`|
|`ctx->used`|9|
|`buf[]`|A2 61 61 01 61 62 82 02 03|

### String longer than the buffer

Using `%s` and the same string as an arg gives same results

```
	uint8_t buf[16];
	lws_lec_pctx_t cbw;

	lws_lec_init(&cbw, buf, sizeof(buf));
	lws_lec_printf(ctx, "'A literal string > one buf'");
	/* not required to be in same function context or same buf,
	 * but the string must remain the same */
	lws_lec_setbuf(&cbw, buf, sizeof(buf));
	lws_lec_printf(ctx, "'A literal string > one buf'");
```

First call

|||
|---|---|
|Return| `LWS_LECPCTX_RET_AGAIN`|
|`ctx->used`|16|
|`buf[]`|78 1A 41 20 6C 69 74 65 72 61 6C 20 73 74 72 69|

Second call

|||
|---|---|
|Return| `LWS_LECPCTX_RET_FINISHED`|
|`ctx->used`|12|
|`buf[]`|6E 67 20 3E 20 6F 6E 65 20 62 75 66|

### Binary blob longer than the buffer

```
	uint8_t buf[16], blob[] = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18 };
	lws_lec_pctx_t cbw;

	lws_lec_init(&cbw, buf, sizeof(buf));
	lws_lec_printf(ctx, "%.*b", (int)sizeof(blob), blob);
	/* not required to be in same function context or same buf,
	 * but the length and blob must remain the same */
	lws_lec_setbuf(&cbw, buf, sizeof(buf));
	lws_lec_printf(ctx, "%.*b", (int)sizeof(blob), blob);
```

First call

|||
|---|---|
|Return| `LWS_LECPCTX_RET_AGAIN`|
|`ctx->used`|16|
|`buf[]`|52 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F|

Second call

|||
|---|---|
|Return| `LWS_LECPCTX_RET_FINISHED`|
|`ctx->used`|3|
|`buf[]`|10 11 12|

## Need for CI

Generally if we're adding something that's supposed to work ongoing, the stuff
should be exercised in CI (at least Travis).

If there are few users for a particular feature, experience has shown that
refactors or other upheaval can easily break it into a state of uselessness
without anyone noticing until later.

Therefore here's a description of how to add something to the CI tests... this
is certainly a nonproductive PITA and I have never been thanked for the work
involved.  But if the promise of the various features working is going to
remain alive, it's necessary to include CI test where possible with new
nontrivial code.

## Integration points

### cmake

`.travis.yml` maps the various test activities to CMake options needed.

### including dependent packages into travis

See `./scripts/travis_install.sh`

### performing prepared test actions

See `./scripts/travis_control.sh`

# Tips about CMake

## Don't be afraid to nuke your build dir

CMake likes to cache options and other things in the build dir... if you stop
asserting the state of something like `-DMY_OPTION=1`, then the last way it was
set it cached.  On order to keep track of what you have set and not set, it's
very advisable to explicitly keep all your options and set them all on one cmake
line.

Then, when you meet a situation you changed something but somehow cmake is
sticking with what it knew before, you can fearlessly delete your build dir
and create a new one with your explicit config.

On Linux, it's usually enough to delete `CMakeCache.txt` to trigger it to config
from the start again, but on, eg, windows, it isn't, for whatever reason it
literally needs the build dir removing.

## CMake presence tests that fail

Lws makes use of various CMake features to figure out what apis your libraries
offer, eg, OpenSSL has many different apis based on version, lws knows how to
work around most of the changes, but to do it it must find out what apis are
available first on your build environment.

CMake basically builds little throwaway test programs using each api in turn, and
if it builds, it understands that the api was available and sets a preprocessor
symbol that's available in the main build accordingly.  Then we can do `#if xxx`
to figure out if we can use `xxx` or need to do a workaround at build-time.

This works very well, but unfortunately if the program didn't build, there are
many possible ways for the build to break even if the api being tested is
really available... for example, some library in your toolchain isn't being
linked for the throwaway test program.

When this happens, cmake indicates that apis that must be available are not available...
CMake keeps a log of what happened with the failed test programs in
`./build/CMakeFiles/CMakeError.log`.  This is appeneded to, so the best way is blow
away the build dir and reconfig a new one from scratch, and go look in there to
find out what the compiler or linker was complaining about.

Notes about coding with lws
===========================

@section era Old lws and lws v2.0+

Originally lws only supported the "manual" method of handling everything in the
user callback found in test-server.c / test-server-http.c.

Since v2.0, the need for most or all of this manual boilerplate has been
eliminated: the protocols[0] http stuff is provided by a generic lib export
`lws_callback_http_dummy()`.  You can serve parts of your filesystem at part of
the URL space using mounts, the dummy http callback will do the right thing.

It's much preferred to use the "automated" v2.0 type scheme, because it's less
code and it's easier to support.

The minimal examples all use the modern, recommended way.

If you just need generic serving capability, without the need to integrate lws
to some other app, consider not writing any server code at all, and instead use
the generic server `lwsws`, and writing your special user code in a standalone
"plugin".  The server is configured for mounts etc using JSON, see
./READMEs/README.lwsws.md.

Although the "plugins" are dynamically loaded if you use lwsws or lws built
with libuv, actually they may perfectly well be statically included if that
suits your situation better, eg, ESP32 test server, where the platform does
not support processes or dynamic loading, just #includes the plugins
one after the other and gets the same benefit from the same code.

Isolating and collating the protocol code in one place also makes it very easy
to maintain and understand.

So it if highly recommended you put your protocol-specific code into the
form of a "plugin" at the source level, even if you have no immediate plan to
use it dynamically-loaded.

@section writeable Only send data when socket writeable

You should only send data on a websocket connection from the user callback
`LWS_CALLBACK_SERVER_WRITEABLE` (or `LWS_CALLBACK_CLIENT_WRITEABLE` for
clients).

If you want to send something, do NOT just send it but request a callback
when the socket is writeable using

 - `lws_callback_on_writable(wsi)` for a specific `wsi`, or
 
 - `lws_callback_on_writable_all_protocol(protocol)` for all connections
using that protocol to get a callback when next writeable.

Usually you will get called back immediately next time around the service
loop, but if your peer is slow or temporarily inactive the callback will be
delayed accordingly.  Generating what to write and sending it should be done
in the ...WRITEABLE callback.

See the test server code for an example of how to do this.

Otherwise evolved libs like libuv get this wrong, they will allow you to "send"
anything you want but it only uses up your local memory (and costs you
memcpys) until the socket can actually accept it.  It is much better to regulate
your send action by the downstream peer readiness to take new data in the first
place, avoiding all the wasted buffering.

Libwebsockets' concept is that the downstream peer is truly the boss, if he,
or our connection to him, cannot handle anything new, we should not generate
anything new for him.  This is how unix shell piping works, you may have
`cat a.txt | grep xyz > remote", but actually that does not cat anything from
a.txt while remote cannot accept anything new. 

@section oneper Only one lws_write per WRITEABLE callback

From v2.5, lws strictly enforces only one lws_write() per WRITEABLE callback.

You will receive a message about "Illegal back-to-back write of ... detected"
if there is a second lws_write() before returning to the event loop.

This is because with http/2, the state of the network connection carrying a
wsi is unrelated to any state of the wsi.  The situation on http/1 where a
new request implied a new tcp connection and new SSL buffer, so you could
assume some window for writes is no longer true.  Any lws_write() can fail
and be buffered for completion by lws; it will be auto-completed by the
event loop.

Note that if you are handling your own http responses, writing the headers
needs to be done with a separate lws_write() from writing any payload.  That
means after writing the headers you must call `lws_callback_on_writable(wsi)`
and send any payload from the writable callback.

@section otherwr Do not rely on only your own WRITEABLE requests appearing

Libwebsockets may generate additional `LWS_CALLBACK_CLIENT_WRITEABLE` events
if it met network conditions where it had to buffer your send data internally.

So your code for `LWS_CALLBACK_CLIENT_WRITEABLE` needs to own the decision
about what to send, it can't assume that just because the writeable callback
came something is ready to send.

It's quite possible you get an 'extra' writeable callback at any time and
just need to `return 0` and wait for the expected callback later.

@section dae Daemonization

There's a helper api `lws_daemonize` built by default that does everything you
need to daemonize well, including creating a lock file.  If you're making
what's basically a daemon, just call this early in your init to fork to a
headless background process and exit the starting process.

Notice stdout, stderr, stdin are all redirected to /dev/null to enforce your
daemon is headless, so you'll need to sort out alternative logging, by, eg,
syslog via `lws_set_log_level(..., lwsl_emit_syslog)`.

@section conns Maximum number of connections

The maximum number of connections the library can deal with is decided when
it starts by querying the OS to find out how many file descriptors it is
allowed to open (1024 on Fedora for example).  It then allocates arrays that
allow up to that many connections, minus whatever other file descriptors are
in use by the user code.

If you want to restrict that allocation, or increase it, you can use ulimit or
similar to change the available number of file descriptors, and when restarted
**libwebsockets** will adapt accordingly.

@section peer_limits optional LWS_WITH_PEER_LIMITS

If you select `LWS_WITH_PEER_LIMITS` at cmake, then lws will track peer IPs
and monitor how many connections and ah resources they are trying to use
at one time.  You can choose to limit these at context creation time, using
`info.ip_limit_ah` and `info.ip_limit_wsi`.

Note that although the ah limit is 'soft', ie, the connection will just wait
until the IP is under the ah limit again before attaching a new ah, the
wsi limit is 'hard', lws will drop any additional connections from the
IP until it's under the limit again.

If you use these limits, you should consider multiple clients may simultaneously
try to access the site through NAT, etc.  So the limits should err on the side
of being generous, while still making it impossible for one IP to exhaust
all the server resources.

@section evtloop Libwebsockets is singlethreaded

Libwebsockets works in a serialized event loop, in a single thread.  It supports
the default poll() backend, and libuv, libev, and libevent event loop
libraries that also take this locking-free, nonblocking event loop approach that
is not threadsafe.  There are several advantages to this technique, but one
disadvantage, it doesn't integrate easily if there are multiple threads that
want to use libwebsockets.

However integration to multithreaded apps is possible if you follow some guidelines.

1) Aside from two APIs, directly calling lws apis from other threads is not allowed.

2) If you want to keep a list of live wsi, you need to use lifecycle callbacks on
the protocol in the service thread to manage the list, with your own locking.
Typically you use an ESTABLISHED callback to add ws wsi to your list and a CLOSED
callback to remove them.

3) LWS regulates your write activity by being able to let you know when you may
write more on a connection.  That reflects the reality that you cannot succeed to
send data to a peer that has no room for it, so you should not generate or buffer
write data until you know the peer connection can take more.

Other libraries pretend that the guy doing the writing is the boss who decides
what happens, and absorb as much as you want to write to local buffering.  That does
not scale to a lot of connections, because it will exhaust your memory and waste
time copying data around in memory needlessly.

The truth is the receiver, along with the network between you, is the boss who
decides what will happen.  If he stops accepting data, no data will move.  LWS is
designed to reflect that.

If you have something to send, you call `lws_callback_on_writable()` on the
connection, and when it is writeable, you will get a `LWS_CALLBACK_SERVER_WRITEABLE`
callback, where you should generate the data to send and send it with `lws_write()`.

You cannot send data using `lws_write()` outside of the WRITEABLE callback.

4) For multithreaded apps, this corresponds to a need to be able to provoke the
`lws_callback_on_writable()` action and to wake the service thread from its event
loop wait (sleeping in `poll()` or `epoll()` or whatever).  The rules above
mean directly sending data on the connection from another thread is out of the
question.

The only lws api that's safe to call from other thread contexts is `lws_cancel_service()`.
This will take a platform-specific action to wake the lws event loop thread wait,
either put a byte into a pipe2() the event loop is waiting on, or send a packet on
a UDP socket pair that the event loop waits on.  When the wake is handled by the
lws event loop thread, it will broadcast a `LWS_CALLBACK_EVENT_WAIT_CANCELLED`
message to every vhost-protocol instantiation, so you can handle this callback,
usually lock a shared data region, and if you see you need to write, call
`lws_callback_on_writeable()` for the wsi(s) that need to write.

There's no restriction on multiple threads calling `lws_cancel_service()`, it's
unconditionally safe due to how it is implemented underneath.

5) The obverse of this truism about the receiver being the boss is the case where
we are receiving.  If we get into a situation we actually can't usefully
receive any more, perhaps because we are passing the data on and the guy we want
to send to can't receive any more, then we should "turn off RX" by using the
RX flow control API, `lws_rx_flow_control(wsi, 0)`.  When something happens where we
can accept more RX, (eg, we learn our onward connection is writeable) we can call
it again to re-enable it on the incoming wsi.

LWS stops calling back about RX immediately you use flow control to disable RX, it
buffers the data internally if necessary.  So you will only see RX when you can
handle it.  When flow control is disabled, LWS stops taking new data in... this makes
the situation known to the sender by TCP "backpressure", the tx window fills and the
sender finds he cannot write any more to the connection.

See the mirror protocol implementations for example code.

If you need to service other socket or file descriptors as well as the
websocket ones, you can combine them together with the websocket ones
in one poll loop, see "External Polling Loop support" below, and
still do it all in one thread / process context.  If the need is less
architectural, you can also create RAW mode client and serving sockets; this
is how the lws plugin for the ssh server works.

@section anonprot Working without a protocol name

Websockets allows connections to negotiate without a protocol name...
in that case by default it will bind to the first protocol in your
vhost protocols[] array.

You can tell the vhost to use a different protocol by attaching a
pvo (per-vhost option) to the 

```
/*
 * this sets a per-vhost, per-protocol option name:value pair
 * the effect is to set this protocol to be the default one for the vhost,
 * ie, selected if no Protocol: header is sent with the ws upgrade.
 */

static const struct lws_protocol_vhost_options pvo_opt = {
	NULL,
	NULL,
	"default",
	"1"
};

static const struct lws_protocol_vhost_options pvo = {
	NULL,
	&pvo_opt,
	"my-protocol",
	""
};

...

	context_info.pvo = &pvo;
...

```

Will select "my-protocol" from your protocol list (even if it came
in by plugin) as being the target of client connections that don't
specify a protocol.

@section closing Closing connections from the user side

When you want to close a connection, you do it by returning `-1` from a
callback for that connection.

You can provoke a callback by calling `lws_callback_on_writable` on
the wsi, then notice in the callback you want to close it and just return -1.
But usually, the decision to close is made in a callback already and returning
-1 is simple.

If the socket knows the connection is dead, because the peer closed or there
was an affirmitive network error like a FIN coming, then **libwebsockets**  will
take care of closing the connection automatically.

If you have a silently dead connection, it's possible to enter a state where
the send pipe on the connection is choked but no ack will ever come, so the
dead connection will never become writeable.  To cover that, you can use TCP
keepalives (see later in this document) or pings.

@section gzip Serving from inside a zip file

Lws now supports serving gzipped files from inside a zip container.  Thanks to
Per Bothner for contributing the code.

This has the advtantage that if the client can accept GZIP encoding, lws can
simply send the gzip-compressed file from inside the zip file with no further
processing, saving time and bandwidth.

In the case the client can't understand gzip compression, lws automatically
decompressed the file and sends it normally.

Clients with limited storage and RAM will find this useful; the memory needed
for the inflate case is constrained so that only one input buffer at a time
is ever in memory.

To use this feature, ensure LWS_WITH_ZIP_FOPS is enabled at CMake.

`libwebsockets-test-server-v2.0` includes a mount using this technology
already, run that test server and navigate to http://localhost:7681/ziptest/candide.html

This will serve the book Candide in html, together with two jpgs, all from
inside a .zip file in /usr/[local/]share-libwebsockets-test-server/candide.zip

Usage is otherwise automatic, if you arrange a mount that points to the zipfile,
eg, "/ziptest" -> "mypath/test.zip", then URLs like `/ziptest/index.html` will be
servied from `index.html` inside `mypath/test.zip`

@section frags Fragmented messages

To support fragmented messages you need to check for the final
frame of a message with `lws_is_final_fragment`. This
check can be combined with `libwebsockets_remaining_packet_payload`
to gather the whole contents of a message, eg:

```
	    case LWS_CALLBACK_RECEIVE:
	    {
	        Client * const client = (Client *)user;
	        const size_t remaining = lws_remaining_packet_payload(wsi);
	
	        if (!remaining && lws_is_final_fragment(wsi)) {
	            if (client->HasFragments()) {
	                client->AppendMessageFragment(in, len, 0);
	                in = (void *)client->GetMessage();
	                len = client->GetMessageLength();
	            }
	
	            client->ProcessMessage((char *)in, len, wsi);
	            client->ResetMessage();
	        } else
	            client->AppendMessageFragment(in, len, remaining);
	    }
	    break;
```

The test app libwebsockets-test-fraggle sources also show how to
deal with fragmented messages.


@section debuglog Debug Logging

See ./READMEs/README.logging.md

@section asan Building with ASAN

Under GCC you can select for the build to be instrumented with the Address
Sanitizer, using `cmake .. -DCMAKE_BUILD_TYPE=DEBUG -DLWS_WITH_ASAN=1`.  LWS is routinely run during development with valgrind, but ASAN is capable of finding different issues at runtime, like operations which are not strictly defined in the C
standard and depend on platform behaviours.

Run your application like this

```
	$ sudo ASAN_OPTIONS=verbosity=2:halt_on_error=1  /usr/local/bin/lwsws
```

and attach gdb to catch the place it halts.

@section extpoll External Polling Loop support

**libwebsockets** maintains an internal `poll()` array for all of its
sockets, but you can instead integrate the sockets into an
external polling array.  That's needed if **libwebsockets** will
cooperate with an existing poll array maintained by another
server.

Three callbacks `LWS_CALLBACK_ADD_POLL_FD`, `LWS_CALLBACK_DEL_POLL_FD`
and `LWS_CALLBACK_CHANGE_MODE_POLL_FD` appear in the callback for protocol 0
and allow interface code to manage socket descriptors in other poll loops.

You can pass all pollfds that need service to `lws_service_fd()`, even
if the socket or file does not belong to **libwebsockets** it is safe.

If **libwebsocket** handled it, it zeros the pollfd `revents` field before returning.
So you can let **libwebsockets** try and if `pollfd->revents` is nonzero on return,
you know it needs handling by your code.

Also note that when integrating a foreign event loop like libev or libuv where
it doesn't natively use poll() semantics, and you must return a fake pollfd
reflecting the real event:

 - be sure you set .events to .revents value as well in the synthesized pollfd

 - check the built-in support for the event loop if possible (eg, ./lib/libuv.c)
   to see how it interfaces to lws
   
 - use LWS_POLLHUP / LWS_POLLIN / LWS_POLLOUT from libwebsockets.h to avoid
   losing windows compatibility

You also need to take care about "forced service" somehow... these are cases
where the network event was consumed, incoming data was all read, for example,
but the work arising from it was not completed.  There will not be any more
network event to trigger the remaining work, Eg, we read compressed data, but
we did not use up all the decompressed data before returning to the event loop
because we had to write some of it.

Lws provides an API to determine if anyone is waiting for forced service,
`lws_service_adjust_timeout(context, 1, tsi)`, normally tsi is 0.  If it returns
0, then at least one connection has pending work you can get done by calling
`lws_service_tsi(context, -1, tsi)`, again normally tsi is 0.

For eg, the default poll() event loop, or libuv/ev/event, lws does this
checking for you and handles it automatically.  But in the external polling
loop case, you must do it explicitly.  Handling it after every normal service
triggered by the external poll fd should be enough, since the situations needing
it are initially triggered by actual network events.

An example of handling it is shown in the test-server code specific to
external polling.

@section cpp Using with in c++ apps

The library is ready for use by C++ apps.  You can get started quickly by
copying the test server

```
	$ cp test-apps/test-server.c test.cpp
```

and building it in C++ like this

```
	$ g++ -DINSTALL_DATADIR=\"/usr/share\" -ocpptest test.cpp -lwebsockets
```

`INSTALL_DATADIR` is only needed because the test server uses it as shipped, if
you remove the references to it in your app you don't need to define it on
the g++ line either.


@section headerinfo Availability of header information

HTTP Header information is managed by a pool of "ah" structs.  These are a
limited resource so there is pressure to free the headers and return the ah to
the pool for reuse.

For that reason header information on HTTP connections that get upgraded to
websockets is lost after the ESTABLISHED callback.  Anything important that
isn't processed by user code before then should be copied out for later.

For HTTP connections that don't upgrade, header info remains available the
whole time.

@section http2compat Code Requirements for HTTP/2 compatibility

Websocket connections only work over http/1, so there is nothing special to do
when you want to enable -DLWS_WITH_HTTP2=1.

The internal http apis already follow these requirements and are compatible with
http/2 already.  So if you use stuff like mounts and serve stuff out of the
filesystem, there's also nothing special to do.

However if you are getting your hands dirty with writing response headers, or
writing bulk data over http/2, you need to observe these rules so that it will
work over both http/1.x and http/2 the same.

1) LWS_PRE requirement applies on ALL lws_write().  For http/1, you don't have
to take care of LWS_PRE for http data, since it is just sent straight out.
For http/2, it will write up to LWS_PRE bytes behind the buffer start to create
the http/2 frame header.

This has implications if you treated the input buffer to lws_write() as const...
it isn't any more with http/2, up to 9 bytes behind the buffer will be trashed.

2) Headers are encoded using a sophisticated scheme in http/2.  The existing
header access apis are already made compatible for incoming headers,
for outgoing headers you must:

 - observe the LWS_PRE buffer requirement mentioned above
 
 - Use `lws_add_http_header_status()` to add the transaction status (200 etc)
 
 - use lws apis `lws_add_http_header_by_name()` and `lws_add_http_header_by_token()`
   to put the headers into the buffer (these will translate what is actually
   written to the buffer depending on if the connection is in http/2 mode or not)
   
 - use the `lws api lws_finalize_http_header()` api after adding the last
   response header
   
 - write the header using lws_write(..., `LWS_WRITE_HTTP_HEADERS`);
 
 3) http/2 introduces per-stream transmit credit... how much more you can send
 on a stream is decided by the peer.  You start off with some amount, as the
 stream sends stuff lws will reduce your credit accordingly, when it reaches
 zero, you must not send anything further until lws receives "more credit" for
 that stream the peer.  Lws will suppress writable callbacks if you hit 0 until
 more credit for the stream appears, and lws built-in file serving (via mounts
 etc) already takes care of observing the tx credit restrictions.  However if
 you write your own code that wants to send http data, you must consult the
 `lws_get_peer_write_allowance()` api to find out the state of your tx credit.
 For http/1, it will always return (size_t)-1, ie, no limit.
 
 This is orthogonal to the question of how much space your local side's kernel
 will make to buffer your send data on that connection.  So although the result
 from `lws_get_peer_write_allowance()` is "how much you can send" logically,
 and may be megabytes if the peer allows it, you should restrict what you send
 at one time to whatever your machine will generally accept in one go, and
 further reduce that amount if `lws_get_peer_write_allowance()` returns
 something smaller.  If it returns 0, you should not consume or send anything
 and return having asked for callback on writable, it will only come back when
 more tx credit has arrived for your stream.
 
 4) Header names with captital letters are illegal in http/2.  Header names in
 http/1 are case insensitive.  So if you generate headers by name, change all
 your header name strings to lower-case to be compatible both ways.
 
 5) Chunked Transfer-encoding is illegal in http/2, http/2 peers will actively
 reject it.  Lws takes care of removing the header and converting CGIs that
 emit chunked into unchunked automatically for http/2 connections.
 
If you follow these rules, your code will automatically work with both http/1.x
and http/2.

@section ka TCP Keepalive

It is possible for a connection which is not being used to send to die
silently somewhere between the peer and the side not sending.  In this case
by default TCP will just not report anything and you will never get any more
incoming data or sign the link is dead until you try to send.

To deal with getting a notification of that situation, you can choose to
enable TCP keepalives on all **libwebsockets** sockets, when you create the
context.

To enable keepalive, set the ka_time member of the context creation parameter
struct to a nonzero value (in seconds) at context creation time.  You should
also fill ka_probes and ka_interval in that case.

With keepalive enabled, the TCP layer will send control packets that should
stimulate a response from the peer without affecting link traffic.  If the
response is not coming, the socket will announce an error at `poll()` forcing
a close.

Note that BSDs don't support keepalive time / probes / interval per-socket
like Linux does.  On those systems you can enable keepalive by a nonzero
value in `ka_time`, but the systemwide kernel settings for the time / probes/
interval are used, regardless of what nonzero value is in `ka_time`.


@section sslopt Optimizing SSL connections

There's a member `ssl_cipher_list` in the `lws_context_creation_info` struct
which allows the user code to restrict the possible cipher selection at
context-creation time.

You might want to look into that to stop the ssl peers selecting a cipher which
is too computationally expensive.  To use it, point it to a string like

	`"RC4-MD5:RC4-SHA:AES128-SHA:AES256-SHA:HIGH:!DSS:!aNULL"`

if left `NULL`, then the "DEFAULT" set of ciphers are all possible to select.

You can also set it to `"ALL"` to allow everything (including insecure ciphers).


@section sslcerts Passing your own cert information direct to SSL_CTX

For most users it's enough to pass the SSL certificate and key information by
giving filepaths to the info.ssl_cert_filepath and info.ssl_private_key_filepath
members when creating the vhost.

If you want to control that from your own code instead, you can do so by leaving
the related info members NULL, and setting the info.options flag
LWS_SERVER_OPTION_CREATE_VHOST_SSL_CTX at vhost creation time.  That will create
the vhost SSL_CTX without any certificate, and allow you to use the callback
LWS_CALLBACK_OPENSSL_LOAD_EXTRA_SERVER_VERIFY_CERTS to add your certificate to
the SSL_CTX directly.  The vhost SSL_CTX * is in the user parameter in that
callback.

@section clientasync Async nature of client connections

When you call `lws_client_connect_info(..)` and get a `wsi` back, it does not
mean your connection is active.  It just means it started trying to connect.

Your client connection is actually active only when you receive
`LWS_CALLBACK_CLIENT_ESTABLISHED` for it.

There's a 5 second timeout for the connection, and it may give up or die for
other reasons, if any of that happens you'll get a
`LWS_CALLBACK_CLIENT_CONNECTION_ERROR` callback on protocol 0 instead for the
`wsi`.

After attempting the connection and getting back a non-`NULL` `wsi` you should
loop calling `lws_service()` until one of the above callbacks occurs.

As usual, see [test-client.c](../test-apps/test-client.c) for example code.

Notice that the client connection api tries to progress the connection
somewhat before returning.  That means it's possible to get callbacks like
CONNECTION_ERROR on the new connection before your user code had a chance to
get the wsi returned to identify it (in fact if the connection did fail early,
NULL will be returned instead of the wsi anyway).

To avoid that problem, you can fill in `pwsi` in the client connection info
struct to point to a struct lws that get filled in early by the client
connection api with the related wsi.  You can then check for that in the
callback to confirm the identity of the failing client connection.


@section fileapi Lws platform-independent file access apis

lws now exposes his internal platform file abstraction in a way that can be
both used by user code to make it platform-agnostic, and be overridden or
subclassed by user code.  This allows things like handling the URI "directory
space" as a virtual filesystem that may or may not be backed by a regular
filesystem.  One example use is serving files from inside large compressed
archive storage without having to unpack anything except the file being
requested.

The test server shows how to use it, basically the platform-specific part of
lws prepares a file operations structure that lives in the lws context.

The user code can get a pointer to the file operations struct

```
	LWS_VISIBLE LWS_EXTERN struct lws_plat_file_ops *
		`lws_get_fops`(struct lws_context *context);
```

and then can use helpers to also leverage these platform-independent
file handling apis

```
	lws_fop_fd_t
	`lws_plat_file_open`(struct lws_plat_file_ops *fops, const char *filename,
			   lws_fop_flags_t *flags)
	int
	`lws_plat_file_close`(lws_fop_fd_t fop_fd)

	unsigned long
	`lws_plat_file_seek_cur`(lws_fop_fd_t fop_fd, lws_fileofs_t offset)

	int
	`lws_plat_file_read`(lws_fop_fd_t fop_fd, lws_filepos_t *amount,
		   uint8_t *buf, lws_filepos_t len)

	int
	`lws_plat_file_write`(lws_fop_fd_t fop_fd, lws_filepos_t *amount,
		   uint8_t *buf, lws_filepos_t len )
```

Generic helpers are provided which provide access to generic fops information or
call through to the above fops

```
lws_filepos_t
lws_vfs_tell(lws_fop_fd_t fop_fd);

lws_filepos_t
lws_vfs_get_length(lws_fop_fd_t fop_fd);

uint32_t
lws_vfs_get_mod_time(lws_fop_fd_t fop_fd);

lws_fileofs_t
lws_vfs_file_seek_set(lws_fop_fd_t fop_fd, lws_fileofs_t offset);

lws_fileofs_t
lws_vfs_file_seek_end(lws_fop_fd_t fop_fd, lws_fileofs_t offset);
```


The user code can also override or subclass the file operations, to either
wrap or replace them.  An example is shown in test server.

### Changes from v2.1 and before fops

There are several changes:

1) Pre-2.2 fops directly used platform file descriptors.  Current fops returns and accepts a wrapper type lws_fop_fd_t which is a pointer to a malloc'd struct containing information specific to the filesystem implementation.

2) Pre-2.2 fops bound the fops to a wsi.  This is completely removed, you just give a pointer to the fops struct that applies to this file when you open it.  Afterwards, the operations in the fops just need the lws_fop_fd_t returned from the open.

3) Everything is wrapped in typedefs.  See lws-plat-unix.c for examples of how to implement.

4) Position in the file, File Length, and a copy of Flags left after open are now generically held in the fop_fd.
VFS implementation must set and manage this generic information now.  See the implementations in lws-plat-unix.c for
examples.

5) The file length is no longer set at a pointer provided by the open() fop.  The api `lws_vfs_get_length()` is provided to
get the file length after open.

6) If your file namespace is virtual, ie, is not reachable by platform fops directly, you must set LWS_FOP_FLAG_VIRTUAL
on the flags during open.

7) There is an optional `mod_time` uint32_t member in the generic fop_fd.  If you are able to set it during open, you
should indicate it by setting `LWS_FOP_FLAG_MOD_TIME_VALID` on the flags.

@section rawfd RAW file descriptor polling

LWS allows you to include generic platform file descriptors in the lws service / poll / event loop.

Open your fd normally and then

```
	lws_sock_file_fd_type u;

	u.filefd = your_open_file_fd;

	if (!lws_adopt_descriptor_vhost(vhost, 0, u,
					"protocol-name-to-bind-to",
					optional_wsi_parent_or_NULL)) {
		// failed
	}

	// OK
```

A wsi is created for the file fd that acts like other wsi, you will get these
callbacks on the named protocol

```
	LWS_CALLBACK_RAW_ADOPT_FILE
	LWS_CALLBACK_RAW_RX_FILE
	LWS_CALLBACK_RAW_WRITEABLE_FILE
	LWS_CALLBACK_RAW_CLOSE_FILE
```

starting with LWS_CALLBACK_RAW_ADOPT_FILE.

The minimal example `raw/minimal-raw-file` demonstrates how to use it.

`protocol-lws-raw-test` plugin also provides a method for testing this with
`libwebsockets-test-server-v2.0`:

The plugin creates a FIFO on your system called "/tmp/lws-test-raw"

You can feed it data through the FIFO like this

```
  $ sudo sh -c "echo hello > /tmp/lws-test-raw"
```

This plugin simply prints the data.  But it does it through the lws event
loop / service poll.

@section rawsrvsocket RAW server socket descriptor polling

You can also enable your vhost to accept RAW socket connections, in addition to
HTTP[s] and WS[s].  If the first bytes written on the connection are not a
valid HTTP method, then the connection switches to RAW mode.

This is disabled by default, you enable it by setting the `.options` flag
LWS_SERVER_OPTION_FALLBACK_TO_APPLY_LISTEN_ACCEPT_CONFIG, and setting
`.listen_accept_role` to `"raw-skt"` when creating the vhost.

RAW mode socket connections receive the following callbacks

```
	LWS_CALLBACK_RAW_ADOPT
	LWS_CALLBACK_RAW_RX
	LWS_CALLBACK_RAW_WRITEABLE
	LWS_CALLBACK_RAW_CLOSE
```

You can control which protocol on your vhost handles these RAW mode
incoming connections by setting the vhost info struct's `.listen_accept_protocol`
to the vhost protocol name to use.

`protocol-lws-raw-test` plugin provides a method for testing this with
`libwebsockets-test-server-v2.0`:

Run libwebsockets-test-server-v2.0 and connect to it by telnet, eg

```
    $ telnet 127.0.0.1 7681
```

type something that isn't a valid HTTP method and enter, before the
connection times out.  The connection will switch to RAW mode using this
protocol, and pass the unused rx as a raw RX callback.
    
The test protocol echos back what was typed on telnet to telnet.

@section rawclientsocket RAW client socket descriptor polling

You can now also open RAW socket connections in client mode.

Follow the usual method for creating a client connection, but set the
`info.method` to "RAW".  When the connection is made, the wsi will be
converted to RAW mode and operate using the same callbacks as the
server RAW sockets described above.

The libwebsockets-test-client supports this using raw:// URLS.  To
test, open a netcat listener in one window

```
 $ nc -l 9999
```

and in another window, connect to it using the test client

```
 $ libwebsockets-test-client raw://127.0.0.1:9999
```

The connection should succeed, and text typed in the netcat window (including a CRLF)
will be received in the client.

@section rawudp RAW UDP socket integration

Lws provides an api to create, optionally bind, and adopt a RAW UDP
socket (RAW here means an uninterpreted normal UDP socket, not a
"raw socket").

```
LWS_VISIBLE LWS_EXTERN struct lws *
lws_create_adopt_udp(struct lws_vhost *vhost, int port, int flags,
		     const char *protocol_name, struct lws *parent_wsi);
```

`flags` should be `LWS_CAUDP_BIND` if the socket will receive packets.

The callbacks `LWS_CALLBACK_RAW_ADOPT`, `LWS_CALLBACK_RAW_CLOSE`,
`LWS_CALLBACK_RAW_RX` and `LWS_CALLBACK_RAW_WRITEABLE` apply to the
wsi.  But UDP is different than TCP in some fundamental ways.

For receiving on a UDP connection, data becomes available at
`LWS_CALLBACK_RAW_RX` as usual, but because there is no specific
connection with UDP, it is necessary to also get the source address of
the data separately, using `struct lws_udp * lws_get_udp(wsi)`.
You should take a copy of the `struct lws_udp` itself (not the
pointer) and save it for when you want to write back to that peer.

Writing is also a bit different for UDP.  By default, the system has no
idea about the receiver state and so asking for a `callback_on_writable()`
always believes that the socket is writeable... the callback will
happen next time around the event loop.

With UDP, there is no single "connection".  You need to write with sendto() and
direct the packets to a specific destination.  To return packets to a
peer who sent something earlier and you copied his `struct lws_udp`, you
use the .sa and .salen members as the last two parameters of the sendto().

The kernel may not accept to buffer / write everything you wanted to send.
So you are responsible to watch the result of sendto() and resend the
unsent part next time (which may involve adding new protocol headers to
the remainder depending on what you are doing).

@section ecdh ECDH Support

ECDH Certs are now supported.  Enable the CMake option

	cmake .. -DLWS_SSL_SERVER_WITH_ECDH_CERT=1 

**and** the info->options flag

	LWS_SERVER_OPTION_SSL_ECDH

to build in support and select it at runtime.

@section sslinfo SSL info callbacks

OpenSSL allows you to receive callbacks for various events defined in a
bitmask in openssl/ssl.h.  The events include stuff like TLS Alerts.

By default, lws doesn't register for these callbacks.

However if you set the info.ssl_info_event_mask to nonzero (ie, set some
of the bits in it like `SSL_CB_ALERT` at vhost creation time, then
connections to that vhost will call back using LWS_CALLBACK_SSL_INFO
for the wsi, and the `in` parameter will be pointing to a struct of
related args:

```
struct lws_ssl_info {
	int where;
	int ret;
};
```

The default callback handler in lws has a handler for LWS_CALLBACK_SSL_INFO
which prints the related information,  You can test it using the switch
-S -s  on `libwebsockets-test-server-v2.0`.

Returning nonzero from the callback will close the wsi.

@section smp SMP / Multithreaded service

SMP support is integrated into LWS without any internal threading.  It's
very simple to use, libwebsockets-test-server-pthread shows how to do it,
use -j n argument there to control the number of service threads up to 32.

Two new members are added to the info struct

	unsigned int count_threads;
	unsigned int fd_limit_per_thread;
	
leave them at the default 0 to get the normal singlethreaded service loop.

Set count_threads to n to tell lws you will have n simultaneous service threads
operating on the context.

There is still a single listen socket on one port, no matter how many
service threads.

When a connection is made, it is accepted by the service thread with the least
connections active to perform load balancing.

The user code is responsible for spawning n threads running the service loop
associated to a specific tsi (Thread Service Index, 0 .. n - 1).  See
the libwebsockets-test-server-pthread for how to do.

If you leave fd_limit_per_thread at 0, then the process limit of fds is shared
between the service threads; if you process was allowed 1024 fds overall then
each thread is limited to 1024 / n.

You can set fd_limit_per_thread to a nonzero number to control this manually, eg
the overall supported fd limit is less than the process allowance.

You can control the context basic data allocation for multithreading from Cmake
using -DLWS_MAX_SMP=, if not given it's set to 1.  The serv_buf allocation
for the threads (currently 4096) is made at runtime only for active threads.

Because lws will limit the requested number of actual threads supported
according to LWS_MAX_SMP, there is an api lws_get_count_threads(context) to
discover how many threads were actually allowed when the context was created.

See the test-server-pthreads.c sample for how to use.

@section smplocking SMP Locking Helpers

Lws provide a set of pthread mutex helpers that reduce to no code or
variable footprint in the case that LWS_MAX_SMP == 1.

Define your user mutex like this

```
	lws_pthread_mutex(name);
```

If LWS_MAX_SMP > 1, this produces `pthread_mutex_t name;`.  In the case
LWS_MAX_SMP == 1, it produces nothing.

Likewise these helpers for init, destroy, lock and unlock


```
	void lws_pthread_mutex_init(pthread_mutex_t *lock)
	void lws_pthread_mutex_destroy(pthread_mutex_t *lock)
	void lws_pthread_mutex_lock(pthread_mutex_t *lock)
	void lws_pthread_mutex_unlock(pthread_mutex_t *lock)
```

resolve to nothing if LWS_MAX_SMP == 1, otherwise produce the equivalent
pthread api.

pthreads is required in lws only if LWS_MAX_SMP > 1.


@section libevuv libev / libuv / libevent support

You can select either or both

	-DLWS_WITH_LIBEV=1
	-DLWS_WITH_LIBUV=1
	-DLWS_WITH_LIBEVENT=1

at cmake configure-time.  The user application may use one of the
context init options flags

	LWS_SERVER_OPTION_LIBEV
	LWS_SERVER_OPTION_LIBUV
	LWS_SERVER_OPTION_LIBEVENT

to indicate it will use one of the event libraries at runtime.

libev and libevent headers conflict, they both define critical constants like
EV_READ to different values.  Attempts to discuss clearing that up with both
libevent and libev did not get anywhere useful.  Therefore CMakeLists.txt will
error out if you enable both LWS_WITH_LIBEV and LWS_WITH_LIBEVENT.

In addition depending on libev / compiler version, building anything with libev
apis using gcc may blow strict alias warnings (which are elevated to errors in
lws).  I did some googling at found these threads related to it, the issue goes
back at least to 2010 on and off

https://github.com/redis/hiredis/issues/434
https://bugs.gentoo.org/show_bug.cgi?id=615532
http://lists.schmorp.de/pipermail/libev/2010q1/000916.html
http://lists.schmorp.de/pipermail/libev/2010q1/000920.html
http://lists.schmorp.de/pipermail/libev/2010q1/000923.html

We worked around this problem by disabling -Werror on the parts of lws that
use libev.  FWIW as of Dec 2019 using Fedora 31 libev 4.27.1 and its gcc 9.2.1
doesn't seem to trigger the problem even without the workaround.

For these reasons and the response I got trying to raise these issues with
them, if you have a choice about event loop, I would gently encourage you
to avoid libev.  Where lws uses an event loop itself, eg in lwsws, we use
libuv.

@section extopts Extension option control from user code

User code may set per-connection extension options now, using a new api
`lws_set_extension_option()`.

This should be called from the ESTABLISHED callback like this
```
	 lws_set_extension_option(wsi, "permessage-deflate",
	                          "rx_buf_size", "12"); /* 1 << 12 */
```

If the extension is not active (missing or not negotiated for the
connection, or extensions are disabled on the library) the call is
just returns -1.  Otherwise the connection's extension has its
named option changed.

The extension may decide to alter or disallow the change, in the
example above permessage-deflate restricts the size of his rx
output buffer also considering the protocol's rx_buf_size member.


@section httpsclient Client connections as HTTP[S] rather than WS[S]

You may open a generic http client connection using the same
struct lws_client_connect_info used to create client ws[s]
connections.

To stay in http[s], set the optional info member "method" to
point to the string "GET" instead of the default NULL.

After the server headers are processed, when payload from the
server is available the callback LWS_CALLBACK_RECEIVE_CLIENT_HTTP
will be made.

You can choose whether to process the data immediately, or
queue a callback when an outgoing socket is writeable to provide
flow control, and process the data in the writable callback.

Either way you use the api `lws_http_client_read()` to access the
data, eg

```
	case LWS_CALLBACK_RECEIVE_CLIENT_HTTP:
		{
			char buffer[1024 + LWS_PRE];
			char *px = buffer + LWS_PRE;
			int lenx = sizeof(buffer) - LWS_PRE;

			lwsl_notice("LWS_CALLBACK_RECEIVE_CLIENT_HTTP\n");

			/*
			 * Often you need to flow control this by something
			 * else being writable.  In that case call the api
			 * to get a callback when writable here, and do the
			 * pending client read in the writeable callback of
			 * the output.
			 */
			if (lws_http_client_read(wsi, &px, &lenx) < 0)
				return -1;
			while (lenx--)
				putchar(*px++);
		}
		break;
```

Notice that if you will use SSL client connections on a vhost, you must
prepare the client SSL context for the vhost after creating the vhost, since
this is not normally done if the vhost was set up to listen / serve.  Call
the api lws_init_vhost_client_ssl() to also allow client SSL on the vhost.

@section clipipe Pipelining Client Requests to same host

If you are opening more client requests to the same host and port, you
can give the flag LCCSCF_PIPELINE on `info.ssl_connection` to indicate
you wish to pipeline them.

Without the flag, the client connections will occur concurrently using a
socket and tls wrapper if requested for each connection individually.
That is fast, but resource-intensive.

With the flag, lws will queue subsequent client connections on the first
connection to the same host and port.  When it has confirmed from the
first connection that pipelining / keep-alive is supported by the server,
it lets the queued client pipeline connections send their headers ahead
of time to create a pipeline of requests on the server side.

In this way only one tcp connection and tls wrapper is required to transfer
all the transactions sequentially.  It takes a little longer but it
can make a significant difference to resources on both sides.

If lws learns from the first response header that keepalive is not possible,
then it marks itself with that information and detaches any queued clients
to make their own individual connections as a fallback.

Lws can also intelligently combine multiple ongoing client connections to
the same host and port into a single http/2 connection with multiple
streams if the server supports it.

Unlike http/1 pipelining, with http/2 the client connections all occur
simultaneously using h2 stream multiplexing inside the one tcp + tls
connection.

You can turn off the h2 client support either by not building lws with
`-DLWS_WITH_HTTP2=1` or giving the `LCCSCF_NOT_H2` flag in the client
connection info struct `ssl_connection` member.

@section vhosts Using lws vhosts

If you set LWS_SERVER_OPTION_EXPLICIT_VHOSTS options flag when you create
your context, it won't create a default vhost using the info struct
members for compatibility.  Instead you can call lws_create_vhost()
afterwards to attach one or more vhosts manually.

```
	LWS_VISIBLE struct lws_vhost *
	lws_create_vhost(struct lws_context *context,
			 struct lws_context_creation_info *info);
```

lws_create_vhost() uses the same info struct as lws_create_context(),
it ignores members related to context and uses the ones meaningful
for vhost (marked with VH in libwebsockets.h).

```
	struct lws_context_creation_info {
		int port;					/* VH */
		const char *iface;				/* VH */
		const struct lws_protocols *protocols;		/* VH */
		const struct lws_extension *extensions;		/* VH */
	...
```

When you attach the vhost, if the vhost's port already has a listen socket
then both vhosts share it and use SNI (is SSL in use) or the Host: header
from the client to select the right one.  Or if no other vhost already
listening the a new listen socket is created.

There are some new members but mainly it's stuff you used to set at
context creation time.


@section sni How lws matches hostname or SNI to a vhost

LWS first strips any trailing :port number.

Then it tries to find an exact name match for a vhost listening on the correct
port, ie, if SNI or the Host: header provided abc.com:1234, it will match on a
vhost named abc.com that is listening on port 1234.

If there is no exact match, lws will consider wildcard matches, for example
if cats.abc.com:1234 is provided by the client by SNI or Host: header, it will
accept a vhost "abc.com" listening on port 1234.  If there was a better, exact,
match, it will have been chosen in preference to this.

Connections with SSL will still have the client go on to check the
certificate allows wildcards and error out if not.
 


@section mounts Using lws mounts on a vhost

The last argument to lws_create_vhost() lets you associate a linked
list of lws_http_mount structures with that vhost's URL 'namespace', in
a similar way that unix lets you mount filesystems into areas of your /
filesystem how you like and deal with the contents transparently.

```
	struct lws_http_mount {
		struct lws_http_mount *mount_next;
		const char *mountpoint; /* mountpoint in http pathspace, eg, "/" */
		const char *origin; /* path to be mounted, eg, "/var/www/warmcat.com" */
		const char *def; /* default target, eg, "index.html" */
	
		struct lws_protocol_vhost_options *cgienv;
	
		int cgi_timeout;
		int cache_max_age;
	
		unsigned int cache_reusable:1;
		unsigned int cache_revalidate:1;
		unsigned int cache_intermediaries:1;
	
		unsigned char origin_protocol;
		unsigned char mountpoint_len;
	};
```

The last mount structure should have a NULL mount_next, otherwise it should
point to the 'next' mount structure in your list.

Both the mount structures and the strings must persist until the context is
destroyed, since they are not copied but used in place.

`.origin_protocol` should be one of

```
	enum {
		LWSMPRO_HTTP,
		LWSMPRO_HTTPS,
		LWSMPRO_FILE,
		LWSMPRO_CGI,
		LWSMPRO_REDIR_HTTP,
		LWSMPRO_REDIR_HTTPS,
		LWSMPRO_CALLBACK,
	};
```

 - LWSMPRO_FILE is used for mapping url namespace to a filesystem directory and
serve it automatically.

 - LWSMPRO_CGI associates the url namespace with the given CGI executable, which
runs when the URL is accessed and the output provided to the client.

 - LWSMPRO_REDIR_HTTP and LWSMPRO_REDIR_HTTPS auto-redirect clients to the given
origin URL.

 - LWSMPRO_CALLBACK causes the http connection to attach to the callback
associated with the named protocol (which may be a plugin).


@section mountcallback Operation of LWSMPRO_CALLBACK mounts

The feature provided by CALLBACK type mounts is binding a part of the URL
namespace to a named protocol callback handler.

This allows protocol plugins to handle areas of the URL namespace.  For example
in test-server-v2.0.c, the URL area "/formtest" is associated with the plugin
providing "protocol-post-demo" like this

```
	static const struct lws_http_mount mount_post = {
		NULL,		/* linked-list pointer to next*/
		"/formtest",		/* mountpoint in URL namespace on this vhost */
		"protocol-post-demo",	/* handler */
		NULL,	/* default filename if none given */
		NULL,
		0,
		0,
		0,
		0,
		0,
		LWSMPRO_CALLBACK,	/* origin points to a callback */
		9,			/* strlen("/formtest"), ie length of the mountpoint */
	};
```

Client access to /formtest[anything] will be passed to the callback registered
with the named protocol, which in this case is provided by a protocol plugin.

Access by all methods, eg, GET and POST are handled by the callback.

protocol-post-demo deals with accepting and responding to the html form that
is in the test server HTML.

When a connection accesses a URL related to a CALLBACK type mount, the
connection protocol is changed until the next access on the connection to a
URL outside the same CALLBACK mount area.  User space on the connection is
arranged to be the size of the new protocol user space allocation as given in
the protocol struct.

This allocation is only deleted / replaced when the connection accesses a
URL region with a different protocol (or the default protocols[0] if no
CALLBACK area matches it).

This "binding connection to a protocol" lifecycle in managed by
`LWS_CALLBACK_HTTP_BIND_PROTOCOL` and `LWS_CALLBACK_HTTP_DROP_PROTOCOL`.
Because of HTTP/1.1 connection pipelining, one connection may perform
many transactions, each of which may map to different URLs and need
binding to different protocols.  So these messages are used to
create the binding of the wsi to your protocol including any
allocations, and to destroy the binding, at which point you should
destroy any related allocations.

@section BINDTODEV SO_BIND_TO_DEVICE

The .bind_iface flag in the context / vhost creation struct lets you
declare that you want all traffic for listen and transport on that
vhost to be strictly bound to the network interface named in .iface.

This Linux-only feature requires SO_BIND_TO_DEVICE, which in turn
requires CAP_NET_RAW capability... root has this capability.

However this feature needs to apply the binding also to accepted
sockets during normal operation, which implies the server must run
the whole time as root.

You can avoid this by using the Linux capabilities feature to have
the unprivileged user inherit just the CAP_NET_RAW capability.

You can confirm this with the test server


```
 $ sudo /usr/local/bin/libwebsockets-test-server -u agreen -i eno1 -k
```

The part that ensures the capability is inherited by the unprivileged
user is

```
#if defined(LWS_HAVE_SYS_CAPABILITY_H) && defined(LWS_HAVE_LIBCAP)
                        info.caps[0] = CAP_NET_RAW;
                        info.count_caps = 1;
#endif
```


@section dim Dimming webpage when connection lost

The lws test plugins' html provides useful feedback on the webpage about if it
is still connected to the server, by greying out the page if not.  You can
also add this to your own html easily

 - include lws-common.js from your HEAD section
 
   \<script src="/lws-common.js">\</script>
   
 - dim the page during initialization, in a script section on your page
 
   lws_gray_out(true,{'zindex':'499'});
   
 - in your ws onOpen(), remove the dimming
 
   lws_gray_out(false);
   
 - in your ws onClose(), reapply the dimming
 
   lws_gray_out(true,{'zindex':'499'});

@section errstyle Styling http error pages

In the code, http errors should be handled by `lws_return_http_status()`.

There are basically two ways... the vhost can be told to redirect to an "error
page" URL in response to specifically a 404... this is controlled by the
context / vhost info struct (`struct lws_context_creation_info`) member
`.error_document_404`... if non-null the client is redirected to this string.

If it wasn't redirected, then the response code html is synthesized containing
the user-selected text message and attempts to pull in `/error.css` for styling.

If this file exists, it can be used to style the error page.  See 
https://libwebsockets.org/git/badrepo for an example of what can be done (
and https://libwebsockets.org/error.css for the corresponding css).

## Using Content Security Policy (CSP)

### What is it?

Modern browsers have recently implemented a new feature providing
a sort of "selinux for your web page".  If the server sends some
new headers describing the security policy for the content, then
the browser strictly enforces it.

### Why would we want to do that?

Scripting on webpages is pretty universal, sometimes the scripts
come from third parties, and sometimes attackers find a way to
inject scripts into the DOM, eg, through scripts in content.

CSP lets the origin server define what is legitimate for the page it
served and everything else is denied.

The CSP for warmcat.com and libwebsockets.org looks like this,
I removed a handful of approved image sources like travis
status etc for clarity...

```
"content-security-policy": "default-src 'none'; img-src 'self' data:; script-src 'self'; font-src 'self'; style-src 'self'; connect-src 'self'; frame-ancestors 'none'; base-uri 'none';",
"x-content-type-options": "nosniff",
"x-xss-protection": "1; mode=block",
"x-frame-options": "deny",
"referrer-policy": "no-referrer"
```

The result of this is the browser won't let the site content be iframed, and it
will reject any inline styles or inline scripts.  Fonts, css, ajax, ws and
images are only allowed to come from 'self', ie, the server that served the
page.  You may inject your script, or deceptive styles: it won't run or be shown.

Because inline scripts are banned, the usual methods for XSS are dead;
the attacker can't even load js from another server.  So these rules
provide a very significant increase in client security.

### Implications of strict CSP

Halfhearted CSP isn't worth much.  The only useful approach is to start
with `default-src 'none'` which disables everything, and then allow the
minimum needed for the pages to operate.

"Minimum needed for the pages to operate" doesn't mean defeat the protections
necessary so everything in the HTML can stay the same... it means adapt the
pages to want the minimum and then enable the minimum.

The main point is segregation of styles and script away from the content, in
files referenced in the document `<head>` section, along these lines:

```
<head>
 <meta charset=utf-8 http-equiv="Content-Language" content="en"/>
 <link rel="stylesheet" type="text/css" href="test.css"/>
 <script type='text/javascript' src="/lws-common.js"></script>
 <script type='text/javascript' src='test.js'></script>
 <title>Minimal Websocket test app</title>
</head>
```

#### Inline styles must die

All styling must go in one or more `.css` file(s) best served by the same
server... while you can approve other sources in the CSP if you have to,
unless you control that server as well, you are allowing whoever gains
access to that server access to your users.

Inline styles are no longer allowed (eg, "style='font-size:120%'" in the
HTML)... they must be replaced by reference to one or more CSS class, which
in this case includes "font-size:120%".  This has always been the best
practice anyway, and your pages will be cleaner and more maintainable.

#### Inline scripts must die

Inline scripts need to be placed in a `.js` file and loaded in the page head
section, again it should only be from the server that provided the page.

Then, any kind of inline script, yours or injected or whatever, will be
completely rejected by the browser.

#### onXXX must be replaced by eventListener

Inline `onclick()` etc are kinds of inline scripting and are banned.

Modern browsers have offered a different system called ["EventListener" for
a while](https://developer.mozilla.org/en-US/docs/Web/API/EventListener)
which allows binding of events to DOM elements in JS.

A bunch of different named events are possible to listen on, commonly the
`.js` file will ask for one or both of

```
window.addEventListener("load", function() {
...
}, false);

document.addEventListener("DOMContentLoaded", function() {
...
}, false);
```

These give the JS a way to trigger when either everything on the page has
been "loaded" or the DOM has been populated from the initial HTML.  These
can set up other event listeners on the DOM objects and aftwards the
events will drive what happens on the page from user interaction and / or
timers etc.

If you have `onclick` in your HTML today, you would replace it with an id
for the HTML element, then eg in the DOMContentLoaded event listener,
apply 

```
   document.getElementById("my-id").addEventListener("click", function() {
   ...
   }, false);
```

ie the .js file becomes the only place with the "business logic" of the
elements mentioned in the HTML, applied at runtime.

#### Do you really need external sources?

Do your scripts and fonts really need to come from external sources?
If your caching policy is liberal, they are not actually that expensive
to serve once and then the user is using his local copy for the next
days.

Some external sources are marked as anti-privacy in modern browsers, meaning
they track your users, in turn meaning if your site refers to them, you
will lose your green padlock in the browser.  If the content license allows
it, hosting them on "self", ie, the same server that provided the HTML,
will remove that problem.

Bringing in scripts from external sources is actually quite scary from the
security perspective.  If someone hacks the `ajax.googleapis.com` site to serve
a hostile, modified jquery, half the Internet will instantly
become malicious.  However if you serve it yourself, unless your server
was specifically targeted you know it will continue to serve what you
expect.

Since these scripts are usually sent with cache control headers for local
caching duration of 1 year, the cost of serving them yourself under the same
conditions is small but your susceptibility to attack is reduced to only taking
care of your own server.  And there is a privacy benefit that google is not
informed of your users' IPs and activities on your site.

## Contributing to lws

### How to contribute

Sending a patch with a bug report is very welcome.

For nontrivial problems, it's probably best to discuss on the mailing list,
or on github if you prefer, how to best solve it.

However your contribution is coming is fine:

 - paste a `git diff`

 - send a patch series by mail or mailing list

 - paste in a github issue

 - github PR

are all OK.

### Coding Standards

Code should look roughly like the existing code, which follows linux kernel
coding style.

If there are non-functional problems I will clean them out when I apply the
patch.

If there are functional problems (eg broken error paths etc) if they are
small compared to the working part I will also clean them.  If there are
larger problems, or consequences to the patch will have to discuss how to
solve them with a retry.

### Funding specific work

If there is a feature you wish was supported in lws, consider paying for the
work to be done.  The maintainer is a consultant and if we can agree the
task, you can quickly get a high quality result that does just what you need,
maintained ongoing along with the rest of lws.

# Lws Crypto Apis

## Overview

![lws crypto overview](/doc-assets/lws-crypto-overview.svg)

Lws provides a "generic" crypto layer on top of both OpenSSL and
compatible tls library, and mbedtls.  Using this layer, your code
can work without any changes on both types of tls library crypto
backends... it's as simple as rebuilding lws with `-DLWS_WITH_MBEDTLS=0`
or `=1` at cmake.

The generic layer can be used directly (as in, eg, the sshd plugin),
or via another layer on top, which processes JOSE JSON objects using
JWS (JSON Web Signatures), JWK (JSON Web Keys), and JWE (JSON Web
Encryption).

The `JW` apis use the generic apis (`lws_genrsa_`, etc) to get the crypto tasks
done, so anything they can do you can also get done using the generic apis.
The main difference is that with the generic apis, you must instantiate the
correct types and use type-specfic apis.  With the `JW` apis, there is only
one interface for all operations, with the details hidden in the api and
controlled by the JSON objects.

Because of this, the `JW` apis are often preferred because they give you
"crypto agility" cheaply... to change your crypto to another supported algorithm
once it's working, you literally just change your JSON defining the keys and
JWE or JWS algorithm.  (It's up to you to define your policy for which
combinations are acceptable by querying the parsed JW structs).

## Crypto supported in generic layer

### Generic Hash

 - SHA1
 - SHA256
 - SHA384
 - SHA512

### Generic HMAC

 - SHA256
 - SHA384
 - SHA512

### Generic AES

 - CBC
 - CFB128
 - CFB8
 - CTR
 - ECB
 - OFB
 - XTS
 - GCM
 - KW (Key Wrap)

### Generic RSA

 - PKCS 1.5
 - OAEP / PSS

### Generic EC

 - ECDH
 - ECDSA
 - P256 / P384 / P521 (sic) curves

## Using the generic layer

All the necessary includes are part of `libwebsockets.h`.

Enable `-DLWS_WITH_GENCRYPTO=1` at cmake.

|api|header|Functionality|
|---|---|---|
|genhash|[./include/libwebsockets/lws-genhash.h](https://libwebsockets.org/git/libwebsockets/tree/include/libwebsockets/lws-genhash.h)|Provides SHA1 + SHA2 hashes and hmac|
|genrsa|[./include/libwebsockets/lws-genrsa.h](https://libwebsockets.org/git/libwebsockets/tree/include/libwebsockets/lws-genrsa.h)|Provides RSA encryption, decryption, signing, verification, key generation and creation|
|genaes|[./include/libwebsockets/lws-genaes.h](https://libwebsockets.org/git/libwebsockets/tree/include/libwebsockets/lws-genaes.h)|Provides AES in all common variants for encryption and decryption|
|genec|[./include/libwebsockets/lws-genec.h](https://libwebsockets.org/git/libwebsockets/tree/include/libwebsockets/lws-genec.h)|Provides Elliptic Curve for encryption, decryption, signing, verification, key generation and creation|
|x509|[./include/libwebsockets/lws-x509.h](https://libwebsockets.org/git/libwebsockets/tree/include/libwebsockets/lws-x509.h)|Apis for X.509 Certificate loading, parsing, and stack verification, plus JWK key extraction from PEM X.509 certificate / private key|

Unit tests for these apis, which serve as usage examples, can be found in [./minimal-examples/api-tests/api-test-gencrypto](https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/api-tests/api-test-gencrypto)

### Keys in the generic layer

The necessary types and defines are brought in by `libwebsockets.h`.

Keys are represented only by an array of `struct lws_jwk_elements`... the
length of the array is defined by the cipher... it's one of

|key elements count|definition|
|---|---|
|`LWS_COUNT_OCT_KEY_ELEMENTS`|1|
|`LWS_COUNT_RSA_KEY_ELEMENTS`|8|
|`LWS_COUNT_EC_KEY_ELEMENTS`|4|
|`LWS_COUNT_AES_KEY_ELEMENTS`|1|

`struct lws_jwk_elements` is a simple pointer / length combination used to
store arbitrary octets that make up the key element's binary representation.

## Using the JOSE layer

The JOSE (JWK / JWS / JWE) stuff is a crypto-agile JSON-based layer
that uses the gencrypto support underneath.

"Crypto Agility" means the JSON structs include information about the
algorithms and ciphers used in that particular object, making it easy to
upgrade system crypto strength or cycle keys over time while supporting a
transitional period where the old and new keys or algorithms + ciphers
are also valid.

Uniquely lws generic support means the JOSE stuff also has "tls library
agility", code written to the lws generic or JOSE apis is completely unchanged
even if the underlying tls library changes between OpenSSL and mbedtls, meaning
sharing code between server and client sides is painless.

All the necessary includes are part of `libwebsockets.h`.

Enable `-DLWS_WITH_JOSE=1` at CMake.

|api|header|Functionality|
|---|---|---|
|JOSE|[./include/libwebsockets/lws-jose.h](https://libwebsockets.org/git/libwebsockets/tree/include/libwebsockets/lws-jose.h)|Provides crypto agility for JWS / JWE|
|JWE|[./include/libwebsockets/lws-jwe.h](https://libwebsockets.org/git/libwebsockets/tree/include/libwebsockets/lws-jwe.h)|Provides Encryption and Decryption services for RFC7516 JWE JSON|
|JWS|[./include/libwebsockets/lws-jws.h](https://libwebsockets.org/git/libwebsockets/tree/include/libwebsockets/lws-jws.h)|Provides signature and verifcation services for RFC7515 JWS JSON|
|JWK|[./include/libwebsockets/lws-jwk.h](https://libwebsockets.org/git/libwebsockets/tree/include/libwebsockets/lws-jwk.h)|Provides signature and verifcation services for RFC7517 JWK JSON, both "keys" arrays and singletons|

Minimal examples are provided in the form of commandline tools for JWK / JWS / JWE / x509 handling:

 - [JWK minimal example](https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/crypto/minimal-crypto-jwk)
 - [JWS minimal example](https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/crypto/minimal-crypto-jws)
 - [JWE minimal example](https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/crypto/minimal-crypto-jwe)
 - [X509 minimal example](https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/crypto/minimal-crypto-x509)

Unit tests for these apis, which serve as usage examples, can be found in [./minimal-examples/api-tests/api-test-jose](https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/api-tests/api-test-jose)

## Crypto supported in the JOSE layer

The JOSE RFCs define specific short names for different algorithms

### JWS

|JSOE name|Hash|Signature|
---|---|---
|RS256, RS384, RS512|SHA256/384/512|RSA
|ES256, ES384, ES521|SHA256/384/512|EC

### JWE

|Key Encryption|Payload authentication + crypt|
|---|---|
|`RSAES-PKCS1-v1.5` 2048b & 4096b|`AES_128_CBC_HMAC_SHA_256`|
|`RSAES-PKCS1-v1.5` 2048b|`AES_192_CBC_HMAC_SHA_384`|
|`RSAES-PKCS1-v1.5` 2048b|`AES_256_CBC_HMAC_SHA_512`|
|`RSAES-OAEP`|`AES_256_GCM`|
|`AES128KW`, `AES192KW`, `AES256KW`|`AES_128_CBC_HMAC_SHA_256`|
|`AES128KW`, `AES192KW`, `AES256KW`|`AES_192_CBC_HMAC_SHA_384`|
|`AES128KW`, `AES192KW`, `AES256KW`|`AES_256_CBC_HMAC_SHA_512`|
|`ECDH-ES` (P-256/384/521 key)|`AES_128/192/256_GCM`|
|`ECDH-ES+A128/192/256KW` (P-256/384/521 key)|`AES_128/192/256_GCM`|

### Keys in the JOSE layer

Keys in the JOSE layer use a `struct lws_jwk`, this contains two arrays of
`struct lws_jwk_elements` sized for the worst case (currently RSA).  One
array contains the key elements as described for the generic case, and the
other contains various nonencrypted key metadata taken from JWK JSON.

|metadata index|function|
|---|---|
|`JWK_META_KTY`|Key type, eg, "EC"|
|`JWK_META_KID`|Arbitrary ID string|
|`JWK_META_USE`|What the public key may be used to validate, "enc" or "sig"|
|`JWK_META_KEY_OPS`|Which operations the key is authorized for, eg, "encrypt"|
|`JWK_META_X5C`|Optional X.509 cert version of the key|
|`JWK_META_ALG`|Optional overall crypto algorithm the key is intended for use with|

`lws_jwk_destroy()` should be called when the jwk is going out of scope... this
takes care to zero down any key element data in the jwk.

## Using CTest with lws

### Updating ancient cmake

You need a recent cmake to have the CTest tests work properly, if you're on an
older distro you need to update your cmake.  Luckily Kitware provide a repo for
common distros.  These instructions work for bionic and xenial.

First remove the old distro cmake and install the pieces needed to get the new repo keys

```
# apt purge --auto-remove cmake
# apt install gnupg wget apt-transport-https ca-certificates
# wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null | sudo apt-key add -
# apt edit-sources
```

Add the line `deb https://apt.kitware.com/ubuntu/ bionic main` at the end
replacing `bionic` with `xenial` as needed, and save (:wq).  Then

```
# apt update
# apt install cmake
```

## Tests live in CMakeLists.txt

The rules for tests are described in ctest / cmake language inside the minimal
examples and api tests that are enabled by current build options, so you need
to build with `-DLWS_WITH_MINIMAL_EXAMPLES=1` to build the examples along with
the library.

The tests are typically running the examples or api tests and regarding the
process exiting with exit code 0 as success, anything else as failure.

## Generating the tests

The main tests just need `-DLWS_WITH_MINIMAL_EXAMPLES=1`.  You can optionally set
`-DLWS_CTEST_INTERNET_AVAILABLE=0` to indicate you can't run the tests that need
internet connectivity.

## Preparing to run the tests

The tests have to be able to run without root and without disturbing any other
install of lws in the build machine.

For that reason you have to do an unprivileged side-install into `../destdir`,
using `make install DESTDIR=../destdir` from the build directory and perform the
tests on the pieces in there.

## Running the tests

We must take care to run the pieces (.so etc) we just built, without having
root access, and not any of the same pieces from some other lws version that may
have been installed on the build machine.  That includes, eg, plugins that
we just built, to ensure precedence of those in the search path we can set our
DESTDIR unprivileged install path in `LD_LIBRARY_PATH`.

Then we can run ctest on the unprivileged install.  The whole step looks
something like this:

```
build $ make -j12 && \
  rm -rf ../destdir && \
  make -j12 DESTDIR=../destdir install && \\
  LD_LIBRARY_PATH=../destdir/usr/local/share/libwebsockets-test-server/plugins ctest -j2 --output-on-failure
```  

On windows, it looks like `ctest . -C DEBUG` or RELEASE if that was the build
type.

Good results look something like this (which tests can run depend on your
build options)

```
Test project /projects/libwebsockets/build
      Start 71: st_wcs_srv
      Start 43: st_hcp_srv
 1/73 Test #71: st_wcs_srv ..................................   Passed    5.01 sec
      Start 19: st_hcmp_srv
 2/73 Test #43: st_hcp_srv ..................................   Passed    5.01 sec
      Start 17: st_hcm_srv
 3/73 Test #19: st_hcmp_srv .................................   Passed    5.01 sec
      Start 55: st_ssproxyctx
 4/73 Test #17: st_hcm_srv ..................................   Passed    5.01 sec
      Start 52: st_ssproxy
 5/73 Test #55: st_ssproxyctx ...............................   Passed    1.02 sec
      Start 67: st_sstfproxy
 6/73 Test #52: st_ssproxy ..................................   Passed    1.02 sec
      Start 60: st_ssprxsmd_sspc
 7/73 Test #67: st_sstfproxy ................................   Passed    1.01 sec
      Start 63: st_mulssprxsmd_sspc
 8/73 Test #60: st_ssprxsmd_sspc ............................   Passed    1.01 sec
      Start 69: sspc-minimaltf
 9/73 Test #63: st_mulssprxsmd_sspc .........................   Passed    1.02 sec
      Start 73: ws-client-spam
10/73 Test #73: ws-client-spam ..............................   Passed   12.21 sec
      Start 57: sspc-minimaltx
11/73 Test #57: sspc-minimaltx ..............................   Passed    5.90 sec
      Start 65: mulsspcsmd_sspc
12/73 Test #65: mulsspcsmd_sspc .............................   Passed    3.58 sec
      Start 62: sspcsmd_sspc
13/73 Test #62: sspcsmd_sspc ................................   Passed    1.73 sec
      Start 22: http-client-multi-h1
14/73 Test #22: http-client-multi-h1 ........................   Passed    5.04 sec
      Start 25: http-client-multi-stag
15/73 Test #25: http-client-multi-stag ......................   Passed    4.53 sec
      Start 26: http-client-multi-stag-h1
16/73 Test #26: http-client-multi-stag-h1 ...................   Passed    4.40 sec
      Start 21: http-client-multi
17/73 Test #21: http-client-multi ...........................   Passed    4.37 sec
      Start 36: http-client-multi-post-h1
18/73 Test #36: http-client-multi-post-h1 ...................   Passed    2.73 sec
      Start 54: sspc-minimal
19/73 Test #54: sspc-minimal ................................   Passed    0.93 sec
      Start 39: http-client-multi-post-stag
20/73 Test #39: http-client-multi-post-stag .................   Passed    2.29 sec
      Start 40: http-client-multi-post-stag-h1
21/73 Test #69: sspc-minimaltf ..............................   Passed   49.83 sec
      Start 35: http-client-multi-post
22/73 Test #40: http-client-multi-post-stag-h1 ..............   Passed    4.30 sec
      Start 33: http-client-multi-restrict-nopipe-fail
23/73 Test #35: http-client-multi-post ......................   Passed    3.23 sec
      Start 28: http-client-multi-stag-h1-pipe
24/73 Test #33: http-client-multi-restrict-nopipe-fail ......   Passed    2.86 sec
      Start 32: http-client-multi-restrict-stag-h1-pipe
25/73 Test #28: http-client-multi-stag-h1-pipe ..............   Passed    2.86 sec
      Start 27: http-client-multi-stag-pipe
26/73 Test #32: http-client-multi-restrict-stag-h1-pipe .....   Passed    1.51 sec
      Start 31: http-client-multi-restrict-stag-pipe
27/73 Test #27: http-client-multi-stag-pipe .................   Passed    1.52 sec
      Start 34: http-client-multi-restrict-h1-nopipe-fail
28/73 Test #34: http-client-multi-restrict-h1-nopipe-fail ...   Passed    2.78 sec
      Start 46: http-client-post-m
29/73 Test #31: http-client-multi-restrict-stag-pipe ........   Passed    2.80 sec
      Start 42: http-client-multi-post-stag-h1-pipe
30/73 Test #42: http-client-multi-post-stag-h1-pipe .........   Passed    1.51 sec
      Start 41: http-client-multi-post-stag-pipe
31/73 Test #46: http-client-post-m ..........................   Passed    1.59 sec
      Start 48: http-client-post-m-h1
32/73 Test #48: http-client-post-m-h1 .......................   Passed    1.10 sec
      Start 23: http-client-multi-pipe
33/73 Test #41: http-client-multi-post-stag-pipe ............   Passed    1.51 sec
      Start 29: http-client-multi-restrict-pipe
34/73 Test #23: http-client-multi-pipe ......................   Passed    1.09 sec
      Start 24: http-client-multi-h1-pipe
35/73 Test #29: http-client-multi-restrict-pipe .............   Passed    0.74 sec
      Start 30: http-client-multi-restrict-h1-pipe
36/73 Test #24: http-client-multi-h1-pipe ...................   Passed    1.14 sec
      Start 45: http-client-post
37/73 Test #30: http-client-multi-restrict-h1-pipe ..........   Passed    1.14 sec
      Start 38: http-client-multi-post-h1-pipe
38/73 Test #45: http-client-post ............................   Passed    0.30 sec
      Start 37: http-client-multi-post-pipe
39/73 Test #38: http-client-multi-post-h1-pipe ..............   Passed    0.49 sec
      Start 47: http-client-post-h1
40/73 Test #37: http-client-multi-post-pipe .................   Passed    0.31 sec
      Start 50: hs_evlib_foreign_event
41/73 Test #47: http-client-post-h1 .........................   Passed    0.29 sec
      Start 66: ss-tf
42/73 Test #50: hs_evlib_foreign_event ......................   Passed   22.02 sec
      Start 49: hs_evlib_foreign_uv
43/73 Test #49: hs_evlib_foreign_uv .........................   Passed   21.03 sec
      Start 51: ss-warmcat
44/73 Test #51: ss-warmcat ..................................   Passed    2.69 sec
      Start 59: ss-smd
45/73 Test #59: ss-smd ......................................   Passed    1.78 sec
      Start 10: api-test-secure-streams
46/73 Test #10: api-test-secure-streams .....................   Passed    1.34 sec
      Start 11: http-client-warmcat
47/73 Test #11: http-client-warmcat .........................   Passed    0.27 sec
      Start 58: sspost-warmcat
48/73 Test #58: sspost-warmcat ..............................   Passed    0.84 sec
      Start 12: http-client-warmcat-h1
49/73 Test #12: http-client-warmcat-h1 ......................   Passed    0.25 sec
      Start  2: api-test-jose
50/73 Test  #2: api-test-jose ...............................   Passed    0.27 sec
      Start 70: ws-client-rx-warmcat
51/73 Test #70: ws-client-rx-warmcat ........................   Passed    0.27 sec
      Start 56: ki_ssproxyctx
52/73 Test #56: ki_ssproxyctx ...............................   Passed    0.12 sec
      Start 68: ki_ssproxy
53/73 Test #68: ki_ssproxy ..................................   Passed    0.11 sec
      Start 64: ki_mulssprxsmd_sspc
54/73 Test #64: ki_mulssprxsmd_sspc .........................   Passed    0.10 sec
      Start 61: ki_ssprxsmd_sspc
55/73 Test #61: ki_ssprxsmd_sspc ............................   Passed    0.11 sec
      Start 13: http-client-h2-rxflow-warmcat
56/73 Test #13: http-client-h2-rxflow-warmcat ...............   Passed    0.28 sec
      Start 14: http-client-h2-rxflow-warmcat-h1
57/73 Test #14: http-client-h2-rxflow-warmcat-h1 ............   Passed    0.34 sec
      Start 16: http-client-hugeurl-warmcat-h1
58/73 Test #16: http-client-hugeurl-warmcat-h1 ..............   Passed    0.16 sec
      Start 15: http-client-hugeurl-warmcat
59/73 Test #15: http-client-hugeurl-warmcat .................   Passed    0.16 sec
      Start 72: ki_wcs_srv
60/73 Test #72: ki_wcs_srv ..................................   Passed    0.12 sec
      Start 44: ki_hcp_srv
61/73 Test #44: ki_hcp_srv ..................................   Passed    0.11 sec
      Start 20: ki_hcmp_srv
62/73 Test #20: ki_hcmp_srv .................................   Passed    0.11 sec
      Start 18: ki_hcm_srv
63/73 Test #18: ki_hcm_srv ..................................   Passed    0.11 sec
      Start  7: api-test-lws_struct_sqlite
64/73 Test  #7: api-test-lws_struct_sqlite ..................   Passed    0.03 sec
      Start  1: api-test-gencrypto
65/73 Test  #1: api-test-gencrypto ..........................   Passed    0.02 sec
      Start  6: api-test-lws_struct-json
66/73 Test  #6: api-test-lws_struct-json ....................   Passed    0.01 sec
      Start  4: api-test-lws_dsh
67/73 Test  #4: api-test-lws_dsh ............................   Passed    0.01 sec
      Start  8: api-test-lws_tokenize
68/73 Test  #8: api-test-lws_tokenize .......................   Passed    0.01 sec
      Start  9: api-test-lwsac
69/73 Test  #9: api-test-lwsac ..............................   Passed    0.00 sec
      Start  3: api-test-lejp
70/73 Test  #3: api-test-lejp ...............................   Passed    0.00 sec
      Start 53: ki_ssproxy
71/73 Test #53: ki_ssproxy ..................................   Passed    0.11 sec
72/73 Test #66: ss-tf .......................................   Passed   55.51 sec
      Start  5: api-test-lws_smd
73/73 Test  #5: api-test-lws_smd ............................   Passed    4.22 sec

100% tests passed, 0 tests failed out of 73

Total Test time (real) = 137.76 sec
```

## Considerations for creating tests

### Timeout

The default test timeout is 1500s, for that reason it's good practice to set
a more suitable `TIMEOUT` property on every test.

### Working Directory

Server-side test apps usually need to be run from their `./minimal-examples/...`
directory so they can access their assets like index.html etc.

However when building with `-DLWS_WITH_MBEDTLS=1` then even client-side apps
need to be run from their directory, since they need to get the trusted CA for
warmcat.com or libwebsockets.org additionally.

For that reason it's good practice to set the `WORKING_DIRECTORY` property to
the home dir of the example app in all cases.

### Spawning Buddies

Many networking tests need to either spawn a client or a server in order to
have a "buddy" to talk to during the test for the opposing side.  This is a
bit awkward in cmake since it does not directly support spawning daemons as
test dependencies.

Lws provides helper scripts for unix type targets in `./scripts/ctest-background.sh`
and `./scripts/ctest-background-kill.sh`, which spawn background processes,
save the pid in a decorated /tmp file and can later take the process down.  This
also has arrangements to dump the log of any background process that exited
early.

To arrange the buddy to run aligned with the test, you first explain to cmake
how to start and stop the buddy using phony tests to make a "fixture" in cmake
terms.

In this example, taken from minimal-http-client-multi, we arrange for
minimal-http-server-tls to be available for our actual test.  The starting and
stopping definition, for "st_hcm_srv" and "ki_hcm_srv":

```
	add_test(NAME st_hcm_srv COMMAND
		${CMAKE_SOURCE_DIR}/scripts/ctest-background.sh
			hcm_srv $<TARGET_FILE:lws-minimal-http-server-tls>
			--port ${PORT_HCM_SRV} )
	add_test(NAME ki_hcm_srv COMMAND
		${CMAKE_SOURCE_DIR}/scripts/ctest-background-kill.sh
			hcm_srv $<TARGET_FILE_NAME:lws-minimal-http-server-tls>
				--port ${PORT_HCM_SRV})
```

... and binding those together so cmake knows they start and stop a specific
named fixture "hcm_srv", itself with an 800s timeout

```
	set_tests_properties(st_hcm_srv PROPERTIES
       		WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}/minimal-examples/http-server/minimal-http-server-tls
		FIXTURES_SETUP hcm_srv
		TIMEOUT 800)
	set_tests_properties(ki_hcm_srv PROPERTIES
		FIXTURES_CLEANUP hcm_srv)
```

... and finally, adding the "hcm_srv" fixture as a requirement on the actual
test (http-client-multi) we are testing

```
	set_tests_properties(http-client-multi
			     PROPERTIES
			     FIXTURES_REQUIRED "hcm_srv"
			     WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}/minimal-examples/http-client/minimal-http-client-multi
			     TIMEOUT 50)
```

Once all that explaining is done, ctest itself will take care about starting
and killing hcm_srv before and after http-client-multi test.

### Buddy sockets and test concurrency

For tests with local buddies using tcp sockets inside the same VM or systemd-
nspawn networking context, you cannot just use a well-known port like 7681.

ctest itself is usually executed concurrently, and Sai is typically building
multiple different instances concurrently as well (typically 3), so it may be
running different ctests inside the same VM simultaneously.

Different tests can have their own convention for port ranges, to solve the
problem about Sai running different tests concurrently inside one ctest.

For the case there are multiple ctests running, we can use the env var
`$ENV{SAI_INSTANCE_IDX}`, which is an ordinal like 0 or 1, to further ensure
that port selections won't conflict.  If not using Sai, you can just set this
in the evironment yourself to reflect your build instance index.

```
       #
       # instantiate the server per sai builder instance, they are running in the same
       # machine context in parallel so they can tread on each other otherwise
       #
       set(PORT_HCM_SRV "7670")
       if ("$ENV{SAI_INSTANCE_IDX}" STREQUAL "0")
               set(PORT_HCM_SRV 7671)
       endif()
       if ("$ENV{SAI_INSTANCE_IDX}" STREQUAL "1")
               set(PORT_HCM_SRV 7672)
       endif()
       if ("$ENV{SAI_INSTANCE_IDX}" STREQUAL "2")
               set(PORT_HCM_SRV 7673)
       endif()
       if ("$ENV{SAI_INSTANCE_IDX}" STREQUAL "3")
               set(PORT_HCM_SRV 7674)
       endif()
```

This is complicated enough that the best approach is copy an existing simple
case like the CMakeLists.txt for minimal-http-client and change the names and
ports to be unique.

# Tips on debugging with lws

## Problem with the library, or your code?

Because lws is only really used when already combined with user code,
it can be a headache figuring out if the actual problem is inside lws
or in the user code.

If it's in lws, I would really like to solve it, but if it's in your
code, that's your problem.  Finding out which side it's on when it
involves your code is also something you need to try to resolve.

The minimal examples are useful because if they demonstrate the same
problem, it's something about your platform or lws itself, I have the
minimal examples so I can test it and find out if it's your platform.
If I can reproduce it, it's my problem.

## Debug builds

With cmake, build with `-DCMAKE_BUILD_TYPE=DEBUG` to build in extra
logging, and use a log level bitmap of eg, 1039 or 1151 to enable
the extra logs for print.

The minimal examples take a -d xxx commandline parameter so you can
select the logging level when you run it.

The extra logging can be very useful to understand the sequencing of
problematic actions.

## Valgrind

If your problems involve heap corruption or use-after-free, Valgrind
is indespensible.  It's simple to use, if you normally run `xxx`, just
run `valgrind xxx`.  Your code will run slower, usually something
like 2 - 4x slower but it depends on the exact code.  However you will
get a backtrace as soon as there is some kind of misbehaviour of either
lws or your code.

lws is developed using valgrind routinely and strives to be completely
valgrind-clean.  So typically any problems reported are telling you
about problems in user code (or my bugs).

## Traffic dumping

The best place for dumping traffic, assuming you are linking against a
tls library, is `lws_ssl_capable_read()` and `lws_ssl_capable_write()`
in either `./lib/tls/openssl/openssl-ssl.c` or
`./lib/tls/mbedtls/mbedtls-ssl.c` according to which tls library you
are using.  There are default-`#if 0` sections in each function like

```
#if 0
	/*
	 * If using mbedtls type tls library, this is the earliest point for all
	 * paths to dump what was received as decrypted data from the tls tunnel
	 */
	lwsl_notice("%s: len %d\n", __func__, len);
	lwsl_hexdump_notice(buf, len);
#endif
```

Enable these to get hexdumps for all unencrypted data in both directions.

ESP32 Support
=============

See \ref esp32 for details on how to build lws as a component in an ESP-IDF project.

Lws provides a "factory" application

https://github.com/warmcat/lws-esp32-factory

and a test application which implements the generic lws server test apps

https://github.com/warmcat/lws-esp32-test-server-demos

The behaviours of the generic factory are are quite rich, and cover uploading SSL certs through factory and user configuration, AP selection and passphrase entry, and managing a switch to allow the user to force entry to user setup mode at boot subsequently.

The factory app comes with partitioning for a 1MB factory partition containing that app and data, and a single 2.9MB OTA partition containing the main app.

The factory app is able to do OTA updates for both the factory and OTA partition slots; updating the factory slot first writes the new image to the OTA slot and copies it into place at the next boot, after which the user can reload the OTA slot.

State|Image|AP SSID|Port|URL|Mode
---|---|---|---|---|---
Factory Reset or Uninitialized|Factory|AP: ESP_012345|80|http://192.168.4.1|factory.html - to set certificates and serial
User configuration|Factory|AP: config-model-serial|443|https://192.168.4.1|index.html - user set up his AP information
Operation|OTA|Station only|443|https://model-serial.local|OTA application

## Basic Auth

The lws-esp32-test-server-demos app also demos basic auth.

On a normal platform this is done by binding a mount to a text file somewhere in the filesystem, which
contains user:password information one per line.

On ESP32 there is not necessarily any generic VFS in use.  So instead, the basic auth lookup is bound to
a given nvs domain, where the username is the key and the password the value.  main/main.c in the test
demos app shows how to both make the mount use basic auth, and how to set a user:password combination
using nvs.

# lws event library support

## v4.0 and below

Before v4.1, lws allowed selecting some event library support for inclusion
in the libwebsockets library

Option|Feature
---|---
`LWS_WITH_GLIB`|glib
`LWS_WITH_LIBEVENT`|libevent
`LWS_WITH_LIBUV`|libuv
`LWS_WITH_LIBEV`|libev

The user code can select by `info->options` flags at runtime which event loop
it wants to use.

The only restriction is that libev and libevent can't coexist, because their
header namespace conflicts.

## v4.1 and above

Lws continues to support the old way described above, but there's an additional
new cmake option that decides how they are built if any are selected,
`LWS_WITH_EVLIB_PLUGINS`.

The old behaviour is set by `LWS_WITH_EVLIB_PLUGINS=0`, for UNIX platforms, this
is set to 1 by default.  This causes the enabled event lib support to each be built into
its own dynamically linked plugin, and lws will bring in the requested support alone
at runtime after seeing the `info->options` flags requested by the user code.

This has two main benefits, first the conflict around building libevent and libev
together is removed, they each build isolated in their own plugin; the libwebsockets
core library build doesn't import any of their headers (see below for exception).
And second, for distro packaging, the event lib support plugins can be separately
packaged, and apps take dependencies on the specific event lib plugin package, which
itself depends on the libwebsockets core library.  This allows just the needed
dependencies for the packageset without forcing everything to bring everything in.

Separately, lws itself has some optional dependencies on libuv, if you build lwsws
or on Windows you want plugins at all.  CMake will detect these situations and
select to link the lws library itself to libuv if so as well, independent of whatever
is happening with the event lib support.

## evlib plugin install

The produced plugins are named

event lib|plugin name
---|---
glib|`libwebsockets-evlib_glib.so`
event|`libwebsockets-evlib_event.so`
uv|`libwebsockets-evlib_uv.so`
ev|`libwebsockets-evlib_ev.so`

The evlib plugins are installed alongside libwebsockets.so/.a into the configured
library dir, it's often `/usr/local/lib/` by default on linux.

Lws looks for them at runtime using the build-time-configured path.

## Component packaging

The canonical package name is `libwebsockets`, the recommended way to split the
packaging is put the expected libs and pkgconfig in `libwebsockets` or `libwebsockets-core`,
the latter is followed by the provided cmake, and produce an additional package per build
event library plugin, named, eg `libwebsockets-evlib_glib`, which has a dependency on
`libwebsockets[-core]`.

Applications that use the default event loop can directly require `libwebsockets[-core]`,
and application packages that need specific event loop support can just require, eg,
`libwebsockets-evlib_glib`, which will bring that in and the core lws pieces in one step.
There is then no problem with multiple apps requiring different event libs, they will
bring in all the necessary pieces which will not conflict either as packages or at
runtime.

## `LWS_WITH_DISTRO_RECOMMENDED`

The cmake helper config `LWS_WITH_DISTRO_RECOMMENDED` is adapted to build all the
event libs with the event lib plugin support enabled.

# Considerations around Event Loops

Much of the software we use is written around an **event loop**.  Some examples

 - Chrome / Chromium, transmission, tmux, ntp SNTP... [libevent](https://libevent.org/)
 - node.js / cjdns / Julia / cmake ... [libuv](https://archive.is/64pOt)
 - Gstreamer, Gnome / GTK apps ... [glib](https://people.gnome.org/~desrt/glib-docs/glib-The-Main-Event-Loop.html)
 - SystemD ... sdevent
 - OpenWRT ... uloop

Many applications roll their own event loop using poll() or epoll() or similar,
using the same techniques.  Another set of apps use message dispatchers that
take the same approach, but are for cases that don't need to support sockets.
Event libraries provide crossplatform abstractions for this functoinality, and
provide the best backend for their event waits on the platform automagically.

libwebsockets networking operations require an event loop, it provides a default
one for the platform (based on poll() for Unix) if needed, but also can natively
use any of the event loop libraries listed above, including "foreign" loops
already created and managed by the application.

## What is an 'event loop'?

Event loops have the following characteristics:

 - they have a **single thread**, therefore they do not require locking
 - they are **not threadsafe**
 - they require **nonblocking IO**
 - they **sleep** while there are no events (aka the "event wait")
 - if one or more event seen, they call back into user code to handle each in
   turn and then return to the wait (ie, "loop")

### They have a single thread

By doing everything in turn on a single thread, there can be no possibility of
conflicting access to resources from different threads... if the single thread
is in callback A, it cannot be in two places at the same time and also in
callback B accessing the same thing: it can never run any other code
concurrently, only sequentially, by design.

It means that all mutexes and other synchronization and locking can be
eliminated, along with the many kinds of bugs related to them.

### They are not threadsafe

Event loops mandate doing everything in a single thread.  You cannot call their
apis from other threads, since there is no protection against reentrancy.

Lws apis cannot be called safely from any thread other than the event loop one,
with the sole exception of `lws_cancel_service()`.

### They have nonblocking IO

With blocking IO, you have to create threads in order to block them to learn
when your IO could proceed.  In an event loop, all descriptors are set to use
nonblocking mode, we only attempt to read or write when we have been informed by
an event that there is something to read, or it is possible to write.

So sacrificial, blocking discrete IO threads are also eliminated, we just do
what we should do sequentially, when we get the event indicating that we should
do it.

### They sleep while there are no events

An OS "wait" of some kind is used to sleep the event loop thread until something
to do.  There's an explicit wait on file descriptors that have pending read or
write, and also an implicit wait for the next scheduled event.  Even if idle for
descriptor events, the event loop will wake and handle scheduled events at the
right time.

In an idle system, the event loop stays in the wait and takes 0% CPU.

### If one or more event, they handle them and then return to sleep

As you can expect from "event loop", it is an infinite loop alternating between
sleeping in the event wait and sequentially servicing pending events, by calling
callbacks for each event on each object.

The callbacks handle the event and then "return to the event loop".  The state
of things in the loop itself is guaranteed to stay consistent while in a user
callback, until you return from the callback to the event loop, when socket
closes may be processed and lead to object destruction.

Event libraries like libevent are operating the same way, once you start the
event loop, it sits in an inifinite loop in the library, calling back on events
until you "stop" or "break" the loop by calling apis.

## Why are event libraries popular?

Developers prefer an external library solution for the event loop because:

 - the quality is generally higher than self-rolled ones.  Someone else is
   maintaining it, a fulltime team in some cases.
 - the event libraries are crossplatform, they will pick the most effective
   event wait for the platform without the developer having to know the details.
   For example most libs can conceal whether the platform is windows or unix,
   and use native waits like epoll() or WSA accordingly.
 - If your application uses a event library, it is possible to integrate very
   cleanly with other libraries like lws that can use the same event library.
   That is extremely messy or downright impossible to do with hand-rolled loops.

Compared to just throwing threads on it

 - thread lifecycle has to be closely managed, threads must start and must be
   brought to an end in a controlled way.  Event loops may end and destroy
   objects they control at any time a callback returns to the event loop.

 - threads may do things sequentially or genuinely concurrently, this requires
   locking and careful management so only deterministic and expected things
   happen at the user data.

 - threads do not scale well to, eg, serving tens of thousands of connections;
   web servers use event loops.

## Multiple codebases cooperating on one event loop

The ideal situation is all your code operates via a single event loop thread.
For lws-only code, including lws_protocols callbacks, this is the normal state
of affairs.

When there is other code that also needs to handle events, say already existing
application code, or code handling a protocol not supported by lws, there are a
few options to allow them to work together, which is "best" depends on the
details of what you're trying to do and what the existing code looks like.
In descending order of desirability:

### 1) Use a common event library for both lws and application code

This is the best choice for Linux-class devices.  If you write your application
to use, eg, a libevent loop, then you only need to configure lws to also use
your libevent loop for them to be able to interoperate perfectly.  Lws will
operate as a guest on this "foreign loop", and can cleanly create and destroy
its context on the loop without disturbing the loop.

In addition, your application can merge and interoperate with any other
libevent-capable libraries the same way, and compared to hand-rolled loops, the
quality will be higher.

### 2) Use lws native wsi semantics in the other code too

Lws supports raw sockets and file fd abstractions inside the event loop.  So if
your other code fits into that model, one way is to express your connections as
"RAW" wsis and handle them using lws_protocols callback semantics.

This ties the application code to lws, but it has the advantage that the
resulting code is aware of the underlying event loop implementation and will
work no matter what it is.

### 3) Make a custom lws event lib shim for your custom loop

Lws provides an ops struct abstraction in order to integrate with event
libraries, you can find it in ./includes/libwebsockets/lws-eventlib-exports.h.

Lws uses this interface to implement its own event library plugins, but you can
also use it to make your own customized event loop shim, in the case there is
too much written for your custom event loop to be practical to change it.

In other words this is a way to write a customized event lib "plugin" and tell
the lws_context to use it at creation time.  See [minimal-http-server.c](https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/http-server/minimal-http-server-eventlib-custom/minimal-http-server.c)

### 4) Cooperate at thread level

This is less desirable because it gives up on unifying the code to run from a
single thread, it means the codebases cannot call each other's apis directly.

In this scheme the existing threads do their own thing, lock a shared
area of memory and list what they want done from the lws thread context, before
calling `lws_cancel_service()` to break the lws event wait.  Lws will then
broadcast a `LWS_CALLBACK_EVENT_WAIT_CANCELLED` protocol callback, the handler
for which can lock the shared area and perform the requested operations from the
lws thread context.

### 5) Glue the loops together to wait sequentially (don't do this)

If you have two or more chunks of code with their own waits, it may be tempting
to have them wait sequentially in an outer event loop.  (This is only possible
with the lws default loop and not the event library support, event libraries
have this loop inside their own `...run(loop)` apis.)

```
	while (1) {
		do_lws_wait(); /* interrupted at short intervals */
		do_app_wait(); /* interrupted at short intervals */
	}
```

This never works well, either:

 - the whole thing spins at 100% CPU when idle, or

 - the waits have timeouts where they sleep for short periods, but then the
   latency to service on set of events is increased by the idle timeout period
   of the wait for other set of events

## Common Misunderstandings

### "Real Men Use Threads"

Sometimes you need threads or child processes.  But typically, whatever you're
trying to do does not literally require threads.  Threads are an architectural
choice that can go either way depending on the goal and the constraints.

Any thread you add should have a clear reason to specifically be a thread and
not done on the event loop, without a new thread or the consequent locking (and
bugs).

### But blocking IO is faster and simpler

No, blocking IO has a lot of costs to conceal the event wait by blocking.

For any IO that may wait, you must spawn an IO thread for it, purely to handle
the situation you get blocked in read() or write() for an arbitrary amount of
time.  It buys you a simple story in one place, that you will proceed on the
thread if read() or write() has completed, but costs threads and locking to get
to that.

Event loops dispense with the threads and locking, and still provide a simple
story, you will get called back when data arrives or you may send.

Event loops can scale much better, a busy server with 50,000 connections active
does not have to pay the overhead of 50,000 threads and their competing for
locking.

With blocked threads, the thread can do no useful work at all while it is stuck
waiting.  With event loops the thread can service other events until something
happens on the fd.

### Threads are inexpensive

In the cases you really need threads, you must have them, or fork off another
process.  But if you don't really need them, they bring with them a lot of
expense, some you may only notice when your code runs on constrained targets

 - threads have an OS-side footprint both as objects and in the scheduler

 - thread context switches are not slow on modern CPUs, but have side effects
   like cache flushing

 - threads are designed to be blocked for arbitrary amounts of time if you use
   blocking IO apis like write() or read().  Then how much concurrency is really
   happening?  Since blocked threads just go away silently, it is hard to know
   when in fact your thread is almost always blocked and not doing useful work.

 - threads require their own stack, which is on embedded is typically suffering
   from a dedicated worst-case allocation where the headroom is usually idle

 - locking must be handled, and missed locking or lock order bugs found

### But... what about latency if only one thing happens at a time?

 - Typically, at CPU speeds, nothing is happening at any given time on most
   systems, the event loop is spending most of its time in the event wait
   asleep at 0% cpu.

 - The POSIX sockets layer is disjoint from the actual network device driver.
   It means that once you hand off the packet to the networking stack, the POSIX
   api just returns and leaves the rest of the scheduling, retries etc to the
   networking stack and device, descriptor queuing is driven by interrupts in
   the driver part completely unaffected by the event loop part.

 - Passing data around via POSIX apis between the user code and the networking
   stack tends to return almost immediately since its onward path is managed
   later in another, usually interrupt, context.

 - So long as enough packets-worth of data are in the network stack ready to be
   handed to descriptors, actual throughput is completely insensitive to jitter
   or latency at the application event loop

 - The network device itself is inherently serializing packets, it can only send
   one thing at a time.  The networking stack locking also introduces hidden
   serialization by blocking multiple threads.

 - Many user systems are decoupled like the network stack and POSIX... the user
   event loop and its latencies do not affect backend processes occurring in
   interrupt or internal thread or other process contexts

## Conclusion

Event loops have been around for a very long time and are in wide use today due
to their advantages.  Working with them successfully requires understand how to
use them and why they have the advantages and restrictions they do.

The best results come from all the participants joining the same loop directly.
Using a common event library in the participating codebases allows completely
different code can call each other's apis safely without locking.

# `lws_fi` Fault Injection

Most efforts during development go towards trying to make the system do what
it is supposed to do during normal operation.

But to provide reliable quality there's a need to not just test the code paths
for normal operation, but also to be able to easily confirm that they act
correctly under various fault conditions that may be difficult to arrange at
test-time. It's otherwise very easy for error conditions that are low
probability to be overlooked and turn out to do the wrong thing, eg, try to
clean up things they had not actually initialized, or forget to free things etc.

Code handling the operational failures we want to check may be anywhere,
including during early initialization or in user code before lws intialization.

To help with this lws has a `LWS_WITH_SYS_FAULT_INJECTION` build option that
provides a simple but powerful api for targeted fault injection in any lws or
user code, and provides a wide range of well-known internal faults inside lws
you can trigger from outside.

## Fault contexts and faults

The basic idea is objects in the user code can choose to initialize "fault
contexts" inside objects, that list named, well-known "faults" that the code
supoorts and that the user wants to inject.

Although these "fault contexts" can be embedded in objects directly at object
creation time, eg, for lws in the lws_context creation info struct, or the
client connection info struct, or Secure Stream info struct, it's usually
inconvenient to pass the desired faults directly deep into the code and attach
them at creation time.  Eg, if you want to cause a fault in a wsi instantiated
by a Secure Stream, that is internal lws code one step removed from the Secure
Stream object creation making it difficult to arrange.

For that reason, faults have a targeted inheritance scheme using namespace
paths, it's usually enough to just list the faults you want at context creation
time and they will be filter down to the internal objects you want to target
when they are created later.

![Fault Injection Overview](../doc-assets/fault-injection.png)

A fault injection request is made in `lws_fi_t` objects, specifying the
fault name and whether, and how often to inject the fault.

The "fault context" objects `lws_fi_ctx_t` embedded in the creation info
structs are linked-lists of `lws_fi_t` objects.  When Fault Injection is enabled
at build-time, the key system objects like the `lws_context`, `lws_vhost`, `wsi`
and Secure Stream handles / SSPC handles contain their own `lws_fi_ctx_t` lists
that may have any number of `lws_fi_t` added to them.

When downstream objects are created, eg, when an lws_context creates a Secure
Stream, in addition to using any faults provided directly in the SS info,
the lws_context faults are consulted to see if any relate to that streamtype
and should be applied.

Although faults can be added to objects at creation, it is far more convenient
to just pass a list of faults you want into the lws_context and have the
objects later match them using namespacing, described later.

## Integrating fault injection conditionals into code in private lws code

A simple query api `lws_fi(fi_ctx, "name")` is provided that returns 0 if no
fault to be injected, or 1 if the fault should be synthesized.  If there is no
rule matching "name", the answer is always to not inject a fault, ie, returns 0.

Similarly for convenience if FAULT_INJECTION is disabled at build, the `lws_fi()`
call always returns the constant `0`.

By default then just enabling Fault Injection at build does not have any impact
on code operation since the user must also add the fault injection rules he
wants to the objects's Fault Injection context.

## Integrating fault injection conditionals into user code with public apis

These public apis query the fault context in a wsi, lws_context, ss handle, or
sspc handle (client side of proxy) to find any matching rule, if so they return
1 if the conditions (eg, probability) are met and the fault should be injected.

These allow user code to use the whole Fault Injection system without having to
understand anything except the common object like a wsi they want to query and
the name of the fault rule they are checking.

|FI context owner|Public API|
|---|---|
|lws_context|`int lws_fi_user_context_fi(struct lws_context *ctx, const char *rule)`|
|wsi|`int lws_fi_user_wsi_fi(struct lws *wsi, const char *rule)`|
|ss handle|`int lws_fi_user_ss_fi(struct lws_ss_handle *h, const char *rule)`|
|sspc handle|`int lws_fi_user_sspc_fi(struct lws_sspc_handle *h, const char *rule)`|

For example, the minimal-http-client user code example contains this in its
ESTABLISHED callback

```
		if (lws_fi_user_wsi_fi(wsi, "user_reject_at_est"))
			return -1;
```

which can be triggered by running it with

`lws-minimal-http-client --fault-injection 'wsi/user_reject_at_est'`, causing

```
...
[2021/03/11 13:41:05:2769] U: Connected to 46.105.127.147, http response: 200
[2021/03/11 13:41:05:2776] W: lws_fi: Injecting fault unk->user_reject_at_est
[2021/03/11 13:41:05:2789] E: CLIENT_CONNECTION_ERROR: HS: disallowed at ESTABLISHED
...
```

When `LWS_WITH_SYS_FAULT_INJECTION` is disabled, these public apis become
preprocessor defines to `(0)`, so the related code is removed by the compiler.

## Types of fault injection "when" strategy

The api keeps track of each time the context was asked and uses this information
to drive the decision about when to say yes, according to the type of rule

|Injection rule type|Description|
|---|---|
|`LWSFI_ALWAYS`|Unconditionally inject the fault|
|`LWSFI_DETERMINISTIC`|after `pre` times without the fault, the next `count` times exhibit the fault`|
|`LWSFI_PROBABILISTIC`|exhibit a fault `pre` percentage of the time|
|`LWSFI_PATTERN`|Reference `pre` bits pointed to by `pattern` and fault if the bit set, pointing to static array|
|`LWSFI_PATTERN_ALLOC`|Reference `pre` bits pointed to by `pattern` and fault if the bit set, pointing to allocated array, freed when fault goes out of scope|

Probabalistic choices are sourced from a PRNG with a seed set in the context
creation info Fault Injection Context.  By default the lws helper
`lws_cmdline_option_handle_builtin()` sets this to the time in us, but it can
be overridden using `--fault-seed <decimal>`, and the effective PRNG seed is
logged when the commandline options are initially parsed.

## Addings Fault Injection Rules to `lws_fi_ctx_t`

Typically the lws_context is used as the central, toplevel place to define
faults.  This is done by adding prepared `lws_fi_t` objects on the stack one by
one to the context creation info struct's `.fic` member, using
`lws_fi_add(lws_fi_ctx_t *fic, const lws_fi_t *fi);`, this will allocate and copy
the provided `fi` into the allocation, and attach it to the `lws_fi_ctx_t` list.

When the context (or other object using the same scheme) is created, it imports
all the faults from the info structure `.fic` and takes ownership of them,
leaving the info `.fic` empty and ready to go out of scope.

## Passing in fault injection rules

A key requirement is that Fault Injection rules must be availble to the code
creating an object before the object has been created.  This is why the user
code prepares a Fault Injection context listing his rules in the creation info
struct, rather than waiting for the object to be created and then attach Fault
Injection rules... it's too late then to test faults during the creation.

## Directly applying fault contexts

You can pass in a Fault Injection context prepared with lws_fi_t added to it
when creating the following kinds of objects

|Object being created|info struct|Fault injection Context member|
|---|---|---|
|lws context|struct lws_context_creation_info|`fic`|
|vhost|struct lws_context_creation_info|`fic`|
|Secure Stream|struct lws_ss_info|`fic`|
|client wsi|struct lws_client_connect_info|`fic`|

However typically the approach is just provide a list of faults at context
creation time, and let the objects match and inherit using namespacing,
described next.

## Using the namespace to target specific instances

Lws objects created by the user can directly have a Fault Injection context
attached to them at creation time, so the fault injection objects directly
relate to the object.

But in other common scenarios, there is no direct visibility of the object that
we want to trigger faults in, it may not exist until some time later.  Eg, we
want to trigger faults in the listen socket of a vhost.  To allow this, the
fault names can be structured with a /path/ type namespace so objects created
later can inherit faults.

Notice that if you are directly creating the vhost, Secure Stream or wsi, you
can directly attach the subrule yourself without the namespacing needed.  The
namespacing is used when you have access to a higher level object at creation-
time, like the lws_context, and it will itself create the object you want to
target without your having any direct access to it.

|namespace form|effect|
|---|---|
|**vh=myvhost/**subrule|subrule is inherited by the vhost named "myvhost" when it is created|
|**vh/**subrule|subrule is inherited by any vhost when it is created|
|**ss=mystream/**subrule|subrule is inherited by SS of streamtype "mystream" (also covers SSPC / proxy client)|
|**ss/**subrule|subrule is inherited by all SS of any streamtype (also covers SSPC / proxy client)|
|**wsi=myname/**subrule|subrule is inherited by client wsi created with `info->fi_wsi_name` "myname"|
|**wsi/**subrule|subrule is inherited by any wsi|

Namespaces can be combined, for example `vh=myvhost/wsi/listenskt` will set the
`listenskt` fault on wsi created by the server vhost "myvhost", ie, it will
cause the listen socket for the vhost to error out on creation.

In the case of wsi migration when it's the network connection wsi on an h2
connection that is migrated to be SID 1, the attached faults also migrate.

Here is which Fault Injection Contexts each type of object inherits matching
Fault Injection rules from:

|Object type|Initialized with|Inherit matching faults from|
|---|---|---|
|context|`struct lws_context_creation_info` .fic|-|
|vhost|`struct lws_context_creation_info` .fic|context FIC|
|client wsi|`struct lws_client_connect_info` .fic|context FIC, vhost FIC|
|ss / sspc|`lws_ss_info_t` .fic|context FIC|
|ss / sspc wsi|-|context FIC, vhost FIC, ss / sspc .fic|

Since everything can be reached from the lws_context fault context, directly or
by additional inheritence, and that's the most convenient to set from the
outside, that's typically the original source of all injected faults.

## Integration with minimal examples

All the minimal examples that use the `lws_cmdline_option_handle_builtin()` api
can take an additional `--fault-injection "...,..."` switch, which automatically
parses the comma-separated list in the argument to add faults with the given
name to the lws_context.  For example,

`lws-minimal-http-client --fault-injection "wsi/dnsfail"`

will force all wsi dns lookups to fail for that run of the example.

### Specifying when to inject the fault

By default, if you just give the name part, if the namespace is absent or
matches an object, the fault will be injected every time.  It's also possible
to make the fault inject itself at a random probability, or in a cyclic pattern,
by giving additional information in brackets, eg

|Syntax|Used with|Meaning|
|---|---|---|
|`wsi/thefault`|lws_fi()|Inject the fault every time|
|`wsi/thefault(10%)`|lws_fi()|Randomly inject the fault at 10% probability|
|`wsi/thefault(.............X.X)`|lws_fi()|Inject the fault on the 14th and 16th try, every 16 tries|
|`wsi/thefault2(123..456)`|lws_fi_range()|Pick a number between 123 and 456|

You must quote the strings containing these symbols, since they may otherwise be
interpreted by your shell.

The last example above does not decide whether to inject the fault via `lws_fi()`
like the others.  Instead you can use it via `lws_fi_range()` as part of the
fault processing, on a secondary fault injection name.  For example you may have
a fault `myfault` you use with `lws_fi()` to decide when to inject the fault,
and then a second, related fault name `myfault_delay` to allow you to add code
to delay the fault action by some random amount of ms within an externally-
given range.  You can get a pseudo-random number within the externally-given
range by calling `lws_fi_range()` on `myfault_delay`, and control the whole
thing by giving, eg, `"myfault(10%),myfault_delay(123..456)"`

## Well-known fault names in lws

|Scope|Namespc|Name|Fault effect|
|---|---|---|---|
|context||`ctx_createfail1`|Fail context creation immediately at entry|
|context||`ctx_createfail_plugin_init`|Fail context creation as if a plugin init failed (if plugins enabled)|
|context||`ctx_createfail_evlib_plugin`|Fail context creation due to event lib plugin failed init (if evlib plugins enabled)|
|context||`ctx_createfail_evlib_sel`|Fail context creation due to unable to select event lib|
|context||`ctx_createfail_oom_ctx`|Fail context creation due to OOM on context object|
|context||`ctx_createfail_privdrop`|Fail context creation due to failure dropping privileges|
|context||`ctx_createfail_maxfds`|Fail context creation due to unable to determine process fd limit|
|context||`ctx_createfail_oom_fds`|Fail context creation due to OOM on fds table|
|context||`ctx_createfail_plat_init`|Fail context creation due to platform init failed|
|context||`ctx_createfail_evlib_init`|Fail context creation due to event lib init failed|
|context||`ctx_createfail_evlib_pt`|Fail context creation due to event lib pt init failed|
|context||`ctx_createfail_sys_vh`|Fail context creation due to system vhost creation failed|
|context||`ctx_createfail_sys_vh_init`|Fail context creaton due to system vhost init failed|
|context||`ctx_createfail_def_vh`|Fail context creation due to default vhost creation failed|
|context||`ctx_createfail_ss_pol1`|Fail context creation due to ss policy parse start failed (if policy enabled)|
|context||`ctx_createfail_ss_pol2`|Fail context creation due to ss policy parse failed (if policy enabled)|
|context||`ctx_createfail_ss_pol3`|Fail context creation due to ss policy set failed (if policy enabled)|
|context||`cache_createfail`|Fail `lws_cache` creation due to OOM|
|context||`cache_lookup_oom`|Fail `lws_cache` lookup due to OOM|
|vhost|`vh`|`vh_create_oom`|Fail vh creation on vh object alloc OOM|
|vhost|`vh`|`vh_create_oom`|Fail vh creation on vh object alloc OOM|
|vhost|`vh`|`vh_create_pcols_oom`|Fail vh creation at protocols alloc OOM|
|vhost|`vh`|`vh_create_access_log_open_fail`|Fail vh creation due to unable to open access log (LWS_WITH_ACCESS_LOG)|
|vhost|`vh`|`vh_create_ssl_srv`|Fail server ssl_ctx init|
|vhost|`vh`|`vh_create_ssl_cli`|Fail client ssl_ctx init|
|vhost|`vh`|`vh_create_srv_init`|Fail server init|
|vhost|`vh`|`vh_create_protocol_init`|Fail late protocol init (for late vhost creation)|
|srv vhost|`vh=xxx/wsi`|`listenskt`|Causes `socket()` allocation for vhost listen socket to fail|
|cli wsi|`wsi`|`dnsfail`|Sync: `getaddrinfo()` is not called and a EAI_FAIL return synthesized, Async: request not started and immediate fail synthesized|
|cli wsi|`wsi`|`sendfail`|Attempts to send data on the wsi socket fail|
|cli wsi|`wsi`|`connfail`|Attempts to connect on the wsi socket fail|
|cli wsi|`wsi`|`createfail`|Creating the client wsi itself fails|
|udp wsi|`wsi`|`udp_rx_loss`|Drop UDP RX that was actually received, useful with probabalistic mode|
|udp wsi|`wsi`|`udp_tx_loss`|Drop UDP TX so that it's not actually sent, useful with probabalistic mode|
|srv ss|`ss`|`ss_srv_vh_fail`|Secure Streams Server vhost creation forced to fail|
|cli ss|`ss`|`ss_no_streamtype_policy`|The policy for the streamtype is made to seem as if it is missing|
|sspc|`ss`|`sspc_fail_on_linkup`|Reject the connection to the proxy when we hear it has succeeded, it will provoke endless retries|
|sspc|`ss`|`sspc_fake_rxparse_disconnect_me`|Force client-proxy link parse to seem to ask to be disconnected, it will provoke endless retries|
|sspc|`ss`|`sspc_fake_rxparse_destroy_me`|Force client-proxy link parse to seem to ask to destroy the SS, it will destroy the SS cleanly|
|sspc|`ss`|`sspc_link_write_fail`|Force write on the link to fail, it will provoke endless retries|
|sspc|`ss`|`sspc_create_oom`|Cause the sspc handle allocation to fail as if OOM at creation time|
|sspc|`ss`|`sspc_fail_metadata_set`|Cause the metadata allocation to fail|
|sspc|`ss`|`sspc_rx_fake_destroy_me`|Make it seem that client's user code *rx() returned DESTROY_ME|
|sspc|`ss`|`sspc_rx_metadata_oom`|Cause metadata from proxy allocation to fail|
|ssproxy|`ss`|`ssproxy_dsh_create_oom`|Cause proxy's creation of DSH to fail|
|ssproxy|`ss`|`ssproxy_dsh_rx_queue_oom`|Cause proxy's allocation in the onward SS->P[->C] DSH rx direction to fail as if OOM, this causes the onward connection to disconnect|
|ssproxy|`wsi`|`ssproxy_client_adopt_oom`|Cause proxy to be unable to allocate for new client - proxy link connection object|
|ssproxy|`wsi`|`ssproxy_client_write_fail`|Cause proxy write to client to fail|
|ssproxy|`wsi`|`sspc_dsh_ss2p_oom`|Cause ss->proxy dsh allocation to fail|
|ssproxy|`ss`|`ssproxy_onward_conn_fail`|Act as if proxy onward client connection failed immediately|
|ssproxy|`ss`|`ssproxy_dsh_c2p_pay_oom`|Cause proxy's DSH alloc for C->P payload to fail|
|ss|`ss`|`ss_create_smd`|SMD: ss creation smd registration fail|
|ss|`ss`|`ss_create_vhost`|Server: ss creation acts like no vhost matching typename (only for `!vhost`)|
|ss|`ss`|`ss_create_pcol`|Server: ss creation acts like no protocol given in policy|
|ss|`ss`|`ss_srv_vh_fail`|Server: ss creation acts like unable to create vhost|
|ss|`ss`|`ss_create_destroy_me`|ss creation acts like CREATING state returned DESTROY_ME|
|ss|`ss`|`ss_create_no_ts`|Static Policy: ss creation acts like no trust store|
|ss|`ss`|`ss_create_smd_1`|SMD: ss creation acts like CONNECTING said DESTROY_ME|
|ss|`ss`|`ss_create_smd_2`|SMD: ss creation acts like CONNECTED said DESTROY_ME|
|ss|`ss`|`ss_create_conn`|Nailed up: ss creation client connection fails with DESTROY_ME|
|wsi|`wsi`|`timedclose`|(see next) Cause wsi to close after some time|
|wsi|`wsi`|`timedclose_ms`|Range of ms for timedclose (eg, "timedclose_ms(10..250)"|

## Well-known namespace targets

Namespaces can be used to target these more precisely, for example even though
we are only passing the faults we want inject at the lws_context, we can use
the namespace "paths" to target only the wsis created by other things.

To target wsis from SS-based connections, you can use `ss=stream_type_name/`,
eg for captive portal detection, to have it unable to find its policy entry:

`ss=captive_portal_detect/ss_no_streamtype_policy` (disables CPD from operating)

...to force it to fail to resolve the server DNS:

`ss=captive_portal_detect/wsi/dnsfail` (this makes CPD feel there is no internet)

...to target the connection part of the captive portal testing instead:

`ss=captive_portal_detect/wsi/connfail` (this also makes CPD feel there is no internet)

### Well-known internal wsi type names

Wsi created for internal features like Async DNS processing can also be targeted

|wsi target|Meaning|
|---|---|
|`wsi=asyncdns/`|UDP wsi used by lws Async DNS support to talk to DNS servers|
|`wsi=dhcpc/`|UDP wsi used by lws DHCP Client|
|`wsi=ntpclient/`|UDP wsi used by lws NTP Client|

For example, passing in at lws_context level `wsi=asyncdns/udp_tx_loss`
will force async dns to be unable to resolve anything since its UDP tx is
being suppressed.

At client connection creation time, user code can also specify their own names
to match on these `wsi=xxx/` namespace parts, so the faults only apply to
specific wsi they are creating themselves later.  This is done by setting the
client creation info struct `.fi_wsi_name` to the string "xxx".

# h2 long poll in lws

lws server and client can support "immortal" streams that are
not subject to normal timeouts under a special condition.  These
are read-only (to the client).

Network connections that contain at least one immortal stream
are themselves not subject to timeouts until the last immortal
stream they are carrying closes.

Because of this, it's recommended there is some other way of
confirming that the client is still active.

## Setting up lws server for h2 long poll

Vhosts that wish to allow clients to serve these immortal
streams need to set the info.options flag `LWS_SERVER_OPTION_VH_H2_HALF_CLOSED_LONG_POLL`
at vhost creation time.  The JSON config equivalent is to set

```
"h2-half-closed-long-poll": "1"
```

on the vhost.  That's all that is needed.

Streams continue to act normally for timeout with the exception
client streams are allowed to signal they are half-closing by
sending a zero-length DATA frame with END_STREAM set.  These
streams are allowed to exist outside of any timeout and data
can be sent on them at will in the server -> client direction.

## Setting client streams for long poll

An API is provided to allow established h2 client streams to
transition to immortal mode and send the END_STREAM to the server
to indicate it.

```
int
lws_h2_client_stream_long_poll_rxonly(struct lws *wsi);
```

## Example applications

You can confirm the long poll flow simply using example applications.
Build and run `http-server/minimal-http-server-h2-long-poll` in one
terminal.

In another, build the usual `http-client/minimal-http-client` example
and run it with the flags `-l --long-poll`

The client will connect to the server and transition to the immortal mode.
The server sends a timestamp every minute to the client, and that will
stay up without timeouts.

# Client http cookie storage, caching and application

lws now has the option to store incoming cookies in a Netscape cookie jar file
persistently, and auto-apply relevant cookies to future outgoing requests.

A L1 heap cache of recent cookies is maintained, along with LRU tracking and
removal of entries from cache and the cookie jar file according to their cookie
expiry time.

The cookie handling is off by default per-connection for backwards compatibility
and to avoid unexpected tracking.

## Enabling at build-time

Make sure `-DLWS_WITH_CACHE_NSCOOKIEJAR=1` is enabled at cmake (it is on by
default now).

## Configuring the cookie cache

The cookie cache is managed through context creation info struct members.

|member|function|
|---|---|
|`.http_nsc_filepath`|Filepath to store the cookie jar file at|
|`.http_nsc_heap_max_footprint`|0, or Max size in bytes for the L1 heap cache|
|`.http_nsc_heap_max_items`|0, or Max number of cookies allowed in L1 heap cache|
|`.http_nsc_heap_max_payload`|0, or Largest cookie we are willing to handle|

## Enabling per-connection in lws

To enable it on connections at lws level, add the flag `LCCSCF_CACHE_COOKIES` to
the client connection info struct `.ssl_connection` flags.

## Enabling per-connection in Secure Streams policy

To enable it on Secure Streams, in the streamtype policy add

```
	"http_cookies":		true
```

# Http fallback and raw proxying

Lws has several interesting options and features that can be applied to get
some special behaviours... this article discusses them and how they work.

## Overview of normal vhost selection

Lws supports multiple http or https vhosts sharing a listening socket on the
same port.

For unencrypted http, the Host: header is used to select which vhost the
connection should bind to, by comparing what is given there against the
names the server was configured with for the various vhosts.  If no match, it
selects the first configured vhost.

For TLS, it has an extension called SNI (Server Name Indication) which tells
the server early in the TLS handshake the host name the connection is aimed at.
That allows lws to select the vhost early, and use vhost-specific TLS certs
so everything is happy.  Again, if there is no match the connection proceeds
using the first configured vhost and its certs.

## Http(s) fallback options

What happens if you try to connect, eg, an ssh client to the http server port
(this is not an idle question...)?  Obviously the http server part or the tls
part of lws will fail the connection and close it.  (We will look at that flow
in a moment in detail for both unencrypted and tls listeners.)

However if the first configured vhost for the port was created with the
vhost creation info struct `.options` flag `LWS_SERVER_OPTION_FALLBACK_TO_APPLY_LISTEN_ACCEPT_CONFIG`,
then instead of the error, the connection transitions to whatever role was
given in the vhost creation info struct `.listen_accept_role` and `.listen_accept_protocol`.

With lejp-conf / lwsws, the options can be applied to the first vhost using:

```
   "listen-accept-role": "the-role-name",
   "listen-accept-protocol": "the-protocol-name",
   "fallback-listen-accept": "1"
```

See `./minimal-examples/raw/minimal-raw-fallback-http-server` for examples of
all the options in use via commandline flags.

So long as the first packet for the protocol doesn't look like GET, POST, or
a valid tls packet if connection to an https vhost, this allows the one listen
socket to handle both http(s) and a second protocol, as we will see, like ssh.

Notice there is a restriction that no vhost selection processing is possible,
neither for tls listeners nor plain http ones... the packet belonging to a
different protocol will not send any Host: header nor tls SNI.

Therefore although the flags and settings are applied to the first configured
vhost, actually their effect is global for a given listen port.  If enabled,
all vhosts on the same listen port will do the fallback action.

### Plain http flow

![plain http flow](/doc-assets/accept-flow-1.svg)

Normally, if the first received packet does not contain a valid HTTP method,
then the connection is dropped.  Which is what you want from an http server.

However if enabled, the connection can transition to the defined secondary
role / protocol.

|Flag|lejp-conf / lwsws|Function|
|---|---|---|
|`LWS_SERVER_OPTION_FALLBACK_TO_APPLY_LISTEN_ACCEPT_CONFIG`|`"fallback-listen-accept": "1"`|Enable fallback processing|

### TLS https flow

![tls https flow](/doc-assets/accept-flow-2.svg)

If the port is listening with tls, the point that a packet from a different
protocol will fail is earlier, when the tls tunnel is being set up.

|Flag|lejp-conf / lwsws|Function|
|---|---|---|
|`LWS_SERVER_OPTION_FALLBACK_TO_APPLY_LISTEN_ACCEPT_CONFIG`|`"fallback-listen-accept": "1"`|Enable fallback processing|
|`LWS_SERVER_OPTION_REDIRECT_HTTP_TO_HTTPS`|`"redirect-http": "1"`|Treat invalid tls packet as http, issue http redirect to https://|
|`LWS_SERVER_OPTION_ALLOW_HTTP_ON_HTTPS_LISTENER`|`"allow-http-on-https": "1"`|Accept unencrypted http connections on this tls port (dangerous)|

The latter two options are higher priority than, and defeat, the first one.

### Non-http listener

![non-http flow](/doc-assets/accept-flow-3.svg)

It's also possible to skip the fallback processing and just force the first
vhost on the port to use the specified role and protocol in the first place.

|Flag|lejp-conf / lwsws|Function|
|---|---|---|
|LWS_SERVER_OPTION_ADOPT_APPLY_LISTEN_ACCEPT_CONFIG|`"apply-listen-accept": "1"`|Force vhost to use listen-accept-role / listen-accept-protocol|

## Using http(s) fallback with raw-proxy

If enabled for build with `cmake .. -DLWS_ROLE_RAW_PROXY=1 -DLWS_WITH_PLUGINS=1`
then lws includes ready-to-use support for raw tcp proxying.

This can be used standalone on the first vhost on a port, but most intriguingly
it can be specified as the fallback for http(s)...

See `./minimal-examples/raw/minimal-raw-proxy-fallback.c` for a working example.

### fallback with raw-proxy in code

On the first vhost for the port, specify the required "onward" pvo to configure
the raw-proxy protocol...you can adjust the "ipv4:127.0.0.1:22" to whatever you
want...

```
	static struct lws_protocol_vhost_options pvo1 = {
	        NULL,
	        NULL,
	        "onward",		/* pvo name */
	        "ipv4:127.0.0.1:22"	/* pvo value */
	};

	static const struct lws_protocol_vhost_options pvo = {
	        NULL,           	/* "next" pvo linked-list */
	        &pvo1,			/* "child" pvo linked-list */
	        "raw-proxy",		/* protocol name we belong to on this vhost */
	        ""              	/* ignored */
	};
```

... and set up the fallback enable and bindings...

```
	info.options |= LWS_SERVER_OPTION_FALLBACK_TO_APPLY_LISTEN_ACCEPT_CONFIG;
	info.listen_accept_role = "raw_proxy";
	info.listen_accept_proxy = "raw_proxy";
	info.pvo = &pvo;
```

### fallback with raw-proxy in JSON conf

On the first vhost for the port, enable the raw-proxy protocol on the vhost and
set the pvo config

```
                "ws-protocols": [{
                        "raw-proxy": {
                         "status": "ok",
                         "onward": "ipv4:127.0.0.1:22"
                        }
                 }],
```

Enable the fallback behaviour on the vhost and the role / protocol binding

```
	"listen-accept-role": "raw-proxy",
	"listen-accept-protocol": "raw-proxy",
	"fallback-listen-accept": "1"
```

### Testing

With this configured, the listen port will function normally for http or https
depending on how it was set up.

But if you try to connect to it with an ssh client, that will also work fine.

The libwebsockets.org server is set up in this way, you can confirm it by
visiting `https://libwebsockets.org` on port 443 as usual, but also trying
`ssh -p 443 invalid@libwebsockets.org`... you will get permission denied from
your ssh client.  With valid credentials in fact that works perfectly for
ssh, scp, git-over-ssh etc all on port 443...

# Notes on http parser corner cases

## Dealing with %00

%00 is considered illegal in

 - the path part of the URL.  A lot of user code handles it as a NUL terminated string,
   even though the header get apis are based around length.  So it is disallowed to
   avoid ambiguity.

 - the name part of a urlarg, like ?name=value

%00 is valid in

 - the value part of a urlarg, like ?name=value

When the parser sees %00 where it is not allowed, it simply drops the connection.

## Note on proper urlarg handling

urlargs are allowed to contain non-NUL terminated binary.  So it is important to
use the length-based urlarg apis

 - `lws_hdr_copy_fragment()`
 - `lws_get_urlarg_by_name_safe()`

The non-length based urlarg api

 - `lws_get_urlarg_by_name()`

...is soft-deprecated, it's still allowed but it will be fooled by the first %00
seen in the argument into truncating the argument.  Use `lws_get_urlarg_by_name_safe()`
instead.

# JIT trust

![JIT Trust logo](../doc-assets/jit-trust-logo.png)

## Background

Most systems using openssl rely on a system trust bundle that openssl was
compiled to load at library init.  This is a bit expensive, since it
instantiates over 120 CA X.509 certs, but most modern Linux systems don't really
notice the permanent use of 1MB or so of heap from init, the advantage is client
connections have all the trusted root certs available in memory to perform
validation.

![Using system trust bundles](../doc-assets/jit-trust-system-trust.png)

For the kind of systems that choose mbedtls, they will typically either be
burdened by or not even have enough ram to take this approach.

If the device only connects to endpoints that are signed by a specific
CA, you can just prepare the connection with the known trusted CA, that's
the approach the examples take.  This method should still be used for critical
connections to the cloud, for example provide the necessary CA cert in the
Secure Streams policy, or at vhost creation time.

![Using system trust bundles](../doc-assets/jit-trust-single-trust.png)

However if you also have a browser type application that could connect anywhere,
but you don't have heap spare to preload all the CAs, you need something like
"JIT trust".

## JIT trust overview

The basic approach is to connect to the server to retrieve its certificates,
then study the certificates to determine the identity of the missing trusted
cert we should be trying to validate with.

![JIT Trust overview](../doc-assets/jit-trust-overview.png)

We attempt to get the trusted cert from some local or remote store, and retry
the connection having instantiated the missing CA cert as trusted for that
connection, if it is one that we do actually trust.  If it lies about what CA it
needs to validate, or we do not trust the one it asks for, subsequent
connections will fail.

If it asked for a trusted CA that we trust, and the relationship was valid, the
tls negotiation should then complete successfully, and we can cache the CA cert
and the host -> CA cert pre-trust requirement so future connections can work
first time.

## Subject Key Id and Authority Key Id

All of the certificates publish a unique-enough personal "Subject Key ID" or
SKID blob.  These are typically 20-byte hashes based on the cert public key.

When a server certificate is issued by the CA, an entry is made first in the
certificate noting the SKID of the certificate that will be used to sign it,
in an "Authority Key ID", or AKID, extension.  The certificate is then signed by
the parent certificate private key to prove it was issued by the real owner of
the CA or intermediate certificate.

![X.509 validation paths](../doc-assets/jit-trust-paths.png)

Basically this AKID on a certificate is guiding the validator with
information about which certificate it claims is next in the chain of trust
leading back to a trusted CA.  Lying about it doesn't help an attacker,
because we're only using the AKID to get the CA certificate and then try to do
the full signature check using it, if it's not really signed by the AKID cert it
told, or anything else wrong, the actual validation will just fail.

A chain that terminates in a CA certificate is complete, and can undergo full
validation using the tls library.

## Converting the Mozilla trust bundle for JIT trust

Lws provides a bash script `./scripts/mozilla-trust-gen.sh` that can fetch the
latest Mozilla CA trust bundle for certs usable for tls validation, and convert
it to three different forms to allow maintaining the trust bundle in different
ways for different kinds of device to consume.

 - as a webroot directory, so you can server trusted DERs, with
   symlink indexes to the CA certs by SKID and issuer/serial

 - as an atomic binary blob, currently about 143KB, with structure
   at the start pointing to DER certs and indexes inside

 - a C-compiler friendly `uint8_t` array version of the blob,
   so it can be compiled into .rodata directly if necessary.

Currently there are 128 certs in the trust bundle, and the whole blob is about
143KB uncompressed.

## Considerations about maintaining the trust blob

Mozilla update their trust bundle at intervals, and there have been at least
three cases where they have removed or distrusted CAs from it by their own
decision, because they have issued dangerous certificates, (like one for `*`
that will validate anything at all).  Certifacte owners may also revoke their
own certificates for any reason and issue replacements.

The certs in the trust bundle expire, currently 10/128 will expire within 3
years and 50/128 over the next 10 years.  So new and replacement certificates
are also being added at intervals.

Part of using the trust bundle is building in some way to update what is trusted
over the lifetime of the device, which may exceed 10 years.

Depending on the device, it may not be any problem to keep the trust blob in the
firmware, and update the firmware ongoing every few months.  So you could build
it into the firmware using the C array include file (the minimal example takes
this approach).

Another device may have difficulty updating the firmware outside of emergencies,
it could keep the trust blob in a separate area and update it separately.
Having it as a single blob makes it easy to fetch and update.

Finally constrained devices, say in ESP32 class, may not have space or desire
to store the trust blob in the device at all, it could query a remote server on
demand to check for any trusted CA matching a given AKID and retrieve and cache
it in volatile ram.  This would use the webroot produced by the script, via tls
and a fixed CA cert outside this system.

## Format of the JIT trust blob

The trust blob layout is currently

```
00:  54 42 4c 42     Magic "TBLB"
04:  00 01           MSB-first trust blob layout version
06:  XX XX           MSB-first count of certificates
08:  XX XX XX XX     MSB-first trust blob generation unix time
0c:  XX XX XX XX     MSB-first offset from blob start of cert length table
10:  XX XX XX XX     MSB-first offset from blob start of SKID length table
14:  XX XX XX XX     MSB-first offset from blob start of SKID table
18:  XX XX XX XX     MSB-first total blob length

1c:  XX .. XX        DER certs (start at +0x1c)
  :  XX .. XX        DER cert length table (MSB-first 16-bit per cert)
  :  XX .. XX        SKID length table (8-bit per cert)
  :  XX .. XX        SKID table (variable per cert)
```

## Enabling JIT Trust

```
$ cmake .. -DLWS_WITH_TLS_JIT_TRUST=1
```

## Minimal example for JIT Trust

`minimal-examples/http-client/minimal-http-client-jit-trust` is built if JIT
Trust is enabled at cmake and `-DLWS_WITH_MINIMAL_EXAMPLES=1`.  This is based on
minimal-http-client, except the loading of the system trust bundle is defeated,
so by default it does not trust anything and cannot complete any tls connection.
It includes the mozilla trust blob as a header file when built.

It tries to do an http client connection twice, the first time fails but JIT
Trust determines which trusted CA cert is missing, retreives it from the trust
blob and creates the necessary temporary vhost with the correct CA cert(s)
trusted.  On the next retry, the connection succeeds.

## Processing of x509 AKID and SKIDs

We study each x509 cert sent by the server in turn.  We parse out the SKID and
AKID on each one and stash them (up to 4 deep).

After the initial validation fails due to lack of any trusted CA, lws has
collected all the AKID and SKIDs that were in certs sent by the server.  Since
these may be sent in any order, may be malicious, and may even contain the
(untrusted) root CA, they are sorted into a trust path using the AKID and SKID
relationships.

To cover cross-signing and cases where the root cert(s) were wrongly sent by
a misconfigured server, all of the AKIDs in the stash are queried against the
trusted CA store.  In cross-signing, multiple intermediates are provided with
the same SKID, that all match the server certificate AKID parent.  Since we
might meet certificates that trust multiple valid CAs that can validate the
certificate, we support up to three CA certs imported.

A user `lws_system_ops` handler performs the query, so it can consist of any
kind of backing store or remote lookup. Helpers are provided to query the JIT
trust mozilla blob, so the system helper is small in the typical case, just
calling lws helpers.

The results (up to three CA certs to account for cross-signing scenarios) are
collected and a 1hr TTL cache entry made for the hostname and the SKIDs of the
matched CAs, if there is no existing JIT vhost with its tls context configured
with the needed trusted CAs, one is created.

When the connection is retried, lws checks the cache for the hostname having
a binding to an existing JIT vhost, if that exists the connection proceeds
bound to that.  If there is a cache entry but no JIT vhost, one is created using
the information in the cache entry.

## Efficiency considerations

From cold, the JIT Trust flow is 

1. A sacrificial connection is made to get the server certs
2. Query the JIT Trust database for AKIDs mentioned in the certs (this may be
done asynchronously)
3. Create a temporary vhost with the appropriate trusted certs enabled in it,
   and add an entry in the cache for this hostname to the SKIDs of the CAs
   enabled on this temporary vhost
4. Retry, querying the cache to bind the connection to the right temporary vhost

An lws_cache in heap is maintained so step 1 can be skipped while hostname->
SKID items exist in the cache.  If the items expire or are evicted, it just
means we have to do step 1 again.

For a short time, the vhost created in step 3 is allowed to exist when idle, ie
when no connections are actively using it.  In the case the vhost exists and
the cache entry exists for the hostname, the connection can proceed successfully
right away without steps 1 through 3.

## APIs related to JIT Trust 

Systems that support JIT trust define an `lws_system_ops` callback
that does whatever the system needs to do for attempting to acquire
a trusted cert with a specified SKID or issuer/serial.

```
int (*jit_trust_query)(struct lws_context *cx, const uint8_t *skid, size_t skid_len, void *got_opaque);
```

The ops handler doesn't have to find the trusted cert immediately before
returning, it is OK starting the process and later if successful calling a
helper `lws_tls_jit_trust_got_cert_cb()` with the `got_opaque` from the query.
This will cache the CA cert so it's available at the next connection retry for
preloading.

An helper suitable for `ops->jit_trust_query` using trust blob lookup in .rodata
is provided in `lws_tls_jit_trust_blob_queury_skid()`, the callback above should
be called with its results as shown in the minimal example.

## Runtime tuning for JIT Trust

The context creation info struct has a couple of runtime-tunable settings
related to JIT Trust.

`.jitt_cache_max_footprint`: default 0 means no limit, otherwise the hostname->
SKID cache is kept below this many bytes in heap, by evicting LRU entries.

`.vh_idle_grace_ms`: default 0 means 5000ms, otherwise sets the length of time
a JIT Trust vhost is allowed to exist when it has no connections using it.
Notice that, eg, h2 connections have their own grace period when they become
idle, to optimize reuse, this period does not start until any h2 network
connection bound to the vhost has really closed.

## Considerations around http redirects

HTTP redirects are transactions that tell the client to go somewhere else to
continue, typically a 301 response with a Location: header explaining where to
go.

JIT Trust supports redirects to hosts with the same or different trust
requirements, each step in the redirect is treated as a new connection that will
fail, try to create a vhost with the right trust and work on the retry.

Lws rejects by default protocol downgrades (https -> http) on redirects, the
example used a context option `LCCSCF_ACCEPT_TLS_DOWNGRADE_REDIRECTS` to
override this.

## Works out of the box on recent mbedtls and openssl

No modifications are needed to either tls library.

## Compatibility Testing

A list of the top 100 sites each from the US and the ROW were combined to
produce 156 unqiue domain names [1]

The Mbedtls build of JIT trust minimal example was run against each of these
doing a GET on path `/` and restricted to h1 (`--server xxx --h1`).  In some
cases, the server at the base domain name is broken or down, as verified using
ssllabs.com as a second opinion.  These domains only resolve properly using
`www.` prefix.

In some cases the sites check the user agent and return a 4xx, these are taken
as success for this test, since there was no problem at the tls layer.

|site|h1|h2|comment|
|---|---|---|---|
|adobe.com|✓|✓||
|allegro.pl|✓|✓||
|allrecipes.com|✓|✓||
|amazon.co.jp|✓|✓||
|amazon.com|✓|✓||
|amazon.co.uk|✓|✓||
|amazon.de|✓|✓||
|amazon.fr|✓|✓||
|amazon.in|✓|✓||
|amazon.it|✓|✓||
|aol.com|✓|✓||
|apartments.com|✓|✓||
|apple.com|✓|✓||
|ar.wikipedia.org|✓|✓||
|att.com|✓|✓||
|bankofamerica.com|✓|✓||
|bbc.com|✓|✓||
|bbc.co.uk|✓|✓||
|bestbuy.com|✕|✓|redirect-> `www.` then h1: timeout, h2: 403 forbidden... geolocated?|
|booking.com|✓|✓||
|britannica.com|✓|✓||
|bulbagarden.net|✓|✓||
|businessinsider.com|✓|✓||
|ca.gov|✓|✓||
|caixa.gov.br|✕|✕|TLS trust works fine.  Continuously redirects to self... sends set-cookie that we don't return yet|
|capitalone.com|✓|✓||
|cbssports.com|✓|✓||
|cdc.gov|✓|✓||
|chase.com|✓|✓||
|chrome.google.com|✓|✓||
|cnbc.com|✓|✓||
|cnet.com|✓|✓||
|cnn.com|✓|✓||
|cookpad.com|✓|✓||
|costco.com|✕|✓|TLS trust works fine.  But with or without `www.` server does not reply within 15s on h1, sends 403 OK on h2... Curl acts the same as we do, firefox works... geolocated?||
|craigslist.org|✓|✓||
|dailymotion.com|✓|✓||
|de.wikipedia.org|✓|✓||
|dictionary.com|✓|✓||
|ebay.com|✓|✓||
|ebay.co.uk|✓|✓||
|en.wikipedia.org|✓|✓||
|epicgames.com|✓|✓||
|espn.com|✓|✓||
|es.wikipedia.org|✓|✓||
|etsy.com|✓|✓||
|expedia.com|✓|✓||
|facebook.com|✓|✓||
|fandom.com|✓|✓||
|fedex.com|✓|✓||
|finance.yahoo.com|✓|✓||
|www.foodnetwork.com|✓|✓|`www.` served correctly, base domain is misconfigured with expired cert, confirmed with ssllabs + curl|
|forbes.com|✓|✓||
|foxnews.com|✓|✓||
|fr.wikipedia.org|✓|✓||
|gamepedia.com|✓|✓||
|genius.com|✓|✓||
|glassdoor.com|✓|✓||
|globo.com|✓|✓||
|google.com|✓|✓||
|healthline.com|✓|✓||
|homedepot.com|✓|✓||
|hulu.com|✓|✓||
|hurriyet.com.tr|✓|✓||
|id.wikipedia.org|✓|✓||
|ign.com|✓|✓||
|ikea.com|✓|✓|`www.` served correctly, base domain is misconfigured with nonresponsive server, confirmed with ssllabs|
|ilovepdf.com|✓|✓||
|imdb.com|✓|✓||
|indeed.com|✓|✓||
|indiatimes.com|✓|✓||
|instagram.com|✓|✓||
|investopedia.com|✓|✓||
|irs.gov|✓|✓||
|it.wikipedia.org|✓|✓||
|ivi.ru|✓|✓||
|ja.wikipedia.org|✓|✓||
|kakaku.com|✓|✓||
|khanacademy.org|✓|✓||
|kinopoisk.ru|✓|✓||
|leboncoin.fr|✓|✓||
|linkedin.com|✓|✓||
|live.com|✓|✓||
|lowes.com|✓|✓||
|macys.com|✕|✓|TLS trust works fine.  Continuously redirects to self... `www.` same, curl acts same but OK if given -b -c, so akami cookie storage issue|
|mail.ru|✓|✓||
|mail.yahoo.com|✓|✓||
|mapquest.com|✓|✓||
|mayoclinic.org|✓|✓||
|medicalnewstoday.com|✓|✓||
|mercadolivre.com.br|✓|✓||
|merriam-webster.com|✓|✓||
|microsoft.com|✓|✓||
|msn.com|✓|✓||
|namu.wiki|✓|✓||
|nbcnews.com|✓|✓||
|netflix.com|✓|✓||
|nih.gov|✓|✓||
|nl.wikipedia.org|✓|✓||
|ny.gov|✓|✓||
|nytimes.com|✓|✓||
|ok.ru|✓|✓||
|onet.pl|✓||
|orange.fr|✓|✓||
|paypal.com|✓|✓||
|pinterest.com|✓|✓||
|pixiv.net|✓|✓||
|play.google.com|✓|✓||
|pl.wikipedia.org|✓|✓||
|www.programme-tv.net|✓|✓|OK with `www.`, without `www.` TLS trust works fine but server does not reply, same with curl|
|pt.wikipedia.org|✓|✓||
|quizlet.com|✓|✓||
|quora.com|✓|✓|||
|rakuten.co.jp|✓|✓||
|realtor.com|✓|✓||
|reddit.com|✓|✓||
|reverso.net|✓|✓||
|roblox.com|✓|✓||
|rottentomatoes.com|✓|✓||
|ru.wikipedia.org|✓|✓||
|sahibinden.com|✓|✓||
|smallpdf.com|✓|✓||
|speedtest.net|✓|✓||
|spotify.com|✓|✓||
|steampowered.com|✓|✓||
|target.com|✓|✓||
|theguardian.com|✓|✓||
|tripadvisor.com|✓|✓||
|tr.wikipedia.org|✓|✓||
|twitch.tv|✓|✓||
|twitter.com|✓|✓||
|uol.com.br|✓|✓||
|ups.com|✓|✓||
|urbandictionary.com|✓|✓||
|usatoday.com|✓|✓||
|usnews.com|✕|✓|TLS trust works fine. Needs `www.` else server doesn't respond in 15s, sends 403 on h2, Curl acts the same, geolocated?|
|usps.com|✓|✓||
|verizon.com|✓|✓||
|vk.com|✓|✓||
|walmart.com|✓|✓||
|washingtonpost.com|✓|✓||
|weather.com|✓|✓||
|webmd.com|✓|✓||
|whatsapp.com|✓|✓||
|wowhead.com|✓|✓||
|wp.pl|✓|✓||
|www.gov.uk|✓|✓||
|xfinity.com|✓|✓||
|yahoo.co.jp|✓|✓||
|yahoo.com|✓|✓||
|yandex.ru|✓|✓||
|yellowpages.com|✓|✓||
|yelp.com|✓|✓||
|youtube.com|✓|✓||
|zh.wikipedia.org|✓|✓||
|zillow.com|✓|✓||

[1]
```
wget -O- https://ahrefs.com/blog/most-visited-websites/ | grep most-visited-websites-us | \
        sed -E 's/class="column-2">/|/g' | tr '|' '\n' | \
        sed 's/<.*//g' | grep -v Domain | grep -v Josh | sort | uniq
```

# LEJP JSON Stream Parser

|||
|---|---|---|
|cmake| `LWS_WITH_LEJP`|
|Header| ./include/libwebsockets/lws-lejp.h|
|api-test| ./minimal-examples/api-tests/api-test-lejp/|
|test app| ./test-apps/test-lejp.c -> libwebsockets-test-lejp|

LEJP is a lightweight JSON stream parser.

The features are:

 - completely immune to input fragmentation, give it any size blocks of JSON as
   they become available, 1 byte, or 100K at a time give identical parsing
   results
 - input chunks discarded as they are parsed, whole JSON never needed in memory
 - nonrecursive, fixed stack usage of a few dozen bytes
 - no heap allocations at all, just requires ~500 byte context usually on
   caller stack
 - creates callbacks to a user-provided handler as members are parsed out
 - no payload size limit, supports huge / endless strings bigger than
   system memory
 - collates utf-8 text payloads into a 250-byte chunk buffer in the json parser
   context object for ease of access

## Type handling

LEJP leaves all numbers in text form, they are signalled in different callbacks
according to int or float, but delivered as text strings in the first
`ctx->npos` chars of `ctx->buf`.

For numeric types, you would typically use `atoi()` or similar to recover the
number as a host type.

## Callback reasons

The user callback does not have to handle any callbacks, it only needs to
process the data for the ones it is interested in.

|Callback reason|JSON structure|Associated data|
|---|---|---|
|`LEJPCB_CONSTRUCTED`|Created the parse context||
|`LEJPCB_DESTRUCTED`|Destroyed the parse context||
|`LEJPCB_COMPLETE`|The parsing completed OK||
|`LEJPCB_FAILED`|The parsing failed||
|`LEJPCB_VAL_TRUE`|boolean true||
|`LEJPCB_VAL_FALSE`|boolean false||
|`LEJPCB_VAL_NULL`|explicit NULL||
|`LEJPCB_PAIR_NAME`|The name part of a JSON `key: value` map pair|`ctx->buf`|
|`LEJPCB_VAL_STR_START`|A UTF-8 string is starting||
|`LEJPCB_VAL_STR_CHUNK`|The next string chunk|`ctx->npos` bytes in `ctx->buf`|
|`LEJPCB_VAL_STR_END`|The last string chunk|`ctx->npos` bytes in `ctx->buf`|
|`LEJPCB_ARRAY_START`|An array is starting||
|`LEJPCB_ARRAY_END`|An array has ended||
|`LEJPCB_OBJECT_START`|A JSON object is starting||
|`LEJPCB_OBJECT_END`|A JSON object has ended||

## Handling JSON UTF-8 strings

When a string is parsed, an advisory callback of `LECPCB_VAL_STR_START` occurs
first.  No payload is delivered with the START callback.

Payload is collated into `ctx->buf[]`, the valid length is in `ctx->npos`.

For short strings or blobs where the length is known, the whole payload is
delivered in a single `LECPCB_VAL_STR_END` callback.

For payloads larger than the size of `ctx->buf[]`, `LECPCB_VAL_STR_CHUNK`
callbacks occur delivering each sequential bufferload.

The last chunk (which may be zero length) is delievered by `LECPCB_VAL_STR_END`.

## Parsing paths

LEJP maintains a "parsing path" in `ctx->path` that represents the context of
the callback events.  As a convenience, at LEJP context creation time, you can
pass in an array of path strings you want to match on, and have any match
checkable in the callback using `ctx->path_match`, it's 0 if no active match,
or the match index from your path array starting from 1 for the first entry.

|CBOR element|Representation in path|
|---|---|
|JSON Array|`[]`|
|JSON Map|`.`|
|JSON Map entry key string|`keystring`|



## Comparison with LECP (CBOR parser)

LECP is based on the same principles as LEJP and shares most of the callbacks.
The major differences:

 - LEJP value callbacks all appear in `ctx->buf[]`, ie, floating-point is
   provided to the callback in ascii form like `"1.0"`.  CBOR provides a more
   strict typing system, and the different type values are provided either in
   `ctx->buf[]` for blobs or utf-8 text strtings, or the `item.u` union for
   converted types, with additional callback reasons specific to each type.

 - CBOR "maps" use `_OBJECT_START` and `_END` parsing callbacks around the
   key / value pairs.  LEJP has a special callback type `PAIR_NAME` for the
   key string / integer, but in LECP these are provided as generic callbacks
   dependent on type, ie, generic string callbacks or integer ones, and the
   value part is represented according to whatever comes.

# JWT support in lws

lws supports the common usage scenarios of JWS (signed) JWT generation,
parsing and transferring in and out as http cookies.  Care is taken to provide
helpers that implement the current security best practices for cookie handling
and JWT validation.  All of the common algorithms like ES512 are supported
along with JWK generation and handling apis.

The build options needed are `-DLWS_WITH_JOSE=1` `-DLWS_WITH_GENCRYPTO=1`.

Underlying JOSE primitives are exposed as apis, some JWT specific primitives
and finally a JWT-via http cookie level creation apis each building on top of
what was underneath.

The higher level APIs are provided additionally because they have the most
opportunity for implementation pitfalls like not validating alg carefully, or
not using the latest cookie security options; the provided APIs handle that
centrally for you.  If your needs vary from what the higher level apis are
doing, you can cut-and-paste out those implementations and create your own
using the public lower level apis.

## LWS JWT fields

Lws JWT uses mainly well-known fields

Field|Std|Meaning
---|---|---
iss|yes|Issuer, typically the domain like "warmcat.com"
aud|yes|Audience, typically a url path like "https://warmcat.com/sai"
iat|yes|Unix-time "Issued At"
nbf|yes|Unix-time "Not Before"
exp|yes|Unix-time "Expired"
sub|yes|Subject, eg, a username or user email
csrf|no|A random 16-char hex token generated with the JWT for use in links specific to the JWT bearer
ext|no|Application-specific JSON sub-object with whatever fields you need, eg, `"authorization": 1`

## Approach for JWT as session token

Once JWTs are produced, they are autonomous bearer tokens, if they are not kept
secret between the browser and the site, they will be accepted as evidence for
having rights to the session from anyone.

Requiring https, and various other cookie hardening techniques make it more
difficult for them to leak, but it is still necessary to strictly constrain the
token's validity time, usually to a few tens of minutes or how long it takes a
user to login and get stuff done on the site in one session.

## CSRF mitigation

Cross Site Request Forgery (CSRF) is a hacking scenario where an authorized
user with a valid token is tricked into clicking on an external link that
performs some action with side-effects on the site he has active auth on.  For
example, he has a cookie that's logged into his bank, and the link posts a form
to the bank site transferring money to the attacker.

Lws JWT mitigates this possibility by putting a random secret in the generated
JWT; when the authorized user presents his JWT to generate the page, generated
links that require auth to perform their actions include the CSRF string from
that user's current JWT.

When the user clicks those links intentionally, the CSRF string in the link
matches the CSRF string in the currently valid JWT that was also provided to
the server along with the click, and all is well.

An attacker does not know the random, ephemeral JWT CSRF secret to include in
forged links, so the attacker-controlled action gets rejected at the server as
having used an invalid link.

The checking and link manipulation need to be implemented in user code / JS...
lws JWT provides the random CSRF secret in the JWT and makes it visible to the
server when the incoming JWT is processed.

## Need for client tracking of short JWT validity times

Many links or references on pages do not require CSRF strings, only those that
perform actions with side-effects like deletion or money transfer should need
protecting this way.

Due to CSRF mitigation, generated pages containing the protected links
effectively have an expiry time linked to that of the JWT, since only the bearer
of the JWT used to generate the links on the page can use them; once that
expires actually nobody can use them and the page contents, which may anyway
be showing content that only authenticated users can see must be invalidated and
re-fetched.  Even if the contents are visible without authentication, additional
UI elements like delete buttons that should only be shown when authenticated
will wrongly still be shown 

For that reason, the client should be informed by the server along with the
authentication status, the expiry time of it.  The client should then by itself
make arrangements to refresh the page when this time is passed,
either showing an unauthenticated version of the same page if it exists, or by
redirecting to the site homepage if showing any of the contents required
authentication.  The user can then log back in using his credientials typically
stored in the browser's password store and receive a new short-term JWT with a
new random csrf token along with a new page using the new csrf token in its
links.

## Considerations for long-lived connections

Once established as authorized, websocket links may be very long-lived and hold
their authorization state at the server.  Although the browser monitoring the
JWT reloading the page on auth expiry should mitigate this, an attacker can
choose to just not do that and have an immortally useful websocket link.

At least for actions on the long-lived connection, it should not only confirm
the JWT authorized it but that the current time is still before the "exp" time
in the JWT, this is made available as `expiry_unix_time` in the args struct
after successful validation.

Ideally the server should close long-lived connections according to their auth
expiry time.

## JWT lower level APIs

The related apis are in `./include/libwebsockets/lws-jws.h`

### Validation of JWT

```
int
lws_jwt_signed_validate(struct lws_context *ctx, struct lws_jwk *jwk,
			const char *alg_list, const char *com, size_t len,
			char *temp, int tl, char *out, size_t *out_len);
```

### Composing and signing JWT

```
int
lws_jwt_sign_compact(struct lws_context *ctx, struct lws_jwk *jwk,
		     const char *alg, char *out, size_t *out_len, char *temp,
		     int tl, const char *format, ...);
```

## JWT creation and cookie get / set API

Both the validation and signing apis use the same struct to contain their
aguments.

```
struct lws_jwt_sign_set_cookie {
	struct lws_jwk			*jwk;
	/**< entry: required signing key */
	const char			*alg;
	/**< entry: required signing alg, eg, "ES512" */
	const char 			*iss;
	/**< entry: issuer name to use */
	const char			*aud;
	/**< entry: audience */
	const char			*cookie_name;
	/**< entry: the name of the cookie */
	char				sub[33];
	/**< sign-entry, validate-exit: subject */
	const char			*extra_json;
	/**< sign-entry, validate-exit:
	 * optional "ext" JSON object contents for the JWT */
	size_t				extra_json_len;
	/**< validate-exit:
	 * length of optional "ext" JSON object contents for the JWT */
	const char			*csrf_in;
	/**< validate-entry:
	 * NULL, or an external CSRF token to check against what is in the JWT */
	unsigned long			expiry_unix_time;
	/**< sign-entry: seconds the JWT and cookie may live,
	 * validate-exit: expiry unix time */
};

int
lws_jwt_sign_token_set_http_cookie(struct lws *wsi,
				   const struct lws_jwt_sign_set_cookie *i,
				   uint8_t **p, uint8_t *end);
int
lws_jwt_get_http_cookie_validate_jwt(struct lws *wsi,
				     struct lws_jwt_sign_set_cookie *i,
				     char *out, size_t *out_len);
```

## Background

libressl is another fork of Openssl.

## Example build for libressl itself

If you unpack or clone into `/path/to/libressl` and enter that dir...

```
$ mkdir build
$ cd build
$ cmake ..
$ make -j8
```

## Example build for lws against libressl

You can just build lws as you would for a specific version of openssl

```
$ mkdir build
$ cd build
$ cmake .. -DLWS_OPENSSL_LIBRARIES='/path/to/libressl/build/tls/libtls.a;/path/to/libressl/build/ssl/libssl.a;/path/to//libressl/build/crypto/libcrypto.a' -DLWS_OPENSSL_INCLUDE_DIRS=/path/to/libressl/include -DLWS_WITH_MINIMAL_EXAMPLES=1
$ make -j8
```

Libressl by default will look for a trust bundle in `/usr/local/etc/ssl/cert.pem`, you either have to
symlink this to your trust bundle if that doesnt happen to be where it is, or give your app the trusted CA
specifically as is done for MBEDTLS and WOLFSSL in the examples.

In Fedora, the system trust store can be found at `/etc/pki/tls/cert.pem`, so you can symlink it

```
$ sudo mkdir -p /usr/local/etc/ssl
$ sudo ln -sf /etc/pki/tls/cert.pem /usr/local/etc/ssl/cert.pem
```

after that you can run examples from the build dir, eg,

```
$ ./bin/lws-minimal-http-client
[2021/02/08 20:10:52:0781] U: LWS minimal http client [-d<verbosity>] [-l] [--h1]
[2021/02/08 20:10:52:0784] N: LWS: 4.1.99-v4.1.0-269-g762ef33fca, loglevel 1031
[2021/02/08 20:10:52:0784] N: NET CLI SRV H1 H2 WS IPv6-absent
[2021/02/08 20:10:52:0786] N:  ++ [wsi|0|pipe] (1)
[2021/02/08 20:10:52:0787] N:  ++ [vh|0|netlink] (1)
[2021/02/08 20:10:52:0802] N:  ++ [vh|1|default] (2)
[2021/02/08 20:10:52:1850] N:  ++ [wsicli|0|GET/h1/warmcat.com] (1)
[2021/02/08 20:10:52:2982] N:  ++ [mux|0|h2_sid1_(wsicli|0|GET/h1/warmcat.com)] (1)
[2021/02/08 20:10:52:3271] U: Connected to 46.105.127.147, http response: 200
[2021/02/08 20:10:52:3335] U: RECEIVE_CLIENT_HTTP_READ: read 4087
[2021/02/08 20:10:52:3335] U: RECEIVE_CLIENT_HTTP_READ: read 4096
[2021/02/08 20:10:52:3526] U: RECEIVE_CLIENT_HTTP_READ: read 4087
[2021/02/08 20:10:52:3526] U: RECEIVE_CLIENT_HTTP_READ: read 4096
[2021/02/08 20:10:52:3543] U: RECEIVE_CLIENT_HTTP_READ: read 4087
[2021/02/08 20:10:52:3543] U: RECEIVE_CLIENT_HTTP_READ: read 4096
[2021/02/08 20:10:52:3545] U: RECEIVE_CLIENT_HTTP_READ: read 3502
[2021/02/08 20:10:52:3546] U: LWS_CALLBACK_COMPLETED_CLIENT_HTTP
[2021/02/08 20:10:52:3546] N:  -- [wsi|0|pipe] (0) 276.019ms
[2021/02/08 20:10:52:3547] N:  -- [mux|0|h2_sid1_(wsicli|0|GET/h1/warmcat.com)] (0) 56.417ms
[2021/02/08 20:10:52:3566] N:  -- [vh|1|default] (1) 276.384ms
[2021/02/08 20:10:52:3566] N:  -- [wsicli|0|GET/h1/warmcat.com|default|h2|h2] (0) 171.599ms
[2021/02/08 20:10:52:3567] N:  -- [vh|0|netlink] (0) 277.974ms
[2021/02/08 20:10:52:3567] U: Completed: OK
```

# lws lifecycles

## Context

![context lifecycle](/doc-assets/lifecycle-context.png)

## Client wsi

![client wsi](/doc-assets/lifecycle-wsi.png)

## Server wsi

![server wsi](/doc-assets/lifecycle-server-wsi.png)

## role-specific events

role|client|server
---|---|---
http COMPLETED|`LWS_CALLBACK_COMPLETED_CLIENT_HTTP`|-
http RECEIVE|`LWS_CALLBACK_RECEIVE_CLIENT_HTTP`|`LWS_CALLBACK_RECEIVE_HTTP`
http WRITEABLE|`LWS_CALLBACK_CLIENT_HTTP_WRITEABLE`|`LWS_CALLBACK_HTTP_WRITEABLE`
http CLOSE|`LWS_CALLBACK_CLOSED_CLIENT_HTTP`|`LWS_CALLBACK_CLOSED_HTTP`
http BIND|`LWS_CALLBACK_CLIENT_HTTP_BIND_PROTOCOL`|`LWS_CALLBACK_HTTP_BIND_PROTOCOL`
http DROP|`LWS_CALLBACK_CLIENT_HTTP_DROP_PROTOCOL`|`LWS_CALLBACK_HTTP_DROP_PROTOCOL`

role|client|server
---|---|---
ws ESTABLISHED|`LWS_CALLBACK_CLIENT_ESTABLISHED`|`LWS_CALLBACK_ESTABLISHED`
ws RECEIVE|`LWS_CALLBACK_CLIENT_RECEIVE`|`LWS_CALLBACK_RECEIVE`
ws WRITEABLE|`LWS_CALLBACK_CLIENT_WRITEABLE`|`LWS_CALLBACK_SERVER_WRITEABLE`
ws CLOSE|`LWS_CALLBACK_CLIENT_CLOSED`|`LWS_CALLBACK_CLOSED`
ws BIND|`LWS_CALLBACK_WS_CLIENT_BIND_PROTOCOL`|`LWS_CALLBACK_WS_BIND_PROTOCOL`
ws DROP|`LWS_CALLBACK_WS_CLIENT_DROP_PROTOCOL`|`LWS_CALLBACK_WS_DROP_PROTOCOL`

role|client|server
---|---|---
raw ESTABLISHED|`LWS_CALLBACK_RAW_CONNECTED`|`LWS_CALLBACK_RAW_ADOPT`
raw RECEIVE|`LWS_CALLBACK_RAW_RX`|`LWS_CALLBACK_RAW_RX`
raw WRITEABLE|`LWS_CALLBACK_RAW_WRITEABLE`|`LWS_CALLBACK_RAW_WRITEABLE`
raw CLOSE|`LWS_CALLBACK_RAW_CLOSE`|`LWS_CALLBACK_RAW_CLOSE`
raw BIND|`LWS_CALLBACK_RAW_SKT_BIND_PROTOCOL`|`LWS_CALLBACK_RAW_SKT_BIND_PROTOCOL`
raw DROP|`LWS_CALLBACK_RAW_SKT_DROP_PROTOCOL`|`LWS_CALLBACK_RAW_SKT_DROP_PROTOCOL`

# lws logging

# `lwsl_` logging apis

LWS has traditionally provided logging arrangements that are not indirected
through the lws context, because logging may be needed before and after the
context existence.  For that reason the original logging arrangements are
processwide.

By default the logs are emitted on stdout, but this can be overridden
using `lws_set_log_level()` and either syslog (provided by `lwsl_emit_syslog()`)
or custom log emission is possible if you point it to your own.

Currently the following log levels are defined

|name|function|release|meaning|
|---|---|---|---|
|`LLL_ERR`|`lwsl_err()`|y|Serious operation errors anyone needs to know|
|`LLL_WARN`|`lwsl_warn()`|y|Operation errors you may need to know|
|`LLL_USER`|`lws_user()`|y|Information user code wants you to know|
|`LLL_NOTICE`|`lwsl_notice()`|y|Information about what lws is doing useful for logging|
|`LLL_INFO`|`lwsl_info()`|n|Detailed information about what lws is doing|
|`LLL_DEBUG`|`lwsl_debug()`|n|Very detailed information about what lws is doing|
|`LLL_PARSER`|`lwsl_parser()`|n|Very detailed information about parsing|
|`LLL_HEADER`|`lwsl_header()`|n|Very detailed information about header processing|
|`LLL_EXT`|`lwsl_ext()`|n|Very detailed information about ws extensions|
|`LLL_CLIENT`|`lwsl_client()`|n|Very detailed information about client connections|
|`LLL_LATENCY`|`lwsl_latency()`|n|detailed latency stats|
|`LLL_THREAD`|`lwsl_thread()`|n|detailed threadpool information|

The first four log levels are built into lws even on Release builds, the others
are only built in Debug builds.

You can select between Debug and Release builds using cmake `-DCMAKE_BUILD_TYPE=`
`DEBUG` or `Release`

`lws_set_log_level()` is used to OR together the logging bitfields you want to
see emitted, only log levels that were built in can be enabled since the code for them
is just not there otherwise.

## Finegrained control of log level build

You can deviate from the default log inclusion for release / debug by overriding it
at cmake, using `LWS_LOGGING_BITFIELD_SET` and `LWS_LOGGING_BITFIELD_CLEAR`.

For example you can set `-DLWS_LOGGING_BITFIELD_SET="LLL_INFO|LLL_DEBUG"`, which will
cause those log level traces to be built in even in Release mode.  Clear works
similarly to defeat build of specific log levels.

## Object tags in lws

Commonly logging wants to refer to an object in a repeatable way, the usual way to
do this is with `%p` to print the object pointer.  But this has a couple of drawbacks,
first the same memory may be freed and reallocated for a different instance of the same
or another object, causing confusion, and second when multiple processes are allocating
objects and logging, the same address may be allocated in different process also causing
confusion.

Lws has introduced unique tag strings to refer to object identity in logging instead, these
contain various information such as a 64-bit ordinal for the group the object belongs
to that won't repeat even if reallocated to the same address (until 2^64 allocations,
anyway).

Tags are fixed at object creation time for the whole object lifetime, although in some
cases the tag may be appended to... accepted server wsis for example don't have much
information available to form the tag until they start to indicate what they want to
do.

At their simplest the tags look like this (in a log indicating creation)

```
[2020/12/27 08:49:19:2956] N:  ++ (4) [wsi|5|h2]
```

It means a wsi has been created with the tag `[wsi|5|h2]`, and after that, there are 4
active objects in the wsi group.

The corresponding object destruction log with the tag is

```
[2020/12/27 08:49:24:4226] N:  -- (3)   5.126s [wsi|5|h2]
```

it indicates the object's tag, that it lived for 5.126s and after its destruction,
there are 3 objects in its group left.

### Compound tags

If the object has bindings, the tag can reflect that, eg

```
[2020/12/27 08:49:19:4787] N:  ++ (2) [wsiSScli|6|d_h1]
[2020/12/27 08:49:19:4793] N:  ++ (2) [wsicli|6|GET/h1/httpbin.org/([wsiSScli|6|d_h1])]
```

the first log is describing a proxied SS client connection at the proxy, and the second
is a wsi bound to the SS object from the first log to do the outgoing client action.

## Tags in user code

When user code wants to refer to a tagged object like a wsi or vhost, there are helpers
that return a `const char *` containing the tag

|tag accessors|
|---|
|`lws_wsi_tag(wsi)`|
|`lws_vh_tag(vh)`|
|`lws_ss_tag(h)`|

# New logging context apis

From v4.3 on lws additionally provides wrappers that issue logs into a
"log context" object, one of these is embedded in the lws_context, lws_vhost,
wsi, ss and sspc handles.  These follow the same general approach as before, but
allow logs to be issued in "the context" of any of those objects, and to fall
back sanely if the object pointer is NULL.

The traditional process scope logs and emit management remain available as
before, and if you do not set custom log contexts, the new log apis use the
processwide log context emit and mask as before too.

Here's a summary of the differences:

|Traditional process scope logs|New log context apis|
|---|---|
|Single processwide log context|Defaults to processwide, but object can use custom log contexts|
|Single processwide emit function|Emit function per log context|
|Single processwide log mask|log mask is in log context, objects can be bound to custom log contexts at creation time|
|Require trailing `\n` in format|Trailing `\n` added if not present|
|Manual `__func__`|`__func__` added in wrapper macros automatically|
|Manual tag addition|Object tag prepended automatically|
|No hierarchy|Log contexts may refer to parent log contexts, which may prepend to child logs|
|Macros per level (eg, `lwsl_err(...)`)|Macros per object type / level (eg, `lwsl_wsi_err(wsi, ...)`)|

In addition to being able to control the emit function and log level for
individual log contexts, eg, for a particular wsi, the log functions understand
how to prepend object-specific information such as tags and `__func__`
automatically.  They also do not need a trailing `\n` in the format string.  So
the new context aware logs remove boilerplate from the logging calls while
making the log information more consistent.

So comparing this kind of logging the processwide and log context aware ways:

```
[2021/06/25 09:39:34:7050] N: [669282|wsicli|4|GET/h1/libwebsockets.org|default]: _lws_generic_transaction_completed_active_conn:  ...
```

|Type|Example code|
|---|---|
|Process scope apis|`lwsl_notice("%s: %s: mylog %d\n", __func__, lws_wsi_tag(wsi), n);`|
|New log context apis|`lwsl_wsi_notice(wsi, "mylog %d", n);`|

The log context / object-aware apis do not replace the processwide logging but
augment it, and the new apis default to use the original processwide emit
function and log mask, so the behaviours are the same.  The original processwide
log apis themselves are unchanged.

At lws_context creation time, you can set the context info `.log_cx` to a user
defined log context which is inherited by objects created in that lws_context by
default.  Vhost creation, wsi creation and ss / sspc creation all allow passing
a user log_cx to customize how logs for that object are handled.

## Using the new logging apis

This table describes the different ways to issue an ERROR verbosity log, it
works the same for info, notice, warn, etc.

|Scope|Api example|Functionality|
|---|---|---|
|Old, Processwide|lwsl_err(...)|Traditional processwide error log|
|lws_context|lwsl_cx_err(context, ...)|error log bound to lws_context|
|lws_vhost|lwsl_vhost_err(vh, ...)|error log bound to lws_vhost|
|lws_wsi|lwsl_wsi_err(wsi, ...)|error log bound to wsi|
|lws_ss|lwsl_ss_err(handle, ...)|error log bound to secure stream|

Similarly hexdumps can be bound to different log contexts

|Scope|Api example|Functionality|
|---|---|---|
|Old, Processwide|lwsl_hexdump_err(...)|Traditional processwide error hexdump|
|lws_context|lwsl_hexdump_cx_err(context, ...)|error hexdump bound to lws_context|
|lws_vhost|lwsl_hexdump_vhost_err(vh, ...)|error hexdump bound to lws_vhost|
|lws_wsi|lwsl_hexdump_wsi_err(wsi, ...)|error hexdump bound to wsi|
|lws_ss|lwsl_hexdump_ss_err(handle, ...)|error hexdump bound to secure stream|

## Creating and using custom log contexts

The log context object is public, in `libwebsockets/lws-logs.h`, currently it
is like this

```
typedef void (*lws_log_emit_t)(int level, const char *line);
typedef void (*lws_log_emit_cx_t)(struct lws_log_cx *cx, int level,
				  const char *line, size_t len);
typedef void (*lws_log_prepend_cx_t)(struct lws_log_cx *cx, void *obj,
				     char **p, char *e);
typedef void (*lws_log_use_cx_t)(struct lws_log_cx *cx, int _new);
typedef struct lws_log_cx {
	union {
		lws_log_emit_t		emit; /* legacy emit function */
		lws_log_emit_cx_t	emit_cx; /* LLLF_LOG_CONTEXT_AWARE */
	} u;
	lws_log_use_cx_t		refcount_cb;
	/**< NULL, or a function called after each change to .refcount below,
	 * this enables implementing side-effects like opening and closing
	 * log files when the first and last object binds / unbinds */
	lws_log_prepend_cx_t		prepend;
	/**< NULL, or a cb to optionally prepend a string to logs we are a
	 * parent of */
	struct lws_log_cx		*parent;
	/**< NULL, or points to log ctx we are a child of */
	void				*opaque;
	/**< ignored by lws, used to pass config to emit_cx, eg, filepath */
	void				*stg;
	/**< ignored by lws, may be used a storage by refcount_cb / emit_cx */
	uint32_t			lll_flags;
	/**< mask of log levels we want to emit in this context */
	int32_t				refcount;
	/**< refcount of objects bound to this log context */
} lws_log_cx_t;
```

The emit function is a union because the traditional logs and the old emit
functions are also implemented using the new log contexts internally.  For
new log context-aware code, you would use `.u.emit_cx` and set the flag
`LLLF_LOG_CONTEXT_AWARE` on `.lll_flags`.

Lws also exports some common emit and refcount functions so you don't have to
reinvent the wheel

|Dest|emit member|`.lll_flags`|emit|`.refcount_cb`|`.opaque`|
|---|---|---|---|---|---|
|stderr|`.u.emit`|-|`lwsl_emit_stderr`|NULL|NULL|
|file|`.u.emit_cx`|`LLLF_LOG_CONTEXT_AWARE`|`lws_log_emit_cx_file`|`lws_log_use_cx_file`|`(const char *)filepath`|

For example, a custom log context that emits to a configurable file can be
declared like this (lws exports the needed helpers already)

```
static lws_log_cx_t my_log_cx = {
        .lll_flags      = LLLF_LOG_CONTEXT_AWARE |
                          LLL_ERR | LLL_WARN | LLL_NOTICE | LLL_USER,
        .refcount_cb    = lws_log_use_cx_file,
        .u.emit_cx      = lws_log_emit_cx_file,
        .opaque	        = "/tmp/mylogpath.log" /* also settable at runtime */
};
```

To bind the lws_context to this log context, set `log_cx` in the context
creation info struct

```
	info.log_cx = &my_log_cx;
```

### Log context hierarchy

Log contexts may also point to a parent log context... the top level log context
defines the emit function to be used, but parent log contexts are consulted by
calling their prepend function if any, to annotate logs with information from
parent levels.

### Log context prepend function

Logs contexts may define a "prepend" function callback, that knows how to
represent the object in a brief string to be prepended to other logs.  For
example the wsi-aware log context layer knows how to provide the wsi tag
when called.

Prepend functions should add `:<space>` after their output, if any, since these
will appear before the start of other logs.

### Log context opaque member

The `.opaque` member is available for passing in configuration to the emit and
refcount_cb members.  Lws does not use this itself at all.

### Log context refcounting

An expected use for custom log contexts is emitting to a specific file, and
then binding one or more objects to that log context.  Since it's too expensive
to keep opening and closing the output file per log, it means we need to know
when we bind to the first object and unbind from the last, so we can keep the
file handle open.

For this reason the log contexts have a refcount, and an opaque `void *stg`
availble for the emit and refounct_cb to use how they see fit, eg, for storing
the output log file descriptor.

# lws_cache: Flexible single and multilevel caching

lws_cache implements a single- or multi-level cache for generic payload items
that are **keyed by a unique string**.

![lws_cache overview](../doc-assets/lws_cache-1.png)

L1 cache is always stored on heap, but it may be hooked up to additional levels
of cache objects with different backing storage.  The last level always contains
a complete set of cached items, earlier levels may be empty or contain a partial
set of objects.

User code can define its own subclassed lws_cache objects with custom storage
formats and media, while being able to take advantage of a suitably-sized L1
heap cache to minimize the cost of repeated access.

![lws_cache overview](../doc-assets/lws_cache-2.png)

You can find examples of how to create, use and destroy single and multilevel
caches in `minimal-examples/api-tests/api-test-lws_cache`

## Cache size restriction, LRU and TTL

The max heap footprint of its items and max number of items can be capped.  LRU
tracking is performed so the least recently relevant items are evicted first.
It's also possible to limit the maximum size of any single payload.

Time To Live (TTL) tracking is also performed automatically, so cached items
auto-expire if a non-zero TTL is provided when the object is created.  A user
callback can be defined to get called when an item is about to be removed from
a particular cache level, in case any housekeeping needed.

## Atomicity

Items in L1 can be accessed in heap casually and reliably if the following is
borne in mind:

 - Any return to the event loop may perform removal of cache items due to TTL
expiry

 - Any operation that writes new items may evict items from non-last
cache levels which have limits to the footprint or item count to make room for
it, using LRU ordering.

In short process cache results before returning to the event loop or writing
or removing items in the cache.

## Cache creation

Caches are created using an info struct `struct lws_cache_creation_info`
that should be zeroed down.  Most members are optional and can be left at zero,
a pointer to the lws_context and a short cache name are mandatory.

```
struct lws_cache_ttl_lru *
lws_cache_create(const struct lws_cache_creation_info *info);
```

How caches work is defined by an "ops struct" that the cache is bound to at
creation time.  `lws_cache_ops_heap` ops struct is provided by lws, you can
define your own to implement your own specialized cache level.  See
`./include/libwebsockets/lws-cache-ttl.h` for the definition.

## Cache destruction

Created cache levels should be destroyed when you are finished with them.

```
void
lws_cache_destroy(struct lws_cache_ttl_lru **cache);
```

For L1, in heap, this frees any allocations.  For other levels, eg, with file
storage for the items, this would close the file and leave any entries as they
are.

## Writethrough

```
int
lws_cache_write_through(struct lws_cache_ttl_lru *cache,
			const char *specific_key, const uint8_t *source,
			size_t size, lws_usec_t expiry, void **ppay);
```

The combined caches are always accessed via the L1 cache, writing new items is
done at L1 and writes through to each cache layer immediately, so new items go
into the backing store without delay, but are available from heap for read.

If existing keys are rewritten, the previous item of the same key is deleted
from all levels of the cache before writing the new one.

## Removal

Removal also is performed at all cache levels at once.

```
int
lws_cache_item_remove(struct lws_cache_ttl_lru *cache, const char *wildcard_key);
```

internally earlier cache levels can evict cached items just at their level, but
this is triggered automatically and not by api.

A wildcard key is supported, removing all items matching, eg "myitem*".

## Get by key

```
int
lws_cache_item_get(struct lws_cache_ttl_lru *cache, const char *specific_key,
		   const void **pdata, size_t *psize);
```

Apis are provided to get the blob related to a specific key, if it exists at
any cache layer.  Again this should use L1, it will bring a copy of the item
into L1 if one is not already there, so it can be accessed from heap.

## Lookup with wildcards

```
int
lws_cache_lookup(struct lws_cache_ttl_lru *cache, const char *wildcard_key,
		 const void **pdata, size_t *psize);
```

lws_cache also supports **lookup** queries that contain wildcards or otherwise match
on multiple keys according to cache-specific rules.  These queries do not return
a single item, instead they return lists of keys that match, in a blob of its
own that is also cached in L1.

The user can walk the lookup results blob using a provided helper api

```
int
lws_cache_results_walk(lws_cache_results_t *walk_ctx);
```

After recovering each result key this way, the user code can use the _get api
to access the blob for each indiviudally.

The lookup results themselves are cached in L1, any new key that matches the
wildcard lookup in any cached results, or any deletion of items with keys
matching the cached wildcard lookup invalidate the affected cached lookup
results so they will be regenerated next time.

In the typical case after a lookup, at least for a while the lookup results blob
and all items mentioned in the lookup results will already be in L1 and cheaply
accessible.

## Expunging

An api is also provided to "expunge" or completely empty all cache levels and
corresponding backing stores.

```
int
lws_cache_expunge(struct lws_cache_ttl_lru *cache);
```

## `lws_conmon` apis

`LWS_WITH_CONMON` build option enables `lws_conmon` apis for user code... these add
some staticistic and information to client connections that can use useful for devices
to introspect how the connection to their servers is actually performing.

The public apis can be found in `libwebsockets/lws-conmon.h`.

A struct is provided that describes

 - the peer sockaddr the wsi actually connected to, if any

 - a deep copy of the aggregate DNS results (struct addrinfo list) that the
   client had access to for the peer

 - the number of us dns lookup took

 - the number of us the socket connection took

 - the number of us the tls link establishment took

 - the number of us from the transaction request to the first response, if
   the protocol has a transaction concept

Because the user code may want to hold on to the DNS list for longer than the
life of the wsi that originated it, the `lws_conmon_wsi_take()` api allows
the ownership of the allocated list to be transferred to the user code (as
well as copying data out into the user's struct so it no longer has any
dependency on wsi lifetime either).  The DNS list copy in the struct must be
released at some point by calling `lws_conmon_release()`, but that
can be at any time afterwards.

The lws-minimal-http-client example shows how user code can use the apis, build
lws with the `LWS_WITH_CONMON` cmake option and run with `--conmon` to get a
dump of the collected information.

# lws_dll Doubly-linked list

## Introduction

Lws supports two kinds of doubly-linked list, `lws_dll` and `lws_dll2`.

Unless memory is at a big premium, or it has to work on lws < v3.2, it's
best to simply use `lws_dll2`.

![lws_dll overview](../doc-assets/lws_dll.svg)

## How to use

The basics are the same for lws_dll and lws_dll2.

The list objects point only to themselves, and you use the `lws_container_of`
macro to get a pointer to your struct that contains the list object.  Doing
it this way

 - the list object does not have to be the first thing in your struct
 
 - your struct can contain multiple list objects and appear on lists belonging
   to multiple owners simultaenously,
   
### lws_dll Minimal example

```
struct mystruct {
	....
	lws_dll list;
	...
};

lws_dll owner;
```

Adding a mystruct to the owner list (...add_tail() works the same way but adds
to the other end of the list)

```
	struct mystruct *p;
	
	...
	
	lws_dll_add_head(&p->list, &owner);
```

Removing the list object from its owner

```
	lws_dll2_remove(&p->list, &owner);
```

If you have a `struct lws_dll *d` pointing to `list` in struct mystruct, you can
convert it to a `struct mystruct *p` ike this

```
	struct mystruct *p = lws_container_of(d, struct lws_dll, list);
```

### lws_dll2 Minimal example


```
struct mystruct {
	....
	lws_dll2 list;
	...
};

lws_dll2_owner owner;
```

Adding a mystruct to the owner list (...add_tail() works the same way but adds
to the other end of the list)

```
	struct mystruct *p;
	
	...
	
	lws_dll2_add_head(&p->list, &owner);
```

Removing the list object from its owner (notice compared to lws_dll, it doesn't
need to be told the owner)

```
	lws_dll2_remove(&p->list);
```

If you have a `struct lws_dll2 *d` pointing to `list` in struct mystruct, you
can convert it to a `struct mystruct *p` ike this

```
	struct mystruct *p = lws_container_of(d, struct lws_dll2, list);
```

## Summary Comparing lws_dll and lws_dll2

 - both offer a doubly-linked list object, and (since v3.2) track both the
   head and tail in an "list owner" object
   
 - both are initalized by memsetting to 0

 - for `lws_dll`, it reuses an `lws_dll` as the "owner", for `lws_dll2`, there's a
   specific `lws_dll2_owner` structure for that
   
 - `lws_dll2_owner` also keeps count of the number of list elements
 
 - `lws_dll2` knows which owner's list it is participating on.  So it can remove
   itself and update the owner without the caller needing to know its owner.
   In the case there are several potential owners list objects may be on, this
   is very convenient.

 - `lws_dll` is simpler and has a smaller footprint (two pointers per entry vs
   three).  But you have to know the exact list owner to perform operations on
   it.
 
## apis
 
|function|lws_dll|lws_dll2|
|---|---|---|
|add entry at head|`void lws_dll_add_head(struct lws_dll *d, struct lws_dll *phead)`|`void lws_dll2_add_head(struct lws_dll2 *d, struct lws_dll2_owner *owner)`|
|add entry at tail|`void lws_dll_add_tail(struct lws_dll *d, struct lws_dll *phead);`|`void lws_dll2_add_tail(struct lws_dll2 *d, struct lws_dll2_owner *owner)`|
|remove entry from its owning list|`void lws_dll_remove_track_tail(struct lws_dll *d, struct lws_dll *phead)`|`void lws_dll2_add_tail(struct lws_dll2 *d, struct lws_dll2_owner *owner)`|
|get owner|(not supported)|`struct lws_dll2_owner * lws_dll2_owner(const struct lws_dll2 *d)`|
|check if item is detached from any list|`lws_dll_is_detached(struct lws_dll *d, struct lws_dll *phead)|int lws_dll2_is_detached(const struct lws_dll2 *d)`|
|iterate through items on list|`int lws_dll_foreach_safe(struct lws_dll *phead, void *user, int (*cb)(struct lws_dll *d, void *user))|int lws_dll2_foreach_safe(struct lws_dll2_owner *owner, void *user, int (*cb)(struct lws_dll2 *d, void *user))`|