TLS Acceleration with relayd

(redirected from Openbsd.Relayd)

relayd can provide secure TLS acceleration.

Features:

  1. Load balancer
  2. Application layer gateway
  3. Transparent proxy
  4. SSL/TLS gateway

Pros:

  1. Simpler to configure than haproxy or stunnel
  2. Tight integration with OpenBSD's packet filter
  3. Secure

Introduction

Sometimes a server does not provide TLS encryption by default. For example, leafnode lacks TLS encryption by default. It serves all its content as plaintext, exposing users' data to eavesdropping. To protect your users, you can use TLS acceleration using relayd.

Another time we will need relayd is when we have two servers that need to both listen on the same socket? (the same IP address and port). For example, the znc web panel and openhttpd both normally serve HTTPS content on the same IP address and on the same port 443. It is not possible, however, to have two servers bind to the same socket. To share the same socket, it is necessary to use relayd with TLS Server Name Indication (SNI).

Check Server in Plaintext

Before using relayd, you must make sure your server can serve its content in plaintext. If the server doesn't respond to plaintext requests, relayd won't work, either.

  1. For providing webhosting, please see the openhttpd hosting guide. Afterwards, test the setup using the telnet troubleshooting guide.
  2. For providing public bouncers, please see the znc chroot guide. Afterwards, test the setup using the netcat troubleshooting guide.

WARNING: If you are using relayd for TLS acceleration for openhttpd, make sure openhttpd does not have a listener on port 443. Look for blocks like the ones below:

server "example.com" {
        listen on * tls port 443
        tls {
                certificate "/etc/ssl/example.com.fullchain.pem"
                key "/etc/ssl/private/example.com.key"
        }
        location "/pub/*" {
                directory auto index
        }
        location "/.well-known/acme-challenge/*" {
                root "/acme"
                request strip 2
        }
}

Notice the line listen on * tls port 443 and the tls {...} block. If httpd is listening on port 443 while relayd is running, the reverse proxy will fail to forward for android, iOS, and other devices!

In the configuration below, we assume you are following the openhttpd hosting guide and listening on port 80.

Request SSL Certs

Make sure you have the SSL certs you need for the domains you want to provide TLS acceleration for. Request them using acme-client if you have not already.

By default, relayd searches /etc/ssl/name:port.crt and /etc/ssl/name:port.key for the public/private keypair. So, we will create symlinks:

$ doas ln -s /etc/ssl/example.com.fullchain.pem /etc/ssl/example.com:443.crt
$ doas ln -s /etc/ssl/private/example.com.key /etc/ssl/private/example.com:443.key

Edit relayd.conf

Let's create /etc/relayd.conf. Here is what we will put, one block at a time:

ip4="192.168.1.1"
ip6="2001:db8::"
table <service1> { 127.0.0.1 }
table <service2> { 127.0.0.1 }
log connection

Replace ip4 and ip6 with the actual IPv4 and IPv6 address you want to listen on. Make sure the IPv4 is DDoS-filtered if you have that option.

http protocol https {
        match request header append "X-Forwarded-For" value "$REMOTE_ADDR"
        match request header append "X-Forwarded-By" \
            value "$SERVER_ADDR:$SERVER_PORT"
        match request header set "Connection" value "close"
        tcp { sack, backlog 128 }
        tls { keypair service1.example.com
              keypair service2.example.com }
        match request header "Host" value "service1.example.com" forward to <service1>
        match request header "Host" value "service2.example.com" forward to <service2>
}

Here we define how to handle the http protocol. We add X-Forwarded-For, X-Forwarded-By, and Connection headers to HTTP requests before forwarding it to openhttpd.

We turn on selective acknowledgments and set the maximum queue to 128 connections in the tcp block.

We then define the keypair names. Here's where relayd searches for them:

NamePublic CertPrivate Key
name/etc/ssl/name:443.crt/etc/ssl/private/name:443.key
service1.example.com/etc/ssl/service1.example.com:443.crt/etc/ssl/private/service1.example.com:443.key
service2.example.com/etc/ssl/service2.example.com:443.crt/etc/ssl/private/service2.example.com:443.key

The last two lines forward to the proper service based on the Host HTTP header.

relay wwwtls {
        listen on $ip4 port 443 tls
        protocol https
        forward to <service1> port 80 check icmp
        forward to <service2> port 80 check icmp
}
relay www6tls {
        listen on $ip6 port 443 tls
        protocol https
        forward to <service1> port 80 check icmp
        forward to <service2> port 80 check icmp
}

We create two relays, one for IPv4 and another for IPv6. Both of them listen on port 443 using TLS. They handle use the protocol template for https and forward to the proper service on port 80 (see the above openhttpd hosting guide). Both check ICMP to see if the service is available.

Complete relayd.conf

Here is the entire /etc/relayd.conf without commentary:

ip4="192.168.1.1"
ip6="2001:db8::"
table <service1> { 127.0.0.1 }
table <service2> { 127.0.0.1 }
log connection

http protocol https {
        match request header append "X-Forwarded-For" value "$REMOTE_ADDR"
        match request header append "X-Forwarded-By" \
            value "$SERVER_ADDR:$SERVER_PORT"
        match request header set "Connection" value "close"
        tcp { sack, backlog 128 }
        tls { keypair service1.example.com
              keypair service2.example.com }
        match request header "Host" value "service1.example.com" forward to <service1>
        match request header "Host" value "service2.example.com" forward to <service2>
}
relay wwwtls {
        listen on $ip4 port 443 tls
        protocol https
        forward to <service1> port 80 check icmp
        forward to <service2> port 80 check icmp
}
relay www6tls {
        listen on $ip6 port 443 tls
        protocol https
        forward to <service1> port 80 check icmp
        forward to <service2> port 80 check icmp
}

Login class permissions

If you have a large number of TLS certs, you will need to increase the maximum number of files that relayd can open. Add this to the bottom of /etc/login.conf:

relayd:\
        :openfiles=4096:\
        :stacksize-cur=96M:\
        :stacksize-max=96M:\
        :tc=daemon:

Then we must make sure there is no login.conf.db database:

$ doas rm /etc/login.conf.db

Starting relayd

$ doas rcctl enable relayd
$ doas rcctl start relayd

WARNING: Make sure that packet filter is enabled! relayd will not run if pf is disabled. You can enable it by typing:

$ doas pfctl -e

To test relayd, we'll use openssl:


Troubleshooting

If relayd fails to start, you will see this message:

relayd(failed)

First, we check the conf file to see if there are any errors:

$ doas relayd -n

When properly configured, relayd will say configuration OK.

Sample errors:

  1. /etc/relayd.conf:NN: cannot load keypair example.com for relay wwwtls
    Check line NN. Your keypair may be missing, have the wrong permissions, or are not labeled correctly.
  2. /etc/relayd.conf:NN: syntax error

Check line NN for syntax errors.

To turn on debugging, first stop any running instances of relayd then run it in the foreground:

$ doas rcctl stop relayd
$ doas relayd -dvv

-d is for debug and -v is to increase verbosity.

log connection

WARNING: This may produce a verbose output which can dramatically increase the size of your /var/log/daemon, especially on busy networks. To avoid this, simply have your syslogd send all relayd messages into its own file. To that, see here.

In addition to splitting relayd logs to its own file, you may wish to create a new entry in your /etc/newsyslog.conf to handle log rotation for your relayd.

common errors

  1. Make sure httpd is not also listening on port 443
  2. Make sure both www.ircnow.org and bnc.ircnow.org have real dns records
  3. Make sure nsd is set up properly
  4. Make sure znc is listening on port 1338
  5. Make sure packet filter is turned on