Scarlet Industries

TLS client

Client TLS for the Scarlet programming language. An encrypted connection is a different type from a cleartext one, and certificate verification cannot be disabled.

Connect

tls.connect opens a connection to host:port and completes the handshake before it returns. host is both the address connected to and the name the certificate is verified against. 

pub fn connect(host String, port Int) Result(TlsSocket, TlsError)
pub fn handshake(sock Socket, server_name String) Result(TlsSocket, TlsError)

The handshake finishes here rather than lazily on the first read, so a certificate failure is reported by the call that made the connection instead of surfacing later at an unrelated read.

connect has one name and uses it twice: host is dialled and host is verified. To connect to one address and verify a different name, take a Socket from net.connect and hand it to tls.handshake with the name to verify. That is the path a proxy takes, and the path a pool takes when it dials an address it resolved once.

The module is scarlet/net/tls. It imports as any other standard library module. 

import scarlet/net/tls
import scarlet/net/tls.{TlsError, TlsSocket, Transport}

Session resumption

Sessions are cached in memory for the life of the program and shared across every Scarlet process in it. A second connection to a host the program has already reached resumes rather than starting a full handshake.

The cache is never written to disk. A session ticket is key material, and persisting it is not a decision a runtime takes on a program's behalf. Nothing resumes across separate runs of a program.

Two types

TlsSocket and Socket are different types. socket.write does not accept a TlsSocket, and tls.write does not accept a Socket. Writing cleartext where encryption was intended is a type error at compile time, not a runtime check and not a packet capture at review time. 

pub type TlsConnection

pub type TlsSocket {
	conn TlsConnection
	peer SocketAddress
	server_name String
}

A cleartext socket and an encrypted one are not one type with a flag. TlsConnection is opaque: the VM constructs it, it carries no fields, and it is a handle into the owning scheduler's connection table. The VM carries the same distinction in the handle's kind, so the two cannot be confused even by a handle that outlives its upgrade.

server_name is retained because it is the identity the certificate was checked against. A caller that pools connections keys on it. Reusing a connection opened for one host to talk to another is exactly the confusion the certificate check exists to prevent.

Upgrade a socket

tls.handshake is the one door into TlsSocket, and it consumes the cleartext Socket. The VM re-keys the connection under a TLS handle, so the Socket that was passed in is stale afterwards and every cleartext operation on it errs with NotConnected. That is what stops a caller from holding on to the plaintext handle and writing around the encryption.

connect is not the only way to arrive at a socket that must be secured. A proxied CONNECT tunnel and a STARTTLS upgrade both hand back a live cleartext socket that has to become a TLS one, which is why handshake is public.

Certificate verification

Certificate verification cannot be disabled. There is no flag, no argument, and no TlsError variant for a check that was skipped. A TLS client that can be talked into accepting any certificate is worse than no TLS client, because it still looks like it works.

A test that needs to trust a private CA adds that CA to the trust store the machine already has. It does not ask the language to stop checking.

Verification is by name. An IP literal as server_name errs with InvalidServerName: a certificate is issued for a DNS name, and an address alone has nothing to verify against.

Errors

Certificate problems are separated from transport problems because they call for different responses. Transport is retryable. A certificate failure is not: a second attempt reaches the same certificate. 

pub type TlsError {
	CertificateUnknownIssuer
	CertificateExpired
	CertificateNotYetValid
	CertificateRevoked
	HostnameMismatch
	BadCertificate
	ProtocolError
	HandshakeFailed
	InvalidServerName
	Transport(cause NetError)
}
fn fetch(host String, target String) String {
	match request(host, target) {
		Ok(body) -> body
		// The network failed. A retry reaches a new connection.
		Err(Transport(e)) -> 'transport failed: ${string.inspect(e)}'
		// The chain did not end at a trusted root. A retry reaches the same
		// certificate. Add the CA to the machine's trust store instead.
		Err(CertificateUnknownIssuer) -> 'untrusted issuer'
		// The chain verified for some other name. Never retried.
		Err(HostnameMismatch) -> 'certificate is not issued for ${host}'
		Err(e) -> 'tls failed: ${string.inspect(e)}'
	}
}

Read and write

pub fn read(c TlsSocket, max Int) Result(Read, TlsError)
pub fn read_exact(c TlsSocket, count Int) Result(Binary, TlsError)
pub fn write(c TlsSocket, data Binary) Result(Nil, TlsError)
pub fn close(c TlsSocket) Result(Nil, TlsError)

read parks the calling process until at least one byte is decrypted or the peer closes. max bounds the plaintext handed back, not the ciphertext read from the socket. A TLS record is decrypted whole or not at all, so a read that returns fewer bytes than arrived has the remainder buffered and ready, not lost.

Read is the type scarlet/net/socket returns: Data(bytes) or Closed. A clean peer close is its own constructor, so failing to handle it is an exhaustiveness error rather than a loop on a dead connection.

read_exact parks until all count bytes have arrived and errs with Transport(UnexpectedEof) if the peer closes first. A count of zero or less returns Ok(<<>>) without touching the connection.

A TLS read carries no deadline. scarlet/net/socket has read_within and read_until. scarlet/net/tls has neither, so a TLS read parks until plaintext arrives or the peer closes.

write encrypts data and drains the ciphertext to the socket before it returns, rather than leaving it buffered in the TLS session for a later call to push.

close sends a close_notify alert and closes the connection. The alert is what tells the peer the stream ended on purpose. Without it a peer cannot tell an orderly shutdown from a truncation attack, and well-behaved ones report an error.

An HTTPS request

There is no HTTP client in the standard library. An HTTPS request is request bytes written by hand over a TlsSocket and a response read straight back off the connection, which is all an HTTPS client is. 

// Read until the peer closes. One tls.read is not a message boundary: it hands
// back whatever has been decrypted so far, so a client that trusts a single
// read will sooner or later see half a response. `Connection: close` makes the
// end of the response the end of the stream.
fn read_to_close(conn TlsSocket, acc Binary) Binary {
	match tls.read(conn, 65536) {
		Ok(Data(bytes)) -> read_to_close(conn, binary.append(acc, bytes))
		Ok(Closed) -> acc
		Err(_) -> acc
	}
}

// There is no HTTP client in the standard library, so the request bytes are
// written by hand and the response is read straight back off the connection.
// tls.connect finishes the handshake before it returns, so a certificate
// failure is the error this function reports.
fn request(host String, target String) Result(String, TlsError) {
	head = 'GET ${target} HTTP/1.1\r\nHost: ${host}\r\nConnection: close\r\n\r\n'
	conn <- result.then(tls.connect(host, 443))
	tls.write(conn, binary.from_string(head)) or Nil
	response = read_to_close(conn, <<>>)
	tls.close(conn) or Nil
	Ok(binary.to_string(response) or '<not utf-8>')
}

One tls.read is not a message boundary: it hands back whatever has been decrypted, so a client that trusts a single read will sooner or later see half a response. Connection: close makes the end of the response the end of the stream, which is the simplest framing there is.

tls.connect completed the handshake before returning, so the certificate was checked against host before the first request byte went out. The TlsError this function reports is that check.