Redis client

Talk to Redis from the Scarlet programming language. 275 commands, typed errors, and nothing that can desync a connection quietly.

Add the library

The client is a Scarlet library. Vendor it as a git submodule.

git submodule add https://github.com/scarletindustries/redis_client lib/redis

Check the library out into a path Scarlet parses as an identifier. lib/redis and vendor/scarlet_redis parse. vendor/scarlet-redis does not: the hyphen reads as subtraction.

Connections

with_conn connects, runs your function, and closes the connection however the function ends: success, failure, or a connect that never happened. A failure to connect and a failure inside the body both come back as the same RedisError.

import scarlet/result
import ./lib/redis/redis.{Conn}
import ./lib/redis/strings

fn greet(c Conn) Result(Nil, redis.RedisError) {
	_ <- result.then(strings.set(c, 'greeting', 'hello'))
	value <- result.then(strings.get(c, 'greeting'))
	println(value or '(nil)')
	Ok(Nil)
}

pub fn main() {
	redis.with_conn('127.0.0.1', 6379, greet)
}

connect dials and then sends HELLO 3 before handing the connection back, so a Conn in hand is always one whose replies this client can read. A server that refuses RESP3, which is anything before Redis 6, would keep answering in RESP2, where a map is a flat array and a double is a bulk string, and the decoders would misread those shapes quietly. Failing at connect is the only honest option.

One connection per process. Two processes sharing a socket would read each other's replies. A Conn is either one socket or a handle to a pool built to be shared; both answer the same command functions, so strings.get(c, k) works against either. A pool cannot promise two consecutive commands land on the same connection, which is what WATCH and MULTI need; pool.with_conn leases a single connection for a block.

Errors

The four ways a command fails are kept apart because they call for different responses.

pub type RedisError {
	Net(err NetError)
	Server(message String)
	Protocol(why String)
	Misuse(why String)
}

• Net: the connection is gone. Reconnect.
• Server: Redis understood the command and refused it. The connection survives.
• Protocol: the bytes on the wire were not the RESP the client expected. A bug, a desync, or a server that is not Redis. Drop the connection.
• Misuse: the call was wrong before it reached the wire, such as subscribing on a pool handle.

redis.show_error renders any of them for a log line. An error reply is a typed failure, and the connection survives it.

// An error reply is a typed failure, and the connection survives it.
refused = match strings.incr(c, 'greeting') {
	Err(e) -> redis.show_error(e)
	Ok(n) -> 'unexpectedly incremented to ${n}'
}

<- with result.then binds the success value and returns the first error, so there is no error plumbing between the lines that do the work.

Get and set

A missing key is None, not '': strings.get returns Result(Option(String), RedisError), and None is distinct from Some('').

_ <- result.then(strings.set_opts(c, 'greeting', 'hello', Always, Some(SetFor(Seconds(60)))))
greeting <- result.then(strings.get(c, 'greeting'))
left <- result.then(keys.ttl(c, 'greeting'))

A condition the key did not meet comes back as Ok(False), not as an error: set_opts with IfNotExists against an existing key is a completed call whose answer is no. Keys and values are String throughout; for values that are bytes rather than text, get_binary, set_binary, and command_raw carry Binary.

Replies

Every wrapped command decodes its reply into the type the command actually has. The raw layer underneath is resp.Value, one variant per RESP3 wire shape, and a family of decoders turns a raw reply into a typed one. Each decoder takes the direct output of command, so an Err passes straight through.

There are around 20 more in the same shape, one per reply contract: expect_opt_int for commands where null means no such element, expect_opt_strings for positional nulls from MGET, expect_opt_strings_or_nil for the null array a blocking timeout returns, which is distinct from an empty one. Write your own on top of value_text, value_int, and value_score when a script returns something bespoke.

Options

Command options are sum types rather than bags of nullable fields, so mutually exclusive choices cannot both be made. SET key v EX 60 KEEPTTL is a server error; SetExpiry makes it unwritable instead of merely ill-advised.

pub type Ttl { Seconds(n Int)  Millis(n Int)  AtSeconds(unix Int)  AtMillis(unix Int) }
pub type SetExpiry { SetFor(ttl Ttl)  KeepTtl }
pub type Condition { Always  IfNotExists  IfExists }
pub type ExpireWhen { Whenever  IfNoExpiry  IfHasExpiry  IfGreater  IfLess }

One expire function covers EXPIRE, PEXPIRE, EXPIREAT, and PEXPIREAT; the Ttl variant picks the verb. IfGreater and IfLess compare against the current expiry, and a key with no expiry counts as infinite, so IfGreater never sets an expiry on a persistent key and IfLess always does.

Pipelines

A round trip costs one network latency whether it carries 1 command or 100. Send them together; the batch reaches the kernel as a single vectored write.

replies <- result.then(
	redis.pipeline(c, [['SET', 'hits', '0'], ['INCR', 'hits'], ['GET', 'hits']]),
)

Measured against a local server, 500 SETs take 152 ms one at a time and 6 ms pipelined.

The outer Result is the connection; the inner ones are per command, in order, so a single refused SET does not discard the 99 replies around it.

match replies {
	[_, incr, get] -> {
		n <- result.then(redis.expect_int(incr))
		v <- result.then(redis.expect_bulk(get))
		println('${n} increment, value ${v or '(nil)'}')
		Ok(Nil)
	}
	_ -> Err(redis.Protocol('expected three replies'))
}

Transactions

transaction sends MULTI, the commands, and EXEC as one pipeline. The step-by-step form (multi, queue, exec) costs a round trip per command; for 20 commands over a 1 ms link that is the difference between 22 ms and 1 ms.

replies <- result.then(
	transactions.transaction(c, [['DECRBY', 'stock:42', '1'], ['LPUSH', 'orders', 'order:99']]),
)

None back means a watched key changed and Redis abandoned the whole thing. That is not an error; it is the signal to read the values again and retry. watch goes before the transaction, never inside it, and that pairing is how a read-modify-write is made safe without holding a lock.

A Redis transaction is isolation, not rollback. Nothing interleaves with it, but a command that fails at runtime does not roll back the commands before it. A command rejected at queue time poisons the transaction, and EXEC then refuses to run a partial batch.

Pub/Sub

Publishing is an ordinary command on any connection.

subscribers <- result.then(pubsub.publish(c, 'news', 'hello'))

Subscribing is not. SUBSCRIBE puts the connection into a mode where the server sends messages nobody asked for, so a subscriber gets its own type: subscribe consumes the Conn and returns a Subscription that owns the socket. Pub/Sub therefore takes two connections: the subscriber's cannot carry a PUBLISH.

import ./lib/redis/pubsub.{Message}

fn listen(c Conn) Result(Nil, redis.RedisError) {
	sub <- result.then(pubsub.subscribe(c, ['news']))
	(rest, event) = pubsub.next_message(sub)
	match event {
		Ok(Message(channel, payload)) -> println('${channel}: ${payload}')
		Ok(_) -> Nil
		Err(e) -> println(redis.show_error(e))
	}
	pubsub.close(rest)
	Ok(Nil)
}

Every call returns the next Subscription alongside its result, because leftover buffer bytes are real state: several messages can arrive in one TCP segment. Thread it through. An Event is Message, PatternMessage from psubscribe, ShardMessage from ssubscribe, or the Subscribed and Unsubscribed bookkeeping, which subscribe consumes before returning, so the first event from next_message is real traffic.

next_message parks the calling process until something arrives. Everything else keeps running; the process is idle, not spinning. next_message_within(sub, ms) returns Ok(None) on timeout, for loops that periodically check a shutdown flag.

Blocking commands

BLPOP and its relatives park a process, not a thread. The scheduler keeps running everything else, so a blocking read costs a process. Give it its own connection.

job <- result.then(lists.blpop(c, ['jobs'], 5.0))
match job {
	Some((key, element)) -> println('${key}: running ${element}')
	None -> Nil
}

Timeouts are seconds as a Float; 0 waits forever, and fractions are allowed. blpop returns the key alongside the element, so a caller waiting on several keys learns which one fired. None is the timeout.

Scores and TTLs

Sorted-set scores are a Score, not a Float, because Redis scores really can be infinite and a Scarlet Float cannot hold an infinity.

pub type Score {
	Finite(value Float)
	PosInf
	NegInf
}

added <- result.then(
	sorted_sets.zadd(c, 'board', [('ada', Finite(120.0)), ('grace', Finite(310.0))]),
)

Range bounds are explicit: Inclusive or Exclusive per end, with LOWEST and HIGHEST as the open ends. TTL queries return a Remaining, which turns the protocol's -1 and -2 sentinels into constructors, so a caller cannot mistake them for durations and do arithmetic on them.

pub type Remaining {
	Expires(in_ Int)
	NoExpiry
	Missing
}

Streams

Stream entries come back as typed records: an Entry is an id and its field pairs, and a read returns them grouped per stream.

id <- result.then(streams.xadd(c, 'events', None, [('kind', 'signup')]))
recent <- result.then(streams.xrange(c, 'events', '-', '+', None))

xadd with None lets the server assign the id. xread blocks with block_ms and returns None on timeout. xreadgroup reads as a named consumer: > asks for undelivered entries, and any other id re-reads that consumer's own pending list, which is how a restarted worker recovers its unfinished work. xpending_range with a minimum idle time finds work abandoned by a dead worker, and xclaim or xautoclaim takes it over. xack retires it.

Commands

275 commands, grouped the way Redis groups them.

strings  keys  hashes  lists  sets  sorted_sets  streams  pubsub  geo
bitmaps  hyperloglog  arrays  transactions  scripting  connection
server  cluster

Scripting keeps keys apart from arguments in the signature, because the key list is what tells Redis Cluster which node a script touches: eval(c, script, keys, args). In production, send the digest instead: script_load returns the SHA1 and evalsha runs it, falling back to eval only on NOSCRIPT.

Hashes carry the per-field TTL family from Redis 7.4 (hexpire, httl, hgetex). The cluster module holds the introspection a routing layer would be built from, but this client does not route: a key on another shard comes back as a Server error carrying the MOVED redirect. Anything not wrapped, send directly.

encoding <- result.then(redis.command(c, ['OBJECT', 'ENCODING', 'user:1']))
stored <- result.then(redis.command_raw(c, [<<'SET'>>, key, jpeg]))

Development

docker compose up -d          # redis on :5379
scarlet run example/tour.scrl
scarlet run test/suite.scrl   # 162 assertions against a live server

The tour walks every module against the compose file's server on port 5379. test/suite.scrl runs 162 assertions against the same server. Run it before sending a change.