Introduction to Renoir

About a month ago, I released a “production ready” Redis cluster client library for Ruby. It is named Renoir and was pushed to rubygems.org: https://rubygems.org/gems/renoir.

It was surprising that there had been no reliable Redis Cluster client library for Ruby in spite of the fact that Redis 3.0 had been released about two years ago. In fact, there exist several implementations but their README always say like “this library is not yet production ready.” Because a reliable library are needed for my work, I tried to develop such a library, and now it would reach a certain degree of quality. This post gives a brief introduction to Renoir; a concept and current features are explained.

Renoir Example

For example, Renoir could be used like this:

require 'renoir'

rc = Renoir::Client.new(cluster_nodes: ['127.0.0.1:30001'])

p rc.set('hoge', 123, nx: true) # keyslot is 1525
p rc.zrange('fuga', 0, -1, with_scores: true) # keyslot is 9097

Suppose that a Redis cluster is running and one of the cluster nodes is listening the port 30001 on the local host. The initializer of Renoir::Client takes at least an address of a node in a cluster and remaining node addresses will be automatically retrieved. In this example, SET and ZRANGE commands are sent to the cluster. Because the keys for these commands have different “keyslots”, each command have to be sent to different nodes unless those keyslots are assigned to same node. But you don’t need to worry about that here because the primary job of Renoir is to manage such directions automatically.

You may notice that the methods and arguments of Renoir::Client are same as the redis gem. So it is! Renoir is designed to provide compatible interface with the redis gem. Thus you can communicate with a Redis cluster as well as communicating with single Redis server by the redis gem.

If you would like to test this code, execute gem install renoir and create a Redis cluster on local machine with a node listening port 30001 (the official create-cluster script is useful to create such a cluster).

Features

Handling MOVED/ASK redirections

Of course, this is the main feature. User can send commands to Redis cluster in a similar manner as sending ones to a single Redis instance because Renoir catches MOVED/ASK redirection responses and resend commands to an appropriate node automatically. Note that the number of redirections can be limited by max_redirection option to prevent too many redirections.

Maintaining cluster information

Renoir maintains information of a cluster. In particular, keyslots assignment information is important because a target node, which holds a key(slot) of a command, could be determined without sending request, i.e., probability of redirection and then latency are dramatically reduced. Although master/slave information is available, all commands are sent to only master nodes and there is no plan to support slaves so far.

Validating singleness of command keyslots

Renoir checks whether keyslots of keys in a command are all the same before sending and the command fails if it is not. This is fine because sending a command accessing multi-keyslots is useless. Besides, Renoir basically refuses a command without a key because a target node is unknown. Sending a command to all nodes could be an option but it would be scary to use because request timing is uncontrollable by user. Instead, Renoir::Client#each_node method could be used to send no-key commands in more controllable way, like this:

keys = rc.each_node.flat_map do |node| # node is a `::Redis` instance
  sleep 0.01
  node.keys('test_*')
end

rc.each_node(&:bgsave)

Supporting transaction and command pipelining

Renoir::Client#multi and #pipelined could be used for transaction and command pipelining:

rc.multi do |tx|
  tx.set('hoge{1}', 123)
  tx.get('hoge{1}')
  tx.get('fuga{1}')
end

rc.pipelined do |pipeline|
  pipeline.set('hoge{1}', 123)
  pipeline.get('hoge{1}')
  pipeline.get('fuga{1}')
end

The commands will be sent to a node only if all keyslots of the commands are same. Hence it is highly recommended to use keys hash tags.

Unfortunately, these commands are not well compatible with the ones of the redis gem; a return value of a command inside the block is useless because a “future” variable is not yet supported. All responses are returned as a return value of #multi/#pipelined, though.

This gem has been used at my work for a messaging delivery system which handles millions of requests in a minute and it works very well so far. But it lacks real world examples. So it would be greatly appreciated if you could use it!

The source code is available on GitHub: https://github.com/saidie/renoir and documentation is also available: http://www.rubydoc.info/gems/renoir.

Future work

• Support “future” variables in #multi and #pipelined
• Add a connection adapter for the redic gem

Acknowledgment

My code is originated to the first reference implementation by antirez.