FOPENP

Redis on Docker Swarm - A bad idea

I made many tests with Docker Swarm to see how the Redis horizontal scale works. I ended up to the conclusion that scaling in real-time with Docker Swarm is a bad idea.
I started trying the examples I found on Internet. I Think I've tried them all! Those examples promise to be working, actually they work without storage persistance or, in another case, there is one instance that, if killed, freezes the entire database.
I have studied the functioning of Redis in depth and have come to the conclusion that Redis, as it works internally, cannot be scale horizontally in real time (as Docker Swarm promises).
Docker configures network interfaces with a dynamic IP address. This IP address changes every time you scale the Redis service. The only solution (as also recommended by the official documentation) is to set the network to "host".
Redis catalogs the keys via a 16384 module. These 16384 elements are spread across the master (cluster) nodes. This system is called “sharding”, and when you want to add a new master server you have to redo the sharding. This thing is not easily automatable with docker swarm.
Another problem is data persistence: Redis does not have a system to DUMP and RESTORE all the database keys; a maximum of one DUMP can be made for a single key. In the official documentation they talk about saving the dump.rdb file, but this is misleading: Redis only saves part of the keys in the dump.rdb file. You must save all dump.rdb files of all instances in order to have a backup of the database. Furthermore, the system is not future-proof. It is therefore necessary to perform a backup and restore at the application level. When this is not possible (because you have thousands of keys with random names), you need to save the dump.rdb files. But dockers are ephemeral and you must therefore use a bind mount to persist the database.
Docker Swarm does not guarantee that instances will always be allocated in the same place. This is a problem, because the shards are cataloged outside dump.rdb and the database is located in dump.rdb (a file that will be on a machine but will not be moved automatically).
When Redis is configured as a cluster, masters and replicas are elected. Each of these has a specific role, but Docker Swarm doesn't take them into account.
I tried Redis with Docker Swarm and found that it complicated my life rather than improved it. In my opinion it is better to manually configure the instances (when the number of servers is low) or use tools like Ansible™ to automate installations.

2023
Dec, 08