multi-leader.asciidoc · GitHub by 262588213843476

A replication topology describes the communication paths along which writes are propagated from one node to another. (View Highlight)

In this case, every device has a local database replica that acts as a leader (it accepts write requests), and there is an asynchronous multi-leader replication process (sync) between the replicas of your calendar on all of your devices. (View Highlight)

Both offline editing and real-time collaboration require a similar replication infrastructure: the application needs to capture any changes that the user makes to a file, and either send them to collaborators immediately (if online), or store them locally for sending later (if offline). Additionally, the application needs to receive changes from collaborators, merge them into the user’s local copy of the file, and update the user interface to reflect the latest version. (View Highlight)

If multiple users have changed the file concurrently, conflict resolution logic may be needed to merge those changes. (View Highlight)

A software library that supports this process is called a sync engine. (View Highlight)

An application that allows a user to continue editing a file while offline (which may be implemented using a sync engine) is called offline-first (View Highlight)

The term local-first software refers to collaborative apps that are not only offline-first, but are also designed to continue working even if the developer who made the software shuts down all of their online services (View Highlight)

when using a sync engine, you have persistent state on the client, and communication with the server is moved into a background process. (View Highlight)

With a sync engine, an app doesn’t need a separate offline mode: being offline is the same as having very large network delay. (View Highlight)

A sync engine simplifies the programming model for frontend apps, compared to performing explicit service calls in application code. (View Highlight)

In order to display edits from other users in real-time, you need to receive notifications of those edits and efficiently update the user interface accordingly. A sync engine combined with a reactive programming model is a good way of implementing this (View Highlight)

Sync engines work best when all the data that the user may need is downloaded in advance and stored persistently on the client. This means that the data is available for offline access when needed, but it also means that sync engines are not suitable if the user has access to a very large amount of data. (View Highlight)

The biggest problem with multi-leader replication—both in a geo-distributed server-side database and a local-first sync engine on end user devices—is that concurrent writes on different leaders can lead to conflicts that need to be resolved. (View Highlight)

Thus, conflict avoidance breaks down if you allow the leader to be changed. (View Highlight)

If conflicts can’t be avoided, the simplest way of resolving them is to attach a timestamp to each write, and to always use the value with the greatest timestamp. (View Highlight)

Therefore the real meaning of LWW is: when the same record is concurrently written on different leaders, one of those writes is randomly chosen to be the winner, and the other writes are silently discarded, even though they were successfully processed at their respective leaders. This achieves the goal that eventually all replicas end up in a consistent state, but at the cost of data loss. (View Highlight)

Automatic conflict resolution ensures that all replicas converge to the same state—i.e., all replicas that have processed the same set of writes have the same state, regardless of the order in which the writes arrived. (View Highlight)

A natural extension of the single-leader replication model is to allow more than one node to accept writes. Replication still happens in the same way: each node that processes a write must forward that data change to all the other nodes. (View Highlight)

An application that allows a user to continue editing a file while offline (which may be implemented using a sync engine) is called offline-first (View Highlight)

There are limits to what is possible with conflict resolution. For example, if you want to enforce that a list contains no more than five items, and multiple users concurrently add items to the list so that there are more than five in total, your only option is to drop some of the items. Nevertheless, automatic conflict resolution is sufficient to build many useful apps. (View Highlight)