Want to skip the details and setup Drizzle replicaiton as quickly as possible?
Then start with the
Simple Master-Slave example <simple_master_slave_example>. Otherwise, start here and read each section in sequence for a complete understanding of Drizzle replication.
Drizzle encodes replication events as Google Protocol Buffer (GPB) messages and transmits them asynchronously through replication streams. A replication stream is a pair of one replicator and one applier. The kernel encodes replication events, sends them to all available replicators, which in turn send them to their paired appliers. For Drizzle, the replication process ends at this point, but appliers are responsible for applying replication events to a service: another Drizzle server (a slave), a message queue, etc.
Drizzle External ========================================================== ======== User Kernel Plugins ===== ================= ========================= Replication Streams ========================= event --> encode GBP(event) +--> replicator1 +--> applier1 --> service1 | | | +--> applier2 --> service2 | +--> replicator2 +--> applier3 --> service3
Replicators and appliers are implemented by plugins, so Drizzle replication is extensible and varies depending on which plugins are used. Drizzle includes plugins to implement several replication systems:
Master-slave replication, including multi-master <slave_applier>
Replication to a RabbitMQ server <rabbitmq_applier>
Replication to a ZeroMQ socket <zeromq_applier>
Master-slave is the most common replication system; it resembles other database servers like MySQL.
Unlike other database servers, the Drizzle kernel plays only a minimal role in the replication process: it simply encodes and sends replication events to available replicators. Consequently, there are very few
drizzled replication options <drizzled_replication_options>, and Drizzle replication is primarily configured by replicator and applier plugin options.
In summary, Drizzle replication:
Learned enough? Ready to start using Drizzle replication? Then jump to the
replication examples <replication_examples>. Otherwise, continue reading for more details about all the parts and processes of Drizzle replication.
Replication events are, in general, any SQL statements which change data or schema objects on a server.
/dml queries are the primary cause of replication events, but other statements like setting global variables or altering schemas or tables may also cause replication events.
The Drizzle kernel sends every replication event to every applier (that is to say, every loaded applier plugin), but appliers determine if replicaiton events are sent to their paired appliers. For example, a replicator may filter certain replication events. Likewise, appliers determine if (and how) replication events are ultimately applied.
Replication events are not logged or saved by the Drizzle kernel or any replicators that ship with Drizzle. An applier may log replication events that it receives from its replicator.
Every replication event is encapsulated as an atomic transaction, including bulk and mutli-statement events.
Drizzle relinquishes all control of replication events once they enter a replication stream. Replicators and appliers are responsbile for handling replication events correctly and efficiently.
Replication stream are logical conduits created by pairing one replicator with one applier. As logical entities, replicaiton streams exist only inside the
drizzled process and cannot be accessed externally. However, some appliers create or access ports or sockets which allows indirect access to the replication stream. Since replicators and appliers are implemented by plugins, one could in theory program a custom applier or replicator to provide a socket or port for direct access into the replication stream.
drizlzed starts, it creates replication streams automatically based on which replicators are loaded and which appliers are loaded and configured to use them. For example, an applier plugin may be configured to use a specific replicator, in which case
drizzled pairs the applier to the specified replicator. The user does not need to perform special steps to create a replication stream.
Replication stream cannot be dynamically recreated; the user must stop Drizzle, reconfigure the replicator or applier, and then restart Drizzle to let it automatically recreate the new replication stream.
The originating server of a replication event is the server on which the SQL statement that caused the replication was first executed. Since one replicaiton event may be applied to several services (by passing through multiple replication streams), the originating server uniquely identifies the true origin of a replication event versus its most immediate upstream origin which may have received the replication event from any number of additional upstream sources.
Drizzle automatically generates a UUID for every server, saved in the
server.uuid file in the
--datadir directory. This UUID is included with every replication event that originates from the server.
An originating server may or may not contain both end points of a replication stream. Replicators are always local to (loaded and ran from) the originating server from which they receive replication events, but appliers may be local or remote (loaded and ran on a different server). The external service to which the applier applies replication events is usually another server, not the originating server, but an applier could, in theory, apply events from and to the same originating server.
Drizzle replication is primarily configured by options specific to each
replicator <replicators> and
The Drizzle kernel has very few
drizzled_replication_options which typically do not need to be changed:
Controls the size, in bytes, of the transaction messages. When a transaction message exceeds this size, a new transaction message with the same transaction ID will be created to continue the replication events. See
Controls whether the originating SQL query will be included within each statement message contained in the enclosing transaction message. The default global value is FALSE which will not include the query in the messages. It can be controlled per session, as well. For example:
drizzle> SET @@replicate_query = 1;
The stored query should be used as a guide only, and never executed on a slave to perform replication as this will lead to incorrect results.