"Fossies" - the Fresh Open Source Software Archive

Member "drizzle-7.1.36-stable/docs/replication/index.rst" (6 May 2012, 7917 Bytes) of package /linux/misc/old/drizzle-7.1.36-stable.tar.gz:


As a special service "Fossies" has tried to format the requested source page into HTML format (assuming markdown format). Alternatively you can here view or download the uninterpreted source code file. A member file download can also be achieved by clicking within a package contents listing on the according byte size field.

drizzled

Drizzle Replication

Want to skip the details and setup Drizzle replicaiton as quickly as possible?

Then start with the Simple Master-Slave example <simple_master_slave_example>. Otherwise, start here and read each section in sequence for a complete understanding of Drizzle replication.

Drizzle encodes replication events as Google Protocol Buffer (GPB) messages and transmits them asynchronously through replication streams. A replication stream is a pair of one replicator and one applier. The kernel encodes replication events, sends them to all available replicators, which in turn send them to their paired appliers. For Drizzle, the replication process ends at this point, but appliers are responsible for applying replication events to a service: another Drizzle server (a slave), a message queue, etc.

Drizzle                                                        External
==========================================================     ========
User      Kernel                 Plugins
=====     =================      =========================
                                 Replication Streams
                                 =========================
event --> encode GBP(event) +--> replicator1 +--> applier1 --> service1
                            |                |
                            |                +--> applier2 --> service2
                            |
                            +--> replicator2 +--> applier3 --> service3

Replicators and appliers are implemented by plugins, so Drizzle replication is extensible and varies depending on which plugins are used. Drizzle includes plugins to implement several replication systems:

Master-slave is the most common replication system; it resembles other database servers like MySQL.

Unlike other database servers, the Drizzle kernel plays only a minimal role in the replication process: it simply encodes and sends replication events to available replicators. Consequently, there are very few drizzled replication options <drizzled_replication_options>, and Drizzle replication is primarily configured by replicator and applier plugin options.

In summary, Drizzle replication:

Learned enough? Ready to start using Drizzle replication? Then jump to the replication examples <replication_examples>. Otherwise, continue reading for more details about all the parts and processes of Drizzle replication.

Replication Events

Replication events are, in general, any SQL statements which change data or schema objects on a server. /dml queries are the primary cause of replication events, but other statements like setting global variables or altering schemas or tables may also cause replication events.

The Drizzle kernel sends every replication event to every applier (that is to say, every loaded applier plugin), but appliers determine if replicaiton events are sent to their paired appliers. For example, a replicator may filter certain replication events. Likewise, appliers determine if (and how) replication events are ultimately applied.

Replication events are not logged or saved by the Drizzle kernel or any replicators that ship with Drizzle. An applier may log replication events that it receives from its replicator.

Every replication event is encapsulated as an atomic transaction, including bulk and mutli-statement events.

Drizzle relinquishes all control of replication events once they enter a replication stream. Replicators and appliers are responsbile for handling replication events correctly and efficiently.

Replication Streams

Replication stream are logical conduits created by pairing one replicator with one applier. As logical entities, replicaiton streams exist only inside the drizzled process and cannot be accessed externally. However, some appliers create or access ports or sockets which allows indirect access to the replication stream. Since replicators and appliers are implemented by plugins, one could in theory program a custom applier or replicator to provide a socket or port for direct access into the replication stream.

When drizlzed starts, it creates replication streams automatically based on which replicators are loaded and which appliers are loaded and configured to use them. For example, an applier plugin may be configured to use a specific replicator, in which case drizzled pairs the applier to the specified replicator. The user does not need to perform special steps to create a replication stream.

Replication stream cannot be dynamically recreated; the user must stop Drizzle, reconfigure the replicator or applier, and then restart Drizzle to let it automatically recreate the new replication stream.

Originating Server

The originating server of a replication event is the server on which the SQL statement that caused the replication was first executed. Since one replicaiton event may be applied to several services (by passing through multiple replication streams), the originating server uniquely identifies the true origin of a replication event versus its most immediate upstream origin which may have received the replication event from any number of additional upstream sources.

Drizzle automatically generates a UUID for every server, saved in the server.uuid file in the --datadir directory. This UUID is included with every replication event that originates from the server.

An originating server may or may not contain both end points of a replication stream. Replicators are always local to (loaded and ran from) the originating server from which they receive replication events, but appliers may be local or remote (loaded and ran on a different server). The external service to which the applier applies replication events is usually another server, not the originating server, but an applier could, in theory, apply events from and to the same originating server.

Configuration

Drizzle replication is primarily configured by options specific to each replicator <replicators> and applier <appliers>.

The Drizzle kernel has very few drizzled_replication_options which typically do not need to be changed:

--transaction-message-threshold

Controls the size, in bytes, of the transaction messages. When a transaction message exceeds this size, a new transaction message with the same transaction ID will be created to continue the replication events. See bulk-operations.

--replicate-query

Controls whether the originating SQL query will be included within each statement message contained in the enclosing transaction message. The default global value is FALSE which will not include the query in the messages. It can be controlled per session, as well. For example:

drizzle> SET @@replicate_query = 1;

The stored query should be used as a guide only, and never executed on a slave to perform replication as this will lead to incorrect results.