Murmur makes use of multiple operating system threads.
Most logic in Murmur happens on the main thread.
Each virtual server in Murmur is represented by the
Server class. Most of the
Server class runs on the main thread, such as all control channel traffic (TLS). Murmur's RPC systems usually run on a separate thread -- but all RPC methods are run on the main thread via
Server class is a subclass of
QThread. The thread that
Server runs, is the voice thread. This thread handles incoming UDP packets (ping and voice), and rebroadcasts them as necessary. The methods that run on the voice thread are:
void Server::processMsg(ServerUser *u, const char *data, int len)
void Server::sendMessage(ServerUser *u, const char *data, int len, QByteArray &cache, bool force)
bool Server::checkDecrypt(ServerUser *u, const char *encrypt, char *plain, unsigned int len)
but these methods may call other methods (such as
ChanACL::hasPermission()) in order to do their job.
In general, when speaking about the voice thread, it refers to any code running in the
Server methods listed above.
The voice thread methods access various data in the
Server class to do their job. Besides being accessed by the voice thread, this data is also read and written to by the main thread.
To properly synchronize the access to the data shared between the two threads, the
Server class contains a non-recursive reader-writer lock,
This lock provides synchronization between the main thread and the voice thread. These are the only two threads that access a
The easiest way to understand the locking strategy and synchronization between the main thread and the
Server's voice thread is by using the concept of ownership.
A thread owning an object means that it is the only thread that is allowed to write to that object. To make changes to it.
Most data in the
Server class is owned by the main thread. That means that the main thread is the only thread that writes/updates those structures.
When processing incoming voice data (and re- broadcasting that voice data), the
Server's voice thread needs to access various parts of
To ensure correct synchronization between the two threads it is important that the rules for accessing data in
Server are clearly documented.
Most of these ownership rules relate to the use of
Server->qrwlVoiceThread mentioned earlier. It is the main reader/writer lock between the voice thread and the main thread. It is important to know how exactly Murmur uses this lock, because it is not immediately obvious.
Since the data protected by
Server->qrwlVoiceThread is only ever accessed by two threads, Murmur is clever in the way that it uses the lock.
It uses the ownership rule defined above ("a thread owning an object means that it is the only thread allowed to write to that object") to avoid taking locks in some situations. This is to avoid littering the code with locks, but also to avoid deadlocks and/or misuse of the non-recursive reader/write locker.
When reading data in the main thread that Murmur knows is only ever written to by the main thread (that is, the data is owned by the main thread), Murmur will avoid taking a read lock on
However, not all data in
Server is owned by the main thread. When data is owned by the voice thread, the same kind of rule applies. The voice thread will not take a read lock when reading data that it knows only it will write to.
This behavior is not immediately obvious, but it is important to know for understanding how the
Server->qrwlVoiceThread lock is used throughout the code base.
Another important detail to keep in mind is that while the main thread can avoid locking
Server->qrwlVoiceThread during most of its operation, the opposite it true for the voice thread. Most of the time, when processing incoming packets, and especially in the
void Server::processMsg(ServerUser *u, const char *data, int len) method, the voice thread holds a read lock
Server->qrwlVoiceThread. This is to ensure that the main thread is properly excluded when the voice thread is reading data, and the voice thread is excluded when the main thread is writing data.
This section documents the owners of data in Murmur that is accessed by multiple threads at once. It documents how one should synchronize access to that data to avoid introducing data races in Murmur.
These are never accessed by the main thread, except in
ServerUser's constructor. There is no synchronization on these.
The rules for accessing these objects are:
The objects are:
The rules for accessing these objects are:
The objects with shared ownership are:
These objects are explicitly synchronized by using atomic types and need no extra synchronization when read or written to. The objects are:
These objects use their own locking schemes. This is can be an internal mutex that is automatically taken when the object's methods are called. It can also be an external mutex that must be held before accessing the object itself.
ServerUser->bwr(Internal locking inside
BandwidthRecord. All methods can be called without extra synchronization.)