Monitoring your disk layout with Relax-and-Recover (ReaR)
A crucial part to properly recreate the system is the disk layout, i.e. the disk partitioning with filesystems and mount points.
When you use "rear mkbackup" to create the ReaR rescue/recovery system together with a backup of all files and directories of your system at the same time, you have both parts (recovery system and backup) in sync which is a prerequirement to properly recreate the system with "rear recover".
But when you use "rear mkrescue" to create the ReaR rescue/recovery system and call "rear mkbackuponly" separatedly to create the backup or you call a third party backup tool separatedly, you may not have both parts (recovery system and backup) in sync so "rear recover" may not properly recreate the system or it may even completely fail to do it.
In particular when the disk layout had changed the ReaR rescue/recovery system must be created anew.
Caution: There are zillions of other ways how the latest created ReaR rescue/recovery system could become outdated or even invalid/useless in general. Each change of the basic system setup (like disk layout, network environment,…) and each change of a software that is used by ReaR (like 'parted', 'mkfs', 'tar',…) and of course also each ReaR version upgrade requires that the ReaR rescue/recovery system gets created anew together with a matching up-to-date backup as prerequirement so that "rear recover" is able to properly recreate the current system. Furthermore after such changes you must carefully and completely re-validate that "rear recover" still works in your particular case/environment.
The disk layout information is stored in var/lib/rear/layout/disklayout.conf
ReaR provides two specific workflows regarding the disk layout:
Saving the current disk layout of the system
ReaR automatically saves the current disk layout of the system when it creates a new ReaR rescue/recovery system. However if you want to only save the current disk layout manually, use "rear savelayout" (this does not update the recovery system).
Checking if the disk layout has changed
When you want to know if the current disk layout of the system has changed compared to the latest saved disk layout, use "rear checklayout". If the disk layout has changed, "rear checklayout" results a non-zero return code so you could use something like
# rear checklayout || rear mkrescue
to create a new ReaR recovery system when the disk layout has changed.
Integration with Nagios and Opsview
If having current DR rescue images is important to your organization, but they cannot be automated (eg. a tape or USB device needs inserting), we provide a Nagios plugin that can send out a notification whenever there is a critical change to the system that requires updating your rescue environment.
Changes to the system requiring an update are:
Changes to hardware RAID
Changes to software RAID
Changes to partitioning
Changes to DRBD configuration
Changes to LVM
Changes to filesystems
The integration is done using our own check_rear plugin for Nagios.
# Purpose: Checks if disaster recovery usb stick is up to date
# Check if ReaR is installed
if [[ ! -x /usr/sbin/rear ]]; then
echo "REAR IS NOT INSTALLED"
# ReaR disk layout status can be identical or changed
# returncode: 0 = ok
if ! /usr/sbin/rear checklayout; then
echo "Disk layout has changed. Please insert Disaster Recovery USB stick into system !"
We also monitor the /var/log/rear/rear-system.log file for ERROR: and BUG strings, so that in case of problems the operator is notified immediately.
Note that error messages may not come from ReaR itself but from programs that are called by ReaR because stdout and stderr are redirected into ReaR’s log file (see the section "What to do with stdin, stdout, and stderr" in https://github.com/rear/rear/wiki/Coding-Style) so in case of error messages one must check if that is actually an error or only false alarm.