Skip to content

Home

Jump to bottom
Daniel J. Bell edited this page Apr 3, 2024 · 15 revisions

This document is being moved to the Zelta Wiki.

This document will help you get started with Zelta. See Use Case Examples, the man page drafts, or the point of use documentation in the .example configuration files or zfs usage for further details.

What can Zelta help you do?

  • Make a local backup and keep it up to date
  • Make sure all of your backups are up to date
  • Make backups of remote systems
  • Migrate data between hosts as quickly as possible
  • Keep a large number of backups up to date automatically
  • Manage a two-way failover system
  • Alert you when something has gone wrong
  • Export policy data into Grafana

Terminology

There are some ambiguities in the language surrounding ZFS and backups in general. Here are some clarifications for terms used throughout Zelta's documentation. Please open an Issue or let us know if you can recommend clearer language.

  • Archives: Static replicas that don't need to be updated incrementally, e.g., decommissioned VMs
  • Backups: Replicas that are expected to receive further replication jobs
  • Dataset: A ZFS volume or filesystem (not a snapshot unless specified)
  • Match: The most recent common snapshot between two replicas
  • Replica: A copy of a ZFS tree used for a backup, archive, or failover standby
  • Tree: A dataset and its children
  • Stub: The relative child dataset name when comparing two replicas, similar to a relative directory path, e.g., given a tree containing copy1/x/y and its replica copy2, both would contain a top level stub plus stubs x and x/y.

To get the most out of Zelta, there are other concepts you should understand.

General ZFS terms: clone, filesystem, permissions, pool, properties, volume, replication, snapshot.

General disaster recovery terms: redundancy versus backups, incremental vs full backups, redundancy vs backups, RPO, RTO.

Installation

install.sh, when run as root, will copy most of Zelta's scripts to /usr/local/share/zelta/ and the zelta shell wrapper to /usr/local/bin/.

git clone https://github.com/bellhyve/zelta.git
cd zelta
sudo sh ./install.sh

Omit sudo to install Zelta scripts for the current user. The script will let you know what was copied and how to configure your environment.

Quick Start: Back up your computer

In this example, we have an OS installed with a ZFS pool called zroot. After attaching a new blank drive to the system, we detect a new drive as da2. We can create a blank pool called backups with a command such as zpool create backups da2.

Run zelta backup and review the output below it.

zelta backup zroot backups/my_zroot_backup
source snapshot created: @2024-01-31_12.38.53
3G sent, 21/21 streams received in 10.34 seconds

Repeat this command to update your backup. To learn more, see the zelta replicate section of this document and this (manpage draft)[https://github.com/bellhyve/zelta/wiki/zelta-replicate-(man-page-draft)].

Quick Start: Back up the universe using a policy

zelta match and zelta backup are useful for migrations and backup scripts that deal with a small number of replication jobs interactively. To deal with large numbers of backup datasets, you can use zelta policy perform many backups and receive a human-readable backup report or JSON detail.

Next, we'll use a policy configuration file to perform the same task as we did in the first example: backing up a local zroot source to backuppool/my_zroot_backup. Set up and edit zelta.conf.

vi /usr/local/etc/zelta/zelta.conf

First, let's make a configuration to test local backups. The first line below is a unique Site name of choice, representing a location or backup set. Note the two spaces before host and dataset names:

MyLocalBackups:
  localhost:
  - zroot: backuppool/my_zroot_backup

Run zelta policy. It will perform the backup operation and report some details about its success or failure. Now we can extend our policy with options and more datasets.

To replicate from or between remote servers, zelta uses ssh. To learn about setting up remote access, ZFS privilege separation, and optimizing ssh with its excellent client-side functions, see the Zelta wiki on (Setting Up and Optimizing Remote Replications)[https://github.com/bellhyve/zelta/wiki/Setting-Up-and-Optimizing-Remote-Replications].

BACKUP_ROOT: backuppool
HOST_PREFIX: 1

MySites:
  localhost:
  - zroot: backuppool/my_zroot_backup
  host1:
  - tank/vm/one
  - tank/vm/two
  host2:
  - tank/jail/three
  - tank/jail/four

And so on. Save again, and as before, run the replication process:

zelta policy

If we want to back up a subset of the policy, name a site, hostname, and/or dataset you'd like to back up. For example, the following will back up all hosts under "MySites", all datasets in host1, and only tank/jail/four for host2:

zelta policy MySites host1 tank/jail/four

If all went well, a zfs list command will show that you have a backup of the five datasets listed in the configuration: backups/my_zroot_backup, backups/host1/one, backups/host1/two, backups/host2/three, and backups/host2/four.

See the configuration example for more details and useful options.

zelta match

zelta match is a tool used for identifying the relationship between the two given dataset trees. This tool is particularly useful for determining if datasets are in sync and identifying the latest common snapshot.

zelta match [options] source_dataset target_dataset

zelta replicate

zelta replicate (previously zpull) handles the actual replication of ZFS snapshots between a source and a target dataset. It uses the output of zelta match to determine which snapshots need to be sent and then performs the replication.

zelta replicate|backup|sync|clone [options] [initiator.host] source_dataset target_dataset

The basic defaults are:

  • Replicate as many intermediate snapshots as possible.
  • Sending using zfs send -Lcp: Large blocks, compression, and send as many properties as are allowed by the user.
  • Delete don't replicate the mountpoint and mount as readonly below the parent mountpoint. (zfs receive -x mountpoint -o readonly=on)

The defaults can be changed by using a different zelta command:

  • zelta backup: Adds a snapshot before replication if needed.
  • zelta sync: Only replicates the latest snapshots, e.g., for faster migration.
  • zelta clone: Creates a read-write view of dataset tree for inspection and recovery.

zelta policy

zelta policy (or just zelta) orchestrates the replication process. The configuration file zelta.conf allows you to specify various parameters, including backup roots, sites, hosts, and datasets.

BACKUP_ROOT: pool/Backups

DAL1:
  fw1.dal1:
  - fw1-boot/jail/webproxy_bti
  host00.bts:
  - ssd00/jail/app1.asi
  ...

In zelta.conf, you can organize your hosts into different Sites which can be used for multi-threaded replication. Several targeting options are provided for any backup naming hierarchy. See the configuration example.

Usage:sh

zelta [optional_site_host_or_dataset] [...]

If one or more arguments are provided, zelta will limit the replication process to the specified sites, hosts, or datasets. If no argument is provided, it will process according to the settings in the configuration file.

Clone this wiki locally