zfs-replicate

https://github.com/alunduil/zfs-replicate

By Alex Brandt alunduil@gmail.com

zfs-replicate backs up Zettabyte File System (ZFS) data sets to a remote host by sending their snapshots over SSH. It does not create snapshots, so pair it with a tool that does.

zfs-replicate relates to several other projects, which fit other niches:

1. sanoid: A full snapshot management system. Its companion, syncoid, handles replication with copious options.
2. zfs-replicate (BASH): A similar project. The major differences include configuration style and system expectations (for example, logging controls). zfs-replicate uses parameters whereas zfs-replicate (BASH) uses a BASH script.
3. znapzend: Another scheduling and replicating system.
4. zrep: A SH script with several control commands for snapshot replication.

1. A local ZFS filesystem and zfs command-line tools
2. Python 3.10 or later on the local system
3. A remote system with a ZFS filesystem and the zfs command-line tools
4. SSH access to that remote system
5. lz4 on both systems, unless you pass --compression off

Don't use the root user on the remote system. Delegate the ZFS permissions replication needs on the backup data set to a regular user instead:

# FreeBSD only: let non-root users mount filesystems.
sysctl -w vfs.usermount=1

zfs allow "${USER}" clone,create,destroy,hold,mount,promote,quota,readonly,receive,rename,reservation,rollback,send,snapshot "${BACKUP_DATASET}"

Install zfs-replicate from PyPI:

pipx install zfs-replicate

Nix and NixOS users install the zfs-replicate package from nixpkgs. NixOS also provides a services.zfs.autoReplication module that runs replication as a system service.

Replicate the snapshots of LOCAL_FS to REMOTE_FS on HOST:

zfs-replicate --user "${USER}" --identity-file ~/.ssh/id_ed25519 HOST REMOTE_FS LOCAL_FS

zfs-replicate --help lists every option. To tune the send stream or set properties on the replica, see How to tune the send and receive streams.

• How to replicate an encrypted data set: Replicate without decrypting, then load the replica's key on the destination.
• How to tune the send and receive streams: Control what the send stream carries and set properties on the replica.
• CHANGELOG: Changes in each release.
• Survey of ZFS Replication Tools: Overview of various ZFS replication tools and their uses.
• Working With Oracle Solaris ZFS Snapshots and Clones: Oracle's guide to working with ZFS snapshots.

• GitHub issues: Report any problems or feature requests to GitHub issues.

Contributions are welcome as issues or pull requests. CONTRIBUTING explains how to set up a development environment and what a pull request needs. Everyone taking part follows the Code of Conduct.

You are free to copy, change, and distribute zfs-replicate with attribution under the terms of the BSD-2-Clause licence. See the LICENSE for details.

zfs-replicate began as a fork of FreeNAS's autorepl.py, so the LICENSE carries the iXsystems, Inc. copyright.