Skip to content

Commit

Permalink
Change docs to introduce the new --incremental option
Browse files Browse the repository at this point in the history
Change docs to explain the new `--incremental` option that can be used
together with the `backup` command. Some sections which mentioned
incremental backups are being something specific to Rsync were also
changed to take the new block-level incremental option into account.

References: BAR-249

Signed-off-by: Gustavo William <gustavo.oliveira@enterprisedb.com>
  • Loading branch information
gustabowill authored and barthisrael committed Aug 22, 2024
1 parent 3195e61 commit bfaddd4
Show file tree
Hide file tree
Showing 7 changed files with 83 additions and 20 deletions.
5 changes: 5 additions & 0 deletions doc/barman.1.d/50-backup.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,11 @@ backup *SERVER_NAME*
Overrides value of the parameter `immediate_checkpoint`, if present
in the configuration file.

--incremental [BACKUP_ID]
: performs a block-level incremental backup. A `BACKUP_ID` or [backup ID shortcut](#shortcuts)
of a previous backup must be provided, which references a previous backup
in the catalog to be used as the parent backup from which the incremental is taken.

--reuse-backup [INCREMENTAL_TYPE]
: Overrides `reuse_backup` option behaviour. Possible values for
`INCREMENTAL_TYPE` are:
Expand Down
7 changes: 7 additions & 0 deletions doc/barman.1.d/70-backup-id-shortcuts.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,10 @@ oldest

last-failed
: Latest failed backup, in chronological order.

last-full
: Latest full-backup eligible for a block-level incremental backup using
the `--incremental` option.

latest-full
: same as *last-full*
4 changes: 2 additions & 2 deletions doc/manual/10-design.en.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,9 +64,9 @@ Barman is able to take backups using either Rsync, which uses SSH as a transport
Choosing one of these two methods is a decision you will need to make, however for general usage we recommend using streaming replication for all currently supported versions of PostgreSQL.

> **IMPORTANT:** \newline
> Because Barman transparently makes use of `pg_basebackup`, features such as incremental backup, parallel backup, and deduplication are currently not available. In this case, bandwidth limitation has some restrictions - compared to the traditional method via `rsync`.
> Because Barman transparently makes use of `pg_basebackup`, features such as parallel backup are currently not available. In this case, bandwidth limitation has some restrictions - compared to the traditional method via `rsync`.
Backup using `rsync`/SSH is recommended in all cases where `pg_basebackup` limitations occur (for example, a very large database that can benefit from incremental backup and deduplication).
Backup using `rsync`/SSH is recommended in cases where `pg_basebackup` limitations pose an issue for you.

The reason why we recommend streaming backup is that, based on our experience, it is easier to setup than the traditional one. Also, streaming backup allows you to backup a PostgreSQL server on Windows[^windows], and makes life easier when working with Docker.

Expand Down
4 changes: 3 additions & 1 deletion doc/manual/25-streaming_backup.en.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
## Streaming backup

Barman can backup a PostgreSQL server using the streaming connection,
relying on `pg_basebackup`.
relying on `pg_basebackup`. Since version 3.11, Barman also supports block-level
incremental backups using the streaming connection, for more information
consult the _"Features in detail"_ section.

> **IMPORTANT:** Barman requires that `pg_basebackup` is installed in
> the same server. It is recommended to install the last available
Expand Down
9 changes: 5 additions & 4 deletions doc/manual/26-rsync_backup.en.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
## Backup with `rsync`/SSH

The backup over `rsync` was the only available method before 2.0, and
is currently the only backup method that supports the incremental
backup feature. Please consult the _"Features in detail"_ section for
more information.
The backup over `rsync` was the only method for backups in Barman before
version 2.0, and before 3.11 it was the only method that supported incremental
backups. Current Barman supports file-level as well as block-level incremental backups.
Backups using `rsync` implements the file-level backup feature. Please consult the
_"Features in detail"_ section for more information.

To take a backup using `rsync` you need to put these parameters inside
the Barman server configuration file:
Expand Down
3 changes: 3 additions & 0 deletions doc/manual/42-server-commands.en.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,9 @@ barman backup <server_name>
> You can use `barman backup <server_1_name> <server_2_name>` to sequentially
> backup both `<server_1_name>` and `<server_2_name>` servers.
For information on how to take incremental backups in Barman, please check the
[incremental backup section]{#incremental-backup}.

Barman 2.10 introduces the `-w`/`--wait` option for the `backup` command.
When set, Barman temporarily saves the state of the backup to
`WAITING_FOR_WALS`, then waits for all the required WAL files to be
Expand Down
71 changes: 58 additions & 13 deletions doc/manual/50-feature-details.en.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,24 +13,21 @@ common patterns.

### Incremental backup

Barman implements **file-level incremental backup**. Incremental
backup is a type of full periodic backup which only saves data changes
from the latest full backup available in the catalog for a specific
PostgreSQL server. It must not be confused with differential backup,
Incremental backup is a type of backup which uses an already existing backup as reference
for copying only necessary data changes from the PostgreSQL server. It must not be confused with differential backup,
which is implemented by _WAL continuous archiving_.

> **NOTE:** Block level incremental backup will be available in
> future versions.
> **IMPORTANT:** The `reuse_backup` option can't be used with the
> `postgres` backup method at this time.
The main goals of incremental backups in Barman are:

- Reduce the time taken for the full backup process
- Reduce the disk space occupied by several periodic backups (**data
deduplication**)

Barman currently supports **file-level incremental backups** (using `rsync`) as well
as **block-level incremental backups** (using `pg_basebackup`).

### File-level incremental backups

This feature heavily relies on `rsync` and [hard links][8], which
must therefore be supported by both the underlying operating system
and the file system where the backup data resides.
Expand All @@ -41,15 +38,15 @@ relevant savings in disk usage. This is particularly true of VLDB
contexts and of those databases containing a high percentage of
_read-only historical tables_.

Barman implements incremental backup through a global/server option
Rsync incremental backups can be enabled through a global/server option
called `reuse_backup`, that transparently manages the `barman backup`
command. It accepts three values:

- `off`: standard full backup (default)
- `link`: incremental backup, by reusing the last backup for a server
- `link`: file-level incremental backup, by reusing the last backup for a server
and creating a hard link of the unchanged files (for backup space
and time reduction)
- `copy`: incremental backup, by reusing the last backup for a server
- `copy`: file-level incremental backup, by reusing the last backup for a server
and creating a copy of the unchanged files (just for backup time
reduction)

Expand All @@ -73,6 +70,54 @@ incremental backup as follows:
barman backup --reuse-backup=link <server_name>
```

> **NOTE:** Unlike Postgres block-level incremental backups, Rsync file-level incremental backups are independent
> on their own, meaning that a backup that reused a previous backup for deduplication is not compromised
> in any way if the parent backup is deleted. As mentioned, deduplication in Rsync backups is implemented with the use
> of hard links, so when a previously reused backup is deleted, files shared with other backups will
> still remain on disk, only being removed when the last backup using those files is also deleted. It also
> means that there’s no need to take full backups when a previous full backup is deleted, each backup
> will still have all files and can be reused without any concerns.
> Along the same lines, there is no need to ever take a full backup with rsync by using `reuse_backup = off`. If it is the first backup being taken with `reuse_backup = link`, in essence
> it behaves like `off` because there are no existing files to create hard-links on.
> **IMPORTANT:** The `reuse_backup` option must be used along with
> `rsync` or `local-rsync` as backup method.

### Block-level incremental backups

Since version 3.11, Barman introduces support for block-level incremental
backups, leveraging the native [incremental backup support introduced in PostgreSQL 17](https://www.postgresql.org/docs/17/continuous-archiving.html#BACKUP-INCREMENTAL-BACKUP).

With block-level incremental backups, deduplication occurs at the data block level
(pages in PostgreSQL). This means that only those pages modified since the
last backup will need to be stored, making it a more efficient option,
especially for large databases with spread write patterns. In PostgreSQL, this feature is
implemented with the use of [WAL Summarization](https://www.postgresql.org/docs/17/runtime-config-wal.html#RUNTIME-CONFIG-WAL-SUMMARIZATION), therefore `summarize_wal` must be enabled
on your database server in order to use it.

You can perform block-level incremental backups in Barman using the `--incremental`
option when running a backup command. It accepts a backup id or backup ID shortcut as argument,
which references a previous backup (full or incremental) in the catalog to be used as a parent
for deduplication. In addition, you can also use `last-full` or `latest-full` to reference the latest
eligible full-backup in the catalog.

``` bash
barman backup --incremental <backup_id> <server_name>
```

To be able to perform a block-level incremental backup in Barman you must:

- Have PostgreSQL 17 or later.
- Have `summarize_wal` enabled.
- Have `postgres` as your backup method.

> **NOTE:** Compressed backups are not **yet** eligible for incremental backups in Barman.
> **IMPORTANT:** If you decide to enable `data_checksums` between incremental backups,
> it is adivised to take a new full-backup as divergent checkum configurations can potentially
> lead to issues during recovery.
### Limiting bandwidth usage

It is possible to limit the usage of I/O bandwidth through the
Expand Down

0 comments on commit bfaddd4

Please sign in to comment.