Command-Line Digital Preservation Utilities

Versioning

• 0.6.0 and earlier - Designed for pasty v1
• 1.0.0 and later - Designed for use with patsy v2

See the Git Tags for versioning.

Installation

Clone this repository and do:

$ pyenv local $(cat .python-version)
$ python -m venv .venv --prompt preserve
$ source .venv/bin/active
$ python setup.py install

If the virtual environment was already setup, just the last two lines are sufficient.

Preserve

The "--help" flag provides information about available commands and arguments:

$ preserve --help

The "--help" flag, coupled with a specific command shows additional information about that command:

$ preserve inventory --help

Subcommands

Annotate

Supplements an inventory CSV by scanning disk for files.

$ preserve annotate [-h] [-i INVENTORY] [-o OUTPUT] [-r ROOT]

Bytecount

Sum the bytes of (visible) files in the specified path, and also count them by extension.

$ preserve bytecount [-h] [-r] [-H] path

Bagcheck

Checks relpath & checksum against bag manifest.

$ preserve bagcheck [-h] [-i INVENTORY] [-b BAG]

Compare

Compare two or more file lists previously generated by various utilities, to ensure that the filesets are identical. Supported report formats include ones created by this script, tab-delimited File Analyzer reports, and Tivoli Storage Manager Backup reports.

$ preserve compare [-h] [-r] first other [other ...]

Inventory

Create a CSV report containing file metadata for all visible files in the specified path. Write CSV to an output file (specified with -o flag) or send the data to stdout for further processing. Resume previously interrupted jobs by specifying path to an existing (with -e flag) partial inventory file. The algorithms provided can be md5, sha1 and sha256.

$ preserve inventory [-h] -b BATCH [-o OUTFILE] [-e EXISTING] [-a ALGORITHMS] [-l LABEL] path

Verify

Compare two existing inventory CSVs or inventories generated at runtime, by verifying their filenames and checksums. If differences are detected, the script will attempt to reason about the nature of the differences (whether files have been changed in place, moved, added, or deleted).

$ preserve verify [-h] [-c] [-r] [-f] first second

Partition

Partition a tree of files based on various schemes. This is only one command.

partition [-h] [-m {copy,move,dryrun}] [-o OUTPUT] [-v] source destination

Development Setup

See docs/DevelopmentSetup.md.