sett packaging file format specifications

sett compresses, encrypts and packages files in a single .tar file whose specifications are described below. Only files adhering to these specifications can be transferred or decrypted by sett, and failure to comply with the specification will generate an error.

File structure

sett .tar files have the following structure:

YYYYMMDDThhmmss.tar
├── metadata.json
├── metadata.json.sig
└── data.tar.gz.gpg
    └── data.tar.gz
        ├── content/
        |   ├── [file1]
        |   ├── [file2]
        |   └── ...
        └── checksum.sha256
metadata.json

Metadata file containing the following information:

  • transfer_id: transfer ID.
  • sender: fingerprint of the sender’s public PGP key.
  • recipients: list of fingerprints of the recipients’ public PGP keys.
  • timestamp: datetime of metadata generation. Uses the datetime format YYYY-MM-DDThh:mm:ss±ZZZZ (where ZZZZ is the UTC offset)
  • checksum: sha256 checksum of the encrypted data.tar.gz.gpg file. This allows verifying the integrity of the transferred file without decrypting it, and is used to make sure nothing was corrupted during the transfer process.
  • checksum_algorithm: The algorithm used to compute the checksum. Currently sha256.
  • compression_algorithm: The algorithm used to compress the inner tarball. gzip is used by default. An empty string indicates no compression.
  • purpose: PRODUCTION or TEST
  • version: The version of the metadata specifications.
metadata.json.sig
Detached PGP signature for the metadata.json file.
data.tar.gz.gpg
A tarball encrypted using the receiver’s public PGP key and signed with the sender’s private key. It is compressed with compression_algorithm.
[file1], [file2]
One or several data files can be transferred. Data to be sent can be in any format, e.g. .txt, .csv, .dat.
checksum.sha256
sha256 checksum file of all files present in data.tar.gz. This is used to make sure nothing was corrupted during the encryption/transfer/decryption process.

File example

Examples of the content and structure of the metadata and checksum files.

metadata.json
{
    "transfer_id": 42,
    "sender": "AAABFBC698539AB6CE60BDBE8220117C2F906548",
    "recipients": ["D99AD936FC83C9BABDE7C33E1CF8C1A2076818C3"],
    "timestamp": "2020-01-29T15:31:42+0100",
    "checksum": "a8eb0ee5a6a53326b1c6f9bf94136da5d98a1dc6dceee21d62f694d71c4cf184",
    "checksum_algorithm": "SHA256",
    "compression_algorithm": "gzip",
    "purpose": "PRODUCTION",
    "version": "0.7"
}
checksum.sha256
41421f0c4a5353a5a0cdd37de3fd80a840a190ca997ad8044a67c4c1683f7b63 file1.csv
35ba157ed1c3269d731a438c466790a4f481bb49805e2d1f380df0c636792ff6 folder1/file.txt
fd9ebdbcc1a5fc35ded6e78a6b16ef658502c9d0b05dd4a2185d0f94ccf165cf folder1/folder2/file.txt