diff options
| author | Marc Vertes <mvertes@free.fr> | 2024-10-07 22:59:29 +0200 |
|---|---|---|
| committer | Marc Vertes <mvertes@free.fr> | 2024-10-07 22:59:29 +0200 |
| commit | 1121b9e07310ad935dccb36d2678b56e9fab777c (patch) | |
| tree | 8ef4765d79847805d74d5a229ef6ec7dbfc7cf8c /README.md | |
| parent | d04ff4c23a6c1a64fc0488c5bf5af7a4ceffd4a3 (diff) | |
write to external repomain
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 23 |
1 files changed, 19 insertions, 4 deletions
@@ -2,7 +2,7 @@ Incremental encrypted backup system -## design v0 +## Archive v0 1. cksum original (sha256) 2. compress (gzip) @@ -22,7 +22,7 @@ Problems: - dedup occurs only for append only files. The same chunk content will lead to a different hmac if located at a different offset. -## design v1 +## Archive v1 - chunk before compression - name chunks from checksum of uncompressed/unencrypted data (invariant => allow dedup). @@ -37,7 +37,7 @@ Problems: with a foreign key (different user), which would a user to download a block which he could not decrypt. -## design v2 +## Archive v2 Each user has a fixed unique id: random 96 bits (12 bytes). This id is added to the content of each block / file prior to compute the invariant checksum @@ -51,8 +51,22 @@ Problems: - in this design, and all previous ones, there is no way to disgard data in an archive. For example, tarsnap does not allow to suppress data. +## Repository + +A repo is associated with a single id/key tuple, ensuring a deduplification space, +i.e. a unique `chunks` directory. + +Each backup is denoted by its archive index $host:$dir:$date + +$dir is the rootdir of the backuped files. + +A repo entrypoint is its current index, containing the list of $host:$dir:$date. + ## Roadmap +- p2p storage +- chunker based on rolling hash (instead of fixed size) + disgarded: - encode checksums in base64 instead of hex. Wrong idea: incompatible with case insensitive filesystems (macos). @@ -68,6 +82,7 @@ disgarded: ## References - tarsnap: https://www.tarsnap.com https://github.com/tarsnap/tarsnap -- chunker: https://github.com/karinushka/chunker +- tarsnap chunker in Go: https://github.com/karinushka/chunker - borg: https://borgbackup.org - rclone: https://rclone.org +- restic: https://restic.readthedocs.io/en/v0.2.0/Design/ |
