Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
bdsync
bdsync.exe
bdsync*.tgz
bdsync.1
bdsync.txt.2
*~
*.swp
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -25,4 +25,4 @@ test: bdsync
./tests.sh

clean:
rm -f bdsync bdsync.txt.2 bdsync.1
rm -f bdsync bdsync.txt.2
271 changes: 271 additions & 0 deletions bdsync.1
Original file line number Diff line number Diff line change
@@ -0,0 +1,271 @@
.\" Automatically generated by Pandoc 3.1.12.1
.\"
.TH "BDSYNC" "1" "July 2017" "" ""
.SH NAME
bdsync \[en] a fast \f[B]b\f[R]lock \f[B]d\f[R]evice
\f[B]sync\f[R]hronizing tool
.SH SYNOPSIS
Client: \f[B]bdsync\f[R] [\[en]verbose] [\[en]digest=DIGEST] REMSHCMD
LOCDEV REMDEV
.PD 0
.P
.PD
Server: \f[B]bdsync\f[R] \[en]server [\[en]verbose]
.PD 0
.P
.PD
Patch: \f[B]bdsync\f[R] \[en]patch[=DSTDEV] [\[en]verbose]
.SH DESCRIPTION
Bdsync is like \[lq]rsync\[rq] but it can operate on block devices,
transferring only the changed blocks, to synchronize block devices over
a network.
It generates a \[lq]binary patchfile\[rq] in an efficient way by
comparing checksums of blocks of the local block device LOCDEV and the
remote block device REMDEV.
.PP
This binary patchfile can be sent to the remote machine and applied to
its block device REMDEV, after which the local blockdev LOCDEV and the
remote block device REMDEV are synchronized.
.PP
Both LOCDEV, REMDEV and DSTDEV can be true block devices, but may be
(image) files as well.
When using an image file for DSTDEV (the \f[B]\[en]patch\f[R] mode), the
\f[B]\[en]diffsize\f[R] may be used to resize the image file when
appropriate.
.PP
bdsync was built to do the only thing rsync isn\[cq]t able to do:
synchronize block devices.
.SH OPTIONS
.TP
\f[B]\-s\f[R], \f[B]\[en]server\f[R]
Start bdsync as a server.
No further arguments are accepted, all controll is done by standard
input and standard output
.TP
\f[B]\-p\f[R], \f[B]\[en]patch[=DEVICE]\f[R]
Make bdsync \[lq]patch\[rq] a device by applying a bdsync file read from
standard input.
The device will be determined from the bdsync data unless an (optional)
device name DEVICE is specified
.TP
\f[B]\-w\f[R], \f[B]\[en]warndev\f[R]
Makes bdsync (in patch mode) warn if the specified device with the
\[en]patch option differs from the device in the patch data
.TP
\f[B]\-v\f[R], \f[B]\[en]verbose\f[R]
Increase the verbosity of bdsync.
Can be repeated multiple times.
.TP
\f[B]\-h\f[R], \f[B]\[en]hash=DIGEST\f[R]
Choose any of openssl\[cq]s digests as a hash for matching blocks on
LOCDEV en REMDEV.
Default is md5.
.TP
\f[B]\-b\f[R], \f[B]\[en]blocksize=BLOCKSIZE\f[R]
Set the blocksize in bytes for which hashes are calculated on both
LOCDEV and REMDEV.
Default 4096.
.TP
\f[B]\-c\f[R], \f[B]\[en]checksum=DIGEST\f[R]
Choose any of openssl\[cq]s digests as a checksum for all LOCDEV data.
The checksum will be written to the binary patchfile.
Default is none.
.TP
\f[B]\-t\f[R], \f[B]\[en]twopass\f[R]
Makes bdsync first match checksums using large blocks (64 * BLOCKSIZE)
and then match checksums using small blocks (BLOCKSIZE).
This may reduce systemcall overhead and networktraffic when the
\[lq]binary patchfile\[rq] has limited size.
.TP
\f[B]\-r\f[R], \f[B]\[en]remdata\f[R]
Makes bdsync (in client mode) write the remote data to standard output
instead of the local blocks that differs.
.TP
\f[B]\-d\f[R], \f[B]\[en]diffsize[=OPTION[,warn]]\f[R]
Specifies how bdsync (in client mode and patch mode) should handle
different sizes of devices.
Possible values for OPTION are \f[B]strict\f[R], \f[B]resize\f[R] and
\f[B]minsize\f[R].
When \f[B]strict\f[R] is specified, different sizes for LOCDEV and
REMDEV are not allowed.
When resize is specified, different sizes are accepted and the LOCDEV
size is applied to REMDEV in patch mode which is only supported for
(image) files (not devices).
When minsize is specified, the smallest size of both LOCDEV and REMDEV
is considered (excess data is ignored).
.RS
.PP
When the \f[B]\[en]diffsize\f[R] command line option is not specified at
all, it defaults to \f[B]\[en]diffsize=strict\f[R].
When the \f[B]\[en]diffsize\f[R] is specified without any of the
additional options, it defaults to \f[B]\[en]diffsize=resize\f[R] which
is consistent with earlier versions of bdsync.
.RE
.RS
.PP
When the additional \f[B]warn\f[R] option is specified, a warning is
issued when sizes differ.
.RE
.TP
\f[B]\-z\f[R], \f[B]\[en]zeroblocks\f[R]
Makes bdsync (in client mode and server mode) identify zero\-filled
blocks and optimize hashes for these blocks.
This may be usefull for sparse files with lots of zero filled blocks.
.TP
\f[B]\-F\f[R], \f[B]\[en]flushcache\f[R]
This client option makes both bdsync client and server actively inform
the OS the data is no longer needed after reading it hence reducing OS
buffer cache polution by bdsync.
This works especially well when deltas are small, because in that case
bdsync itself won\[cq]t be reading blocks twice.
.TP
\f[B]\-P\f[R], \f[B]\[en]progress=[noscroll]\f[R]
This client option makes the client periodically report progress to
stderr during operation.
The format is:
.RS
.PP
\f[CR]PROGRESS:\f[R]\f[B]<pct>\f[R]\f[CR]%,\f[R]\f[B]<diffsize>\f[R]\f[CR],\f[R]\f[B]<pos>\f[R]\f[CR],\f[R]\f[B]<size>\f[R]\f[CR],\f[R]\f[B]<elapsed
s>\f[R]\f[CR],\f[R]\f[B]<remaining s>\f[R]
.RE
.RS
.PP
Where: \f[B]<pct>\f[R] is progress in %, \f[B]<diffsize>\f[R] is the
current size of the generated diff, \f[B]<pos>\f[R] is the current
position reading LOCDEV, \f[B]<size>\f[R] is the total size of LOCDEV,
\f[B]<elapsed s>\f[R] is the elapsed time in seconds and \f[B]<remaining
s>\f[R] is an estimate of the remaining time left in seconds.
.RE
.RS
.PP
When \f[B]noscroll\f[R] is specified, each progress update will be
printed on the same line.
.RE
.TP
\f[B]\-H\f[R], \f[B]\[en]help\f[R]
Display brief help information.
.SH USAGE
Bdsync assumes a client is connecting to a server.
The connection isn\[cq]t established by the client itself, but by a
remote\-shell\-command REMSHCMD.
This REMSHCMD can be any kind of command to make a connection: rsh, ssh,
netcat..
If using rsh or ssh REMSHCMD should be the full command to make the
connection including the remote bdsync command te be executed in server
mode.
If using netcat to make the connection, the remote server mode bdsync
command should be executed by inetd, xinetd etc.
.PP
The \[en]verbose option results in verbose output.
In Server mode the output will be sent to syslog.
The \[en]verbose option can be repeated raising the verbosity level.
.PP
Bdsync can be initiated like this in its most simple form:
.IP
.EX
bdsync \[dq]bdsync \-s\[dq] /dev/LOCDEV /dev/REMDEV > DEV.bdsync
.EE
.PP
This generates a patchfile \f[B]DEV.bdsync\f[R] containing the blocks in
the \f[B]/dev/LOCDEV\f[R] device that differ from the blocks in the
\f[B]/dev/REMDEV\f[R] device which both are local.
A more realistic example is this:
.IP
.EX
bdsync \[dq]ssh doe\[at]remote bdsync \-\-server\[dq] /dev/LOCDEV > /dev/REMDEV \[rs]
| gzip > DEV.bdsync.gz
.EE
.PP
When run as \f[B]john\f[R] at \f[B]local\f[R] the bdsync client makes an
ssh connection to \f[B]remote\f[R] as user \f[B]doe\f[R] and executes a
bdsync server by passing it the \[en]server option.
The generated patchfile output is passed on to gzip which creates a
compressed patchfile \f[B]DEV.bdsync.gz\f[R] on local.
The patchfile contains all blocks in \f[B]/dev/LOCDEV\f[R] at local that
differ from \f[B]/dev/REMDEV\f[R] at \f[B]remote\f[R].
.PP
On the remote machine remote the user doe can apply the patch by
executing:
.IP
.EX
gzip \-d < DEV.bdsync.gz | bdsync \-\-patch=/dev/REMDEV
.EE
.PP
The reason to use a binary patch file instead of instantly patching the
remote block device REMDEV is twofold:
.IP \[bu] 2
Transferring a complete patchfile allows to synchronize in a consistent
way in case of an interruption (powerloss, network malfunction) since
you can choose to apply the (complete) patchfile or not.
.IP \[bu] 2
Compression of the patchfile can easily be done, without introducing
complexity in bdsync itself.
.PP
A more complex example of transferring without the intermediate storage:
.IP
.EX
bdsync \-\-diffsize=resize \-\-zeroblocks \-\-flushcache \-\-progress \[rs]
\[dq]ssh doe\[at]remote bdsync \-\-server\[dq] /dev/LOCDEV /dev/REMDEV \[rs]
| ssh doe\[at]remote bdsync \-\-diffsize \-\-flushcache \-\-patch
.EE
.PP
The above command pipes the resulting patch directly to the remote
server to apply.
A compression step could be included in there such as using
\[lq]zstd\[rq], or by passing \[lq]\-C\[rq] to the second SSH command.
.SH EXIT STATUS
0 completed successfully
.PD 0
.P
.PD
1 invalid or conflicting parameters supplied
.PD 0
.P
.PD
2 invalid patch format
.PD 0
.P
.PD
3 size mismatch of source and destination blockdevice
.PD 0
.P
.PD
4 protocol error
.PD 0
.P
.PD
5 checksum error
.PD 0
.P
.PD
6 read error
.PD 0
.P
.PD
7 failed to collect randomness
.PD 0
.P
.PD
8 process management error
.PD 0
.P
.PD
9 write error
.PD 0
.P
.PD
10 digest error
.PD 0
.P
.PD
11 transmission error
.PD 0
.P
.PD
12 IO error
.PD 0
.P
.PD
13 connection error
.SH AUTHORS
Rolf Fokkens.