# `gdal vsi sync`

Added in version 3.11.

Synchronize source and target file/directory located on GDAL Virtual System Interface (VSI)

## Synopsis

    Usage: gdal vsi sync [OPTIONS] <SOURCE> <DESTINATION>

    Synchronize source and target file/directory located on GDAL Virtual System Interface (VSI).

    Positional arguments:
      --source <SOURCE>                Source file or directory name [required]
      --destination <DESTINATION>      Destination file or directory name [required]

    Common Options:
      -h, --help                       Display help message and exit
      --json-usage                     Display usage as JSON document and exit
      --config <KEY>=<VALUE>           Configuration option [may be repeated]
      --progress                       Display progress bar

    Options:
      -r, --recursive                  Synchronize recursively
      --strategy <STRATEGY>            Synchronization strategy. STRATEGY=timestamp|ETag|overwrite (default: timestamp)
      -j, --num-threads <NUM-THREADS>  Number of jobs (or ALL_CPUS)

## Description

gdal vsi sync synchronize files and directories located on GDAL Virtual File Systems (compressed, network hosted, etc...): /vsimem, /vsizip, /vsitar, /vsicurl, ....

This is an analog to the UNIX rsync command. In the current implementation, rsync would be more efficient for local file copying, but gdal vsi sync main interest is when the source or target is a remote file system like /vsis3/ or /vsigs/, in which case it can take into account the timestamps of the files (or optionally the ETag/MD5Sum) to avoid unneeded copy operations.

This is implemented efficiently for:

- local filesystem <--> remote filesystem.

- remote filesystem <--> remote filesystem, where the source and target remote filesystems are the same and one of /vsis3/, /vsigs/ or /vsiaz/. Or when the target is /vsiaz/ and the source is /vsis3/, /vsigs/, /vsiadls/ or /vsicurl/

Similarly to rsync behavior, if the source filename ends with a slash, it means that the content of the directory must be copied, but not the directory name. For example, assuming `/home/even/foo` contains a file `bar`, `gdal vsi sync -r /home/even/foo/ /mnt/media` will create a `/mnt/media/bar` file. Whereas `gdal vsi sync -r /home/even/foo /mnt/media` will create a `/mnt/media/foo` directory which contains a `bar` file.

This is implemented by `VSISync()`.

### Options

-r, --recursive

  Synchronize recursively.

--strategy timestamp|ETag|overwrite

  Determines which criterion is used to determine if a target file must be replaced when it already exists and has the same file size as the source. Only applies for a source or target being a network filesystem.

  The default is `timestamp` (similarly to how aws s3 sync works). For an upload operation, a remote file is replaced if it has a different size or if it is older than the source. For a download operation, a local file is replaced if it has a different size or if it is newer than the remote file.

  The `ETag` strategy assumes that the ETag metadata of the remote file is the MD5Sum of the file content, which is only true in the case of /vsis3/ for files not using KMS server side encryption and uploaded in a single PUT operation (so smaller than 50 MB given the default used by GDAL). Only to be used for /vsis3/, /vsigs/ or other filesystems using a MD5Sum as ETAG.

  The `overwrite` strategy will always overwrite the target file with the source one.

-j, --num-threads <value>

  Number of jobs to run at once

## Examples

### Example 1: Synchronize a local directory onto a S3 bucket

    $ gdal vsi sync -r my_directory/ /vsis3/bucket/my_directory