diff --git a/README.md b/README.md index 6966162..2f0c715 100644 --- a/README.md +++ b/README.md @@ -16,15 +16,22 @@ ADA is a client that talks to the [dCache storage system](https://dcache.org/) [ * Work with labels on files: set labels, remove them, and search directories for files with labels * Work with file metadata (extended attributes): set attributes, delete them, and search directories for files with certain attributes * Stage files (restore from tape), check whether they are online or not -* Show available space in dCache +* Show your disk and tape quotas * Subscribe to server-sent-events to set up automated workflows +* Upload and download files + +Many of these operations can be done recursively. For authentication, ADA supports X509, tokens (macaroons and OIDC) and basic auth (username/password), depending on the dCache configuration. + +When something goes wrong, Ada tries to provide clear, understandable and useful error messages, so you can help yourself, or otherwise your support team will be able to help you better. + +With the `--debug` flag, Ada provides a wealth of information that may help you to troubleshoot problems. If you are a developer, this will help you to understand how to talk to the dCache API. -Many of these operations can be done recursively. For authentication, ADA supports X509, tokens (macaroons and OIDC) and basic auth (username/password), depending on the dCache configuration. #### Limitations -* ADA does not transfer files; we suggest you use [Rclone](https://rclone.org/) for that. +* ADA can upload and download files, but not in bulk, and it's not as efficient as [Rclone](https://rclone.org/). * ADA depends on dCache. The dCache system you work with may have limitations that impact ADA. +* Ada is currently implemented in bash. Bash has limitations; for instance, trying to stage more than 1500 files in a single operation may fail. ### Installation diff --git a/ada/ada b/ada/ada index dd0c053..8602f5f 100755 --- a/ada/ada +++ b/ada/ada @@ -99,6 +99,58 @@ usage() { Please note, that dCache storage systems usually don't have an undelete option. + --upload /localpath/file /remotepath/[file] [--verify-checksum] [--allow-insecure-redirects] + Upload a file from the local system to dCache. + + With --verify-checksum, Ada computes the local file's + MD5 checksum and sends it along with the upload. + If the uploaded file's checksum does not match, + dCache aborts the transfer and removes the corrupt file. + This ensures data integrity during transfer. + + By default, dCache uses Adler32 as checksum. + With --verify-checksum, dCache will store both the + default (Adler32) and the MD5 checksum in its database. + Without --verify-checksum, only the Adler32 checksum + is stored, and not verified. + + Ada does not overwrite files. If the destination file + exists, Ada will print an error message, except when + you specify --verify-checksum: then Ada will compare + checksums of source and destination. + + Ada will try several ways to discover a WebDAV door + to use for the transfer. + If that fails, you can specify the door explicitly by + prepending it to the remote path, for example: + https://webdav.example.org/remotepath/remotefile + + During uploads, Ada will prevent redirects from + HTTPS to HTTP for security reasons. If you want to + allow HTTP (for performance reasons), you can use + --allow-insecure-redirects. + + --download /remotepath/file /localpath/[file] [--verify-checksum] [--allow-insecure-redirects] + Download a file from dCache. + With --verify-checksum, Ada will verify the file's checksum + after download, and show an error message if it doesn't match. + + Ada does not overwrite files. If the destination file + exists, Ada will print an error message, except when + you specify --verify-checksum: then Ada will compare + checksums of source and destination. + + Ada will try several ways to discover a WebDAV door + to use for the transfer. + If that fails, you can specify the door explicitly by + prepending it to the remote path, for example: + https://webdav.example.org/remotepath/remotefile + + During downloads, Ada will prevent redirects from + HTTPS to HTTP for security reasons. If you want to + allow HTTP (for performance reasons), you can use + --allow-insecure-redirects. + --setlabel