The Courier Who Speaks Every Cloud — rclone

1. The Drive That Started Clicking

Selvi shoots weddings in Madurai. Eleven years of work lived in three places, and none of them felt safe.

There were two external hard drives on her desk, one labelled WORK and one labelled WORK COPY in her own handwriting. There was a Google Drive plan that had sent her a "storage almost full" email every week since March. And there was her laptop, which held whatever she was editing that month.

Eight days before she had to deliver an album to a family in Theni, the WORK drive began to click. A soft, regular tick, like a clock in another room.

She copied the Theni folder off it in a panic. It came out whole. Then she opened WORK COPY to check what else was safe, and found that her "copy" had stopped in October of the previous year. She had dragged folders across by hand whenever she remembered, and she had stopped remembering.

She called Imran, who looks after servers for a logistics company and has helped her with every piece of technology she owns.

"I don't need a lecture," she said. "I need one tool that can move my photos between the drives, Google, and wherever else is cheaper, and I need it to happen without me."

"Then you need a courier who speaks every cloud's language," Imran said. "It's called rclone. People describe it as rsync for cloud storage. It's a single program, it's open source, and it talks to more storage services than you have heard of."

2. What rclone Is

rclone is a command-line program, written in Go, that copies, syncs, moves, checks and mounts files between storage systems. The project was started by Nick Craig-Wood and is released under the MIT licence. It ships as one binary with no server to run and no account to create.

The idea at its heart is the remote. A remote is a named connection to some storage, such as gdrive: for a Google Drive, b2: for a Backblaze B2 account, or nas: for an SFTP server in a cupboard. Your own disk needs no name at all, it is just a path. Every rclone command takes a source and a destination, and either side can be any remote.

rclone copy /mnt/photos gdrive:Photos
rclone copy gdrive:Photos b2:selvi-photos
rclone copy b2:selvi-photos/2019 /tmp/restore

Same verbs, same flags, whichever cloud is on either end. That is the whole reason it exists.

rclone supports dozens of backends, including Google Drive, OneDrive, Dropbox, Amazon S3 and every S3-compatible service (Backblaze B2, Cloudflare R2, Wasabi, MinIO), SFTP, WebDAV, FTP and plain local disks. It also has a few special remotes that wrap other remotes, and one of those, crypt, becomes very important later in this chapter.

3. Installing It

On Linux and macOS the project's install script fetches the latest stable release.

sudo -v ; curl https://rclone.org/install.sh | sudo bash
rclone version

Distribution packages (sudo apt install rclone) work too, but they are often several versions behind, and some backends and flags in this chapter need a recent release. Windows users can download the zip from the website or use winget install Rclone.Rclone.

Selvi ran it on the small Ubuntu box Imran had set up for her ntfy server, where both external drives were now plugged in. The WORK drive was mounted at /mnt/photos.

4. The First Remote: Google Drive

Every remote is created with one interactive command.

rclone config

It asks a series of questions. For Google Drive the answers were short.

n) New remote
name> gdrive
Storage> drive
client_id>                     (blank for now)
client_secret>                 (blank for now)
scope> 1                       (full access)
service_account_file>          (blank)
Edit advanced config?> n
Use web browser to automatically authenticate?> y

A browser tab opened, she signed in to Google and approved access, and rclone saved a token. On a server without a browser, answer n to the last question and rclone prints a command to run on any machine that does have one, then you paste the result back.

Then she asked the remote some questions of her own.

rclone listremotes          # gdrive:
rclone lsd gdrive:          # top-level folders
rclone size gdrive:Photos   # how many files and how many bytes
rclone about gdrive:        # quota used and free

lsd lists directories, ls lists every file with its size, and lsl adds modification times. rclone tree gdrive:Photos --max-depth 2 draws a quick picture of the layout, and rclone ncdu gdrive: opens a little interactive browser that shows which folders eat the most space. That last one told her that 180 GB of her full Drive was duplicated video exports she had forgotten about.

Tip. Google rate-limits the shared client ID that rclone ships with. If transfers feel slow or you see many 403 rate limit messages, create your own OAuth client ID in the Google Cloud console and enter it at the client_id step. The rclone docs walk through it.

5. Copy, Sync, Move, and the Command That Can Hurt You

Imran made her learn the three main verbs before letting her run anything.

Command	What happens at the destination
`rclone copy SRC DST`	New and changed files are added. Nothing is ever deleted.
`rclone sync SRC DST`	The destination is made identical to the source, including deleting files that are not in the source.
`rclone move SRC DST`	Like copy, then the files are deleted from the source.

"Copy is a guest who brings gifts," he said. "Sync is a guest who rearranges your house to match theirs. Move is a guest who takes your furniture home."

To prove the point, he asked her to type a sync with the arguments the wrong way round and add one flag.

rclone sync gdrive:Photos /mnt/photos --dry-run

The output scrolled for a while and ended with a line saying that 3,412 files would have been deleted. Those were the photos on the WORK drive that had never made it to Google.

--dry-run shows what would happen without touching anything. --interactive (or -i) asks before every change. Imran's rule was simple. Any new sync command runs with --dry-run first, every time, and you read the last lines before removing the flag.

The right direction, with progress shown, looked like this.

rclone copy /mnt/photos gdrive:Photos --progress

rclone compares each file by size and modification time (and by hash where the backend offers one) and skips what is already there, so running the same command again a minute later finished in seconds.

6. A Cheaper, Second Home: Backblaze B2

Google Drive was full, and paying for the next tier just to store eleven years of RAW files felt wrong. Imran suggested object storage, which charges per gigabyte actually stored. Selvi chose Backblaze B2. Any S3-compatible service would work the same way with rclone.

In the B2 web console she created a private bucket called selvi-photos-vault and an application key restricted to that one bucket. Then she added a remote.

n) New remote
name> b2
Storage> b2
account> 0051a2b3c4d5e6f0000000001     (the keyID)
key> K005xxxxxxxxxxxxxxxxxxxxxxxxxxx  (the applicationKey)
hard_delete> false

rclone lsd b2:
rclone mkdir b2:selvi-photos-vault/test

With hard_delete off, B2 hides deleted files rather than destroying them, keeping older versions until a lifecycle rule in the bucket settings removes them. That is a useful safety net, and it also means storage you are paying for, so she set the bucket to keep old versions for 30 days.

7. Locking the Boxes: the crypt Remote

Then Selvi stopped. "These are other people's weddings. Brides, children, grandparents. I promised them their photos stay with me."

"Then the cloud should only ever see noise," Imran said.

rclone's crypt remote wraps another remote. Files you write through it are encrypted on your machine before they leave, and decrypted when you read them back. File names and folder names can be encrypted too. The storage provider sees only scrambled names and scrambled bytes.

n) New remote
name> vault
Storage> crypt
remote> b2:selvi-photos-vault/data
filename_encryption> standard
directory_name_encryption> true
Password or pass phrase for encryption> y   (type your own)
Password or pass phrase for salt> y         (type a second, different one)

Now there were two names for the same bucket.

rclone ls vault:      # 2019/Theni-Kumar-Lakshmi/IMG_0412.CR3
rclone ls b2:selvi-photos-vault/data
                      # p0e52nreeaj0a5ea7s64m4j72s/3ib4dl0o8t...

Imran was very serious about the next part, and so is this book.

The two passwords are the data. rclone stores them in rclone.conf in an obscured form, which hides them from a casual glance but is not real encryption. If you lose the config file and do not know the passwords, nobody, including Backblaze and the rclone authors, can recover a single photo. Write both passwords down, keep a copy of rclone.conf somewhere that is not the same machine, and protect the file itself.

rclone config file prints where the config lives. Inside rclone config, the s option sets a configuration password, which really does encrypt the whole file. rclone then asks for that password whenever it starts, or reads it from the RCLONE_CONFIG_PASS environment variable for scheduled jobs.

8. The Safety Net: Sync With a Backup Folder

Selvi wanted the vault to mirror the WORK drive exactly, so that a renamed folder did not leave a stale twin behind. That meant sync. But sync deletes, and the whole point of a backup is to survive mistakes, including her own.

The answer was --backup-dir. When sync would delete or overwrite a file, it moves the old version into another folder on the same remote instead.

TODAY=$(date +%F)
rclone sync /mnt/photos vault:current \
    --backup-dir "vault:archive/$TODAY" \
    --progress

vault:current always looks like her drive today. vault:archive/2026-11-03 holds anything that was deleted or replaced on the third of November. If she ever deletes a client's folder by accident, the files are waiting in that day's archive. Once a year she can review old archive folders and remove what she no longer needs.

8.1 Filters

Not everything belonged in the vault. Lightroom previews can be rebuilt, and operating systems leave litter everywhere. She wrote a filter file.

# /home/selvi/bin/photo-filters.txt
- .DS_Store
- Thumbs.db
- *.tmp
- **/Lightroom Previews.lrdata/**
- **/Smart Previews.lrdata/**
+ **

Rules are read top to bottom and the first match wins. A minus excludes, a plus includes, and the final + ** includes everything that was not excluded. She tested it before trusting it.

rclone ls /mnt/photos --filter-from /home/selvi/bin/photo-filters.txt | head

Other useful filters are --max-age 7d for recent files only, --min-size 1M to skip tiny files, and --exclude "*.xmp" on the command line for a quick one-off.

9. Trust, Then Verify

A copy that finished without errors is good news. A copy whose contents have been compared is proof.

rclone check /mnt/photos gdrive:Photos --one-way

check compares sizes and hashes between two sides and lists anything missing or different. --one-way only reports files in the source that are missing or different in the destination, which is what you want when the destination holds extra files.

Encrypted remotes need a different command. The hashes B2 stores are of the encrypted bytes, so a plain check can only compare sizes. cryptcheck encrypts each local file's hash the same way and compares that.

rclone cryptcheck /mnt/photos vault:current

The first full run took most of a night. It ended with 0 differences found, and Selvi took a screenshot of it like it was a certificate.

10. Making It Happen Without Her

The whole reason for this chapter was "without me". Imran wrote a small script, and he started it with a guard that comes straight from chapter 1.

"Meera's backup failed because her drive unmounted and nobody noticed," he said. "Yours would fail in a worse way. If /mnt/photos is not mounted, it is an empty folder. And syncing an empty folder makes the vault empty too."

#!/usr/bin/env bash
# /home/selvi/bin/photo-backup.sh
set -uo pipefail

TODAY=$(date +%F)
LOG=/home/selvi/logs/rclone-$TODAY.log
BELL=https://ntfy.selvi.example/backups

ring() { curl -s -H "Title: $1" -H "Priority: $2" -d "$3" "$BELL" > /dev/null; }

if ! mountpoint -q /mnt/photos; then
    ring "Backup skipped" high "/mnt/photos is not mounted. Nothing was synced."
    exit 1
fi

if rclone sync /mnt/photos vault:current \
        --backup-dir "vault:archive/$TODAY" \
        --filter-from /home/selvi/bin/photo-filters.txt \
        --max-delete 200 \
        --transfers 8 \
        --bwlimit "08:00,1M 23:00,off" \
        --log-file "$LOG" --log-level INFO; then
    ring "Backup done" default "Photos synced to the vault on $TODAY."
else
    ring "Backup FAILED" high "rclone exited with an error. Read $LOG on the server."
    exit 1
fi

Three flags in there deserve a sentence each.

--max-delete 200 makes rclone stop with an error if a run would delete more than 200 files. A normal day deletes a handful, so a large number means something is wrong.
--transfers 8 uploads eight files at a time instead of the default four, which suits large RAW files on a decent line.
--bwlimit "08:00,1M 23:00,off" is a timetable. From eight in the morning the upload is held to 1 MiB per second, so her family's video calls and the TV still work. From eleven at night there is no limit.

She ran it once by hand, watched her phone buzz with Backup done, and then handed it to systemd.

# /etc/systemd/system/photo-backup.service
[Unit]
Description=Sync photos to the encrypted vault
Wants=network-online.target
After=network-online.target

[Service]
Type=oneshot
User=selvi
Environment="RCLONE_PASSWORD_COMMAND=cat /home/selvi/.config/rclone/pass"
ExecStart=/home/selvi/bin/photo-backup.sh

# /etc/systemd/system/photo-backup.timer
[Unit]
Description=Nightly photo backup

[Timer]
OnCalendar=*-*-* 01:30
Persistent=true

[Install]
WantedBy=timers.target

sudo systemctl daemon-reload
sudo systemctl enable --now photo-backup.timer
systemctl list-timers photo-backup.timer

Persistent=true means that if the box was off at 01:30, the job runs as soon as it boots. The environment line tells rclone which command prints the config password. Here it reads a file that only selvi can open (chmod 600). A command that asks a password manager works the same way.

11. Opening the Vault Like a Folder

Restoring a single album with rclone copy is easy. Browsing eleven years to find "that photo of the bride's grandmother with the parrot" is not. For that, rclone can mount a remote as a folder.

sudo apt install fuse3
mkdir -p ~/vault
rclone mount vault:current ~/vault --read-only --vfs-cache-mode full --daemon

Now ~/vault behaves like a disk. Her photo viewer opened it, generated thumbnails, and fetched only the files she looked at. --read-only means nothing she does in the viewer can change the backup. --vfs-cache-mode full keeps recently opened files on the local disk so scrolling back is fast. To unmount it, run fusermount3 -u ~/vault.

11.1 Serving files to her assistant

Her assistant Asha edits on her own laptop and needed read access to the current year only. rclone can serve a remote over WebDAV, HTTP, SFTP, FTP, DLNA or even the S3 protocol.

htpasswd -B -c /home/selvi/.config/rclone/htpasswd asha
rclone serve webdav vault:current/2026 \
    --addr 127.0.0.1:8081 \
    --htpasswd /home/selvi/.config/rclone/htpasswd \
    --read-only

It listens only on localhost. In front of it, three lines of Caddy from chapter 3 gave it a real name and HTTPS.

files.selvi.example {
    reverse_proxy localhost:8081
}

Asha added the address as a network drive on her laptop and signed in with her own password. She never saw the encrypted names, and she never needed the vault passwords.

12. Leaving Google Without Losing Anything

The last job was moving out of the full Google Drive. rclone copies cloud to cloud with the same command.

rclone copy gdrive:Photos vault:google-export --progress --transfers 4

The bytes flow through the machine running rclone, so the home connection was the bottleneck. A cheap VPS for a weekend with the same rclone.conf would have been faster. Where both sides are on the same provider (two buckets in the same S3 service, for example), rclone uses server-side copy and the data never leaves the provider at all.

Google Drive also lets two files share the same name in one folder, which most other storage does not. rclone dedupe gdrive:Photos finds those and asks what to do with each one. Selvi ran it in interactive mode, removed exact duplicates, and renamed the rest.

She ran rclone check gdrive:Photos vault:google-export --one-way --size-only before deleting anything from Google (sizes are the honest comparison here, because the destination is encrypted), and only then did she drop to a smaller plan.

13. When Things Go Wrong

Imran left a list taped to the side of her monitor.

Symptom	Likely cause
`403 rateLimitExceeded` from Google Drive	The shared client ID is busy. Create your own client ID, or lower `--transfers` and `--tpslimit`.
`directory not found` at the start of a sync	The source path is wrong or the drive is not mounted. Check before syncing.
`--max-delete` stopped the run	Something removed many files at the source. Look before you raise the limit.
Mount fails with `fusermount` errors	`fuse3` is not installed, or the mount folder is not empty.
Every file transfers again on every run	Modification times are not kept by that backend or were changed. Try `--checksum` or `--size-only`.
`couldn't decrypt` or garbled names	The crypt passwords or the remote path in the config are not the ones used when the data was written.
Uploads are slow on a fast line	Raise `--transfers` and, for many small files, `--checkers`. Add `--fast-list` on bucket storage.

For anything not on the list, run the failing command again with -vv for debug output, and read the backend's page on rclone.org. Most quirks are documented there.

14. Keeping the Courier Honest

Back up rclone.conf and the crypt passwords somewhere separate from the server. Without them, the vault is noise forever.
Update rclone every few months with the install script (rclone selfupdate also works on the official binary).
Read the log after anything unusual. The nightly ntfy message tells you that something happened. The log tells you what.
Follow the 3-2-1 rule. Three copies of anything important, on two kinds of storage, with one copy somewhere else. For Selvi that became the WORK drive, the WORK COPY drive refreshed weekly by the same script, and the encrypted vault.
Test a restore. Not a check. An actual restore of real files to a different disk, opened in a real program.

15. Epilogue: The Restore Drill

The WORK drive died completely three weeks later, on a Sunday. It did not click any more. It did nothing at all.

Selvi did not call Imran. She plugged in a new drive, mounted it at /mnt/photos, and typed one command she had practised the month before.

rclone copy vault:current /mnt/photos --progress --transfers 8

It ran through the night. In the morning she ran rclone cryptcheck /mnt/photos vault:current, read 0 differences found, and opened the Theni album to be sure. Every frame was there, including the grandmother with the parrot.

On Monday the family from Theni called to say they loved the album. On Monday night, at 01:30, her phone buzzed once.

Backup done. Photos synced to the vault.

She did not even pick it up.

Quick Reference Cheat-Sheet

I want to	Do this
Add a remote	`rclone config`
List remotes	`rclone listremotes`
List folders or files	`rclone lsd remote:` / `rclone ls remote:path`
See size and quota	`rclone size remote:path` / `rclone about remote:`
Find what uses space	`rclone ncdu remote:`
Copy without deleting	`rclone copy SRC DST --progress`
Mirror exactly	`rclone sync SRC DST` (try `--dry-run` first)
Keep what sync replaces	`--backup-dir remote:archive/DATE`
Stop a runaway delete	`--max-delete N`
Exclude files	`--exclude "*.tmp"` or `--filter-from file.txt`
Encrypt before upload	a `crypt` remote that wraps another remote
Verify a copy	`rclone check SRC DST` / `rclone cryptcheck SRC CRYPT`
Limit bandwidth by time	`--bwlimit "08:00,1M 23:00,off"`
Browse as a folder	`rclone mount remote: ~/dir --vfs-cache-mode full`
Share over WebDAV	`rclone serve webdav remote:path --addr 127.0.0.1:8081`
Fix duplicate names	`rclone dedupe remote:path`
Debug a failure	add `-vv`

Key takeaways

A remote is a name for some storage. Every rclone command works the same whichever remotes are on either side.
copy adds, sync mirrors and deletes, move takes away. Run every new sync with --dry-run first.
--backup-dir turns a mirror into a history, so a mistake at the source is not a loss in the backup.
Encrypt with crypt on your own machine, and treat the passwords and rclone.conf as part of the data.
Guard the source. Check the drive is mounted and set --max-delete, so an empty folder never empties the vault.
Schedule it, ring a bell, and test a real restore. A backup is only as good as the last time you got files back from it.

Self-hosting gives your files a home. rclone makes sure they also have a second one, somewhere far away, that only you can open.