Using Syncthing to back up photos from an Android phone

There is a Syncthing client for Android phones, which can automatically sync the DCIM folder (or any folder) from an Android phone to a server. The problem is, that Syncthing works both ways: when you rename, move or delete an image on the server, it will also be renamed, moved or deleted on the phone. Or, if you delete old photos on your phone, because you need more space, those files will also be deleted on the server.

This describes a setup, where we use an intermediate folder to sync the photos from the mobile phone to the server, and then use a small script to copy new files to another folder. Thus, the images in the new folder are completely independent from the intermediate folder (and thus from the mobile phone).

You could also add some lines to the script, to automatically delete images that are older than a few month from your mobile phone, by deleting them from the intermediate sync folder.

You need syncthing running on your mobile phone. Unfortunatelly, there is no good solution for iPhones, because Apple.

On the server, there are two folders, /srv/phone-sync/ and /srv/phone-images/. We then install syncthing on the server, and sync the DCIM folder on the phone with the /srv/phone-sync/ folder on the server. See the syncthing documentation for this.

To sync the new files from one folder into the other, a small bash script helps; you could save it at ~/bin/copy-photos.sh:

#!/bin/bash
 
SOURCEFOLDER="/srv/phone-sync/"   # with trailing slash
TARGETFOLDER="/srv/phone-images/" # with trailing slash
 
# make sure we are not running already:
if [[ `pgrep -f $0` != "$$" ]]; then
	echo "Another instance of this script is already running! Exiting"
	exit 0
fi
 
echo "copying .."
rsync --times --exclude '.*' "$SOURCEFOLDER"* "$TARGETFOLDER"
exit 0

we then need to make sure we can start this script with chmod +x ~/bin/copy-photos.sh

This script calls rsync. It excludes all files starting with a dot (while syncing, Syncthing first creates a hidden .pending-* file before renaming it to the correct filename; we don't want to sync these .pending files, or other files that are hidden, like .trashed* files, that may be trashed photos on the mobile phone). It also and keeps the timestamp (--times) while syncing. You could also add some more flags to rsync, for example --user= and --group= if you want to set a specific user/group combination, or --chmod= if you want to set specific file permissions. Pay attention when using the -a flag, because this will delete files in the target directory that are not present in the source directory, so use -rlptgo instead.

☥ Automatically Copy New Photos

I wanted to automate the copying as soon as new photos are synced via syncthing. I tried out two solutions that kind of worked: inotifywait and Syncthing Folder Event Daemon (both solutions described below). However, I could not get them to run reliably; after a few hours or days it stopped working until I restarted the script, or used systemctl to look at the output (at which point systemd notices that the script is no longer running, and restarts it automatically). So, I ended up with a simple cronjob that runs every night and calls the copy-photos.sh script, which is good enough for my use case.

I've documented my previous attempts below, but I no longer recommend them:

☥ Old Solution: Syncthing Folder Event Daemon

In the past, I used the Syncthing Folder Event Daemon by desbma to watch the synced folder for changes. But I could not get it to run reliably, so I no longer recommend the following setup, but still leave it here for documentation purposes:

desbma wrote a small daemon called Syncthing Folder Event Daemon, or stfed, to run alongside Syncthing and running custom commands when certain folder events happen. This is perfect for our usecase.

See their GitHub page for installation instructions. I write them down here, but they may have changed in the meantime:

# you need a Rust build environment:
sudo apt install build-essential # required for some rust crates
# this uses https://rustup.rs for the installation:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# logout and login again, so that the PATH variable gets updated
 
# download and unzip the code:
wget https://github.com/desbma/stfed/archive/refs/heads/master.zip
unzip master.zip
rm master.zip
cd stfed-master/
 
# build and install stfed:
cargo build --release
sudo install -Dm 755 -t /usr/local/bin target/release/stfed
sudo install -D -t ~/.config/systemd/user ./systemd/stfed.service

The config file should be put into ~/.config/stfed/hooks.toml and looks something like this:

[[hooks]]
 
# Syncthing folder path
folder = "/srv/phone-sync"
 
# Event type, one of:
# file_down_sync_done: triggers when a file has been fully synchronized locally (see filter to match for a specific file)
# folder_down_sync_done: triggers when a folder has been fully synchronized locally
# file_conflict: triggers when Syncthing creates a .stconflict file due to a synchronization conflict
event = "file_down_sync_done"
 
# glob rule for specific file matching for file_down_sync_done events: only include files that do not start with a dot
filter = "[^.]*"
 
# command to run when event triggers
command = "~/bin/copy-photos.sh"
 
# Whether to allow several commands for the same hook to run simultaneously
# if false, and a burst of events comes, the commands will be skipped while the previous one is still running
# optional, defaults to false
allow_concurrent = false

You can then start stfed and see, if everything works:

/usr/local/bin/stfed

It should find the syncthing config by itself. If it doesn't, you may need another config file, ~/.config/stfed/config.toml with the syncthing configuration:

url = "http://127.0.0.1:8384/"  # Syncthing URL
api_key = "xyz"  # Syncthing API key

You can find the API Key in the web ui. See the Syncthing docs for details.

When everything runs correctly, you can add a service to automatically start stfed (and restart it on failure):

systemctl --user daemon-reload
systemctl --user enable --now stfed.service
systemctl --user status stfed.service
# if you ever change the config files, restart the service:
systemctl --user restart stfed.service

☥ Even older Solution: inotifywait

In the past, I used inotifywait to watch the synced folder for changes. But I could not get it to run reliably, so I no longer recommend the following setup, but still leave it here for documentation purposes:

To watch the phone-sync folder, we use inotifywait. If you are on Ubuntu, you can install it via sudo apt install inotify-tools, if it isn't installed already.

To make our lifes a bit easier, we create a bash script, for example in ~/bin/watch-images.sh with this content:

#!/bin/bash
# config
WATCHFOLDER="/srv/phone-sync/"    # with trailing slash
TARGETFOLDER="/srv/phone-images/" # with trailing slash
 
# make sure we are not running already:
if [[ `pgrep -f $0` != "$$" ]]; then
	echo "Another instance of this script is already running! Exiting"
	exit 0
fi
 
echo "starting ..."
 
inotifywait --monitor --event moved_to --exclude '\./.*' "$WATCHFOLDER" | while read line; do
	echo "new file was created: $line"
	rsync --times --exclude '.*' "$WATCHFOLDER"* "$TARGETFOLDER"
done
 
echo "exiting ..."
exit 1

we then need to make sure we can start this script with chmod +x ~/bin/watch-images.sh

In the script, we first define our two folders, then we make sure that the script isn't already running (if it is, we exit). We then call inotifywait and listen to MOVED_TO events. Syncthing works by syncing the file to a hidden file first (.pending-*), and after the sync is complete it renames the hidden file to the correct filename. This renaming triggers the MOVED_TO event. We also exclude all files starting with . from being watched (--exclude '\./.*'). We use the --monitor flag to run forever. If a MOVED_TO event is detected, we call rsync (for every event that is detected). rsync also excludes all files starting with a dot, and keeps the time (--times) while syncing. You could also add some more flags to rsync, for example --user= and --group= if you want to set a specific user/group combination, or --chmod= if you want to set specific file permissions. Pay attention when using the -a flag, because this will delete files in the target directory that are not present in the source directory, so use -rlptgo instead.

You can now start this script manually and test it. If we want to autostart the script after a reboot (or on failure), we can use systemd (if you are on Ubuntu). Create a new file ~/.config/systemd/user/watch-images.service (if the directory does not exist, create it), with the following content:

[Unit]
Description=sync mobile phone images
 
[Service] 
ExecStart=%h/bin/watch-images.sh
Type=Simple
Restart=always
RestartSec=5s
 
[Install]
WantedBy=default.target

you can then enable and start this service with:

systemctl --user daemon-reload
systemctl --user enable watch-images.service
systemctl --user start watch-images.service
systemctl --user status watch-images.service

• Linux
• Backup
• Syncthing

Have a comment? Drop me an email!
This helped you? Consider buying me a ♥ coffee ♥