Tag Archives: Syncthing

Syncthing v0.14.40, Raspberry Pi, 100% CPU

I think Syncthing is an amazing piece of software, but I ran into problem last week.

I have a library of 10 different folders, 120000 files, 42000 directories and 428GB of data.

I thought that was a little bit too much for my RPi V1 (Syncthing 0.14.40, Arch Linux), because it constantly ran at 100%. I raised Rescan Interval to several hours (so it would finish before staring over).

After startup it took about 10-15 min to get the web GUI up, and about an hour to scan all folders for the first time. Well, that is ok, but after that it still constantly used 100% CPU despite all folders were “up to date”.

It turned out it crashed and started over. I found panic logs in .config/syncthing and error messages in ./config/syncthing/index-v0.14.0.db/LOG.

Some errors indicated Bad Magic Number and Checksum Corruption. The usual reason for this seems to be hardware problem (!?!).

I upgraded my RPi V1 to an RPi V2, with little success. Then I found that I had similar problems on another RPi V2. So after shutting down Syncthing I tried the quite scary:

  $ syncthing -reset-database      ( does not start syncthing )      
  $ syncthing                      ( start syncthing )

After several hours of scanning everything seems to work perfectly!
Let us see how long that lasts.

Syncthing: breaking upgrades

Syncthing is usually very easy to upgrade: it upgrades itself silently or via apt-get. Problem is, when it changes version (lastly from 0.12 to 0.13) it is not compatible anymore. It is not a big problem since it is easy to upgrade, but you kind of have to upgrade your entire Syncthing cluster at the same time, otherwise you get an undesired fork.

This was the quite confusing error message i got on the newly updated system that failed to connect to non upgraded systems:

[THC2C] 19:17:52 INFO: Failed to exchange Hello messages with <ID> (<ADDRESS>): EOF

So, be mindful when upgrading syncthing so you dont get a non breaking upgrade when you dont have time to upgrade everything.

Syncthing on Android

I installed Syncthing a few weeks ago. Now I found it was time to connect my Android mobile to it. Installing Syncthing via Google Play was easy. Configuring it, not that easy. The amount of useful error messages… close to zero.

I found:

  1. When I manually write the address to my other syncthing unit (like my NAS), only IP address works (with :port after it). Writing a domain name fails.
  2. When sharing a folder, I can not share a folder on the SD card: I get something like “Error (100%)”. To me, this is a pity, because I could put a big SD card (32-64GB) and have synchronised music there… but it seems not possible.

Update 2015-11-14: Upgraded to new syncthing version (0.12.2). Syncthing (for Android) now does not start properly. It just keeps “Loading”. No error message. No way to interact with it.

Syncthing rocks!

For quite a while I have been using ownCloud to syncronize and backup files. But enough about ownCloud, I think I have found something much better for my needs: Syncthing.

It is still early, but a few things I like:

  • Simple command line server with webGUI, makes it easy to run on headless machines, and the same GUI for all platforms
  • Light on resources (it actually works well on my QNAP-TS109: ARMv5@500MHz and 128MB RAM)
  • It had Debian packages, even for armel
  • It is easy to use – just start using it and you will get it immediately
  • No need for a webserver or database
  • There is no single centralized server that can crash, requires special updates, or that requires a recovery plan.

I have now synced ~13000 files, ~75GB, to my QNAP. It uploads/downloads at about 250-500kb/s (kilobytes), and it uses about 60-70MB of RAM in doing so. With other hardware is it much faster – I think a Raspberry Pi with a USB drive will be excellent.

Synching seems based on Go, and Go seems not to be available for PowerPC, so my old G4 that has been struggling with ownCloud for a while will need something else to do.

Update 2015-10-11: RAM Usage
Adding more files (and more devices?) to your Syncthing environment increases RAM demands. Consider the below table a draft.

Device Shares Files MBs RAM MBs
QNAP TS-109 2 1045 687 27
Raspberry Pi1 9 8094 46500 82