Experimenting with clustering and data replication in Nextcloud with MariaDB Galera and SyncThing

(Jason Bayton) #1

Originally published at: https://bayton.org/2017/06/experimenting-with-clustering-and-data-replication-in-nextcloud-with-mariadb-galera-and-syncthing/

Update After discussions with the Nextcloud team and guys at TU Berlin, the below could be officially supported with some small changes. See the updates noted against the challenges. A rewrite or additional post will be coming soon to address and test the changes. Nextcloud works really well as a standalone, single-server deployment. They additionally have some great recommendations for larger deployments supporting thousands of users and terabytes of data: Up to 100,000 users and 1PB of data What wasn’t so apparent until last week, however, is how someone might deploy Nextcloud across multiple datacentres (or locations) in a distributed manner wherein each Node can act as the “master” at any point in time; federation is obviously a big feature in Nextcloud and works very well for connecting systems and building a trusted network of nodes, but that doesn’t do an awful lot for those wanting the type of enterprise deployment pictured above, without having all of the infrastructure on one network. Now that Global Scale has been announced this will likely be the way forward when it’s ready, however given I’d already started a proof of concept (PoC) before NC12 was officially made available, I kept working away at it regardless – more for my own amusement than anything else for reasons explained further down. The concept The theory was as follows: If I’m at home, the server in the office is the best place to connect to since it’s on the LAN and performance will be excellent. In this case, a DNS override…

(Johan Van Huyssteen) #2

Hi Jason

I have build a solution for my customers as follows:
ubuntu 16.04 lts server
freenas for storage
syncthing to backup the data to my cloud storage

and its working excellent.

I read through your post and are impressed that you got syncthing to work with the apache www-data user in the service.

How did you achieve this I would love to backup some user data form the /var/www/html/owncloud/data folder without the permission challenge.

Keep up the good work and be blessed

Thanks in Advance


Installing Nextcloud on Ubuntu 16.04 LTS with Redis, APCu, SSL & Apache
(Jason Bayton) #3

Hey, thanks for the comment!

I created a new systemd file that essentially ran syncthing as www-data. Nothing fancy. Scroll up a little from testing and you’ll see them: