Build Your Own Real-Time Filesharing and Automatic Backup System

A few months ago, Heidi and I bought a desktop computer. We each have working laptops, and while my trusty old Surface Pro 2 shows no sign of slowing down, I’m doing more with 4K video these days and needed something with a little more power. As fast as today’s laptops are, yesterday’s desktop computers offer considerably more bang for your buck, especially with the ability to cheaply add more RAM, more screens, more hard drives, and more GPUs, which are pretty useful for video production.

Since Heidi and I were going to share this machine, I wanted a way to put all of our laptop files on it, so each of us would have all of our work available, regardless of which computer we were using. The tricky part is keeping all of those files up-to-date, so that any change that Heidi makes to a spreadsheet on her laptop get synchronized to the copy of that spreadsheet sitting on the desktop computer. The easy way to do this is simply to store all the files on the desktop and let the laptop just open them over the network, but then she wouldn’t see any of her files if the desktop computer was off or she wasn’t connected to the network.

Today, most people solve this problem by storing files on Google Drive or Dropbox, or some other file sharing service in The Cloud. In addition to keeping files synchronized between as many computers as you like, it also lets users see those files from any web browser. This is handy, but there are several downsides. Dropbox is not secure, only as fast as your internet connection, and, if you’ve got hundreds of gigabytes of video files for each project, kind of expensive. Even for smaller files, it’s remarkably inefficient. From here in Tennessee, my data usually has to hop through more than a dozen servers just to get to Dropbox in San Francisco. I’d rather not send a sensitive document all the way across the country and then back, especially when I’m just moving it to a laptop sitting five feet away.

The best solution I have found is FreeFileSync. It is, as the name suggests, free, and it’s also open source, and operates without ever needing to connect to the internet. Its main downside is a lack of comprehensive documentation for what I wanted to use it for, so it took me a bit of experimenting to get it set up, and that’s why I’m writing these instructions now. If you want real-time file-syncing between more than two computers, read on.

Essentially, FreeFileSync allows you to link two directories, which can be anywhere on the computer or network, and the software will synchronize those two directories. It can be two-way communication, keeping these two folders absolutely identical regardless of which one changes, or a one-way transfer that simply copies everything from Directory A to Directory B, regardless of what is in Directory B. That one-way sync is ideal for backups, but I wanted real-time two-way syncing.

There are three main ways that you can use the software. You can select the directories you want synced and then just hit the transfer button manually whenever you want files to be copied, or you can set up a task to be scheduled and run at certain times, or you can let the program run in the background, constantly watching for changes to a directory and then copying files every time they are created or updated.

Now, I recommend that anyone planning to use it skim through the manual before setting it up. Bear in mind that if you are not careful you can cause file conflicts, save over documents, and if you delete a file in one place it will disappear in the other. There are safeguards against these things, but just remember that any program with the power to move and delete your files can, well, move and delete your files.

When I first started messing with it, I wasn’t entirely sure how it worked. Turns out it’s actually pretty simple. You install FreeFileSync, open it, and create a task, which you then export as a batch file that you can run later. You don’t need server and client side apps for syncing; as long as one computer can see the others on a network, it’s the only one that needs to run the software. I installed it on the desktop, because what I wanted to do is this:

The simpler your file management system is, the easier this is to set up. Heidi stores all of her files in a single directory on the C: drive of her laptop, which is named “Klara.” So, I created a job called “SyncKlara” that synchronizes “C:\Heidi” on her laptop with “C:\Heidi” on the desktop, then saved that as a batch file. I have work files in a bunch of directories on my Surface’s D: drive, and so I linked those to identically named directories on the desktop’s D: drive, and saved that job as a batch file.

Now, inside FreeFileSync’s folder, is another program called RealTimeSync.exe, which does the real time observation of the directories so it knows when new files have been created or old files have been updated. To tell RealTimeSync.exe to load our batch files, we need to create shortcuts that link both to it and the batch files that we created earlier. That link, depending on where you put your batch files, looks like this: “C:\Program Files\FreeFileSync\RealTimeSync.exe” “C:\Program Files\FreeFileSync\SyncSurface.ffs_batch”

Since I have two computers that I want to sync, and I have two batch files, I made two shortcuts and am going to be running two instances of RealTimeSync. Putting these two shortcuts in Window’s startup directory (just for reference: %AppData%\Microsoft\Windows\Start Menu\Programs\Startup) means that next time I reboot I will see two tasks running in my taskbar; one job for each of the two laptops. A red icon means monitoring for changes, green means copying, and grey means the other computer is offline.

The end result works very well. If Heidi wants to work on spreadsheets using the desktop’s two big monitors, all her files are there. If she decides to move to the couch, she just needs to save what she’s doing, close it to prevent conflicts, and all the updated files jump to her laptop as soon as she wakes it up.

If I want to tweak some After Effects projects while I’m at the shop waiting for the CNC machine to warm up, I have everything on my Surface. When I get home and want to render those projects on a beefier computer, I only need to power up my laptop, wait a few moments for the new After Effects files to be copied over, and I’m ready to go.

Also, we have the added security of all of our work files being in two places at the same time. If any of these three drives dies, we still have everything. Of course, I’m still occasionally backing everything up to our giant external drive, but now I’m doing that automatically with a third (scheduled) task in FreeFileSync, and rather than back up the laptops one at a time, I can just grab all the data from the desktop in one shot.

This sort of setup could be done using Dropbox or Google Drive, but only at the speed of our poky rural-Tennessee internet, and everyone from AT&T to the NSA could see our files flying back and forth across the country all day. Our entirely local FreeFileSync solution means that our files are kept up-to-date as fast as Wi-Fi or ethernet can go, we have everything we need even when we travel away from working internet, and all the software runs on our home network, not our laptops.

It seems like everything is moving into The Cloud these days. Pretty much every computing task, either processing, interfaces, or storage, is being done on giant server farms. But personal computers are fast, network hardware is cheap, and even individual hard drives are huge. More importantly, there are some significant advantages to doing things locally. Next time you need some kind of software solution, on a desktop, a laptop, or a phone, think about what you can do offline, with local resources, before signing up for the newest and easiest server-based service.

  1. Thanks for that overview, I have been using Microsoft Sync Toy but am interested in alternatives.

    - Bart

Comments are closed.