How to Use rclone to Back Up to Google Drive on Linux

Configurare noua (How To)

Situatie

The Google Drive logo on a terminal window-colored background.

There’s still no official Linux client for Google Drive, but you can back up to your Google Drive using the rclone utility right from the command line. We show you how.

Solutie

Where’s Google Drive on Linux?

Despite promising Linux support “coming soon” back in 2012, there’s no indication that Google will ever produce a native Linux client for Google Drive. There are several unofficial third-party solutions, such as InSync, overGrive and ODrive, and some file browsers allow integration with your Google Drive, such as Files in GNOME.

The third-party applications are commercial products, requiring either an outright purchase or a subscription. They work well they don’t cost much, and in fact, overGrive does have a free version, offering limited functionality for no cost.

But what if you want to create and run backups from the command line? Or to incorporate that functionality into scripts? That’s all possible thanks to an amazing application called rclone. In fact, with rclone you can back up, download, and synchronize files to over forty different cloud solutions. It’s like rsync for clouds.

Installing rclone

rclone almost certainly won’t be installed on your Linux computer by default. Happily, there’s an installation script that should work on all distributions. The installation process uses curl. On the computers used to research this article, Fedora 31 and Manjaro 18.1.0 already had curl installed but  curl had to be installed on Ubuntu 18.04 LTS.

On Ubuntu, run this command to install it:

sudo apt-get install curl
sudo apt-get install curl in a terminal window

Once curl has been installed, install rclone with this command:

curl https://rclone.org/install.sh | sudo bash
curl https://rclone.org/install.sh | sudo bash in a terminal window

When the rclone installation has finished, you’ll see a success message.

Success message from the rclone installtion script in a terminal window

This has installed the rclone program on your Linux computer. The next step is to run through the setup process and authenticate rclone to access your Google Drive.

Creating an rclone Remote Connection

Connections to remote cloud services are called “remotes” in the rclone world. We need to create one for Google Drive. Start the rclone configuration process with this command:

rclone config
rclone config in a terminal window

There are a lot of questions in the configuration process. But don’t be disheartened, many of them can be left at their default values and simply accepted by pressing “Enter.”

rclone tells us there are no remotes configured. Press “n” and press “Enter” to create a new remote. It will prompt you for a name. We’re going to call it “google-drive.” Use whatever name you like.

rclone menu to create a new remote, in a terminal window

A long menu allows you to choose the type of storage you’re creating a remote connection to.

rclone remote connection type menu in a terminal window

Scroll through the list until you see the entry for Google Drive, and make a note of its number.

rclone config in a terminal window

We can see that in this instance, it is number 13. Enter this as the storage type and press “Enter.”

Setting the rclone storage type in a terminal window

You’re prompted for a Google Application Client ID. Press “Enter” to accept the default.

Prompt for a Google application client id in a terminal window

You’re then prompted for a Google Application Client Secret.

prompt for a Google application client secret in a terminal window

Again, just press “Enter.” You’re asked to provide the scope that rclone will have when it is operating on your Google Drive. Press “1” and then press “Enter.”

providing the scope for rclone in a terminal window

For the “ID of the root folder”, just press “Enter.”

prompt for root folder ID ina terminal window

At the “Service Account Credentials” prompt, press “Enter.”

Prompt for Service Account Credentials in a terminal window

At the “Edit advanced config”  prompt, just press “Enter.” At the “Use auto config” menu, press “y” and then press “Enter.”

The "use auto config" menu in a terminal window

This causes rclone to communicate to your Google Drive, and to launch your browser to allow you to give permission for rclone to interact with your Google Drive.

rclone about to launch a browser, in a terminal window

In your browser window, click on the Google account you wish to use.

Choosing the Google account to use in a browser window

Click the “Allow” button to allow rclone to have access to your Google Drive.

Allowing rclone to work with Google Drive in a terminal window

When authenticate has completed, you’ll see a “Success!” message in the browser window. You can close the browser and return to the terminal window.

Success message in a browser window

At the “Configure this as a team drive” prompt, type “n” and then press “Enter.”

The rclone "configure this as a team drive" prompt in a terminal window

At the “Yes, Edit, Delete” menu type “y” and then press “Enter.”

The rclone "yes,edit, delete" menu in a terminal window

At the final menu, type “q” and press “Enter.”

The rclone final menu in a terminal window

The rclone Back Up Script

The rclone application is very feature-rich. That’s great, but it does mean there are a lot of options. The command we’re going to look at below copies files from your local computer to your Google Drive. This is a one-way copy to the cloud; it isn’t a two-way synchronization between your Google Drive and your local computer—although rclone can do that. We’re using this as a basic form of off-site backup.

Type (or copy and paste) this into a text editor and save it to your computer. We called it gbk.sh. You can call it whatever makes sense to you.

#!/bin/bash

/usr/bin/rclone copy --update --verbose --transfers 30 --checkers 8 --contimeout 60s --timeout 300s --retries 3 --low-level-retries 10 --stats 1s "/home/dave/Documents" "google-drive:LinuxDocs"

Here’s what the parameters mean:

  • copy: Copy the files from the local computer to the remote storage, skipping over files that are already present on the remote storage.
  • –update: Skip any files that are on the remote storage that have a modified time that is newer than the file on the local computer.
  • –verbose: Gives information about every file that is transferred.
  • –transfers 30: This sets the number of files to copy in parallel.
  • –checkers 8: How many “checkers” to run in parallel. Checkers monitor the transfers that are in progress.,
  • –contimeout 60s: The connection timeout. It sets the time that rclone will try to make a connection to the remote storage.
  • –timeout 300s: If a transfer becomes idle for this amount of time, it is considered broken and is disconnected.
  • –retries 3: If there are this many errors, the entire copy action will be restarted.
  • –low-level-retries 10: A low-level retry tries to repeat one failing operation, such as a single HTTP request. This value sets the limit for the number of retries.
  • –stats 1srclone can provide statistics on the transferred files. This sets the frequency of update of the statistics to one second.
  • “/home/dave/Documents”: The local directory to we’re going to copy to the remote storage.
  • “google-drive:LinuxDocs”: The destination directory in the remote storage. Note the use of “google-drive”, which is the name we gave to this remote connection during the the rclone config sequence. Also note the colon “:” that is used as a separator between the remote storage name and the directory name. Subdirectories are separated by the usual “/” forward slash. If the destination directory does not exist, it will be created.

Some of these values are the defaults, but we’ve included them here so that we can discuss them. That way, if you need to change a value, you know which parameter to adjust.

Make the script executable with this command:

chmod +x gbk.sh

Running the Back Up Script

Our back up script is going to copy our Documents folder to our Google Drive. In our Documents folder, we’ve got a collection of sheet music.

Collection of sheet music in ~/Documents in a file browser

We can launch the back up script with this command:

./gbk.sh
./gbk.sh in a terminal window

We asked for statistics updates every one second (--stats 1s), and we also asked for verbose output (--verbose). It’ll come as no surprise then that we get a lot of screen output. It’s usually a good option to turn on verbose output for new functionality so that you can spot problems. You can turn down the amount of output once you’re happy things are running smoothly.

We get a final summary telling us 60 files were transferred with no errors. The transfer took roughly 24 seconds.

Let’s check on our Google Drive and see what happened in our cloud storage.

LinuxDocs folder in Google Drive

A “LinuxDocs” directory has been created, so that looks promising. If we double-click it to take a look inside, we can see that the files have all been transferred to our Google Drive.

Files in "LinuxDocs" on Google Drive

Using rclone to View Files On Google Drive

We can use rclone to peek into the folder on Google Drive, right from the terminal window:

rclone ls google-drive:/LinuxDocs

rclone ls google-drive:/LinuxDocs in a terminal window

The Tip of the Iceberg

That’s great that we can perform this type of copy straight from the command line. We can incorporate the use of our cloud storage into scripts, and we could schedule the execution of back up scripts using cron.

rclone has an absolute wealth of commands, you’re encouraged to check out their documentation and excellent website. We’ve barely scratched the surface here, and a bit of reading and playing with rclone will pay back the effort many times over.

Strictly speaking, this isn’t a true backup. It is an off-site, remote copy of your files and data, which is definitely a good thing to have, but it is just a copy of files. It doesn’t offer versioning or other features that true backup solutions would offer.

So use rclone in conjunction with other backup techniques. As another layer to an existing backup regime rclone is an easy way to get your data stored in a location that is geographically removed from your home or office. And that’s got to be a good thing.

 

Tip solutie

Permanent

Voteaza

(11 din 22 persoane apreciaza acest articol)

Despre Autor

Leave A Comment?