Friday, August 11, 2017

Drobo to Drobo Serverless backup using SSH, RSYNC and CRON

Would you like to automatically backup your data from one drobo to another drobo automatically, without a computer doing the copying?

Heres how:

Use Case

I have two identical drobo5ns. One has more capacity than the other. They are both connected to a GB router.

Drobo5n1 - 5 x 2TB
Drobo5n2 - 5 x 3TB
Mac Desktop

I am paranoid about losing my data so I keep a backup of my main Drobo5N on another one. Typically I open both shares on my MAC, then drag and drop files between them. This has lots of problems with it which I won't go into but basically it takes ages and i hate leaving my MAC on doing the copying.

I want to automatically backup my data form one Drobo to the other, without using my MAC, and do some fancy synchronising so i only copy newer files over... AND...  i don't want to copy permissions. I'll cover this shortly.

Good plan huh?

Well, yes. BUT FUCK ME this was REALY difficult to accomplish and took alot of research and testing

Hopefully this will help others.

So this is what i did...

Format the Target Drobo (Its optional but i'll tell you why)

I formatted the target Drobo.. this is because the original data is from many places, has both MAC and Windows permissions all over it, and a bunch of MAC metadata files that i don't like or need. Trying to compare and copy these files was not going so great from one to another doesn't work very well at the best of times.

Starting fresh or just in a new share means you'll have none of these problems. But i never want to have the same problems again either. Even on a fresh target, once the files are copied i'm going to have the same problems again I think as the permissions will still be there.

For the most part, most of the files have Windows NTFS permissions, but the computers and users i created those files with don't exist anymore. I use a MAC now, and maybe one day I'll use windows again or a Google table or whatever.

So for my use case I don't want to fight with the old permissions. I want nice and bland files with the default Drobo permissions. Then i can access my files anyway i like later.

* this is the plan at least I haven't fully compared permissions after copying yet

If you are interested in how Drobo can manage to copy the files in the first place.. its using a sticky bit on at the root level of each share, effectively giving Drobo admin full access to all files on the system. Here is more info on sticky bit

Lastly: Do this at your own risk, and if you don't want to, try it on a new share at least ;)

Update your Drobo Firmware and Drobo Dashboard

Connect to the second drobo
- I set the username to lowercase admin as i like it that way.
- Set the password
- Rename drobo
- Enable DroboApps in the admin console
- Set a static ip
- Create a taget Share with admin has full permission - eg DATA
- Restart

Upload OpenSSH

You need to do this on both drobos. On your computer connect to the Drobo as admin. This makes the drobapps folder appear. Download the file openssh.tgz and save it in the /drobapps/ folder


Install Dropbear 

You need to do this on both drobos. Connect to the Drobo Dashboard app again and goto Droboapps. Install dropbear and start it.

use putty, or on a mac just ssh to the drobo with the admin username and password
$ ssh admin@


Switch to the root user
# sudo su -
run this to unpack and install (this is basically how you install any drobo ported app)
# /usr/bin/ install

Configure OpenSSH on port 2222

Dropbear runs on port 22 so i avoid using OpenSSH on that port by changing the config using vi

# vi /mnt/DroboFS/Shares/DroboApps/openssh/etc/sshd_config

Press i to start insert mode. edit this this line
#Port 22
to this
Port 2222

Press :
then wq (write quit)

All done, goto the drobo dashboard and stop start the OpenSSH app (down the bottom of all the apps).

From here on, connecting via putty or SSH is done on port 2222 to take advantage of OpenSSH. So from your computer you can do
$ ssh -p 2222 admin@192.168.1.x

Create .ssh folder

You need to do this on both drobos.

I am not sure but i think dropbear creates this folder, and this file
# /mnt/DroboFS/home/admin/.ssh/
# /mnt/DroboFS/home/admin/.ssh/authorized_keys

If not create them, and set correct permissions. I 'think' this should do it...
# mkdir /mnt/DroboFS/home/admin/.ssh/
# touch /mnt/DroboFS/home/admin/.ssh/authorized_keys
# chmod 700 -R /mnt/DroboFS/home/admin/.ssh/

Before we move on...

Why use OpenSSH at all? Well dropbear is super slow is why... i believe this is due to windowsizes

Create SSH keys

You need to create the keys on drobo2, and copy the public key to drobo1. This is because drobo2 will be running  the sync script and needs to connect to drobo1 and pull the files. You can do it in reverse but i like the second drobo doing all the work. Also, i feel safer messing around on my second drobo..

Now lets make some keys yeah? On Drobo2... cd to the openssh app and list files

Drobo5N2:~ # cd /mnt/DroboFS/Shares/DroboApps/openssh/bin
Drobo5N2:~/Shares/DroboApps/openssh/bin # ls
scp*         sftp*        slogin@      ssh*         ssh-add*     ssh-agent*   ssh-keygen*  ssh-keyscan*

Run this and accept the defaults, but please read the bloody prompts at least and make sure the files end up as per below

# ./ssh-keygen

Save files to


Cat the pub key and copy that garbage to clipboard

# cat /mnt/DroboFS/home/admin/.ssh/

from your computer logon to drobo1 via OpenSSH on port 2222
$ ssh -p 2222 admin@192.168.1.x
Create the directories you did before.
edit the authorized_keys file with vi, and paste the SSH key into the file and :wp to save



Login with no password

Drobo2 is going to do all the copying so go back to drobo2. Unfortunately, the ssh executable is not in the 'PATH' so you have to run it with the full path. I tried adding it to the path but it gets magically overwritten.

Run this with the PRIVATE key (id_rsa) and you should log straight into drobo1 with no password

# /mnt/DroboFS/Shares/DroboApps/openssh/bin/ssh -i /mnt/DroboFS/home/admin/.ssh/id_rsa admin@192.168.1.x

exit drobo1 and re-run that command as the root user on drobo2. Accept the prompt to add this connection to known hosts. If you don't do this step, when you run CRON later as root it'll fail. 
# sudo su -
# /mnt/DroboFS/Shares/DroboApps/openssh/bin/ssh -i /mnt/DroboFS/home/admin/.ssh/id_rsa admin@192.168.1.x

Did it connect with no pwd - it should have, If it didn't work... thats sucks. Maybe your SSH keys might not have right permissions. Look that shiz up. chmod some shit

Now for the SSH / RSYNC part

The basic copying process works by creating an tunnel, then contacting rsync on the destination, comparing the files on both ends, creating a list, and transfering the files.

First off lets talk about how rsync works for for me:

The trick to do this on drobo is to tell the destination where to find the rsync executable. You need to include this..


For my setup i specifically include the folders i want and exclude the files / patterns i don't want copied

--files-from=/mnt/DroboFS/home/admin/include.txt: include these files. eg.
--exclude-from=/mnt/DroboFS/home/admin/exclude.txt exclude these

I exclude this MAC metadata crap in my file


Then there are the switches to do the sync. once apon a time i would use -a, but that copies the permissions, so i just use -r

r: recursive
v: verbose
h: human readable
--progress: show the progress
--ignore-existing: only copy new stuff
--delete-after: when done, delete stuff on the target that is not on the source
--delete-excluded; also delete stuff i put in the excluded file on the source

Rsync stages the file copy on the destination when it copies. I like to dump the partial files in a completely different place for better visibility etc

--temp-dir=/mnt/DroboFS/Shares/Public/partial: copy temporary, partial files here

Lastly, log the copy process. We'll be running this as a CRON job so you need this or you won't know what going on.



Now for the full SSH command

So how does the SSH part work in more detail? Basically we

execute rsync on the target machine over SSH on port 2222 using the PRIVATE key (id_rsa)
...then all the rsync commands
...then the destination
.. then the source (and folders from the include file)

rsync -e "/mnt/DroboFS/Shares/DroboApps/openssh/bin/ssh -p 2222 -i /mnt/DroboFS/home/admin/.ssh/id_rsa"

--rsync-path="/sbin/rsync" -rvh --temp-dir=/mnt/DroboFS/Shares/Public/partial --progress --ignore-existing --delete-after --delete-excluded --files-from=/mnt/DroboFS/home/admin/include.txt --log-file=/mnt/DroboFS/home/admin/data.log --exclude-from=/mnt/DroboFS/home/admin/exclude.txt



Put it all together

rsync -e "/mnt/DroboFS/Shares/DroboApps/openssh/bin/ssh -p 2222 -i /mnt/DroboFS/home/admin/.ssh/id_rsa2" --rsync-path="/sbin/rsync" -rvh --temp-dir=/mnt/DroboFS/Shares/Public/partial --progress --ignore-existing --delete-after --delete-excluded --files-from=/mnt/DroboFS/home/admin/include.txt --log-file=/mnt/DroboFS/home/admin/data.log --exclude-from=/mnt/DroboFS/home/admin/exclude.txt admin@ /mnt/DroboFS/Shares/Data/

NB: Keep in mind that the folders i am copying are in my includes text file so really this command will be doing this..

admin@ /mnt/DroboFS/Shares/Data/folder1
admin@ /mnt/DroboFS/Shares/Data/folder2
admin@ /mnt/DroboFS/Shares/Data/folder3
admin@ /mnt/DroboFS/Shares/Data/folder4 ...


Now lets put that command in a script

Simply create a bash script with vi, put your command in it, save it, then chmod it with +x and run it.

# vi
# /bin/bash
# your command

Save, then
# chmod +x
Execute ./

You should see rsync calculate the file list then start copying like mad. Make sure this works as the root user before you attempt doing a cron job.


Setup crontabs

Drobo uses busybox for most of its linux style commands including crond.

To get crond working you need to sudo to root, create 'some' of the default path, then create a crontab, and start crond

Drobo5N2:~ $ crond --help
BusyBox v1.23.2 (2015-11-06 13:39:54 PST) multi-call binary.

Usage: crond -fbS -l N -d N -L LOGFILE -c DIR

-f Foreground
-b Background (default)
-S Log to syslog (default)
-l N Set log level. Most verbose:0, default:8
-d N Set log level, log to stderr
-c DIR Cron dir. Default:/var/spool/cron/crontabs

# sudo su -
# mkdir -p /var/spool/cron
# /bin/busybox crond

The its basic crontab stuff, which runs as root so thats cool. Run the command below to create a job list and use cron numbering to schedule a script to run the SSH / RSYNC command when you want it run.

# crontab -e

Keep in mind the drobo uses PDT time so set your job accordingly

Drobo5N2:~ # date

Fri Aug 11 05:00:54 PDT 2017

To view the crontab you created.. as root

Drobo5N2:~ # crontab -l
51 16 * * 6 /mnt/DroboFS/home/admin/*

Drobo5N2:~ # 


When getting started, you can put a -n in your rsync switches to do a dry run copy
Assuming crond kicks off, you can tail the log file and see if copying has started
You can also do ps aux | grep rsync on either drobo and watch the process kick off if its working


You can do a -I if you want to just overwrite whatever is on the target. Sometimes i find this handy.
Kill the crond command if its not working nicely on first go

Last but not least here are some references i used for this magic.

If this page has helped you .. well sweet. You r welcome.




busybox tools and  commands


Drobo dropbear windowsize sucks the dick - windowsizes

i tried using a drobo ported version of rsync at some stage but gave up

dropbear man page

dropbears equivalent to 'ssh' (dbclilent)

basic rsync operation

rsync with includes

cron scheduling

busybox tools and  commands

drobo pinhole reset

1 comment: