How to set up a decentralized storage infrastructure with Infinit and Digital Ocean

Infinit is a decentralised file storage platform allowing you to create storage infrastructures that are scalable, secure, flexible, fault-tolerant and easy to deploy. You can read more about the project here.

With this tutorial, our goal is to create a simple NFS-like infrastructure with only two machines. One will be used to store files (a Digital Ocean server) and the other one (your computer) will access them through a mountable volume.

The infrastructure we want to create:

What follows is a step-by-step guide to set up this infrastructure for a single user with these two devices (named A and B). Device A will contribute storage capacity to store the blocks of data composing the files while device B will read and write blocks over the network. This creates an NFS-like system even though it is also possible to create infrastructure where more than one node stores blocks.

From now on, the device A will refer to your Digital Ocean droplet, and device B to your computer.

Warning: this guide has been published with version 0.6 of the Infinit file system and is written for Linux users. If you're a Mac user or if something does not work as expected, be sure to check our complete and up-to-date Get Started guide on our website directly.



Create your DigitalOcean Droplet

Go to your DigitalOcean account and create a new droplet on Ubuntu. Once it's done you will need to use ssh to install Infinit on this machine. DigitalOcean published a guide on how to connect to your droplet with ssh, so simply follow their guide!

Download and install Infinit’s dependencies

Infinit relies on FUSE to create filesystems in userland. Once connected to your DigitalOcean machine with ssh, you will need to install FUSE using apt:

$> sudo apt-get install fuse

Download and install the Infinit command-line tools

First import the public key used by the package management system:

$> sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 3D2C3B0B
...
gpg: key 6821EB43: public key "Infinit <contact@infinit.one>" imported  

Then add the repository locally:

$> sudo add-apt-repository "deb https://debian.infinit.sh/ trusty main"

Finally, you can update your local list of packages and install the command-line tools as you would any other package:

$> sudo apt-get update
$> sudo apt-get install infinit
...
Unpacking infinit (0.6.0)...  
Setting up infinit (0.6.0)...  

You can now change to the install directory:

$> cd /opt/infinit
  • The bin/ subdirectory contains the actual Infinit binaries such as infinit-user, infinit-network, etc.
  • The lib/ subdirectory contains all the libraries the above binaries depend upon to operate (excluding the FUSE library you installed earlier).
  • The share/infinit/filesystem/test/ subdirectory is provided for you to quickly test the command-line tools.

Create Infrastructure

It is now time for you to create your own storage infrastructure. First, add the bin/ directory to the PATH environment variable to be able to invoke the command-line tools from anywhere:

$> export PATH=$PWD/bin/:$PATH

Create a user

The first step consists of creating a user on the Hub. All the commands that follow use the user name ‘alice’ but you should pick your own unique user name:

$> infinit-user --signup --name alice --email alice@company.com --fullname "Alice"
Generating RSA keypair.  
Remotely pushed user "alice".  

Create a storage resource

A storage resource behaves like a hard disk, storing data blocks without understanding the data’s meaning. The beauty of Infinit is that it is completely agnostic of the nature of such storage resources.

Next, we are going to declare a local storage resource. A local storage stores data blocks as files in a directory (such as /var/storage/infinit/) on the local filesystem.

The binary infinit-storage is used for this purpose. The option --filesystem is used to indicate that the storage will be on the local filesystem:

Device A
$> infinit-storage --create --filesystem --name local --capacity 1GB Created storage "local".

Create a network

Now that we have at least one storage resource to store data, we can create a network interconnecting different machines.

The infinit-network command is used to create the network, specifying a name along with the list of storage resources to rely upon. We will ignore the other options now but you can read about them in the reference documentation. In this example, only the ‘local’ storage resource is used but you could plug as many as you like. Obviously, you need to substitute 'alice' with your username:

Device A
$> infinit-network --create --as alice --storage local --kelips --name my-network --push Locally created network "alice/my-network". Remotely pushed network "alice/my-network".

NOTE: The --push option is used to publish the created network (likewise for other objects) onto the Hub for it to be easily fetched on another device or shared with other users.

Create a volume

The last step on this device consists of creating a logical volume to store and access files. Volumes can be manipulated through the infinit-volume binary as shown next:

Device A
$> infinit-volume --create --as alice --network my-network --name my-volume --push Locally created volume "alice/my-volume". Remotely pushed volume "alice/my-volume".

That’s it, you’ve created a volume named ‘my-volume’ i.e. a filesystem. The blocks that the files are composed of will be distributed across the network named ‘my-network’, currently composed of a single computer with a single storage resource.

Mount the volume

Let’s access this volume by mounting it as easily as any other filesystem:

Device A
$> infinit-volume --mount --as alice --name my-volume --mountpoint ~/mnt-my-volume/ --async --cache --publish Fetched endpoints for "alice/my-network". Running network "alice/my-network". Remotely pushed endpoints for "alice/my-network". Running volume "alice/my-volume". ...

NOTE: This command does not return. You can make it run in the background if you prefer. To stop it and unmount the volume, just hit CTRL^C or interrupt the process. You should wait until the end of the guide to stop this process though.

That’s it! You can now create, list and access files from the mount point ~/mnt-my-volume. Try creating a file right now:

Device A
$> echo "everything is" > ~/mnt-my-volume/awesome.txt $> cat ~/mnt-my-volume/awesome.txt everything is

Access from another machine

Now that you have successfully created and mounted a volume on your droplet, it would be interesting to access the data from your computer.

In order to access your volume from your computer, you will need to transfer your user’s identity to that device. A user’s identity is analogous to an SSH key pair and should be treated in the same way. Because of its critical nature, it is not stored on the Hub like network and volume descriptors could be for instance. For this guide, we will rely on the Hub to transmit the identity in encrypted form to the other device.

Let’s transmit the user’s identity to the other device. Note that it will be encrypted with a passphrase chosen by you. Also you will only have 5 minutes to retrieve it on device B.

Device A
$> infinit-device --transmit --user --as alice Passphrase: ******** Transmitted user identity for "alice". User identity on the Hub for: 297 seconds

Now let’s move to device B. First, you need to download, install and configure the Infinit command-line tools on your computer as you did for device A. Please refer to the first sections of this guide for that purpose (for Mac users, we also offer a Mac version you can download on our website).

Device B
$> infinit-device --receive --user --name alice Passphrase: ******** Received user identity for "alice".

NOTE: The pairing process may have expired on device A. If so, please try again and enter the passphrase on device B in order to retrieve your user identity before the counter runs down.

The next steps consist of fetching the resources that you previously pushed on the Hub: networks and volumes.

Device B
$> infinit-network --fetch --as alice --name my-network Fetched network "alice/my-network". $> infinit-volume --fetch --as alice --name my-volume Fetched volume "alice/my-volume".

Let’s link this device to the ‘my-network’ network you created on device A.

Device B
$> infinit-network --link --as alice --name my-network Linked device to network "alice/my-network".

Finally, the volume can be mounted on device B as simply as on device A:

Device B
$> infinit-volume --mount --mountpoint ~/mnt-my-volume2/ --as alice --name my-volume --async --cache --publish Fetch endpoints for "alice/my-network". Running network “alice/my-network”. Running volume "alice/my-volume". ...

It is now time to check if the file you created on device A is synchronized with device B:

Device B
$> ls ~/mnt-my-volume2/ awesome.txt $> cat ~/mnt-my-volume/awesome.txt everything is

That’s it, you’ve created a filesystem that you quickly connected to with two devices, without having to go through a complicated administrative process.

Go further

If you want to learn more about our file system and build different infrastructures, please see our complete Get Started Guide or our Deployment Examples on our official website.


Get started with Infinit in less than 10 minutes! Test it!

Gaëtan Rochel

designer / web dev @ docker (infinit crew)

Subscribe to Infinit's Blog

Get the latest posts delivered right to your inbox.

or subscribe via RSS with Feedly!