Selfhosting a cloud. Part 0: Basic setup

Selfhosting a cloud

  1. Basic setup (this post)
  2. Reverse Proxy with Traefik v2
  3. Secure your services with HTTPS
  4. Nextcloud - everything you need and more
  5. Some nice services you might want
  6. Use keycloak for user management and SSO

Intro

Since you're reading this, you probably want to enter the frustrating marvelous world of selfhosting. Maybe it's because you want to run away from Google, or you simply like messing around with computers and want to see what you can do.

Regardless of your reasons or previous knowledge, this series will teach you how to run your own cloud and how to easily add additional services to it as needed.

In this post I will guide you through the most basic steps from getting a server to running a very basic web service. I assume you have the most basic knowledge of Linux (cd, ls, installing packages)

Disclaimer

Due to how disorganised I am, there will be a lot of tangents and (more or less) off-topic paragraphs.

Any paragraph that looks like this is one of those paragraphs. You can ignore them and information on this page will still make sense.

I will sometimes use these paragraphs to explain something in more details.

Virtual Private Server

The most important thing you need to start selfhosting is a server that will run 24/7. You can use a spare computer you have lying around or buy a professional server to put in your basement, but then you would need to open your local network to the Internet. That could become a security problem if you don't secure your server correctly and your ISP might not be happy with that (not to mention bandwidth caps). You can just rent a server in a data center somewhere. But you probably don't want a whole server (and probably can't afford it). Luckily it's possible to rent just a part of the whole server as a Virtual Private Server (VPS).

A VPS is a virtual machine running on an actual server. It has a limited amount of resources like RAM or CPU cores, but you can get one very cheap. The cheapest VPS I could find (and still recommend) was 3 euros (<4 dollars) a month for some pretty decent specs.

Where to buy a VPS?

I personally recommend Hetzner (that's a refferal link). They have great pricing (about 1/4 the price others want for similar servers). Their cheapest option for 3 Euro is perfect when you're just starting. You can always move to a more expensive plan later if you need/want to.

I'm currently using their 11 Euro plan (CX31) since I needed the extra RAM for hosting game servers. All this SSD space is also useful for cloud file storage.

The only problem is that all their servers are in Europe (2 in Germany and one in Finland). If you're located outside of Europe you might want to look for something closes to you. I've used Digital Ocean before and was quite happy with the service. I've also heard good things about Linode and Vultr, though I didn't use either of them long enough to say anything.

The main reason why you want your server somewhere close to you is latency. If you rent a server in Germany, but you're in USA then every single time you want to access your server the packet has to travel halfway around the world. The lag might be "only" 200ms, but consider that this delay happens for every single packet: every HTTP request, every character typed through SSH... It really quickly goes from slight inconvenience to almost unusable.

When creating a server you will most likely have to choose an operating system that will be installed on your server. If you're not sure which to choose, Debian is usually a good choice. If, however, you're more familiar with something like CentOS or Fedora, then choose them.

Before you buy a server you need SSH keys

SSH keys are mostly used to authenticate users connecting to a different host through SSH. Most VPS providers create only root account on their servers and configure SSH so that you can only connect to that root user using SSH keys.

To generate your own SSH keys paste the following command in your terminal (assuming you have SSH installed):


ssh-keygen -t rsa -b 4096
By default it will put your keys in ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub. It will also ask you to create a password to secure those keys. If you choose a password you will have to input it every time you want to connect to your server.

We can argue about whether rsa is the best algorithm to use here or if 4096 bits is too much/not enough. While in some enterprise environments these keys would be considered too weak, I find that they're secure enough for personal use. Just the fact that SSH doesn't accept passwords is enough to prevent most "hacker" attacks. Plus that's the command Github recommended when I was learning about this stuff.

The file id_rsa.pub is your public key. You can share it everywhere. When your VPS provider asks you for your SSH keys you give them contents of this file. File id_rsa is your private key. Do NOT share it with anyone. If you accidentally leak it, generate new keys and update everything that used that key pair before.

Domain name

Next thing you need is a domain. If you don't know what a domain is, it's the thing you put your browser's address bar. In this guide I will use example.com as the domain name. Whenever you see example.com, replace it with your domain.

A Top Level Domain (TLD) is the end of every domain, like .com or .me.

Where to get one

For most TLDs, you have to buy your desired domain from a registrar. However, if you don't mind an unusual TLD like .tk, you can get a domain for free. You can grab a free .tk domain on freenom.com. For my paid domains I personally use namecheap.com, but I've heard good things about gandi.net.

Setting nameservers

Nameservers are what control what IP address a domain points to. Your registrar uses their own internal nameservers. While you can use them, I recommend you use your VPS provider's nameservers. This has the great benefit of not having to paste the IP address of your server for every record (we'll get to what records are later).

The exact process is different for every registrar and VPS provider, but the general idea is the same: add your domain to your VPS provider's domain interface, get their nameservers and tell your registrar to use those. It's usually pretty easy, but if you don't know what to do then google "{VPS Provider} add domain" and "{Registrar} set nameservers".

Adding DNS records

Now that we (hopefully) have a server and a domain, it's time to actually point the domain to the server. Open your VPS provider's domain configuration interface.

You might see a lot of confusing terms, like record or TTL. Here's a (very) brief explanation on what is what:

There is some more records, but they're not relevant to us right now.

If you want to learn more about DNS, Computerphile on Youtube has some videos that explain it better than I ever would

Create a new "A" record. As host either leave blank or insert "@", depending on your provider. Set Target/Value/IP Address (depending on provider) to your server. Leave TTL as whatever is default, or set it to 1800 (half an hour). If your VPS supports IPv6 then repeat this step but for an "AAAA" record.

Configuring the server

Now that we have a server with a domain pointing to it, we need to configure it. The absolutely first step is adding an user, configuring sudo and adding securing SSH.

In order to configure the server we need to connect to it via SSH. Run this command in your terminal:


ssh root@example.com
This is the only time we'll log in as root.

Adding an user

Choose an username you want to use, then run this command:


useradd -m -U USER
Substitute USER with your username. To create a password run this:

passwd USER
You won't see what you type as the password. You'll need to type the password twice.

Configuring sudo

First we need to know if sudo is even installed:


which sudo
If the result is "sudo not found", then you don't have sudo installed and you must install it yourself.

Now that we know sudo is installed, we need to allow our user to use sudo. The only good way to do it is via the command visudo. This will require a text editor like nano or vi installed. For new people I recommend using nano. Install it if it's not installed by default.

Technically you can edit the file directly, but visudo will make sure you don't make a mistake that would render sudo unusable.

To run visudo with a specific editor run it like this (example for nano):


EDITOR=nano visudo
Now scroll down until you find this line:

root ALL=(ALL) ALL
The syntax for this file is a bit confusing. For now you just need to copy this line but change "root" to your user so it looks like this:

root ALL=(ALL) ALL
USER ALL=(ALL) ALL

Secure SSH

Now we'll configure SSH to not allow logging in via password. This will prevent most brute force attacks.

First we need to add our SSH keys to the new user (they're stored per user). If you want to access the server from multiple computers you need to generate keys on them now. Then to add keys for a computer run this command (on this computer):


ssh-copy-id user@example.com

Log out from your SSH connection to server either by running "exit" or pressing Ctrl-D. After adding keys to the new user, you can now log in as that user to your server:


ssh user@example.com

To secure SSH you need to edit the file /etc/ssh/sshd_config. Since it's in /etc you need root privileges to edit it. Luckily that's exactly what sudo does. This command will run nano as root, allowing you to edit this file:


sudo nano /etc/ssh/sshd_config
It's important you edit sshd_config and not ssh_config.

This file is organised in key-value pairs where the first word is an option and the rest of the line is what this option is set to. Each line starting with a # is a comment that has no effect on configuration. When I tell you to uncomment a line I mean that you should remove the #. Comments are used to show default values and sometimes provide some information on various options

Scroll down until you find the option "PermitRootLogin". Uncomment it and set it's value to "no". Do the same with "PasswordAuthentication". This will completely forbid root login and make make SSH keys required.

To make sure our changes took effect you need to restart SSH server:


sudo systemctl restart sshd
If you don't get any error, then you configured it correctly.

Now try and connect to the server from another terminal. Do NOT close your previous connection until you verified that you can still connect. If you can't connect then you most likely didn't add your keys.

Run a basic service

Now that we have a more or less secure server, it's time to host something on it. For now we'll use a simple demonstration service "whoami". We'll use docker for hosting pretty much everything.

Docker and docker-compose

You're probably wondering what docker is. The (very) simple version is that docker let's you run multiple services on one server, even if they would normally be incompatible (for whatever reason). It also simplifies dependency management - a docker container contains the application as well as everything this app needs.

Running an app in docker requires executing commands which can get pretty long (over one thousand characters is pretty common). Docker-compose allows you to describe your services using a YAML file. For this first service I'll show you the docker command and the equivalent YAML file. Docker-compose has another benefit: you can easily modify how a container is running and you can add these files to git for version control.

Install docker and docker-compose

To install docker head over to official documentation and follow instructions for your OS. Instructions for docker-compose are here. After installing docker you might want to add your user to the docker group:


sudo usermod -aG docker $USER
If you don't do that, then every time you want to do anything with docker you'll have to use sudo.

Now you need to start and enable the docker service:


sudo systemctl enable --now docker

To test if docker works run this command:


docker run hello-world
This should download the hello-world image and print some messages telling you that docker works fine.

Run a whoami service

Docker command

To run the whoami service using plain docker execute this command:


docker run -p 80:80 --name=whoami containous/whoami
Now open your domain in your browser. It should show you some information that's completely useless to you.

A simple explanation for this command:

Pressing Ctrl+C will stop the container. I recommend also running docker system prune to remove the container from disk.

Docker-compose

A good practice when using docker-compose is to keep every service and their requirements (things like DB servers) in their own files. To keep organised we'll keep those files in separate folders.

Create a main folder to keep those files. I recommend the folder to be called "servers" and created in your home directory:


mkdir ~/servers

Now cd into that folder, create a new subfolder called whoami and cd into that. In that folder create and edit a file called docker-compose.yml:


nano docker-compose.yml

In this file paste the following:


version: "3"
services:
		whoami:
				image: containous/whoami
				container_name: whoami
				ports:
						- 80:80
Close the file and run this command to start the service:

docker-compose up -d

Most of this file is straight forward: in services section you define a service similarly to how you did with docker command. The first whoami right after services is how the service is called (that's not container name).

If you're wondering about that version: "3" line, it's pretty much boilerplate. It tells docker-compose which syntax you want available in your docker-compose.yml. Since syntax from versions 1 and 2 work in version 3, there's no reason (that I know of) to use older versions.

The command docker-compose up creates and starts the container. -d flag means that the service should run in background.

Once you test that everything works you can stop the service with


docker-compose down
However, the beauty of docker-compose is that you can modify the docker-compose.yml file and when you run docker-compose up -d the container will be automatically modified and restarted, even if it was already running.

You now have a working server

If you've made it this far, then congratulations! Not only did you endure my attempts at explaining stuff, but if those attempts weren't complete failures you now should have a working server with a simple service and with bare minimum security.

The next post (which should come out in a day or two) will show you how to run and configure a reverse proxy. But don't expect posts 2-4 to be released with the same frequency.