Installation

The whole installation is performed with ansible so it is required to install it on the computer that will run playbooks. Also, ssh access to all hosts need to be setup.

Prerequisites

Following tools needs to be installed on the computer running the playbook :

• ansible - infrastructure as code software tools.
• age - simple, modern and secure encryption tool.
• htpasswd - user files for basic authentication managerment.
• jq - lightweight and flexible command-line JSON processor.
• mc - command-line tool that allows you to manage your s3.
• openssl - cryptography and SSL/TLS toolkit.
• hvac - HashiCorp Vault API client for Python 3.x.
• sshpass - non-interactive ssh password auth.
• sops - Simple and flexible tool for managing secrets.
• yq - portable command-line YAML, JSON, XML, CSV, TOML and properties processor.

# Clone the repository
git clone --depth 1 https://github.com/this-is-tobi/homelab.git && cd ./homelab && rm -rf ./.git

# Copy inventories examples to inventories
cp -R ./infra/ansible/inventory-example ./infra/ansible/inventory
cp -R ./kubernetes/ansible/inventory-example ./kubernetes/ansible/inventory

Because crowdsec is used as the firewall, it is required to create an account to share attack detection on the local network with the community as the community share it with us.

Notes:

Crowdsec, PiHole and Wireguard installation can be ignored by setting enabled: false in gateway group_vars.

Every kubernetes services can be disabled by commenting its declaration in the Argocd applicationset. Ansible will determine which service is enabled and create the appropriate secrets in vault, it will also update the dashy configmap and may ask to push the updated file for gitops needs.

Settings

Update the hosts file and group_vars files to provide the appropriate infra and services settings.

Actions Runner Controller uses Sops encrypted secret to store information about Github applications. These secrets are managed (encrypted/decrypted) using the wrapper script run.sh following the keys provided in .sops.yaml.

Notes:

Update Sops keys with your own but leave the first age key blank as it is used by the cluster's automated key management system.

Decrypt secrets by running ./run.sh -d and encrypt secrets by running ./run.sh -e, do not forget to recrypt secrets when changes are made.

To create user access to the bastion, it is required to provide their informations in the groups_vars/bastion.yml file :

• Set setup: true to setup the working environment for the given user.
• Put user ssh public key in the inventory file, this will grant user access to the bastion by adding authorized_keys.

Notes:

During setup, every password, token and so on are randomly generated and stored into kubernetes secrets / vault secrets.

Deploy

Two playbooks are available, one for infrastructure installation and another one for services installation. Various tags are available in the playbooks (for more details, take a look at the files), it allows to launch only some part of the installation, the main ones are :

Infra :

# Deploy bastion
./run.sh -p ./infra/ansible/install.yml -t bastion

# Deploy gateway
./run.sh -p ./infra/ansible/install.yml -t gateway

# Deploy cluster
./run.sh -p ./infra/ansible/install.yml -t k3s

Services :

# Deploy kubernetes services
./run.sh -p ./kubernetes/ansible/services.yml

# Deploy only core services
./run.sh -p ./kubernetes/ansible/services.yml -t core

# Deploy only platform services
./run.sh -p ./kubernetes/ansible/services.yml -t additional

# Deploy only keycloak
./run.sh -p ./kubernetes/ansible/services.yml -t keycloak

Notes:

By default tag all is used so every roles are played on playbooks launch.Multiple tags can be passed as follows : ./run.sh -p ./infra/ansible/install.yml -t gateway,k3s

First gateway init can take a long time to run because of openvpn key genereration (5-10min).

Destroy

It is possible to cleanly detroy the k3s cluster by running :

# Destroy cluster
./run.sh -p ./infra/ansible/install.yml -t k3s-destroy

Kubernetes services

Kubernetes services are deployed within 2 steps, the first one deploy core services that are needed to deploy one or more platforms, core services are composed of :

• Longhorn - storage management in the cluster.
• Traefik - ingress controler to expose services.
• Cert Manager - certificate management for tls.
• Vault - secret management for services deployments.
• Argocd - deployment management for services deployments.

Other services follow the gitops workflow, they are configured through files stored in a Git repository that is watched by Argocd. An applicationSet is responsible to deploy an app of apps for each environement (or platform) wanted to be spin up, then the app of apps will deploy all others services with their dependencies by reading secrets into Hashicorp Vault.

The next step would be to deploy each platform environment to a dedicated cluster as described in the following schema.

Known issues

At the moment, mattermost and outline images are not arm64 compatible so their deployment are using custom mirror image with compatibility (see. this repo and and associated Argocd applications).

The official Harbor helm chart cannot be used due to arm64 incompatibility, the Bitnami distribution is used instead.