1 - Terraform K3s
Provisions K3s nodes and is able to build a cluster from multiple nodes.
You can use the k3s module to template the necessary cloudinit files for creating a K3s cluster node. Modules for OpenStack and Hetzner hcloud that bundle all necessary resources are available.
Supported Cloud Providers
- OpenStack
- Hetzner Cloud (hcloud)
Modules
k3s
This module provides the templating of the user_data for use with cloud-init.
module "k3s_server" {
source = "git::https://github.com/nimbolus/tf-k3s.git//k3s"
name = "k3s-server"
cluster_token = "abcdef"
k3s_ip = "10.11.12.13"
install_k3s_exec = "server --disable traefik --node-label az=ex1"
}
output "server_user_data" {
value = module.k3s_server.user_data
sensitive = true
}
k3s-openstack
With this module a single K3s node can be deployed with OpenStack. It internally uses the k3s module. Depending on the supplied parameters the node will initialize a new cluster or join an existing cluster as a server or agent.
module "server" {
source = "git::https://github.com/nimbolus/tf-k3s.git//k3s-openstack"
name = "k3s-server"
image_name = "ubuntu-20.04"
flavor_name = "m1.small"
availability_zone = "ex"
keypair_name = "keypair"
network_id = var.network_id
subnet_id = var.subnet_id
security_group_ids = [module.secgroup.id]
cluster_token = "abcdef"
install_k3s_exec = "server --disable traefik --node-label az=ex" // if using bootstrap-auth include "--kube-apiserver-arg=\"enable-bootstrap-token-auth\""
bootstrap_token_id = "012345"
bootstrap_token_secret = "0123456789abcdef"
}
k3s-openstack/security-group
The necessary security-group for the K3s cluster can be deployed with this module.
module "secgroup" {
source = "git::https://github.com/nimbolus/tf-k3s.git//k3s-openstack/security-group"
}
k3s-hcloud
With this module a single K3s node can be deployed with hcloud. It internally uses the k3s module. Depending on the supplied parameters the node will initialize a new cluster or join an existing cluster as a server or agent.
module "server" {
source = "git::https://github.com/nimbolus/tf-k3s.git//k3s-hcloud"
name = "k3s-server"
keypair_name = "keypair"
network_id = var.network_id
network_range = var.ip_range
cluster_token = "abcdef"
install_k3s_exec = "server --disable traefik --node-label az=ex" // if using bootstrap-auth include "--kube-apiserver-arg=\"enable-bootstrap-token-auth\"""
bootstrap_token_id = "012345"
bootstrap_token_secret = "0123456789abcdef"
}
bootstrap-auth
To access the cluster an optional bootstrap token can be installed on the cluster. To install the token specify the parameters bootstrap_token_id and bootstrap_token_secret on the server that initializes the cluster.
For ease of use this module can be used to retrieve the CA certificate from the cluster. The module also outputs a kubeconfig with the bootstrap token.
Please keep in mind that the connection to retrieve the CA certificate cannot be secure as the certificate cannot be verified. Additionally this module makes use of the scottwinkler/shell provider. Please make sure you only supply trusted values to the module.
module "bootstrap_auth" {
source = "git::https://github.com/nimbolus/tf-k3s.git//bootstrap-auth"
// depends_on = [module.secgroup] // if using OpenStack
k3s_url = module.server1.k3s_external_url
token = local.token
}
Examples
- basic: basic usage of the k3s module with one server and one agent node
- ha-hcloud: 3 Servers and 1 Agent with bootstrap token on Hetzner Cloud
- ha-openstack: 3 Servers and 1 Agent with bootstrap token on OpenStack
Requirements
MacOS users need to install coreutils for the timeout command used by the bootstrap-auth module:
brew install coreutils
Tests
Basic
cd tests/basic
go test -count=1 -v
OpenStack
cd tests/ha-openstack
cp env.sample .env
$EDITOR .env
go test -count=1 -v
hcloud
cd tests/ha-hcloud
cp env.sample .env
$EDITOR .env
go test -count=1 -v
2 - Terraform Vault
These Terraform modules can provision a highly available Vault cluster with the Consul storage backend.
You can use the vault-consul module to deploy the Vault cluster using a pre-existing Consul installation. The vault-transit module can be used to setup the transit engine used for auto-unsealing another Vault cluster (they rely on each other for auto-unsealing).
Usage
- Deploy Vault 1 (e.g. with vault-consul) - do NOT enable auto-unseal yet.
- Repeat for Vault 2.
- Deploy vault-transit for Vault 1 and Vault 2.
- Enable auto-unseal for Vault 1 and init seal migration. You need to restart the Vault server pods.
- Repeat for Vault 2.
Examples
HA Vault with Consul storage
module "vault1" {
source = "github.com/nimbolus/tf-vault/vault-consul"
release_name = "vault1"
namespace = kubernetes_namespace.vault.metadata[0].name
vault_domain = yamldecode(kubectl_manifest.vault1_certificate.yaml_body_parsed)["spec"]["dnsNames"][0]
consul_address = "consul.example.com"
auto_unseal_enable = true
auto_unseal_vault_service_address = "https://vault2.example.com"
auto_unseal_vault_mount_path = "vault-transit"
auto_unseal_vault_role = "vault-unseal"
tls_secret_name = yamldecode(kubectl_manifest.vault1_certificate.yaml_body_parsed)["spec"]["secretName"]
ingress_ssl_passthrough_enable = true
}
Transit Secret Engine for Auto Unseal
module "vault_transit1" {
depends_on = [module.vault1]
providers = {
vault = vault.vault1
}
source = "github.com/nimbolus/tf-vault/vault-transit"
name = "vault2"
namespace = "vault2"
}