Home-Lab Refresh: DNS

January 2022

Following setting up the KVM Hypervisors for my Homelab Refresh, the next step was to add DNS. I settled on using PowerDNS for several reasons.

  1. It has an external API.
  2. Kubernetes can use it as external DNS.
  3. I am already familiar with it through work.

To make PowerDNS work effectively, A MariaDB Galera cluster will be set-up to act as the database backend to PowerDNS. No GUI is required for PowerDNS so this will be ignored.

While this could be setup inside a Kubernetes cluster, I am electing to keep these separate and outside of Kubernetes. This maintains a healthy separation of failure domains between DNS and Kubernetes, meaning that a failure of one will not impact the other. This is ideal in the event that Kubernetes experiences issues, as troubleshooting will not then complicated by DNS potentially being down. In the event that the DNS servers are experiencing issues, there is no need to touch the Kubernetes side.

Setting this up has a few steps:

  1. Terraform
  2. Ansible - Node 1
  3. Manually Bootstrap Galera
  4. Ansible - Remaining Nodes
  5. PowerDNS

Terraform

The first step for deploying the DNS servers is to provision them using the Terraform I previously made. This is fairly straight forward and is done using a new directory so that the Terraform state is separated. I’ve published the Terraform configuration that I’ve used for deploying the DNS servers.

Variables

terraform/infrastructure/core_services/terraform.tfvars

hypervisor_hosts = {
  "kvm1" = {
    "ip"   = "10.1.1.21",
    "user" = "root",
  },
  "kvm2" = {
    "ip"   = "10.1.1.22",
    "user" = "root",
  },
  "kvm3" = {
    "ip"   = "10.1.1.23",
    "user" = "root",
  },
}

virtual_machines = {
  "dns1" = {
    "ip"   = "10.1.1.31",
    "os"   = "debian_10"
  },
  "dns2" = {
    "ip"   = "10.1.1.32",
    "os"   = "debian_10"
  },
  "dns3" = {
    "ip"   = "10.1.1.33",
    "os"   = "debian_10"
  },
}

domain = "lab.alexgardner.id.au"

host_admin_users = {
  "adminuser" = "ssh-rsa AAAAB[...truncated...]NZe19",
}

network_gateway_ip     = "10.1.1.1"
network_nameserver_ips = "10.1.1.31, 10.1.1.32, 10.1.1.33"

Commands

cd terraform/infrastructure/core_services
terraform1.1 apply

Ansible - Node 1

The next step is to run Ansible on the node selected to bootstrap the Galera cluster. I’ve published the Ansible configuration that I used to do this. This initial run also creates the PowerDNS database in MariaDB and imports the database schema.

Variables

ansible/group_vars/all.yml

---
domain: lab.alexgardner.id.au
email: alex+homelab@alexgardner.id.au

nameservers:
  - '10.1.1.1'
network_subnets:
  - '10.1.1.0/24'
  - '10.1.2.0/24'
  - '10.1.3.0/24'

firewall_servers_subnet: 10.1.1.0/24
firewall_wireless_subnet: 10.1.2.0/24
firewall_clients_subnet: 10.1.3.0/24

timezone: Australia/Sydney

admin_users:
  - adminuser

ansible/group_vars/dns_servers.yml

---
debian_version: buster
mariadb_version: mariadb-10.5
mariadb_root_password: MySuperSecretPassword
#checkov:skip=CKV_SECRET_6:Unencrypted secrets are git-ignored

mariadb_galera_auth_user: mariabackup
mariadb_galera_auth_password: MySuperSecretPassword
#checkov:skip=CKV_SECRET_6:Unencrypted secrets are git-ignored
mariadb_galera_bootstrap_host: dns1
mariadb_galera_hosts_list:
  - '10.1.1.31'
  - '10.1.1.32'
  - '10.1.1.33'

powerdns_version: '45'
powerdns_mysql_password: MySuperSecretPassword
#checkov:skip=CKV_SECRET_6:Unencrypted secrets are git-ignored
powerdns_forward_recursors: 10.1.1.1;1.0.0.1;1.1.1.1
powerdns_foward_zones:
  - 'lab.alexgardner.id.au'
  - '1.1.10.in-addr.arpa'

Commands

cd ansible
ansible-playbook -i production dns-servers.yml -l dns1

Manually Bootstrap Galera

Once Ansible has been run on the bootstrapping node, we then SSH into it and manually bootstrap the Galera cluster. This is manual because it reduces the complexity in having Ansible try to work out when the galera_new_cluster command should be run. This could be achieved with Ansible, however, this is much easier and should only need to be run once per cluster. Perhaps if I have time I’ll update Ansible to do it all.

adminuser@dns1:~$ sudo systemctl stop mariadb
adminuser@dns1:~$ sudo galera_new_cluster
adminuser@dns1:~$ sudo mysql -e "SHOW GLOBAL STATUS LIKE 'wsrep_cluster_s%';"
+--------------------------+--------------------------------------+
| Variable_name            | Value                                |
+--------------------------+--------------------------------------+
| wsrep_cluster_size       | 1                                    |
| wsrep_cluster_state_uuid | 4cc81535-738f-11ec-89ae-d2c65843490d |
| wsrep_cluster_status     | Primary                              |
+--------------------------+--------------------------------------+

Ansible - Remaining nodes

Once Galera has been bootstrapped, Ansible can be run on the remaining hosts. Since the remaining hosts are not the bootstrap host, Ansible will restart MariaDB on them, which will bring them into the Galera cluster.

Commands

cd ansible
ansible-playbook -i production dns-servers.yml

Once Ansible is completed, confirm that the Galera cluster is healthy.

adminuser@dns1:~$ sudo mysql -e "SHOW GLOBAL STATUS LIKE 'wsrep_cluster_s%';"
+--------------------------+--------------------------------------+
| Variable_name            | Value                                |
+--------------------------+--------------------------------------+
| wsrep_cluster_size       | 3                                    |
| wsrep_cluster_state_uuid | 36acd23e-7393-11ec-8a4e-b60b353a9920 |
| wsrep_cluster_status     | Primary                              |
+--------------------------+--------------------------------------+

PowerDNS

You should now be able to add DNS records to PowerDNS.

adminuser@dns1:~$ sudo pdnsutil list-all-zones
adminuser@dns1:~$ sudo pdnsutil create-zone lab.alexgardner.id.au
Creating empty zone 'lab.alexgardner.id.au'
adminuser@dns1:~$ sudo pdnsutil list-all-zones
lab.alexgardner.id.au
adminuser@dns1:~$ sudo pdnsutil add-record lab.alexgardner.id.au dns1 a 900 10.1.1.31
New rrset:
test.example.test. 900 IN A 10.1.1.31
adminuser@dns1:~$ dig dns1.lab.alexgardner.id.au @127.0.0.1

; <<>> DiG 9.11.5-P4-5.1+deb10u6-Debian <<>> dns1.lab.alexgardner.id.au @127.0.0.1
;; global options: +cmd
;; Got answer:
;; WARNING: .local is reserved for Multicast DNS
;; You are currently testing what happens when an mDNS query is leaked to DNS
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 18539
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;dns1.lab.alexgardner.id.au. IN  A

;; ANSWER SECTION:
dns1.lab.alexgardner.id.au. 900  IN  A 10.1.1.31

;; Query time: 6 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Thu Jan 13 07:45:27 AEDT 2022
;; MSG SIZE  rcvd: 70

Next up: Kubernetes Installation

Back to article list