Managing FreeBSD Jails with Ansible - part 1

• Init Ansible
  • Create project and configure
  • Make your project configurable
  • Prepare the jails host
• Shared IP Jail
  • Configure IP aliases
  • Provision jails environments
  • Declare your jails
  • Run and test

When it comes to manage an OS configuration, I always fully automate the process, it is a oneshot work, you write it, test it, forget it, until you need to modify the process. I kind of use Ansible to make the work concrete as I use my blog to clarify my ideas.

Jails are like containers for FreeBSD, it lets you isolate your services from each others. Each Jail has its own IP, there are different ways to manage networking, let’s explore automation for each.

Init Ansible

Create project and configure

Create your Ansible project root and git it.

$ mkdir jails && cd jails
$ git init
$ mkdir roles group_vars

Create inventory, if your host is configured with DHCP, check its IP.

$ echo "host-test ansible_host=192.168.0.44" > hosts

Configure Ansible.

$ cat << EOF > ansible.cfg
[defaults]
stdout_callback = yaml
inventory = hosts
remote_user = root
interpreter_python = auto_silent
EOF

And create the playbook, with a role jails.

$ cat << EOF > playbook.yml
---
- hosts: host-test
  roles:
    - { role: jails, tags: jails }
EOF

$ mkdir -p roles/jails/tasks

Install community.general collection, it contains sysrc ansible module to safely edit rc.conf

$ ansible-galaxy collection install community.general

Make your project configurable

Add a configuration file as group_vars/all.yml, with 3 vars. We will increment the static IP of the host in tasks, so choose an IP which can be incremented as many time as you have jails.

inet: 192.168.0.100
netmask: 255.255.255.0
gateway: 192.168.0.254

Prepare the jails host

Bootstrap FreeBSD for Ansible.

$ ansible host-test -m raw -a "pkg install -y python37"

Configure your default network interface.

$ ansible host-test -m sysrc -a 'name="ifconfig_{{ ansible_default_ipv4.interface }}" value="inet {{ inet }} netmask {{ netmask }}"'
$ ansible host-test -m sysrc -a 'name="defaultrouter" value="{{ gateway }}"'

Restart netif and routing services to read you network configuration for rc.conf.

$ ansible host-test -m raw -a "service netif restart"
$ ansible host-test -m raw -a "service routing restart"

Adapt your inventory with the new static IP and you’re ready.

$ echo "host-test ansible_host=192.168.0.100" > hosts

Shared IP Jail

The simpler way to create a Jail is to add IP alias to you network interface and then bind that IP to your Jail.
For the exemple, let’s create 2 jails, bind and nginx.
First we need to create two IP aliases, default IP is incremented with ipmath which needs netaddr python package installed on the controller.

Let’s declare our jails in a list in group_vars/all.yml.

jails:
  - bind
  - nginx

Configure IP aliases

We use extended loop vars to increment inet IP with ipmath which needs netaddr python package installed on the controller. ansible_loop.index0 starts index at 0 instead of 1.
To be sure that the list will always be processed in the same order, it needs to be explicitly sorted.

In roles/jails/tasks/main.yml.

---
- name: create IP aliases for jails
  vars:
    alias_ip: "{{ inet | ipmath(ansible_loop.index) }}"
  community.general.sysrc:
    name: "ifconfig_{{ ansible_default_ipv4.interface }}_alias{{ ansible_loop.index0 }}"
    value: "inet {{ alias_ip }} netmask {{ netmask }}"
  loop: "{{ jails | sort | flatten(levels=1) }}"
  loop_control:
    extended: yes
  notify: restart netif

You need a handler to trigger the netif service restart on configuration update.

In roles/jails/handlers/main.yml.

---
- name: restart netif
  service:
    name: netif
    state: restarted

Run your playbook, and test.

$ ansible-playbook playbook.yml

PLAY [host-test] ***************************************************************************************************

TASK [Gathering Facts] *********************************************************************************************
ok: [host-test]

TASK [network : create IP aliases for jails] ***********************************************************************
changed: [host-test] => (item=bind)
changed: [host-test] => (item=nginx)

RUNNING HANDLER [network : restart netif] **************************************************************************
changed: [host-test]

PLAY RECAP *********************************************************************************************************
host-test                  : ok=3    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Your host now has two aliases configured.

$ ansible host-test -m shell -a "ifconfig vtnet0 | grep inet"
host-test | CHANGED | rc=0 >>
	inet 192.168.0.100 netmask 0xffffff00 broadcast 192.168.0.255
	inet 192.168.0.101 netmask 0xffffff00 broadcast 192.168.0.255
	inet 192.168.0.102 netmask 0xffffff00 broadcast 192.168.0.255

Provision jails environments

- name: create zfs jails dataset
  community.general.zfs:
    name: zroot/jails
    state: present
    extra_zfs_properties:
      mountpoint: /usr/local/jails

- name: create zfs per jail dataset
  community.general.zfs:
    name: "zroot/jails/{{ item }}"
    state: present
  loop: "{{ jails | sort | flatten(levels=1) }}"

Install FreeBSD on the jails with a custom bsdinstall script.
I faced an issue here, bsdinstall uses its jail argument to target the specific environment, and its script argument to automate the process. Using both make it ignore the second one.

I improved the jail script to be able to use a SCRIPT env var with the script path, to automate in jail provisionning and created a PR.

Now let’s create templates for automated provisionning.
Let’s install python3 and enable sshd to be able to use Ansible on our jails.

- name: template bsdinstall script
  copy:
    dest: "/usr/local/jails/{{ item }}.template"
    content: |
      DISTRIBUTIONS="base.txz"
      export nonInteractive="YES"
      #!/bin/sh
      pkg install -y python37
      sysrc sshd_enable="YES"
      mkdir /root/.ssh
      chmod 600 /root/.ssh
  loop: "{{ jails | sort | flatten(levels=1) }}"

Trigger the provisionning with the shell module and args: creates: to make it idempotent.

- name: bsdinstall jails
  shell: bsdinstall jail /usr/local/jails/"{{ item }}"
  environment:
    SCRIPT: "/usr/local/jails/{{ item }}.template"
  args:
    creates: "/usr/local/jails/{{ item }}/bin"
  loop: "{{ jails | sort | flatten(levels=1) }}"

Authorize your ssh public key in jails.

- name: authorize your ssh key
  copy:
    src: ~/.ssh/id_rsa.pub
    dest: "/usr/local/jails/{{ item }}/root/.ssh/authorized_keys"
    mode: 0600
  loop: "{{ jails | sort | flatten(levels=1) }}"

We finally need to permit root login.

- name: permit root login
  replace:
    path: "/usr/local/jails/{{ item }}/etc/ssh/sshd_config"
    regexp: '^#(PermitRootLogin).*'
    replace: '\1 yes'
  loop: "{{ jails | sort | flatten(levels=1) }}"

Declare your jails

Last thing we need is to declare jails in /etc/jail.conf.

- name: set default jails config
  blockinfile:
    path: /etc/jail.conf
    create: yes
    marker: "# {mark} ANSIBLE MANAGED: default"
    block: |
      exec.start = "/bin/sh /etc/rc";
      exec.stop = "/bin/sh /etc/rc.shutdown";
      exec.clean;
      mount.devfs;

- name: declare jails
  vars:
    alias_ip: "{{ inet | ipmath(ansible_loop.index) }}"
  blockinfile:
    path: /etc/jail.conf
    marker: "# {mark} ANSIBLE MANAGED: {{ item }}"
    block: |
      {{ item }} {
          host.hostname = "{{ item }}.domain.local";
          path = "/usr/local/jails/{{ item }}";
          exec.consolelog = "/var/log/jail_{{ item }}.log";
          ip4.addr = {{ alias_ip }};
      }
  loop: "{{ jails | sort | flatten(levels=1) }}"
  loop_control:
    extended: yes

Let’s tell rc.conf to run jails at startup.

- name: start jails at startup
  community.general.sysrc:
    name: "jail_enable"
    value: "YES"

Finally, add the tasks to start the jails now. We can’t use service module here, because args argument don’t pass its value to service jail $action. The service has a rc of 0 if the service is already running, so it’s not a problem to trigger the start at each playbook run.

- name: start jails
  shell: service jail start "{{ item }}"
  loop: "{{ jails | sort | flatten(levels=1) }}"

Run and test

Run the playbook to provision jails.

$ ansible-playbook playbook.yml
[...]

TASK [jails : create zfs jails dataset] ****************************************************************************
changed: [host-test]

TASK [jails : create zfs per jail dataset] *************************************************************************
changed: [host-test] => (item=bind)
changed: [host-test] => (item=nginx)

TASK [jails : template bsdinstall script] **************************************************************************
changed: [host-test] => (item=bind)
changed: [host-test] => (item=nginx)

TASK [jails : bsdinstall jails] ************************************************************************************
changed: [host-test] => (item=bind)
changed: [host-test] => (item=nginx)

TASK [jails : authorize your ssh key] ********************************************
changed: [host-test] => (item=bind)
changed: [host-test] => (item=nginx)

TASK [jails : permit root login] *************************************************
changed: [host-test] => (item=bind)
changed: [host-test] => (item=nginx)

TASK [jails : set default jails config] ****************************************************************************
changed: [host-test]

TASK [jails : declare jails] ***************************************************************************************
changed: [host-test] => (item=bind)
changed: [host-test] => (item=nginx)

TASK [jails : start jails at startup] ******************************************************************************
changed: [host-test]

TASK [jails : start jails] *****************************************************************************************
changed: [host-test] => (item=bind)
changed: [host-test] => (item=nginx)

PLAY RECAP *********************************************************************************************************
host-test                  : ok=4    changed=10    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

List your running jails and check their IP.

$ ansible host-test -m shell -a "jls"
host-test | CHANGED | rc=0 >>
   JID  IP Address      Hostname                      Path
     1  192.168.0.101   bind.domain.local             /usr/local/jails/bind
     2  192.168.0.102   nginx.domain.local            /usr/local/jails/nginx

$ ansible host-test -m shell -a "jexec bind ifconfig | grep inet"
host-test | CHANGED | rc=0 >>
	inet 192.168.0.101 netmask 0xffffff00 broadcast 192.168.0.255

$ ansible host-test -m shell -a "jexec nginx ifconfig | grep inet"
host-test | CHANGED | rc=0 >>
	inet 192.168.0.102 netmask 0xffffff00 broadcast 192.168.0.255

You can now reach your jails with Ansible, just add their entries in your inventory.

$ cat << EOF >> hosts
bind ansible_host=192.168.0.101
nginx ansible_host=192.168.0.102
EOF

$ ansible nginx:bind -m ping
bind | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/local/bin/python3.7"
    },
    "changed": false,
    "ping": "pong"
}
nginx | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/local/bin/python3.7"
    },
    "changed": false,
    "ping": "pong"
}

In the next part, we will see how to provision vnet jails.