Category Archives: System Administration

Ansible Configuration Management

What is it?


Ansible is a Configuration Management tool designed to automate the deployment of defined configurations from a single host to many hosts. It allows one to define hosts into specific groups, and then run a specific set of tasks against those hosts, allowing one to set up a web server or a mail server quickly and perfectly configured every time. These qualities are essential for productivity in a large server environment.

Ansible is similar to Puppet and Chef in terms of configuration management, but Ansible does not require a client to be installed on the target server in order to perform its job. Instead, Ansible accesses each server through SSH and then performs all its commands through the command line — just like a regular user would — in order to leave as little mark on the system as possible; in fact, as Ansible uses an unobtrusive method of control, it can be used alongside Puppet and Chef.

Installation


The first step is to get Ansible installed on the ‘master’ system (the system you plan to control all of your defined servers from). In this case, all of our servers will be CentOS 6.7 Minimal based. As root:

yum install epel-release
yum install ansible

Ansible is available from the ‘Extra Packages for Enterprise Linux’ repository (commonly known as epel), and so this needs to be installed first. Ansible is now installed on the system and is ready to be configured.

Configuration


SSH keys are necessary to allow access from the ‘master’ server (the server one wishes to control the other machines from) to each ‘client’ machine without requiring a password. Passwords can be defined in Ansible, but keys make life a lot easier.

ssh-keygen

And then a quick loop to copy the new SSH public key on to the target machines

[root@server1 ~]# for i in {2..4}; do
> ssh-copy-id server$i
> done
The authenticity of host 'server2 (10.44.16.152)' can't be established.
RSA key fingerprint is 12:7e:90:4b:af:11:bb:56:40:84:4e:84:e6:78:b8:d3.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'server2,10.44.16.152' (RSA) to the list of known hosts.
root@server2's password: 
Now try logging into the machine, with "ssh 'server2'", and check in:

  .ssh/authorized_keys

to make sure we haven't added extra keys that you weren't expecting.

The authenticity of host 'server3 (10.44.16.153)' can't be established.
RSA key fingerprint is 12:7e:90:4b:af:11:bb:56:40:84:4e:84:e6:78:b8:d3.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'server3,10.44.16.153' (RSA) to the list of known hosts.
root@server3's password: 
Now try logging into the machine, with "ssh 'server3'", and check in:

  .ssh/authorized_keys

to make sure we haven't added extra keys that you weren't expecting.

The authenticity of host 'server4 (10.44.16.154)' can't be established.
RSA key fingerprint is 12:7e:90:4b:af:11:bb:56:40:84:4e:84:e6:78:b8:d3.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'server4,10.44.16.154' (RSA) to the list of known hosts.
root@server4's password: 
Now try logging into the machine, with "ssh 'server4'", and check in:

  .ssh/authorized_keys

to make sure we haven't added extra keys that you weren't expecting.

All of the required configuration files are available from /etc/ansible. The one that is the most important at this point is the hosts file; this allows hosts to be defined in groups in order to run particular sets of Ansible playbooks against them.

[root@server1 ~]# cat /etc/ansible/hosts
[group1]
server3

[group2]
server2
server4

[servers:children]
group1
group2

[servers:vars]
ansible_ssh_user=root

The group names are arbitrary and can be anything that the user chooses, such as ‘webservers’, ‘mailservers’, ‘dbservers’, etc.

CentOS uses SELinux, and this tends to get in the way when changes are made to the system. To get around this, a necessary package is installed on the clients that allows such changes to be made

ansible servers -m yum -a "name=libselinux-python state=latest"

Ansible should now be ready for full usage.

Using Ansible


Now everything is in place to start doing interesting things with Ansible. The following is an example of what can be done

[root@server1 ~]# ansible group2 -m command -a "df -h"
server2 | success | rc=0 >>
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_server-lv_root
                      8.3G  628M  7.3G   8% /
tmpfs                 499M     0  499M   0% /dev/shm
/dev/sda1             477M   30M  422M   7% /boot

server4 | success | rc=0 >>
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_server-lv_root
                      8.3G  627M  7.3G   8% /
tmpfs                 499M     0  499M   0% /dev/shm
/dev/sda1             477M   30M  422M   7% /boot

[root@server1 ~]# ansible servers -m service -a "name=ntpd state=restarted"
server3 | success >> {
    "changed": true, 
    "name": "ntpd", 
    "state": "started"
}

server4 | success >> {
    "changed": true, 
    "name": "ntpd", 
    "state": "started"
}

server2 | success >> {
    "changed": true, 
    "name": "ntpd", 
    "state": "started"
}

Referring back to the /etc/ansible/hosts file one can see that Ansible is executing the commands only on the servers specified within particular groups. You may have noticed a parent group ‘servers’ which all available groups have been specified as children of. This allows much tighter control over which commands are run on which servers.

Ansible Playbooks

The above doesn’t save much time if we have 40 packages to install and a bunch of configuration files to copy across. Sure, it saves us ‘time of configuration’ x ‘amount of servers to configure’, but what if new servers are constantly added which require identical configurations? We’d be back to square one of having to type things out command by command for every server. This is where the power of playbooks comes in.

To demonstrate this, let’s start with a simple playbook that creates a MOTD on our servers. In this example, I’ve created a group in /etc/ansible/hosts called maintenance that contain all the servers that will suffer downtime due to planned maintenance.

[root@server1 playbooks]# cat /etc/ansible/hosts | grep -A4 maintenance
[maintenance]
server2
server3

[servers:children]
[root@server1 playbooks]# cat test_playbook.yml 
---
- hosts: maintenance
  user: root
  vars:
    motd_warning: '\nWARNING! The system will be shut down for maintenance at 17:00 on 16/11/15\n\n'
  tasks:
    - name: setup a MOTD
      copy: dest=/etc/motd content="{{ motd_warning }}"

[root@server1 playbooks]# ansible-playbook test_playbook.yml 

PLAY [maintenance] ************************************************************ 

GATHERING FACTS *************************************************************** 
ok: [server3]
ok: [server2]

TASK: [setup a MOTD] ********************************************************** 
changed: [server3]
changed: [server2]

PLAY RECAP ******************************************************************** 
server2                    : ok=2    changed=1    unreachable=0    failed=0   
server3                    : ok=2    changed=1    unreachable=0    failed=0   

[root@server1 playbooks]# ssh server3
Last login: Thu Nov 12 01:40:01 2015 from 10.44.16.151

WARNING! The system will be shut down for maintenance at 17:00 on 16/11/15

[root@server3 ~]#

If more servers require maintenance, just add them to the group and re-run the playbook.

[root@server1 playbooks]# cat /etc/ansible/hosts | grep -A4 maintenance
[maintenance]
server2
server3
server4

[root@server1 playbooks]# ansible-playbook test_playbook.yml 

PLAY [maintenance] ************************************************************ 

GATHERING FACTS *************************************************************** 
ok: [server3]
ok: [server2]
ok: [server4]

TASK: [setup a MOTD] ********************************************************** 
ok: [server3]
ok: [server2]
changed: [server4]

PLAY RECAP ******************************************************************** 
server2                    : ok=2    changed=0    unreachable=0    failed=0   
server3                    : ok=2    changed=0    unreachable=0    failed=0   
server4                    : ok=2    changed=1    unreachable=0    failed=0

More Complex Ansible Playbooks

Now that the essence of Ansible playbooks has been established, we’ll design a playbook to take care of all of the common tasks that are necessary when a new server is set up.

In this example, there are a series of templates for Ansible to copy to the target systems in order to configure things the way we want them. Here is one such template

[root@server1 playbooks]# cat ssh_banner.j2 

****************************************************************************

 WARNING! This is a private server. The use of this system is restricted to
 authorized users only. Unauthorized access is forbidden. All information
 and communications on this system are monitored.

****************************************************************************

Now for the playbook

[root@server1 playbooks]# cat common_setup.yml 
---
- hosts: servers
  user: root
  tasks:
  - name: Install latest version of libselinux-python
    yum: name=libselinux-python state=latest

  - name: Update server to latest package versions
    yum: name=* state=latest

  - name: Add hosts file
    template: src=hosts.j2 dest=/etc/hosts mode=644 owner=root group=root

  - name: Add SSHD config file
    template: src=sshd_config.j2 dest=/etc/ssh/sshd_config mode=644 owner=root group=root

  - name: Add SSH banner
    template: src=ssh_banner.j2 dest=/etc/ssh/banner.txt mode=644 owner=root group=root

  - name: Restart SSH Service
    service: name=sshd state=restarted

  - name: Add motd.sh file
    template: src=motd.sh.j2 dest=/etc/motd.sh mode=755 owner=root group=root

  - name: Update /etc/profile
    template: src=profile.j2 dest=/etc/profile mode=644 owner=root group=root

[root@server1 playbooks]# ansible-playbook common_setup.yml 

PLAY [servers] **************************************************************** 

GATHERING FACTS *************************************************************** 
ok: [server3]
ok: [server4]
ok: [server2]

TASK: [Install latest version of libselinux-python] *************************** 
ok: [server3]
ok: [server4]
ok: [server2]

TASK: [Update server to latest package versions] ****************************** 
ok: [server4]
ok: [server2]
ok: [server3]

TASK: [Add hosts file] ******************************************************** 
changed: [server2]
changed: [server3]
changed: [server4]

TASK: [Add SSHD config file] ************************************************** 
changed: [server4]
changed: [server2]
changed: [server3]

TASK: [Add SSH banner] ******************************************************** 
changed: [server2]
changed: [server4]
changed: [server3]

TASK: [Restart SSH Service] *************************************************** 
changed: [server4]
changed: [server3]
changed: [server2]

TASK: [Add motd.sh file] ****************************************************** 
changed: [server3]
changed: [server4]
changed: [server2]

TASK: [Update /etc/profile] *************************************************** 
changed: [server2]
changed: [server3]
changed: [server4]

PLAY RECAP ******************************************************************** 
server2                    : ok=9    changed=6    unreachable=0    failed=0   
server3                    : ok=9    changed=6    unreachable=0    failed=0   
server4                    : ok=9    changed=6    unreachable=0    failed=0

And finally just check that the new configurations have done what you expected them to do. Once you know that a playbook does everything you expect it to you should be able to trust it.

[root@server1 playbooks]# ssh server2

****************************************************************************

 WARNING! This is a private server. The use of this system is restricted to
 authorized users only. Unauthorized access is forbidden. All information
 and communications on this system are monitored.

****************************************************************************

Last login: Thu Nov 12 12:13:17 2015 from 10.44.16.151

WARNING! The system will be shut down for maintenance at 17:00 on 16/11/15

      Host: server2.hostname.com
    Uptime: 12:25:19 up 4:29, 1 user, load average: 0.08, 0.02, 0.01
    Memory: 105MB used / 890MB free
      Disk: 791M used / 7.1G free

[root@server2 ~]# logout
Connection to server2 closed.
[root@server1 playbooks]# ssh server3

****************************************************************************

 WARNING! This is a private server. The use of this system is restricted to
 authorized users only. Unauthorized access is forbidden. All information
 and communications on this system are monitored.

****************************************************************************

Last login: Thu Nov 12 03:41:27 2015 from 10.44.16.151

WARNING! The system will be shut down for maintenance at 17:00 on 16/11/15

      Host: server3.hostname.com
    Uptime: 03:53:35 up 4:30, 2 users, load average: 0.00, 0.00, 0.00
    Memory: 105MB used / 890MB free
      Disk: 887M used / 7.0G free