Operating loop results with dict or array

The scenario is that we have two different loops, and we want to operate their result together with a with_nested. So, here we have two loops:

- name: Get home files with names from 1 to 4 
  shell: "ls /home/ansible/{{ item }}"
  ignore_errors: yes
  register: files_1_4
  with_items:
    - 1
    - 2
    - 3
    - 4 #doesn't exist to do the example more interesting

- name: Get home files with names from 5 to 8
  shell: "ls /home/ansible/{{ item }}"
  ignore_errors: yes
  register: files_5_8
  with_items:
    - 5
    - 6
    - 7
    - 8

If you want to operate this together you will need to create a dict (or array) in order to be able to operate them.

So, what we should do is use set_fact to create this dict:

- name: Creating the dict with files that exists
  set_fact:
    existing_files_1_4: "{{ existing_files_1_4|default([]) + [ {'name': item.item } ] }}"
  when: ( item.rc == 0 ) # Just as example
  with_items:
    - "{{ files_1_4.results }}"

- name: Creating the dict with files that exists
  set_fact: 
    existing_files_5_8: "{{ existing_files_5_8|default([]) + [ {'name': item.item } ] }}"
  when: ( item.rc == 0 ) # Just as example
  with_items:
    - "{{ files_5_8.results }}"

And now we’re able to use with_nested with both items. Take into account that we were saving item.item, but you could save item.stdout_lines for example.

- name: Echo result
  shell: "echo {{ item.0.name }} {{ item.1.name }}"
  with_nested:
    - "{{ existing_files_1_4 }}"
    - "{{ existing_files_5_8 }}"
  ignore_errors: yes

And if you check the result you will see something like:

TASK [Echo result] *************************************************************
changed: [server] => (item=[{'name': 1}, {'name': 5}])
changed: [server] => (item=[{'name': 1}, {'name': 6}])
changed: [server] => (item=[{'name': 1}, {'name': 7}])
changed: [server] => (item=[{'name': 2}, {'name': 5}])
changed: [server] => (item=[{'name': 2}, {'name': 6}])
changed: [server] => (item=[{'name': 2}, {'name': 7}])
changed: [server] => (item=[{'name': 3}, {'name': 5}])
changed: [server] => (item=[{'name': 3}, {'name': 6}])
changed: [server] => (item=[{'name': 3}, {'name': 7}])

If instead of using a dict, you want to use an array, you should set_fact as follows

- set_fact:
    existing_files_1_4: "{{ existing_files_1_4|default({}) | combine ( { item.item: item.stdout } ) }}"
  when: (item.rc == 0)
  with_items:
    - "{{ files_1_4.results }}"
Advertisements

Debug (properly) data registered with a loop

If you take a look at official documentation of Ansible to learn how to use register with a loop, you will see that the data that is registered is not really usable when debugging.

Example:

- shell: echo "{{ item }}"
  with_items:
    - one
    - two
  register: echo

If you want to show only the ‘stdout_lines’ of this task you will need to do:

- debug:
  msg: "{{ echo.results|map(attribute='stdout_lines')|list }}"

The output won’t be perfect, but for sure will be more usable:

ok: [server] => {
 "msg": [
     [
         "one"
     ],
     [
         "two"
     ]
   ]
}

Using tags for each role

I normally put a tags for each role, so it makes me easier to choose what I need to do for my hosts with only one tag. Then if I need to specify something inside the role, I can specify a second tag and apply both tags easily.

So for example I have here frontend.yml playbook with two roles. Each role deploys one application and each of them has its own tag name:

- name: "Deploy of frontend applications: frontend1, frontend2"
  hosts: frontend
  roles:
    - { role: frontend1,
        tags: frontend1 }
    - { role: frontend2, 
        tags: frontend2 }

So then I can easily choose what I want to do if I run:

ansible-playbook -i inventory/prod frontend.yml -t frontend1
ansible-playbook -i inventory/prod frontend.yml -t frontend2

Then, imagine that inside each playbook, as part of the deployment process, I stop and start my app. Then I can add a tag called restart for that specific tasks. For example inside roles/frontend1/tasks/main.yml:

[...]
- name: Stop {{ service_name }}
  shell: /etc/init.d/{{ service_name }} stop
  tags:
    - restart

- name: Start {{ service_name }}
  shell: /etc/init.d/{{ service_name }} start"
  tags:
    - restart
[...]

Then, you could think that in order to restart the app you could run:

ansible-playbook -i inventory/prod frontend.yml -t frontend1,restart

Sadly you can’t do that because it will restart also frontend2 app. In order to fix that you should add –skip-tags:

ansible-playbook -i inventory/prod frontend.yml -t frontend1,restart --skip-tags frontend2

I know that it doesn’t look really nice, but that’s the best way I’ve found to do it right now. Also I recommend to use an Operation Tool like Rundeck to perform these kind of tasks.

How to register the result of a mysql query with Ansible

When you run a query on mysql you normally have a output like:

+-----------+
| USERNAME  |
+-----------+
| rcastells |
+-----------+
1 row in set (0.00 sec)

If you need to register only ‘rcastells’, how can you register the result of query above with ansible? You need to add following args to your mysql query:

  • -N, –skip-column-names: Don’t write column names in results.
  • -s, –silent: Be more silent. Print results with a tab as separator, each row on new line.

So your task will look like:

- name:  Get username
  command: mysql -u {{ username }} -p {{ mysql_root_password }} {{ database }} -Ns -e "<your_query>"
  register: username

- debug: msg="{{ username.stdout_lines }}"

 

Jinja2: How to gather_facts from other hosts

If you have this content in your inventory file:

[loadbalancers]
ha01 ansible_host=ec2-52-50-164-246.eu-west-1.compute.amazonaws.com
ha02 ansible_host=ec2-52-48-101-228.eu-west-1.compute.amazonaws.com

[webservers]
web01 ansible_host=ec2-52-50-143-215.eu-west-1.compute.amazonaws.com
web02 ansible_host=ec2-52-31-14-253.eu-west-1.compute.amazonaws.com

And you want to create a template inside your loadbalancers with information from webservers, like for example following haproxy configuration:

{% for host in groups['webservers'] %}
server {{ hostvars[host]['ansible_hostname'] }} {{ hostvars[host]['ansible_default_ipv4']['address'] }}:80
{% endfor %}

You first need to connect to webservers. No need to do anything. So, your playbook will look like:

---
# No need to run a task for webservers. This will gather their facts.
- hosts: webservers
  become: yes

- hosts: loadbalancers
  become: yes
  roles:
- haproxy-ha

This will connect first to your webservers and gather their facts that will be used in haproxy.cfg.j2 template file.

Using vars_prompt to create menu with options

Normally I prefer to reduce the probability that a user does mistakes. So sometimes is better if you ask through a menu what do you want to do when running a playbook. For example, if you’re using the same playbook to perform deployments of different applications is a good idea to create a menu with options. So you can reuse the same playbook for different actions. If you have following menu created by using vars_prompt:

vars_prompt:
 - name: "application"
   prompt: "Which application do you want to deploy?\n1- Web\n2- Wiki\n"
   private: no

You will have following prompt when running the playbook:

Which application do you want to deploy?
1- Web
2- Wiki

Then inside your playbook you can work with this condition.

For example you can define variables using set_fact:

 - set_fact: app=web
 when: ( application == "1" )
 
 - set_fact: app=wiki
 when: ( application == "2" )

Or perform actions taking into account which option you have chosen:

 - name: Build the war for website
   shell: /opt/builds/buildwebwar.sh
   when: ( application == "1" ) 


- name: Build the war for wiki
   shell: /opt/builds/buildwikiwar.sh
   when: ( application == "2" )

In that way you can be sure that user can do only one mistake: press the wrong number 🙂

Using variables in hosts declaration inside playbooks

When you declare in which hosts do you want to run a playbook you usually do it like this:

- hosts: databases

But let’s say that you want to be more flexible and you want to have some variable inside your hosts declaration. How can you do that? One possible solution is using extra vars.

If we decide to call our variable ‘env’ we should define our hosts inside the playbook like:

- hosts: "{{ env }}_master"

And then run the playbook as follows:

ansible-playbook -i inventory/databases binlogs.yml --extra-vars "env=database01"

Obviously inside your inventory file, these hosts must exist:

database01_master ansible_host=db01
database02_master ansible_host=db03
database03_master ansible_host=db05

 

How to use more than one inventory file in a playbook in one single command

If you want to run a playbook in more than one inventory file in one command you just need to put every inventory file you want inside a directory and then run:

ansible-playbook -i <inventory_file_directory> <playbook>

For example:

ansible-playbook -i inventory/rackspace_prod/ update_config.yml

So you can easily run a playbook and work with all servers that you need no matter in which environment file they are.

Let’s go a little deeper in the example above because you could think: why not create a bigger inventory file with everything inside it?

Imagine that you have one static inventory file (static) and a dynamic inventory file. For example Rackspace dynamic inventory file (rax.py). In both environments you have a group name called webservers because you use Rackspace Cloud Servers to scale up and down your static webservers.

If you want to operate all your webservers (static and dynamic servers) you could run the playbook twice:

ansible webservers -i inventory/static -m ping
ansible webservers -i inventory/rax.py -m ping

But you can create a directory called rackspace_prod and put there both inventory files and then run:

ansible webservers -i inventory/rackspace_prod/ -m ping

I normally use this feature to update configuration files like in apaches, loadbalancers or /etc/hosts file.

For example you can update haproxy configuration with your static and dynamic webservers by using a template file:

[...]
backend my_backend
option httpchk
cookie JSESSIONID prefix nocache
 balance roundrobin
{% for host in groups['webservers'] %}
 server {{ hostvars[host]['ansible_hostname'] }} {{ hostvars[host]['ansible_eth1']['ipv4']['address'] }}:80 check  {{ hostvars[host]['ansible_hostname'] }}
 {% endfor %}
[...]

How to forward your SSH key to hosts in Ansible

I prefer to work with SSH keys in order to connect to all my hosts. It makes the daily work easier. Now imagine that you need to use your SSH key in a host. For example I needed that in order to run a checkout of a SVN repo. You can’t specify SSH key with svn module.

Just edit your /etc/ansible/ansible.cfg file and search for [ssh_connection]. There you can edit all the options you want for your SSH connection. A really useful option. So just add the option ForwardAgent=yes to ssh_args.

[ssh_connection]
ssh_args = -o ForwardAgent=yes

Now in every SSH connection Ansible will forward your key.

How to know the amount of data that has been transferred using copy module

Ansible doesn’t have a progress bar and sometimes when you use copy module in a poor network or transferring a big file, we could be running the playbook and got stuck here:

TASK [Copy war into directory /deploy] ***************

And we don’t know what’s going on, if it’s working, if it’s going fast, slow,…

How can we know the status of our copy?

You should connect into the server where you’re copying your file and go into the temporary directory that has been created in the user home that is being used by Ansible to connect. So, if your playbook looks like:

- hosts: appserver
  user: rcastells
  become: yes

You should connect into appserver and go into temporary directory created by ansible:

/home/rcastells/.ansible/tmp/ansible-tmp-1462436406.52-187369121476281

Take in consideration that inside tmp you will find a lot of different temporary directories created by Ansible for every playbook execution. You’re only interested in the latest one.

Inside this temporary directory you can find a file called ‘source’ that will be increasing its size. This temporary file is the one that is using copy module.

56M -rw-rw-r-- 1 rcastells rcastells 56M May 5 16:34 source

Is not a really nice solution, but at least you can know what’s going on and how fast is your copy going.