Ansible collect df output and convert to dictionary from set_fact

Question

I need a help with my playbook. Im currently capturing filesystem and utilization related details from Restricted Bash VM (Management Console) and would like to create Splunk Event & upload to HTTP Event Collector using URI

Using set_fact I tried to create 2 variables for filesystem and utilization, Im unable to create it as a dictionar or convert into to single event

Could you please help me with workaround. ` Ansible PLAY:

name: HMC Filesystems 
    raw: monhmc -r disk -n0 | sed -r 's/ ([^ ]*)$/-\1/' | sed "s/.* //g"  | grep -v Mounted | sed -e "s/^/fs_utilization=/g" -e "s/-/  Filesystem=/g" 
    register: result2
    ignore_errors: yes 

  - name: print fs outputs
    debug: 
      var: "{{ result2.stdout_lines | to_json }}" 
 
  - name: HMC Filesystem related variable Assignment 
    set_fact: 
        hmc_filesytem: "{{ item | regex_search('Filesystem.*') | regex_replace('Filesystem=', '')  }}"
        hmc_filesystem_util: "{{ item | regex_search('fs_utilization.*') | regex_replace('fs_utilization=', '') | regex_replace('%.*', '') | regex_replace('%', '')  }}"
        #hmc_fs_dict: "{{ hmc_fs_dict | default([]) + [{ 'hmc_filesystem': item.hmc_filesystem, 'hmc_fs_util': item.hmc_filesystem_util }] }}"
        #hmc_fs_dict: "{{ hmc_fs_dict + hmc_filesystem_util | to_dict }}"
        #dict_var: "{{ dict_var | default({}) | combine({item.hmc_filesystem: item.hmc_fileystem_util}) }}"
        #dict_var: "{{ dict_var | default([]) + [ {'hmc_fs': hmc_filesystem, 'hmc_utils': hmc_filesystem_util } ] }}"
        #hmc_dict: "{{ hmc_dict|default({}) | combine( {item.ansible_facts.hmc_filesystem: item.ansible_facts.hmc_filesystem_util} ) }}"
    with_items: 
      "{{ result2.stdout_lines }}"

  - name: print var stuff
    debug: var=hmc_filesytem
    debug: var=hmc_fs_dict

Output from Filesystems task 
---------------------------------
TASK [print fs outputs] ***********************************************************************
ok: [hmc1lab] => {
    "[\"fs_utilization=0%  Filesystem=/dev\", \"fs_utilization=1%  Filesystem=/dev/shm\", \"fs_utilization=3%  Filesystem=/run\", \"fs_utilization=0%  Filesystem=/sys/fs/cgroup\", \"fs_utilization=44%  Filesystem=/\", \"fs_utilization=5%  Filesystem=/extra\", \"fs_utilization=2%  Filesystem=/data\", \"fs_utilization=4%  Filesystem=/home\", \"fs_utilization=1%  Filesystem=/dump\", \"fs_utilization=13%  Filesystem=/var\", \"fs_utilization=18%  Filesystem=/var/hsc/log\", \"fs_utilization=0%  Filesystem=/run/user/601\", \"fs_utilization=1%  Filesystem=/run/user/604\", \"fs_utilization=0%  Filesystem=/run/user/600\"]": [
        "fs_utilization=0%  Filesystem=/dev",
        "fs_utilization=1%  Filesystem=/dev/shm",
        "fs_utilization=3%  Filesystem=/run",
        "fs_utilization=0%  Filesystem=/sys/fs/cgroup",
        "fs_utilization=44%  Filesystem=/",
        "fs_utilization=5%  Filesystem=/extra",
        "fs_utilization=2%  Filesystem=/data",
        "fs_utilization=4%  Filesystem=/home",
        "fs_utilization=1%  Filesystem=/dump",
        "fs_utilization=13%  Filesystem=/var",
        "fs_utilization=18%  Filesystem=/var/hsc/log",
        "fs_utilization=0%  Filesystem=/run/user/601",
        "fs_utilization=1%  Filesystem=/run/user/604",
        "fs_utilization=0%  Filesystem=/run/user/600"

`

Would like to create a dictionary from df command with filesystem and their utilisation and feed to Splunk HTTP event collector as a single event using URI.

Vladimir Botka · Accepted Answer · 2022-12-16T14:17:46.247

Q: "Create a dictionary from the df command with the filesystem and their utilization."

A: Use the community.general.jc filter. Declare the variable

  df: "{{ df_out.stdout|community.general.jc('df') }}"

and get the data

    - command: df
      register: df_out

^{The implementation of the df utility and the data format differs among the systems. For example, Linux reports Use%, e.g.
shell> df /
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/nvme0n1p6 40054040 29539152 8450504 78% /

As a result, you get
df:
- 1k_blocks: 40054040
available: 8452212
filesystem: /dev/nvme0n1p6
mounted_on: /
use_percent: 78
used: 29537444

but *BSD systems reports Capacity, e.g.
shell> df /
Filesystem 512-blocks Used Avail Capacity Mounted on
zroot/jails/test_11 202537888 1114560 201423328 1% /

As a result, you get
df:
- 512_blocks: 202537888
available: 201423328
capacity_percent: 1
filesystem: zroot/jails/test_11
mounted_on: /
used: 1114560}

Use the filter ansible.utils.replace_keys and rename the capacity_percent key to use_percent

    df_used: "{{ df|ansible.utils.replace_keys(target=_target) }}"
    _target:
      - before: capacity_percent
        after: use_percent

and create the dictionary

    mount_percent: "{{ df_used|items2dict(key_name='mounted_on',
                                          value_name='use_percent') }}"

^{Example of a complete playbook for testing
shell> cat pb.yml
- hosts: test_11,test_13,localhost

vars:

df: "{{ df_out.stdout|community.general.jc('df') }}"
df_used: "{{ df|ansible.utils.replace_keys(target=_target) }}"
_target:
- before: capacity_percent
after: use_percent
mount_percent: "{{ df_used|items2dict(key_name='mounted_on',
value_name='use_percent') }}"

tasks:

- command: df
register: df_out
- debug:
var: df

- debug:
var: df_used

- debug:
var: mount_percent

gives
shell> ansible-playbook pb.yml

PLAY [test_11,test_13,localhost] ************************************************************

TASK [command] ******************************************************************************
changed: [localhost]
changed: [test_11]
changed: [test_13]

TASK [debug] ********************************************************************************
ok: [test_13] =>
df:
- 512_blocks: 201990424
available: 201424472
capacity_percent: 0
filesystem: zroot/jails/test_13
mounted_on: /
used: 565952
ok: [test_11] =>
df:
- 512_blocks: 202538384
available: 201424472
capacity_percent: 1
filesystem: zroot/jails/test_11
mounted_on: /
used: 1113912
ok: [localhost] =>
df:
- 1k_blocks: 3974864
available: 3974864
filesystem: udev
mounted_on: /dev
use_percent: 0
used: 0
- 1k_blocks: 801304
available: 798812
filesystem: tmpfs
mounted_on: /run
use_percent: 1
used: 2492
- 1k_blocks: 40054040
available: 8450272
filesystem: /dev/nvme0n1p6
mounted_on: /
use_percent: 78
used: 29539384
- 1k_blocks: 4006520
available: 3627692
filesystem: tmpfs
mounted_on: /dev/shm
use_percent: 10
used: 378828
- 1k_blocks: 5120
available: 5116
filesystem: tmpfs
mounted_on: /run/lock
use_percent: 1
used: 4
- 1k_blocks: 4006520
available: 4006520
filesystem: tmpfs
mounted_on: /sys/fs/cgroup
use_percent: 0
used: 0
- 1k_blocks: 107323996
available: 15491352
filesystem: /dev/nvme0n1p7
mounted_on: /export
use_percent: 85
used: 86337860
- 1k_blocks: 98304
available: 29569
filesystem: /dev/nvme0n1p2
mounted_on: /boot/efi
use_percent: 70
used: 68735
- 1k_blocks: 801304
available: 801288
filesystem: tmpfs
mounted_on: /run/user/1000
use_percent: 1
used: 16

TASK [debug] ********************************************************************************
ok: [test_13] =>
df_used:
- 512_blocks: 201990424
available: 201424472
filesystem: zroot/jails/test_13
mounted_on: /
use_percent: 0
used: 565952
ok: [test_11] =>
df_used:
- 512_blocks: 202538384
available: 201424472
filesystem: zroot/jails/test_11
mounted_on: /
use_percent: 1
used: 1113912
ok: [localhost] =>
df_used:
- 1k_blocks: 3974864
available: 3974864
filesystem: udev
mounted_on: /dev
use_percent: 0
used: 0
- 1k_blocks: 801304
available: 798812
filesystem: tmpfs
mounted_on: /run
use_percent: 1
used: 2492
- 1k_blocks: 40054040
available: 8450272
filesystem: /dev/nvme0n1p6
mounted_on: /
use_percent: 78
used: 29539384
- 1k_blocks: 4006520
available: 3627692
filesystem: tmpfs
mounted_on: /dev/shm
use_percent: 10
used: 378828
- 1k_blocks: 5120
available: 5116
filesystem: tmpfs
mounted_on: /run/lock
use_percent: 1
used: 4
- 1k_blocks: 4006520
available: 4006520
filesystem: tmpfs
mounted_on: /sys/fs/cgroup
use_percent: 0
used: 0
- 1k_blocks: 107323996
available: 15491352
filesystem: /dev/nvme0n1p7
mounted_on: /export
use_percent: 85
used: 86337860
- 1k_blocks: 98304
available: 29569
filesystem: /dev/nvme0n1p2
mounted_on: /boot/efi
use_percent: 70
used: 68735
- 1k_blocks: 801304
available: 801288
filesystem: tmpfs
mounted_on: /run/user/1000
use_percent: 1
used: 16

TASK [debug] ********************************************************************************
ok: [test_11] =>
mount_percent:
/: 1
ok: [test_13] =>
mount_percent:
/: 0
ok: [localhost] =>
mount_percent:
/: 78
/boot/efi: 70
/dev: 0
/dev/shm: 10
/export: 85
/run: 1
/run/lock: 1
/run/user/1000: 1
/sys/fs/cgroup: 0

PLAY RECAP **********************************************************************************
localhost: ok=4 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
test_11: ok=4 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
test_13: ok=4 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0}

Hi Vladimir, Thank you for very detailed information. The restricted bash VM doesn't have df command itself. It is customised Linux Console the alternative DF command is "monhmc -r disk -n0" — satsensort, Dec 16 '22 at 11:58
Output looks like below Filesystem 1K-blocks Used Available Use% Mounted on devtmpfs 15781376 0 15781376 0% /dev — satsensort, Dec 16 '22 at 12:00
Your solution worked. Many many thanks for quick help and fix. — satsensort, Dec 16 '22 at 12:08

Ansible collect df output and convert to dictionary from set_fact

1 Answers1

Linked