1

Reason for question

I have a requirement to check for any hard-coded IP address which may have been left within the Playbooks in our environment.

Scenario

The following code which will find files containing IPv4 addresses and store them in playbook_files.

- hosts: localhost
  tasks:
    - name: Loop through Playbooks
      find:
        paths: "{{ playbook_dir }}/.."
        file_type: file
        recurse: yes
        contains: ^.*(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$
      register: playbook_files

    - name: "Test-1"
    - debug:
        msg: 
          - "{{ playbook_files.files | type_debug }}"
          - "{{ playbook_files.files }}"}}"

    - name: "Test-2"
      debug:
        msg: "Path: {{ item }}"
      loop: "{{ playbook_files.files | map(attribute='path') | list  }}"


    - set_fact:
        files_with_hardcoded_ips:
          - "/home/ansible/playbooks/README.md"
          - "/home/ansible/playbooks/test/vars.yml"

    - name: "Test 3"
      debug:
        msg: 
          - "files_with_hardcoded_ips: {{ files_with_hardcoded_ips | type_debug }}"
          - "{{ files_with_hardcoded_ips }}"

The result of that being:

TASK 1/5 [Loop through Playbooks] *******************************************************************************************
ok: 1/1 [localhost] => {"changed": false, "examined": 295, "files": [{"atime": 1677573594.1507878, "ctime": 1677573594.1507878, "dev": 66305, "gid": 1009, "gr_name": "ec2_user", "inode": 13302670, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mode": "0775", "mtime": 1677573594.1507878, "nlink": 1, "path": "/home/ansible/playbooks/README.md", "pw_name": "ec2_user", "rgrp": true, "roth": true, "rusr": true, "size": 5571, "uid": 1009, "wgrp": true, "woth": false, "wusr": true, "xgrp": true, "xoth": true, "xusr": true}, 
{"atime": 1677573594.1507878, "ctime": 1677573594.1507878, "dev": 66305, "gid": 1009, "gr_name": "ec2_user", "inode": 16787335, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mode": "0775", "mtime": 1677573594.1507878, "nlink": 1, "path": "/home/ansible/playbooks/test/vars.yml", "pw_name": "ec2_user", "rgrp": true, "roth": true, "rusr": true, "size": 232, "uid": 1009, "wgrp": true, "woth": false, "wusr": true, "xgrp": true, "xoth": true, "xusr": true}], "matched": 30, "msg": ""}

TASK 2/5 [Test-1] ***********************************************************************************************************
ok: 1/1 [localhost] => {
    "msg": [
        "list",
        [
            {
                "atime": 1677573594.1507878,
                "ctime": 1677573594.1507878,
                "dev": 66305,
                "gid": 1009,
                "gr_name": "ansible",
                "inode": 13302670,
                "isblk": false,
                "ischr": false,
                "isdir": false,
                "isfifo": false,
                "isgid": false,
                "islnk": false,
                "isreg": true,
                "issock": false,
                "isuid": false,
                "mode": "0775",
                "mtime": 1677573594.1507878,
                "nlink": 1,
                "path": "/home/ansible/playbooks/README.md",
                "pw_name": "ansible",
                "rgrp": true,
                "roth": true,
                "rusr": true,
                "size": 5571,
                "uid": 1009,
                "wgrp": true,
                "woth": false,
                "wusr": true,
                "xgrp": true,
                "xoth": true,
                "xusr": true
            },
            {
                "atime": 1677573594.1507878,
                "ctime": 1677573594.1507878,
                "dev": 66305,
                "gid": 1009,
                "gr_name": "ansible",
                "inode": 16787335,
                "isblk": false,
                "ischr": false,
                "isdir": false,
                "isfifo": false,
                "isgid": false,
                "islnk": false,
                "isreg": true,
                "issock": false,
                "isuid": false,
                "mode": "0775",
                "mtime": 1677573594.1507878,
                "nlink": 1,
                "path": "/home/ansible/playbooks/test/vars.yml",
                "pw_name": "ansible",
                "rgrp": true,
                "roth": true,
                "rusr": true,
                "size": 232,
                "uid": 1009,
                "wgrp": true,
                "woth": false,
                "wusr": true,
                "xgrp": true,
                "xoth": true,
                "xusr": true
            }
        ]
    ]
}
TASK 3/5 [Test-2] ***********************************************************************************************************
ok: 1/1 [localhost] => {

TASK 4/5 [set_fact] *********************************************************************************************************
ok: 1/1 [localhost] => {"ansible_facts": {"files_with_hardcoded_ips": ["/home/ansible/playbooks/README.md", "/home/ansible/playbooks/test/vars.yml"]}, "changed": false}

TASK 5/5 [Test 3] ***********************************************************************************************************
ok: 1/1 [localhost] => {
    "msg": [
        "files_with_hardcoded_ips: list",
        [
            "/home/ansible/playbooks/README.md",
            "/home/ansible/playbooks/test/vars.yml"
        ]
    ]
}

Test-1: Shows the data_type and the file information.

Test-2: Unsuccessfully attempts to loop through the list of files found.

Test-3: Shows what I'd like to achieve.

Symptom

I am unable to loop through the list of files found to contain IP address and make use of the results. The output from Test-1 shows the playbook_files.files is a list, and gives me a list of two objects detailing the files found.

However, when I try and loop through the list to display just the path value, I get nothing. I think this is because playbook_files.files is a list of dictionaries.

Required Result

I would like to be able to loop through playbook_files.files and do the following:

  • Ignore files README.md files.

  • Register the fact: files_with_hardcoded_ips: containing the path to the files within playbook_files.files

Resulting example:

ok: 1/1 [localhost] => {
    "msg": [
        [
            "/home/ansible/playbooks/README.md",
            "/home/ansible/playbooks/test/vars.yml"
        ]
    ]
}
  • Output the line(s) containing the hardcoded IP with the file.

Resulting example:

In this example the vars.yml file has two lines containing an IP. One is a comment the other is not.

ok: 1/1 [localhost] => {
    "test4": {
        "home/ansible/playbooks/test/vars.yml": [
            "# Example IP Address 192.168.1.20",
            "    ip: 192.168.1.20"
        ]
    }
}

As always, your expert help is very much appreciated!

Simon.

Thank you to @Zeitounato, the selection method suggested by works a treat, as does:

- set_fact:
    files_with_hardcoded_ips: "{{ playbook_files | json_query('files[*].path') }}"

However, when I try to loop over the files_with_hardcoded_ips list it fails, see below.

- hosts: localhost
  tasks:
    - name: Loop through Playbooks
      find:
        paths: "{{ playbook_dir }}/.."
        file_type: file
        recurse: yes
        contains: ^.*(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$
      register: playbook_files
      
  - name: store file paths containing ips excluding any README.md in a fact 
    set_fact:
      files_with_hardcoded_ips: "{{ playbook_files.files | map(attribute='path') | reject('search', 'README.md') }}"

  - name: Show list of files and data type.
    debug:
      msg:
        - "{{ files_with_hardcoded_ips | type_debug }}"
        - "{{ files_with_hardcoded_ips }}"

  - name: Loop over list of files and show data type.
    debug:
      msg: "{{ item }} {{ item | type_debug }}"
    loop: "{{ files_with_hardcoded_ips }}"

Resulting in:

TASK [store file paths containing ips excluding any README.md in a fact] **************************************************************************************************************************
task path: /home/ansible/playbooks/test/test_hardcoded_ips.yml:23
ok: 1/1 [localhost] => {
    "ansible_facts": {
        "files_with_hardcoded_ips": [
            "/home/ansible/playbooks/test/vars.yml"
            "/home/ansible/playbooks/test-play-2/test-play-2.yml"
        ]
    },
    "changed": false
}

TASK [Show list of files.] ************************************************************************************************************************************************************************
task path: /home/ansible/playbooks/test/test_hardcoded_ips.yml:27

What I'm not understanding now is why I cannot loop over the files_with_hardcoded_ips list, and display or use those files. It's really really odd!

user51760
  • 127
  • 8
  • Not related to your direct problem: I suggest you see [this](https://stackoverflow.com/a/36760050/9401096) for much shorter yet fully functionnal alternatives to your ipv4 regex. – Zeitounator Feb 28 '23 at 13:31
  • Thanks again, I had see that page, very interesting, I think I took the regex I used from there tbh! – user51760 Mar 02 '23 at 16:09
  • The output is not aligned to the playbook your are showing, i.e. the name of task after set_fact is `Show list of files and data type.` in your playbook and `Show list of files.` in your output. So please double check the content. – Zeitounator Mar 02 '23 at 16:15

1 Answers1

0

In a nutshell:

  - name: store file paths containing ips excluding any README.md in a fact
    set_fact:
      files_with_hardcoded_ips: "{{ playbook_files.files | map(attribute='path') | reject('search', 'README.md') }}
Zeitounator
  • 38,476
  • 7
  • 53
  • 66