-1

I am trying to create local users on remote systems using Ansible.
The user list is read from a CSV formatted file using Ansible's read_csv.
The CSV file is formatted in UTF-8 with Byte order mark (BOM).

I am getting the error 'dict object' has no attribute 'Username'.
But 'Username' does in fact exist in the CSV file's header line, and it is the first word of the file.

Property file:

Username,UID
user1,70001

Playbook:

---
- name: Reading users from CSV file
  community.general.read_csv:
    path: files/users.csv
  register: users
  delegate_to: localhost

- name: Create from CSV file
  user:
    name: "{{ item.Username }}"
    uid: "{{ item.UID }}"
    state: present
  loop: "{{ users.list }}"

Error:

TASK [manage_local_users : Create from CSV file] ******************************************************************************************************************************************************************************
task path: /local/ansible/manage-users/playbooks/roles/manage_local_users/tasks/config.yml:12
fatal: [myhost.acme.com]: FAILED! => {
    "msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'Username'

The error appears to be in '/local/ansible/manage-users/playbooks/roles/manage_local_users/tasks/config.yml': line 12, column 3, but maybe elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:
- name: Create from CSV file
  ^ here"
}

Output from previous task ("Reading users from CSV file") in -vvvv mode:

ok: [myhost.acme.com -> localhost] => {
    "changed": false,
    "dict": {},
    "invocation": {
        "module_args": {
            "delimiter": null,
            "dialect": "excel",
            "fieldnames": null,
            "key": null,
            "path": "files/users.csv",
            "skipinitialspace": null,
            "strict": null,
            "unique": true
        }
    },
    "list": [
        {
            "UID": "70001",
            "Username": "user1"
        }
    ]
}

I tried:

  • Running against another host

  • Create a new property file from scratch

  • Reduce data in the property file

  • Tried both loop: "{{ users.list }}" and with_dict: "{{ users }}"

  • Reduce number of tasks in the Ansible role

EDIT:

I have figured out that the input csv file has caused this issue.
This is the file causing the issue:

# file files/users_BROKEN.csv 
files/users_BROKEN.csv: UTF-8 Unicode (with BOM) text

# hexdump -C files/users_BROKEN.csv 
00000000  ef bb bf 55 73 65 72 6e  61 6d 65 2c 55 49 44 0a  |...Username,UID.|
00000010  75 73 65 72 31 2c 37 30  30 30 31 0a              |user1,70001.|
0000001c

While this file is not causing the issue:

# file files/users_CORRECT.csv 
users.csv: ASCII text

# hexdump -C files/users_CORRECT.csv 
00000000  55 73 65 72 6e 61 6d 65  2c 55 49 44 0a 75 73 65  |Username,UID.use|
00000010  72 31 2c 37 30 30 30 31  0a                       |r1,70001.|
00000019
norinori
  • 11
  • 3
  • 3
    `with_dict: "{{ users }}"` => `loop: "{{ users.list }}"` – Zeitounator Oct 31 '22 at 18:34
  • Agree. But both flavors give the same error. – norinori Oct 31 '22 at 20:00
  • Same error when usinig `loop: "{{ users.list }}"` rather than `when_dict: "{{ users }}`. Why is this question closed, when no solution was provided? – norinori Oct 31 '22 at 20:04
  • @Zeitounator the question should be reopened, as I was able to find a reasonable solution. – norinori Oct 31 '22 at 23:46
  • 1
    The csv input file was causing the problem. `file users.csv` outputs `UTF-8 Unicode (with BOM) text`. With a ASCII plaintext file, Ansible runs without error. It should be pointed out, that no difference in file contents is visible to the eye between the two seemingly identical file versions, but `diff` tells that there is a difference in the header line, so that "`Username`" (UTF-8) is indeed different than "`Username`" (ASCII). – norinori Nov 01 '22 at 00:02
  • 1
    Please read the help section to understand how this site works. I don't have the power to reopen a question on my own. [Edit] your question with the new details and see if you get enough votes (i.e. three) for this to happen. – Zeitounator Nov 01 '22 at 05:18
  • @Zeitounator I already edited the question, changed what's necessary, and set the flag to reopen - but nothing happens. Thanks for explaining. – norinori Nov 01 '22 at 12:19
  • In respect to your comment it might additionally be necessary to provide the output of `hexdump -C files/users.csv` since with the given information your issue is still not be reproducible. This is because of [Can I use US-ASCII and UTF-8 encoding interchangeably?](https://softwareengineering.stackexchange.com/a/263701/302852) and therefore "_"Username" (UTF-8) is indeed different than "Username" (ASCII)_" can't be the case. – U880D Nov 01 '22 at 17:45
  • @U880D I have added the `hexdump -C` to the problem description. Many thanks, that's a really useful idea! – norinori Nov 01 '22 at 20:45
  • So the question is more "_Is Ansible `read_csv` module capable to read UTF-8 encoded files which have byte-order mark (BOM) signature `EF BB BF`?_". See [The byte-order mark (BOM)](https://www.w3.org/International/questions/qa-byte-order-mark). – U880D Nov 02 '22 at 06:28
  • You may have a look into Ansible Issue #[2189](https://github.com/ansible-collections/community.general/issues/2189) specificly and as it could answer the question already or Ansible Issue #[26048](https://github.com/ansible/ansible/issues/26048) more generally. – U880D Nov 02 '22 at 06:38
  • 1
    @U880D Thanks for the additional links, they are quite useful. I disagree that the title should be changed. The current title is what people will experience, when the use `read_csv` on UTF-8 with BOM, and the search engine may take them here. It's a pity that the question isn't reopened. This discussion is quite useful to the world. – norinori Nov 02 '22 at 09:06
  • What you've experienced during your question is just the outcome of something which is caused on an other part or domain of software. The title may also be mislead and not help to narrow down the cause fully. So you could either rephrase your question fully or maybe open an other one. – U880D Nov 02 '22 at 09:36
  • @U880D I changed the title and initial description. It should be clearer now. – norinori Nov 02 '22 at 15:06

1 Answers1

0

Avoid using Unicode with BOM (Byte order mark), when reading files with Ansible.
Convert text files to a format, that does not use BOM (e.g. UTF-8 without BOM, ASCII, etc)

It seems that Ansible (as of today, 2022) is not supporting BOM:
'utf8' codec can't decode byte 0xff in position 0: invalid start byte" #23903

On Linux, dos2unix may be used to get rid of BOM.
How can I remove the BOM from a UTF-8 file?

norinori
  • 11
  • 3