Dumping custom class objects to a YAML file

Question

I would like some help on dumping a custom class object to a YAML file. A representation of my class looks like below:

from classes.enum_classes import Priority, Deadline

class Test(yaml.YAMLObject):
    
    yaml_tag = u'!RandomTestClass'
 
    def __init__(self):
        
        self._name = ""
        self._request_id = ""
        self._order = None
        self._priority = Priority.medium
        self._deadline = Deadline.label_c

The last two parameters are objects of different classes, both of which are Enum derived classes. I am trying to dump the contents of an object of Test class to a YAML output file. My __repr__ method for the Test class looks so:

    def __repr__(self):
        return "(name=%r, request=%r, order=%r, priority=%r, deadline=%r)" % \
               ((str(self.name) + str(self.order)), self.request_id, self.order,
                self.priority.name, self._deadline.name)

Working off the constructors and representers section of the PyYAML documentation and the example provided there (especially considering that the authors use lists for some of the class variables), I would expect my YAML file to display the actual display tags in the __repr__ method, rather than the variable names themselves. This is what I see currently:

--- !ContainerClass
_requests: !!python/object/apply:collections.defaultdict
  args:
  - !!python/name:builtins.list ''
  dictitems:
    '108':
    - !RandomTestClass
      _deadline: &id002 !Deadline '3'
      _name: '108.1'
      _order: '1'
      _priority: &id006 !Priority '1'
      _request_id: '108'
    - !RandomTestClass
      _deadline: &id002 !Deadline '3'
      _name: '108.2'
      _order: '2'
      _priority: &id006 !Priority '1'
      _request_id: '108'
_name: TestContainer

I want it to look like so:

---
Requests:
    - name: '108.1'
    - requestID: '108'
    - priority: '1'
    - order: '1'
    - deadline: '3'
    - name: '108.2' <for the second entry in the list and so on>
Name: ContainerClass

No amount of fiddling around with the __repr__ method or anything else has resulted in the output I would like. So there are two issues I would love to get some help with.

1. How do I get a sane, readable representation? I am guessing I will have to write a custom representer, so if someone could guide me with some pointers, since I was not able to find much information on that any way.

2. Getting rid of those pointers, or whatever we would want to call them next to priority and deadline. Priority and Deadline are the two classes referenced in my __init___ method above, that are Enum subclasses. Since they are already subclasses, any attempts to subclass them to yaml.YAMLObject result in an error with mixins. To get around that, some posts suggested I do so:

class Priority(Enum):

    low = 0
    medium = 1
    high = 2


class Deadline(Enum):

    label_a = 0
    label_b = 1
    label_c = 2
    label_d = 3


def priority_enum_representer(dumper, data):
    return dumper.represent_scalar('!Priority', str(data.value))


def deadline_enum_representer(dumper, data):
    return dumper.represent_scalar('!Deadline', str(data.value))


yaml.add_representer(Deadline, deadline_enum_representer)
yaml.add_representer(Priority, priority_enum_representer)

Any information/pointers on solving these two issues will be much much appreciated. Apologies for the long post, but I have learnt that more information generally leads to much more precise help.

UPDATE:

My YAML file is written based on a list of these RandomTestClass objects that are stored in a defaultdict(list) in a ContainerClass.

class ContainerClass(yaml.YAMLObject):
    
    yaml_tag = u'ContainerClass'

    def __init__(self):
        
        self._name = ""
        self._requests = defaultdict(list)

    def __repr__(self):
        
        return "(Name=%r, Requests=%r)" % \
               (self._name, str(self._requests))

    @property
    def requests(self):
        return self._requests

    @requests.setter
    def requests(self, new_req, value=None):

        if type(new_req) is dict:
            self._requests = new_req
        else:
            try:
                self._requests[new_req].append(value)
            except AttributeError:
                # This means the key exists, but the value is a non-list
                # entity. Change existing value to list, append new value
                # and reassign to the same key
                list_with_values \
                    = [self._requests[new_req], value]
                self._requests[new_req] = list_with_values            

The ContainerClass holds instances of Test objects. In another class, which is the entry point for my code containing __main__, I create multiple instances of Test objects, that are then stored in an ```ContainerClass`` object and dumped out to the YAML file.

# requisite imports here, based on 
# how the files are stored

from classes.Test import Test
from classes.ContainerClass import ContainerClass

class RunTestClass:

if __name__ == '__main__':
    
    yaml_container = ContainerClass()

    test_object_a = Test()
    test_object_a._name = '108.1'
    test_object_a._order = 1
    test_object_a._request_id = '108'

    yaml_container._name = "TestContainer"
    yaml_container._requests[test_object_a._request_id] = test_object_a

    test_object_b = Test()
    test_object_b._name = '108.2'
    test_object_b._order = 2
    test_object_b._request_id = '108'

    yaml_container._name = "TestContainer"
    yaml_container._requests[test_object_b._request_id] = test_object_b

with open(output_file, mode='w+') as outfile:
    for test_class_object in yaml_container._requests:
        yaml.dump(test_class_object, outfile, default_flow_style=False,
                  explicit_start=True, explicit_end=True)

UPDATE:

Adding a single, consolidated file to the question, executable to replicate the issue.

import yaml
from enum import Enum
from collections import defaultdict


class Priority(Enum):

    low = 0
    medium = 1
    high = 2


class Deadline(Enum):

    label_a = 0
    label_b = 1
    label_c = 2
    label_d = 3


def priority_enum_representer(dumper, data):
    return dumper.represent_scalar('!Priority', str(data.value))


def deadline_enum_representer(dumper, data):
    return dumper.represent_scalar('!Deadline', str(data.value))


yaml.add_representer(Deadline, deadline_enum_representer)
yaml.add_representer(Priority, priority_enum_representer)


class Test(yaml.YAMLObject):
    
    yaml_tag = u'!RandomTestClass'
 
    def __init__(self):
        
        self._name = ""
        self._request_id = ""
        self._order = None
        self._priority = Priority.medium
        self._deadline = Deadline.label_c

    @property
    def name(self):
        return self._name

    @name.setter
    def name(self, name):
        self._name = name

    @property
    def request_id(self):
        return self._request_id

    @request_id.setter
    def request_id(self, r_id):
        self._request_id = r_id

    @property
    def order(self):
        return self._order

    @order.setter
    def order(self, order):
        self._order = order

    @property
    def priority(self):
        return self._priority

    @priority.setter
    def priority(self, priority):
        self._priority = priority

    @property
    def deadline(self):
        return self._deadline

    @deadline.setter
    def deadline(self, deadline):
        self._deadline = deadline

    def __str__(self):
        return self.name + ", " + self._request_id + ", " + str(self.order) + ", " \
               + str(self.priority) + ", " + str(self.deadline)


class ContainerClass(yaml.YAMLObject):
    
    yaml_tag = u'ContainerClass'

    def __init__(self):
        
        self._name = ""
        self._requests = defaultdict(list)

    def __repr__(self):
        
        return "(Name=%r, Requests=%r)" % \
               (self._name, str(self._requests))

    @property
    def name(self):
        return self._name

    @name.setter
    def name(self, name):
        self._name = name

    @property
    def requests(self):
        return self._requests

    def set_requests(self, new_req, value=None):

        if type(new_req) is dict:
            self._requests = new_req
        else:
            try:
                self._requests[new_req].append(value)
            except AttributeError:
                # This means the key exists, but the value is a non-list
                # entity. Change existing value to list, append new value
                # and reassign to the same key
                print("Encountered a single value, converting to a list and appending new value")
                list_with_values \
                    = [self._requests[new_req], value]
                self._requests[new_req] = list_with_values


yaml_container = ContainerClass()
yaml_container.name = "TestContainer"

test_object_a = Test()
test_object_a._name = '108.1'
test_object_a._order = 1
test_object_a._request_id = '108'
yaml_container.set_requests(test_object_a.request_id, test_object_a)

test_object_b = Test()
test_object_b._name = '108.2'
test_object_b._order = 2
test_object_b._request_id = '108'
yaml_container.set_requests(test_object_b.request_id, test_object_b)

with open('test.yaml', mode='w+') as outfile:
    yaml.dump(yaml_container, outfile, default_flow_style=False,
              explicit_start=True, explicit_end=True)

Answer 1

There are different ways to solve this. Generally, you want to define a classmethod to_yaml in ContainerClass that will be called by the dumper, and implement the transition from native structure to YAML node graph there.

A minimal solution would be to just construct the structure you want to have as normal dict with a list in it, and tell the dumper to use that instead of the real structure:

    @classmethod
    def to_yaml(cls, dumper, data):
        rSeq = []
        for value in data._requests.values():
            for t in value:
              rSeq.extend([
                {"name": t._name},
                {"requestID": t._request_id},
                {"priority": t._priority},
                {"order": t._order},
                {"deadline": t._deadline}
              ])
        return dumper.represent_mapping("tag:yaml.org,2002:map",
                {"Requests": rSeq, "Name": data._name})

This will give you

---
Name: TestContainer
Requests:
- name: '108.1'
- requestID: '108'
- priority: &id001 !Priority '1'
- order: 1
- deadline: &id002 !Deadline '2'
- name: '108.2'
- requestID: '108'
- priority: *id001
- order: 2
- deadline: *id002
...

YAML generates anchors & aliases for priority and requestID because the values refer the same objects. You can avoid those by doing

yaml.Dumper.ignore_aliases = lambda *args : True

before you dump the YAML.

You can be more sophisticated and iterate the properties instead of hard coding the names, but that is beyond the scope of this answer. If you want to load this YAML again into the same structure, you will need to add another classmethod from_yaml implementing the reverse transformation.