Python

JSON (De)Serialization of nested objects

During my first encounter of handling JSON (de)serialization in Python, I faced the problem of (de)serializing objects that have properties that are instances of another class.

Using the json module, one has to write two methods, a complex_handler and a class_mapper that are fed to json.dumps and json.loads respectively.

The design problem here is that the class_mapper needs to compare a dict that is to be deserialized with the properties of potential classes in order to find the matching type. In the first level of deserialization one could potentially provide the type as a parameter to the deserialization-function, but as soon as there is a child property, the type may be unknown.

Therefore each class that is to be deserialized has to be registered into a collection of (properties) -> class mappings.

To simplify the handling, I wrote the following JsonConvert class:

import json


class JsonConvert(object):
    mappings = {}
    
    @classmethod
    def class_mapper(clsself, d):
        for keys, cls in clsself.mappings.items():
            if keys.issuperset(d.keys()):   # are all required arguments present?
                return cls(**d)
        else:
            # Raise exception instead of silently returning None
            raise ValueError('Unable to find a matching class for object: {!s}'.format(d))
    
    @classmethod
    def complex_handler(clsself, Obj):
        if hasattr(Obj, '__dict__'):
            return Obj.__dict__
        else:
            raise TypeError('Object of type %s with value of %s is not JSON serializable' % (type(Obj), repr(Obj)))

    @classmethod
    def register(clsself, cls):
        clsself.mappings[frozenset(tuple([attr for attr,val in cls().__dict__.items()]))] = cls
        return cls

    @classmethod
    def ToJSON(clsself, obj):
        return json.dumps(obj.__dict__, default=clsself.complex_handler, indent=4)

    @classmethod
    def FromJSON(clsself, json_str):
        return json.loads(json_str, object_hook=clsself.class_mapper)
    
    @classmethod
    def ToFile(clsself, obj, path):
        with open(path, 'w') as jfile:
            jfile.writelines([clsself.ToJSON(obj)])
        return path

    @classmethod
    def FromFile(clsself, filepath):
        result = None
        with open(filepath, 'r') as jfile:
            result = clsself.FromJSON(jfile.read())
        return result

Now we can define some classes and make them (de)serializable by decorating them with JsonConvert.register:

@JsonConvert.register
class Employee(object):
    def __init__(self, Name:int=None, Age:int=None):
        self.Name = Name
        self.Age = Age
        return

@JsonConvert.register
class Company(object):
    def __init__(self, Name:str="", Employees:[Employee]=None):
        self.Name = Name
        self.Employees = [] if Employees is None else Employees
        return

When the class definition is parsed, this will register it with the static mappings in JsonConvert.

Now we can easily serialize and deserialize our Company to a JSON-string:

company = Company("Contonso")
company.Employees.append(Employee("Werner", 38))
company.Employees.append(Employee("Mary"))

asJson = JsonConvert.ToJSON(company)
fromJson = JsonConvert.FromJSON(asJson)
asJsonFromJson = JsonConvert.ToJSON(fromJson)

assert(asJsonFromJson == asJson)

print(asJsonFromJson)

Or directly to and from a file:

filepath = JsonConvert.ToFile(company, "company.json")
fromFile = JsonConvert.FromFile(filepath)

The JSON string looks like this:

{
    "Name": "Contonso",
    "Employees": [
        {
            "Name": "Werner",
            "Age": 38
        },
        {
            "Name": "Mary",
            "Age": null
        }
    ]
}

It’s not as convenient as our beloved Json.NET for C#/.NET, but it can get the job done.
cheers

PS.: if you want to learn more about decorators, have a look at this excellent YouTube tutorial.

5 comments

  1. 1oglop1

    Hi, nice work!
    I noticed that you adopted the convention from Json.NET but this is python and this code violates PEP8 and probably other PEPs too, could you please fix it? with black and pylint/flake8?
    In case you need any help or you are curious about it, let me know 😉

    1. theCake Post author

      Thanks!

      Unfortunately, I have to admit you are absolutely right…
      I’ll put it on my list, but due to a busy week, you’re of course invited to propose a revision.

      cheers

  2. Anonymous

    I get this error with Python 3.6:

    File “C:\Projects\PycharmProjects\Random\lib\JsonConvert.py”, line 31, in ToJSON
    return json.dumps(obj.__dict__, default=clsself.complex_handler, indent=4)
    AttributeError: ‘list’ object has no attribute ‘__dict__’

    Any ideas?

  3. Nitin Muteja`

    What if two classes have same attribute names? For instance a response object from two service? How do we handle the conflict in that case?

    1. theCake Post author

      Yes, that’s problematic with the code I showed. Note that my post is from 2016..

      I guess you could include the class name with in the key that’s used in the `mappings` dict.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.