Convert Django Model object to dict with all of the fields intact

Question

How does one convert a Django Model object to a dict with all of its fields? All ideally includes foreign keys and fields with editable=False.

Let me elaborate. Let's say I have a Django model like the following:

from django.db import models

class OtherModel(models.Model): pass

class SomeModel(models.Model):
    normal_value = models.IntegerField()
    readonly_value = models.IntegerField(editable=False)
    auto_now_add = models.DateTimeField(auto_now_add=True)
    foreign_key = models.ForeignKey(OtherModel, related_name="ref1")
    many_to_many = models.ManyToManyField(OtherModel, related_name="ref2")

In the terminal, I have done the following:

other_model = OtherModel()
other_model.save()
instance = SomeModel()
instance.normal_value = 1
instance.readonly_value = 2
instance.foreign_key = other_model
instance.save()
instance.many_to_many.add(other_model)
instance.save()

I want to convert this to the following dictionary:

{'auto_now_add': datetime.datetime(2015, 3, 16, 21, 34, 14, 926738, tzinfo=<UTC>),
 'foreign_key': 1,
 'id': 1,
 'many_to_many': [1],
 'normal_value': 1,
 'readonly_value': 2}

---

Questions with unsatisfactory answers:

Django: Converting an entire set of a Model's objects into a single dictionary

How can I turn Django Model objects into a dictionary and still have their foreign keys?

Answer 1

There are many ways to convert an instance to a dictionary, with varying degrees of corner case handling and closeness to the desired result.

---

1. instance.__dict__

instance.__dict__

which returns

{'_foreign_key_cache': <OtherModel: OtherModel object>,
 '_state': <django.db.models.base.ModelState at 0x7ff0993f6908>,
 'auto_now_add': datetime.datetime(2018, 12, 20, 21, 34, 29, 494827, tzinfo=<UTC>),
 'foreign_key_id': 2,
 'id': 1,
 'normal_value': 1,
 'readonly_value': 2}

This is by far the simplest, but is missing many_to_many, foreign_key is misnamed, and it has two unwanted extra things in it.

---

2. model_to_dict

from django.forms.models import model_to_dict
model_to_dict(instance)

which returns

{'foreign_key': 2,
 'id': 1,
 'many_to_many': [<OtherModel: OtherModel object>],
 'normal_value': 1}

This is the only one with many_to_many, but is missing the uneditable fields.

---

3. model_to_dict(..., fields=...)

from django.forms.models import model_to_dict
model_to_dict(instance, fields=[field.name for field in instance._meta.fields])

which returns

{'foreign_key': 2, 'id': 1, 'normal_value': 1}

This is strictly worse than the standard model_to_dict invocation.

---

4. query_set.values()

SomeModel.objects.filter(id=instance.id).values()[0]

which returns

{'auto_now_add': datetime.datetime(2018, 12, 20, 21, 34, 29, 494827, tzinfo=<UTC>),
 'foreign_key_id': 2,
 'id': 1,
 'normal_value': 1,
 'readonly_value': 2}

This is the same output as instance.__dict__ but without the extra fields. foreign_key_id is still wrong and many_to_many is still missing.

---

5. Custom Function

The code for django's model_to_dict had most of the answer. It explicitly removed non-editable fields, so removing that check and getting the ids of foreign keys for many to many fields results in the following code which behaves as desired:

from itertools import chain

def to_dict(instance):
    opts = instance._meta
    data = {}
    for f in chain(opts.concrete_fields, opts.private_fields):
        data[f.name] = f.value_from_object(instance)
    for f in opts.many_to_many:
        data[f.name] = [i.id for i in f.value_from_object(instance)]
    return data

While this is the most complicated option, calling to_dict(instance) gives us exactly the desired result:

{'auto_now_add': datetime.datetime(2018, 12, 20, 21, 34, 29, 494827, tzinfo=<UTC>),
 'foreign_key': 2,
 'id': 1,
 'many_to_many': [2],
 'normal_value': 1,
 'readonly_value': 2}

---

6. Use Serializers

Django Rest Framework's ModelSerialzer allows you to build a serializer automatically from a model.

from rest_framework import serializers
class SomeModelSerializer(serializers.ModelSerializer):
    class Meta:
        model = SomeModel
        fields = "__all__"

SomeModelSerializer(instance).data

returns

{'auto_now_add': '2018-12-20T21:34:29.494827Z',
 'foreign_key': 2,
 'id': 1,
 'many_to_many': [2],
 'normal_value': 1,
 'readonly_value': 2}

This is almost as good as the custom function, but auto_now_add is a string instead of a datetime object.

---

Bonus Round: better model printing

If you want a django model that has a better python command-line display, have your models child-class the following:

from django.db import models
from itertools import chain

class PrintableModel(models.Model):
    def __repr__(self):
        return str(self.to_dict())

    def to_dict(instance):
        opts = instance._meta
        data = {}
        for f in chain(opts.concrete_fields, opts.private_fields):
            data[f.name] = f.value_from_object(instance)
        for f in opts.many_to_many:
            data[f.name] = [i.id for i in f.value_from_object(instance)]
        return data

    class Meta:
        abstract = True

So, for example, if we define our models as such:

class OtherModel(PrintableModel): pass

class SomeModel(PrintableModel):
    normal_value = models.IntegerField()
    readonly_value = models.IntegerField(editable=False)
    auto_now_add = models.DateTimeField(auto_now_add=True)
    foreign_key = models.ForeignKey(OtherModel, related_name="ref1")
    many_to_many = models.ManyToManyField(OtherModel, related_name="ref2")

Calling SomeModel.objects.first() now gives output like this:

{'auto_now_add': datetime.datetime(2018, 12, 20, 21, 34, 29, 494827, tzinfo=<UTC>),
 'foreign_key': 2,
 'id': 1,
 'many_to_many': [2],
 'normal_value': 1,
 'readonly_value': 2}