Dynamically creating Django models with `type`

Question

I have 20+ MySQL tables, prm_a, prm_b, ... with the same basic structure but different names, and I'd like to associate them with Django model classes without writing each one by hand. So, feeling ambitious, I thought I'd try my hand at using type() as a class-factory:

The following works:

def get_model_meta_class(prm_name):
    class Meta:
        app_label = 'myapp'
    setattr(Meta, 'db_table', 'prm_%s' % prm_name)
    return Meta

prm_class_attrs = {
    'foo': models.ForeignKey(Foo),
    'val': models.FloatField(),
    'err': models.FloatField(blank=True, null=True),
    'source': models.ForeignKey(Source),
    '__module__': __name__,
}

###
prm_a_attrs = prm_class_attrs.copy()
prm_a_attrs['Meta'] = get_model_meta_class('a')
Prm_a = type('Prm_a', (models.Model,), prm_a_attrs)

prm_b_attrs = prm_class_attrs.copy()
prm_b_attrs['Meta'] = get_model_meta_class('b')
Prm_b = type('Prm_b', (models.Model,), prm_b_attrs)
###

But if I try to generate the model classes as follows:

###
prms = ['a', 'b']
for prm_name in prms:
    prm_class_name = 'Prm_%s' % prm_name
    prm_class = type(prm_class_name, (models.Model,), prm_class_attrs)
    setattr(prm_class, 'Meta', get_model_meta_class(prm_name))
    globals()[prm_class_name] = prm_class
###

I get a curious Exception on the type() line (given that __module__ is, in fact, in the prm_class_attrs dictionary):

File ".../models.py", line 168, in <module>
    prm_class = type(prm_class_name, (models.Model,), prm_class_attrs)
  File ".../lib/python2.7/site-packages/django/db/models/base.py", line 79, in __new__
    module = attrs.pop('__module__')
KeyError: u'__module__' 

So I have two questions: what's wrong with my second approach, and is this even the right way to go about creating my class models?

OK - thanks to @Anentropic, I see that the items in my prm_class_attrs dictionary are being popped away by Python when it makes the classes. And I now have it working, but only if I do this:

attrs = prm_class_attrs.copy()
attrs['Meta'] = get_model_meta_class(prm_name)
prm_class = type(prm_class_name, (models.Model,), attrs)

not if I set the Meta class as an attribtue with

setattr(prm_class, 'Meta', get_model_meta_class(prm_name))

I don't really know why this is, but at least I have it working now.

Answer 1

The imediate reason is because you are not doing prm_class_attrs.copy() in your for loop, so the __modules__ key is getting popped out of the dict on the first iteration

As for why this doesn't work:

setattr(prm_class, 'Meta', get_model_meta_class(prm_name))

...it's to do with the fact that Django's models.Model has a metaclass. But this is a Python metaclass which customises the creation of the model class and is nothing to do with the Meta inner-class of the Django model (which just provides 'meta' information about the model).

In fact, despite how it looks when you define the class in your models.py, the resulting class does not have a Meta attribute:

class MyModel(models.Model):
    class Meta:
        verbose_name = 'WTF'

>>> MyModel.Meta
AttributeError: type object 'MyModel' has no attribute 'Meta'

(You can access the Meta class directly, but aliased as MyModel._meta)

The model you define in models.py is really more of a template for a model class than the actual model class. This is why when you access a field attribute on a model instance you get the value of that field, not the field object itself.

Django model inheritance can simplify a bit what you're doing:

class GeneratedModelBase(models.Model):
    class Meta:
        abstract = True
        app_label = 'myapp'

    foo = models.ForeignKey(Foo)
    val = models.FloatField()
    err = models.FloatField(blank=True, null=True)
    source = models.ForeignKey(Source)

def generate_model(suffix):
    prm_class_name = 'Prm_%s' % prm_name
    prm_class = type(
        prm_class_name,
        (GeneratedModelBase,),
        {
            # this will get merged with the attrs from GeneratedModelBase.Meta
            'Meta': {'db_table', 'prm_%s' % prm_name},
            '__module__': __name__,
        }
    )
    globals()[prm_class_name] = prm_class
    return prm_class

prms = ['a', 'b']
for prm_name in prms:
    generate_model(prm_name)

Answer 2

You can use ./manage.py inspectdb This will print out a python models file for the DB you're pointing at in your settings.py Documentation

EDIT For dynamic models, try django-mutant or check out this link

Answer 3

For anyone still wondering how to do this, I took the genius answer from @Anentropic and made it work with some modifications.

I also changed the db table name to something more like django uses to name the tables of the models ("appname_lowercaseclassname") by stripping all non alphabetic characters from the class name and converting the resulting string to lower case.

I Tested it on Django 2.2.6

def generate_model(class_name):

    clean_name_for_table = ''.join(filter(str.isalpha, class_name)).lower() # Could also use regex to remove all non alphabetic characters.
    db_table_name = f"{__package__}_{clean_name_for_table}"
    prm_class = type(
        class_name,
        (BaseClass,),
        {
            # this will get merged with the attrs from GeneratedModelBase.Meta
            'Meta': type("Meta",(),{'db_table':db_table_name}),
            '__module__': __name__,
        }
    )
    globals()[class_name] = prm_class
    return prm_class