How do I handle exceptions with Django objects.bulk_create()

Question

I am using bulk_createto create objects of a model. Whenever an exception is raised, how do I handle it?

aList = [
    Student(name="Jason"),
    Student(name="Mark"),
    Student(name="Tom"),
    Student(name="Jason"),
    Student(name="Tom"),
]

Student.objects.bulk_create(aList)

as the model has field name that is unique=True, I have to process the inputs as the exception is raised. How do I handle exceptions one by one.

This don't work as intended,

try:
    # bulk_create
except:
    # handlers

as the exception is raised, the bulk_create process is terminated.

PS. I am looking forward for the bulk_create. no loops with create or update_or_create or get_or_create

Possible duplicate of [Django bulk_create with ignore rows that cause IntegrityError?](https://stackoverflow.com/questions/12451053/django-bulk-create-with-ignore-rows-that-cause-integrityerror/54189488) — shad0w_wa1k3r, Jan 12 '21 at 13:55

ivan Kir · Answer 1 · 2020-08-12T08:57:28.693

2

actually there is a flag ignore_conflicts=False, that catches exceptions, so it makes sense to try ModelName.objects.bulk_create([ModelName(name='spam', slug='eggs')...], ignore_conflicts=True)

edited Aug 12 '20 at 08:57

answered Dec 05 '19 at 18:54

ivan Kir

31
4

score 1 · Accepted Answer · answered Apr 17 '16 at 09:20

Well, that's impossible. That's how bulk_create is defined:

def bulk_create(self, objs, batch_size=None):
        for parent in self.model._meta.get_parent_list():
            if parent._meta.concrete_model is not self.model._meta.concrete_model:
                raise ValueError("Can't bulk create a multi-table inherited model")
        if not objs:
            return objs
        self._for_write = True
        connection = connections[self.db]
        fields = self.model._meta.concrete_fields
        objs = list(objs)
        self._populate_pk_values(objs)
        with transaction.atomic(using=self.db, savepoint=False):
            if (connection.features.can_combine_inserts_with_and_without_auto_increment_pk
                    and self.model._meta.has_auto_field):
                self._batched_insert(objs, fields, batch_size)
            else:
                objs_with_pk, objs_without_pk = partition(lambda o: o.pk is None, objs)
                if objs_with_pk:
                    self._batched_insert(objs_with_pk, fields, batch_size)
                if objs_without_pk:
                    fields = [f for f in fields if not isinstance(f, AutoField)]
                    self._batched_insert(objs_without_pk, fields, batch_size)

        return objs

And the _batched_insert:

def _batched_insert(self, objs, fields, batch_size):
    """
    A little helper method for bulk_insert to insert the bulk one batch
    at a time. Inserts recursively a batch from the front of the bulk and
    then _batched_insert() the remaining objects again.
    """
    if not objs:
        return
    ops = connections[self.db].ops
    batch_size = (batch_size or max(ops.bulk_batch_size(fields, objs), 1))
    for batch in [objs[i:i + batch_size]
                  for i in range(0, len(objs), batch_size)]:
        self.model._base_manager._insert(batch, fields=fields,
                                         using=self.db)

So, as you can see bulk_create is basically a for loop inside an transaction.atomic.

One more thing. It is also impossible to save only some entries inside the transaction block. It either is executed fully or not executed at all.

score 0 · Answer 3 · answered Oct 06 '22 at 10:40

0

you can handle it by just doing

objs = [(Event), (Event), (Event)...]

try:
  Event.objects.bulk_create(objs)
except IntegrityError:
   for obj in objs:
    try:
      obj.save()
    except IntegrityError:
      continue

answered Oct 06 '22 at 10:40

Pape Ibrahima Diawara

1

Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Oct 10 '22 at 15:29

How do I handle exceptions with Django objects.bulk_create()

3 Answers3