3.15 regression: Serializer validation failed for unique together constraint #9358

anndoc · 2024-04-02T15:27:44Z

The new 3.15.0 release introduced a bug with validation unique constraint.

Code to reproduce an error:

from django.db import models
from rest_framework import serializers

class Pet(models.Model):
    name = models.CharField(max_length=100)
    animal_type = models.CharField(max_length=100)
    can_fly = models.BooleanField(null=True)

    class Meta:
        constraints = (
            UniqueConstraint(
                fields=["name", "animal_type"],
                name="unique_pet",
                condition=Q(can_fly__isnull=True),
            ),
        )


class PetSerializer(serializers.ModelSerializer):
    class Meta:
        model = Pet
        fields = ('name', 'animal_type', 'can_fly')


Pet.objects.create(animal_type='dog', name='Fluffy', can_fly=None)
serializer = PetSerializer(data={
    'can_fly': False,
    'animal_type': 'dog',
    'name': 'Fluffy'
})
serializer.is_valid(raise_exception=True)

The last line raises the error:

Error
Traceback (most recent call last):
    rest_framework.exceptions.ValidationError: {
        'non_field_errors': [ErrorDetail(string='The fields name, animal_type must make a unique set.', code='unique')]
    }

Validation ignores that can_fly field is present in the serializer.initial_data and just runs UniqueTogetherValidator for animal_type and name fields.

The text was updated successfully, but these errors were encountered:

AGarrow · 2024-04-22T19:50:09Z

bump on this 😄 , causing some issues for us in prod after upgrading 😅

terjekv · 2024-05-27T06:59:09Z

This looks similar to what we're seeing here: unioslo/mreg#537, except that we're seeing a RelatedObjectDoesNotExist exception:

ERROR    django.request:log.py:241 Internal Server Error: /api/v1/hosts/
Traceback (most recent call last):
  File ".../projects/uio/mreg/env/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File ".../projects/uio/mreg/env/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../projects/uio/mreg/env/lib/python3.11/site-packages/django/views/decorators/csrf.py", line 65, in _view_wrapper
    return view_func(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../projects/uio/mreg/env/lib/python3.11/site-packages/django/views/generic/base.py", line 104, in view
    return self.dispatch(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../projects/uio/mreg/env/lib/python3.11/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../projects/uio/mreg/env/lib/python3.11/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File ".../projects/uio/mreg/env/lib/python3.11/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File ".../projects/uio/mreg/env/lib/python3.11/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../projects/uio/mreg/mreg/api/v1/views.py", line 354, in post
    if ipserializer.is_valid():
       ^^^^^^^^^^^^^^^^^^^^^^^
  File ".../projects/uio/mreg/env/lib/python3.11/site-packages/rest_framework/serializers.py", line 223, in is_valid
    self._validated_data = self.run_validation(self.initial_data)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../projects/uio/mreg/env/lib/python3.11/site-packages/rest_framework/serializers.py", line 444, in run_validation
    self.run_validators(value)
  File ".../projects/uio/mreg/env/lib/python3.11/site-packages/rest_framework/serializers.py", line 477, in run_validators
    super().run_validators(to_validate)
  File ".../projects/uio/mreg/env/lib/python3.11/site-packages/rest_framework/fields.py", line 553, in run_validators
    validator(value, self)
  File ".../projects/uio/mreg/env/lib/python3.11/site-packages/rest_framework/validators.py", line 169, in __call__
    checked_values = [
                     ^
  File ".../projects/uio/mreg/env/lib/python3.11/site-packages/rest_framework/validators.py", line 172, in <listcomp>
    if field in self.fields and value != getattr(serializer.instance, field)
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../projects/uio/mreg/env/lib/python3.11/site-packages/django/db/models/fields/related_descriptors.py", line 264, in __get__
    raise self.RelatedObjectDoesNotExist(
mreg.models.host.Ipaddress.host.RelatedObjectDoesNotExist: Ipaddress has no host.

The issue comes from this bit of code: https://github.com/unioslo/mreg/blob/fa6ca20a41bd486cc8053a116f412b1d526a72ef/mreg/api/v1/views.py#L345-L351 combined with the unique_together constraint in Ipaddress: https://github.com/unioslo/mreg/blob/fa6ca20a41bd486cc8053a116f412b1d526a72ef/mreg/models/host.py#L25-L36.

This (very old) code works fine if we use 3.14, but running 3.15.* gives the above.

Django 5 support. * Add python 3.12 to github actions. * Drop python 3.7 support due to djangorestframework and django-auth-ldap having dropped support. * Drop Django 3 support due to django-filter 24.* needing 4.2+. * Add tzdada dependency (Needed for docker unit tests) * Clean up implementation of manually finding conflicts from requests via get_object_from_request. * "Fix" and test filtering for HostFilterSet. :( Note: This relies on django-rest-framework 3.14.0 and not 3.15.1 (which has formal Django 5 support)... This is due to changes in 3.15.* with regards to unique_together in models. For us, that hits Ipaddress when creating a host, leading to, where we an error as follows (this is reported as part of encode/django-rest-framework#9358). ```python ERROR django.request:log.py:241 Internal Server Error: /api/v1/hosts/ Traceback (most recent call last): File ".../projects/uio/mreg/env/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner response = get_response(request) ^^^^^^^^^^^^^^^^^^^^^ File ".../projects/uio/mreg/env/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".../projects/uio/mreg/env/lib/python3.11/site-packages/django/views/decorators/csrf.py", line 65, in _view_wrapper return view_func(request, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".../projects/uio/mreg/env/lib/python3.11/site-packages/django/views/generic/base.py", line 104, in view return self.dispatch(request, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".../projects/uio/mreg/env/lib/python3.11/site-packages/rest_framework/views.py", line 509, in dispatch response = self.handle_exception(exc) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".../projects/uio/mreg/env/lib/python3.11/site-packages/rest_framework/views.py", line 469, in handle_exception self.raise_uncaught_exception(exc) File ".../projects/uio/mreg/env/lib/python3.11/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception raise exc File ".../projects/uio/mreg/env/lib/python3.11/site-packages/rest_framework/views.py", line 506, in dispatch response = handler(request, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".../projects/uio/mreg/mreg/api/v1/views.py", line 354, in post if ipserializer.is_valid(): ^^^^^^^^^^^^^^^^^^^^^^^ File ".../projects/uio/mreg/env/lib/python3.11/site-packages/rest_framework/serializers.py", line 223, in is_valid self._validated_data = self.run_validation(self.initial_data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".../projects/uio/mreg/env/lib/python3.11/site-packages/rest_framework/serializers.py", line 444, in run_validation self.run_validators(value) File ".../projects/uio/mreg/env/lib/python3.11/site-packages/rest_framework/serializers.py", line 477, in run_validators super().run_validators(to_validate) File ".../projects/uio/mreg/env/lib/python3.11/site-packages/rest_framework/fields.py", line 553, in run_validators validator(value, self) File ".../projects/uio/mreg/env/lib/python3.11/site-packages/rest_framework/validators.py", line 169, in __call__ checked_values = [ ^ File ".../projects/uio/mreg/env/lib/python3.11/site-packages/rest_framework/validators.py", line 172, in <listcomp> if field in self.fields and value != getattr(serializer.instance, field) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".../projects/uio/mreg/env/lib/python3.11/site-packages/django/db/models/fields/related_descriptors.py", line 264, in __get__ raise self.RelatedObjectDoesNotExist( mreg.models.host.Ipaddress.host.RelatedObjectDoesNotExist: Ipaddress has no host. ```

dfn-certling · 2024-06-12T15:18:47Z

We see the same issue as @terjekv . This is due to #9154 . Before you could create a bare model of the serializer's instance class in memory and provide it to the serializer, resulting in a model stored to the DB. Now the UniqueTogetherValidator tries to check for changes in the unique values in the provided instance that has never been stored to the DB and does not have the corresponding fields initialized.

- We are stuck on drf 3.14.0 due to encode/django-rest-framework#9358 until encode/django-rest-framework#9483 goes into prod, hopefully 3.15.3.

* Add query string to logging output. * Type checking fix (type->isinstance) * Fix filters for Mreg (HostPolicy outstanding). * Safer version of JSON filter to avoid SQL injections. * Refactor, cleanup. * Hostpolicy filter "fixes", also bump dependencies. - We are stuck on drf 3.14.0 due to encode/django-rest-framework#9358 until encode/django-rest-framework#9483 goes into prod, hopefully 3.15.3. * Skeleton for testing filters. * Add iexact support, add cases. Fix toml. * Move to ruff formater. * More tests. - Add ip support. - Add reverse lookups. * Hostpolicy filter tests. - Also reformat as per ruff. * Add a test for filtering on ?id= for hosts. - This should catch the generic issue of filtering on IDs. Ideally we'd do this for every model that supports ID... * support `__in`. * Support CIDR matching. - Match exact CIDR or IP within a CIDR.

eddielu · 2024-10-02T03:05:09Z

Is this a recent change? When did DRF start adding UniqueTogetherValidator for ModelSerializer?

Edit: I know it's broken now, but I'm wondering if this feature was always there or just added recently

rogelho-junior-byx · 2024-10-06T04:45:25Z

Is this a recent change? When did DRF start adding UniqueTogetherValidator for ModelSerializer?

Edit: I know it's broken now, but I'm wondering if this feature was always there or just added recently

Has been added on DRF 3.15.0 (15th March 2024)

kalekseev linked a pull request Apr 3, 2024 that will close this issue

Fix unique together validator doesn't respect condition's fields #9360

Open

shamoon mentioned this issue Apr 12, 2024

Chore(deps): Bump the small-changes group with 10 updates paperless-ngx/paperless-ngx#6382

Closed

terjekv mentioned this issue May 27, 2024

Django 5 support. unioslo/mreg#537

Merged

shamoon mentioned this issue Jun 30, 2024

Chore: upgrade to DRF 3.15 paperless-ngx/paperless-ngx#7134

Merged

12 tasks

B4dM4n mentioned this issue Jul 22, 2024

authentik: Fix runtime errors NixOS/nixpkgs#329151

Merged

13 tasks

max-muoto mentioned this issue Jul 31, 2024

Revert Unique Together Checking Regression #9483

Closed

terjekv added a commit to terjekv/mreg that referenced this issue Aug 7, 2024

Hostpolicy filter "fixes", also bump dependencies.

6757d93

- We are stuck on drf 3.14.0 due to encode/django-rest-framework#9358 until encode/django-rest-framework#9483 goes into prod, hopefully 3.15.3.

terjekv mentioned this issue Aug 7, 2024

Further filter fixes. unioslo/mreg#547

Merged

glennmatthews mentioned this issue Sep 9, 2024

[2.3] Front Ports API schema incorrectly labels rear_port_position field as not required nautobot/nautobot#6100

Open

peterthomassen mentioned this issue Sep 10, 2024

3.15.2 includes two undocumented breaking changes #9450

Open

browniebroke mentioned this issue Sep 12, 2024

3.15 is raising required error on model nullable fields #9378

Open

browniebroke mentioned this issue Oct 14, 2024

Fix raising on nullable fields part of UniqueConstraint #9531

Open

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

3.15 regression: Serializer validation failed for unique together constraint #9358

3.15 regression: Serializer validation failed for unique together constraint #9358

anndoc commented Apr 2, 2024

AGarrow commented Apr 22, 2024

terjekv commented May 27, 2024

dfn-certling commented Jun 12, 2024

eddielu commented Oct 2, 2024 •

edited

Loading

rogelho-junior-byx commented Oct 6, 2024

3.15 regression: Serializer validation failed for unique together constraint #9358

3.15 regression: Serializer validation failed for unique together constraint #9358

Comments

anndoc commented Apr 2, 2024

AGarrow commented Apr 22, 2024

terjekv commented May 27, 2024

dfn-certling commented Jun 12, 2024

eddielu commented Oct 2, 2024 • edited Loading

rogelho-junior-byx commented Oct 6, 2024

eddielu commented Oct 2, 2024 •

edited

Loading