Why Does (De)Serialization Fails When JsonConverter.CanConvert(Type) returns false?

[roy-cohen273]

When applying [JsonConverter(typeof(MyJsonConverter))] to a type, (de)serialization fails if MyJsonConverter.CanConvert returns false. Why dosn't the (de)serialization continue on as if MyJsonConverter didn't exist, i.e. use the default (de)serialization?

Example:

class MyJsonConverter : JsonConverter<object> {
    public override bool CanConvert(Type typeToConvert) {
        return false;
    }

    public override object Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) {
        throw new NotImplementedException();
    }

    public override void Write(Utf8JsonWriter writer, object value, JsonSerializerOptions options) {
        throw new NotImplementedException();
    }
}

[JsonConverter(typeof(MyJsonConverter))]
public class SimplePoco {
    public string Name { get; set; }
}

class Program {
    static void Main(string[] args) {
        var obj = new SimplePoco { Name = "<some name>" };
        Console.WriteLine(JsonSerializer.Serialize(obj));
    }
}

I would like the output of this program to be {"Name": "<some name>"} but instead I get an error: Unhandled exception. System.InvalidOperationException: The converter specified on 'JsonOptionConverter.SimplePoco' is not compatible with the type 'JsonOptionConverter.SimplePoco'..
Why is this like that? Why not ignore a converter if that converter's CanConvert method returns false? To avoid changing code that's already written, this behaviour can be activated only when a property on the JsonConverter is true (like the HandleNull property).

I've looked at the source code and I don't think that would be hard to implement (if people even want this). I think all it takes is adding public virtual bool Optional { get; } = false; in JsonConverter, and changing a little bit the private function JsonSerializerOptions.GetConverterFromAttribute in JsonSerializerOptions.Converters.cs line 321:

if (!converter.CanConvert(typeToConvert))
{
    Type? underlyingType = Nullable.GetUnderlyingType(typeToConvert);
    if (underlyingType != null && converter.CanConvert(underlyingType))
    {
        if (converter is JsonConverterFactory converterFactory)
        {
            converter = converterFactory.GetConverterInternal(underlyingType, this);
        }

        // Allow nullable handling to forward to the underlying type's converter.
        return NullableConverterFactory.CreateValueConverter(underlyingType, converter);
    }
    // <code that I added>
    if (converter.Optional) return null;
    // </code that I added>
    ThrowHelper.ThrowInvalidOperationException_SerializationConverterOnAttributeNotCompatible(classTypeAttributeIsOn, memberInfo, typeToConvert);
}

This is my first time starting a discussion so please tell me if I'm doing something wrong.
Should this have been filed as an issue instead?

Edit: I added more elaboration about specific code changes.

Edit: After some more reading I noticied such a change will fall under bucket 2 of breaking changes:

A new instance field is added to a type (impacts serialization)

I don't see a reason why you would serialize a JsonConverter but I thought I mentioned it anyway.

[huoyaoyuan]

If you annotate the converter on a type or a member, the it's intended to support the thing it's annotated to.
Selecting converter based on CanConvert should use JsonSerializerOptions.Converters.

[huoyaoyuan]

Applying of converter is decided statically. The CanConvert always gets the type or type of property it's annotated on, not the GetType result of the actual object. It has no different from simple removing the JsonConverterAttribute annotation.

[roy-cohen273]

It is different from simply removing the annotation, because annotations are decided statically and CanConvert is in the run time. It's impossible to remove an attribute after the program starts running so I think it would be useful if there was a way to 'disable' a converter applied using JsonConverterAttribute, I figured returning false from CanConvert would be the simplest way to do that.

[huoyaoyuan]

CanConvert is assumed to be unchanged and will be called just once. Changing it's return value may be easily broken with the assumption. It's quite fragile.

Inheriting from JsonConverterAttribute and override CreateConverter is more actionable.

[roy-cohen273]

I don't think CanConvert is only called once, I tested it a few days ago and and it's called for every (de)serialization.

Regarding inhereting from JsonConverterAttribute and overriding CreateConverter, it is still imposible to disable the converter like that because you must return a JsonConverter from CreateConverter (returning null raises an exception).
This is what I thought of when I suggested to add a field optional to JsonConverterAttribute, but I later realized that the second suggestion (adding a virtual property Optional to JsonConverter) is more flexible and if a subclass of JsonConverterAttribute would llike to disable itself it can return a dummy JsonConverter who's CanConvert method always returns false (and Optional property always set to true).

[roy-cohen273]

After checking again, I concluded that you were right, CanConvert is only called once, I assume because some sort of caching.
I think now that the only solution to my issue is some sort af addition to the API in a form of a JsonConverter<T> or a JsonConverterFactory that will act like the normal (de)serialization, i.e. ignore all custom converters. But that's a seperate issue.
See this SO question.

What should I do with this discussion, should I delete it?

fiddle I used for testing: https://dotnetfiddle.net/MMHgWk