Support configure tag when marshal/unmarshal

[surethink]

For example, I have a struct as follows， I want the id field to be omitted in json when writing http response， so I set tag json:"-", it's ok。 But when I want to marshal the same struct to redis (or unmarshal from redis) , I want the id field not to be omitted .So would you please allow developers to decide whether to ignore tag when marshal/unmarshal

type Example struct {
Id int64 json:"-"
IdStr string json:"id"
}

[liuq19]

There are some ways to do it. For example, use a mock type, and change - to other tags if you doest not want to ignore it.

struct Foo{
Id int64 json:"-"
}

struct FooMock{
Id int64 json:"id"
}

[surethink]

Yes, it works.but if there are a lot of such structs. mocking them would not be very elegant.

to the same struct, In some scenario i want them to be omitted, and others not :)

Maybe my request can be expressed as a more generic one:

support setting more than one tag key, such as. json:"id",other:"id2"
when marshal/unmarshal , allow developers to use different tag

then, I can use a tagKey not exists, the process will fallback to no tag

[liuq19]

this case is not generic, you can use a marshaler/unmarshaler interface.

[acynothia]

golang/go#45812 It's really useful feature in GraphQL or FieldMask.

[liuq19]

golang/go#45812 It's really useful feature in GraphQL or FieldMask.

Thank you for sharing, we will survey it~

[surethink]

In jsoniter, I can appoint specific tagKey to marshal, like this:
jsoniter.Config{TagKey: "redis"}.Froze().Marshal(v)

If this feature is supported by sonic, I can remove jsoniter totally. thanks
FYI: https://github.com/json-iterator/go/blob/master/config.go

[liuq19]

thanks, we will discuss it

[AsterDY]

Since sonic's JIT assumes field tag is immutable and complies the field key into assembly (https://github.com/bytedance/sonic/blob/main/internal/encoder/compiler.go#L738), I don't think it is worth supporting this config. The simplest way is compiling and caching different JIT-produced codes according to this config, but it will cost more memory and CPU in runtime

[ii64]

I think this can be done in resolver package. Currently we are using Go's encoding/json.typeFields to read JSON tag from field tags and it's by default looking up for sf.Tag.Get("json").

Higher level the JSON key lookup is like: make{Encoder,Decoder} -> compiler -> compileStruct -> compileStructBody -> ResolveStruct. If we want to have a configurable tag name/selector, then we can have our implementation of encoding/json.typeFields.

EDIT 0: But then we'll need to have different cache storage for them.
EDIT 1: I agree that we'll end up having to recompile the bytecode, resulting more memory usage for storing the machine code.

Ref: https://go.googlesource.com/go/+/go1.20.4/src/encoding/json/encode.go#1257
Ref: internal/resolver/resolver.go#L114
Ref: internal/resolver/stubs.go#L45

[liuq19]

Thank you. I think we can create a fieldmap that contains the mapping of each field's JSON tag name to its field name and its corresponding quoted field name when resolving the struct. Then, we can pass the tag name as a parameter to the encoder and decoder so that we don't need to compile and cache different machine code for different tags.

To make it more convenient, we can store the tag on a fixed location on the stack, and select the appropriate fieldmap during decoding based on the tag. For encoding, we can also select the quoted field name constant to copy based on the tag.

Chinese:
谢谢。我认为可以将struct resolve之后得到不同tag的fieldmap，fieldmap 里面同时存放 fieldname以及quoted fieldname。然后将tag当做decode/encode 的参数传入，这样就不用为不同tag编译和缓存不同的机器码。

为了方便，我们可以将tag 存放到 stack上面的固定位置，然后decode 时根据tag 来选择fieldmap，encode时根据 tag 来选择需要拷贝的 quoted fieldname 常量。

[AsterDY]

Thank you. I think we can create a fieldmap that contains the mapping of each field's JSON tag name to its field name and its corresponding quoted field name when resolving the struct. Then, we can pass the tag name as a parameter to the encoder and decoder so that we don't need to compile and cache different machine code for different tags.

To make it more convenient, we can store the tag on a fixed location on the stack, and select the appropriate fieldmap during decoding based on the tag. For encoding, we can also select the quoted field name constant to copy based on the tag.

Chinese: 谢谢。我认为可以将struct resolve之后得到不同tag的fieldmap，fieldmap 里面同时存放 fieldname以及quoted fieldname。然后将tag当做decode/encode 的参数传入，这样就不用为不同tag编译和缓存不同的机器码。

为了方便，我们可以将tag 存放到 stack上面的固定位置，然后decode 时根据tag 来选择fieldmap，encode时根据 tag 来选择需要拷贝的 quoted fieldname 常量。

this will make encoder a little bit slower, since query field name need at least O(1) calculation

[surethink]

Thank you. I think we can create a fieldmap that contains the mapping of each field's JSON tag name to its field name and its corresponding quoted field name when resolving the struct. Then, we can pass the tag name as a parameter to the encoder and decoder so that we don't need to compile and cache different machine code for different tags.

To make it more convenient, we can store the tag on a fixed location on the stack, and select the appropriate fieldmap during decoding based on the tag. For encoding, we can also select the quoted field name constant to copy based on the tag.

Chinese: 谢谢。我认为可以将struct resolve之后得到不同tag的fieldmap，fieldmap 里面同时存放 fieldname以及quoted fieldname。然后将tag当做decode/encode 的参数传入，这样就不用为不同tag编译和缓存不同的机器码。

为了方便，我们可以将tag 存放到 stack上面的固定位置，然后decode 时根据tag 来选择fieldmap，encode时根据 tag 来选择需要拷贝的 quoted fieldname 常量。

any update on this? any hero can make it done?

[liuq19]

Thanks. We will change lots of code and don't have enough time to do it now. Maybe support in the future. cc @AsterDY

[surethink]

Thanks. We will change lots of code and don't have enough time to do it now. Maybe support in the future. cc @AsterDY

Got it. Hope it will be done as soon as possible. it's not easy to find such an excellent library, but unfortunately, I can only watch it with nothing to do :)

[surethink]

Thank you. I think we can create a fieldmap that contains the mapping of each field's JSON tag name to its field name and its corresponding quoted field name when resolving the struct. Then, we can pass the tag name as a parameter to the encoder and decoder so that we don't need to compile and cache different machine code for different tags.
To make it more convenient, we can store the tag on a fixed location on the stack, and select the appropriate fieldmap during decoding based on the tag. For encoding, we can also select the quoted field name constant to copy based on the tag.
Chinese: 谢谢。我认为可以将struct resolve之后得到不同tag的fieldmap，fieldmap 里面同时存放 fieldname以及quoted fieldname。然后将tag当做decode/encode 的参数传入，这样就不用为不同tag编译和缓存不同的机器码。
为了方便，我们可以将tag 存放到 stack上面的固定位置，然后decode 时根据tag 来选择fieldmap，encode时根据 tag 来选择需要拷贝的 quoted fieldname 常量。

this will make encoder a little bit slower, since query field name need at least O(1) calculation

If there is only one tagKey or no tagKey, keep the current solution ; If there is more than one tagKey, then use the tag field map solution. Only the user who uses more tagKeys pay the cost

does this feasible?