Question: For some models, only categrical inputs passed on. Why?

[gladomat]

I was wondering, when I set up a model, for example a Wide and Deep model with an AutoInt Net and a CIN Net, only the Wide and Deep model gets fed with numerical variables, whereas the others only get the embedded categorical variables. See the attached figure. Why is that? wouldn't that mean, that interaction values between numerical variables get ignored? The other problem is, some numerical variables I feed into the model were originally categorical variables of differing lengths, that have been embedded through a different method. These also must exhibit interaction effects in a non-linear fashion. I thus find it a pity to not be able to take advantage of this extra information. Or I have a completely wrong understanding of the underlying structure.

I certainly would appreciate an explanation. Thank you!

[diff7]

From AutoInt

To allow the interaction between categorical and numerical features, we also represent the numerical features in the same low-dimensional feature space. Specifically, we represent the numerical feature an embedding vector.

https://arxiv.org/pdf/1810.11921v2.pdf

But I did not find in the code where real values are embedded.

[gladomat]

I'm not really seeing any embedding of the continuous features. They also don't enter the CIN or the AutoInt models.

[gladomat]

I'm also interested in computing explanations using integrated gradients working (Here's a nice Keras tutorial). Unfortunately, the embedding layers are not differentiable and gradient calculations are then impossible. A solution may be that I embed the categorical variables externally and feed the embeddings to the model. However, if only the categorical variables are passed on to compute higher order interactions, then passing the already embedded categorical variables as continuous variables will result in no interaction calculation. Correct me if I'm wrong.