Optimize Text embedding error: can't optimize a non-leaf Tensor

[hizuka590]

Hi author

Thank you for releasing your amazing work.When I try to retrain your model, I find the original source code perhaps got something wrong and failed to optimize text embedding because of the ''can't optimize a non-leaf tensor'' error.

could you help check out what goes wrong?

Many thanks

[hizuka590]

After tracing is_leaf and requires_grad propserty of your code, I believe this nonleaf error is caused by: 'embeddings_to_optimize' is a slice of the original 'text_embedding'.