Dehao Yuan , Furong Huang , Cornelia Fermüller , Yiannis Aloimonos
ICLR2024 Poster [Paper] [Poster]
We propose Hyper-Dimensional Function Encoding (HDFE). Left: Given samples of a continuous object (e.g. a function), HDFE encodes the object into a fixed-length vector without any training. The encoding is not affected by the distribution and size with which the object is sampled. The encoding can be decoded to reconstruct the continuous object. Right: Applications of HDFE. HDFE can be used to perform machine learning tasks (e.g. classification, regression) on continuous objects. HDFE also enables neural networks to regress continuous objects by predicting their encodings.
HDFE.py contains all the functionalities of HDFE. Feel free to explore them with the following example script:
python example.py
If you want to use HDFE to encode your own function, please prepare your function samples as a pytorch.Tensor of shape X: (num_samples, input_dim) and Y: (num_samples, ) and pass it to the FPE_Encoder or BVE_Encoder. Please make sure your X is normalized and Y is scaled between (0,1). The encoding may require some tuning on the alpha parameter. If you find the code does not produce a nice encoding, it is quite likely that 1. the function samples are not scaled properly; 2. the alpha is not chosen properly; 3. the dim is not chosen properly. Suppose you find any difficulty reproducing the results or making it work in your application, feel free to contact me.
Reconstruction of Function Encoding: Under a suitable selection of receptive field and dimensionality, HDFE can produce a decodable encoding of the original function. Left: suitable receptive field and suitable dimension. Mid: too large receptive field and suitable dimension. Right: suitable receptive field and too small dimension.
Binary Vector Implementation of HDFE: In addition to the fractional power encoding introduced in the paper, we also implement a binary vector encoding version of HDFE. It runs faster and yields comparable performance as the FPE encoding. Below shows the reconstruction of the binary vector implementation of HDFE. It has not been optimized for the memory consumption and takes a lot of GPU memory. However, it is not difficult to do that optimization from the methodology perspective. It just requires a lot of engineering, so I haven't done it yet. If you are interested in using this architecture in some real-time applications, and require that implementation. Feel free to contact me.
Isometry Property: In the paper we show that HDFE preserves the function distance. We have an experiment to demonstrate the property.
Receptive Field of HDFE: We provide a functionality to check and visualize the receptive field of HDFE.
We also provide the source code for reproducing the experiments in the paper. Please checkout the corresponding directories below.
To solve PDEs using neural networks, we first encode the PDE and its solution into their vector embeddings using HDFE. Then, we train a multi-layer perceptron to map the embedding of the PDE to the embedding of its solution. The optimization target is the cosine similarity between the predicted embedding and the true embedding. Since the embeddings are complex vectors, we adopt a Deep Complex Network as the architecture of the multi-layer perceptron.
We compare the vanilla HDFE with the PCPNet, which is a vanilla PointNet architecture. We replace the PointNet with our HDFE appended with a deep complex network. Such replacement yields significantly higher performance.
We integrate HDFE into the HSurf-Net, which is the state-of-the-art PointNet-based normal estimator. Such integration yields significantly higher performance.
Point cloud normal estimation error visualization.
If you find this paper useful, please consider citing:
@misc{yuan2023decodable,
title={Decodable and Sample Invariant Continuous Object Encoder},
author={Dehao Yuan and Furong Huang and Cornelia Fermüller and Yiannis Aloimonos},
year={2023},
eprint={2311.00187},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
If you have any question or have any issue running the codes, feel free to contact me at dhyuan@umd.edu.