orml-style-transfer

orml-style-transfer

Encodes style from one image and transfers style to another image.

What can I do with it?

One can capture style from an image (style image) and encode it into a numerical representation (a style vector). This style can be applied –or transferred– to another image (content image).

Style vectors can be transformed, which may lead to interesting new styles.

How do I use it?

orml-style-transfer has two main components: StyleEncoder for the encoding of styles in an image, and StyleTransformer to transfer the encoded style to an image.

Using StyleEncoder

Load the encoder once

val encoder = StyleEncoder.load()

Encode a style image into a style vector

val styleVector: FloatArray = encoder.encodeStyle(styleImage)

Note that styleVector is a FloatArray which values can easily be changed. For example to blend between two style vectors one can

Using StyleTransformer

StyleTransformer comes in two tastes, an accurate one and a faster one.

To load the accurate version:

val transformer = StyleTransformer.load() 

To load the faster version:

val transformer = StyleTransformer.loadSeparable() 

To transfer style:

val transformed = transformer.transformStyle(contentImage, styleVector)

Result

Content	Style	Result

Blending style vectors

One can make blends between two style vectors to create new styles.

Consider two style vectors produced by the encoder:

val styleVector0 = encoder.encodeStyle(styleImage0)
val styleVector1 = encoder.encodeStyle(styleImage1)

We can blend them as follows:

val blendFactor = 0.5f
val styleVector = (styleVector0 zip styleVector1).map {
    it.first * blendFactor + it.second * (1.0f - blendFactor)
}.toFloatArray()

Then we use styleVector in the transformer like we'd use any style vector.

See BlendST01.kt for a demonstration of style blending.

Example work

• Collager project by @voorbeeld (Twitter)

Credits and references

Based on:

• https://github.com/reiinakano/arbitrary-image-stylization-tfjs
• https://github.com/magenta/magenta/tree/master/magenta/models/arbitrary_image_stylization

Exploring the structure of a real-time, arbitrary neural artistic stylization network. Golnaz Ghiasi, Honglak Lee, Manjunath Kudlur, Vincent Dumoulin, Jonathon Shlens, Proceedings of the British Machine Vision Conference (BMVC), 2017.