From 749e8703fe10443f97db1518b1ad3d8b76af8b07 Mon Sep 17 00:00:00 2001 From: Alejandro Serrano Date: Mon, 1 Jul 2024 15:44:02 +0200 Subject: [PATCH 01/19] Expose boxed inline value classes --- proposals/jvm-expose-boxed.md | 195 ++++++++++++++++++++++++++++++++++ 1 file changed, 195 insertions(+) create mode 100644 proposals/jvm-expose-boxed.md diff --git a/proposals/jvm-expose-boxed.md b/proposals/jvm-expose-boxed.md new file mode 100644 index 000000000..5e764a6e7 --- /dev/null +++ b/proposals/jvm-expose-boxed.md @@ -0,0 +1,195 @@ +# Expose boxed inline value classes + +* **Type**: Design proposal +* **Author**: Alejandro Serrano +* **Contributors**: Mikhail Zarechenskii, Ilmir Usmanov +* **Discussion**: ?? +* **Status**: ?? +* **Related YouTrack issue**: [KT-28135](https://youtrack.jetbrains.com/issue/KT-28135/Add-a-mechanism-to-expose-members-of-inline-classes-and-methods-that-use-them-for-Java) +* **Previous proposal**: [by @ommandertvis](https://github.com/Kotlin/KEEP/blob/commandertvis/jvmexpose/proposals/jvm-expose-boxed-annotation.md) + +## Abstract + +We propose modifications to how value classes are exposed in JVM, with the goal of easier consumption from Java. This includes exposing the constructor publicly, and giving the ability to exposed boxed variants of operations. + +## Table of contents + +* [Abstract](#abstract) +* [Table of contents](#table-of-contents) +* [Motivation](#motivation) + * [Design goals](#design-goals) +* [Expose boxed constructors](#expose-boxed-constructors) + * [No argument constructors](#no-argument-constructors) + * [Other design choices](#other-design-choices) +* [Expose operations and members](#expose-operations-and-members) + * [Other design choices](#other-design-choices-1) +* [Problems with reflection](#problems-with-reflection) + * [Other design choices](#other-design-choices-2) + +## Motivation + +[Value (or inline) classes](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md) are an important tool to create a _lightweight wrapper_ over another class. One important example are classes which introduce additional invariants over another one, like an integer which is always greater than zero. + +```kotlin +@JvmInline value class PositiveInt(val number: Int) { + init { require(number >= 0) } +} +``` + +In order to keep its lightweight nature, the Kotlin compiler tries to use the _unboxed_ variant -- in the case above, an integer itself -- instead of its _boxed_ variant -- the `PositiveInt` class -- whenever possible. However, in order to prevent clashes between different overloads, the name of those functions is _mangled_; those functions operate directly on the underlying unboxed representation, with additional checks coming from the `init` block. + +```kotlin +fun PositiveInt.add(other: PositiveInt): PositiveInt = + PositiveInt(this.number + other.number) + +// is compiled down to code similar to +fun Int.add-1bc5(other: Int): Int = + PositiveInt.check-invariants(this.number + other.number) +``` + +Note: we use `PositiveInt.check-invariants` for illustrative purposes, the [inline classes](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md) defines the concrete compilation scheme that the compiler ought to follow. + +The main drawback of this compilation scheme is that a type like `PositiveInt` cannot be (easily) consumed from Java (or other JVM languages other than Kotlin). One benefit is that library authors no longer need to choose between using a value class (which is more performant, but only available for Kotlin consumers) or a (data) class (which always requires boxing but can be used anywhere). + +### Design goals + +"Exposing" a `value` class is not a single entity. We have found (at least) four different scenarios that we would like to support: + +1. Creating boxed values: for example, creating an actual instance of `PositiveInt`. +2. Calling operations defined over value classes over their boxed representations. +3. Making inline value classes with Java-idiomatic patterns, especially those using reflection. + +--- + +Apart from the particular problems, we set the following cross-cutting goals for the proposal. + +**Minimize API surface duplication**: if we expose every single member and function taking an value class with the corresponding version using the boxed type, this essentially duplicates the API surface. Developers may want full control over which functions are exposed this way; on the other hand, having to annotate every single function is also tiring. + +**Ensure that invariants are never broken**: especially in the "frontier" between Kotlin and Java, we need to provide an API which does all the corresponding checks. We are OK with introducing some performance penalties when using the API from Java, but no additional penalties should be paide if the class is used in Kotlin. + +**Compatibility with current compilation scheme**: if we alter the way inline classes are currently compiled in a non backward compatible way, we could create a huge split in the community. Or even worse, we will end up supporting two different schemes in the compiler. + +## Expose boxed constructors + +The [current compilation scheme](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md#inline-classes-abi-jvm) exposes the constructor of the boxed class as _synthetic_, which makes it unavailable from Java. We propose to remove that modifier from the constructor, and use the same visibility in the constructor as the one defined in the value class. + +```kotlin +@JvmInline value class PositiveInt(val number: Int) { + init { require(number >= 0) } +} + +// compiles down to +class PositiveInt { + public (Int): void + + // and others +} +``` + +We think this is the right choice because of a couple of reasons: + +1. It does not increase the binary size, as it only exposes a constructor which was already there (albeit synthetic); +2. It is binary compatible with previous users of this code. + +### No argument constructors + +Frameworks like Java Persistence require classes to have a [default no argument constructor](https://www.baeldung.com/jpa-no-argument-constructor-entity-class). This case would be covered by a combination of the proposal above, and the ability to give default values to arguments of a value class. Continuing with our example of positive integers, the following code, + +```kotlin +@JvmInline value class PositiveInt(val number: Int = 0) { + init { require(number >= 0) } +} +``` + +generates the constructor taking a single integer, but also another one without the optional argument. Since value classes may only have one argument, this means that making it optional creates a no argument constructor. + +### Other design choices + +**Put it under a flag**, in other words, only expose the boxed constructor when some annotation or compiler flag is present. In this case, we could not find a realistic scenario where this expose would be counter-productive; in the worst case in which you expose the constructor but not operations you just have a way to create useless values. + +The only drawback is that the ABI exposed in the JVM changes from a version of the compiler to the next, albeit in a binary compatible way. There is a risk involved in downgrading the compiler version, as it may break Java clients, but this scenario is quite rare. + +**Exposing a factory method**: instead of exposing the constructor, use `PositiveInt.of(3)`. This option is nowadays quite idiomatic in the Java world (for example, in `List.of(1, 2)`), but not as much as constructors. Using a factory method also has the drawback of possible clashes for the name of the factory method, which we avoid when using the constructor. + +## Expose operations and members + +The current compilation scheme transforms every operation where a value class is involved into a static function that takes the unboxed variants, and whose named is mangled to prevent clashes. This means those operarions are not available for Java consumers. We propose introducing a new `@JvmExposeBoxed` annotation that exposes a variant of the function taking the boxed versions instead. + +The `@JvmExposeBoxed` annotation may be applied to a declaration, or to a declaration container (classes, files). In the latter case, it should be takes as applied to every single declaration within it. + +If the `@JvmExposeBoxed` annotation is applied to a member of a value class (or the value class itself, or the containing file), then it is compiled to a member function of the corresponding boxed class, not as a static method. + +The name of the declaration must coincide with the name given in the Kotlin code, unless a `@JvmName` annotation is present. In that case, the name defined in the annotation should be used. + +The following is an example of the compilation of some operations over `PositiveInt`. The `box-impl` and `unbox-impl` refer to the operations defined in the [current compilation scheme](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md#inline-classes-abi-jvm) for boxing and unboxing without checks. + +```kotlin +@JvmInline @JvmExposeBoxed +value class PositiveInt(val number: Int) { + init { require(number >= 0) } + + fun add(other: PositiveInt): PositiveInt = ... +} + +@JvmExposeBoxed +fun PositiveInt.duplicate(): PositiveInt = ... + +// compiles down to +class PositiveInt { + public (Int): void + + public add(other: PositiveInt): PositiveInt = + box-impl(add-1df3(unbox-impl(this), unbox-impl(other))) + // mangled version + public static add-1df3($this: Int, other: Int): Int + + // and others +} + +public duplicate($this: PositiveInt): PositiveInt = + box-impl(duplicate-26b4(unbox-impl($this))) +// mangled version +fun duplicate-26b4($this: Int): Int +``` + +### Other design choices + +**Annotate the value class**: in a [previous proposal](https://github.com/Kotlin/KEEP/blob/commandertvis/jvmexpose/proposals/jvm-expose-boxed-annotation.md) the annotation was applied to the value class itself, not to the operations, and would force every user of that class to create the boxed and unboxed versions. We found this approach not flexible enough: it was completely on the hand of the developer controlling the value class to decide whether Java compatibility was important. This opinion may not coincide with that of another author, which may restrict or expand the boxed variants in their own code. + +**Use a compiler flag instead of an annotation**: we have also considered exposing this feature using a compiler flag, `-Xjvm-expose-boxed=C,D,E`, where `C`, `D`, and `E` are the names of the classes for which boxed variants of the operations should be generated. We found two drawbacks for this choice: + +- Users are not accostumed to tweaking compiler flags, which are often hidden in the build file; in contrast to annotations. +- It does not give the flexibility of exposing only a subset of operations. The current proposal allows that scenario if you annotate each operation independently, but also allows `@file:JvmExposeBoxed` if you want a wider stroke. + +## Problems with reflection + +The current proposal does not solve all of the problems that inline value classes have with reflection. Many of them stem from the back that, once compiled to the unboxed variant, there is no trace of the original boxed variant. + +```kotlin +class Person(val name: String, val age: PositiveInt) + +// compiles down to +class Person { + fun getName(): String + fun getAge(): Int // ! not PositiveInt +} +``` + +In the case above, libraries may unadvertedly break some of the requirements of `PositiveInt` if a `Person` instance is created using reflection. + +We are currently investigating the required additional support. Some Java serialization libraries, like [Jackson](https://github.com/FasterXML/jackson-module-kotlin/blob/master/docs/value-class-support.md) already include specialized support for value classes. At this point, collaboration with the maintainers of the main libraries in the Java ecosystem seems like the most productive route. + +### Other design choices + +**Mark usages of unboxed classes**: we have investigated the possibility of annotating every place where an unboxed variant is used with a reference to the boxed variant. + +```kotlin +// compiles down to +class Person { + fun getName(): String + fun getAge(): @JvmBoxed(PositiveInt::class) Int +} +``` + +**Validation**: building on the previous idea, `@JvmBoxed` could be implemented as a [Bean validator](https://beanvalidation.org/), so frameworks like Hibernate would automatically execute the corresponding checks. Alas, the Bean Validation specification is a huge mess right now. By Java 7 it was part of the standard distribution in the `javax.validation` package, but now it's moved to `jakarta.validation` (this is in fact the [main change](https://beanvalidation.org/3.0/) between versions 2 and 3 of the specification). It is impossible to provide good compatibility with many Java versions on those grounds. + From aa608b74090328b12fe8df2aaf5e0b70241bf260 Mon Sep 17 00:00:00 2001 From: Ilmir Usmanov Date: Tue, 2 Jul 2024 15:38:39 +0000 Subject: [PATCH 02/19] fixup! Expose boxed inline value classes --- proposals/jvm-expose-boxed.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/jvm-expose-boxed.md b/proposals/jvm-expose-boxed.md index 5e764a6e7..3bfcfc6df 100644 --- a/proposals/jvm-expose-boxed.md +++ b/proposals/jvm-expose-boxed.md @@ -6,7 +6,7 @@ * **Discussion**: ?? * **Status**: ?? * **Related YouTrack issue**: [KT-28135](https://youtrack.jetbrains.com/issue/KT-28135/Add-a-mechanism-to-expose-members-of-inline-classes-and-methods-that-use-them-for-Java) -* **Previous proposal**: [by @ommandertvis](https://github.com/Kotlin/KEEP/blob/commandertvis/jvmexpose/proposals/jvm-expose-boxed-annotation.md) +* **Previous proposal**: [by @commandertvis](https://github.com/Kotlin/KEEP/blob/commandertvis/jvmexpose/proposals/jvm-expose-boxed-annotation.md) ## Abstract From 1d3de33be20053a73d418119218dab68eeae854c Mon Sep 17 00:00:00 2001 From: Alejandro Serrano Date: Wed, 3 Jul 2024 10:52:15 +0200 Subject: [PATCH 03/19] Mention serialization and Scala --- proposals/jvm-expose-boxed.md | 65 ++++++++++++++++++++++++----------- 1 file changed, 45 insertions(+), 20 deletions(-) diff --git a/proposals/jvm-expose-boxed.md b/proposals/jvm-expose-boxed.md index 3bfcfc6df..f4314627c 100644 --- a/proposals/jvm-expose-boxed.md +++ b/proposals/jvm-expose-boxed.md @@ -17,14 +17,16 @@ We propose modifications to how value classes are exposed in JVM, with the goal * [Abstract](#abstract) * [Table of contents](#table-of-contents) * [Motivation](#motivation) - * [Design goals](#design-goals) + * [Design goals](#design-goals) * [Expose boxed constructors](#expose-boxed-constructors) - * [No argument constructors](#no-argument-constructors) - * [Other design choices](#other-design-choices) + * [Serialization](#serialization) + * [No argument constructors](#no-argument-constructors) + * [Other design choices](#other-design-choices) * [Expose operations and members](#expose-operations-and-members) - * [Other design choices](#other-design-choices-1) -* [Problems with reflection](#problems-with-reflection) - * [Other design choices](#other-design-choices-2) + * [Other design choices](#other-design-choices-1) +* [Further problems with reflection](#further-problems-with-reflection) + * [Other design choices](#other-design-choices-2) +* [Comparison with Scala](#comparison-with-scala) ## Motivation @@ -63,9 +65,9 @@ The main drawback of this compilation scheme is that a type like `PositiveInt` c Apart from the particular problems, we set the following cross-cutting goals for the proposal. -**Minimize API surface duplication**: if we expose every single member and function taking an value class with the corresponding version using the boxed type, this essentially duplicates the API surface. Developers may want full control over which functions are exposed this way; on the other hand, having to annotate every single function is also tiring. +**Minimize API surface duplication**: if we expose every single member and function taking a value class with the corresponding version using the boxed type, this essentially duplicates the API surface. Developers may want full control over which functions are exposed this way; on the other hand, having to annotate every single function is also tiring. -**Ensure that invariants are never broken**: especially in the "frontier" between Kotlin and Java, we need to provide an API which does all the corresponding checks. We are OK with introducing some performance penalties when using the API from Java, but no additional penalties should be paide if the class is used in Kotlin. +**Ensure that invariants are never broken**: especially in the "frontier" between Kotlin and Java, we need to provide an API that does all the corresponding checks. We are OK with introducing some performance penalties when using the API from Java, but no additional penalties should be paid if the class is used in Kotlin. **Compatibility with current compilation scheme**: if we alter the way inline classes are currently compiled in a non backward compatible way, we could create a huge split in the community. Or even worse, we will end up supporting two different schemes in the compiler. @@ -86,11 +88,15 @@ class PositiveInt { } ``` -We think this is the right choice because of a couple of reasons: +We think this is the right choice for a couple of reasons: -1. It does not increase the binary size, as it only exposes a constructor which was already there (albeit synthetic); +1. It does not increase the binary size, as it only exposes a constructor that was already there (albeit synthetic); 2. It is binary compatible with previous users of this code. +### Serialization + +`kotlinx.serialization` currently ensures that [value classes are correctly serialized](https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/value-classes.md#serializable-value-classes), even when allocated. This must remain the case once this KEEP is implemented. + ### No argument constructors Frameworks like Java Persistence require classes to have a [default no argument constructor](https://www.baeldung.com/jpa-no-argument-constructor-entity-class). This case would be covered by a combination of the proposal above, and the ability to give default values to arguments of a value class. Continuing with our example of positive integers, the following code, @@ -107,19 +113,19 @@ generates the constructor taking a single integer, but also another one without **Put it under a flag**, in other words, only expose the boxed constructor when some annotation or compiler flag is present. In this case, we could not find a realistic scenario where this expose would be counter-productive; in the worst case in which you expose the constructor but not operations you just have a way to create useless values. -The only drawback is that the ABI exposed in the JVM changes from a version of the compiler to the next, albeit in a binary compatible way. There is a risk involved in downgrading the compiler version, as it may break Java clients, but this scenario is quite rare. +The only drawback is that the ABI exposed in the JVM changes from one version of the compiler to the next, albeit in a binary compatible way. There is a risk involved in downgrading the compiler version, as it may break Java clients, but this scenario is quite rare. **Exposing a factory method**: instead of exposing the constructor, use `PositiveInt.of(3)`. This option is nowadays quite idiomatic in the Java world (for example, in `List.of(1, 2)`), but not as much as constructors. Using a factory method also has the drawback of possible clashes for the name of the factory method, which we avoid when using the constructor. ## Expose operations and members -The current compilation scheme transforms every operation where a value class is involved into a static function that takes the unboxed variants, and whose named is mangled to prevent clashes. This means those operarions are not available for Java consumers. We propose introducing a new `@JvmExposeBoxed` annotation that exposes a variant of the function taking the boxed versions instead. +The current compilation scheme transforms every operation where a value class is involved into a static function that takes the unboxed variants, and whose name is mangled to prevent clashes. This means those operations are not available for Java consumers. We propose introducing a new `@JvmExposeBoxed` annotation that exposes a variant of the function taking the boxed versions instead. -The `@JvmExposeBoxed` annotation may be applied to a declaration, or to a declaration container (classes, files). In the latter case, it should be takes as applied to every single declaration within it. +The `@JvmExposeBoxed` annotation may be applied to a declaration, or a declaration container (classes, files). In the latter case, it should be taken as applied to every single declaration within it. If the `@JvmExposeBoxed` annotation is applied to a member of a value class (or the value class itself, or the containing file), then it is compiled to a member function of the corresponding boxed class, not as a static method. -The name of the declaration must coincide with the name given in the Kotlin code, unless a `@JvmName` annotation is present. In that case, the name defined in the annotation should be used. +The name of the declaration must coincide with the name given in the Kotlin code unless a `@JvmName` annotation is present. In that case, the name defined in the annotation should be used. The following is an example of the compilation of some operations over `PositiveInt`. The `box-impl` and `unbox-impl` refer to the operations defined in the [current compilation scheme](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md#inline-classes-abi-jvm) for boxing and unboxing without checks. @@ -154,16 +160,16 @@ fun duplicate-26b4($this: Int): Int ### Other design choices -**Annotate the value class**: in a [previous proposal](https://github.com/Kotlin/KEEP/blob/commandertvis/jvmexpose/proposals/jvm-expose-boxed-annotation.md) the annotation was applied to the value class itself, not to the operations, and would force every user of that class to create the boxed and unboxed versions. We found this approach not flexible enough: it was completely on the hand of the developer controlling the value class to decide whether Java compatibility was important. This opinion may not coincide with that of another author, which may restrict or expand the boxed variants in their own code. +**Annotate the value class**: in a [previous proposal](https://github.com/Kotlin/KEEP/blob/commandertvis/jvmexpose/proposals/jvm-expose-boxed-annotation.md), the annotation was applied to the value class itself, not to the operations, and would force every user of that class to create the boxed and unboxed versions. We found this approach not flexible enough: it was completely in the hands of the developer controlling the value class to decide whether Java compatibility was important. This opinion may not coincide with that of another author, which may restrict or expand the boxed variants in their code. -**Use a compiler flag instead of an annotation**: we have also considered exposing this feature using a compiler flag, `-Xjvm-expose-boxed=C,D,E`, where `C`, `D`, and `E` are the names of the classes for which boxed variants of the operations should be generated. We found two drawbacks for this choice: +**Use a compiler flag instead of an annotation**: we have also considered exposing this feature using a compiler flag, `-Xjvm-expose-boxed=C,D,E`, where `C`, `D`, and `E` are the names of the classes for which boxed variants of the operations should be generated. We found two drawbacks to this choice: -- Users are not accostumed to tweaking compiler flags, which are often hidden in the build file; in contrast to annotations. +- Users are not accustomed to tweaking compiler flags, which are often hidden in the build file; in contrast to annotations. - It does not give the flexibility of exposing only a subset of operations. The current proposal allows that scenario if you annotate each operation independently, but also allows `@file:JvmExposeBoxed` if you want a wider stroke. -## Problems with reflection +## Further problems with reflection -The current proposal does not solve all of the problems that inline value classes have with reflection. Many of them stem from the back that, once compiled to the unboxed variant, there is no trace of the original boxed variant. +The current proposal does not solve all of the problems that inline value classes have with reflection. Many of them stem from the fact that, once compiled to the unboxed variant, there is no trace of the original boxed variant. ```kotlin class Person(val name: String, val age: PositiveInt) @@ -175,7 +181,7 @@ class Person { } ``` -In the case above, libraries may unadvertedly break some of the requirements of `PositiveInt` if a `Person` instance is created using reflection. +In the case above, libraries may inadvertently break some of the requirements of `PositiveInt` if a `Person` instance is created using reflection. We are currently investigating the required additional support. Some Java serialization libraries, like [Jackson](https://github.com/FasterXML/jackson-module-kotlin/blob/master/docs/value-class-support.md) already include specialized support for value classes. At this point, collaboration with the maintainers of the main libraries in the Java ecosystem seems like the most productive route. @@ -193,3 +199,22 @@ class Person { **Validation**: building on the previous idea, `@JvmBoxed` could be implemented as a [Bean validator](https://beanvalidation.org/), so frameworks like Hibernate would automatically execute the corresponding checks. Alas, the Bean Validation specification is a huge mess right now. By Java 7 it was part of the standard distribution in the `javax.validation` package, but now it's moved to `jakarta.validation` (this is in fact the [main change](https://beanvalidation.org/3.0/) between versions 2 and 3 of the specification). It is impossible to provide good compatibility with many Java versions on those grounds. +## Comparison with Scala + +The closest feature to inline classes in the JVM comes from Scala. In this case, there's quite a big difference between Scala 2 and 3. + +Scala 3 is based on [_opaque type aliases_](https://docs.scala-lang.org/scala3/reference/other-new-features/opaques-details.html) ([SIP-35](https://docs.scala-lang.org/sips/opaque-types.html)). The core idea is that when you mark a type alias as opaque, the translation is only visible in the scope it was defined. For example, inside `opaquetypes` below you can treat `Logarithm` and `Double` interchangeably, but outside they are treated as completely separate types. + +```scala +package object opaquetypes { + opaque type Logarithm = Double +} +``` + +You then define operations over the opaque type using a companion. In Kotlin terms, that amounts to defining all operations as extension methods. The aforementioned SIP does not consider compatibility with Java. In fact, it mentions that opaque types are actually erased before going into each of the backends. + +The closest feature to inline classes in Scala is _value classes_ ([SIP-15](https://docs.scala-lang.org/sips/value-classes.html)). They are available in both Scala 2 and 3 but are considered deprecated in the latter. Value classes follow a compilation scheme very close to Kotlin's, but the original class implementation stays available. In other words, it works as if the constructors and members were exposed, using our terminology. + +- One detail is that instead of mangling, the "unboxed versions" of the methods are available in a (companion) object. Nevertheless, they still require some name adjustments to prevent clashes. + + From 06baad4bad87141c286b0a497911deda9bd0be4e Mon Sep 17 00:00:00 2001 From: Alejandro Serrano Date: Wed, 17 Jul 2024 12:47:38 +0200 Subject: [PATCH 04/19] Introduce `of` factory method --- proposals/jvm-expose-boxed.md | 40 +++++++++++++++-------------------- 1 file changed, 17 insertions(+), 23 deletions(-) diff --git a/proposals/jvm-expose-boxed.md b/proposals/jvm-expose-boxed.md index f4314627c..1420e2eb8 100644 --- a/proposals/jvm-expose-boxed.md +++ b/proposals/jvm-expose-boxed.md @@ -18,9 +18,8 @@ We propose modifications to how value classes are exposed in JVM, with the goal * [Table of contents](#table-of-contents) * [Motivation](#motivation) * [Design goals](#design-goals) -* [Expose boxed constructors](#expose-boxed-constructors) +* [Expose factory method](#expose-factory-method) * [Serialization](#serialization) - * [No argument constructors](#no-argument-constructors) * [Other design choices](#other-design-choices) * [Expose operations and members](#expose-operations-and-members) * [Other design choices](#other-design-choices-1) @@ -55,7 +54,7 @@ The main drawback of this compilation scheme is that a type like `PositiveInt` c ### Design goals -"Exposing" a `value` class is not a single entity. We have found (at least) four different scenarios that we would like to support: +"Exposing" a `value` class is not a single entity. The following are some scenarios that we would like to support: 1. Creating boxed values: for example, creating an actual instance of `PositiveInt`. 2. Calling operations defined over value classes over their boxed representations. @@ -71,9 +70,13 @@ Apart from the particular problems, we set the following cross-cutting goals for **Compatibility with current compilation scheme**: if we alter the way inline classes are currently compiled in a non backward compatible way, we could create a huge split in the community. Or even worse, we will end up supporting two different schemes in the compiler. -## Expose boxed constructors +--- + +It is a **non-goal** for this KEEP to decide what may happen in a world in which value classes are part of the JVM platform ("post-Valhalla") and potentially do not need inlining and boxing. Any future KEEP touching those parts of the language should also consider its interactions with this proposal. -The [current compilation scheme](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md#inline-classes-abi-jvm) exposes the constructor of the boxed class as _synthetic_, which makes it unavailable from Java. We propose to remove that modifier from the constructor, and use the same visibility in the constructor as the one defined in the value class. +## Expose factory method + +The [current compilation scheme](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md#inline-classes-abi-jvm) exposes the constructor of the boxed class as _synthetic_, which makes it unavailable from Java. Note that this constructor does _not_ execute the `init` block, it's essentially boxing the value. We propose to introduce a static factory method `of` that executes any prerequisites and builds the value, with the same visibility as the constructor of the value class. ```kotlin @JvmInline value class PositiveInt(val number: Int) { @@ -83,49 +86,40 @@ The [current compilation scheme](https://github.com/Kotlin/KEEP/blob/master/prop // compiles down to class PositiveInt { public (Int): void - + public constructor-impl(Int): Int // executes the 'init' block // and others + + public static of(number: Int): PositiveInt = + (constructor-impl(number)) } ``` We think this is the right choice for a couple of reasons: -1. It does not increase the binary size, as it only exposes a constructor that was already there (albeit synthetic); +1. Using `of` as a factory method is well-known in the Java ecosystem (for example, to create collections); 2. It is binary compatible with previous users of this code. ### Serialization `kotlinx.serialization` currently ensures that [value classes are correctly serialized](https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/value-classes.md#serializable-value-classes), even when allocated. This must remain the case once this KEEP is implemented. -### No argument constructors - -Frameworks like Java Persistence require classes to have a [default no argument constructor](https://www.baeldung.com/jpa-no-argument-constructor-entity-class). This case would be covered by a combination of the proposal above, and the ability to give default values to arguments of a value class. Continuing with our example of positive integers, the following code, - -```kotlin -@JvmInline value class PositiveInt(val number: Int = 0) { - init { require(number >= 0) } -} -``` - -generates the constructor taking a single integer, but also another one without the optional argument. Since value classes may only have one argument, this means that making it optional creates a no argument constructor. - ### Other design choices **Put it under a flag**, in other words, only expose the boxed constructor when some annotation or compiler flag is present. In this case, we could not find a realistic scenario where this expose would be counter-productive; in the worst case in which you expose the constructor but not operations you just have a way to create useless values. The only drawback is that the ABI exposed in the JVM changes from one version of the compiler to the next, albeit in a binary compatible way. There is a risk involved in downgrading the compiler version, as it may break Java clients, but this scenario is quite rare. -**Exposing a factory method**: instead of exposing the constructor, use `PositiveInt.of(3)`. This option is nowadays quite idiomatic in the Java world (for example, in `List.of(1, 2)`), but not as much as constructors. Using a factory method also has the drawback of possible clashes for the name of the factory method, which we avoid when using the constructor. +**Exposing a constructor directly**: instead of using a factory method, `PositiveInt.of(3)`, expose a constructor, `PositiveInt(3)`. The problem is that the current compilation scheme already defines a constructor, which does _not_ execute the `init` block. Changing the behavior is possible -- after all, we just add more checks -- but this may have a significant performance impact. ## Expose operations and members -The current compilation scheme transforms every operation where a value class is involved into a static function that takes the unboxed variants, and whose name is mangled to prevent clashes. This means those operations are not available for Java consumers. We propose introducing a new `@JvmExposeBoxed` annotation that exposes a variant of the function taking the boxed versions instead. +The current compilation scheme transforms every operation where a value class is involved into a static function that takes the unboxed variants, and whose name is mangled to prevent clashes. This means those operations are not available for Java consumers. We propose introducing a new `@JvmExposeBoxed` annotation that exposes a variant of the function taking and returning the boxed versions instead (if more than one argument or return type is a value class, the aforementioned variant uses the boxed versions for _all_ of them). We call it the _boxed variant_ of the function. The `@JvmExposeBoxed` annotation may be applied to a declaration, or a declaration container (classes, files). In the latter case, it should be taken as applied to every single declaration within it. -If the `@JvmExposeBoxed` annotation is applied to a member of a value class (or the value class itself, or the containing file), then it is compiled to a member function of the corresponding boxed class, not as a static method. +If the `@JvmExposeBoxed` annotation is applied to a member of a value class (or the value class itself, or the containing file), the boxed variant is compiled to a member function of the corresponding boxed class. The unboxed variant is still compiled as a static member of the class. -The name of the declaration must coincide with the name given in the Kotlin code unless a `@JvmName` annotation is present. In that case, the name defined in the annotation should be used. +The name of the boxed variant coincides with the name given in the Kotlin code unless a `@JvmName` annotation is present. In that case, the name defined in the annotation should be used. The following is an example of the compilation of some operations over `PositiveInt`. The `box-impl` and `unbox-impl` refer to the operations defined in the [current compilation scheme](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md#inline-classes-abi-jvm) for boxing and unboxing without checks. From 9e2995a96bc0e1945624db8d26aa63bcd6962254 Mon Sep 17 00:00:00 2001 From: Alejandro Serrano Date: Wed, 17 Jul 2024 13:34:40 +0200 Subject: [PATCH 05/19] Bring back "no argument" section --- proposals/jvm-expose-boxed.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/proposals/jvm-expose-boxed.md b/proposals/jvm-expose-boxed.md index 1420e2eb8..ad762667f 100644 --- a/proposals/jvm-expose-boxed.md +++ b/proposals/jvm-expose-boxed.md @@ -22,6 +22,7 @@ We propose modifications to how value classes are exposed in JVM, with the goal * [Serialization](#serialization) * [Other design choices](#other-design-choices) * [Expose operations and members](#expose-operations-and-members) + * [No argument constructors](#no-argument-constructors) * [Other design choices](#other-design-choices-1) * [Further problems with reflection](#further-problems-with-reflection) * [Other design choices](#other-design-choices-2) @@ -152,6 +153,20 @@ public duplicate($this: PositiveInt): PositiveInt = fun duplicate-26b4($this: Int): Int ``` +### No argument constructors + +Frameworks like Java Persistence require classes to have a [default no argument constructor](https://www.baeldung.com/jpa-no-argument-constructor-entity-class). We propose to expose a no argument constructor whenever a default value is given to the underlying property of the value class (in addition to the factory methods). Continuing with our example of positive integers, the following code, + +```kotlin +@JvmInline value class PositiveInt(val number: Int = 0) { + init { require(number >= 0) } +} +``` + +generates (1) `PositiveInt.of(n: Int)`, (2) `PositiveInt.of()` which uses the default value, and (3) a constructor `PositiveInt()` with the same visibility as the one defined in the class. + +Note that exposing this constructor directly is OK, as we expect the given default value to satisfy any requirement in the `init` block. + ### Other design choices **Annotate the value class**: in a [previous proposal](https://github.com/Kotlin/KEEP/blob/commandertvis/jvmexpose/proposals/jvm-expose-boxed-annotation.md), the annotation was applied to the value class itself, not to the operations, and would force every user of that class to create the boxed and unboxed versions. We found this approach not flexible enough: it was completely in the hands of the developer controlling the value class to decide whether Java compatibility was important. This opinion may not coincide with that of another author, which may restrict or expand the boxed variants in their code. From 2bf2ccd73cd9453cd622dd7df34920eb03b1f6ac Mon Sep 17 00:00:00 2001 From: Alejandro Serrano Date: Thu, 18 Jul 2024 15:42:41 +0200 Subject: [PATCH 06/19] Introduce names for @JvmExposeBoxed --- proposals/jvm-expose-boxed.md | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/proposals/jvm-expose-boxed.md b/proposals/jvm-expose-boxed.md index ad762667f..0cbece286 100644 --- a/proposals/jvm-expose-boxed.md +++ b/proposals/jvm-expose-boxed.md @@ -106,7 +106,7 @@ We think this is the right choice for a couple of reasons: ### Other design choices -**Put it under a flag**, in other words, only expose the boxed constructor when some annotation or compiler flag is present. In this case, we could not find a realistic scenario where this expose would be counter-productive; in the worst case in which you expose the constructor but not operations you just have a way to create useless values. +**Put it under a flag**, in other words, only expose the boxed constructor when some annotation or compiler flag is present. In this case, we could not find a realistic scenario where this expose would be counter-productive; in the worst case in which you expose the factory method but not operations you just have a way to create useless values. The only drawback is that the ABI exposed in the JVM changes from one version of the compiler to the next, albeit in a binary compatible way. There is a risk involved in downgrading the compiler version, as it may break Java clients, but this scenario is quite rare. @@ -116,11 +116,19 @@ The only drawback is that the ABI exposed in the JVM changes from one version of The current compilation scheme transforms every operation where a value class is involved into a static function that takes the unboxed variants, and whose name is mangled to prevent clashes. This means those operations are not available for Java consumers. We propose introducing a new `@JvmExposeBoxed` annotation that exposes a variant of the function taking and returning the boxed versions instead (if more than one argument or return type is a value class, the aforementioned variant uses the boxed versions for _all_ of them). We call it the _boxed variant_ of the function. +```kotlin +annotation class JvmExposedBoxed(val jvmName: String = "") +``` + The `@JvmExposeBoxed` annotation may be applied to a declaration, or a declaration container (classes, files). In the latter case, it should be taken as applied to every single declaration within it. If the `@JvmExposeBoxed` annotation is applied to a member of a value class (or the value class itself, or the containing file), the boxed variant is compiled to a member function of the corresponding boxed class. The unboxed variant is still compiled as a static member of the class. -The name of the boxed variant coincides with the name given in the Kotlin code unless a `@JvmName` annotation is present. In that case, the name defined in the annotation should be used. +The name of the boxed variant coincides with the name given in the Kotlin code unless the `jvmName` argument of the annotation is present. In that case, the name defined in the annotation should be used. Note that `@JvmName` applies to the unboxed variant. + +> [!NOTE] +> There is a corner case in which `@JvmExposeBoxed` with a name is always needed: when the function has no argument which is a value class, but returns a value class. +> In that case the compilation scheme performs no mangling, so not renaming the boxed version results in a platform clash. The following is an example of the compilation of some operations over `PositiveInt`. The `box-impl` and `unbox-impl` refer to the operations defined in the [current compilation scheme](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md#inline-classes-abi-jvm) for boxing and unboxing without checks. @@ -132,7 +140,7 @@ value class PositiveInt(val number: Int) { fun add(other: PositiveInt): PositiveInt = ... } -@JvmExposeBoxed +@JvmExposeBoxed("dupl") fun PositiveInt.duplicate(): PositiveInt = ... // compiles down to @@ -147,7 +155,7 @@ class PositiveInt { // and others } -public duplicate($this: PositiveInt): PositiveInt = +public dupl($this: PositiveInt): PositiveInt = box-impl(duplicate-26b4(unbox-impl($this))) // mangled version fun duplicate-26b4($this: Int): Int From 72319f3dd7786e8903651a18cee278e7ae753238 Mon Sep 17 00:00:00 2001 From: Alejandro Serrano Date: Fri, 19 Jul 2024 10:20:41 +0200 Subject: [PATCH 07/19] Grammar and clarifications --- proposals/jvm-expose-boxed.md | 80 +++++++++++++++++++---------------- 1 file changed, 44 insertions(+), 36 deletions(-) diff --git a/proposals/jvm-expose-boxed.md b/proposals/jvm-expose-boxed.md index 0cbece286..46310fbc1 100644 --- a/proposals/jvm-expose-boxed.md +++ b/proposals/jvm-expose-boxed.md @@ -10,7 +10,7 @@ ## Abstract -We propose modifications to how value classes are exposed in JVM, with the goal of easier consumption from Java. This includes exposing the constructor publicly, and giving the ability to exposed boxed variants of operations. +We propose modifications to how value classes are exposed in JVM, with the goal of easier consumption from Java. This includes exposing the constructor publicly and giving the ability to expose boxed variants of operations. ## Table of contents @@ -20,17 +20,17 @@ We propose modifications to how value classes are exposed in JVM, with the goal * [Design goals](#design-goals) * [Expose factory method](#expose-factory-method) * [Serialization](#serialization) + * [No argument constructors](#no-argument-constructors) * [Other design choices](#other-design-choices) * [Expose operations and members](#expose-operations-and-members) - * [No argument constructors](#no-argument-constructors) * [Other design choices](#other-design-choices-1) -* [Further problems with reflection](#further-problems-with-reflection) - * [Other design choices](#other-design-choices-2) + * [Further problems with reflection](#further-problems-with-reflection) +* [Discarded potential features](#discarded-potential-features) * [Comparison with Scala](#comparison-with-scala) ## Motivation -[Value (or inline) classes](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md) are an important tool to create a _lightweight wrapper_ over another class. One important example are classes which introduce additional invariants over another one, like an integer which is always greater than zero. +[Value (or inline) classes](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md) are an important tool to create a _lightweight wrapper_ over another class. One important example is given by classes that introduce additional invariants over another one, like an integer always greater than zero. ```kotlin @JvmInline value class PositiveInt(val number: Int) { @@ -38,28 +38,38 @@ We propose modifications to how value classes are exposed in JVM, with the goal } ``` -In order to keep its lightweight nature, the Kotlin compiler tries to use the _unboxed_ variant -- in the case above, an integer itself -- instead of its _boxed_ variant -- the `PositiveInt` class -- whenever possible. However, in order to prevent clashes between different overloads, the name of those functions is _mangled_; those functions operate directly on the underlying unboxed representation, with additional checks coming from the `init` block. +In order to keep its lightweight nature, the Kotlin compiler tries to use the _unboxed_ variant -- in the case above, an integer itself -- instead of its _boxed_ variant -- the `PositiveInt` class -- whenever possible. To prevent clashes between different overloads, the names of some of those functions must be _mangled_, that is, transformed in a specific way. The mangled functions operate directly on the underlying unboxed representation, with additional checks coming from the `init` block. The [inline classes](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md) KEEP defines the whole compilation scheme, for this example is enough to know that `constructor-impl` is where the `init` block ends up living. ```kotlin fun PositiveInt.add(other: PositiveInt): PositiveInt = PositiveInt(this.number + other.number) // is compiled down to code similar to +class PositiveInt { + // static member from the JVM perspective + public static constructor-impl(number: Int): Int { + require(number >= 0) + return number + } +} +// 1bc5 is completely made up fun Int.add-1bc5(other: Int): Int = - PositiveInt.check-invariants(this.number + other.number) + PositiveInt.constructor-impl(this.number + other.number) ``` -Note: we use `PositiveInt.check-invariants` for illustrative purposes, the [inline classes](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md) defines the concrete compilation scheme that the compiler ought to follow. +The main drawback of this compilation scheme is that a type like `PositiveInt` cannot be (easily) consumed from Java (or other JVM languages other than Kotlin). This KEEP tries to rectify this problem, providing a way to consume value classes easily from Java. As a side benefit, library authors no longer need to choose between using a value class (which is more performant, but only available for Kotlin consumers) or a (data) class (which always requires boxing but can be used anywhere). -The main drawback of this compilation scheme is that a type like `PositiveInt` cannot be (easily) consumed from Java (or other JVM languages other than Kotlin). One benefit is that library authors no longer need to choose between using a value class (which is more performant, but only available for Kotlin consumers) or a (data) class (which always requires boxing but can be used anywhere). +> [!NOTE] +> In the following code examples we use `static` to highlight those places where the compilation scheme produced a static method in JVM bytecode. +> This does not imply any particular stance on future inclusion of `static` members in the Kotlin language. ### Design goals -"Exposing" a `value` class is not a single entity. The following are some scenarios that we would like to support: +"Exposing" a `value` class is not a single concept. The following are some scenarios that we would like to support: 1. Creating boxed values: for example, creating an actual instance of `PositiveInt`. -2. Calling operations defined over value classes over their boxed representations. -3. Making inline value classes with Java-idiomatic patterns, especially those using reflection. +2. Calling operations that take or return value classes using their boxed representations. +3. Making inline value classes work with Java-idiomatic patterns, especially those using reflection. --- @@ -69,7 +79,7 @@ Apart from the particular problems, we set the following cross-cutting goals for **Ensure that invariants are never broken**: especially in the "frontier" between Kotlin and Java, we need to provide an API that does all the corresponding checks. We are OK with introducing some performance penalties when using the API from Java, but no additional penalties should be paid if the class is used in Kotlin. -**Compatibility with current compilation scheme**: if we alter the way inline classes are currently compiled in a non backward compatible way, we could create a huge split in the community. Or even worse, we will end up supporting two different schemes in the compiler. +**Compatibility with current compilation scheme**: if we alter the way inline classes are currently compiled in a non-backward compatible way, we could create a huge split in the community. Or even worse, we will end up supporting two different schemes in the compiler. --- @@ -77,7 +87,7 @@ It is a **non-goal** for this KEEP to decide what may happen in a world in which ## Expose factory method -The [current compilation scheme](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md#inline-classes-abi-jvm) exposes the constructor of the boxed class as _synthetic_, which makes it unavailable from Java. Note that this constructor does _not_ execute the `init` block, it's essentially boxing the value. We propose to introduce a static factory method `of` that executes any prerequisites and builds the value, with the same visibility as the constructor of the value class. +The [current compilation scheme](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md#inline-classes-abi-jvm) exposes the constructor of the boxed class as _synthetic_, which makes it unavailable from Java. Note that this constructor does _not_ execute the `init` block, so it is just boxing the value. We propose to introduce a static factory method called `of` that executes any prerequisites and builds the value, with the same visibility as the constructor of the value class. ```kotlin @JvmInline value class PositiveInt(val number: Int) { @@ -87,7 +97,7 @@ The [current compilation scheme](https://github.com/Kotlin/KEEP/blob/master/prop // compiles down to class PositiveInt { public (Int): void - public constructor-impl(Int): Int // executes the 'init' block + public static constructor-impl(Int): Int // executes the 'init' block // and others public static of(number: Int): PositiveInt = @@ -104,13 +114,25 @@ We think this is the right choice for a couple of reasons: `kotlinx.serialization` currently ensures that [value classes are correctly serialized](https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/value-classes.md#serializable-value-classes), even when allocated. This must remain the case once this KEEP is implemented. -### Other design choices +### No argument constructors -**Put it under a flag**, in other words, only expose the boxed constructor when some annotation or compiler flag is present. In this case, we could not find a realistic scenario where this expose would be counter-productive; in the worst case in which you expose the factory method but not operations you just have a way to create useless values. +Frameworks like Java Persistence require classes to have a [default no argument constructor](https://www.baeldung.com/jpa-no-argument-constructor-entity-class). We propose to expose a no argument constructor whenever a default value is given to the underlying property of the value class (in addition to the factory methods). Continuing with our example of positive integers, the following code, -The only drawback is that the ABI exposed in the JVM changes from one version of the compiler to the next, albeit in a binary compatible way. There is a risk involved in downgrading the compiler version, as it may break Java clients, but this scenario is quite rare. +```kotlin +@JvmInline value class PositiveInt(val number: Int = 0) { + init { require(number >= 0) } +} +``` -**Exposing a constructor directly**: instead of using a factory method, `PositiveInt.of(3)`, expose a constructor, `PositiveInt(3)`. The problem is that the current compilation scheme already defines a constructor, which does _not_ execute the `init` block. Changing the behavior is possible -- after all, we just add more checks -- but this may have a significant performance impact. +generates both the factory method `PositiveInt.of(n: Int)` and the no argument constructor `PositiveInt()` with the same visibility as the one defined in the class. + +Note that exposing this constructor directly is OK, as we expect the given default value to satisfy any requirement in the `init` block. + +### Other design choices + +**Put it under a flag or annotation**, in other words, only introduce the factory method when some annotation `@JvmExposeFactory` or compiler flag is present. In this case, we could not find a realistic scenario where this expose would be counter-productive; in the worst case in which you expose the factory method but not operations you just have a way to create useless values. + +**Exposing a constructor directly**: instead of using a factory method, `PositiveInt.of(3)`, expose a constructor, `PositiveInt(3)`, also when no default value is given. The problem here is backward compatibility: the current compilation scheme already defines a constructor, which does _not_ execute the `init` block. Changing the behavior is possible -- after all, we just add more checks -- but this may have a significant performance impact. ## Expose operations and members @@ -161,20 +183,6 @@ public dupl($this: PositiveInt): PositiveInt = fun duplicate-26b4($this: Int): Int ``` -### No argument constructors - -Frameworks like Java Persistence require classes to have a [default no argument constructor](https://www.baeldung.com/jpa-no-argument-constructor-entity-class). We propose to expose a no argument constructor whenever a default value is given to the underlying property of the value class (in addition to the factory methods). Continuing with our example of positive integers, the following code, - -```kotlin -@JvmInline value class PositiveInt(val number: Int = 0) { - init { require(number >= 0) } -} -``` - -generates (1) `PositiveInt.of(n: Int)`, (2) `PositiveInt.of()` which uses the default value, and (3) a constructor `PositiveInt()` with the same visibility as the one defined in the class. - -Note that exposing this constructor directly is OK, as we expect the given default value to satisfy any requirement in the `init` block. - ### Other design choices **Annotate the value class**: in a [previous proposal](https://github.com/Kotlin/KEEP/blob/commandertvis/jvmexpose/proposals/jvm-expose-boxed-annotation.md), the annotation was applied to the value class itself, not to the operations, and would force every user of that class to create the boxed and unboxed versions. We found this approach not flexible enough: it was completely in the hands of the developer controlling the value class to decide whether Java compatibility was important. This opinion may not coincide with that of another author, which may restrict or expand the boxed variants in their code. @@ -184,7 +192,7 @@ Note that exposing this constructor directly is OK, as we expect the given defau - Users are not accustomed to tweaking compiler flags, which are often hidden in the build file; in contrast to annotations. - It does not give the flexibility of exposing only a subset of operations. The current proposal allows that scenario if you annotate each operation independently, but also allows `@file:JvmExposeBoxed` if you want a wider stroke. -## Further problems with reflection +### Further problems with reflection The current proposal does not solve all of the problems that inline value classes have with reflection. Many of them stem from the fact that, once compiled to the unboxed variant, there is no trace of the original boxed variant. @@ -202,7 +210,7 @@ In the case above, libraries may inadvertently break some of the requirements of We are currently investigating the required additional support. Some Java serialization libraries, like [Jackson](https://github.com/FasterXML/jackson-module-kotlin/blob/master/docs/value-class-support.md) already include specialized support for value classes. At this point, collaboration with the maintainers of the main libraries in the Java ecosystem seems like the most productive route. -### Other design choices +## Discarded potential features **Mark usages of unboxed classes**: we have investigated the possibility of annotating every place where an unboxed variant is used with a reference to the boxed variant. @@ -228,7 +236,7 @@ package object opaquetypes { } ``` -You then define operations over the opaque type using a companion. In Kotlin terms, that amounts to defining all operations as extension methods. The aforementioned SIP does not consider compatibility with Java. In fact, it mentions that opaque types are actually erased before going into each of the backends. +You then define operations over the opaque type using a companion. In Kotlin terms, that amounts to defining all operations as extension methods. The aforementioned SIP does not consider compatibility with Java. In fact, it mentions that opaque types are simply erased before going into each of the backends. The closest feature to inline classes in Scala is _value classes_ ([SIP-15](https://docs.scala-lang.org/sips/value-classes.html)). They are available in both Scala 2 and 3 but are considered deprecated in the latter. Value classes follow a compilation scheme very close to Kotlin's, but the original class implementation stays available. In other words, it works as if the constructors and members were exposed, using our terminology. From 01863874878c45566f3b258f853343209654f654 Mon Sep 17 00:00:00 2001 From: Alejandro Serrano Date: Mon, 26 Aug 2024 12:38:37 +0200 Subject: [PATCH 08/19] New scheme to expose constructors --- proposals/jvm-expose-boxed.md | 52 +++++++++++++++++++++++++---------- 1 file changed, 37 insertions(+), 15 deletions(-) diff --git a/proposals/jvm-expose-boxed.md b/proposals/jvm-expose-boxed.md index 46310fbc1..4a3e8b5e3 100644 --- a/proposals/jvm-expose-boxed.md +++ b/proposals/jvm-expose-boxed.md @@ -18,7 +18,7 @@ We propose modifications to how value classes are exposed in JVM, with the goal * [Table of contents](#table-of-contents) * [Motivation](#motivation) * [Design goals](#design-goals) -* [Expose factory method](#expose-factory-method) +* [Expose boxed constructors](#expose-boxed-constructors) * [Serialization](#serialization) * [No argument constructors](#no-argument-constructors) * [Other design choices](#other-design-choices) @@ -38,7 +38,7 @@ We propose modifications to how value classes are exposed in JVM, with the goal } ``` -In order to keep its lightweight nature, the Kotlin compiler tries to use the _unboxed_ variant -- in the case above, an integer itself -- instead of its _boxed_ variant -- the `PositiveInt` class -- whenever possible. To prevent clashes between different overloads, the names of some of those functions must be _mangled_, that is, transformed in a specific way. The mangled functions operate directly on the underlying unboxed representation, with additional checks coming from the `init` block. The [inline classes](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md) KEEP defines the whole compilation scheme, for this example is enough to know that `constructor-impl` is where the `init` block ends up living. +In order to keep its lightweight nature, the Kotlin compiler tries to use the _unboxed_ variant — in the case above, an integer itself — instead of its _boxed_ variant — the `PositiveInt` class — whenever possible. To prevent clashes between different overloads, the names of some of those functions must be _mangled_, that is, transformed in a specific way. The mangled functions operate directly on the underlying unboxed representation, with additional checks coming from the `init` block. The [inline classes](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md) KEEP defines the whole compilation scheme, for this example is enough to know that `constructor-impl` is where the `init` block ends up living. ```kotlin fun PositiveInt.add(other: PositiveInt): PositiveInt = @@ -85,9 +85,22 @@ Apart from the particular problems, we set the following cross-cutting goals for It is a **non-goal** for this KEEP to decide what may happen in a world in which value classes are part of the JVM platform ("post-Valhalla") and potentially do not need inlining and boxing. Any future KEEP touching those parts of the language should also consider its interactions with this proposal. -## Expose factory method +## Expose boxed constructors -The [current compilation scheme](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md#inline-classes-abi-jvm) exposes the constructor of the boxed class as _synthetic_, which makes it unavailable from Java. Note that this constructor does _not_ execute the `init` block, so it is just boxing the value. We propose to introduce a static factory method called `of` that executes any prerequisites and builds the value, with the same visibility as the constructor of the value class. +The [current compilation scheme](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md#inline-classes-abi-jvm) exposes the constructor of the boxed class as _synthetic_, which makes it unavailable from Java. This constructor does _not_ execute the `init` block, it just boxes the value. + +Note that when boxing needs to occur, the compilation scheme does not call this constructor directly. Rather, it uses the static `box-impl` method for that type. + +```kotlin +fun positiveIntSingleton(number: PositiveInt): List = listOf(number) +// is "translated" into the following code +fun positiveIntSingleton-3bd7(number: Int): List = listOf(PositiveInt.box-impl(number)) +``` + +We propose a new compilation scheme which _replaces_ the previous one with respect to constructors. + +- The previous synthetic constructor, which does not execute the `init` block, now takes an additional `BoxingConstructorMarker` argument. This argument is always passed as `null`, but allows us to differentiate the constructor from other (like `DefaultConstructorMarker`). The implementation of `box-impl` is updated to call this constructor. +- We get a new constructor with the same visibility in the constructor as the one defined in the value class. This constructor is exposed to Java, and should execute any `init` block; in the current compilation scheme, this amounts to calling `constructor-impl` over the input value. ```kotlin @JvmInline value class PositiveInt(val number: Int) { @@ -96,19 +109,21 @@ The [current compilation scheme](https://github.com/Kotlin/KEEP/blob/master/prop // compiles down to class PositiveInt { - public (Int): void + private synthetic (number: Int, marker: BoxingConstructorMarker) + + public (number: Int): void = (constructor-impl(number), null) + public static constructor-impl(Int): Int // executes the 'init' block + public static box-impl(number: Int): PositiveInt = (number, null) // and others - - public static of(number: Int): PositiveInt = - (constructor-impl(number)) } ``` -We think this is the right choice for a couple of reasons: +Note that this change is binary compatible with previous users of this code: -1. Using `of` as a factory method is well-known in the Java ecosystem (for example, to create collections); -2. It is binary compatible with previous users of this code. +- The previous compilation scheme never calls the constructor directly; it always uses `box-impl` if boxing is required in the Kotlin side. But the signature of `box-impl` is not changed, only its implementation. +- Java users were not able to call the constructor, since it was marked as synthetic (and private). So exposing the constructor is compatible. +- In the scenario in which the constructor was called directly (for example, by reflection), the code is still correct. There might be some performance hit, since now the `init` block is executed, but this is not a big concern. ### Serialization @@ -124,15 +139,22 @@ Frameworks like Java Persistence require classes to have a [default no argument } ``` -generates both the factory method `PositiveInt.of(n: Int)` and the no argument constructor `PositiveInt()` with the same visibility as the one defined in the class. +exposes both the constructor taking a single integer, but also another one without the optional argument. -Note that exposing this constructor directly is OK, as we expect the given default value to satisfy any requirement in the `init` block. +```kotlin +class PositiveInt { + private synthetic (number: Int, marker: BoxingConstructorMarker) + + public (number: Int): void = (constructor-impl(number), null) + public (): void = (0) +} +``` ### Other design choices -**Put it under a flag or annotation**, in other words, only introduce the factory method when some annotation `@JvmExposeFactory` or compiler flag is present. In this case, we could not find a realistic scenario where this expose would be counter-productive; in the worst case in which you expose the factory method but not operations you just have a way to create useless values. +**Put it under a flag or annotation**, in other words, move to the new generation scheme only when some annotation `@JvmExposeFactory` or compiler flag is present. In this case, we could not find a realistic scenario where this expose would be counter-productive; in the worst case in which you expose the new constructor but not operations you just have a way to create useless values. -**Exposing a constructor directly**: instead of using a factory method, `PositiveInt.of(3)`, expose a constructor, `PositiveInt(3)`, also when no default value is given. The problem here is backward compatibility: the current compilation scheme already defines a constructor, which does _not_ execute the `init` block. Changing the behavior is possible -- after all, we just add more checks -- but this may have a significant performance impact. +**Exposing a factory method**: instead of exposing the constructor, use `PositiveInt.of(3)`. This option is nowadays quite idiomatic in the Java world (for example, in `List.of(1, 2)`), but not as much as constructors. Using a factory method also has the drawback of possible clashes for the name of the factory method, which we avoid when using the constructor. ## Expose operations and members From cdd4a9ba4053f769abc2dc5f5227ecf8b0ed4155 Mon Sep 17 00:00:00 2001 From: Alejandro Serrano Date: Mon, 26 Aug 2024 13:36:11 +0200 Subject: [PATCH 09/19] Mention JEP-401 --- proposals/jvm-expose-boxed.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/proposals/jvm-expose-boxed.md b/proposals/jvm-expose-boxed.md index 4a3e8b5e3..ecd585570 100644 --- a/proposals/jvm-expose-boxed.md +++ b/proposals/jvm-expose-boxed.md @@ -21,6 +21,7 @@ We propose modifications to how value classes are exposed in JVM, with the goal * [Expose boxed constructors](#expose-boxed-constructors) * [Serialization](#serialization) * [No argument constructors](#no-argument-constructors) + * [JVM value classes](#jvm-value-classes) * [Other design choices](#other-design-choices) * [Expose operations and members](#expose-operations-and-members) * [Other design choices](#other-design-choices-1) @@ -81,10 +82,6 @@ Apart from the particular problems, we set the following cross-cutting goals for **Compatibility with current compilation scheme**: if we alter the way inline classes are currently compiled in a non-backward compatible way, we could create a huge split in the community. Or even worse, we will end up supporting two different schemes in the compiler. ---- - -It is a **non-goal** for this KEEP to decide what may happen in a world in which value classes are part of the JVM platform ("post-Valhalla") and potentially do not need inlining and boxing. Any future KEEP touching those parts of the language should also consider its interactions with this proposal. - ## Expose boxed constructors The [current compilation scheme](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md#inline-classes-abi-jvm) exposes the constructor of the boxed class as _synthetic_, which makes it unavailable from Java. This constructor does _not_ execute the `init` block, it just boxes the value. @@ -150,6 +147,10 @@ class PositiveInt { } ``` +### JVM value classes + +It is not a goal of this KEEP to decide what should happen with Kotlin value classes once the JVM exposes that feature ([JEP-401](https://openjdk.org/jeps/401)). The compilation scheme presented above follows the guidelines for _Migration of existing classes_, so it should be possible to neatly migrate to a JVM value class without further changes. + ### Other design choices **Put it under a flag or annotation**, in other words, move to the new generation scheme only when some annotation `@JvmExposeFactory` or compiler flag is present. In this case, we could not find a realistic scenario where this expose would be counter-productive; in the worst case in which you expose the new constructor but not operations you just have a way to create useless values. From 8e8614d6451c3e75a09114ef974a0feafc3532a5 Mon Sep 17 00:00:00 2001 From: Alejandro Serrano Date: Thu, 12 Sep 2024 11:51:28 +0200 Subject: [PATCH 10/19] Fix comment about mangling --- proposals/jvm-expose-boxed.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/jvm-expose-boxed.md b/proposals/jvm-expose-boxed.md index ecd585570..9f36384e1 100644 --- a/proposals/jvm-expose-boxed.md +++ b/proposals/jvm-expose-boxed.md @@ -1,4 +1,4 @@ -# Expose boxed inline value classes +# Expose boxed inline value classes in JVM * **Type**: Design proposal * **Author**: Alejandro Serrano @@ -173,7 +173,7 @@ The name of the boxed variant coincides with the name given in the Kotlin code u > [!NOTE] > There is a corner case in which `@JvmExposeBoxed` with a name is always needed: when the function has no argument which is a value class, but returns a value class. -> In that case the compilation scheme performs no mangling, so not renaming the boxed version results in a platform clash. +> In that case the compilation scheme performs no mangling. As a result, without the annotation the Java compiler would produce an ambiguity error while resolving the name. The following is an example of the compilation of some operations over `PositiveInt`. The `box-impl` and `unbox-impl` refer to the operations defined in the [current compilation scheme](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md#inline-classes-abi-jvm) for boxing and unboxing without checks. From ee7c5a2adccd0d3b7133a9d282e5dd265a6fd8c3 Mon Sep 17 00:00:00 2001 From: Alejandro Serrano Date: Thu, 12 Sep 2024 11:55:15 +0200 Subject: [PATCH 11/19] Update link to discussion --- proposals/jvm-expose-boxed.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/jvm-expose-boxed.md b/proposals/jvm-expose-boxed.md index 9f36384e1..58ba5b7da 100644 --- a/proposals/jvm-expose-boxed.md +++ b/proposals/jvm-expose-boxed.md @@ -3,8 +3,8 @@ * **Type**: Design proposal * **Author**: Alejandro Serrano * **Contributors**: Mikhail Zarechenskii, Ilmir Usmanov -* **Discussion**: ?? -* **Status**: ?? +* **Discussion**: [#394](https://github.com/Kotlin/KEEP/issues/394) +* **Status**: Under discussion * **Related YouTrack issue**: [KT-28135](https://youtrack.jetbrains.com/issue/KT-28135/Add-a-mechanism-to-expose-members-of-inline-classes-and-methods-that-use-them-for-Java) * **Previous proposal**: [by @commandertvis](https://github.com/Kotlin/KEEP/blob/commandertvis/jvmexpose/proposals/jvm-expose-boxed-annotation.md) From b3e0ff999f5da818813bff7ef54559029f3e6ca7 Mon Sep 17 00:00:00 2001 From: Alejandro Serrano Date: Fri, 20 Sep 2024 10:45:08 +0200 Subject: [PATCH 12/19] Clarification about goals --- proposals/jvm-expose-boxed.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/proposals/jvm-expose-boxed.md b/proposals/jvm-expose-boxed.md index 58ba5b7da..0dcfcda88 100644 --- a/proposals/jvm-expose-boxed.md +++ b/proposals/jvm-expose-boxed.md @@ -82,6 +82,12 @@ Apart from the particular problems, we set the following cross-cutting goals for **Compatibility with current compilation scheme**: if we alter the way inline classes are currently compiled in a non-backward compatible way, we could create a huge split in the community. Or even worse, we will end up supporting two different schemes in the compiler. +> [!IMPORTANT] +> This KEEP changes _nothing_ about how value classes are handled in Kotlin code. +> The new boxed variants are only accessible to other JVM languages, +> the Kotlin compiler shall hide them, as it currently does with the +> already-implemented compilation scheme. + ## Expose boxed constructors The [current compilation scheme](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md#inline-classes-abi-jvm) exposes the constructor of the boxed class as _synthetic_, which makes it unavailable from Java. This constructor does _not_ execute the `init` block, it just boxes the value. From 2b7a05f44b6208543c5e267f4706cc27e4d9a5c2 Mon Sep 17 00:00:00 2001 From: Alejandro Serrano Date: Mon, 28 Oct 2024 13:55:26 +0100 Subject: [PATCH 13/19] `Result` should be excluded --- proposals/jvm-expose-boxed.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/proposals/jvm-expose-boxed.md b/proposals/jvm-expose-boxed.md index 0dcfcda88..c69da3df7 100644 --- a/proposals/jvm-expose-boxed.md +++ b/proposals/jvm-expose-boxed.md @@ -24,6 +24,7 @@ We propose modifications to how value classes are exposed in JVM, with the goal * [JVM value classes](#jvm-value-classes) * [Other design choices](#other-design-choices) * [Expose operations and members](#expose-operations-and-members) + * [`Result` is excluded](#result-is-excluded) * [Other design choices](#other-design-choices-1) * [Further problems with reflection](#further-problems-with-reflection) * [Discarded potential features](#discarded-potential-features) @@ -212,6 +213,21 @@ public dupl($this: PositiveInt): PositiveInt = fun duplicate-26b4($this: Int): Int ``` +### `Result` is excluded + +Even though on paper the [`Result`](https://kotlinlang.org/api/core/kotlin-stdlib/kotlin/-result/) in the standard library should be covered by this KEEP, this is _not_ the case. The reason is that the compiler performs no mangling, and every `Result` value is treated as `java.lang.Object`. + +```kotlin +fun weirdIncrement(x: Result): Result = + if (x.isSuccess) Result.success(x.getOrThrow() + 1) + else x + +// compiles down to +fun weirdIncrement(x: java.lang.Object): java.lang.Object +``` + +More concretely, if `@JvmExposeBoxed` is applied to a callable using `Result` either as parameter or return type, that position should be treated as a non-value class. + ### Other design choices **Annotate the value class**: in a [previous proposal](https://github.com/Kotlin/KEEP/blob/commandertvis/jvmexpose/proposals/jvm-expose-boxed-annotation.md), the annotation was applied to the value class itself, not to the operations, and would force every user of that class to create the boxed and unboxed versions. We found this approach not flexible enough: it was completely in the hands of the developer controlling the value class to decide whether Java compatibility was important. This opinion may not coincide with that of another author, which may restrict or expand the boxed variants in their code. From 2a2af9cb8177e5886a8d2add2e0514f371d19a52 Mon Sep 17 00:00:00 2001 From: Alejandro Serrano Date: Thu, 7 Nov 2024 16:06:20 +0100 Subject: [PATCH 14/19] Explicit API mode --- proposals/jvm-expose-boxed.md | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/proposals/jvm-expose-boxed.md b/proposals/jvm-expose-boxed.md index c69da3df7..9035d09a3 100644 --- a/proposals/jvm-expose-boxed.md +++ b/proposals/jvm-expose-boxed.md @@ -27,6 +27,7 @@ We propose modifications to how value classes are exposed in JVM, with the goal * [`Result` is excluded](#result-is-excluded) * [Other design choices](#other-design-choices-1) * [Further problems with reflection](#further-problems-with-reflection) +* [Explicit API mode](#explicit-api-mode) * [Discarded potential features](#discarded-potential-features) * [Comparison with Scala](#comparison-with-scala) @@ -166,13 +167,15 @@ It is not a goal of this KEEP to decide what should happen with Kotlin value cla ## Expose operations and members -The current compilation scheme transforms every operation where a value class is involved into a static function that takes the unboxed variants, and whose name is mangled to prevent clashes. This means those operations are not available for Java consumers. We propose introducing a new `@JvmExposeBoxed` annotation that exposes a variant of the function taking and returning the boxed versions instead (if more than one argument or return type is a value class, the aforementioned variant uses the boxed versions for _all_ of them). We call it the _boxed variant_ of the function. +The current compilation scheme transforms every operation where a value class is involved into a static function that takes the unboxed variants, and whose name is mangled to prevent clashes. This means those operations are not available for Java consumers. We propose introducing a new `@JvmExposeBoxed` annotation. ```kotlin -annotation class JvmExposedBoxed(val jvmName: String = "") +annotation class JvmExposedBoxed(val jvmName: String = "", val expose: Boolean = true) ``` -The `@JvmExposeBoxed` annotation may be applied to a declaration, or a declaration container (classes, files). In the latter case, it should be taken as applied to every single declaration within it. +If the `expose` argument is set to `true` (the default), then a new variant of the function taking and returning the boxed versions is produced (if more than one argument or return type is a value class, the aforementioned variant uses the boxed versions for _all_ of them). We call it the _boxed variant_ of the function. + +If the `expose` argument is set to `false`, then only the unboxed variant is produced. This corresponds to the behavior of the compiler prior to this KEEP. Having such a boolean argument is needed in the [explicit API mode](#explicit-api-mode). If the `@JvmExposeBoxed` annotation is applied to a member of a value class (or the value class itself, or the containing file), the boxed variant is compiled to a member function of the corresponding boxed class. The unboxed variant is still compiled as a static member of the class. @@ -182,6 +185,8 @@ The name of the boxed variant coincides with the name given in the Kotlin code u > There is a corner case in which `@JvmExposeBoxed` with a name is always needed: when the function has no argument which is a value class, but returns a value class. > In that case the compilation scheme performs no mangling. As a result, without the annotation the Java compiler would produce an ambiguity error while resolving the name. +The `@JvmExposeBoxed` annotation may be applied to a declaration, or a declaration container (classes, files). In the latter case, it should be taken as applied to every single declaration within it, with the same value for `expose`. It is not allowed to give an explicit value to `jvmName` when using the annotation in a declaration container. + The following is an example of the compilation of some operations over `PositiveInt`. The `box-impl` and `unbox-impl` refer to the operations defined in the [current compilation scheme](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md#inline-classes-abi-jvm) for boxing and unboxing without checks. ```kotlin @@ -255,6 +260,17 @@ In the case above, libraries may inadvertently break some of the requirements of We are currently investigating the required additional support. Some Java serialization libraries, like [Jackson](https://github.com/FasterXML/jackson-module-kotlin/blob/master/docs/value-class-support.md) already include specialized support for value classes. At this point, collaboration with the maintainers of the main libraries in the Java ecosystem seems like the most productive route. +## Explicit API mode + +Whether an operation is available or not in its boxed variant is very important for interoperability with other languages. For that reason, this KEEP mandates that whenever an operation mentions a value class, and [explicit API mode](https://github.com/Kotlin/KEEP/blob/master/proposals/explicit-api-mode.md) is enabled, the developer _must_ indicate whether the operation should or not be exposed. The latter is done using `@JvmExposeBoxed(expose = false)`. + +We expect in most cases for the annotation to be applied file-wise, `@file:JvmExposeBoxed`, since this uniformly enables or disables the feature. + +> [!NOTE] +> If the developer wants to disable any usage of the value class from Java, the following two actions need to be performed: +> 1. Mark the constructor as `private`, so that no new instances may be created from Java; +> 2. Annotate every operation with `@JvmExposeBoxed(expose = false)`. + ## Discarded potential features **Mark usages of unboxed classes**: we have investigated the possibility of annotating every place where an unboxed variant is used with a reference to the boxed variant. From 3aade665c1520815f4daa6dbb435f7f11fe2642c Mon Sep 17 00:00:00 2001 From: Alejandro Serrano Date: Fri, 8 Nov 2024 09:12:40 +0100 Subject: [PATCH 15/19] Clarifications over the annotation --- proposals/jvm-expose-boxed.md | 79 ++++++++++++++++++++++------------- 1 file changed, 49 insertions(+), 30 deletions(-) diff --git a/proposals/jvm-expose-boxed.md b/proposals/jvm-expose-boxed.md index 9035d09a3..3c09567f2 100644 --- a/proposals/jvm-expose-boxed.md +++ b/proposals/jvm-expose-boxed.md @@ -18,6 +18,8 @@ We propose modifications to how value classes are exposed in JVM, with the goal * [Table of contents](#table-of-contents) * [Motivation](#motivation) * [Design goals](#design-goals) +* [The `JvmExposeBoxed` annotation](#the-jvmexposeboxed-annotation) + * [Explicit API mode](#explicit-api-mode) * [Expose boxed constructors](#expose-boxed-constructors) * [Serialization](#serialization) * [No argument constructors](#no-argument-constructors) @@ -27,7 +29,6 @@ We propose modifications to how value classes are exposed in JVM, with the goal * [`Result` is excluded](#result-is-excluded) * [Other design choices](#other-design-choices-1) * [Further problems with reflection](#further-problems-with-reflection) -* [Explicit API mode](#explicit-api-mode) * [Discarded potential features](#discarded-potential-features) * [Comparison with Scala](#comparison-with-scala) @@ -90,6 +91,47 @@ Apart from the particular problems, we set the following cross-cutting goals for > the Kotlin compiler shall hide them, as it currently does with the > already-implemented compilation scheme. +## The `JvmExposeBoxed` annotation + +We propose introducing a new `@JvmExposeBoxed` annotation, defined as follows. + +```kotlin +@Target( + CONSTRUCTOR, PROPERTY, FUNCTION, // callables + CLASS, FILE, // containers +) +annotation class JvmExposedBoxed(val jvmName: String = "", val expose: Boolean = true) +``` + +Whenever a _callable declaration_ (function, property, constructor) is annotated with `@JvmExposeBoxed` and `expose` is set to `true` (the default), a new _boxed_ variant of that declaration should be generated. How this is done differs between constructors and other operations, as discussed below. + +Since annotating every single callable declaration would be incredibly tiresome, the annotation may also be applied to declaration _containers_, such as classes and files. In that case, it should be taken as applied to every single declaration within it, with the same value for `expose`. It is not allowed to give an explicit value to `jvmName` when using the annotation in a declaration container. + +The consequence of the rules above is that if we annotate a class, + +```kotlin +@JvmExposeBoxed @JvmInline value class PositiveInt(val number: Int) { + fun add(other: PositiveInt): PositiveInt = ... +} +``` + +this is equivalent to annotating the constructor and every callable member, + +```kotlin +@JvmInline value class PositiveInt @JvmExposeBoxed constructor (val number: Int) { + @JvmExposeBoxed fun add(other: PositiveInt): PositiveInt = ... +} +``` + +We actually expect in most cases for the annotation to be applied class-wise or even file-wise, `@file:JvmExposeBoxed`, since this uniformly enables or disables the feature. + +> [!IMPORTANT] +> Whether you expose the (boxing) constructor of value class has no influence on whether you can expose boxed variants of operations over that same class. The compilation scheme never calls that constructor, moving between the boxed and unboxed worlds is done via `box-impl` and `unbox-impl`, as described below. + +### Explicit API mode + +Whether an operation is available or not in its boxed variant is very important for interoperability with other languages. For that reason, this KEEP mandates that whenever an operation mentions a value class, and [explicit API mode](https://github.com/Kotlin/KEEP/blob/master/proposals/explicit-api-mode.md) is enabled, the developer _must_ indicate whether the operation should or not be exposed. The latter is done using `@JvmExposeBoxed(expose = false)`. + ## Expose boxed constructors The [current compilation scheme](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md#inline-classes-abi-jvm) exposes the constructor of the boxed class as _synthetic_, which makes it unavailable from Java. This constructor does _not_ execute the `init` block, it just boxes the value. @@ -105,10 +147,10 @@ fun positiveIntSingleton-3bd7(number: Int): List = listOf(PositiveI We propose a new compilation scheme which _replaces_ the previous one with respect to constructors. - The previous synthetic constructor, which does not execute the `init` block, now takes an additional `BoxingConstructorMarker` argument. This argument is always passed as `null`, but allows us to differentiate the constructor from other (like `DefaultConstructorMarker`). The implementation of `box-impl` is updated to call this constructor. -- We get a new constructor with the same visibility in the constructor as the one defined in the value class. This constructor is exposed to Java, and should execute any `init` block; in the current compilation scheme, this amounts to calling `constructor-impl` over the input value. +- We get a new non-synthetic constructor which executes any `init` block; in the current compilation scheme, this amounts to calling `constructor-impl` over the input value. The visibility of this new constructor should coincide with that of the constructor of the value class if the boxed variant of the constructor ought to be exposed, and be `private` otherwise. ```kotlin -@JvmInline value class PositiveInt(val number: Int) { +@JvmExposeBoxed @JvmInline value class PositiveInt(val number: Int) { init { require(number >= 0) } } @@ -139,7 +181,7 @@ Note that this change is binary compatible with previous users of this code: Frameworks like Java Persistence require classes to have a [default no argument constructor](https://www.baeldung.com/jpa-no-argument-constructor-entity-class). We propose to expose a no argument constructor whenever a default value is given to the underlying property of the value class (in addition to the factory methods). Continuing with our example of positive integers, the following code, ```kotlin -@JvmInline value class PositiveInt(val number: Int = 0) { +@JvmExposeBoxed @JvmInline value class PositiveInt(val number: Int = 0) { init { require(number >= 0) } } ``` @@ -161,32 +203,20 @@ It is not a goal of this KEEP to decide what should happen with Kotlin value cla ### Other design choices -**Put it under a flag or annotation**, in other words, move to the new generation scheme only when some annotation `@JvmExposeFactory` or compiler flag is present. In this case, we could not find a realistic scenario where this expose would be counter-productive; in the worst case in which you expose the new constructor but not operations you just have a way to create useless values. - **Exposing a factory method**: instead of exposing the constructor, use `PositiveInt.of(3)`. This option is nowadays quite idiomatic in the Java world (for example, in `List.of(1, 2)`), but not as much as constructors. Using a factory method also has the drawback of possible clashes for the name of the factory method, which we avoid when using the constructor. ## Expose operations and members -The current compilation scheme transforms every operation where a value class is involved into a static function that takes the unboxed variants, and whose name is mangled to prevent clashes. This means those operations are not available for Java consumers. We propose introducing a new `@JvmExposeBoxed` annotation. - -```kotlin -annotation class JvmExposedBoxed(val jvmName: String = "", val expose: Boolean = true) -``` - -If the `expose` argument is set to `true` (the default), then a new variant of the function taking and returning the boxed versions is produced (if more than one argument or return type is a value class, the aforementioned variant uses the boxed versions for _all_ of them). We call it the _boxed variant_ of the function. - -If the `expose` argument is set to `false`, then only the unboxed variant is produced. This corresponds to the behavior of the compiler prior to this KEEP. Having such a boolean argument is needed in the [explicit API mode](#explicit-api-mode). +The current compilation scheme transforms every operation where a value class is involved into a static function that takes the unboxed variants, and whose name is mangled to prevent clashes. This means those operations are not available for Java consumers. -If the `@JvmExposeBoxed` annotation is applied to a member of a value class (or the value class itself, or the containing file), the boxed variant is compiled to a member function of the corresponding boxed class. The unboxed variant is still compiled as a static member of the class. +In the new compilation scheme, if a boxed variant is requested, the compiler shall produce a callable taking and returning the boxed versions. If more than one argument or return type is a value class, the aforementioned variant uses the boxed versions for _all_ of them. -The name of the boxed variant coincides with the name given in the Kotlin code unless the `jvmName` argument of the annotation is present. In that case, the name defined in the annotation should be used. Note that `@JvmName` applies to the unboxed variant. +The name of the boxed variant coincides with the name given in the Kotlin code unless the `jvmName` argument of the annotation is present. In that case, the name defined in the annotation should be used. Note that `@JvmName` annotations apply to the unboxed variant. > [!NOTE] > There is a corner case in which `@JvmExposeBoxed` with a name is always needed: when the function has no argument which is a value class, but returns a value class. > In that case the compilation scheme performs no mangling. As a result, without the annotation the Java compiler would produce an ambiguity error while resolving the name. -The `@JvmExposeBoxed` annotation may be applied to a declaration, or a declaration container (classes, files). In the latter case, it should be taken as applied to every single declaration within it, with the same value for `expose`. It is not allowed to give an explicit value to `jvmName` when using the annotation in a declaration container. - The following is an example of the compilation of some operations over `PositiveInt`. The `box-impl` and `unbox-impl` refer to the operations defined in the [current compilation scheme](https://github.com/Kotlin/KEEP/blob/master/proposals/inline-classes.md#inline-classes-abi-jvm) for boxing and unboxing without checks. ```kotlin @@ -260,17 +290,6 @@ In the case above, libraries may inadvertently break some of the requirements of We are currently investigating the required additional support. Some Java serialization libraries, like [Jackson](https://github.com/FasterXML/jackson-module-kotlin/blob/master/docs/value-class-support.md) already include specialized support for value classes. At this point, collaboration with the maintainers of the main libraries in the Java ecosystem seems like the most productive route. -## Explicit API mode - -Whether an operation is available or not in its boxed variant is very important for interoperability with other languages. For that reason, this KEEP mandates that whenever an operation mentions a value class, and [explicit API mode](https://github.com/Kotlin/KEEP/blob/master/proposals/explicit-api-mode.md) is enabled, the developer _must_ indicate whether the operation should or not be exposed. The latter is done using `@JvmExposeBoxed(expose = false)`. - -We expect in most cases for the annotation to be applied file-wise, `@file:JvmExposeBoxed`, since this uniformly enables or disables the feature. - -> [!NOTE] -> If the developer wants to disable any usage of the value class from Java, the following two actions need to be performed: -> 1. Mark the constructor as `private`, so that no new instances may be created from Java; -> 2. Annotate every operation with `@JvmExposeBoxed(expose = false)`. - ## Discarded potential features **Mark usages of unboxed classes**: we have investigated the possibility of annotating every place where an unboxed variant is used with a reference to the boxed variant. From 82a0e043325a1477945d3a1cbfce61bf4158c87a Mon Sep 17 00:00:00 2001 From: Alejandro Serrano Date: Mon, 18 Nov 2024 10:44:24 +0100 Subject: [PATCH 16/19] Clarifications --- proposals/jvm-expose-boxed.md | 48 +++++++++++++++++++++++++++++------ 1 file changed, 40 insertions(+), 8 deletions(-) diff --git a/proposals/jvm-expose-boxed.md b/proposals/jvm-expose-boxed.md index 3c09567f2..0f16945ee 100644 --- a/proposals/jvm-expose-boxed.md +++ b/proposals/jvm-expose-boxed.md @@ -27,6 +27,7 @@ We propose modifications to how value classes are exposed in JVM, with the goal * [Other design choices](#other-design-choices) * [Expose operations and members](#expose-operations-and-members) * [`Result` is excluded](#result-is-excluded) + * [Annotations](#annotations) * [Other design choices](#other-design-choices-1) * [Further problems with reflection](#further-problems-with-reflection) * [Discarded potential features](#discarded-potential-features) @@ -97,15 +98,19 @@ We propose introducing a new `@JvmExposeBoxed` annotation, defined as follows. ```kotlin @Target( - CONSTRUCTOR, PROPERTY, FUNCTION, // callables - CLASS, FILE, // containers + // function-like + FUNCTION, CONSTRUCTOR, PROPERTY_GETTER, PROPERTY_SETTER, + // properties + PROPERTY, + // containers + CLASS, FILE, ) annotation class JvmExposedBoxed(val jvmName: String = "", val expose: Boolean = true) ``` -Whenever a _callable declaration_ (function, property, constructor) is annotated with `@JvmExposeBoxed` and `expose` is set to `true` (the default), a new _boxed_ variant of that declaration should be generated. How this is done differs between constructors and other operations, as discussed below. +Whenever a _function-like declaration_ (function, constructor, property accessor) is annotated with `@JvmExposeBoxed` and `expose` is set to `true` (the default), a new _boxed_ variant of that declaration should be generated. How this is done differs between constructors and other operations, as discussed below. -Since annotating every single callable declaration would be incredibly tiresome, the annotation may also be applied to declaration _containers_, such as classes and files. In that case, it should be taken as applied to every single declaration within it, with the same value for `expose`. It is not allowed to give an explicit value to `jvmName` when using the annotation in a declaration container. +Since annotating every single declaration in a file or class would be incredibly tiresome, the annotation may also be applied to declaration _containers_, such as classes and files. In that case, it should be taken as applied to every single declaration within it, with the same value for `expose`. It is not allowed to give an explicit value to `jvmName` when using the annotation in a declaration container. The consequence of the rules above is that if we annotate a class, @@ -115,7 +120,7 @@ The consequence of the rules above is that if we annotate a class, } ``` -this is equivalent to annotating the constructor and every callable member, +this is equivalent to annotating the constructor and every member, ```kotlin @JvmInline value class PositiveInt @JvmExposeBoxed constructor (val number: Int) { @@ -123,7 +128,27 @@ this is equivalent to annotating the constructor and every callable member, } ``` -We actually expect in most cases for the annotation to be applied class-wise or even file-wise, `@file:JvmExposeBoxed`, since this uniformly enables or disables the feature. +For the purposes of this KEEP, a _property_ counts as a container for its getter and setter. As a result, if you want to give a name for the accessors, you need to apply the annotation separately to each accessor or use a explicit target. + +```kotlin +@JvmInline value class PositiveInt constructor (val number: Int) { + @get:JvmExposeBoxed("negated") // instead of 'getNegated' + val negated: NegativeInt = ... +} +``` + +We actually expect in most cases for the annotation to be applied class-wise or even file-wise, `@file:JvmExposeBoxed`, since this uniformly enables or disables the feature. If several `@JvmExposeBoxed` annotations apply to a single declaration, the closest one (in the nesting sense of the term) takes precedence. + +```kotlin +@file:JvmExposeBoxed + +@JvmExposeBoxed(expose = false) @JvmInline value class Foo(val foo: String) { + // constructor is not exposed (the closest `JvmExposeBoxed` is `false`) + + // `zoom` is exposed (the closest `JvmExoseBoxed` is `true`) + @JvmExposeBoxed fun zoom(other: Foo) { } +} +``` > [!IMPORTANT] > Whether you expose the (boxing) constructor of value class has no influence on whether you can expose boxed variants of operations over that same class. The compilation scheme never calls that constructor, moving between the boxed and unboxed worlds is done via `box-impl` and `unbox-impl`, as described below. @@ -209,7 +234,7 @@ It is not a goal of this KEEP to decide what should happen with Kotlin value cla The current compilation scheme transforms every operation where a value class is involved into a static function that takes the unboxed variants, and whose name is mangled to prevent clashes. This means those operations are not available for Java consumers. -In the new compilation scheme, if a boxed variant is requested, the compiler shall produce a callable taking and returning the boxed versions. If more than one argument or return type is a value class, the aforementioned variant uses the boxed versions for _all_ of them. +In the new compilation scheme, if a boxed variant is requested, the compiler shall produce a function taking and returning the boxed versions. If more than one argument or return type is a value class, the aforementioned variant uses the boxed versions for _all_ of them. The name of the boxed variant coincides with the name given in the Kotlin code unless the `jvmName` argument of the annotation is present. In that case, the name defined in the annotation should be used. Note that `@JvmName` annotations apply to the unboxed variant. @@ -261,7 +286,14 @@ fun weirdIncrement(x: Result): Result = fun weirdIncrement(x: java.lang.Object): java.lang.Object ``` -More concretely, if `@JvmExposeBoxed` is applied to a callable using `Result` either as parameter or return type, that position should be treated as a non-value class. +More concretely, if `@JvmExposeBoxed` is applied to a function or accessor using `Result` either as parameter or return type, that position should be treated as a non-value class. + +### Annotations + +Annotations should be carried over to the boxed variant of a declaration, with the exception of `@JvmName` and `@JvmExposeBoxed`. + +- If a declaration has a `@JvmName` annotation, that should appear only in the _unboxed_ variant, which is the one whose name is affected. +- If a declaration has a `@JvmExposeBoxed` annotation, that should appear only in the _boxed_ variant. ### Other design choices From 1172df5e0fae125c299ea06f6a35f44b177b4a2b Mon Sep 17 00:00:00 2001 From: Alejandro Serrano Date: Mon, 18 Nov 2024 15:47:25 +0100 Subject: [PATCH 17/19] Clarify propagation even more --- proposals/jvm-expose-boxed.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/proposals/jvm-expose-boxed.md b/proposals/jvm-expose-boxed.md index 0f16945ee..1c1a8f695 100644 --- a/proposals/jvm-expose-boxed.md +++ b/proposals/jvm-expose-boxed.md @@ -108,30 +108,34 @@ We propose introducing a new `@JvmExposeBoxed` annotation, defined as follows. annotation class JvmExposedBoxed(val jvmName: String = "", val expose: Boolean = true) ``` -Whenever a _function-like declaration_ (function, constructor, property accessor) is annotated with `@JvmExposeBoxed` and `expose` is set to `true` (the default), a new _boxed_ variant of that declaration should be generated. How this is done differs between constructors and other operations, as discussed below. +Whenever a _function-like declaration_ (function, constructor, property accessor) is annotated with `@JvmExposeBoxed` and `expose` is set to `true` (the default), a new _boxed_ variant of that declaration should be generated. How this is done differs between constructors and other operations, as discussed below. The compiler should report a _warning_ if no boxed variant of the annotated declaration exists. -Since annotating every single declaration in a file or class would be incredibly tiresome, the annotation may also be applied to declaration _containers_, such as classes and files. In that case, it should be taken as applied to every single declaration within it, with the same value for `expose`. It is not allowed to give an explicit value to `jvmName` when using the annotation in a declaration container. +Since annotating every single declaration in a file or class would be incredibly tiresome, the annotation may also be applied to declaration _containers_, such as classes and files. In that case, it should be taken as applied to every single declaration within it _for which the boxed variant exists_, with the same value for `expose`. It is not allowed to give an explicit value to `jvmName` when using the annotation in a declaration container. The consequence of the rules above is that if we annotate a class, ```kotlin @JvmExposeBoxed @JvmInline value class PositiveInt(val number: Int) { fun add(other: PositiveInt): PositiveInt = ... + + fun toInt(): Int = number } ``` -this is equivalent to annotating the constructor and every member, +this is equivalent to annotating the constructor and the `add` member, leaving `toInt` without annotation as no boxed variant exists, ```kotlin @JvmInline value class PositiveInt @JvmExposeBoxed constructor (val number: Int) { @JvmExposeBoxed fun add(other: PositiveInt): PositiveInt = ... + + fun toInt(): Int = number } ``` For the purposes of this KEEP, a _property_ counts as a container for its getter and setter. As a result, if you want to give a name for the accessors, you need to apply the annotation separately to each accessor or use a explicit target. ```kotlin -@JvmInline value class PositiveInt constructor (val number: Int) { +@JvmExposeBoxed @JvmInline value class PositiveInt(val number: Int) { @get:JvmExposeBoxed("negated") // instead of 'getNegated' val negated: NegativeInt = ... } From 8f7312f7fb90c05a36875e90ea0c27ffa56ce926 Mon Sep 17 00:00:00 2001 From: Alejandro Serrano Date: Tue, 19 Nov 2024 08:41:31 +0100 Subject: [PATCH 18/19] Clarify retention --- proposals/jvm-expose-boxed.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/proposals/jvm-expose-boxed.md b/proposals/jvm-expose-boxed.md index 1c1a8f695..c0cadc3db 100644 --- a/proposals/jvm-expose-boxed.md +++ b/proposals/jvm-expose-boxed.md @@ -105,6 +105,7 @@ We propose introducing a new `@JvmExposeBoxed` annotation, defined as follows. // containers CLASS, FILE, ) +@Retention(RUNTIME) annotation class JvmExposedBoxed(val jvmName: String = "", val expose: Boolean = true) ``` @@ -299,6 +300,8 @@ Annotations should be carried over to the boxed variant of a declaration, with t - If a declaration has a `@JvmName` annotation, that should appear only in the _unboxed_ variant, which is the one whose name is affected. - If a declaration has a `@JvmExposeBoxed` annotation, that should appear only in the _boxed_ variant. +Note that `@JvmExposeBoxed` annotations on containers "travel" to the boxed variant. As a result, both annotation processors and runtime reflection do not see `@JvmExposeBoxed` on the containers (file, class) but rather on each individual operation. + ### Other design choices **Annotate the value class**: in a [previous proposal](https://github.com/Kotlin/KEEP/blob/commandertvis/jvmexpose/proposals/jvm-expose-boxed-annotation.md), the annotation was applied to the value class itself, not to the operations, and would force every user of that class to create the boxed and unboxed versions. We found this approach not flexible enough: it was completely in the hands of the developer controlling the value class to decide whether Java compatibility was important. This opinion may not coincide with that of another author, which may restrict or expand the boxed variants in their code. From f76ebcbf76b2216422fd5ecb9c28aaf77d2d6c11 Mon Sep 17 00:00:00 2001 From: Alejandro Serrano Date: Thu, 5 Dec 2024 11:02:23 +0100 Subject: [PATCH 19/19] Clarification about suspend --- proposals/jvm-expose-boxed.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/proposals/jvm-expose-boxed.md b/proposals/jvm-expose-boxed.md index c0cadc3db..4e392a997 100644 --- a/proposals/jvm-expose-boxed.md +++ b/proposals/jvm-expose-boxed.md @@ -109,6 +109,8 @@ We propose introducing a new `@JvmExposeBoxed` annotation, defined as follows. annotation class JvmExposedBoxed(val jvmName: String = "", val expose: Boolean = true) ``` +It is not allowed to apply the annotation to members marked with the `suspend` modifier. + Whenever a _function-like declaration_ (function, constructor, property accessor) is annotated with `@JvmExposeBoxed` and `expose` is set to `true` (the default), a new _boxed_ variant of that declaration should be generated. How this is done differs between constructors and other operations, as discussed below. The compiler should report a _warning_ if no boxed variant of the annotated declaration exists. Since annotating every single declaration in a file or class would be incredibly tiresome, the annotation may also be applied to declaration _containers_, such as classes and files. In that case, it should be taken as applied to every single declaration within it _for which the boxed variant exists_, with the same value for `expose`. It is not allowed to give an explicit value to `jvmName` when using the annotation in a declaration container.