Swifts C++ interoperability

[Schaltfehler]

Bidirectional C++ interoperability has been added in Swift 5.9/Xcode 15.

I would like to gather thoughts and points that speak for or against a conversion of ggml-metal.m to ggml-metal.swift.
Or maybe someone already started to work on this or notice some limits or blockers?

• I would say on MacOS/iOS most if not all apps that use llama.cpp are/will be written in swift. By interopting directly with swift we could avoid the bridging and dependence on the objc run time and potentially simply the usage of the library.
• Not sure if true for this community but in objc devs are in decline and a swift to swift could ease up the usage and contribution from swift devs.

[ggerganov]

I'm interested in replacing Obj-c with Swift, but will definitely need an initial implementation on top of which we can build. I don't have enough experience to do it myself. If is is simple enough and covers everything that we currently have, I will be fine to switch to Swift

[slaren]

Is there any reason to not use the C++ bindings instead? https://developer.apple.com/metal/cpp/

[Schaltfehler]

Ohh Good question. This might actually be the better option, we could keep the code base completely in c/cpp.
Minimum version seems to be iOS15/MacOS12, current spm file has it set to iOS14/MacOS11, not sure if there is a specific reason for current supported minimum version.

[jhen0409]

Ohh Good question. This might actually be the better option, we could keep the code base completely in c/cpp.

Minimum version seems to be iOS15/MacOS12, current spm file has it set to iOS14/MacOS11, not sure if there is a specific reason for current supported minimum version.

iOS 14/macOS 12 for int64_t support in Metal, this is the minimum requirement found so far, so I think we still can increase it. macOS 11 in the swift package might be a mistake.

[Schaltfehler]

Nice, thanks for correcting MacOS min version.
#3564

[philippzagar]

Would love to see the Metal C++ bindings integrated into the project, making ObjC redudant! As far as I'm seeing, the bindings are "technically" just available for the iOS17 Beta's (also here: https://github.com/bkaradzic/metal-cpp). Probably also works with the stable iOS17 version, just as a small heads-up that this might be troublesome.
Sadly, I don't have the knowledge or experience to code this quite big change on my own, are there any plans to work on that? @Schaltfehler

[Schaltfehler]

Apples Metal C++ link above provides headers for all major OS version starting with MacOS12/iOS15 up to MacOS14/iOS17.
There is script that merged them all into one headerfile which we could consider to use if we don't want to pollute the repository with all these files.
I am not sure if someone is currently already diving deeper into this. I though on gather more though first.
My c++ is a bit rusty but maybe ok for converting from objc to c++.

[jsflax]

Swift/C++ interoperability is still new, somewhat experimental, and the compilation is slow. I have been experimenting with the interop feature with llama.cpp, and though it works, it's not pleasant. Many C++ types do not bridge well/at all, templates are useless, and you end up having to write a bunch of bridging code anyways. There are too many unknowns as far as compatibility until the feature matures.

I would recommend exposing an ObjC++ interface over the current C interface to better support Swift users. As it stands, there are already a number of missing features not exposed in the llama.cpp C interface that Swift users cannot reasonably interact with.