Exploring Kotlin/Native – Part 2 – Interoperability

In part 1 of this blog series, we looked at how one might compile Kotlin code for 8 different platforms using the Kotlin/Native compiler. But you may have noticed that I only compiled functional executables for three platforms. That’s because building an Android or iOS app requires more project files than a simple executable in order to run on a device. Setting up an Android/iOS multi-platform app requires a different set of steps to set up. This topic requires a little more in-depth understanding of Android and iOS development and is beyond the scope of this blog series. You can learn more about Kotlin Multiplatform through Big Nerd Ranch’s Advanced Kotlin course. Look up the Advanced Kotlin course in our Course Schedule or we’re happy to answer any questions.

In this post, we’ll explore how well the Kotlin/Native compiler handles both pre-built and non-pre-built libraries.

Pre-built Libraries

The Kotlin/Native distribution includes “Platform Libraries”, which are a set of prebuilt libraries specific to each target.

A few of the prebuilt libraries that come “out-of-the-box” when you download Kotlin/Native are:

• POSIX (available for all platforms except WebAssembly)
• OpenGL
• Zlib
• Gzip
• Metal
• Foundation … along with many other libraries and Apple frameworks.

Source: Kotlin/Native Interoperability

How it Works

According to kotlinlang.org:

“The Kotlin/Native compiler automatically detects which of the platform libraries have been accessed and automatically links the needed libraries. … [Platform] libs in the distribution are merely just wrappers and bindings to the native libraries. That means the native libraries themselves (.so, .a, .dylib, .dll, etc.) should be installed on the machine.”

This means that you can write a Kotlin program that uses any of the supported platform libraries. Since the platform libraries are wrappers and bindings to the native libraries, any accesses to the platform libraries will be routed to where the actual implementation of the native library is located on the machine. In this way, the program is able to access the operating system services of the platform that it targets.

No additional action needs to be taken in order to utilize the platform libraries. Platform libraries seem easy and straightforward to use, so let’s try it out.

Test Driving a Platform Library

There are several platform libraries to choose from. OpenGL appeared to be the most interesting to me, so we’ll experiment with it. (Note: Apple deprecated OpenGL in 2018 in favor of Metal, but we are using OpenGL 3.3, the version upon which all future versions of OpenGL are based; macOS supports up to OpenGL 4.1.)

First, we need to build a window and an OpenGL context. Some background information about OpenGL is strongly recommended for this tutorial, but not strictly required. Chapter 2 of LearnOpenGL is a great source of background information and serves as inspiration for this tutorial.

We could manually program our window and context, or we can use a pre-existing library that contains functions that can create a window and an OpenGL context for us. Some commonly used libraries which handle these tasks are GLUT, SDL, SFML and GLFW. I noticed that GLUT is included in the macOS platform libraries. For Linux, the equivalent library will need to be installed on the machine. Regardless of the platform, the libraries need to exist directly on the machine. The actual implementation of the libraries is not included in the platform libraries, as the platform libraries are merely wrappers/bindings to actual libraries. OpenGL and GLUT come with the OS and Xcode installations.

JetBrains provides a neat, simple demonstration of this out-of-the-box capability, which can be found here.

Figure 1. Running JetBrains’s OpenGL sample.

Non-pre-built Libraries

What if I want to use a library that is not pre-included in the platform libraries? What if I wanted to use GLFW instead of GLUT so that I can follow along in the OpenGL tutorial?

We will need to create Kotlin bindings from the C library. This example is a good reference. We’ll use the cinterop tool. “It takes a C library and generates the corresponding Kotlin bindings for it, which then allows us to use the library as if it were Kotlin code.”

First, download the GLFW source files. You can download the source package or clone the git repo. Note: We’ll be using a macOS. The steps in this tutorial may not translate directly to Windows or Linux operating systems.

If you want to produce a library compatible with all three platforms, you have to create bindings that will act as an interface to the real library (written in C/C++). There’s some good documentation about how to produce the .klib file to build our program with.

• C Interop: Official documentation on how to use Kotlin/Native’s cinterop tool to generate everything your Kotlin program needs to interact with an external library.
• Kotlin/Native Libraries: Basic instructions on how to produce a library with the Kotlin/Native compiler and how to link the library to your Kotlin program; also contains documentation about helpful klib utilities included in Kotlin/Native.

We’ll use these resources to produce our own .klib file.

The first step before we start building the library is to create a Kotlin program that will use GLFW. I created a file named test.kt. In test.kt, import the OpenGL platform libraries.

import kotlinx.cinterop.*
import platform.OpenGL.*
import platform.OpenGLCommon.*

Below these import statements, import the GLFW libraries and instantiate a window.

...
import glfw.*
fun main() {
    var glfwInitialized = glfwInit()
    if (glfwInitialized == 0) {
        println("Failed to initialize GLFW")
        return
    }
    var window = glfwCreateWindow(640, 480, "OpenGL on Kotlin! Wow!", null, null);
    if (window == null)
    {
        println("Failed to create GLFW window")
        glfwTerminate()
        return
    }
    println("hello world! glfw initialized? = $glfwInitialized, window pointer = $window")
    glfwMakeContextCurrent(window)
    while(glfwWindowShouldClose(window) == 0)
    {
        glClear(GL_COLOR_BUFFER_BIT)
        glfwSwapBuffers(window);
        glfwPollEvents();
    }
    glfwTerminate()
    return
}

This program uses the function glfwInit(), among others, whose actual implementation is contained inside the GLFW C library. We used this program to test that the Kotlin program can properly access the GLFW library.

Next, you need to define the .def file. It is important that you get this part right. If you do not have the proper contents included in your custom .def file, the cinterop tool may not be able to generate the necessary files properly. You will know if you did something wrong if you are trying to use a function or property belonging to the C library in your Kotlin program and you receive compile-time errors stating that the function or property is not recognized. A good way to check if the wrapper/bindings are being generated correctly is by using the klib contents <library-name> command (explained in more detail later). If you are inducing C interoperability on a library, you must ensure that you include all of the necessary header files and include the proper compiler and linker options. To know if you are doing this right may require some background knowledge about how to build and link the particular C library you are using. For the sake of this demonstration, however, the contents of the .def file will be provided to you.

Create a glfw.def file. The contents should look like this:

headers = /usr/local/include/GLFW/glfw3.h /usr/local/include/GLFW/glfw3native.h
compiler-options = -framework OpenGl
package = glfw
linkerOpts.osx = -L/opt/local/lib -L/usr/local/lib -lglfw
linkerOpts.linux = -L/usr/lib64 -L/usr/lib/x86_64-linux-gnu -lglfw
linkerOpts.mingw = -lglfw

Note: According to the documentation on Creating Bindings for a New Library, if you wish to insert compiler options that are specific to a particular platform, you can set fields such as compilerOpts.osx and compilerOpts.linux equal to the options you would like to specify.

To avoid lengthy terminal commands, we are going to add the Kotlin/Native distribution to the system PATH. (If you already added Kotlin/Native to the PATH, you can skip this part.) When adding the distribution to the PATH, you are telling the terminal to look in this location (along with the other locations specified in your PATH) when using terminal commands (such as cinterop and kotlinc-native). The terminal needs to know where to find the scripts behind the commands in order to run them. If you do not add the distribution location path to your PATH, the terminal will not recognize your cinterop and kotlinc-native commands when you try to use them (unless you type the entire file path specifying the location of the respective scripts). First, identify the location of your Kotlin/Native distribution that you downloaded. I downloaded my instance of the Kotlin/Native distribution from the releases page (scroll all the way down and you can see links to download Kotlin/Native for Windows, Linux, and macOS).

Copy the file path to wherever you downloaded and unzipped your Kotlin/Native distribution. You can add the distribution location to the PATH temporarily or permanently.

The Temporary Way

Enter the following command into your terminal, substituting <path-to-your-kotlin-native-distribution> with the file path leading to your Kotlin/Native distribution located on your local machine. Don’t forget to append /bin.

export PATH=$PATH:<path-to-your-kotlin-native-distribution>/bin

To verify that your path was added to PATH, you can run:

echo $PATH

and check that your distribution location was added to the PATH variable. File paths are separated by the : delimiter.

The Permanent Way

Run:

open ~/.bash_profile

or, if the file does not exist:

touch ~/.bash_profile
open ~/.bash_profile

Append the following text to the subsequent file that opens, substituting <path-to-your-kotlin-native-distribution> with the file path leading to your Kotlin/Native distribution located on your local machine:

PATH=$PATH:<path-to-your-kotlin-native-distribution>/bin

Save the file. Close the terminal and open a new session for the changes to take effect. Verify that your path was added to PATH by running:

echo $PATH

Now you should be able to use the cinterop and kotlinc-native commands without issue.

Set Up C Interoperability and Link the Library

Run the cinterop command:

$ cinterop -def glfw.def -compiler-option -I/usr/local/include -o glfw3

You can verify if the bindings were generated correctly by running:

$ klib contents glfw3

You should see a bunch of code containing type aliases, constants, function headers, etc. output to the terminal. The code excerpt below displays snippets of glfw3.klib‘s contents. Notice that the code is in Kotlin. This output indicates that the Kotlin bindings were generated. These bindings correspond to the properties and functions present in the C library. If you run the klib contents command but nothing outputs, it likely means that something went wrong. You should check that the right header files and options were declared in the .def file in the appropriate fields. Refer to the library’s documentation for guidance on properly building and linking the library.

package glfw {
@CStruct(spelling = "struct { unsigned char p0[15]; float p1[6]; }") class GLFWgamepadstate constructor(rawPtr: NativePtr /* = NativePtr */) : CStructVar {
    val axes: CArrayPointer<FloatVar /* = FloatVarOf<Float> */> /* = CPointer<FloatVarOf<Float>> */
    val buttons: CArrayPointer<UByteVar /* = UByteVarOf<UByte> */> /* = CPointer<UByteVarOf<UByte>> */
    companion object : CStructVar.Type
}
...

typealias GLFWcharfun = CPointer<CFunction<(CPointer<GLFWwindow>?, UInt) -> Unit>>
typealias GLFWcharfunVar = CPointerVarOf<GLFWcharfun>
typealias GLFWcharmodsfun = CPointer<CFunction<(CPointer<GLFWwindow>?, UInt, Int) -> Unit>>
typealias GLFWcharmodsfunVar = CPointerVarOf<GLFWcharmodsfun>
typealias GLFWcursorenterfun = CPointer<CFunction<(CPointer<GLFWwindow>?, Int) -> Unit>>
...

package glfw {
const val GLFW_ACCUM_ALPHA_BITS: Int = 135178
const val GLFW_ACCUM_BLUE_BITS: Int = 135177
const val GLFW_ACCUM_GREEN_BITS: Int = 135176
...

package glfw {
const val GLFW_KEY_EQUAL: Int = 61
const val GLFW_KEY_ESCAPE: Int = 256
const val GLFW_KEY_F: Int = 70
const val GLFW_KEY_F1: Int = 290
...

package glfw {
const val GLFW_OPENGL_API: Int = 196609
const val GLFW_OPENGL_COMPAT_PROFILE: Int = 204802
const val GLFW_OPENGL_CORE_PROFILE: Int = 204801
const val GLFW_OPENGL_DEBUG_CONTEXT: Int = 139271
...

@CCall(id = "knifunptr_glfw77_glfwCreateCursor") external fun glfwCreateCursor(image: CValuesRef<GLFWimage>?, xhot: Int, yhot: Int): CPointer<GLFWcursor>?
@CCall(id = "knifunptr_glfw78_glfwCreateStandardCursor") external fun glfwCreateStandardCursor(shape: Int): CPointer<GLFWcursor>?
@CCall(id = "knifunptr_glfw25_glfwCreateWindow") external fun glfwCreateWindow(width: Int, height: Int, @CCall.CString title: String?, monitor: CValuesRef<GLFWmonitor>?, share: CValuesRef<GLFWwindow>?): CPointer<GLFWwindow>?
@CCall(id = "knifunptr_glfw22_glfwDefaultWindowHints") external fun glfwDefaultWindowHints()
@CCall(id = "knifunptr_glfw79_glfwDestroyCursor") external fun glfwDestroyCursor(cursor: CValuesRef<GLFWcursor>?)
@CCall(id = "knifunptr_glfw26_glfwDestroyWindow") external fun glfwDestroyWindow(window: CValuesRef<GLFWwindow>?)
...

Once the glfw3.klib file is generated, install the library by running:

$ klib install glfw3.klib

Note: The klib install command will install the associated library files to the default repository. The headers will be installed to /usr/local/include/GLFW and the library files will be installed to /usr/local/lib for macOS. For Linux, the corresponding install locations are /opt/local/include/GLFW and /opt/local/lib.

Compile the program, linking our program to the GLFW library.

$ kotlinc-native test.kt -l glfw3

(This outputs an executable file named program.kexe by default. To specify a name, append -o program-name to the command, substituting “program-name” with your desired program name.)

To run the program, type ./program.kexe into the command line and hit enter.

You should see something like hello world! glfw initialized? = 1, window pointer = CPointer(raw=0x7f858c40c8a0) in the terminal. So glfwInit() returned 1 (i.e. true) and the window, which is accessed via pointer, was created. You should also see an empty black window pop up, like so: Now you have a place to put your 3D OpenGL objects.

You have successfully created a Kotlin program that is capable of using the OpenGL and GLFW C libraries! Thanks to Kotlin/Native, you can take advantage of the functionality provided by platform libraries and external C libraries using the Kotlin programming language.