In this post, we’ll explore how well the Kotlin/Native compiler handles both pre-built and non-pre-built libraries.

Pre-built Libraries

The Kotlin/Native distribution includes “Platform Libraries”, which are a set of prebuilt libraries specific to each target.

A few of the prebuilt libraries that come “out-of-the-box” when you download Kotlin/Native are:

• POSIX (available for all platforms except WebAssembly)
• OpenGL
• Zlib
• Gzip
• Metal
• Foundation … along with many other libraries and Apple frameworks.

Source: Kotlin/Native Interoperability

How it Works

According to kotlinlang.org:

“The Kotlin/Native compiler automatically detects which of the platform libraries have been accessed and automatically links the needed libraries. … [Platform] libs in the distribution are merely just wrappers and bindings to the native libraries. That means the native libraries themselves (.so, .a, .dylib, .dll, etc.) should be installed on the machine.”

This means that you can write a Kotlin program that uses any of the supported platform libraries. Since the platform libraries are wrappers and bindings to the native libraries, any accesses to the platform libraries will be routed to where the actual implementation of the native library is located on the machine. In this way, the program is able to access the operating system services of the platform that it targets.

No additional action needs to be taken in order to utilize the platform libraries. Platform libraries seem easy and straightforward to use, so let’s try it out.

Test Driving a Platform Library

There are several platform libraries to choose from. OpenGL appeared to be the most interesting to me, so we’ll experiment with it. (Note: Apple deprecated OpenGL in 2018 in favor of Metal, but we are using OpenGL 3.3, the version upon which all future versions of OpenGL are based; macOS supports up to OpenGL 4.1.)

First, we need to build a window and an OpenGL context. Some background information about OpenGL is strongly recommended for this tutorial, but not strictly required. Chapter 2 of LearnOpenGL is a great source of background information and serves as inspiration for this tutorial.

We could manually program our window and context, or we can use a pre-existing library that contains functions that can create a window and an OpenGL context for us. Some commonly used libraries which handle these tasks are GLUT, SDL, SFML and GLFW. I noticed that GLUT is included in the macOS platform libraries. For Linux, the equivalent library will need to be installed on the machine. Regardless of the platform, the libraries need to exist directly on the machine. The actual implementation of the libraries is not included in the platform libraries, as the platform libraries are merely wrappers/bindings to actual libraries. OpenGL and GLUT come with the OS and Xcode installations.

JetBrains provides a neat, simple demonstration of this out-of-the-box capability, which can be found here.

Figure 1. Running JetBrains’s OpenGL sample.

Non-pre-built Libraries

What if I want to use a library that is not pre-included in the platform libraries? What if I wanted to use GLFW instead of GLUT so that I can follow along in the OpenGL tutorial?

We will need to create Kotlin bindings from the C library. This example is a good reference. We’ll use the cinterop tool. “It takes a C library and generates the corresponding Kotlin bindings for it, which then allows us to use the library as if it were Kotlin code.”

First, download the GLFW source files. You can download the source package or clone the git repo. Note: We’ll be using a macOS. The steps in this tutorial may not translate directly to Windows or Linux operating systems.

If you want to produce a library compatible with all three platforms, you have to create bindings that will act as an interface to the real library (written in C/C++). There’s some good documentation about how to produce the .klib file to build our program with.

• C Interop: Official documentation on how to use Kotlin/Native’s cinterop tool to generate everything your Kotlin program needs to interact with an external library.
• Kotlin/Native Libraries: Basic instructions on how to produce a library with the Kotlin/Native compiler and how to link the library to your Kotlin program; also contains documentation about helpful klib utilities included in Kotlin/Native.

We’ll use these resources to produce our own .klib file.

The first step before we start building the library is to create a Kotlin program that will use GLFW. I created a file named test.kt. In test.kt, import the OpenGL platform libraries.

import kotlinx.cinterop.*
import platform.OpenGL.*
import platform.OpenGLCommon.*

Below these import statements, import the GLFW libraries and instantiate a window.

...
import glfw.*
fun main() {
    var glfwInitialized = glfwInit()
    if (glfwInitialized == 0) {
        println("Failed to initialize GLFW")
        return
    }
    var window = glfwCreateWindow(640, 480, "OpenGL on Kotlin! Wow!", null, null);
    if (window == null)
    {
        println("Failed to create GLFW window")
        glfwTerminate()
        return
    }
    println("hello world! glfw initialized? = $glfwInitialized, window pointer = $window")
    glfwMakeContextCurrent(window)
    while(glfwWindowShouldClose(window) == 0)
    {
        glClear(GL_COLOR_BUFFER_BIT)
        glfwSwapBuffers(window);
        glfwPollEvents();
    }
    glfwTerminate()
    return
}

This program uses the function glfwInit(), among others, whose actual implementation is contained inside the GLFW C library. We used this program to test that the Kotlin program can properly access the GLFW library.

Next, you need to define the .def file. It is important that you get this part right. If you do not have the proper contents included in your custom .def file, the cinterop tool may not be able to generate the necessary files properly. You will know if you did something wrong if you are trying to use a function or property belonging to the C library in your Kotlin program and you receive compile-time errors stating that the function or property is not recognized. A good way to check if the wrapper/bindings are being generated correctly is by using the klib contents <library-name> command (explained in more detail later). If you are inducing C interoperability on a library, you must ensure that you include all of the necessary header files and include the proper compiler and linker options. To know if you are doing this right may require some background knowledge about how to build and link the particular C library you are using. For the sake of this demonstration, however, the contents of the .def file will be provided to you.

Create a glfw.def file. The contents should look like this:

headers = /usr/local/include/GLFW/glfw3.h /usr/local/include/GLFW/glfw3native.h
compiler-options = -framework OpenGl
package = glfw
linkerOpts.osx = -L/opt/local/lib -L/usr/local/lib -lglfw
linkerOpts.linux = -L/usr/lib64 -L/usr/lib/x86_64-linux-gnu -lglfw
linkerOpts.mingw = -lglfw

Note: According to the documentation on Creating Bindings for a New Library, if you wish to insert compiler options that are specific to a particular platform, you can set fields such as compilerOpts.osx and compilerOpts.linux equal to the options you would like to specify.

To avoid lengthy terminal commands, we are going to add the Kotlin/Native distribution to the system PATH. (If you already added Kotlin/Native to the PATH, you can skip this part.) When adding the distribution to the PATH, you are telling the terminal to look in this location (along with the other locations specified in your PATH) when using terminal commands (such as cinterop and kotlinc-native). The terminal needs to know where to find the scripts behind the commands in order to run them. If you do not add the distribution location path to your PATH, the terminal will not recognize your cinterop and kotlinc-native commands when you try to use them (unless you type the entire file path specifying the location of the respective scripts). First, identify the location of your Kotlin/Native distribution that you downloaded. I downloaded my instance of the Kotlin/Native distribution from the releases page (scroll all the way down and you can see links to download Kotlin/Native for Windows, Linux, and macOS).

Copy the file path to wherever you downloaded and unzipped your Kotlin/Native distribution. You can add the distribution location to the PATH temporarily or permanently.

The Temporary Way

Enter the following command into your terminal, substituting <path-to-your-kotlin-native-distribution> with the file path leading to your Kotlin/Native distribution located on your local machine. Don’t forget to append /bin.

export PATH=$PATH:<path-to-your-kotlin-native-distribution>/bin

To verify that your path was added to PATH, you can run:

echo $PATH

and check that your distribution location was added to the PATH variable. File paths are separated by the : delimiter.

The Permanent Way

Run:

open ~/.bash_profile

or, if the file does not exist:

touch ~/.bash_profile
open ~/.bash_profile

Append the following text to the subsequent file that opens, substituting <path-to-your-kotlin-native-distribution> with the file path leading to your Kotlin/Native distribution located on your local machine:

PATH=$PATH:<path-to-your-kotlin-native-distribution>/bin

Save the file. Close the terminal and open a new session for the changes to take effect. Verify that your path was added to PATH by running:

echo $PATH

Now you should be able to use the cinterop and kotlinc-native commands without issue.

Set Up C Interoperability and Link the Library

Run the cinterop command:

$ cinterop -def glfw.def -compiler-option -I/usr/local/include -o glfw3

You can verify if the bindings were generated correctly by running:

$ klib contents glfw3

You should see a bunch of code containing type aliases, constants, function headers, etc. output to the terminal. The code excerpt below displays snippets of glfw3.klib‘s contents. Notice that the code is in Kotlin. This output indicates that the Kotlin bindings were generated. These bindings correspond to the properties and functions present in the C library. If you run the klib contents command but nothing outputs, it likely means that something went wrong. You should check that the right header files and options were declared in the .def file in the appropriate fields. Refer to the library’s documentation for guidance on properly building and linking the library.

package glfw {
@CStruct(spelling = "struct { unsigned char p0[15]; float p1[6]; }") class GLFWgamepadstate constructor(rawPtr: NativePtr /* = NativePtr */) : CStructVar {
    val axes: CArrayPointer<FloatVar /* = FloatVarOf<Float> */> /* = CPointer<FloatVarOf<Float>> */
    val buttons: CArrayPointer<UByteVar /* = UByteVarOf<UByte> */> /* = CPointer<UByteVarOf<UByte>> */
    companion object : CStructVar.Type
}
...

typealias GLFWcharfun = CPointer<CFunction<(CPointer<GLFWwindow>?, UInt) -> Unit>>
typealias GLFWcharfunVar = CPointerVarOf<GLFWcharfun>
typealias GLFWcharmodsfun = CPointer<CFunction<(CPointer<GLFWwindow>?, UInt, Int) -> Unit>>
typealias GLFWcharmodsfunVar = CPointerVarOf<GLFWcharmodsfun>
typealias GLFWcursorenterfun = CPointer<CFunction<(CPointer<GLFWwindow>?, Int) -> Unit>>
...

package glfw {
const val GLFW_ACCUM_ALPHA_BITS: Int = 135178
const val GLFW_ACCUM_BLUE_BITS: Int = 135177
const val GLFW_ACCUM_GREEN_BITS: Int = 135176
...

package glfw {
const val GLFW_KEY_EQUAL: Int = 61
const val GLFW_KEY_ESCAPE: Int = 256
const val GLFW_KEY_F: Int = 70
const val GLFW_KEY_F1: Int = 290
...

package glfw {
const val GLFW_OPENGL_API: Int = 196609
const val GLFW_OPENGL_COMPAT_PROFILE: Int = 204802
const val GLFW_OPENGL_CORE_PROFILE: Int = 204801
const val GLFW_OPENGL_DEBUG_CONTEXT: Int = 139271
...

@CCall(id = "knifunptr_glfw77_glfwCreateCursor") external fun glfwCreateCursor(image: CValuesRef<GLFWimage>?, xhot: Int, yhot: Int): CPointer<GLFWcursor>?
@CCall(id = "knifunptr_glfw78_glfwCreateStandardCursor") external fun glfwCreateStandardCursor(shape: Int): CPointer<GLFWcursor>?
@CCall(id = "knifunptr_glfw25_glfwCreateWindow") external fun glfwCreateWindow(width: Int, height: Int, @CCall.CString title: String?, monitor: CValuesRef<GLFWmonitor>?, share: CValuesRef<GLFWwindow>?): CPointer<GLFWwindow>?
@CCall(id = "knifunptr_glfw22_glfwDefaultWindowHints") external fun glfwDefaultWindowHints()
@CCall(id = "knifunptr_glfw79_glfwDestroyCursor") external fun glfwDestroyCursor(cursor: CValuesRef<GLFWcursor>?)
@CCall(id = "knifunptr_glfw26_glfwDestroyWindow") external fun glfwDestroyWindow(window: CValuesRef<GLFWwindow>?)
...

Once the glfw3.klib file is generated, install the library by running:

$ klib install glfw3.klib

Note: The klib install command will install the associated library files to the default repository. The headers will be installed to /usr/local/include/GLFW and the library files will be installed to /usr/local/lib for macOS. For Linux, the corresponding install locations are /opt/local/include/GLFW and /opt/local/lib.

Compile the program, linking our program to the GLFW library.

$ kotlinc-native test.kt -l glfw3

(This outputs an executable file named program.kexe by default. To specify a name, append -o program-name to the command, substituting “program-name” with your desired program name.)

To run the program, type ./program.kexe into the command line and hit enter.

You should see something like hello world! glfw initialized? = 1, window pointer = CPointer(raw=0x7f858c40c8a0) in the terminal. So glfwInit() returned 1 (i.e. true) and the window, which is accessed via pointer, was created. You should also see an empty black window pop up, like so: Now you have a place to put your 3D OpenGL objects.

You have successfully created a Kotlin program that is capable of using the OpenGL and GLFW C libraries! Thanks to Kotlin/Native, you can take advantage of the functionality provided by platform libraries and external C libraries using the Kotlin programming language.

The post Exploring Kotlin/Native – Part 2 – Interoperability appeared first on Big Nerd Ranch.

After all, Java’s phenomenal widespread popularity (and profitability) in the enterprise world was fueled by its ability to run on any machine, eliminating the need to replicate software tailored to different hardware platforms. The creators of Java, Sun Microsystems, had a memorable and catchy slogan: “Write once, run anywhere.” This implied that Java could be written by anyone and run on any device — so long as that device was equipped with a Java Virtual Machine (JVM).

This caveat can prove to be a costly obstacle because, for some devices, the need to have a JVM pre-installed is impractical or impossible. Embedded devices with limited memory may not have the capacity to house a JVM. Some operating systems, such as iOS, simply do not allow it.

The Kotlin compiler has a method to compile the language to native binaries of several popular operating systems, eliminating the need for a special virtual machine to be installed on the device. For a handful of supported operating systems, the Kotlin compiler can remove the need to download and install a virtual machine in order to run your Kotlin program. In theory, this means that you can write a Kotlin program and compile it to produce eight different executable files that can run natively on each of their respective platforms, right?

In this blog series, we will explore Kotlin/Native in-depth: What it is, what it can do, and what it can’t.

Expectation:

Through my research, I found that some Kotlin/Native target platforms must be built on the appropriate host machine. Windows targets, for example, must be built on a Windows host. So you cannot create native binaries for all eight platforms using only one host machine.

Figure 1. Informational message received when building a Kotlin/Native project in IntelliJ IDEA. The Windows Kotlin/Native target cannot be built on a host running MacOS.

Since I am operating on a MacBook, let’s instead explore creating native binaries that can be compiled from MacOS, i.e. Mac, Linux, and iosARM native binaries.

Test Driving Kotlin/Native

The Kotlin/Native compiler is already included in the Kotlin/Native distribution in the form of a command line tool. Using the command line is straight-forward when compiling for a target platform that is the same as the host machine, but we are trying to take one Kotlin source file and compile it down to eight native binary files that can run on the eight supported hardware platforms. Additionally, compiling one Kotlin source code file as opposed to compiling an entire project with multiple Kotlin source code files is unlikely in the real world. In the scenario where multiple source files and libraries need to be compiled, it is easier to use an IDE and Gradle build system. IntelliJ is a great choice because it allows you to compile multiple source files in a project at once, and can allow you to target multiple platforms for the compilation of those source files simultaneously by using Gradle. Because it is overall more practical to use the Kotlin/Native compiler via IntelliJ, we will also be using IntelliJ to accomplish our objective.

I followed the instructions on Creating a new Kotlin/Native project in IntelliJ IDEA, but added support for the other platforms in the Gradle file in the Kotlin block:

plugins {
   id 'org.jetbrains.kotlin.multiplatform' version '1.3.61'
}
repositories {
   mavenCentral()
}
kotlin {
   // For ARM, should be changed to iosArm32 or iosArm64
   // For Linux, should be changed to e.g. linuxX64
   // For MacOS, should be changed to e.g. macosX64
   // For Windows, should be changed to e.g. mingwX64
   macosX64("macos") {
       binaries {
           executable {
               // Change to specify fully qualified
               // name of your application's entry point:
              entryPoint = 'sample.main'
               // Specify command-line arguments, if necessary:
               runTask?.args('')
           }
       }
   }
   linuxX64("linux") {
       binaries {
           executable {
               entryPoint = 'sample.main'
               runTask?.args('')
           }
       }
   }
   mingwX64("windows") {
       binaries {
           executable {
               entryPoint = 'sample.main'
               runTask?.args('')
           }
       }
   }
   iosArm64("ARM") {
       binaries {
           executable {
               entryPoint = 'sample.main'
               runTask?.args('')
           }
       }
   }
   sourceSets {
       // Note: To enable common source sets please 
       // comment out 'kotlin.import.noCommonSourceSets' property
       // in gradle.properties file and re-import your project in IDE.
       macosMain {
       }
       macosTest {
       }
   }
}
// Use the following Gradle tasks to run your application:
// :runReleaseExecutableMacos - without debug symbols
// :runDebugExecutableMacos - with debug symbols

I tried to build but was hit with this error:

Execution failed for task ':compileKotlinMacos'.
> Process 'command '/Applications/IntelliJ IDEA CE.app/Contents/jbr/Contents/Home/bin/java''
> finished with non-zero exit value 2

If you see this error message, you may be missing Xcode Command Line Tools. Here’s how you can set them:

Open Xcode. In the top menu bar, navigate to Xcode > Preferences. In the window that opens, select the Locations tab in the top menu bar.

In the dropdown menu next to “Command Line Tools,” select the latest version available. This should fix the issue.

Now it is time to build. Hit the green, hammer-shaped build button towards the top right of the IDE window. 

The program should compile without issue. You may notice that nothing seemed to happen. No new folders were generated for our platforms. What gives?

Once you build the program, the IDE compiles the source code and produces an executable file. You can find this executable file in the project’s build files.

Figure 2. The executable file is placed in the ./build/bin/macos/releaseExecutable directory of your project files.

Assuming you are currently in your project root file, you can go to ./build/bin/macos and you’ll be able to find your runnable executable file in releaseExecutable. You can even run it directly in the terminal.

Alternatively, you can run the program in the IDE. To run the program in the IDE, double click the runReleaseExecutableMacos task in the Gradle tasks menu. To get to the Gradle task menu, click the Gradle side tab located on the right side of the IDE window. Open your project’s drop-down and then open Tasks > run and you should find the run task for the MacOS executable.

As you can see in the Run view, the executable ran successfully!

Creating Multiple Native Binaries from a Common Source

Next, we will create a common source module. Comment out the kotlin.import.noCommonSourceSets=true line in the gradle.properties file.

Next, in the project view located in the left hand drawer menu, create a new Directory.

Once you click Directory, a pop-out options menu should appear. Select commonMain/kotlin and commonTest/kotlin and hit enter. To select multiple items, hold down the control key on Windows or command key on Mac while selecting the items with your mouse.

You should see some new folders appear: commonMain and commonTest.

These folders are for our common source code. We will place our Kotlin program in the commonMain/kotlin directory and compile it down to native binaries that target the Mac, Linux, and iOS (ARM) platforms.

After making a common Kotlin source file, like so:

I made a few changes to the build.gradle file:

kotlin {
   // For ARM, should be changed to iosArm32 or iosArm64
   // For Linux, should be changed to e.g. linuxX64
   // For MacOS, should be changed to e.g. macosX64
   // For Windows, should be changed to e.g. mingwX64
   macosX64("macos") {
       binaries {
           executable {
               // Change to specify fully qualified
               // name of your application's entry point:
              entryPoint = 'main'
               // Specify command-line arguments, if necessary:
               runTask?.args('')
           }
       }
   }
   linuxX64("linux") {
       binaries {
           executable {
               entryPoint = 'main'
               runTask?.args('')
           }
       }
   }
   mingwX64("windows") {
       binaries {
           executable {
               entryPoint = 'main'
               runTask?.args('')
           }
       }
   }
   iosArm64("ARM") {
       binaries {
           executable {
               entryPoint = 'main'
               runTask?.args('')
           }
       }
   }
   sourceSets {
       // Note: To enable common source sets please
       // comment out 'kotlin.import.noCommonSourceSets' property
       // in gradle.properties file and re-import your project in IDE.
       commonMain {
           dependencies {
               implementation(kotlin("stdlib-common"))
           }
       }
       commonTest {
           dependencies {
               implementation(kotlin("test-common"))
               implementation(kotlin("test-annotations-common"))
           }
       }
   }
}

I rebuilt the project and verified that native binaries were produced for each of the three target platforms I specified in the build.gradle file, except for Windows. Since I am using a Mac to compile the source code, the compiler just ignores the Windows instructions in build.gradle.

Voila! Executables were created for all three platforms.

Reality:

In reality and for more complex projects, multi-platforming will likely require platform-specific code to be implemented, especially for Android and iOS. So the multi-platforming structure probably looks closer to this (ignoring host machine requirements):

Next time, we can try to build a program using different libraries to see how Kotlin/Native will hold up under the added complexity.

Conclusion

Kotlin/Native allows you to compile a single Kotlin program to eight different types of native binaries. Each binary file produced is intended to run on a specific target platform. The only caveat is that some targets require a specific type of host in order to compile the native binary. So if you wanted to compile eight different native binaries, one for each of the supported eight platforms, you would have to make sure you compile each one on the specific hardware platform it requires. Fortunately, having access to Windows and Mac systems will allow you to cover the most commonly used supported platforms. Alternatively, you could try compiling on a virtual machine or using an automated build service such as CircleCI. Apart from the emerging multiplatform capabilities, you will have the option to create Kotlin programs for a variety of platforms with direct access to native libraries and frameworks.

The post Exploring Kotlin/Native – Part 1 appeared first on Big Nerd Ranch.