HawtJNI Developer Guide

Features

Getting Started with HawtJNI

Implementing JNI libraries is a piece of cake when you use HawtJNI. It will code generate the JNI code needed to access the native methods defined in your Java classes. Lets say you wanted to access to the c library's classic printf function:

int printf(const char *format, ...);

To do that, you would only need to define a simple Java class like:

import org.fusesource.hawtjni.runtime.*

@JniClass
public class Simple {
    
  private static final Library LIBRARY = new Library("simple", Simple.class);    
  static {
    LIBRARY.load();
  }

  public static native int printf(String message);
}

That's it. No JNI coding required. It's composed of a static class initializer and a native method interface definition for the printf function. For folks who have done JNI before this looks familiar except for the way the library is loaded. You could have also loaded the library the traditional textbook way:

  static {
    System.loadLibrary("simple");
  }

The benefit of using the Library class to load the native library is that it can automatically unpack the native library from a jar resource and use that so that you don't have to worry installing it to the java library path.

The HawtJNI build process will take care of implementing your simple jni library by using some Maven tooling which we will cover in the next example.

Building with Maven

If you are not familiar with Maven, please checkout Maven by Example.

The easiest way to get started with HawtJNI is copy and use the example Maven project as a template for your module.

At the root of the Maven project run:

mvn install

The Maven build will produce the following artifacts:

These artifacts will be deployed to the Maven repository so that other users can easily use them as dependencies in their builds.

You may also be interesting in How to Add HawtJNI to an Existing Maven Build.

Mapping Native Methods

Overview

HawtJNI looks for all classes annotated with @JniClass. For every static native method found, it will generate the corresponding JNI function which calls a the platform function of the same name as the java method.

The JNI method mapping can be customized by applying the @JniMethod annotation to the method and the @JniArg to each method argument.

Without additional configuration native methods automatically convert the method arguments and return types to the corresponding type of the same size on the platform.

Java Type Native Type Description Windows Types
byte char 8-bit integer BYTE, TCHAR
short short 16-bit integer WORD
char wchar_t 16 or 32-bit character TCHAR
int int 32-bit integer DWORD
long long long 64-bit integer LONG
boolean int boolean value BOOL
float float 32-bit FP
double double 64-bit FP

If a primitive array type or String is used, it gets converted to the corresponding native array/pointer type.

Java Type Native Type Description Windows Types
byte[] char* 8-bit array BYTE, TCHAR
short[] short* 16-bit array WORD
char[] wchar_t* 16 or 32-bit array TCHAR
int[] int* 32-bit array DWORD
long[] long long* 64-bit array LONG
float[] float* 32-bit FP array
double[] double* 64-bit FP array
String char* 8-bit array LPTCSTR

It's important to note that when dealing with arrays and structures, HawtJNI must copy the contents of the java object to the native type since the JVM can any time move java objects in memory. It will then call the native function and then copy back the native array back over the original java array so that the original java array picks up any changes.

When a Java string is converted to a char * it applies a UTF-8 conversion. If your native code can handle wide character (i.e. double byte unicode characters), then you annotate the argument with UNICODE flag. For example:

  public static native int printf(
    @JniArg(flags={UNICODE}) String message);

Passing Primitives by Reference

It is common to run into native methods similar to the following:

void adder(int *result, int left, int right) {
  *result = left + right;
}

They use a pointer to a simple type to store the result of function call. It may not be obvious at first, but this can be mapped in java using a primitive array. For example:

  public static native void adder(int []result, int left, int right);

Just make sure you use the method with 1 element array:

  byte[] result = new byte[1];
  adder(result, 3, 4);
  System.out.println("The result was: "+result[0]);

Mapping Native Structures

You define a Java class for each native structure that you want map and replicate all the fields that you will need to access as regular java fields.

For example, say you had a C structure and function that was defined as follows:

struct COORD {
  int x;
  int y;
};
void display_coord(struct COORD* position);

Then the the corresponding Java class for the structure would look like:

@JniClass(flags={STRUCT})
public static class COORD {
  public short x;
  public short y;
}

The native method definition can then just take COORD java object as an argument.

public static native void display_coord(COORD position);

Nested Structures

Nested native structures are also easy. For example:

struct RECT {
  struct COORD top_left;
  struct COORD bottom_right;
};

Would be mapped as:

@JniClass(flags={STRUCT})
public static class RECT {
  public COORD top_left = new COORD();
  public COORD bottom_right = new COORD();
}

Passing Structures By Value

You probably noticed that structures are passed by reference by default. If your native method accepts the structure by value instead, then you need to annotate the method argument with the BY_VALUE flag.

For example, if your native method was defined as:

int validate_coord(struct COORD position);

Then your Java method mapping would look like

  public static native int validate_coord(
    @JniArg(flags={BY_VALUE}) COORD position);

The passed object MUST not be null.

Using Typedefed Structures

If the structure name your mapping is actually a typedef, in other words, the type is referred to in native code by just the plain name and not the struct name, then you need to add the TYPEDEF flag to struct definition.

For example, if the native definition was:

typedef struct _COORD {
  int x;
  int y;
} COORD;

Then the the corresponding Java class for the structure would look like:

@JniClass(flags={STRUCT,TYPEDEF})
public static class COORD {
  public short x;
  public short y;
}

Zero Out Structures

You do NOT have to map all the native fields in a structure it's corresponding Java structure class. You will actually get better performance if you only map the fields that will be accessed by your Java application.

If not all the fields of the structure are mapped, then when the native structure is created from a Java structure, the unmapped fields will have whatever random data was in the allocated memory location the native structure was allocated on. If you prefer or NEED to zero out unmapped fields, then add the ZERO_OUT flag. For example:

@JniClass(flags={STRUCT,ZERO_OUT})
public static class COORD {
  public short x;
}

Skipping Fields

If you need to have a Java field which is not mapped to a native structure field, annotate it with @JniField(flags={FIELD_SKIP}). This can be useful, to holding java side computations of the structure.

For example, if you want to cache the hash computation of the structure you could do the following:

@JniClass(flags={STRUCT,ZERO_OUT})
public static class COORD {
  public short x;
  public short y;

  @JniField(flags={FIELD_SKIP})
  public int hash;
  public int hashCode() {
    if( hash==0 ) {
      hash = (x << 16) & y;
    }
    return hash;
  }
}

Binding to C++ Classes

HawtJNI support binding to C++ classes. Here's an example of bind to the “std::string” class.

@JniClass(name="std::string", flags={ClassFlag.CPP})
private static class StdString {

    @JniMethod(flags={CPP_NEW})
    public static final native long create();

    @JniMethod(flags={CPP_NEW})
    public static final native long create(String value);

    @JniMethod(flags={CPP_DELETE})
    static final native void delete(long self);

    @JniMethod(flags={CPP_METHOD}, cast="const char*")
    public static final native long c_str_ptr (long self);

    @JniMethod(flags={CPP_METHOD}, cast="size_t")
    public static final native long length (long self);
}

The CPP_NEW flagged methods are constructor methods. They allocate a new instance of the class or structure on the native heap and return a pointer to it.

The CPP_DELETE flagged method is used to destruct a previously created object. The CPP_METHOD flagged methods are used to call methods on the object.

You can also get and set fields on the object using the GETTER and SETTER method flags. For example, if we had an object with a int count field.

    @JniMethod(flags={GETTER})
    public static final native int count(long self);

    @JniMethod(flags={SETTER})
    public static final native void count(long self, int value);

Notice that the CPP_DELETE, CPP_METHOD, GETTER, and SETTER all take a pointer the object they operating against as the first argument of the java method definition.

Advanced Mapping Topics

Loading Constants

Many times you need to access the value of platform constants. To load platform constants you will need to:

  1. Define a constant initializer method which when called will sets all the static fields annotated as constant fields with the constant value. Example:

          @JniMethod(flags={CONSTANT_INITIALIZER})
      private static final native void init();

  2. For each constant you want to load define a static field for it with the @JniField(flags={CONSTANT}) annotation. Example:

          @JniField(flags={CONSTANT})
      public static short FOREGROUND_BLUE;

  3. Call the constant initializer method in the class initializer, after the library is loaded. For example:

          private static final Library LIBRARY = new Library("simple", Simple.class);    
      static {
        LIBRARY.load();
        init();
      }

Renaming

If you want to call your java method something different from the native method name, you can set the accessor attribute on a @JniMethod or @JniField annotation.

For example, to call the native printf function from a Java print method, you would set it up as follows:

  @JniMethod(accessor="printf")
  public static native int print(String message);

In the case of field on a structure this can be used to do simple renames or to map field to the field of a nested structure or union:

  // Just a simple rename..
  @JniField(accessor="dwSize")
  public int size;

  // Mapping to a nested field.
  @JniField(accessor="u.pos")
  public int deep;

The accessor also comes in handy when you want to load constants which are not a simple symbol, for example the size of a structure:

  @JniField(accessor="sizeof(struct COORD)", flags={CONSTANT})
  public static int SIZEOF_COORD;

Pointers

If you want the same java class to be able to work with 32 bit and 64 bit pointers, you should map pointers to Java long. If your only targeting 32 bit platforms you can map pointers to int.

When HawtJNI is on a 32 bit platform but is mapping pointers to Java's 64 bit longs, it needs to know that the value it's working with is a pointer so that it properly up and down casts the pointer value.

This is typically done by setting the cast attribute on the the @JniMethod, @JniArg and @JniField annotations to the pointer type. In general it's good practice to set the cast attribute to the defined type of value.

For example, to let HawtJNI know that the malloc function returns a void pointer you would define it as follows:

  @JniMethod(cast="void *")
  public static final native long malloc(
    @JniArg(cast="size_t") long size);

If the cast end with * and it's being mapped to a Java long then HawtJNI knows that it's dealing with a pointer type. But if your using a typedef to pointer, then you need to flag the condition with one of the following:

This is very common on the Windows platform where tend to typedef pointer types like LPTCSTR.

You may be tempted to do pointer arithmetic on the java side long value, but DON'T. The native pointer is a combination of signed/unsigned, and 32/64 bit value which may more may not match java's memory model. Adding offsets to the pointer on the java side will likely result in a invalid pointer location.

Using the Native Heap

The memory associated with a passed structure or array is reclaimed at the end of every native method call. Therefore, a method expects a passed array or structure reference to remain valid after the method call then that array or structure needs to be allocated on the native heap instead.

This is accomplished using the standard native malloc function call. For example to create a char array 80 bytes big:

  long buffer = malloc(80);

To create a structure on the the heap, your going to need a couple of helper methods.

  1. You need to know the size of the structure.
  2. You need define a versions of memmove which copy to and from the your Java structure object and memory buffer (void *).

I recommend keeping those defined in the structure class itself. For example:

@JniClass(flags={STRUCT})
public static class COORD {

  public short x;
  public short y;

  // To hand loading the SIZE_OF constant
  @JniMethod(flags={CONSTANT_INITIALIZER})
  private static final native void init();
  @JniField(flags={CONSTANT}, accessor="sizeof(struct COORD)")
  public static short SIZE_OF;
  static {
    init();
  }

  public static final native void memmove (
    @JniArg(cast="void *", flags={NO_IN, CRITICAL})) COORD dest, 
    @JniArg(cast="const void *", flags={NO_OUT, CRITICAL}) long src, 
    @JniArg(cast="size_t") long size);

  public static final native void memmove (
    @JniArg(cast="void *", flags={NO_IN, CRITICAL})) long dest, 
    @JniArg(cast="const void *", flags={NO_OUT, CRITICAL}) COORD src, 
    @JniArg(cast="size_t") long size);
}

Now you can allocate and initialize a structure on the heap:

  long buffer = malloc(COORD.SIZE_OF);
  COORD tmp = new COORD()
  tmp.x = 3;
  tmp.y = 4;
  COORD.memmove(buffer, tmp, COORD.SIZE_OF);

Callbacks: Calling Java methods from native functions.

Some native functions require a callback function pointer as an argument. You can create one using a Callback object. For example, lets say needed to call the following native function:

long foo(void (*fp)(char *buffer, int n));

It you could map it in Java as:

  public static final native void foo (
    @JniArg(cast="void *") long fp);

Next you need to create a java method that can accept the callback. The method MUST return long and can have any number of arguments, but they must also all be longs. The method can be a static or instance method.

For example:

class MyObject {
  public long mymethod(long buffer, long n) {
    System.out.println("Was given a buffer "+n+" byte big at "+buffer);
  }
}

Then to create a function pointer which points back to a Java method, you create a Callback object with a reference to the Java object, method name, and number of arguments on the method takes.

For example:

  MyObject object = new MyObject();
  Callback callback = new Callback(object, "mymethod", 2);
  long fp = callback.getAddress();
  foo(fp);

Warning: you can only create up to 128 Callbacks concurrently. If you exceed this number callback.getAddress() returns zero. You should use the callback.dispose() method to release a callback once it's not being used anymore.

Attaching Native Threads to the JVM

If you have a thread that was not started by the JVM try to call into the JVM you must first “attach” it to the JVM. HawtDispatch provides some helper methods to make it simpler and more efficient. These methods are only available on JVMs supporting JNI 1.2.

If your platform supports pthreads and you have the HAVE_PTHREAD_H define enabled, then the attach operation is cached and the detach is only performed when the thread stops.

Optimizations

If you have performance sensitive method that works on an Array or Structure, setting on of the following flags may help your performance.

These optimization flags are typically used for the memmove C library function. for example, to copy the the contents of a byte[] into a java int[], you would define the memmove method as follows:

  public static final native void memmove (
    @JniArg(cast="void *", flags={NO_IN, CRITICAL})) int[] dest, 
    @JniArg(cast="const void *", flags={NO_OUT, CRITICAL}) byte[] src, 
    @JniArg(cast="size_t") long size);

Conditionals

You can use the conditional attribute on the @JniClass, @JniMethod and @JniField annotations to control if JNI code associated with the class, method, or field gets conditionally compiled out.

This is very useful if your mapping to a native function, structure, or field that may not be available on all the platforms which your Java class is going to be made available to.

Example:

  @JniMethod(conditional="defined(_WIN32)")
  public static native int printf(String message);

Would produce a JNI printf method implementation which is surrounded by the following pre-processor directives:

  #if defined(_WIN32)
  // ... the JNI method implementation would be here
  #endif

The conditional on a @JniMethod or @JniField defaults to value configured on the enclosing @JniClass. So if most of the fields or methods in a class need to have have the same conditional applied, just set it on the @JniClass annotation.

Maven Plugin Reference

The HawtJNI provides a Maven plugin which makes it easy code generate and build the native library for your current platform.

The Maven tooling takes care of:

Usage

Once you have a working Maven build for a java module, you can then update it to use HawtJNI to generate your JNI libraries. With the following steps:

  1. Add the hawtjni-runtime dependency to pom. This small 19k jar file contains the HawtJNI annotations and a few helper classes that you will be developing against.

        <pom>
      <dependencies>
        ...
        <dependency>
          <groupId>org.fusesource.hawtjni</groupId>
          <artifactId>hawtjni-runtime</artifactId>
          <version>1.17</version>
        </dependency>
        ...
      </dependencies>
    <pom>

  2. Add the HawtJNI Maven plugin to the pom.

        <project>
      <build>
        <plugins>
          ...
          <plugin>
            <groupId>org.fusesource.hawtjni</groupId>
            <artifactId>hawtjni-maven-plugin</artifactId><!-- was maven-hawtjni-plugin until 1.15 -->
            <version>1.17</version>
            <executions>
              <execution>
                <goals>
                  <goal>generate</goal>
                  <goal>build</goal>
                  <goal>package-jar</goal>
                  <goal>package-source</goal>
                </goals>
              </execution>
            </executions>        
          </plugin>
          ...
        </plugins>
      </build>
    <project>

Build Phases

You may have noticed that HawtJNI is generating a slew of code in different directories. Here is a breakdown of what gets generated where and during which Maven build phase:

  1. process-classes: Processes the annotated java classes:

    1. generates JNI .c and .h files to target/generated-sources/hawtjni/native-src

    2. generates an autoconf and msbuild build project in the target/generated-sources/hawtjni/native-package directory

  2. generate-test-resources: Compiles the native library:

    1. The project is built in target/native-build

    2. The project installed the libraries to target/native-dist

    3. The libraries are copied to target/generated-sources/hawtjni/lib which gets added as a test resource path.

  3. package-jar: The contents of target/generated-sources/hawtjni/lib get jarred and attached to the Maven build with a platform specific classifier.

  4. package-source: The contents of target/generated-sources/hawtjni/native-package get zipped up into a platform agnostic source package for building the native library.

Platform Build Tools Requirements

Windows

Download and install the free Microsoft Windows SDK. The SDK includes all the headers, libraries, and build tools needed to compile the JNI library.

Set the JAVA_HOME environment variable to the location where your JDK is installed.

Make sure the msbuild (vcbuild for legacy SDK versions) tool is on in your system PATH. The simplest way is to use SDK command prompt.

Ubuntu Linux

On Ubuntu you need a JDK and the build-essential package installed to do native library builds.

If you want to be able to generate the native-src.zip which contains a GNU style make project, then you will also need the following packages installed:

Install them is by running:

sudo apt-get install build-essential automake1.10 libtool

Fedora Core Linux

On Ubuntu you need a JDK and the and gcc package installed to do native library builds.

Install them is by running:

yum install gcc java-1.6.0-openjdk-devel

If you want to be able to generate the native-src.zip which contains a GNU style make project, then you will also need the following packages installed:

Install them is by running:

yum install automake libtool