How to configure NDK with Android Gradle plugin 0.7

Question

The new Android gradle plugin (0.7) seems to include new support for the NDK, but in the documentation there is little to no mention of it (the only reference I found is a test called ndkSanAngeles).

It looks like gradle is looking for the NDK, which I have included in my PATH. However, building the project fails with

• What went wrong: Execution failed for task ':OGLTests:compileDefaultFlavorDebugNdk'. NDK not configured

How can I configure the NDK in gradle?

My current build.gradle looks like this:

task nativeLibsToJar(type: Zip, description: 'create a jar with native libs') {
    destinationDir file("$buildDir/native-libs")
    baseName 'native-libs'
    extension 'jar'
    from fileTree(dir: 'src/main/libs', include: '**/*.so')
    from fileTree(dir: 'src/main/libs', include: '**/gdb*')
    into 'lib/'
}

tasks.withType(JavaCompile) {
    compileTask -> compileTask.dependsOn nativeLibsToJar
}

dependencies {
    compile fileTree(dir: "$buildDir/native-libs", include: '*.jar')
}

android {
    compileSdkVersion 19
    buildToolsVersion '19.0.0'

    defaultConfig {
        minSdkVersion 14
        targetSdkVersion 19
        versionCode 1
        versionName "0.1"

    }
    buildTypes {
        release {
            runProguard false
        }
        debug {
           // jniDebugBuild true
            runProguard false
            debuggable true
        }
    }
    productFlavors {
        defaultFlavor {
            proguardFile 'proguard-rules.txt'
        }
    }
}

Thanks.

Answer 1

Looking through the gradle plugin code, I found the following that helped me use both NDK and prebuilt native libraries:

To simply Link in Prebuilt Native Libraries, just add an ndk section to your task. For instance, I added it below in productFlavors. The abiFilter is the folder name the libs are stored in. abiFilters means both libs from the comma separated list will be added to your final APK (so you could theoretically have "armeabi", "armeabi-v7a", "x86", and "mips" all in one APK, and the O/S would choose the supported architecture lib on install):

productFlavors {
    arm {
        ndk {
            abiFilters "armeabi", "armeabi-v7a"
        }
    }
    x86 {
        ndk {
            abiFilter "x86"
        }
    }
}

In this example, the arm build will create an APK with the V5 and V7A arm libs in it, and the x86 build will create an APK with just x86 libs in it. This will search for the native libraries in your project jniLibs directory. The jniLibs directory should be structures as the old jni directory, i.e.:

[project]/[app]/src/main/jniLibs/armeabi/libmyNative.so
[project]/[app]/src/main/jniLibs/armeabi-v7a/libmyNative.so
[project]/[app]/src/main/jniLibs/x86/libmyNative.so

Then you can load it in Java as follows:

static
{
    loadLibrary("myNative");
}

Now, let's say one native library depends on another one. You MUST (if setting your min API to API 17 or lower) load the dependent libraries first:

static
{
    loadLibrary("myDependency");
    loadLibrary("myNative");
}

You can also place the ndk {} section in your defaultConfig or a buildType (such as debug or release or whatever else you may use). For example:

buildTypes {
    debug {
        ndk {
            abiFilters "armeabi", "armeabi-v7a"
        }
    }
}

By prebuilt, I mean 3rd party libs you downloaded or a library you built using the NDK toolchain or your own ARM toolchain (not the ndk-build script itself).

In API 18, they fixed a long standing architectural issue that prevented the native lib loader from "automatically" loading dependencies because it didn't know about the application's lib directory (security reasons, etc). In API 18 and above, if myNative depends on myDependency above, you can just call loadLibrary("myNative") and the OS will handle loading myDependency. Don't RELY on this though, until the market penetration of devices running API 17 and below is at a low number acceptable to you.

To explicitly Build NDK Libraries From Source in the current version of Android Studio, you may do the following:

Set the ndk.dir value in your local.properties to point to NDK home as mentioned previously. Does anyone know if you can use env vars directly in local.properties? :)

In your build.gradle file, add something like this to your task (again, can be defaultConfig, debug, release, a productFlavor, etc):

ndk {
    moduleName "myNDKModule"
    stl "stlport_shared"
    ldLibs "log", "z", "m"
    cFlags "-I/some/include/path"
}

This is the basic structure with currently supported types (moduleName, stl, ldLibs, and cFlags). I looked and did not find more than this. There is an issue I believe with ldLibs because it automatically adds "-l" to the front of each field above. You can trick it though (I had to) by saying: ldLibs "log -lz -lm -Wl,-whole-archive -l/path/to/someOtherLib -Wl,-no-whole-archive"

In this line, you're just tagging on to the end of the first parameter to add parameters that don't start with -l, so you can get by for now. In the case above, I'm linking a whole static library into my NDK module for use from within Java. I've asked the google developer to add additional features to allow this or even ability to merge your own Android.mk file into the NDK build process, but as this is all new, it may be a while.

Currently, whatever you put in build.gradle deletes the temp build directory and recreates it every time, so unless you want to download and modify the gradle android plugin source code (which would be fun), there are some "make due"'s like this required to get your stuff copied into the build. The android gradle script that provides this ndk support in essence generates an Android.mk file and builds using the NDK system in a temporary directory.

Sidetracked for a sec. The moduleName should match a c or cpp file in your project under the jni directory like:

[project]/[app]/src/main/jni/myNDKModule.cpp

The stl value should be set to a value of "stlport_shared" or "stlport_static" if you want to use the stlport libraries for C++. You can leave stl out if you don't need extended C++ support. Remember Android provides very basic C++ support by default. For other supported C++ libraries, view your NDK documentation guidelines in the NDK you downloaded. Note that by setting it to stlport_shared here, gradle copies the libstlport_shared.so lib from your NDK's sources/cxx-stl/stlport/libs directory to your APK's lib directories. It also handles the include path in the compiler (technically the gradle doesn't do all of this, but the Android NDK build system). So don't put your own copy of stlport into your jniLibs directory.

Lastly, I think cFlags is pretty obvious.

You can not set ANDROID_NDK_HOME on the Mac OSX (see below), but from some research I've done appears maybe this still works on other OSs. It will be removed though.

I wanted to comment but don't have the reputation yet. Dennis, the environment variables are ignored completely, not just overridden. In fact, you don't get any of your environment variables. From what I can tell, the Android Studio IDE creates its own environment with just a few specific environment variables (check System.getenv() and print it out from a gradle script).

I wrote this up as a bug here because using env vars builds fine from cmd line:
https://code.google.com/p/android/issues/detail?id=65213

but as you can see, Google decided they didn't want environment variables being used by the IDE at all; I'm still on the fence on that decision. It makes my life painful to have to update local.properties to point to absolute paths that can be loaded in my gradle scripts, which I still haven't figured out how to do yet (but haven't looked that hard). That means I either force my team members to use the same path as me, play with links, make them all type them in every time they pull the repo, or add an automation script. I believe it's a bad decision that will cost time for any developers that rely on env vars which may be small at the micro level but huge at the macro level.

groundloop, I believe the IDE will be updated soon with the ability to add the NDK folder path to your project, and it will auto generate the local.properties file from that (at least it wouldn't make sense if they had not thought of this).

For more detailed examples from Google, here are the latest examples (search for jni or ndk): https://docs.google.com/viewer?a=v&pid=sites&srcid=YW5kcm9pZC5jb218dG9vbHN8Z3g6NDYzNTVjMjNmM2YwMjhhNA

cross-platform fat APKs using NDK:

Lastly, there is a drawback to using gradle and not being able to provide your own Android.mk file so that you could only link in 3rd party native libraries from a single architecture to your NDK. Note I said "link in". You can build the NDK Modules (moduleName above) in several architectures with the "abiFilters" command, and they will be placed in your app such that the same APK could be used on multiple architectures. If you need to link in your own 3rd party libs or even have different values for cFlags depending on your architecture, there's not a simple way.

I tried the following and it appeared to work at first, but then I found out it was simply building the NDK by appending everything together from the two ndk sections (or something like that, it did somehow build multiple architecture libraries though):

android {
    compileSdkVersion 23
    buildToolsVersion '23.0.1'
    defaultConfig {
        minSdkVersion 14
        targetSdkVersion 23
        versionCode 28
        versionName "3.0"
    }
    buildTypes {
        def commonLibs = " -lfoo -lbar -lwhatever"
        def armV7LibsDir = "/whatever/armv7a/libs"
        def armX86LibsDir = "/whatever/x86/libs"
        def armV7IncDir = "/whatever/armv7a/include"
        def x86IncDir = "/whatever/x86/include"
        debug {
            ndk {
                cFlags = "-I" + armV7IncDir
                moduleName "myNativeCPPModule"
                stl "stlport_shared"
                abiFilter "armeabi-v7a"
                ldLibs "log -L" + armV7LibsDir + commonLibs
            }
            ndk {
                cFlags = "-I" + armX86IncDir
                moduleName "myNativeCPPModule"
                stl "stlport_shared"
                abiFilter "x86"
                ldLibs "log -L" + armX86LibsDir + commonLibs
            }
        }
    }
}

After much grief trying to create a fat binary in a clean manor with gradle and native 3rd party libs, I finally came to the conclusion that Google Play's built-in multi-architecture support for APKs is really the best route to go anyway, so create individual APKs for each architecture.

So I created multiple buildTypes, no product flavors, and added the following code to generate the version code for each type.

// This is somewhat nasty, but we need to put a "2" in front of all ARMEABI-V7A builds, a "3" in front of 64-bit ARM, etc.
// Google Play chooses the best APK based on version code, so if a device supports both X86 and
// ARM, it will choose the X86 APK (preferred because Inky ARM running on an X86 with Houdini ARM Emulator crashes in our case)
android.applicationVariants.all { variant ->
    if (variant.buildType.name.equals('release')) {
        variant.mergedFlavor.versionCode = 2000 + defaultConfig.versionCode
    } else if (variant.buildType.name.equals('debug')) {
        variant.mergedFlavor.versionCode = 2000 + defaultConfig.versionCode
    } else if (variant.buildType.name.equals('debugArmV8a')) {
        variant.mergedFlavor.versionCode = 3000 + defaultConfig.versionCode
    } else if (variant.buildType.name.equals('releaseArmV8a')) {
        variant.mergedFlavor.versionCode = 3000 + defaultConfig.versionCode
    } else if (variant.buildType.name.equals('debugMips')) {
        variant.mergedFlavor.versionCode = 5000 + defaultConfig.versionCode
    } else if (variant.buildType.name.equals('releaseMips')) {
        variant.mergedFlavor.versionCode = 5000 + defaultConfig.versionCode
    } else if (variant.buildType.name.equals('debugMips64')) {
        variant.mergedFlavor.versionCode = 6000 + defaultConfig.versionCode
    } else if (variant.buildType.name.equals('releaseMips64')) {
        variant.mergedFlavor.versionCode = 6000 + defaultConfig.versionCode
    } else if (variant.buildType.name.equals('debugX86')) {
        variant.mergedFlavor.versionCode = 8000 + defaultConfig.versionCode
    } else if (variant.buildType.name.equals('releaseX86')) {
        variant.mergedFlavor.versionCode = 8000 + defaultConfig.versionCode
    } else if (variant.buildType.name.equals('debugX86_64')) {
        variant.mergedFlavor.versionCode = 9000 + defaultConfig.versionCode
    } else if (variant.buildType.name.equals('releaseX86_64')) {
        variant.mergedFlavor.versionCode = 9000 + defaultConfig.versionCode
    }
}

Now all you have to do is set the value for versionCode in your defaultConfig object, as you would do normally, and this appends it to the end of the architecture-specific version string, based on build type. The version string remains the same for all builds but varies the code to provide an order of precedence from ARM all the way up to X86_64. It's a bit hackish or hard-coded, but it gets the job done. Note that this provides you with up to 999 versions, so if you need more, multiply the numbers above by 10, not sure what the max value is that you can put in for the version code.

In my case, we have a fairly complex build system. We build CPython for 9 architectures, 3 of which are Android, then build a bunch of our own libraries, and link them all together into a single library for each architecture. We use the ndk command line build tools, automake, and python to build everything, instead of Android.mk files. The final libraries are then linked into a single JNI interface cpp file (called myNativeCPPModule above). One click of the button, and everything is built all at once, very nice Android Studio.