Build Gradle repository for offline development

Question

I am working on implementing a Gradle build system for a piece of software that has parts that are developed in area without Internet connectivity or the ability to install a Maven/Ivy server (like Nexus). To support development in these environments, I am putting together a Gradle plugin that allows the generation of an "Offline Workspace".

I originally implemented this functionality by triggering the resolution of each configuration in the project (triggering the download of all dependencies), then traversing the entire dependency tree of each configuration and copying the local cached copy of the dependency into the Offline Workspace. (A Copy task was generated for each copy operation.) These JARs would then be referenced using a flatDir repository.

This implementation performed its job using an afterEvaluate block. While this worked fine in Gradle 2.0, it triggers a deprecation warning in Gradle 2.2.1 because triggering the resolution is somehow seen as modifying a configuration after it has already been resolved (Attempting to change configuration ':core:runtime' after it has been included in dependency resolution. This behaviour has been deprecated and is scheduled to be removed in Gradle 3.0). In all, this approach feels rather hacky since it also requires me to modify the build.gradle files to explicitly list all transitive dependencies since there are no POM files available to properly specify dependencies.

A more elegant approach seems like it would build a local Maven repository of all dependencies (including POM files, source JARs, javadoc JARs, etc) and then just use the mavenLocal() repository type. Unfortunately, I'm not sure how to do this properly where I don't need to trigger artifact resolution in order to perform this operation.

Is there some better way that I can achieve the full artifact download into an easy-to-package way than just zipping up my entire $USER_HOME/.gradle directory?

Answer 1

To have an offline build you need somehow to provide all required dependencies. One of the options here is just to commit those jars into version control. The hard part is to collect all those dependencies. For that it's possible to have a build.gradle file that can operate in two modes (online and offline):

buildscript {
    repositories {
        if ('allow' == System.properties['build.network_access']) {
            mavenCentral()
        } else {
            maven {
                url 'dependencies'
            }
        }
    }
    dependencies {
        classpath 'com.android.tools.build:gradle:1.2.0-beta2'
    }
}

To run in offline mode type:

gradle --offline

And to run in online mode:

gradle -Dbuild.network_access=allow

And to collect all dependencies use this script that will run gradle in online mode, fetch dependencies to cache inside ${project_dir}/.gradle_home and copy artifacts to local maven repository in dependencies folder.

#!/usr/bin/python

import sys
import os
import subprocess
import glob
import shutil

# Place this in build.gradle:
# repositories {
#     if ('allow' == System.properties['build.network_access']) {
#         mavenCentral()
#     } else {
#         maven { url 'dependencies' }
#     }
# }
def main(argv):
    project_dir = os.path.dirname(os.path.realpath(__file__))
    repo_dir = os.path.join(project_dir, "dependencies")
    temp_home = os.path.join(project_dir, ".gradle_home")
    if not os.path.isdir(temp_home):
        os.makedirs(temp_home)
    subprocess.call(["gradle", "-g", temp_home, "-Dbuild.network_access=allow"])
    cache_files = os.path.join(temp_home, "caches/modules-*/files-*")
    for cache_dir in glob.glob(cache_files):
        for cache_group_id in os.listdir(cache_dir):
            cache_group_dir = os.path.join(cache_dir, cache_group_id)
            repo_group_dir = os.path.join(repo_dir, cache_group_id.replace('.', '/'))
            for cache_artifact_id in os.listdir(cache_group_dir):
                cache_artifact_dir = os.path.join(cache_group_dir, cache_artifact_id)
                repo_artifact_dir = os.path.join(repo_group_dir, cache_artifact_id)
                for cache_version_id in os.listdir(cache_artifact_dir):
                    cache_version_dir = os.path.join(cache_artifact_dir, cache_version_id)
                    repo_version_dir = os.path.join(repo_artifact_dir, cache_version_id)
                    if not os.path.isdir(repo_version_dir):
                        os.makedirs(repo_version_dir)
                    cache_items = os.path.join(cache_version_dir, "*/*")
                    for cache_item in glob.glob(cache_items):
                        cache_item_name = os.path.basename(cache_item)
                        repo_item_path = os.path.join(repo_version_dir, cache_item_name)
                        print "%s:%s:%s (%s)" % (cache_group_id, cache_artifact_id, cache_version_id, cache_item_name)
                        shutil.copyfile(cache_item, repo_item_path)
    shutil.rmtree(temp_home)
    return 0

if __name__ == "__main__":
    sys.exit(main(sys.argv))

So, after each dependency change just run this script and commit changes in dependencies folder. Then you can build offline with gradle --offline or just gradle.