Extremely fast builds with Gradle 4

Cédric Champeau (@CedricChampeau), Gradle

Who am I

speaker {
    name 'Cédric Champeau'
    company 'Gradle Inc'
    oss 'Apache Groovy committer',
    successes 'Static type checker',
              'Static compilation',
              'Traits',
              'Markup template engine',
              'DSLs'
        failures Stream.of(bugs),
        twitter '@CedricChampeau',
        github 'melix',
        extraDescription '''Groovy in Action 2 co-author
Misc OSS contribs (Gradle plugins, deck2pdf, jlangdetect, ...)'''
}

Agenda

  • The Gradle Daemon

  • Continuous builds

  • Profiling

  • Incremental builds

  • Incremental compilation

  • Composite builds

  • Build cache

What is Gradle?

  • A multi-purpose build tool

  • Gradle Enterprise : build analytics

The Gradle daemon

  • Long-lived background process

  • Listens and executes build actions

  • Faster startup / execution

  • Enabled by default since 3.0

The Gradle daemon

Cold daemon

daemon keynote cold

Warm daemon

daemon keynote warm

Continuous builds

  • Gradle watches for changes in task inputs

  • Re-executes tasks as changes occur

  • Enabled with -t

gradle -t asciidoctor

Build scans

  • Insights into your build

  • View and share via URL

  • Debug, optimize and refine

  • Analyze all of your builds

  • Available for free

build scans

Build scan demo

https://scans.gradle.com/s/cmihj7rsqszle (Groovy) https://scans.gradle.com/s/cxwz7amaxhcdc (This talk) https://scans.gradle.com/s/ly5tnqtuoqlwe (Gradle serial) https://scans.gradle.com/s/7ftlptyifoqna (Android) http://localhost:5050/s/jzgcf2gxxrobu (Gradle parallel)

Incremental builds

  • Gradle is meant for incremental builds

  • clean is a waste of time

  • Prepare your builds for incrementalness

Example: building a shaded jar

task shadedJar(type: ShadedJar) {
   jarFile = file("$buildDir/libs/shaded.jar")
   classpath = configurations.runtime
   mapping = ['org.apache': 'shaded.org.apache']
}

  • What are the task inputs?

  • What are the task outputs?

  • What if one of them changes?

Declaring inputs

@InputFiles
FileCollection getClasspath() { ... }

@Input
Map<String, String> getMapping() { ... }

Declaring outputs

@OutputFile
File getJarFile() { ... }

Incremental compilation

  • Given a set of source files

  • Only compile the files which have changed…​

  • and their dependencies

  • Language specific

Incremental compilation

Gradle has support for incremental compilation of Java

compileJava {
    //enable incremental compilation
    options.incremental = true
}

Incremental compilation

gradle 3.4 perf

Gradle 3.4

https://blog.gradle.org/incremental-compiler-avoidance

Compile avoidance

Compile classpath vs runtime classpath

  • Gradle makes the difference

  • Ignores irrelevant (non ABI) changes to compile classpath

Usage

import com.acme.model.Person;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Iterables;

...

public Set<String> getNames(Set<Person> persons) {
   return ImmutableSet.copyOf(Iterables.transform(persons, TO_NAME))
}

Declaring dependencies

dependencies {
   compile project(':model')
   compile 'com.google.guava:guava:18.0'
}

But…​

import com.acme.model.Person; // exported dependency
import com.google.common.collect.ImmutableSet; // internal dependency
import com.google.common.collect.Iterables; // internal dependency

...

public Set<String> getNames(Set<Person> persons) {
   return ImmutableSet.copyOf(Iterables.transform(persons, TO_NAME))
}

The Java Library Plugin

apply plugin: 'java-library' // instead of 'java'

Separating API from implementation

dependencies {
   api project(':model')
   implementation 'com.google.guava:guava:18.0'
}

Benefit

  • No more compile classpath leakage

  • Downstream dependencies not recompiled when internal dependency changes

Composite builds

  • Compose various projects as if there were one

    • Each project can live in its own repository

    • Each project has its own Gradle build

    • Composition unites them through dependency resolution

Composite builds

  • Split monolithic projects

    • For large multiproject builds, allows splitting them into several pieces

    • Each piece can be versioned independently

    • Developers can choose what subprojects they care about

Composite builds demo time!

Gradle 3.5+

Build cache

  • Avoid doing work even after clean

  • Share binaries between projects on a single machine

  • Share binaries between projects on a network

  • Backend agnostic

Enabling the build cache

From CLI

./gradlew --build-cache <some tasks>

Globally

gradle.properties
org.gradle.caching=true

What to expect?

Enable it on CI!

  • Dramatically reduces feedback loop time

  • Can separate push and pull

ext.isCiServer = System.getenv().containsKey("CI")

buildCache {
    local {
        enabled = !isCiServer
    }
    remote(HttpBuildCache) {
        url = 'https://example.com:8123/build-cache/'
        push = isCiServer
    }
}

Build cache use cases

  • Medium to large multi-projects

  • Bisecting

  • Multiple checkouts of same project on local machine

What’s next?

  • Parallel by default

    • Parallel dependency resolution

    • Parallel execution of tasks

    • Intra-task parallelism

  • Open API for worker threads/process

Performance in larger ecosystem

What about Android?

http://tools.android.com/tech-docs/new-build-system/2-5-alpha-gradle-plugin

android 25 preview

Performance guide

https://gradle.github.io/performance-guide/

perf graph

Thank you!

Learn more at www.gradle.org

#DevoxxFR - @CedricChampeau