Variant-aware dependency management with Gradle

Cédric Champeau

Who am I

speaker {
    name 'Cédric Champeau'
    company 'Gradle Inc'
    oss 'Apache Groovy committer',
    successes (['Static type checker',
                    'Static compilation',
                    'Traits',
                    'Markup template engine',
                    'DSLs'])
        failures Stream.of(bugs),
        twitter '@CedricChampeau',
        github 'melix',
        extraDescription '''Groovy in Action 2 co-author
Misc OSS contribs (Gradle plugins, deck2pdf, jlangdetect, ...)'''
}
GradleLogoReg

Groovy in Action 2

koenig2

https://www.manning.com/books/groovy-in-action-second-edition

Coupon ctwgr8conftw

Principles

  • Apps target different platforms

  • Apps have different flavors

  • Each of them have different dependency set

  • How can you create your own variants?

Declaring a dependency

Declare a dependency on a library of the same project

model {
    components {
        main(JvmLibrarySpec) {
            sources {
                java {
                    dependencies {
                        library 'dep'
                    }
                }
            }
        }
        dep(JvmLibrarySpec)
    }
}

Declare a dependency on a library from another project

model {
    components {
        main(JvmLibrarySpec) {
            sources {
                java {
                    dependencies {
                        project ':other' library 'dep'
                    }
                }
            }
        }
    }
}

Shortcut notation if target project only has one library

model {
    components {
        main(JvmLibrarySpec) {
            sources {
                java {
                    dependencies {
                        project ':other'
                    }
                }
            }
        }
    }
}

Platform-aware dependency management

Declaring target platform of a component

model {
    components {
        main(JvmLibrarySpec) {
            targetPlatform 'java6'
            targetPlatform 'java7'
        }
	dep(JvmLibrarySpec) {
            targetPlatform 'java7'
        }
    }
}

Dependencies with platforms (1/3)

model {
    components {
        main(JvmLibrarySpec) {
            targetPlatform 'java6'
            targetPlatform 'java7'
            sources {
		java {
		    dependencies {
		        library 'dep'
		    }
		}
	    }
        }
	dep(JvmLibrarySpec) {
            targetPlatform 'java7'
        }
    }
}

Dependencies with platforms (2/3)

$ ./gradlew platform-aware:java7MainJar
:platform-aware:compileDepJarDepJava
:platform-aware:createDepJar
:platform-aware:depJar
:platform-aware:compileJava7MainJarMainJava
:platform-aware:createJava7MainJar
:platform-aware:java7MainJar

BUILD SUCCESSFUL

Total time: 1.264 secs

Dependencies with platforms (3/3)

$ ./gradlew platform-aware:java6MainJar

FAILURE: Build failed with an exception.

* What went wrong:
Could not resolve all dependencies for 'Jar 'java6MainJar'' source set 'Java source 'main:java''
> Cannot find a compatible binary for library 'dep' (Java SE 6). Available platforms: [Java SE 7]

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 1.169 secs

In short

  • A component can target multiple versions of Java

  • Produces as many jars as there are target platforms

  • By default, will use the platform Gradle is executed with

  • A dependency will resolve to the highest compatible version

  • Error messages tell which platforms are available

Custom libraries

  • It is possible to define custom library types

  • To be compatible with JVM dependency management, they have to

    • extend LibrarySpec

    • produce at least one binary of type JarBinarySpec

Example

interface MyLib extends LibrarySpec {}
class DefaultMyLib extends BaseComponentSpec implements MyLib {}

class ComponentTypeRules extends RuleSource {

    @ComponentType
    void registerComponent(
            TypeBuilder<MyLib> builder) {
        builder.defaultImplementation(DefaultMyLib)
    }

    @ComponentBinaries
    void createBinaries(
            ModelMap<JarBinarySpec> binaries,
            MyLib library,
            PlatformResolvers platforms,
            @Path("buildDir") File buildDir,
            JavaToolChainRegistry toolChains) {

        def platform = platforms.resolve(JavaPlatform, DefaultPlatformRequirement.create("java${JavaVersion.current().majorVersion}"))
        def toolChain = toolChains.getForPlatform(platform)
        def baseName = "${library.name}"
        String binaryName = "${baseName}Jar"
        binaries.create(binaryName) { jar ->
            jar.toolChain = toolChain
            jar.targetPlatform = platform
        }
    }

}

apply type: ComponentTypeRules

Usage

model {
    components {
        main(MyLib) {
            sources {
                java(JavaSourceSet)
            }
        }
    }

Dependencies onto custom components

model {
    components {
        main(MyLib) {
            sources {
                java(JavaSourceSet) {
                    dependencies {
                        library 'dep'
                    }
                }
            }
        }
        dep(MyLib) {
            sources {
                java(JavaSourceSet)
            }
        }
    }
}

What depends on what?

  • A JvmLibrarySpec can depend on a CustomLib

  • A CustomLib can depend on a JvmLibrarySpec

  • Combibations of the above

Defining custom binaries

  • Custom binaries are the entry point for custom variants

  • Can be unmanaged or @Managed

Example

@Managed
interface MyJar extends JarBinarySpec {}

...

    @ComponentBinaries
    void createBinaries(
            ModelMap<MyJar> binaries,
            MyLib library,
            PlatformResolvers platforms,
            @Path("buildDir") File buildDir,
            JavaToolChainRegistry toolChains) {

        def platform = platforms.resolve(JavaPlatform, DefaultPlatformRequirement.create("java${JavaVersion.current().majorVersion}"))
        def toolChain = toolChains.getForPlatform(platform)
        def baseName = "${library.name}"
        String binaryName = "${baseName}Jar"
        binaries.create(binaryName) { jar ->
            jar.toolChain = toolChain
            jar.targetPlatform = platform
        }
    }

...

Custom variant dimensions

Main steps

  • Define a custom library type

  • Define a custom binary type

  • Annotate some properties with @Variant

@Variant

  • Must be set on a getter

  • The type can be either a String

  • or a ? extend Named

Define a custom library type

  • The library interface also defines the DSL

trait MyLib implements LibrarySpec {
    List<String> flavors = []
    void flavors(String... flavors) { this.flavors.addAll(flavors as List) }
}
class DefaultMyLib extends BaseComponentSpec implements MyLib {}

Define the binary type

@Managed
interface MyJar extends org.gradle.jvm.internal.JarBinarySpecInternal {
    @Variant
    String getFlavor()
    void setFlavor(String flavor)
}

Update the @ComponentBinaries method to create binaries

        library.flavors.each { flavor ->
            String binaryName = "${baseName}${flavor.capitalize()}Jar"
            binaries.create(binaryName, MyJar) { jar ->
                jar.toolChain = toolChain
                jar.targetPlatform = platform
                jar.flavor = flavor
            }
        }

Describe the components

model {
    components {
        main(MyLib) {
            flavors 'free','paid'
            sources {
                java(JavaSourceSet) {
                    dependencies {
                        library 'dep'
                    }
                }
            }
        }
        dep(MyLib) {
            flavors 'free'
            sources {
                java(JavaSourceSet)
            }
        }
    }
}

Build the "free" variant

$ ./gradlew customvariants:mainFreeJar
...
:customvariants:compileDepFreeJarDepJava
:customvariants:createDepFreeJar
:customvariants:depFreeJar
:customvariants:compileMainFreeJarMainJava
:customvariants:createMainFreeJar
:customvariants:mainFreeJar

BUILD SUCCESSFUL

Total time: 2.08 secs

Build the "paid" variant

$ gw customvariants:mainPaidJar
...

FAILURE: Build failed with an exception.

* What went wrong:
Could not resolve all dependencies for 'Jar 'mainPaidJar'' source set 'Java source 'main:java''
> Cannot find a compatible binary for library 'dep' (Java SE 8).
      Required platform 'java8', available: 'java8'
      Required flavor 'paid', available: 'free'


* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

Detailed errors (1/2)

$ ./gradlew customvariants:fourthFreeJar
...

FAILURE: Build failed with an exception.

* What went wrong:
Could not resolve all dependencies for 'Jar 'fourthFreeJar'' source set 'Java source 'fourth:java''
> Cannot find a compatible binary for library 'third' (Java SE 8).
      Required platform 'java8', available: 'java8' on Jar 'thirdDemoJar','java8' on Jar 'thirdSharewhareJar'
      Required flavor 'free', available: 'demo' on Jar 'thirdDemoJar','sharewhare' on Jar 'thirdSharewhareJar'

Detailed errors (2/2)

$ gw customvariants:fifthJar
...

FAILURE: Build failed with an exception.

* What went wrong:
Could not resolve all dependencies for 'Jar 'fifthJar'' source set 'Java source 'fifth:java''
> Multiple binaries available for library 'third' (Java SE 8) :
     - Jar 'thirdDemoJar':
         * flavor 'demo'
         * targetPlatform 'java8'
     - Jar 'thirdSharewhareJar':
         * flavor 'sharewhare'
         * targetPlatform 'java8'

How matching works

  • JavaPlatform matches the highest compatible platform

  • If a variant is null in either the consumer or the candidate library: dimension is ignored

  • If a variant is not null:

    • if the type is String, matches stricly

    • if the type is Named, matches strictly on getName(), unless a custom VariantDimensionSelector is provided

    • JavaPlatform is handled through a specific VariantDimensionSelector (DefaultJavaPlatformVariantDimensionSelector)

Bonus: binary-level dependencies

Variant-specific dependencie

  • A source set can define its own dependencies

model {
    components {
        main(AndroidLikeLib) {
            flavors 'free', 'paid'
            sources {
                paid {
                    // some extra dependencies for paid flavor
                }
            }
        }
    }
}

Shared source set dependencies

  • But for a shared source set?

model {
    components {
        main(JvmLibrarySpec) {
            targetPlatform 'java6'
            targetPlatform 'java7'
            binaries.named('java6MainJar') {
                sources.named('java') {
                    source.srcDir file("$projectDir/src/main/java7compat")
                }
            }
            binaries.named('java7MainJar') {
                sources.named('java') {
                    dependencies {
                        library 'dep'
                    }
                }
            }
        }
        dep(JvmLibrarySpec) {
            targetPlatform 'java7'
        }
    }
}

What we can do?

  • Create new binary level source sets

  • But no variation of dependencies for a specific source set

model {
    components {
        main(JvmLibrarySpec) {
            targetPlatform 'java6'
            targetPlatform 'java7'
            binaries.named('java6MainJar') {
                sources {
                    java6(JavaSourceSet) // creates a Java 6 specific source set
                    dependencies {
                        project ':other' library 'lib'
                    }
                }
            }
        }
    }
}

Conclusion : Gradle…​

  • Natively supports Java variant aware dependency management

  • Supports defining custom variant dimensions

But…​

  • New (experimental) Java software model only

  • No support for external dependencies yet

  • No support for publishing either

Questions?