Cédric Champeau's blog: Gradle myth busting: scripting

The good old debate

In the end, most of the answers cycle around the same, good old, debate: Gradle uses scripting (Groovy or Kotlin) vs Maven uses declarative. You’ll aways find people telling you that XML is better because it locks you down, its fully declarative (is it, really?) and everybody is forced to do the same. I don’t counter those arguments, this is a strength of Maven, but it also comes with a number of drawbacks.

I promised some of the folks in the conversation some answers (please look down for direct answers to tweets), so here they are. I’m answering on a blog post because again Twitter is not good for this, it’s causing a lot of misunderstandings, because you get into multiple, parallel conversations with different people who accidentally get mentioned, and get scuds fired at you without even having time to answer… Even a blog post is not enough, there’s so much to say on this topic.

First, on the so called "declarative vs imperative" model, I will always disagree on this dichotomy. I disagree that Gradle isn’t declarative. It’s as declarative as you want it to be. Take this build file I wrote recently, which is an Asciidoctor Reveal.js presentation template (it allows writing slide decks with Asciidoctor and reveal.js). Here’s what my build file looks like:

plugins {
    id("org.gradle.presentation.asciidoctor")
}

presentation {
    githubUserName.set("melix")
}

dependencies {
    asciidoctor("org.asciidoctor:asciidoctorj-diagram:1.5.11")
}

tasks {
    asciidoctor {
        requires("asciidoctor-diagram")
    }

}

I wouldn’t particularly say this is imperative. It looks very declarative to me. Concise too. The fact it uses an imperative language is orthogonal, but it does, however, create the ability to write imperative code in the build. Note, however, that a dependency was declared for asciidoctor. This is a major, and probably the most important, difference with Maven: compile or runtime doesn’t make sense here. We declare a dependency for asciidoctor rendering. There’s no Java library being built here, it’s a presentation. Gradle lets you model precisely what you build.

So, in the end, I think what matters is not declarative vs scripting. I think what people really want is to reduce the risks of writing bad things. Locking down using XML is one way to achieve this, but it’s not the only one. For example, Gradle build scripts may be linted. In other words, you can apply on a Gradle build the same tooling you are used to work with when dealing with your own code: checkstyle, findbugs, … You don’t have to, but you can.

The consequence is that yes, there are many different ways you can layout your build with Gradle. This is not different from how you can layout your code in a project: we don’t tell you in which package you should put your beans, services, … However, there’s a big misconception that I’d like to fight:

By default, for a Java project, Gradle follows the same conventions as Maven.

That’s as simple as that. Sources will be in src/main/java. Tests will be in src/test/java. Gradle gives you the freedom to diverge from this convention, but this is not encouraged, and to be honest, I’ve almost never seen any build diverging from those conventions. On rare occasions, those were actually builds migrated from other build systems (in particular Ant) where at the time there wasn’t any convention. Gradle offers the flexibility to reuse an existing layout without much hassle.

Gradle is too flexible?

All in all, the argument that "Gradle is too flexible" is a fallacy. It’s all about good engineering practices, putting the right tools in place, and this is nothing different from any engineering work we do everyday. If you can do it for your code, you can do it for your build. The interesting thing is why you think you shouldn’t have the same quality expectation levels for your build as you have for your code. Often the answer is just "I don’t care much about the build, I’m writing code, this is what I’m paid for".

And this is where the discussion becomes interesting, because I think this is a bias that lots of developers have. They don’t even realize how much time they are losing.

Tell me, how likely is it that you change your build scripts, compared to the number of times you’re effectively going to run mvn clean install or gradle test? The reality is that you’re running the build much more often that you change it. Therefore, correctness, incremental builds, incremental compilation, compile avoidance, task output caching, are far more important to developer productivity than the declarativeness aspect.

Sure pure declarativeness is a good thing and this is why I encourage Gradle users to write nice, synthetic build files, but this is not the most important aspect for developer productivity.

My point is therefore that if you only focus on the surface, that is to say the language used to express the build (XML vs Groovy/Kotlin), then you’re missing the most important part to me, which is the underlying Gradle model, far more advanced than what you have in other tools. The Gradle API surfaces this model and has a number of advantages:

A task can be seen as a function. It declares inputs and outputs. For the same inputs, the output are always the same: this provides up-to-date checking and cache-ability.
A task inputs can be another task outputs. This provides implicit dependencies: Gradle knows that if you want to run "test", you have to compile first, but it also knows that whatever else is an input to the tests need to be executed.

As a consequence, I already wrote about why it’s wrong to think that Gradle doesn’t have lifecycle tasks. In fact, Gradle has them, but is also significantly more precise. The "phase" approach of Maven is way to coarse: it’s doomed to execute too much, prevents smart parallelism, and leads to dirty workarounds (-x …. on the CLI to avoid things you know are not necessary).

Similarly, say you want to test your application on different JDKs and have a single build execute tests for all target JVMs, which is different from the JVM which runs the build tool. With a scripting approach like Gradle, this is totally doable. I won’t say easy because we can definitely do better to make this use case better, but the underlying model makes it quite simple. You don’t want to rebuild your application for each target VM. All you want is to test on different platforms, and therefore the only step should be a different target VM for test execution. Tools like Maven force you into arbitrary things like defining Maven profiles, and force you into rebuilding everything. This is a giant waste of time for something you don’t need!

In a different topic, this is no surprise that Gradle can build for different ecosystems: Java, Scala, C++, Kotlin, Kotlin Native, Python, … The underlying infrastructure makes it possible. Even for a single ecosystem, Gradle can declare what the difference between a Java Library, a Java Platform or an application is.

I could talk hours about why it’s important to model properly software, and actually with the release of Gradle 6 we’ll have a series of blog posts explaining why we think it’s a game changer in terms of dependency management. If you’re tired of having to fix the same "multiple slf4j bindings" in each and every project, tired of Guava being upgraded from jre to android, frustrated by incompatibilities of Scala 2.11 and 2.12 dependencies, tired of not knowing which of those Maven optional dependencies is important for you to add, you’ll understand what I mean.

Direct answer to some tweets

I’ll try to answer more direct questions in this section. Sorry if I missed yours, I got quite a few comments/answers…

I would sacrifice caches, dependency locks and better plugins to version to have a declarative build process instead of an imperative one. Give me a declarative Gradle and I will love it.

Again, I think this is the wrong tradeoff. Given that you run the build way more often that you change it, declarativeness (that you can have with Gradle) shouldn’t be the goal. Your goal should be to reduce your build times, make your build reproducible, improve your developer productivity. Declarativeness is not a goal, it’s at best a mean, but not sufficient by itself. A declarative Gradle, whatever that means, would help you reduce the cognitive overhead, but wouldn’t help you better model what your application needs.

make a one liner the ability to publish on different repos the snapshot and release artifacts. The way it was done on Gradle 4.x was broken on 5.x and the only way we found to do it is a horrible hack

Here’s a webinar about publishing. Publishing is not complicated with Gradle. It used to be poorly documented, and the old publishing plugins didn’t help. But publishing to a snapshot repository should be trivial already.

Some people will prefer to do their own way, some people will prefer to have a less expressive tool that will produce similar build processes on their projects. Gradle give you the former, Maven the latter. As I say, a matter of taste.

A less expressive tool reduces the risks of writing bad builds. It doesn’t help, however, in developing correct, reproducible, fast builds. An, again, I disagree that Gradle leads to "custom builds" everywhere. Most people stick to the defaults and are very happy with them. The more complex builds you find in the wild are those which have indeed very specific needs, or need to be tweaked for performance, producing more artifacts, combinations of artifacts or testing. Things that you can’t easily do with Maven profiles, for example, because profiles are adhoc solutions which do not combine well.

Gradle tries to create a fake sense of declarativeness, but it is just an illusion.

It’s not an illusion. Gradle has a clear separation between its configuration model and execution model. All tasks have declared inputs. The plugins create either new tasks or conventions. This is not an illusion, this is the reality. Now, because you can write if or loops doesn’t mean it’s not declarative, it’s imperative-declarativeness. And yes, you can end up with giant build scripts with "code" inside. If you have, do yourself a favor, refactor your build like you would with your code, because no one should tolerate this. Use buildSrc, this is your friend.

I think library dependencies is not correctly supported by IDEs and Java modules are better.

That’s not correct. We’ve been using the native Gradle IntelliJ support for years at Gradle, with api and implementation separation, and it works exactly like it should. Implementation dependencies are hidden from consumers, like they should. If you don’t see this, either you didn’t declare the dependencies or you have a bug in the IDE, in which case it needs to be reported.

Even worst, Gradle don’t have an official plugin to deal with module-info. there was an old post that says it is not necessay with Gradle because lib dependencies were better (they don’t)

I don’t think anyone said you don’t need module-info. There are different things in play:

separation of API and implementation: Gradle supports this, and it maps to requires vs requires transitive
declaration of public API packages: Gradle used to have this with the deprecated "software model". It still has to be backported to the current configuration model. For this, module-info works fine but it forces you into using the "modular world", which a lot of libraries, frameworks and IDEs are not ready for.
declaration of services: Gradle doesn’t support this.

Can you use modules with Gradle? Yes, there’s a quite good plugin to do it. We are planning to support modules and modularity in general better in Gradle, but not short term, because we have bigger pain to solve for our users first. It doesn’t mean we don’t consider this important.

I don’t get why Gradle allow you to explain what your app is better than Maven. In fact I think it is more difficult to explain it on a script that descriptivelly.

I think the question is what Gradle models better than Maven. A good example here is api vs implementation dependencies. Because Maven uses the same descriptor (pom.xml) for the producer and the consumer, a dependency declared in the <compile> scope ends up on the compile classpath of the consumers. This is leaking internal implementation details to consumers, which is very bad because it makes it very hard to evolve libraries, because changing an internal dependency would break consumers which accidentally started depending on your own transitives. This is just an example of course, there are many other differences (like, why we consider that exclude is a bad workaround in general, more on this topic in Gradle 6, if you want to read our docs).

The builds I’ve seen have been very spaghettish and clearly copy-and-pasted together un-understood recipes from SO.

Yes, there are bad builds out there. With Gradle it’s frequent for quite old builds from early adopters. More recent builds tend to be much cleaner, because we made a significant effort in guides, getting started samples, documentation. You’ll always find bad things, and it should be encouraged to fix. On this topic, tools like build scans really help. And copy/pasting from SO is indeed a bad thing. If you copy and paste without understanding what it does, well, bad things can happen… That said I’ve seen very scary Maven builds too, and believe me or not, some of our customers wouldn’t be proud to show you their Maven builds. It’s the "personal experience fallacy". I’ve experienced very clean Gradle builds, you’ve experienced very bad Gradle builds. I’ve also written bad Gradle builds, which I dramatically improved, making them more correct, faster, … Gradle is like any other technology: learn it and you can understand what it brings.

Gradle performance/caching are very attractive but the scripting possibility is a deal breaker. A « declarative-only » Gradle would be perfect for people like me.

Again I think "declarative" is the wrong term here. Locked down to reduce the risks of doing bad things is what you want. It doesn’t matter if it’s Kotlin, Groovy, XML or whatever else. It doesn’t matter if you can use if or for loops. What matters is what you can express, and what should be limited. It’s all good engineering that we must share within the industry, find the best patterns, discourage the bad ones. There are quite a few things in the Android world (which uses Gradle) in this direction. We, at Gradle, should do more, but it’s always a matter of priorities: fixing the most important user pain first. By the way, we provide a Maven build cache with Gradle Enterprise. That is to say, the ability to cache Maven builds using Gradle Enterprise. However, this is limited to "clean builds" (which Maven users are used to do in any case), because of the limitations of the Maven execution model (no knowledge of what each plugin or mojo does, where it writes files, …).

IMHO the biggest feature of Gradle that Maven doesn’t have is the ability to change the version of the project

Well, this is just an accidental example of the interest of having access to the API in a build script. It offers a number of options for the release process, but that’s not the only one.

My only complain about @gradle is how it is unnecessarily complex to deploy a multi-module project to central. Too much copy & paste, or you need to make an init script, which I still haven’t managed to do.

Technically the problem is not "how to deploy a multi-module project to Central", but rather, how do I avoid duplicating configuration between scripts. This is what buildSrc is for. As soon as you have repetition, then, it means a plugin makes sense. buildSrc can be seen as "local plugins", and this is where you should write your common code. Then each project applies a plugin to publish. This is a composition model, as opposed to the inheritance model of Maven.

I like all those (caching, incrementality, …) , in theory, but for my needs they are more complexity than feature

I don’t think those are complexity. A task declares its inputs. If you do, you benefit from up-to-date checking, and with a bit more configuration, caching. You don’t have to. If you don’t declare the inputs/outputs, you’re back to the "Maven" approach where the build tools knows nothing about what a task does, at the difference that Gradle knows that it knows nothing, so can be a bit smarter. As soon as you start declaring your inputs, you benefit from more. It’s more work, for sure, but it’s not that complex and the benefit is huge.