Groovy 2: Type checking to the rescue!

Cédric Champeau, VMware

SpringOne2GX, October 17th, 2012

@CedricChampeau

About me

Past : Groovy contributor

Bugfixes
Modules: @Bytecode AST xform, GFreeMarker
Core: compilation customizers, @xInterrupt, ...
Used Groovy as a DSL for natural language processing

Present: Core Groovy committer

Working on bugfixes but main focus on Groovy 2.0
Static type checking
Static compilation

Follow me

About Groovy 2

Modularity

several jars for modules (swing, ant, xml, json, ...)
groovy.jar now 3MB
extension modules

JDK 7 enhancements

project coin
invoke dynamic support (use -indy jars)

Static theme

Static type checking
Static compilation

Static type checking

Goal

Find errors at compile time (fail early)
because lots of code do not use dynamic features of Groovy
so many bugs can be discovered before production
Make java developers even happier

Turn the compiler grumpy

Report typos
missing method/property
Extension methods (aka DefaultGroovyMethods)
Type check assignments
Perform type inference
method/closure return type inference
generics type inference

Static Type Checking

It's optional

Triggered using @TypeChecked annotation

Annotate a class or a method

@groovy.transform.TypeChecked
class TypoInMethodName {
    void foo() {}
    void bar() {}

    void method() {
        foo() // no error, method is defined
        baz() // error at compile time!
    }
}

Static type checking basics

@groovy.transform.TypeChecked
class TypoInFieldName {
    String name = 'Cedric'

    void method() {
        println naame // typo, the variable [naame] is undeclared
    }
}

@groovy.transform.TypeChecked
class TypoInGString {
    String name = 'Cedric'

    void method() {
        println "My name is $naame" // in gstrings too, the variable [naame] is undeclared
    }
}

@groovy.transform.TypeChecked
class OperatorChecking {
    void method() {
        int x = 0
        x += new Object() // Cannot find matching method int#plus(Object)
    }
}

STC basics (cont'd)

@groovy.transform.TypeChecked
class AssignmentError {
    void method() {
        Set set = 'Bar' // Cannot assign a value of type String to variable of type Set
    }
}

@groovy.transform.TypeChecked
class AssignmentToArrayElement {
    void method() {
        Date[] arr = new Date[2]
        arr[0] = new Date() // ok
        arr[1] = 'Hello' // Cannot assign value of type String to variable of type Date
    }
}

@groovy.transform.TypeChecked
class IncorrectReturnType {
    int method() {
        'foo' // cannot return value of type String on method returning type int
    }
}

STC basics (cont'd)

@groovy.transform.TypeChecked
class ReturnTypePropagation {
    boolean test() { true }
    void m() {
        int x = 1 + test() // return type checking
    }
}

@groovy.transform.TypeChecked
class TransparentToString {
    String test() { new StringBuilder('transparent conversion to string') }
    void m() {
        println "${test()}"
    }
}

@groovy.transform.TypeChecked
class GDKMethods {
    int test(String val) {
        val.toInteger() // DGM methods
    }
}

STC basics (cont'd)

Multiple assignments

        def (x,y) = [ 1, 2 ] // valid only because the number of components correspond

def (int x, Date d) = [1, new Date()] // did you know about that syntax?

Type checking options

if set on a class, type checks:

the whole class
inner classes
closures defined in the class

if set on a method, type checks:

the method only
closures defined in the method body

You can prevent some code from beeing type checked:

import groovy.transform.TypeChecked
import static groovy.transform.TypeCheckingMode.*

@TypeChecked(SKIP)
int foo() { 'STRING SHOULD NOT PASS TYPE CHECKING' }

Type inference (1/3)

def with list literals

@groovy.transform.TypeChecked
    class ListComponentTypeInference {
        void test() {
            def list = ['a','b','c']
            println list[0].toUpperCase() // list type inferrence
        }
    }

Type inference (2/3)

Verify that declared types are compatible with assignments

@groovy.transform.TypeChecked
    class GenericsTypeInference {
        void test() {
            List<String> list = ['a','b','c'] // generic type argument inferrence
            println list[0].toUpperCase()
            List<Number> list2 = ['a','b','c'] // Cannot assign List<String> to List<Number>
        }
    }

Type inference (3/3)

LUB = Lowest Upper Bound (also called Least Upper Bound)

represents the "common super type"
granularity is the class + interfaces

@groovy.transform.TypeChecked
    class LowestUpperBound {
        void test() {
            List<? extends Comparable> list = ['a',123] // LUB inferrence
        }
    }

Instanceof type inference

@groovy.transform.TypeChecked
class InstanceOfNotRequired {
    void test(Object val) {
        if (val instanceof String) {
            println val.toUpperCase() // no need for explicit cast
        } else if (val instanceof Number) {
            println ("X"*val.intValue()) // String#multiply(int) from DGM
        }
    }
}

Type checked Grooviness (1/4)

Groovy constructors

Dimension d = [100,200] // type checked!
Dimension d2 = [100] // fails at compile time!

Map-style constructors

class Bean {
    int x
    int y
}
Bean myBean = [x:100,y:200] // valid
Bean myBean = [x:'foo', y:200] // compile-time error

Type checked Grooviness (2/4)

Closure return type inference

def cl = { new Date() }
long time = cl().time // no compile time error!
boolean ok = cl().time // compile time error!

Closure arguments

def sum = { int x, int y -> x+y }
assert sum(8,8) == 16 // no compile time error, types are compatible
assert sum('8','8') == '88' // compile time error!

Type checked Grooviness (3/4)

Lists

def list = [new Date(), new Date()]
def listOfTime = list.time
long time = listOfTime[0]

Maps

def map = [Java: 'verbose', Groovy: 'Baby!']
println "Groovy ${map.Groovy}"

Type checked Grooviness (4/4)

Compatible with @AST transformations

@groovy.transform.TupleConstructor
class Person {
    String name, city
    static Person create() {
        new Person("Guillaume")
    }
}

Person.create()

True as long as the AST transformation runs before the INSTRUCTION_SELECTION phase

Flow typing

Use type inference to remove the need for explicit casts
If using totally different types with same variable, bad practice

def var = 123 // store an int into var
int x = var // assignment is valid, compiler doesn't complain
var = '123' // assign a String to var
x = var.toInteger() // valid, no compile-time error because we know it's a String
var = 123 // still valid
x = var.toUpperCase() // at this point, compile-time error

What's cooking?

Planned for Groovy 2.1

Support for @DelegatesTo
Improved DSL type checking

Wishlist

ability to enable @TypeChecked by default
probably through CompilerConfiguration

Static type checking gotchas

Dynamic vs Static

Static type checking in a dynamic language?!

by definition, dynamic features are not compatible
metaclass, categories, builders, mocks...

@groovy.transform.TypeChecked
void wontPassTypeChecker() {
        String.metaClass.myMethod = { 'a new method' }
        'foo'.myMethod() // works with dynamic Groovy but not with STC
}

Metaclasses can be changed from any thread
even if you change the metaclass in a not statically checked method
the changes won't be "visible" to a static type checker

Closure parameter types

What's wrong with this code?

    @groovy.transform.TypeChecked
    List<String> wontCompile() {
       ['foo','bar','baz'].collect { it.toUpperCase() }
    }

Is the role of the receiver the same in this code?

100.times { println it }

The type checker doesn't know about the type of "it"

An enhancement proposal exists, nothing implemented yet

Closure shared variables (1/2)

Tracking assignments of closure shared variables
The type checker enforces "good practice"
ensure that method calls are only possible on the LUB of all assignments

@groovy.transform.TypeChecked
class ClosureSharedVariable {
    void test() {
        def var = "String" // var is a closure shared variable
        def cl = { var = new Date() } // assignment of a Date inside a closure
        cl()
        var = var.toUpperCase() // method calls only allowed on the LUB
    }
}

Closure shared variables (2/2)

This is allowed

class A { void foo() {} }
class B extends A { void bar() {} }

@groovy.transform.TypeChecked
class ClosureSharedVariable {
    void test() {
        def var = new A()
        def cl = { var = new B() }
        cl()
        var.foo() // var is at least an instance of A
    }
}

Closure delegates and strategy

In 2.0

Only "with" is supported
No support for delegate
No support for strategy

In 2.1 (upcoming)

User may "help" the type checker
Uses the @DelegatesTo annotation

Method selection gotchas (1/3)

Dynamic vs static

In dynamic Groovy, methods are selected at runtime
In dynamic Groovy+STC, it's still the case, yet the compiler tries to find them at compile time
By definition, the type checker is wrong!

What's wrong with this (statically checked) code?

static String foo(String s) {
        'String'
}
static String foo(Integer s) {
    'Integer'
}
static String foo(Boolean s) {
    'Boolean'
}
['foo',123,true].each {
    foo(it)
}

Method selection gotchas (2/3)

Step 1: you need to help the compiler

static String foo(String s) {
    'String'
}
static String foo(Integer s) {
    'Integer'
}
static String foo(Boolean s) {
    'Boolean'
}
['foo',123,true].each {
    if (it instanceof String) {
        foo((String)it)
    } else if (it instanceof Boolean) {
        foo((Boolean)it)
    } else if (it instanceof Integer) {
        foo((Integer)it)
    }
}

Method selection gotchas (3/3)

Step 2: you can remove the casts!

static String foo(String s) {
    'String'
}
static String foo(Integer s) {
    'Integer'
}
static String foo(Boolean s) {
    'Boolean'
}
['foo',123,true].each {
    if (it instanceof String) {
        foo(it)
    } else if (it instanceof Boolean) {
        foo(it)
    } else if (it instanceof Integer) {
        foo(it)
    }
}

Static compilation

Why do you want static compilation?

Results from a survey before we started working on static compilation

Why static compilation?

If you do static type checking...

you cannot ensure dynamic semantics
so why can't you statically compile ?

Benefits

Static type checking (type safety)
Improved performance
not everyone has JDK7+ (InvokeDynamic)
Bytecode size (work on progress)
Immunity to monkey patching (useful for framework developers)

But you loose...

Dynamic features (categories, metaclasses, ...)
Dynamic method dispatch
Dynamic Groovy semantics (part of)

Static compilation in Groovy 2

Static compilation is optional

Relies on the @TypeChecked AST transformation
Built to be as close as possible as dynamic Groovy semantics

But there are important differences

Unlike Groovy++, it's just static compilation
no added features
no grammar changes
Features and design choices discussed with the community

Static compilation: how to use

Just use the @CompileStatic annotation instead of @TypeChecked

@groovy.transform.CompileStatic
int fib(int n) {
    n<2?1:fib(n-1)+fib(n-2)
}

Generated bytecode is very close (if not equal) to what Java produces

CompileStatic and semantics

Does it follow the semantics of dynamic Groovy?

Short answer: no

Long answer: as close as possible

Problem is the dynamic dispatch

Static method dispatch (1/2)

What does the following code outputs?

def foo(String str) { 'STRING' }
def foo(Date d) { 'DATE' }
Object o = new Date()
foo(o) // in Java, would fail at compile-time

In dynamic Groovy, prints 'DATE'
Passes type checking thanks to flow typing
Prints 'DATE' with static compilation too!

Static method dispatch (2/2)

Be careful!

@CompileStatic
class StaticDispatch {
    static def foo(Object a) { 'OBJECT' }
    static def foo(String a) { 'STRING' }
    static def foo(Date a) { 'DATE' }

    static def bar(Object o) {
        foo(o)
    }

    static void main(String... args) {
        println foo('string') // prints 'STRING'
        println bar('string') // prints 'OBJECT'
    }
}

Compat with extension modules (1/2)

Static compilation (thus static type checking) is compatible!

Define an extension

public class MyStringExtension {
    public static String reverseToUpperCase(String self) {
        StringBuilder sb = new StringBuilder(self.toUpperCase());
        return sb.reverse().toString();
    }
}

Write the descriptor file

moduleName=My test module
moduleVersion=1.0-test
extensionClasses=org.example.MyStringExtension
staticExtensionClasses=

Compat with extension modules (2/2)

Static compilation (thus static type checking) is compatible!

Use it in your statically compiled code!

@groovy.transform.CompileStatic
void codeUsingExtensionModule() {
    def str = 'yvoorg'
    assert str.reverseUpperCase() == 'GROOVY'
}

Inside the type checker

Inside the type checker (1/2)

How does it work?

It's an AST transformation!
Technically, it doesn't really transform anything but it makes intensive use of node metadata (see ASTNode#getNodeMetaData)
runs at instruction selection phase (late)
Some compiler framework changes were needed to support
- regular compilation
- invokedynamic (1.7+ JVMs)
- static compilation

Inside the type checker (2/2)

Node metadata

Groovy allows storing arbitrary metadata on each AST node
The type checker uses this facility to store type information
For details, see org.codehaus.groovy.transform.stc.StaticTypesMarker

Static compilation

Puts additional node metadata (see org.codehaus.groovy.transform.sc.StaticCompilationMetadataKeys)
Performs AST transformations (sometimes generates optimized bytecode directly)

Testing the type checker (1/2)

ASTTest transformation

Developped to help testing the type checker
An AST transform aimed at testing any AST transformation

Map.Entry<Date, Integer> entry

@ASTTest(phase=INSTRUCTION_SELECTION, value={
    assert node.getNodeMetaData(INFERRED_TYPE) == make(Date)
})
def k = entry?.key

@ASTTest(phase=INSTRUCTION_SELECTION, value={
    assert node.getNodeMetaData(INFERRED_TYPE) == Integer_TYPE
})
def v = entry?.value

Testing the type checker (2/2)

ASTTest transformation

@ASTTest(phase=INSTRUCTION_SELECTION, value= {
    lookup('forLoop').each {
        assert it instanceof org.codehaus.groovy.ast.stmt.ForStatement
        def collection = it.collectionExpression // MethodCallExpression
        def inft = collection.getNodeMetaData(INFERRED_TYPE)
        assert inft == make(Set)
        def entryInft = inft.genericsTypes[0].type
        assert entryInft == make(Map.Entry)
        assert entryInft.genericsTypes[0].type == STRING_TYPE
        assert entryInft.genericsTypes[1].type == Integer_TYPE
    }
})
void test() {
    def result = ""
    def sum = 0
    forLoop:
    for ( Map.Entry<String, Integer> it in [a:1, b:3].entrySet() ) {
       result += it.getKey()
       sum += it.getValue()
    }
    assert result == "ab"
    assert sum == 4
}

Q&A

Thank you!

Slides written using deck.js

http://melix.github.com/talks/s2gx-typechecking/slides.html