Configure Global, Customise per Task

Summary

Use of extensions is a common pattern for global configuration. In cases where multiple tasks rely on such global configuration, it is sometimes required to customise specific tasks not to use the global configuration. This leads to additional properties on a task with additional testing and complexity.

Solution

Create an extension that can be applied to both the project and tasks. Make the tasks in your plugin that would previously have relied only on a project extension to be task-extension aware. Build the extension in such a way that task extensions are aware of the global project extension. If the properties of the task extension is not explicitly configured by the build script author, the task extension will retrieve the value from the project extension.

Examples

Consider for a moment that you are writing an extension for the GNU Make tool. In this extension you want to set the number of concurrent jobs and as well as the flags that should always be passed to an invocation of the tool. Conceptually you may model this in your extension as follows.

 1 @CompileStatic
 2 class GnuMakeExtension {
 3     void setNumJobs(int numJobs) {
 4         this.numJobs = numJobs
 5     }
 6 
 7     Integer getNumJobs() {
 8         this.numJobs
 9     }
10 
11     void flags(final Map<String,String> extraFlags) {
12         flags.putAll(extraFlags)
13     }
14 
15     Map<String,String> getFlags() {
16         this.flags
17     }
18 
19     private Integer numJobs
20     private Map<String,String> flags = [:]
21 }

The first step is to add the constructors for both task and project.

 1 final static String NAME = 'gnumake'
 2 
 3 GnuMakeExtension(Project project) {
 4     this.project = project
 5 }
 6 
 7 GnuMakeExtension(Task task) {
 8     this.task = task
 9 }
10 
11 private final Project project
12 private final Task task

The second step is to modify in the internal fields so that they can be recognised as being uninitialised. The most common case is to use null, but depending on your context this might be different. For containers you might want to simply leave them as empty. Now modify your project constructor to set some default values and leave them uninitialised in the task form of the extension.

 1 GnuMakeExtension(Project project) {
 2     this.project = project
 3     this.numJobs = 4
 4 }
 5 
 6 GnuMakeExtension(Task task) {
 7     this.task = task
 8 }
 9 
10 private Integer numJobs
11 private final Map<String,Object> flags = [:]

The final step is to retrieve the values in an intelligent manner. The logic is always to look in the task extension first and then in the project extension. However, this latter logic should only occur when an extension is attached to a task. All other entities in Gradle should not be aware of this behaviour. Start by adding a helper function that will always return the project extension

1 private GnuMakeExtension getProjectExtension() {
2     project ? this : (GnuMakeExtension)(task.project.extensions.getByName(NAME))
3 }

Now continue to modify the getters in the extension to check first whether the latter is attached to a project or a task and modify behaviour accordingly:

 1 Integer getNumJobs() {
 2     if(project) { 
 3         return this.numJobs
 4     } else {
 5         this.numJobs ?: getProjectExtension().getNumJobs()
 6     }
 7 }
 8 
 9 Map<String,Object> getFlags() {
10     if(project) {
11         this.flags
12     } else {
13         this.flags.isEmpty() ? getProjectExtension().getFlags() : this.flags
14     }
15 }

Having coded your extension it is now time to add it your plugin code and to your plugin’s task types. Create the global extension as per usual when the plugin is applied.

1 void apply(final Project project) {
2 
3     project.extensions.create(
4         GnuMakeExtension.NAME,
5         GnuMakeExtension,
6         project
7     )
8 }

The task extension is added when the task is instantiated. Assuming you have created a task type called GnuMakeTask, just add one line to the constructor.

1 GnuMakeTask() {
2     gnumake = extensions.create(GnuMakeExtension.NAME,GnuMakeExtension,this)
3 }
4 
5 private final GnuMakeExtension gnumake

Accessing the values in the extension simply becomes a case of referring to the local reference. For instance, you may want to add a method that returns the correct command-line parameter for concurrent jobs, This can be done as follows.

1 String getNumJobsParameter() {
2     "-j${this.gnumake.getNumJobs()}"
3 }

At this point all that remains is for a build script author to apply your plugin and make use of. Assuming that the build script creates two GnuMakeTask instances called makeProjectA and makeProjectB, configuration becomes readable and comprehensible.

 1 gnumake {
 2     numJobs = 10
 3 }
 4 
 5 makeProjectA {
 6     gnumake {
 7         defaultFlags BUILDNUM : '1234'
 8     }
 9 }
10 
11 makeProjectB {
12     gnumake {
13         numJobs = 1
14     }
15 }

A number of plugins in the field already make use of this recipe. If you want to study their usage and implementations have a look at Node.js + NPM plugins and well as the Packer plugin.

Gradle API Updates

The Gradle team has made advances in moving conventions to a public API. This approach are in many cases still more flexible and useful than conventions.

Caveats

• The above example simplifies the construction of task extensions and makes the assumption that it will always be added with the same name. If you are concerned about potential mis-use, then you should add a second parameter to the task constructor which takes the name of the project extension.
• When values in the task extension is modified, the task will not be out of date. If this behaviour is important to you, implement a task input property which can monitor changes in the task extension.

Grolifant

AbstractCombinedProjectTaskExtension is a base class inf Grolifant that can be used to simplify building of this recipe.

Migrate Extension to Unmanaged Model

Summary

Although Link Extension to Model is a good for a first migration, a plugin author might just decide to completely remove compatibility with older version of a plugin and embrace a new model approach. On the other hand, performing a Migrate Extension to Managed Model might be a step too far as keeping the syntax as close as possible to previous versions of the plugin might make migration for script authors less painful.

A plugin author might also be new to the new software model and might not be au fait with some of the intricacies of managed model elements and would prefer to settle for a more familiar unmanaged element.

Solution

Keep the existing extension class the same and enhance it with some declarative DSL methods. The latter will be necessary as unmanaged DSL object are not decorated as would have been the case with project extension objects. Finish it off by adding a model creation rule to create the unmanaged object.

Examples

Continuing on from Link Extension to Model, we can stay with the external tool metaphor. In this case out legacy extension might look something like below for GNU Make.

1 class ExternalToolExtension {
2     String executable = 'make'
3     List<String> execArgs = []
4 
5     void execArgs(String... args) {
6         this.execArgs.addAll(args as List)
7     }
8 }

We can keep our extension code mostly intact and enhance it with some declarative methods. (If we have anything that depends on the legacy Project object we need to rework those sections as we won’t have access to it anymore).

 1 class ExternalToolExtension {
 2     String executable = 'make'
 3     List<String> execArgs = []
 4 
 5     void execArgs(String... args) {
 6         this.execArgs.addAll(args as List)
 7     }
 8 
 9     void executable(String exe) {
10         this.executable = exe
11     }
12 }

All that remains is to add a model creation rule. As this is an unmanged model element our rule has to return an instance of the model element instead of void as would be the case for managed model elements.

1 class ExtensionContainerRules extends RuleSource {
2     @Model
3     ExternalToolExtension externalTool() { 
4         new ExternalToolExtension()
5     }
6 }

Once the rules are applied we have all of the greatness of the original extension object available within the model element.

 1 model {
 2     externalTool {          
 3         executable = 'gmake'
 4         execArgs = ['-i']   
 5         execArgs '-s','-B'  
 6     }
 7 
 8     externalTool {
 9         executable 'amake'  
10     }
11 }

Caveats

This is a suggested approach if DSL-compatibility with an older plugin version needs to be maintained. However, the following needs to be kept in mind:

• As the extension is unmanaged, Gradle can never guarantee the configuration to be immutable.
• Gradle will not decorate the extension with any other methods, and it is up to the plugin author to add the appropriate enhancements.

References & Credits

• Mark Viera clarified a number of caveats with this approach. [MViera3]

Appendix: Understanding the legacy native software model

Managed Data Annotations

@Managed

1 @Managed
2 interface ManagedDocker {
3     String getDockerHost()
4     void setDockerHost(String host)
5 
6     File getCertPath()
7     void setCertPath(File path)
8 }

At this point the managed class can already be used in a build script as long as it is available on the classpath.

1 model {
2     mysqlContainer(ManagedDocker) {
3         dockerHost 'https://192.168.99.100:2376'
4     }
5 }

This is effectively the equivalent of what a @Model annotation will create. By keeping a mental picture of this equivalence in mind, it will be easier for many of us plugins authors to transition from the classic Gradle way of thinking to this new style.

RuleSource Annotations

These are special annotations that are applied to methods in a class that extends RuleSource.

@Model

When applied to a method it indicates that a new top level element is created in the model space. This element takes the name of the method unlike the annotation is given a value. In the latter case the provided name must start with a lowercase character and only consist of ASCII word characters (Regex \w).

The element also has to provide a type, but the way of providing the type depends on whether this is a [Managed] type or not.

• Managed: The return type is always void. The first parameter is a type defined somewhere else and annotated with @Managed. The first parameter is not allowed to be unmanaged or annotated with @Path. (See [Managed] for more details. Any additional parameters are considered inputs.
• Unmanaged: The return type is the model type. All parameters are considered inputs.

1 class ExampleRules extends RuleSource {
2 
3     @Model
4     void docker(ManagedDocker md) {}
5 
6 }

When building the model rules for a managed type, the first parameter must always be the type and the return type is void. (This pattern is common across RuleSource rule annotation, but it is worth repeating here). The name of the method becomes the name of the model in the model registry.

1 apply plugin : ExampleRules

Reinforcing this equivalence of code and script, we should visualise this code as simply being.

1 model {
2     docker(ManagedDocker)
3 }

It is also to create a model without using a managed type. Consider that we rather wanted to implement the data class for out Docker descriptor ourselves. A simplified version of it might have looked something like this:

1 class NonManagedDocker {
2     String dockerHost = 'http://192.168.1.1:1234'
3     File certPath
4 }

In order to use this as a model, the method in our rules needs to return an instance of this class (as opposed to void in the managed case).

1 @Model
2 NonManagedDocker dockerNonManaged() {
3     new NonManagedDocker()
4 }

The @Model annotation also allows for setting a custom name that might be more suitable for use within a DSL than the method name. Simply use value=NewName as parameter to the annotation.

1 @Model(value='DrDocker')
2 void docker(ManagedDocker md) {}

@Defaults

Use the @Defaults annotation to set default property values for a subject. The name of the method is irrelevant as far as the build script author is concerned and should rather be named to describe the intent of setting the default value. The first parameter of the method is always the subject, which will be mutable for the duration of the method call. The return type is always void.

1 class DefaultExampleRules extends RuleSource {
2     @Model
3     void docker(ManagedDocker md) {}
4 
5     @Defaults
6     void defaultServer(ManagedDocker md) {
7         md.dockerHost = 'https://192.168.99.100:2376'
8     }
9 }

@Mutate

These are some of the most powerful kind of rules as they are not only used to set values on model subjects, but also to create tasks and other entities. The first parameter of a mutate method is always the subject, which will be mutable for the duration of the method call. Additional parameters can be supplied which can be used as inputs. The return type is always void.

@Finalize

According to gradle core developer, Mark Viera, there is very little difference between @Finalize and @Mutate rules. They are effectively just two groups of the same things, with all @Mutate rules being executed, before any @Finalize rules[MViera1]. Anything else that has been mentioned previously for @Mutate will apply equally here. From a idiomatic point of view, it is recommended that @Finalize rules are only used for final configuration in a similar fashion that project.afterEvaluate has been used in the classic model.

@Validate

Validation rules are used to check the integrity of properties before execution can begin. The first parameter of the method is always the subject, which will be immutable for the duration of the method call. It is the responsibility of the plugin author to terminate execution upon validation failure by throwing an appropriate exception. Groovy power asserts can also be used as shown below [MViera2]:

1 @Validate
2 void alwaysUseHttp(ManagedDocker md) {
3     assert md.dockerHost?.startsWith('http')
4 }

Inspecting the Model

Script authors should not need to do this, but as plugin author at some stage will probably have th need to look under the hood. This is especially the case when attempting to understand how the new model work the first time or even for some low-level unit tests. For this purpose use a method on the internal Project class namely modelRegistry. This is best described by means of a little Spock Framework test.

Revisiting our DefaultExampleRules class from earlier

1 class DefaultExampleRules extends RuleSource {
2     @Model
3     void docker(ManagedDocker md) {}
4 
5     @Defaults
6     void defaultServer(ManagedDocker md) {
7         md.dockerHost = 'https://192.168.99.100:2376'
8     }
9 }

we can author a test to check that the dockerHost property has a default value set.

 1 class DefaultExampleRulesSpec extends Specification {
 2     def project = ProjectBuilder.builder().build()
 3 
 4     def "Default rule must set dockerHost"() {
 5         given: "A simple model"
 6         project.allprojects {
 7             apply plugin : DefaultExampleRules
 8         }
 9 
10         def node = project.modelRegistry.find('docker',ManagedDocker)
11 
12         expect: "dockerHost to have default value"
13         node?.dockerHost == 'https://192.168.99.100:2376'
14     }
15 
16     def "Configuration overrides Default rules"() {
17         given: "A simple model"
18         project.allprojects {
19             apply plugin : DefaultExampleRules
20 
21             model {
22                 docker {
23                     dockerHost 'https://192.168.99.100:1234'
24                 }
25             }
26         }
27 
28         when:
29         def node = project.modelRegistry.find('docker',ManagedDocker)
30 
31         then:
32         node?.dockerHost == 'https://192.168.99.100:1234'
33     }
34 
35 }

Appendix: Legacy native software model configuration order

Bibliography

Discussion Forums

[AMurdoch1] Adam Murdoch. Obtaining the name of a component in model rules.

[LPelletier] Luc Pelletier. Limitation of generatedBy in native software model.

[MViera1] Mark Viera. @Finalize is not well described anywhere in the docs.

[MViera2] Mark Viera. @Validate needs a a little more clarification.

[MViera3] Mark Viera. Extension objects in new model.

Software

[GNUMake] Schalk W. Cronjé. GNU Make Gradle Plugin

[NodePlugin] Schalk W. Cronjé. Node.js Gradle Plugin

[PackerPlugin] Schalk W. Cronjé. Packer Gradle Plugin

[Terraform]

Books

[SCronje] Schalk Cronjé. ‘Idiomatic Gradle: 25 recipes for plugin authors’. Leanpub.