AWS CodeCommit – Update Workflow Tool on Commit

AWS CodeCommit – Update Workflow Tool on Commit

If you use a workflow tool such as Trello, Assembla or Jira to help manage software development and are also using Git for your source control, you will often want to automatically update or add a comment to cards whenever a new commit is made. Luckily, you can do this easily if you are using AWS CodeCommit by creating a new AWS Lambda function that is automatically triggered when a push occurs to a repository. The Lambda can then retrieve the commit message, and if it contains enough information, can automatically update the ticket for you.

In this open source tutorial (all code is available on GitHub), we will be building a Java Lambda function that does just that. For this tutorial we will be using the Java 8 JDK and Gradle and so I recommend using the free IntelliJ Community Edition as it all integrates very well together, but any IDE should work.

The first thing we want to do is create our build.gradle file and directory structure. In the gradle file, we do a few things, but importantly we bring in the aws-lambda-java-core as a compileOnly dependency as it is already provided at runtime by AWS. We then bring in aws-lambda-java-events as we want the type safety of the repository push event (cover more of the later) and finally to retrieve the commit message we need the aws-java-sdk-code-commit library.

Then we will enhance the jar task provided by the java plugin to build a “Fat Jar” that contains all these dependencies in our single deployable artifact that we will upload to AWS Lambda as our function.

apply plugin: 'java'
apply plugin: 'idea'

repositories {
    jcenter()
}

dependencies {
    compileOnly 'com.amazonaws:aws-lambda-java-core:1.2.0'
    compile 'com.amazonaws:aws-lambda-java-events:2.2.6'
    compile 'com.amazonaws:aws-java-sdk-codecommit:1.11.592'

    testCompile 'junit:junit:4.12'
    testCompile 'org.mockito:mockito-core:2.10.0'
}

jar {
    dependsOn configurations.runtime
    from {
        configurations.runtime.collect {
            it.isDirectory() ? it : zipTree(it)
        }
    }
}

Then in src/main/java/com/mannanlive/domain we are going to create a new interface to communicate to our workflow tool, WorkflowClient.java. This interface defines the two functions we are going to invoke when we find a matching commit message, place a comment on the card and to update it’s status (if possible). This could be easily extended to add labels, close cards or even create new ones just from the contents of a commit message.

package com.mannanlive.domain;

public interface WorkflowClient {
    boolean addComment(final String board, final String cardId, final String comment);
    boolean updateStatus(final String board, final String cardId, final String status);
}

You can view the implementation of these client’s for Assembla and Trello on GitHub, or you can make your own. To use these you will need to generate an API Key and Secret to authenticate and connect to the workflow tool, which you can find details online on how to do this, for example Assembla and for Trello.

The next thing is to write a class that can extract relevant information from a commit message. The approach taken in src/main/java/com/mannanlive/service/TicketExtractor.java is to use a symbol to indicate that the following characters is the ticket or card identifier. For example if the symbol is hash, #1234 would indicate that the card id is 1234. There is also a regular expression for the card id format itself, for instance Assembla only uses numbers so \d+ will work to match it, Trello uses a mix of lowercase or numbers so you can use [a-z0-9]+, Jira uses letters and numbers like ABC-123 so you could use [A-Z]+-[0-9]+ to match these formats.

The third and final purpose of this class is to identify the preceding word to the card id, so it can detect “Test #1234” and assign the Test status or column to the card, if you had a status of Done, you could in your commit message write “Fixed issue with code, done #1234”. You can also mention multiple cards in the same message like “Ready to Test #1234, #1235” will add both cards to the test phase, or you can do “Done #1234, Starting Dev #1235” to update 1234 to “Test” and 1235 to “Dev”.

We then need to create a class to format the commit message into a comment for a card. We want to include the author of the commit, the message in the commit and a link to AWS CodeCommit console to view the affected files and other details. To do this, different workflow tools have different markups so we need to create one for each provider, for example this is the Trello one.

package com.mannanlive.domain;

import com.amazonaws.services.codecommit.model.Commit;
import static java.lang.String.format;

public abstract class WorkflowComment {
    private final String region;
    private final String repositoryName;

    public WorkflowComment(final String region, final String repositoryName) {
        this.region = region;
        this.repositoryName = repositoryName;
    }

    public abstract String calculate(final Commit commit);

    protected String getUrl(final Commit commit) {
        return format("https://%s.console.aws.amazon.com/codesuite/codecommit/repositories/%s/commit/%s?region=%s",
                region, repositoryName, commit.getCommitId(), region);
    }

    public String getRepositoryName() {
        return repositoryName;
    }
}

package com.mannanlive.domain.trello;

import com.amazonaws.services.codecommit.model.Commit;
import com.mannanlive.domain.WorkflowComment;
import static java.lang.String.format;

public class TrelloComment extends WorkflowComment {
    public TrelloComment(final String region, final String repositoryName) {
        super(region, repositoryName);
    }

    public String calculate(final Commit commit) {
        return format("%s has [committed a change](%s) related to this ticket:\n\n```%s```",
                commit.getAuthor().getName(), getUrl(commit), commit.getMessage());
    }
}

Now we need to put this all together in a src/main/java/com/mannanlive/service/WorkflowTool.java class that will get the message from the commit, for each ticket or card it finds in the message add a comment to it, and if possible, update it’s status.

public void process(final Commit commit) {
    ticketExtractor.extract(commit.getMessage()).forEach(ticket -> {
        final String message = comment.calculate(commit);
        if (workflowClient.addComment(spaceOrBoard, ticket.getTicketId(), message)) {
            workflowClient.updateStatus(spaceOrBoard, ticket.getTicketId(), ticket.getAction());
        }
    });
}

In order to know which workflow tool to use (Trello, Jira etc) we can use the Factory or Builder Pattern to create a WorkflowToolBuilder.java class to create an instance of a WorkflowTool for our Lambda function to use. This factory receives a CodeCommit record that our handler will be given (we build this in the next step) and can extract the custom data that is used to setup the CodeCommit commit trigger to know; the type of workflow tool, the related board or space and finally the Git repository name.

public class WorkflowToolBuild {
    public WorkflowTool create(final CodeCommitEvent.Record record) {
        final String[] segments = record.getCustomData().split(":");
        final String tool = segments[0];
        final String spaceOrBoard = segments[1];
        final String region = record.getAwsRegion();
        final String repositoryName = record.getEventSourceArn().split(":")[5];

        switch (tool.toLowerCase()) {
            case "assembla":
                return new WorkflowTool(
                        new AssemblaComment(region, repositoryName),
                        new AssemblaWorkflowClient(),
                        new TicketExtractor("#", "\\d+"),
                        spaceOrBoard);
            case "trello":
                return new WorkflowTool(
                        new TrelloComment(region, repositoryName),
                        new TrelloWorkflowClient(),
                        new TicketExtractor("#", "[a-z0-9]+"),
                        spaceOrBoard);
            default:
                throw new IllegalArgumentException(tool + " is not a supported workflow tool");
        }
    }
}

Now we can build our workflow tool, all that is left is to plug it into a Lambda handler. Creating a src/main/java/com/mannanlive/CodeCommitHandler.java class, it accepts a CodeCommitEvent and returns void (null). You might wonder where did the RequestHandler and CodeCommitEvent come from? Remember when in the build.gradle we added aws-lambda-java-core and aws-lambda-java-events lets us take advantage of these ready to use components to rapidly build Lambda functions.

package com.mannanlive;

import com.amazonaws.services.codecommit.model.Commit;
import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;
import com.amazonaws.services.lambda.runtime.events.CodeCommitEvent;
import com.mannanlive.service.WorkflowTool;
import com.mannanlive.service.CodeCommitRepository;
import com.mannanlive.service.WorkflowToolBuilder;

public class CodeCommitHandler implements RequestHandler<CodeCommitEvent, Void> {
    private final CodeCommitRepository codeCommit = new CodeCommitRepository();

    public Void handleRequest(final CodeCommitEvent input, final Context context) {
        input.getRecords().forEach(record -> {
            final WorkflowTool workflowTool = new WorkflowToolBuilder().create(record);
            record.getCodeCommit().getReferences().forEach(reference -> {
                final Commit commit = codeCommit.getCommit(workflowTool.getRepositoryName(), reference.getCommit());
                workflowTool.process(commit);
            });
        });
        return null;
    }
}

Now we are ready to ship our new function to the cloud. Gradle has a great gradle-aws-plugin to automate a lot of annoying or repetitive tasks. If you enhance the build.gradle file to the value below, you will add the ability to create a new IAM role that the Lambda can use to access CodeCommit and also create and update a new Lambda function. Hopefully most of it should be self explanatory or covered in previous posts.

import jp.classmethod.aws.gradle.lambda.AWSLambdaMigrateFunctionTask
import jp.classmethod.aws.gradle.lambda.AWSLambdaUpdateFunctionCodeTask
import jp.classmethod.aws.gradle.identitymanagement.AmazonIdentityManagementCreateRoleTask

// grab the AWS plugin
buildscript {
    repositories {
        maven { url 'https://plugins.gradle.org/m2/' }
    }
    dependencies {
        classpath 'jp.classmethod.aws:gradle-aws-plugin:0.37'
    }
}

apply plugin: 'jp.classmethod.aws.lambda'
apply plugin: 'jp.classmethod.aws.iam'

// Original Content //

aws {
    profileName = 'default'
    region = 'ap-southeast-2'
}

task createRole(type: AmazonIdentityManagementCreateRoleTask) {
    doFirst {
        roleName = 'lambda-codecommit'
        assumeRolePolicyDocument = '{"Version": "2012-10-17", "Statement": [' +
                '{"Effect": "Allow", "Principal": {"Service": "lambda.amazonaws.com"}, "Action":"sts:AssumeRole"}' +
                ']}'
        policyArns = [
                'arn:aws:iam::aws:policy/AWSCodeCommitReadOnly',
                'arn:aws:iam::aws:policy/AWSLambdaBasicExecutionRole'
        ]
    }
}

task createLambda(type: AWSLambdaMigrateFunctionTask, dependsOn: jar) {
    doFirst {
        role = "arn:aws:iam::${aws.accountId}:role/lambda-codecommit"
        zipFile = jar.archiveFile.get().getAsFile()
        memorySize = 256
        timeout = 60
        runtime = 'java8'
        functionName = 'codecommit-lambda'
        environment = [
                API_KEY   : 'YOUR_API_KEY!',
                API_SECRET: 'YOUR_API_SECRET!',
        ]
        handler = 'com.mannanlive.CodeCommitHandler::handleRequest'
        description = 'Updates project management tool when a new push is received'
    }
}

task updateLambda(type: AWSLambdaUpdateFunctionCodeTask, dependsOn: jar) {
    doFirst {
        functionName = 'codecommit-lambda'
        zipFile = jar.archiveFile.get().getAsFile()
    }
}

Now we are ready to create our Lambda, to do this we can run ./gradlew createRole createLambda that will create everything that we need. If you make a code change, you can run ./gradlew updateLambda to push an update. Finally we need to configure our CodeCommit repository to invoke this function and let it know what Workflow tool to use and what board or space is related to the repository. We can do this using the AWS CLI or you can use Console if you prefer. Just remember to change the <variables> to your specified values.

aws codecommit put-repository-triggers --repository-name <repository name> --triggers \
  name=Commit-Trigger,destinationArn=arn:aws:lambda:<region>:<account number>:function:codecommit-lambda,customData=<worktool>:<space>,branches=[],events=updateReference

With your fingers crossed you should now be able to make a commit to your repository and then check your workflow tool and see a comment and possibly an updated status! If not, make sure to check the CloudWatch logs for details, or you can try a test event (with a real commit hash) and see if you can pinpoint any errors.

If anything is unclear or missing please have a look at the GitHub source repo or leave a reply below. Thanks!