How to consume Amazon SWF

Question 1

How to consume Amazon SWF

amazon-web-services amazon-swf

sdolgy · Feb 22, 2012 · Viewed 11.8k times · Source

Answer

Answer

I'm using Amazon Simple Workflow Service (SWF) to implement asynchronous business processing using the AWS Flow Framework. It was important to me to have my development build setup using Maven, so that I could easily build from my IDE of choice (IntelliJ IDEA) as well as automate my test builds for continuous integration and production builds for release and deploy.

Most of my time was spent trying to get the auto-generated proxy classes created using AspectJ. This was initially a problem for me in Eclipse as I was using version 3.7 (Indigo) and even after following both the load-time and compile-time weaving instructions in the Setting up the Development Environment documentation I was unable to successfully get the classes blown out. On a hunch I remembered that the documentation says they used Eclipse 3.6 (Helios), so I downloaded this specific version of Eclipse and retried using the load-time weaving approach and it worked like a champ. Looking at the Eclipse logs between these two versions I was able to see that Eclipse 3.7 is missing a dependency for log4j and freemarker. I didn't bother going too far down the road to troubleshoot this further with Eclipse as I'm more of an IntelliJ IDEA user, but I'm sure that it's most definitely possible to get Eclipse working properly.

My next effort was to get an IntelliJ IDEA Maven project up and running with the minimum amount of information in my pom.xml to enable the auto-generation of the proxy classes. The clue was the last paragraph of the instructions for load-time weaving in the Setting up the Development Environment documentation, which states:

If you are building your project from the command line, ensure that aws-java-sdk-flow-build-tools-1.3.3.jar is in the classpath. This jar file contains the AWS Flow Framework annotation processor that must be run to generate code. For an example, see the build.xml file included in the samples folder.

Unless I'm mistaken, the research I've done to date indicates that the aspectj-maven-plugin doesn't not currently support load-time weaving. Breaking away from doing load-time weaving and utilizing an aop.xml file in conjunction with aspectjweaver running as a Java agent, I moved to compile-time weaving supported by this plugin. I can't help but think that when I installed the AWS SDK for Java support directly in Eclipse that it in turn baked in the support found in the aws-java-sdk-flow-build-tools-1.3.3.jar dependency. Taking the aforementioned clue, I was finally able to get the proxy classes blowing out by including a dependency for aws-java-sdk-flow-build-tools-1.3.3.jar in my pom.xml after installing this JAR into my local Maven repository. The rest of this entry outlines the steps taken to make this all happen.

IntelliJ IDEA Project Creation

Start the IntelliJ IDEA application.
Create a New Project.
Input the Project name and Project files location.
Select type should be Maven Module.
Adjust any Maven properties if appropriate on next screen and click Finish.
Follow the Maven instructions below for setting up the pom.xml.

AWS SDK for Java

In order to reference the necessary types for developing workflows and activities, you will need to download the AWS SDK for Java. It is recommended that you use Maven to include a dependency for this library for your project and build, but you should still download this library in order to get the aws-java-sdk-flow-build-tools library which can be found under the lib directory.

SWF Flow Build Tools

The AWS SDK for Java download includes a aws-java-sdk-flow-build-tools-<version>.jar JAR under the lib directory. In order to allow the inclusion of a Maven dependency for this library, you'll need to install the JAR into your local Maven repository. You can achieve this by running the following from the lib directory of the AWS SDK download:

mvn install:install-file 
    -Dfile=aws-java-sdk-flow-build-tools-<version>.jar 
    -DgroupId=com.amazonaws 
    -DartifactId=aws-java-sdk-flow-build-tools 
    -Dversion=<version> 
    -Dpackaging=jar

NOTE: Be sure to replace the tokens in the above command with the appropriate version found in your AWS SDK download.

Maven

Your pom.xml file should include the following dependencies. I've included the version numbers I'm using in case you run into breaking changes using different versions:

<dependencies>
    <dependency>
        <groupId>org.aspectj</groupId>
        <artifactId>aspectjrt</artifactId>
        <version>1.6.11</version>
    </dependency>
    <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-java-sdk-flow-build-tools</artifactId>
        <version>1.3.3</version>
    </dependency>
    <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-java-sdk</artifactId>
        <version>1.3.3</version>
    </dependency>
    <dependency>
        <groupId>org.freemarker</groupId>
        <artifactId>freemarker</artifactId>
        <version>2.3.18</version>
    </dependency>
</dependencies>

Your pom.xml file should also include the following plugin. I'm using source include patterns that following a packaging and interface naming convention I use in my code, though you don't necessarily need to do things this way:

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>aspectj-maven-plugin</artifactId>
    <version>1.4</version>
    <configuration>
        <complianceLevel>1.5</complianceLevel>
        <showWeaveInfo>true</showWeaveInfo>
        <verbose>true</verbose>
        <sources>
            <source>
                <basedir>src/main/java</basedir>
                <includes>
                    <include>*/**/workflow/*Workflow.java</include>
                    <include>*/**/workflow/activities/*Activities.java</include>
                </includes>
            </source>
        </sources>
    </configuration>
    <executions>
        <execution>
            <goals>
                <goal>compile</goal>
                <goal>test-compile</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Once you've include the plugin listed above and have at least run a Maven compile, you should notice an aspectj node appear under the Plugins node within the Maven Projects tool window in IntelliJ IDEA. You can also optional add or tweak the elements of the configuration section of the aspectj-maven-plugin plugin if you desire. The various supported settings can be found in the aspectj:compile goal documentation found here. I've haven't tweaked my plugin configuration yet to ensure that the .java files are generated in the proper location under my source directory, though I'm sure this is quite doable.

External Libraries

Once you've include the set of dependencies listed above and have at least run a Maven compile, you should notice at minimum the following set of dependencies listed under the External Libraries node within the Project tool window in IntelliJ IDEA:

com.amazonaws:aws-java-sdk-flow-build-tools
com.amazonaws:aws-java-sdk
commons-codec:commons-codec
commons-logging:commons-logging
org.apache.httpcomponents:httpclient
org.apache.httpcomponents:httpcore
org.aspectj:aspectjrt
org.codehaus.jackson:jackson-core-asl
org.codehaus.jackson:jackson-mapper-asl
org.freemarker:freemarker

Example pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>swf_example</groupId>
    <artifactId>swf_example</artifactId>
    <version>1.0</version>

    <dependencies>
        <dependency>
            <groupId>org.aspectj</groupId>
            <artifactId>aspectjrt</artifactId>
            <version>1.6.11</version>
        </dependency>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-java-sdk-flow-build-tools</artifactId>
            <version>1.3.3</version>
        </dependency>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-java-sdk</artifactId>
            <version>1.3.3</version>
        </dependency>
        <dependency>
            <groupId>org.freemarker</groupId>
            <artifactId>freemarker</artifactId>
            <version>2.3.18</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>aspectj-maven-plugin</artifactId>
                <version>1.4</version>
                <configuration>
                    <complianceLevel>1.5</complianceLevel>
                    <showWeaveInfo>true</showWeaveInfo>
                    <verbose>true</verbose>
                    <sources>
                        <source>
                            <basedir>src/main/java</basedir>
                            <includes>
                                <include>*/**/workflow/*Workflow.java</include>
                                <include>*/**/workflow/activities/*Activities.java</include>
                            </includes>
                        </source>
                    </sources>
                </configuration>
                <executions>
                    <execution>
                        <goals>
                            <goal>compile</goal>
                            <goal>test-compile</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

</project>

Defining Workflows

To define a workflow you must create a Java interface that meets the following criteria:

The interface is annotated with @Workflow.
A single method is defined for the interface that is annotated with @Execute and has a version property set.
The interface has a name that ends with the string Workflow.
The interface resides under a package that ends with workflow.

The following is an example workflow interface named MyWorkflow, and would be contained a file named MyWorkflow.java, located under the source directory src/main/java, and under the package directory structure com/some/package/workflow. I haven't included the @WorkflowRegistrationOptions annotation or other bells and whistles as these aren't required and are dependent on your particular needs:

package com.some.package.workflow;

import com.amazonaws.services.simpleworkflow.flow.annotations.Execute;
import com.amazonaws.services.simpleworkflow.flow.annotations.Workflow;

@Workflow
public interface MyWorkflow {

    @Execute(version="1.0")
    void doWorkflow();

}

Defining Activities

To define activities you must create a Java interface that meets the following criteria:

The interface is annotated with @Activities and has a version property set.
The interface has a name that ends with the string Activities.
The interface resides under a package that ends with workflow/activities.

The following is an example activities interface named MyActivities, and would be contained in a file named MyActivities.java, located under the source directory src/main/java, and under the package directory structure com/some/package/workflow/activities. I haven't included the @ActivityRegistrationOptions annotation or other bells and whistles as these aren't required and are dependent on your particular needs:

package com.some.package.workflow.activities;

import com.amazonaws.services.simpleworkflow.flow.annotations.Activities;

@Activities(version="1.0")
public interface MyActivities {

    void doActivity1();
    void doActivity2();
    void doActivity3();

}

Building

In order to ensure that the build process is the same during everyday development as well as for non-development environments (e.g. test, production, etc.), you should run builds in your development tool of choice through Maven. Various aspects have been included in the aws-java-sdk and aws-java-sdk-flow-build-tools JARs which are weaved into your workflows and activities, and the aws-java-sdk-flow-build-tools JAR includes the necessary mechanism to auto-generate the required proxy classes to execute workflows and activities. In order to ensure that you're working with the latest generated proxy classes, you should take care to clean the generated artifacts before a build in order to throw away unneeded and/or old classes. This can be achieved by running the following command or equivalent in your development tool of choice:

mvn clean install

If you keep the showWeaveInfo configuration option enabled in the aspectj-maven-plugin plugin, you should see something like the following snippet in your build output, albeit there are only have a few lines of output here due to only having a single workflow and single activities for this run:

Mar 12, 2012 5:21:22 PM com.amazonaws.eclipse.simpleworkflow.asynchrony.annotationprocessor.AsynchronyDeciderAnnotationProcessor process
INFO: AsynchronyDeciderAnnotationProcessor.process() invoked.
Mar 12, 2012 5:21:22 PM com.amazonaws.eclipse.simpleworkflow.asynchrony.annotationprocessor.AsynchronyDeciderAnnotationProcessor process
INFO: Processing @Activities for MyActivities
Mar 12, 2012 5:21:22 PM com.amazonaws.eclipse.simpleworkflow.asynchrony.annotationprocessor.AsynchronyDeciderAnnotationProcessor process
INFO: Processing @Workflow for MyWorkflow
Mar 12, 2012 5:21:22 PM com.amazonaws.eclipse.simpleworkflow.asynchrony.annotationprocessor.AsynchronyDeciderAnnotationProcessor process
INFO: AsynchronyDeciderAnnotationProcessor.process() invoked.

Auto-Generated Proxies

Once you've compiled your workflows and activities you should find the follow set of auto-generated proxy classes have been created. These proxies are to be used within your workflows to call upon your various activities, execute child workflows within other workflows, and also to execute workflows at a top level. NOTE: The strings "Workflow" and "Activities" in the following bullets would actually be the name of your actual workflow and activities interfaces respectively, and you should see the following set of classes created for each of your defined workflow and activities interfaces:

WorkflowClient.java
WorkflowClientExternal.java
WorkflowClientExternalFactory.java
WorkflowClientExternalFactoryImpl.java
WorkflowClientExternalImpl.java
WorkflowClientFactory.java
WorkflowClientFactoryImpl.java
WorkflowClientImpl.java
WorkflowSelfClient.java
WorkflowSelfClientImpl$1.java
WorkflowSelfClientImpl.java
ActivitiesClient.java
ActivitiesClientImpl.java

I'm also including some background information to help clarify the type of development environment I'm working on and tools I'm using for day to day coding.

OS

Mac OS X version 10.7.3

Java

java version "1.6.0_29"
Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-11D50b)
Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode)

Maven

Apache Maven 3.0.3 (r1075438; 2011-02-28 12:31:09-0500)
Maven home: /usr/share/maven
Java version: 1.6.0_29, vendor: Apple Inc.
Java home: /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
Default locale: en_US, platform encoding: MacRoman
OS name: "mac os x", version: "10.7.3", arch: "x86_64", family: "mac"

IntelliJ IDEA (Community Edition)

IntelliJ IDEA 11.0.2
Build #IC111.277
Built on February 1, 2012

Question 2

Amazon SWF was launched today. How best to consume it with Java / PHP / etc. ?

The current SDK support doesn't appear to include it. I know it's new, but does anyone have any good resources on how to consume it, or what changes I'd need to implement in the any of the following SDK's to get going right away?

Personally, my interest is on the Java & PHP SDK's...

Updated releases are visible at: http://aws.amazon.com/releasenotes Thanks Bjorn!

How to consume Amazon SWF

Answer

IntelliJ IDEA Project Creation

AWS SDK for Java

SWF Flow Build Tools

Maven

External Libraries

Example pom.xml

Defining Workflows

Defining Activities

Building

Auto-Generated Proxies

Related questions