Last week, I had the privilege of having lunch with Keith Swenson. I have to admit I’m a fan of his “Go Flow” blog.

Keith Swenson is a fellow of the Workflow Management Coalition and he works for the R&D of Fujitsu Computers Systems. He has more than 15 years of experience in the workflow and business process management fields.

We talked about XPDL, federated workflows, RESTful BPM, WfXML-R and he gave me some explanations about the new Fujitsu process discovery service (I’d love to work on such a tool).

This meeting could have happened a bit earlier as we realized that our offices are just 4 blocks away from each other.

Keith gladly accepted to give a speech soon at a BPMinna meeting (Wakizaka-san, we have to talk about that).

This lunch was very instructive as well as friendly, I just forgot to ask for his business card, well, next time.

just released the “rufus-scheduler” 1.0.9 [gem].

It fixes a few bugs, mainly making jobs visible via get_job(job_id) even when they are right in the middle of execution.

It also introduces 4 new shorter aliases : “at”, “in”, “every” and “cron” (to “schedule_at”, “schedule_in”, “schedule_every” and “schedule” respectively) :

require 'rubygems'
require 'rufus/scheduler' # gem 'rufus-scheduler'

scheduler = Rufus::Scheduler.start_new

scheduler.in("3d") do
  regenerate_monthly_report()
end
  #
  # will call the regenerate_monthly_report method
  # in 3 days from now

scheduler.every "10m10s" do
  # every 10 minutes and 10 seconds
  check_score(favourite_team)
end

scheduler.cron "0 22 * * 1-5" do
  log.info "activating security system..."
  activate_security_system()
end

scheduler.at "Sun Oct 07 14:24:01 +0900 2009" do
  init_self_destruction_sequence()
end

Changelog :

- bug #21262 : brought back (proxy) duration_to_f to Rufus::Scheduler
- todo #21251 : added ‘at’, ‘cron’, ‘in’ and ‘every’ shortcut methods
- todo #21203 : keeping track of At and EveryJob until last trigger is over
- todo #20823 : now raising exception in case of bad ‘at’ parameter
- todo #21194 : added trigger_thread to Job class
- todo #21193 : spinned CronLine out to rufus/cronline.rb
- bug #20893 : sometimes unschedule(every_job) not working when job was active (performing). Fixed.

Links :

http://github.com/jmettraux/rufus-scheduler
http://rubyforge.org/projects/rufus/
http://rufus.rubyforge.org/rufus-scheduler/files/README_txt.html
http://groups.google.com/group/rufus-ruby

Thanks to Sean Liu, Adam and Rael for their help.

On Monday 30th of June, we were lucky enough to have a presentation by Petia Wohed of the Information Systems Laboratory of the University of Stockholm (and Royal Institute of Technology).

Petia contacted me last year while she was reviewing OpenWFE[ja] (and other open source workflow systems) in the light of the workflow patterns. I already blogged about the result of this work (see links at the bottom of this post).

Petia is a member of the YAWL project (workflow language and open source project) developed by the research groups of Prof. Wil van der Aalst and Prof. Arthur ter Hofstede. She delivered an excellent talk about this research effort and especially about the advantages of systems based on [Colored] Petri Nets (think Verification).

Before Petia’s presentation and my talk about OpenWFEru ruote, we had a demonstration by Zaky-san of iKnow! a social networking site for learning languages. It’s extremely successful in Japan (where it started) and it’s now expanding to the rest of the world.

Here is Petia’s presentation, it was graciously translated by Wakizaka-san to japanese :

What interested me the most was the worklet and the declarative workflow concepts. I will read carefully the papers they published about those.

Next was my talk about OpenWFEru, the slides are skinny, I rushed through them and then I demoed ruote-web, ruote-rest and the new fluo (process editing / rendering) :

Here are a few links :

• Petia’s blog post about the presentation
• YAWL - Yet Another Workflow Language - the project
• my blog post about Petia’s evaluation work
• the BPMinna, a Tokyo based japanese BPM interest group (ja)
• the workflow patterns
• Wakizaka-san’s first blog post (ja)
• Wakizaka-san’s second blog post (ja)
• Mark-Wada post (ja)
• Keep on Smilin’ post (ja)
• Shuyo-san’s report (ja)
• hikiko_mori’s report (ja)

(I’m certainly missing some blog reports, sorry)

Many thanks to Wakizaka-san and id:GoTheDistance for organizing those excellent BPMinna meetings (and many thanks as well to Sawada-san for inviting me to those meetings).

Until now I mainly wrote about exposing a workflow/BPM engine as a set of resources. I’ll write a few line about the other side. I could have named this post “RESTful orchestration”, but BPM is shorter and is music related as well.

For orchestration vs choreography, the best quote, via Stefan Tilkov, might be :

In orchestration, there’s someone — the conductor — who tells everybody in the orchestra what to do and makes sure they all play in sync.

One might argue that the conductor is merely a metronome that cares about BPM and that every participant knows where he fits in the ensemble. Would ‘orchestration’ rather refer to the preparation work of the execution ? Arrangements ?

Let’s forget about big words and acronyms, there’s work to do and no time for taking part in an already old argument. Let’s get started with the resources and where they live. I’ll go with a content management example, it’s a just a few steps away (not hardcore ROA).

There is one staging server, one preview server and two production servers. Content is developed on the staging server, pushed on the preview server for approval and then passed to the production servers.

shell scripts

The “push to the preview server” might look like :

#!/bin/bash

# to_preview.sh

curl http://staging.example.com/pages/$0 > tmp.out

curl -X PUT \
  --data-binary "@tmp.out" \
  -H "Content-Type: text/plain" \
  http://preview.example.com/pages/$0

curl -X POST \
  -d "uri=http://preview.example.com/pages/$0" \
  -d "title=please approve" -d "team=review" \
  http://preview.example.com:4567/tasks

Which deals with “/pages” resources getting and putting, and finally creating a task for the review team (a “/tasks” resource on the preview host). (thanks curl)

Note that this script (and this post) doesn’t care about authentication and authorization, nor about what’s the content type of our pages (it’s also limited to one ‘page’ what about batches of them ?).

The content producer and designers push for preview by running :

./to_preview.sh summer_catalog.html

I could show the scripts for refusing the preview material or for approving and publishing it, but they’re not that hard to deduce.

All is well, provided that the producers/designers are not too averse to command line tools. There are just two kinds of resource to orchestrate, pages and tasks, and three sets of page resources (staging, preview, production).

But this workflow is kind of loose, there seem to be no “conductor”, just “agents” pushing stuff around with tiny scripts. Business Process Management is hard, let’s go shopping… I mean, this business process is hard to manage, because it is fragmented.

business process scripting

“There he goes again…”, let’s try to do that with a workflow/BPM engine. Feel free to just click away.

A workflow engine deals with “long running asynchronous processes”, that means it is patient enough to wait for replies, to play its conductor role. And if interrupted/restarted, it will wait again.

It would be nice to have one process definition to track this and all the implicit requirements :

There is one staging server, one preview server and two production servers. Content is developed on the staging server, pushed on the preview server for approval and then passed to the production servers.

The “business process” could look like :

class PubProcess0 < OpenWFE::ProcessDefinition
  cursor do

    participant "producer"

    hget "http://staging.example.com/pages/${f:page}"
    hput "http://preview.example.com/pages/${f:page}"

    participant "reviewer"

    cancel :if => "${f:decision} == cancel"
    rewind :if => "${f:decision} == redo"
    # else publish...

    hget "http://staging.example.com/pages/${f:page}"
    concurrence do
      hput "http://prod1.example.com/pages/${f:page}"
      hput "http://prod2.example.com/pages/${f:page}"
    end
  end
end

This definition uses the cursor expression and its ‘rewind’ and ‘cancel’ subexpressions, as well as the new http expression (hget, hput, hpost, … more on it later)

Looks neat but the interaction with the /tasks resource vanished, maybe I’ll be using the worklist embedded in ruote-web or ruote-rest.

But what if we really wanted to use that /tasks resource ?

cursor do
  #...

  #participant "reviewer"

  hpost(
    "http://preview.example.com/tasks",
    :params => {
      :uri => "http://preview.example.com/pages/$0"
    })

  hpoll(
    "${f:hheaders.location}",
    :until => "${f:hcode} == 404")

  #...
end

Assuming that the ‘hpost’ created successfully the task and was kind enough to reply with a “Location” header, the ‘hpoll’ expression would block the process, waiting for the ‘until’ condition to realize (the condition here is quite silly, it will poll every ten seconds until the /task/{id} resource goes 404, ‘task not found’ being interpreted as ‘task executed’).

RESTful orchestration

Has it been reached ? By adding the HTTP verbs to a business process definition language ?

Isn’t that a kind of specialization ? BPEL was born with a similar specialization a few years ago.

The move from “participant ‘reviewer’” to the “hpost/hpoll” pair is a specialization. With the ‘participant’ the details are abstracted away from the business process [definition], with hpost/hpoll there is an assumption of a resource oriented architecture/context.

the hget/hpost/…/hpoll expressions are a few hours of work, they are not finished. I meant to include them in Ruote (OpenWFEru) since long ago but this “specialization” issue refrained me. I feel they are not true to the “participants” vs “definitions” concept / dichotomy.

I don’t want to invest too much time into those new expressions. They should only cover basic cases. Complex cases should be abstracted to participant implementations.

(I started OpenWFE in Java a few years ago. I wanted to implement an open source workflow engine. In 2006, I switched to Ruby, this language has this huge expressiveness gain. I could [re]write the engine with less code. But I was/am also banking on people doing the last mile by themselves. Need a report ? Check prawn and write it by yourself. I don’t have to write all the participant implementations, I don’t have to cover all cases and all options. Ruby with its huge expressiveness and its openness (and its stock of gems) is making it easy for people to plug in their participants)

Anyway, those new Ruote expressions were fun to implement, they are available in the upcoming 0.9.19 release of OpenWFEru Ruote. Feedback is welcome on the user mailing list.

post scriptum / 303

I have not shown how to do authentication here. I have not talked about conditional GETs. Exceptions and retrying have not been discussed either. Maybe yet another incentive for hiding those details under the roof of a [custom] participant implementation.

RESTful BPM… Jason thinks there might be an elephant in the room.