Reading and Rabbit Holes – The Workflow Project Gone Astray

Doug Gault ALL-THE-THINGS 7 Comments

Although I’ve been quiet as of late with regard to my “All-The-Things” workflow project, that doesn’t mean I’ve been sitting idle. No, in fact I’ve been running in circles at quite an alarming rate.

My last related post mentioned that I was ordering and would be reading “BPMN Method & Style” by Bruce Silver to try to get a better handle on BPMN. Little did I know the rabbit hole I was sending myself down.

I read it… twice! The first time was to get a handle on what BPMN really is, how it is intended to be used, and who it is aimed at. The second time was … Well, let me explain.

The book, although relatively small at 259 pages, is packed with extreme amounts of information. It includes history on BPMN and the author’s involvement in its guidance and definition, what makes “good” and “bad” BPMN and introduces the notation itself along with rules and guidelines for use. The author’s premise is that, armed with the same information about a process, if two very different people were to be given this information and the details outlined in his book, they would draw the same model. A lofty (if not virtually unattainable) goal.

The real kick in the gut comes when you reach Parts IV and V – The BPMN Implementor’s Guide (for Non-Executable and Executable BPMN respectively). The first section talks about serializing the model and the implied data flow, how to store it, reinstate it, etc. The second section talks about all things related to interpreting the model’s executability and the implied data flow therein, including differentiating between human-executed and automatable tasks, assignments, etc.

When it came time to talk about reference implementations, the author chose BonitaSoft Open Source, which was part of my review of the competition.  Mr. Silver walks through some example models and drills down into the data collected to make the tasks executable and/or automatable. He also exposes the XML serialization of the different parts of the model, the captured data and variables and some of the built-in function definitions available in the BonitaSoft product.

To say that the XSD that outlines the XML Serialization is confusing is akin to saying that the principles of thermodynamics are “a bit troublesome”.  But in all honesty, it needs to be complex. The enormity of the possible process definitions (and models that represent them) requires it to be so. The problem is that you can’t really cut this down to a “manageable” group. Even if you were to limit the possible number of symbols available in the model to Mr. Silver’s “Level 1 Pallet”, you’d still have an enormous job to do.

It was at this point that I put the book down and walked away.  I wasn’t necessarily giving up. I just needed to absorb what I had read. Digest it,  ruminate on it.

The book sat on my desk for a number of weeks. Occasionally I would pick it up, flip through to a section that I had a question about. Or even flip to a random section and see if luck or fate would provide any serendipitous insight.

Finally, about 2 weeks ago, I picked it up again and read it in full. This time, the core of what was being explained wasn’t what I was focused on. Instead I was focusing on how BPMN in general fits in to the project I had in mind, and whether it was forcing me outside of the parameters I had set for the project.

To remind you (and myself) of what those parameters are, here they are again:

  • Build a flexible, understandable and well documented workflow engine that is easy to integrate into any application and that will serve the majority of people’s needs.
  • Provide an easy to use GUI that allows users to create and change workflows visually. 
  • Provide a REST based API layer that allows external applications to start, progress, stop and query the state of a given workflow.
  • Provide several reference implementations in different technologies to show how the workflow API layer can be leveraged.
  • Make it freely available to anyone who wants it. 
  • Use Oracle based development tools to build the product.
  • Don’t try to boil the ocean!

At the end of the second reading, i was deflated. My initial excitement at BPMN’s potential to provide the framework for a flexible, understandable and well documented workflow engine with a solid commons based UI had turned in to a sense of dread to even move forward. So much complexity and work would be added just by introducing BPMN that the end product would not only not be simple, but would force the end user to learn a notation language that could take months if not years to master.

Looking back at the parameters I had set for this project I realized that not only would I completely shatter the first two in the list, it would mean that the time it would take to release anything useful would be extreme, even if i was working on it full time.

So I am at a crossroads. I have multiple paths ahead of me and I’m not sure which one to take. But since you’re on this journey with me, you can help me decide.  As I see it, my choices are as follows:

  1. I can keep going full steam ahead, build something that fully implements BPMN, is standards compliant  and that will likely take a very long time to complete. The end product would reach into the realms of products that are already available, such as BonitoSoft. The reason I don’t like this particular option is that my main overarching reason for starting this project was to explore, use and demonstrate the various Oracle development technologies. If I continue down this path, way over half of my time would be dedicated to integrating BPMN and related tools into the project.
  2. I can take what I’ve learned from reading about BPMN and build something that is non-standards based. The engine and UI would benefit from the ideas behind BPMN, but would not implement the modeling language, nor adhere to the restrictions. This would mean that it would be an isolated product that wouldn’t be able to share process information with other 3rd party systems. Building this system would also likely take a while as building a modeling UI from scratch wouldn’t be easy.
  3. I could set my focus on the core workflow engine itself, building a flexible and well documented system with comprehensive examples, documentation and API’s. One way to do this would be to completely re-work PL/FLOW. The core of what is there is good, it’s just hard to understand and the documentation and examples are horrible. By starting with something that I already know works, this could be a much quicker solution.
  4. I could abandon workflow completely. Before you judge to harshly, here is what I would suggest in it’s place. I would solicit requests from readers about topics related to APEX, ORDS, REST, PL/SQL, etc. Whether it be solving a specific problem, or doing a deep dive into a specific facet of one of the technologies, the blog series would be driven by what YOU want to see.

So here’s your chance.  I make no promises, but I’m definitely interested in your opinion.  Let me know what you think! The poll will be open through July 14, 2018.

What should I do about the Workflow Project?
  • Set your focus on a simple and well documented workflow engine akin to PL/FLOW. I'll create my own UI. 64%, 43 votes
    43 votes 64%
    43 votes - 64% of all votes
  • Take what you've learned and build a non-standards based engine and UI. I'm OK to wait for something good. 19%, 13 votes
    13 votes 19%
    13 votes - 19% of all votes
  • Ditch Workflow. I'd rather see a blog series of deep dives based on reader suggestions. 9%, 6 votes
    6 votes 9%
    6 votes - 9% of all votes
  • Keep going full steam ahead! Even if it takes Forever 7%, 5 votes
    5 votes 7%
    5 votes - 7% of all votes
Total Votes: 67
20-Jun-2018 - 14-Jul-2018
Voting is closed

Comments 7

  1. I’m afraid I will have to recuse myself from this vote. As I have a conflict of interest, and I have never wished to deter anyone from an ambitious venture.

    I can tell you that I feel your pain with BPMN. It does not lend itself well to a workflow engine that wants to keep it simple and ensure that users don’t need an advanced education in Business Process Design and Management. In that area you the programmer and designer have to bear the burden of work to put so much in the back-end so that the front works like magic.

    In any event, I vote for you to succeed, not matter the direction the community gives you.

  2. Most of the popular workflow systems (bonita, processmaker, decisions) started out with non-compliant systems and evolved into their current state. It would say you are probably right where one should expect to be on this journey. Market forces will push you towards more standards compliance in due time. Happy to guinea pig what you come up with no matter what.

  3. Maybe you could take a cue from AWS Simple Workflow Service and start simple and skip standards. Maybe just work towards a solution that fulfills some of your most pressing use cases.

    1. Post
      Author

      Thanks for mentioning this. Didn’t even know this existed so I’ll definitely take a look at what they offer. May be a decent model on which to base a new and improved PL/FLOW.

  4. i vote for “Set your focus on a simple and well documented workflow engine akin to PL/FLOW. I’ll create my own UI.”

    1. Post
      Author

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.