ALL THE THINGS – Project Outline

Doug Gault ALL-THE-THINGS 22 Comments

In my previous post I mentioned that I’d be developing a non-trivial, real-world application from scratch and approach it in the same way I would if I were doing it for a paying client. The first step in that direction is to gather requirements and define the project:

  • What is the main purpose of the project and what need does it satisfy?
  • Are there real and artificial limitations placed on the project?
  • What technologies will be used to satisfy the requirements? Why?

There are obviously loads more questions that we need to ask, and answer. But lets start with these three. The rest of this post will provide a broad outline for the project, where I’m going to try to set the boundaries, and why I chose this specific idea to tackle.

The Problem of Workflow

Since my early days working with the Oracle database, all the way into my tenure with Oracle APEX, workflow has been something of a conundrum. When most people see or hear workflow it usually goes one of two ways: They think of something specific to their business and therefore fairly concrete (in their minds at least) or they think of something wildly generic that will suit every case that ever existed and ever will exist.

As a long time consultant programmer, every time I discuss workflow with a client it usually goes something like this.

Client: We need some sort of workflow engine so that we can track the progress of Process A through its multiple steps.

Me: Ok, I’d love to understand more about the steps in this process, how complex is it?

Client: Oh, very complex. There are multiple steps, various people involved, various decision points that may change the outcome and we have restrictions on how long the process can take.

Me: How “static” is the process? Do the steps and process change over time or has the process been fairly stable?

Client: It’s fairly stable, but we’d like to be able to react quickly to new business drivers that may push changes to the process. We pride ourselves on being a nimble company and that’s what keeps us competitive.

Me: Understood. Sounds like you need a full-fledged workflow engine that allows you full flexibility to adjust process definitions. There are a number of commercial solutions out there. Are you interested in looking at any of them?

Client: We’ve actually done a bit of research and the commercial solutions are either too complex, too expensive or too hard to integrate. We’d rather have something written that suits our specific needs.

Me: [ … silence with a blank expression … ]

At this point it turns into the age-old discussion of the value of buy vs. build. The cost of maintaining home-spun software and the value of a supported product. If they truly choose to go down the “roll-your-own” path, then there is inevitably the discussion of re-inventing the wheel.

In my experience, more times than not, the client abandons the entire idea of a flexible workflow engine and ends up hardcoding the process steps into whatever platform they’re using to develop. I can’t tell you how many clients I’ve built code for that, as soon as the process steps need to change, they’d have to rip apart and rewrite.

There have been a few clients over my 30+ years of doing this that, while they didn’t want to buy a huge workflow package (feeling it was overkill), did want truly flexible workflow. For these customers we’d develop something that was flexible enough to suit their business model, but it was so tightly integrated to their core system that it really wasn’t usable by anyone else in the company.

One (1!) customer let me convince them to implement PL/FLOW and integrate that into their system. This worked well, and they were happy with it for quite a while. Then when they wanted to integrate it with another system, they were so confounded by the whole thing, and because they weren’t able to get support (and didn’t want to pay me to help them) they ripped it out. I’m still not sure what they did to replace it, but alas it is not my problem now.

What do we take away from all this? Workflow is not straight forward. It’s hard – hard to understand, hard to implement, hard to integrate into existing systems. It’s just hard. So this, in essence, defines the answer to the first question I outlined above. What is the main purpose of the project and what need does it satisfy:

Build a flexible, understandable and well documented workflow engine that is easy to integrate into any application and that will serve the majority of people’s needs.

Provide an easy to use GUI that allows users to create and change workflows visually. 

Provide a REST based API layer that allows external applications to start, progress, stop and query the state of a given workflow.

Provide several reference implementations in different technologies to show how the workflow API layer can be leveraged.

Make it freely available to anyone who wants it. 

Don’t try to boil the ocean!

Lofty goals indeed, but that’s the gist of what I’ll be trying to do here.

Building Real Workflow or Re-Inventing the Wheel?

As I mentioned above, there are quite a few commercial, scalable and flexible workflow products out there on the market. Just Google “Workflow Engine” and you’ll get north of 6,200,000 entries.  Google “Free Workflow Engine” and that number gets cut in half. Still a significant number, don’t get me wrong!

So why build Yet Another Workflow Engine1?  There are a few reasons.

  1. To Simplify – As I said, workflow is complex and is often made even more complicated by systems that try to implement a solution that can be all things to all people. My experience is that the greatest share of workflow requirements, while not exactly simple, aren’t rocket science either.
  2. To Educate – Not only those who are reading this, but myself. Workflow is something that most people understand the concept of. But how to implement it, how to use it, what it technically entails to develop – maybe not so much. This is, after all, the whole reason for this blog series!
  3. To Push Myself – I can happily say that I know a lot about Oracle in general. I’ve been doing it since 1988 and have picked up a fair amount of knowledge on the way. However there are many dark corners of the Oracle toolsets that I have never visited. Part of the reason I’m doing this is completely selfish. I want to learn more about my chosen tools of the trade.

This brings us to the second question in the list above. Are there real and artificial limitations placed on the project?

First lets address the real limitations:

  • While I may, nay will,  take inspiration from external sources, everything I will be doing will be using, as much as possible, pure Oracle technology and will be coded from scratch. This will limit some design and implementation decisions that, in a green-field project devoid of technological baggage, may point toward different ways of doing things. That’s fine. We all know why we’re here. Lets Move on.
  • With a goal of creating something that will run on just about any Oracle platform, I shall not use any Enterprise level features. That means I’ll be developing in the shallow end of the database pool, using Oracle SE or even Oracle XE.
  • This is not an academic exercise to try to see how fancy or clever I can be. In the end, the bloody thing not only needs to work, but needs to be useful. I have long-held the mantra “Implement the simplest thing possible to get the job done” as a design mandate. That, along with the limitation of using only Oracle technologies, will obviously shape the project from beginning to end.

Now what about those artificial limitations:

  • Probably the biggest of these, and the only one worth mentioning, is my time. Not only do I have a day job doing cool things at Oracle, but I also have a life. I won’t be locking myself in a developers dungeon until the code is done. I’ll be doing this as I get time and I’ll likely be learning along the way. That means there will be gaps in productivity while I bone up on something like how to build a proper JET application or how VBCS works. Bare with me – I’ll be sharing my insights (and pain) along the way.

1  Hmm.. YAWE might be a great name for the end product! I shall build a workflow system and it shall be called YAWE!

It’s an Oracle thing…

The last of the three questions is probably the easiest to answer:  What technologies will be used to satisfy the requirements? Why?

Technologies? Oracle
Why? You have to ask?

Now having said that I can take a swipe at what I’ll likely use for each major area:


  • Database Design – Oracle Data Modeler
  • Screen Design – Balsamiq Mockups (one of the few non-Oracle tools I’ll probably use)
  • Code Editor – Oracle SQL Developer / SQLcl
  • Javascript Debugging – Chrome/Firefox (again, non-Oracle, but you know)
  • Code Repository – GitHub (
  • API Documentation –


  • Core Engine – Oracle Database and PL/SQL
  • Workflow Builder Interface – Oracle APEX and ORACLE JET
  • APIs – PL/SQL and Oracle REST Data Services
  • Client Applications – APEX, JET, VBCS, ???
  • Communication standard between Client and Core – JSON

These are obviously open to change and influence by you, the reader. But I think that’s a reasonable estimation of where we’re headed.

In Summary

Workflow is my chosen project. Its been an idea that’s been stuck in my head like a bad 80’s song for years. Now I finally get to set it free.

While you’re not likely to dissuade me from going down this path, I’d love to hear your thoughts on this choice and the topic in general.

Until next time…


Comments 22

  1. Doug, I think it’s the perfect application to build 🙂

    I’m just involved in a project where the customer also asked to integrate a workflow, so I’m happy to be your beta customer, as I know you build great things!

    I look forward reading your adventures.


    1. Post

      Thanks Dimitri! Good to hear the vote of confidence. We’re a long way from beta, but I’d love to continually get your (and other’s) feedback during the design process. Although I’m not trying to “Boil the ocean”, making the end product as flexible as possible is a design goal.

      Keep Watching (and commenting)!

    1. Post

      Hi Niels,

      Yup, I’ve already read it and it will be mentioned in my next post! You did a great job on outlining what was out there at the time. I may want to get with you so you can show me your demonstrations (if you still have them available).

      I’m currently in the process of loading up and looking at PL/FLOW and FCT. I’ll look at Camunda separately!


      1. Yes, I still have the demo available. Let me know if you want to see it in action.

        As for Dietmar: yes, I still would recommend Camunda. Here is why:

        – Camunda is a fork of activiti and is open source, but you can also get support / extra features for it
        – Camunda is actively worked on and has a large install base (see
        – Major patch sets are included in the community edition (which is sufficient)
        – Camunda can be integrated seemlessly in an existing APEX architecture (the Java engine runs also on Tomcat and its metadata can be kept in Oracle)
        – It supports BPMN 2.0 (you can reuse your existing processes when exchanging Camunda for something else)
        – It scales well
        – The graphical representation is run by the open source library
        – There is a solid German company behind it (“Made in Germany”) 🙂
        – You can read/write using RESTfull Web Services, but it also possible to read directly from the tables
        – You can keep the logic from the process flow inside PL/SQL


        – It is not available natively in APEX, so you still need to do some integration work.
        – You need to be familiar with BPMN 2.0 to model your process.

        Hope this helps.

        1. Excellent, this is exactly what I wanted to hear :). Yes, when building a workflow into your application you will always have two sides of the same story in parallel: The state of the workflow but also the state of the underlying object that you wish to modify. These need to be in sync to only allow the proper next steps and update the object state as required.

          Our main focus is to the support the business users in changing the workflow themselves (within reason, perhaps we do the final validation and deploy the new workflow version later). So this still looks like a valid approach.


        2. Post
        3. Hi Niels,
          I remember your presentation at 2017. It’s was great.
          did you tried to integrate camunda dynamic forms within an Apex’s page ?

        4. Hi Niels,

          Is this sample apex workflow application available for download? I want to try this application.

    2. Hi Niels,

      would you still recommend Camunda? At the German Telecom Shops we want to migrate from pl/flow to Camunda … I guess we need to talk some time ;).


    1. Post
    1. Post

      Hi Jason, I would love to chat! I am aware of your project and have even watched the videos. Your product looks much more far reaching than what I’m planning, but I’d love to know more! Ping me and we can set up a screen share. Would be good to catch up anyway.

    1. Post

      Good question. I think I mentioned that my goal is to be independent of database level. But which database version is another question all together. I’d like to say that I can keep it to 11g, but I can’t guarantee that. As I move forward with design there may be thing that would be made significantly easier by moving to 12c then I’ll make that decision at the time.

      But be sure that if there is a dependency on something in a specific version of the database, it will be considered thoughtfully and the reasons and dependencies will be documented.

      Hope that is a satisfying enough answer.

  2. Hello,
    We are using pl/flow in a project, since 2016. We created a java CLI to deploy a bpmn2 process directly into pl/flow schema. It works fine, but pl/flow has a lot of limitation :event handling, signals, and gateways. Also pl/flow lacks business process monitoring tools. I assume Camunda didn’t not have it in community version
    we are seriously thinking of migrating to Camunda, and all ideas are welcome.

  3. Much needed. I have experience with clients who want workflow but once we get into the actual flows they balk and opt for something that doesn’t force them to deal with requirements (or admitting their workflows are mostly pointless). If they could define flows/steps visually and on-the-fly I can imagine this being a big hit

  4. Great idea, Doug. Ive come across the issues youve outlined time and time again. Happy to help with any beta testing

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.