Search for a command to run...
I'm a developer that has spent most of his professional career in the Oracle world with extremely brief detours into JAVA, JavaScript, C#, shell scripting and Python.
Current areas of expertise are Oracle APEX, Data Model Design, PL/SQL, Oracle REST with a little performance tuning thrown in.
No comments yet. Be the first to comment.
Quite some time ago I was asked by a customer how we could track the changes to a row over time. They had a couple of very important tables for which they wanted to keep track of who changed what. The interesting thing was that they were not interest...
There are many times when you're coding an application where there is a requirement for seed data that is required for the system to run correctly. If the seed data is changed or missing, some programming functions will fail due to the missing data. ...
This article was first published on blogs.oracle.com in September of 2019. Not long after it was published I was made aware of a couple of security flaws. However by that time the underlying blogging platform Oracle was using had been migrated and I ...
In honor of #JoelKallmanDay, I thought I’d break back into blogging after a long hiatus with an update on some of my more popular posts of the past – Oracle APEX and Workflow. In January of 2018, I kicked off a little personal research project. The g...
After my initial post about this, I had a discussion with John Snyders of HardLikeSoftware fame, and it turns out there is a better and more efficient way to maintain the tree state. While what I outlined definitely works, what follows is the “prefer...
Remember when I said that, for my All The Things project, “My goal is to go through the steps I would normally traverse when building an application for a paying client”? Well one of those steps would surely be to understand what was already available on the market and see if any of them fit the bill for the client’s needs.
I’m going to try to do the same thing here, but since there isn’t a real client (and therefore no real set of needs) I won’t spend the time and go to the depth i might. After all the purpose of this whole exercise is to create a system from scratch, right?
That being said I wanted to have a look at some of the more popular options out there. This can help us gain an understanding of what people are requiring and what companies are building in to their products.
First let me apologize for the marathon post. It turned out way longer than I thought it was going to. You may want to grab a drink (and a snack). It’s a long trail you’re about to hike.
One of the very first things I ran into when searching for available workflow solutions is that no one really talks about “Workflow” any more. Although a straight google search for “Workflow Engines” does bring back a lot of hits, you quickly begin to notice that most of the sites actually talk much more about Business Processes and Business Process Management way more than they talk about Workflow directly.
This sent me down a rabbit hole which consumed the better part of a Saturday afternoon – What’s the difference between Business Process Management and Workflow?
My first thought was that Business Process Management was just a fancy new updated way of saying Workflow. But that turned out to be too much of a simplification.
How you think about Workflow vs BPM seems to hinge quite heavily on your perspective, but I’ll do my best to boil down what I learned to a single paragraph :
“Workflow focuses more specifically on the automated execution of a set of defined steps (or tasks) and knowing where you are in those steps. Business Process Management focuses more on the definition of the overarching processes (which may contain one to many workflows), their overall efficiency and how to improve said processes to the benefit of the company.”
So where we used to talk about Workflow as the solution to the problem of process automation, now workflow is talked about as a mere component of a BPM solution, this time to the more all-encompassing problem of business process efficiency.
I don’t want to beleaguer the point, but I will obviously need to return to this discussion in relation to the software that I’m going to try to build. For now, lets just understand that when researching existing solutions, I had to keep in mind that there is more at play than just simple process automation.
When you dive in to the deep end of the Google search pool you could spend weeks if not months exploring all the options that come up when searching for either Workflow or Business Process Management. Because of this, I’ve limited my discussion to those that I have had either direct experience with, or that have been recommended to me by those I trust.
My goal is not to compare and contrast each option, nor choose a “winner”. I’ve obviously not used most of these in any type of production environment. My goal is to simply explore some of the more popular options and potentially use them for reference when moving forward with this project.
URL: http://plflow.sourceforge.net/
Availability: Open Source (LGPL)
Status: Inactive (No Activity since 06-2006)
According to the PL/FLOW SourceForge repository, “PL/FLOW is a workflow engine written in Oracle PL/SQL, implementing interfaces 1 and 2 (Process Definition and the client API) as specified by the Workflow Management Coalition. (www.wfmc.org)”
This is one of the few options I’ve spent any significant time with and have actually implemented for a customer. I won’t lie, it takes a while to understand what PL/FLOW is trying to do, how it can be used and where the boundaries of the system are. This is partially because the documentation is lacking beyond the very basics and isn’t helped by the object and column names being relatively cryptic and hard to remember.
With a little study and a thorough reading of the source code, it starts to become clear that this is exactly what it purports to be – A simple workflow execution engine.
On the positive side, the implementation is open source, it is written in PL/SQL, the code is heavily commented and provides a set of PL/SQL API’s that can be used to integrate the engine into your application.
However, there are definitely challenges with the implementation.
First and foremost, the code depends on a set of SYS owned Advanced Replication packages being available. If you’re using any database before 12.2, you should be fine, but Advanced Replication has been de-supported in version(s) 12.2 and beyond. Getting around this isn’t a huge challenge. The advanced replication packages are used in the triggers on each table to identify whether a change was generated by replication or by actual manipulation of the data by the engine. You can rip these out of the triggers and everything should work without issue.
Second, the definition of the workflows is not based on current standards. The WfMC Standards that PL/FLOW is based on are ancient in IT Terms, and the world has moved on. BPMN is the current standard and PL/FLOW natively knows nothing about it.
And when it comes to designing workflows, the documentation does walk you through the metadata of a couple of examples, but they are simple and don’t necessarily show the full functionality and feature set of the engine. For that you have to experiment.
Next, although the metadata breaks out the definition of a workflow from a running instance, the table structure doesn’t allow for multiple versions of a workflow to exists at once. And with the way PL/FLOW implements instantiation of a workflow (by only copying data about the current step to the instance tables as it is reached in the process), if you change the definition of a workflow on the fly, you might get unexpected results.
There’s also no way to export the definition of a workflow. Sure, all the metadata is there in the database tables, but moving a workflow definition from development to QA or PROD becomes a process of building your own scripts.
Finally, there is no visual way to design workflows. The website does mention (and link to) a Java based workflow editor, but those links have not worked for a very long time. While this isn’t a show-stopper, people will likely want a way to visualize their workflow designs and the state of in-process workflows. You’ll have to do this on your own here.
All this, coupled with the fact that there is no support and that there have been no updates or releases of the software since June of 2005, means that you are truly on your own implementing PL/FLOW.
I truly believe that much of this accounted for the one client I implemented this for eventually ripping PL/FLOW out by the roots, even though it was working for them.
Having said all that, I had a really interesting web conference with a gentleman by the name of Robert Meyer from Leipzig, Germany. He showed me a system that he is in the process of creating using bpmn.io as the designer and modified version of PL/FLOW as the execution engine. I have to say I was quite impressed with what he had done. Whether Robert intends to release his work to the world at large remains to be seen, but I definitely encouraged him to keep moving forward!
URL: https://github.com/j-sieben/FCT/
Availability: Open Source (MIT)
Status: Inactive (No Activity since 06-2016)
FTC takes a different approach to implementing workflow. Jürgen Sieben, the author, has implemented a Finite State Machine using Oracle database types and PL/SQL and has done it in a very specifically object-oriented way.
I was excited to try this specific engine out as it comes with an example that includes an APEX app.
The entire system hinges around an Oracle Object Type definition called FCT_TYPE. The object type defines an abstract object definition including a number of getter methods as well of a number of helper methods.
The act of defining a workflow consists of a number of steps.
Confused yet? I definitely am. And the fact that I could not get the code to install cleanly didn’t help. The installation scripts are wrought with issues, including what seems to be a requirement for one of Jürgen’s other projects (PIT). But even after installing PIT, getting it working and executing the proper grants, I found that the signatures of the PIT package calls don’t match those defined in the PIT repository. It was at this point that I stopped.
Based on the docs in github, I think this is a really interesting idea. However I do think it looks very code heavy and may put a lot of people off.
Again it’s not standards based and does not have any type of graphical interface to build or track workflows. The fact that it is open source and doesn’t have a supported option will put many companies off.
It doesn’t look like anything has been done to the core code since its initial release in June of 2016. There are a couple outstanding bugs logged against it, specifically to do with the failed installation that don’t look like they’ve been acknowledged or actioned.
The only conclusion I can come to is that the FCT project is dormant and the author has moved on to other priorities.
URL: https://www.patreon.com/TribalKnowledge
Availability: TBD (Likely Commercial)
Status: ACTIVE
Jason Aughenbaugh, a long time personal friend and friend of the APEX community, is in the process of developing a workflow engine based on his years of experience and knowledge in working with process automation.
Although he is still in the throws of core development and there is nothing available for download at the moment, he was good enough to walk me through what he has done thus far. He has obviously put a lot of work into the system and it shows.
Process modeling is done graphically and uses IBM’s flow notation. The system allows you to drill down into process steps and decision points to further define variables, conditions, forms, responsible parties, etc.
Looking at the stated goals for Relay and the broad list of functionality on the website, it’s definitely one to watch in the future.
URL: https://www.activiti.org/
Availability: Open Source (Apache 2.0) / Commercial support provided by Alfresco
Status: ACTIVE
Activiti is a light-weight, java-based Business Process Management platform that has been around since 2010. The documentation states that it is aimed at Business Users, Administrators and Developers. It supports both BPMN 2.0 and DMN standards and has open REST APIs allowing integration with external systems that may not inhabit the same server space.
The implementations can be as simple as a working JRE and Apache Tomcat, or as complex as bundling Activiti JAR files in your own Java application.
Data for Activiti is stored in one of a number of supported relational databases, the default being h2 due to its simplicity and portability. Oracle, mySQL, Postgres, DB2, and MSSQL are also supported databases.
For testing purposes I chose to go with the JRE and Tomcat option, and I have to say it was as easy as copying the WA files to the webapps directory and navigating to the correct URL. I was immediately able to log in to the demo environment, had access to create BPMN models, Reusable Forms, Decision Tables and Application Definitions.
The REST APIs are what you would expect and allow access and manipulation of process definitions, models, process Instances, executions, tasks, history, forms, Users, etc. It even provides REST APIs to retrieve metadata information about data from the underlying database tables.
Although REST APIs are provided and well documented, the bulk of the documentation assumes that you’ll be implementing Activiti as part of a Java project. The examples and the quick start guide are all Java heavy, so if you’re not familiar with Java, following the concepts of their examples will be challenging.
There’s no PL/SQL level API so you’d have to do everything via REST, but that shouldn’t be a problem as it’s becoming easier to do that.
The fact that Alfresco provides an enterprise level supported version makes it a good choice for companies that want to implement a supported solution, and the fact that it easily supports Oracle as a database means that you could tap in to the underlying tables for reporting purposes.
URL: http://www.flowable.org
Availability: Open Source (Apache 2.0) / Commercial support Available from EdorasWare
Status: ACTIVE
In October 2016, 3 key engineers from Activiti tendered their resignation and very shortly thereafter forked the main Activiti repository and released the first version of Flowable. The goal was to continue development in the direction they saw the project going without the restraints put on them by the commercial needs of Alfresco.
It looks as if their business model is not to create an enterprise version of the software, but instead to provide full support, training and consulting on the open source Flowable core.
Looking at the docs, screenshots and examples, they so far have not strayed to far from the origin when it comes to features and functionality. The biggest change that I can see is the addition fo CMMN to the original BPMN and DNN.
Other than that, it looks much the same as Activiti thus far.
URL: https://camunda.org
Availability: Open Source (Apache 2.0) / Commercially supported Enterprise Edition available
Status: ACTIVE
Camunda came about in 2013 with a code fork from the Activiti code base. Since then the Camunda team seems to have done a lot to differentiate it from its Activiti base.
Looking through the documentation, you’ll see quite a bit that is similar to both Activiti and Flowable, which is understandable seeing that they all branch from the same root code. However the user interface looks to be a complete departure from its Activiti ancestor. And, like Flowable, Camunda have added CMMN to the original BPMN and DNN.
Camunda provides REST API’s for pretty much everything you’d want to do. Like Activiti, the documentation and examples assume that you’ll be implementing everything using Java, so be prepared to mentally parse some Java code to get to the meat.
The software provides support for various relational databases (including Oracle) as data stores, and this makes it nice if you want to read data directly from the underlying tables. However the table structure is confusing and there isn’t a lot of good documentation that I found on what is stored where.
On the plus side, it has already been proven by Niels de Bruijn that Camunda can be integrated into an Oracle/APEX environment. A number of months ago he did a presentation on workflow integration with APEX. I had a very enlightening Skype call with him where he shared with me how he integrated the two products. He had obviously done a lot of leg work in understanding the underlying data structures, APIs and required touch points. Niels even created an APEX application that read data from the Camunda base tables to show users information about the status of in process workflows.
I’m sure that could be done for any of the applications mentioned here that allow for installation into an Oracle database, but it was quite impressive to see what he had achieved.
URL: https://bonitasoft.com
Availability: Open Source (LGPL 2.0) / Commercially supported Subscription Edition available
Status: ACTIVE
BonitaSoft comes from a different stable entirely and, as you would expect, looks and feels completely different than the other options listed above.
Bonita Studio provides both desktop and web portal based environments that together provide the ability to build models, define execution specifics for steps, manage predefined integration with external systems, build forms, and even create full “applications” with linked pages, forms and interactions with process models.
As you would expect, it provides a full REST based API suite and again, pretty much everything you want to do is available.
Modeling uses BPMN based models, as seems to be the standard, however the feel of the modeler is completely different and uses a lot of color. Not that that’s a bad thing – just different. One of the nice things in the Bonita modeler that I have not seen in others is the tree view that shows the process components and allows you to drill in on them.
The docs are fairly clear but again, they are Java heavy including a whole section of javaDoc on the classes that are available for implementation.
Overall the product seems a bit more cohesive and usable but it is obviously complex and, like most of the systems mentioned here, would take a long time to learn how to get the best out of it.
URL: https://www.processmaker.com
Availability: Dual License (AGPL V3 / Various Commercial Levels )
Status: ACTIVE
Like BonitaSoft, ProcessMaker is different from those that sprung from the roots of Activiti. However, instead of being a cross between desktop and web portal, ProcessMaker is all web-based.
Like all of the major systems discussed here, the designer uses BPMN notation to design the flows and has a sophisticated interface to define and link together variables, forms, users, groups, process steps, group assignments, etc. I have to say, the interface in ProcessMaker was really easy to use and well laid out. It didn’t overcomplicate things and only showed you what you needed to see when you needed to see it.
The process modeler was more akin to Camunda in it’s look and feel, but with the pallet of controls across the top instead of along the side.
Again a full set of REST based APIs let you control every aspect from outside of the system, including APIs for Administration, Design and Execution.
The docs are the best I’ve seen as they are well structured and have lots of examples and walk throughs. There are also lots of videos that demonstrate the various features.
Although Process maker has the capability to integrate with external Databases, the metadata defined within process maker is stored in a local mySQL database. Integration comes in the form of being able to define and provide tables in remote databases based on the data in its core mySQL database.
Like BonitaSoft, there are a number of built in integrations with external software vendors, making triggering and integrating workflows easier.
Overall ProcessMaker seemed to be the friendliest to get started with.
So after having worked my way through the octet of options listed above, what did I learn?
No two people (or companies) seem to think of workflow in the same way. Even the three packages that share the same code base are all moving in different directions. Although I didn’t specifically mention this in my synopsis of each above, many of the workflow engines tout solutions targeted at specific industries. Whether this comes in the form of pre-built process models, or extensions/plugins to the core, each obviously understand that every industry is different and has different needs.
Every major option that I looked at incorporated BPMN as their modeling standard. This is no coincidence – BPMN and Business Process Management is big business and each of these vendors want a piece of that pie. And not only is BPMN a must, but many have added CMMN and DNN to the mix – All the better to serve you with, my dear.
Here is one that I disagree with. Most of the products I looked at were not only implementing the ability to model, execute and track a process, but they were also trying to provide a full platform on which to build Process Automation Applications. The ability to design linked forms to collect and store data about a process step, and even the ability to create “applications” that encompass full or even multiple linked processes is provided by many of the systems.
Ok I understand that a business user starting from scratch might find this alluring. They don’t have to get IT involved to implement their business process. But I can’t help but think that, in the long run, this is just another place to build and maintain business rules outside of the core development environment that already exist in a company. And how many times have we seen systems “developed by business users” either fail because they are left to languish after their initial implementation, or turn into data islands that are not integrated into the business architecture as a whole. And doesn’t that last one defeat the whole purpose of Business Process Management?
The world of BPM seems to be heavily Java-Centric. From documentation to examples, Java seemed to be the only thing on most companies minds. Even the examples and documentation around the REST APIs were heavily bent toward Java developers.
But then again, you code to what you know. Most of the engines listed above are written in Java, so why wouldn’t they use their native language to provide examples.
Apart from the obvious Oracle-centric options, pretty much every platform used either H2 or mySQL as their default database, and it’s obvious why. They’re cheap and portable, don’t need a lot of DBA love, and can stand up to a reasonable amount of thrashing before they start to creak under the strain.
And of the ones that chose H2 or mySQL as their base, all but one offered the option to configure the platform to use a more robust database of your choosing. And the usual suspects were always there – MS-SQL, Oracle, PostgeSQL, DB2.
But the very fact that each platform is fairly database agnostic means that they are simply using the chosen database as a datastore and not using RDBMS system for what they’re good at; manipulating and maintaining data. Instead, they’re pushing all of the data manipulation away from the data into the middle (Java) tier. Having seen a number of systems (not just Java) that have been coded to be database agnostic, that does not fill me with joy and sunshine.
Examining all of these platforms has really been an interesting process. There are obviously huge similarities between the platforms. The very fact that most use BPM means that the language they speak when it comes to talking about processes, models, instantiation and execution are very similar. This is a good thing and made the task of looking at multiple platforms far easier.
But there were huge differences too – In UI, deployment options, documentation, examples, REST interfaces, etc. And you would expect this from software companies who are trying to differentiate their products in a relatively crowded marketplace.
As I said earlier, my purpose for this wasn’t to come up with a winner, although some stars shone brighter than others. My goal here was to understand what companies were doing, what was common, and what I could and should take forward into my project.
Below is a list of things that I want to document for my own purposes as the feelings I came away with after completing this process.
This has been a long process, as the long post shows. But I think it has provided a huge amount of information for me. Like I said, workflow is hard – it is not a light and breezy skip through a daisy-laden field. More like a crawl through a culvert of questionable content.
But I do invite you to continue the journey with me. I look forward to the next installment, but before then I have a lot of reading to do.