One year ago I penned Event Processing in Twitter Space, and today parts of the net are buzzing about Twitter.
In a nutshell, Twitter is a one-to-many communications service that uses short messages (140 chars or less). Following on the heels of the blogging phenomena, Twitter has been primarily used for microblogging and group communications.
Twitter, and Twitter-like technologies, has great promise in many areas. For example, you could be subscribed to the @tsunamiwarning channel on your dream island vacation and get instant updates on potential disasters. A team of people working in network management could subscribe to the @myserverstatus channel and receive updates on their health of their company IT services. Passengers could subscribe to the @ourgatestatus channel and follow up-to-date information on their fight.
Twitter was created to answer the simple question, “What are you doing now?”
Bruce makes an interesting comment on business rules too: that “routing logic in process gateways” are not “business rules”. That doesn’t really make sense: for sure some gateways will be process-housekeeping decisions of little interest to the business user, but others will surely embed business-critical decisions. On the other hand, it has long been acknowledged that a best practice for BPM is to delegate such business decisions to a managed decision service - hence the explicit new business rule (aka decision) task in BPMN 2.0. And,in the CEP world, for tools like TIBCO BusinessEvents to invoke a decision managed by its Decision Manager tool.
The main characteristic to be aware of in these tools is that BE is primarily rule-based (using an embedded rule engine), whereas BW and iProcess are orchestration / flow engines. In BE we can use a state diagram to indicate a sequence of states which may define what process / rules apply, but this is really just another way of specifying a particular type of rules (i.e. state transition rules).
The main advantages to specifying behavior as declarative rules are:
Handling complex, event-driven behavior and choreography
Iterative development, rule-by-rule
The main advantages of flow diagrams and BPMN-type models are:
Ease of understanding (especially for simpler process routes)
Process paths are pre-determined and therefore deemed guaranteeable.
In combination these tools provide many of the IT capabilities required in an organization. For example, a business automation task uses BW to consolidate information from multiple existing sources, with human business processes for tasks such as process exceptions managed by iProcess. BE is used to consolidate (complex) events from systems to provide business information, or feed into or drive both BW and iProcess, and also monitors end-to-end system and case performance.
On Event Processing Agents implies a “new” event processing reference architecture with terms like,
(1) simple event processing agents for filtering and routing,
(2) mediated event processing agents for event enrichment, transformation, validation,
(3) complex event processing agents for pattern detection, and
(4) intelligent event processing agents for prediction, decisions.
Frankly, while I generally agree with the concepts, I think the terms in On Event Processing Agents tend to add to the confusion because these concepts in On Event Processing Agents are following, almost exactly, the same reference architecture (and terms) for MSDF, illustrated again below to aid the reader.
The success of Service-Oriented Architecture (SOA) has created the foundation for information
and service sharing across application and organizational boundaries. Through the use of SOA,
organizations are demanding solutions that provide vast scalability, increased reusability of
business services, and greater efficiency of computing resources. More importantly,
organizations need agile architectures that can adapt to rapidly changing business requirements
without the long development cycles that are typically associated with these efforts. Event-Driven
Architecture (EDA) has emerged to provide more sophisticated capabilities that address these
dynamic environments. EDA enables business agility by empowering software engineers with
complex processing techniques to develop substantial functionality in days or weeks rather than
months or years. As a result, EDA is positioned to enhance the business value of SOA.
The purpose of this white paper is to describe the approach employed to overcome the significant
technical challenges required to design a dynamic grid computing architecture for a US
government program. The program required optimization of the overall business process while
maximizing scalability to support dramatic increases in throughput. To realize this goal, an
architecture was developed to support the dynamic placement and removal of business services
across the enterprise.
JT has posted his view on rules and decisions and how they relate. Given that James talks more about services than events, I thought it would be worth reviewing his post from both a Complex Event Processing and a TIBCO BusinessEvents event processing platform perspective.
”Decision Services:
Support business processes by making the business decisions that allow a process to continue.
Support event processing systems by adding business decisions to event correlation decisions (they are often called Decision Agents in this context).
Allow crucial and high-maintenance parts of legacy enterprise applications to be externalized for reuse and agility.
Can be plugged into a variety of systems using Enterprise Service Bus approaches.”
Folks who have been in event processing fields like network management (NMS) or security management for many years have a very high expectation for processing complex events. Most of the network and security management platforms on the market have basic rule-based processing available “out of the box” and most of these platforms have had the capability to process events in near-real time for decades. Adding a new “rules-based event processing platform” to the network and security management software mix does little to add any additional capability and certainly does not solve any nagging complex detection problems.
- leave anything related to transport, communication to other layers- use this revised CEP to express and execute event-relevant logic, the purpose of which is to translate the ambient events into relevant business events- have these business events trigger business processes (however lightweight you want to make them)- have these business processes invoke decision services implemented through decision management to decide what they should be doing at every step- have the business processes invoke action services to execute the actions decided by the decision services- all the while generating business events or ambient events- etc.
A common task in many event processing systems is to detect patterns of events.
If combined, these patterns will eventually form a situation consisting of multiple patterns over time.
So basically a detected instance of a situation is a specific sequence of events.
CEP module receives or intercepts a flurry of events and processes them with the objective of figuring out what those events are relevant for; it triggers the appropriate business processes or decision services
BPM module receives the request for a given process to be applied to a higher level entity (an application, a document...); it automates the steps defined in the business process
BRMS module is invoked with a given context to apply business rules; it makes a business decision
Function answers the question --- what is being done?Technique answers the question -- how something being done?
Application answers the question --- what is the problem being solved?
ExamplesBusiness Activity Monitoring (BAM) is an application type, it solves the problem of controlling the business activities in order to optimize the business, deal with exceptions etc...Business Rules are type of technique --- which can be used to infer facts from other facts or rules (inference rules) , or to determine action when event occurs and condition is satisfied (ECA rules) and more (there are at least half a dozen types of rules, which are techniques to do something).Event Processing is really a set of functions which does what the name indicates -- process events --- processing can be filtering, transforming, enriching, routing, detect patterns, deriving and some more.
EDA is more a manifestation of finite state machines going all the way back to Alan Turing. Old_State + Event = Some_Action + New_State. It is the simplest, yet most powerful way to design robust systems. I only wish more people would give it due consideration.
A very old implementation example is I/O interrupts (hardware events) for asynchronous I/O - real time event handling which enabled multitasking operating systems.
Many want to use web services for everything now and at times it is hard to convince people that other messaging schemes and standards are a better fit for some problems.
Rule-processing is just a style of computation. Of course it is used in BRMS, but it is also used in CEP. CEP systems typically employ rules-based processing to infer higher-order events by matching patterns across many event streams within the event ‘cloud’. BRMS’s use rule processing to match patterns within data tuples representing business-orientated data. CEP systems may support the use of advanced analytics to manage predictive analysis, reasoning under uncertainty and other requirements in relation to the event cloud. Some of the better BRMS’s offer similar analytics in regard to processing business data.
The Center for Multisource Information Fusion (CMIF) is a research center based at the University at Buffalo and at a non-profit Western New York research center called CUBRC Inc.. Information fusion allows users to assess complex situations more accurately by combining effectively the core evidence in the massive, diverse and sometimes conflicting data received from multiple sources. CUBRC/UB's partners in the center are the Rochester Institute of Technology, which has expertise in image analysis and visualization, and Pennsylvania State University, which also has a long history in information fusion research focused on the human and cognitive aspects.
The “big elephant in the room” in the ongoing CEP dialog is that most of the current (CEP) software on the market is not capable of machine learning and statistical analysis of dynamic real-time situations. Software vendors have been promoting and selling business process automation solutions and calling this approach “CEP” when, in fact, nothing is new. There is certainly no “technology leap” in these systems, as sold today.
J. Llinas, C. Bowman, G. Rogova, A. Steinberg, and F. White. In P. Svensson and J. Schubert (Eds.), Proceedings of the Seventh International Conference on Information Fusion (FUSION 2004, page 1218--1230. (2004)