Business Enabled Integration...

A few years ago I worked at an incubator called 12 Entrepreneuring. It was during the heady but waning days of the internet boom. 12 was in the midst of incubating Grand Central Networks, an integration network company. I was asked to help temporarily with the conceptual design and architecture on the project, which was based in the San Francisco office, until the New York office (my eventual homebase) was up and running.

My background was in systems integration at the enterprise level. I felt that the value of the network would be to provide a rich set of pre-integrated business partners and industry aware components. My point was to align with the business and not be purely technology focused. Founder Halsey Minor pushed to keep the service simple. His catch phrase was "a better FTP". Alas, Grand Central Networks has gone the way of many a startup.

Alignment with business is where the industry is finally heading... In an effort to differentiate themselves in the inevitable arms race like spiral of the integration market, webMethods has sought and received SWIFT Certification. Banks looking to upgrade their SWIFT integration technology now have a reason to seriously consider webMethods. iWay Software, as the adapter company has been at the forefront of supporting industry protocols such as SWIFT, FIX, HIPAA, etc. and is pushing into insurance with ACORD. Yes, vendors will continue to add bells and whistles to their products but I expect them to become more tightly aligned with various industries and their standards. A tool that is already aware of your industry's standard processes and vocabulary is incredibly valuable.

Security Onus Is on Developers

I actually read this eWeek article on my way into work in it's print form. (A one hour train ride into NY City every day can do wonders for your reading backlog...) I liked some of the points so much, that I tracked down the online version so I could blog it here.

Essentially, poor code quality can have as much of an impact on security as the hackers themselves.
"In 2004, Internet Explorer had a publicly revealed vulnerability that had not been patched on 98 percent of the days [of that year]. Firefox was vulnerable on 7 percent of the days [ of that year]. That tells you that what the application developers are doing can make a big difference."

David Wagner
I'm a strong proponent of code inspections, especially automation of the more mundane but insane little detail checking. But it has always amazed me how hard it is to get managements approval to purchase inspection tools. It usually comes quickly though, once one of those small mistakes has caused a big problem. Bill Pugh points out the other side of the coin... that everyone makes mistakes, even the smartest of developers.
"A lot of people think that errors and defects and stupid mistakes are things that the "lesser programmers" make. One of the things that I've found is that tools find insanely embarrassing bugs, written in production code, by some of the very best programmers I know."

Bill Pugh

2006 JavaOne Conference - It's a Wrap

I'm contemplating the week @ JavaOne while I wait for my flight at the airport.

AJAX/REST
AJAX was obviously a popular topic at the conference. There is a clear consensus that AJAX is cool, gives you the appearance of rich client capabilities, but is still hard to do well. A number of speakers pointed to bugs in various Google products. The point being, is that the Google brain trust does AJAX very well and even they run into problems. JavaScript features are uneven across browsers, so this has lead to a wealth of vendor server side "Frameworks". Frameworks that try and solve this problem. There seems to be dissagreement as to weather or not this is the way to go... See the JackBe/Sun discussion.

Representation State Transfer or REST popped up in a number of talks... Simplicity and leverage of the standard Web infrastructure are it's primary benefits, but a big drawback is that it is not discoverable... although there is work being done in this area.

XML

XML processing continues to receive improved support..... Mark Reinhold gave a brain dump of his thoughts on how XML can be made "native" to Java in Dolphin.

Back To Basics....

JEE - The Java Enterprise Edition version 5 has made good use of Tigers annotation capabilities and lessons learned from the Spring Framework and simplified the EJB specification considerable. Some would still say it wasn't enough and that Spring's POJO model is still the way to go.

SOA - Although vendors continue to introduce products that support all the latest standards and clain "Buzzword" compliance, there is an undercurrent of chatter from attendees that "things" are overly complex... The S in SOAP stands for Simple, but anyone who has read the WS-* specifications can attest to the fact that the Web Services space seems to be spiraling out of control. There were a number of well attended talks in the REST space and mention of WOA as a simpler more scalable alternative to SOA.

Participation

Finally, there was a continuous call for participation of individuals in the JCP. Doug Lea, spec lead for JSR 166, was called out as an excellent example of how individuals can have large impact on the language and platform.

DayThree @ JavaOne

I finally listened to the Oracle Keynote from yesterday... Oracle is excited about the Java persistence API specification. Surprise, surprise! The demoed the JPA from within JDeveloper. Point and squish as the saying goes. Thomas Kurian went on to discuss Oracle's concept of a Service Fabric. Basically a Service Neutral, transport and protocol independent infrastructure on top of the Spring Framework. Hmm, sounds like something that Grahm Glass, founder of Minda Electric was working on before his company was aquired by WebMethods.

So here is today's annotated schedule.
09:45AM It's Not Over Till the Fat Client Sings - An interesting take on why AJAX is a poor mans substitute for a good Swing Application. I don't entirely agree, but AJAX has it's issues and Sun has been polishing Swing for some time now. There is something to be said for working in a less than ideal application (Browser) that has 100% desktop penetration versus an environment that is significantly lower (Java @ 60% if you believe Sun)
11:00AM Java Technology, AJAX, Web 2.0 and SOA - a buzzword compliant industry panel.
12:15PM New Compiler Optimizations in the Java HotSpot Virtual Machine - I always like to stay on top of the new plumbing, especially in terms of performance. This talk is always worth it.
01:30PM Desktop Patterns and Data Binding - not what I expected... Just an esoteric discussion on Swing and pure implementation strategies of the MVC pattern.
02:45PM Extreme GUI Makeover: Lookin' Better - This talk is always excellent. The presenters take a how-hum user interface and do an extreme make over using the latest UI techniques... Always cool. The slides don't do it justice. You need to see a demo of the application, before then after.
04:00PM Practical SOA Business Integration using OpenESB - A "How To" session on the use of OpenESB. Nothing special here.
07:30PM Discovery and Dependency Injection Patterns in Modular Architectures - Discusses the NetBeans Lookup Library. The presenters kept calling it dependancy injection, but I thought it sounded more like a modified Locator Pattern. Martin fowler has a nice write-up about Dependency Injection and the Locator Pattern.
08:30PM Memory Leaks in Java Technology-Based Applications: Different Tools for Different Types of Leaks - Excellent talk on this subject by Gregg Sporar. He has promised to post a writeup of this talk on his web log in the comming weeks.
09:30PM JSRs 236 and 237: Concurrency Utilities for the JavaÃ'™ EE Platform in Practice - This is basically the adaption of JSR 166 Concurrency Utilities to Java Enterprise Edition. I had particular need for this capability while at a Power Utility client awhile back so I've felt the pain. I'm happy that they're working on this.
10:30PM Database Refactoring: Enabling Evolutionary Database Development - I ran out of gas before attending this one. I'll have to pickup the book;)

DayTwo @ JavaOne

Jet lag is a cruel thing... I missed the Oracle Keynote this morning. I Will have to catch the web cast of that one first thing tomorrow morning. JavaOne is relentless. This is a busy day, like yesterday, but without the jet lag. The last session I plan to attend is the "XMLBeans 2.1 Java Technology Developers Perspective" which ends at 11:20 PM.

Sooo... here is the annotated schedule:

DayOne @ JavaOne

No sign of Scott McNeally today, Sad... I loved his bravado and humor. I attended the first JavaOne conference in 1996. It was like attending Woodstock. There was an incredible amount of excitement and anticipation for the platform and Scott set the tone for the week. Jonathon Schwartz is a sharp individual, but doesn't grip the room like Scott can. I know this sounds like superficial rumination, but can't help voicing my first impression of the keynote...

Anyway... on to some "real" information... This is the largest JavaOne ever, exact numbers have yet to be announced. The conference definitely feels much more crowded than last year, but it's hard to tell if there are significantly more people or that it is just due to the new requirement to register for a seat for sessions. Waiting to get in seemed silly. Event managers held up attendees while the previous session cleared. This led to long lines as people waved their RFID badges on the way in and get the green light to enter.

There is a definite push to get individuals involved in the JCP. This was repeated several times in several sessions. Maybe Sun hopes it'll feel more like open source if more individuals are involved in the JCP.

Motorola CEO Ed Zander arrived to push his cell phones and announce that Java is "the platform" for motorola's phones going forward. The industry will ship 1 billion Java devices this year with motorola shipping 2oo million of them. Ed also announced motorola's participation in the NetBeans project. In fact there were a number of announcements around new membership in the NetBeans develop community. Seem's that Sun has put a big push behind getting NetBeans viewed as a viable platform against Eclipse.

Changes in licensing were announced to allow Linux vendors to include Java as part of there distribution. In fact, shwartz stated that 60% of the PC's sold today include Java out of the box and Sun is pushing hard, working with PC vendors to include Java on their platforms.

One of Schwartz's first acts as CEO was to bring back Rich Green as Excutive Vice President of Software. Schwartz brought Green on stage and immediately asked the question.... "Are you going to open source Java?". Basically, after some banter, Green said It's not a question of When Java will be open sourced, it's a question of how."

Business Rules: A Report from the Field

I'll preface this entry by stating that I'm not a "Business Rules" expert by any stretch. I write this as one who has experienced an implementation from the trenches and who is about to dive into the deep once again at my latest account.

I arrived on a project that needed an injection of technical leadership. The project was basically a complex settlement system for a large energy market. There were hundreds of steps required to create a bill for a customer with numerous regulatory considerations. The complexity of each step could range from a simple atomic calculation to an extremely complex set of "business rules" with 10 to 15 pages of single spaced specifications. In general it was determined by the client that a business rules engine should be used to construct these steps. The goal would be to allow the business users to maintain the rules for the new system. I a'm not going to explain business rules basics, but going to assume youÂ've read the basic marketing available. This article contains the implementation experience that I pass on to you.

Business People Should Write Business Rules?

The ability for a business person to directly control the business rules, this is the Holy Grail when implementing business rules engines. The thinking here is that a visual tool can allow business users to simply "draw" a business flow. Additionally, by bypassing the Business Analyst, developer and associated typical software development life cycle (SDLC) that rules can be constructed more quickly and managed more easily. It is true that Business Rule Management Systems (BRMS) can accelerate and streamline the implementation of business rules, but buyers should understand some of the realities of adopting the technology and adjust their expectations accordingly.

It has been my experience that even the most advanced rules engine with the slickest interface requires training and, in some respects, structured thinking about how to organize and construct the rules. There are two factors at work here. First, visual rule environments provided by Business Rules Engines are not "Visio". There are structural constraints enforced during the definition of a valid visual rule flow. How long before a business user becomes frustrated because they just can't draw the "picture" they can clearly see in their minds eye.

The second issue with business users and direct manipulation of rules is rule decomposition. Rules engines that implement a Rete algorithm allow large numbers of simple rule statements to be automatically organized into a network of nodes that allow efficient execution of highly interdependent rules. Business users typically do not think in terms of rules, they think in terms of business policies.

Merriam-Webster defines policy as
"a high-level overall plan embracing the general goals and acceptable procedures"


So in essence policies are much broader than individual rules. Someone must decompose the various business policies into discrete rules and structure them appropriately in order to represent these policies in a form that can be executed by the engine. Although rules can be structured such that they can be easily read and understood by business users, rules engines are not "Natural Language" processors. They require strict syntax in order to operate. An example would be the Yasu QuickRules engine. The user interface allows rules to be entered either visually or as text that adhere to a VB like syntax. Rules that do not conform to the syntax can not be saved and therefore never executed by the engine. The product gives relatively vague error messages, so good knowledge of the syntax and operation of the engine are typically required.

So if the business person is not the most appropriate person to implement business rules, who is? We could naturally gravitate towards the programmer as an option, but my experience is that a typical programmer does not easily become an effective business rules designer. There is the natural tendency for a programmer to try and use the engine's rule language like another general purpose programming language. This usually ends in frustration because general purpose "programming" is not the goal of a rules engine and therefore can seem a poor substitute for languages targeted for the general purpose space like Java or C#.

Business rules in general should be recognized as a separate disciplinep. Special roles such as rules analyst, designer and administrator should be created. Talent for these roles can be sourced from disciplines like business analysis, software development and operations, but there must be the appropriate investment in training, mentoring and time to allow existing employees to grow into these roles. Think back to the investment required to move functional programmers to a more object oriented methodology. It took significant investment and effort for a successful transition. You can expect your results to match your investment.

Business Rules Require Facts (Data)

Business rules operate on facts.Facts are essentially the data provided to the business rules for execution and also derived during their operation. Facts are compiled, or loaded, by the calling application and passed into the rules engine for evaluation. The Fact Model is the entire set of facts that has been exposed for use to the rules engine. This model is typically constructed incrementally over time as facts are added to support new or modified business rules. The complete fact model can be developed in advance of implementing any rules, but in practice this can be very time consuming.

There are two issues that may arise in the management and transfer of facts (data) to the business rules engine. The first is version control. New facts required because of rule changes result in associated changes to the calling application. Additionally this requires that the modified application be deployed along with the new business rules. Something that we would hope to minimize as part of a business rules strategy. So in this case, the SDLC overhead can not be avoided and synchronized deployment of business rules and application software must be managed appropriately.

The second issue arises from the overhead of moving data in and out of the rules engine. Depending on the amount of data and the location of the engine (remote?), this could be a relatively costly endeavor. There are a number of engines that can execute within the same memory space as the calling application and therefore reduce the overhead of moving data between the two. But again, depending on the amount of data required it may be questionable as to whether data, not explicitly required for rule execution, should be loaded at all. Some engines have the ability to read and write application databases directly which can simplify the data movement process, but this mixes IO overhead with rule execution. This feature also removes the ability to use optimization techniques like write-through share memory caching in applications.

Business Rules Portability?

With industry support for standards such as the Java Rule Engine API or Java Specification Requests (JSR) 94, applications can be developed that can plug in different rules engines. JSR-94 standardizes the way rules engines are called, but does not standardize how rules are written. We can plug a new rules engine into an application quickly, but moving the rules between engines can be problematic. There are initiatives out there like RuleML that attempt to facilitate interoperability among vendors, but none have reach critical mass. Do not count on JSR-94 to ease replacement of a rules engine significantly.

Testability

When constructing a suite of business rules, it is important to be able to verify that we achieve the desired results. Testing all rules in isolation will not necessarily give a definitive answer as to expected outcomes. Dependencies between rules may have unexpected effects. Yes we can do "what if" scenario testing, but the real work is in defining a comprehensive set of test cases that exercise the dark corners of the rule network. This was our most difficult part of the project, generating the volume of scenarios and associated data to ensure coverage across all the business rules.