f3yourmind

I’ve seen many enterprise service bus (ESB) implementations that are, well, quite extraordinary.  Sadly, they are extraordinary for more wrong reasons than right.  Given that we had a SOA frenzy not too long ago, many developers got caught in that feeding frenzy.  Hey, when you’re a shark off the south coast of Kwazulu-Natal in winter, everything looks like a sardine.  In fact, if you were a tiny sole, just wandering around at the bottom of a big sea, chances are you also wanted some of them some sardines.

That was a long time ago, but now that we get to see the extraordinary things that developers built with their ESB.  Here are my top five extraordinary things.

#5 Look, we can synchronize our databases with our ESB

You have two application databases and you want the changes in one to be propagated to the other.  The ESB seemed like a perfect way to spray data around.  Hey, just give me an end point and I’d pump some data through it.  Well, syncing data is a replication problem with different challenges such as change detection, conflict resolution, retry or abort strategies, and bulk materialization on both ends when things get horribly out of sync.

#4 You can call my stored proc from the other side of my ESB

Somewhere in your app you had something that called a stored procedure. The ESB seemed like a really easy way to wrap that SProc with a service and hand out a new end point.  Cool, now everyone can call your stored proc.  Well, perhaps that original thing that was calling your stored proc was a wrong architecture decision in the first place.  If it was the write decision, suddenly opening it up to multiple callers that you have no control over means you got to be certain about whether your SProc is re-entrant, can handle the concurrency, and a whole lot more. 

#3 My ESB now manages my transactions

Enough said.

#2 My authentication runs as a centralized service on my ESB

Login seems like an easy enough “hello world” service to get up and running.  Hey, since our ESB has all of these apps hanging off it, we can get our Login Service to do single sign on (SSO).  We just need to call all the login services for each app and front it with our single sign on service and then this service will dish out authentication tokens.  Oh, we might as well put in a centralised authorisation service too.  Well, firstly, login is not a service.  You can take that thought further from here.

#1 I replaced my call stack with my ESB

To be fair, most developers don’t know that they did this.  Let me explain.  You had some app that called a method that calls a few more methods and so on.  We all know that this builds up a call stack that gets popped on the return path.  When you take those classes that have those methods and dump them as services on your ESB, the call stack has not changed, but now there is an ESB in between.  Well, you can’t just shift classes wrapped as services to your ESB.  You actually have to get service oriented in your architecture.  Only then will that call stack change.

It’s not surprising that we see this now.  It takes a long time for an architectural style to be understood.  Unfortunately, there are some costly mistakes along the way.  We also can’t blame the ESB, but, beware, the sardine run happens every year.

Recently I wrote about the fruitless plight of a schizophrenic service.  Now, I think that some of that schizophrenia exists in the ESB too (or is it rubbing off onto the ESB?).  I’ve always felt that the ESB was just another pattern that showed how to isolate things and deal with routing and transformations.  The most common implementation was a messaging gadget with some pluggable framework of sorts for the transformations, and some configurable framework for routing.

With such isolation of parts, it was convenient to not worry about what happened elsewhere when something was thrown to this gadget for processing.  And we started wondering about scalability things and decided that asynchronous was the way to go … disconnected, stateless, etc, all good, well-intentioned things and useful things.

Then the pattern became a product.  And on top of this product we had more products like business process orchestrators or workflow managers.  And below this product we had applications and databases and ftp locations and all sorts of things that catered for every imaginable protocol.  And around all of this we had some enterprise-ish sort of management thing to keep on eye on everything that was happening inside this very busy product.

Then, services moved from applications to the ESB product.  After all, it’s a service and that’s an enterprise service bus, right?  And when the services where moved over to their new home, all the dependencies had to come along too.  And then we started arguing about getting granularity right in the ESB.  I used to just think that the ESB had a proxy of sorts to the service that still was at the application.  Maybe I got it all wrong.

Now this ESB is starting to feel like an Application Server with a messaging gadget, workflow gadget, transformers, routers, protocol handlers.  And some ESB’s have a web server too, since they have browser based management consoles.

Some people also like the idea of a rules engine for their complex domain rules and embedded those in their applications.  Hold on, those content based routers in the ESB also used a rules engine.  Ok, let’s move our rules over to the ESB too.  Cool, my ESB is also a rules engine.

Now, I see people writing the most hellish XML that is meant to do everything from configure routing, define transformations, execute code, persist messages, fire off sets of rules and more.  It reminds me so much of those weird and wonderful stored procedures and cascading triggers that we wrote.  The other day I got a laugh out of a friend when I told him that ESB’s are now DB servers and everyone writes sprocs in XML.

And we tried to do everything in the database server – rules, custom types, defaults, constraints, sprocs, triggers, batch jobs … even jump into a shell and execute something else.  It did not work out very well then.

If I was an ESB, I’d be very confused.  I started life as a pattern with a reasonable implementation using messaging and transformation and routing.  Now all of this.  In fact, I’d be more stressed than confused.

Then again, maybe the ESB is not confused, and maybe the people that use the ESB that are confused.  In fact, if I was one of those people, I’d be stressed too.

Many big vendors have invested a lot on blue print or reference architectures.  I came across another in recent months.  I witnessed a vendor team moving from client to client implementing this reference architecture as part of their SOA solution.

What were they actually doing? They were mapping the client’s domain to the reference architecture domain and thereby identified reference architecture services that supported the client’s needs.  This most probably works for some people.   But I feel uncomfortable with it because…

• It means translating from one domain to another and back again.  It’s like having one massive bounded context around the reference architecture with a gigantic set of adaptors and transformers.
• There is a very real possibility of semantic impedance on the boundary of the two domains.
• There is likely to be two domain vocabularies or one large polluted vocabulary with synonyms, etc.

There are other reasons but these few are just old problems and habits coming back again.  Things that we accepted as dangerous and limits success in creating good software.

So, are reference architectures bad? Yes and no.  Maybe you should consider adopting its domain vocabulary as a first step.  A reference architecture with a rich metamodel is more likely to be more valuable than one without a metamodel.

And the moment you start thinking at a meta level, then you’re moving into a higher level of abstraction.  In this higher level, you will have a greater opportunity to describe your intentions agnostic of the reference architecture and the vendor’s technology stack.

The way I see it, services are defined at a meta level.  They describe your intentions and are independent of a reference architecture.  However, if you chose a reference architecture up front, then describe your intentions in the vocabulary of the reference architecture.

Does this make sense?  Because I’m just hypothesising here.

There will always be someone who will scream that Cloud Computing is new.  And there’s someone else that will tell you it’s old-school-been-there-done-that.  And there will always be some vendor that will try to sell something too.

Since we shrink in body and mind to the state of children when thrown into a new context, beware of stranger danger.  It’s tempting to follow strangers for cheap candy in the cloud.

And yes, like SOA, we’ve seen and done this before.  But I think it is important that we let the passage of time to gel our experiences and ideas over and over again.  Why?  Because contexts change with time and we can use new contexts to innovate again and again and again.

You’re allowed to think … for yourself … and decide … for yourself.

“Hello.  My name is Hope and I am a Service.  Last week I was asked by a View to do something.  I told the View that I can’t do it.  So he asked the Fat Controller.  The Controller sent me the same message and I just took exception.  What kind of service does he think I am?”

“Hello Hope.  We are glad you joined us.  You are not alone.  Look around the room.  We’re all struggling to find our own identity.  Those of us that have been around for a long time are still recovering from being forced to convert DomainObjects to DTOs.  The more recent ones feel like meaningless proxies, and last week we had a guy who thought he was a Service but he was just a HelperClass.”

“It’s as if I am losing touch with reality.  My own DomainObjects don’t even interact with me directly.  Now processes want a piece of me too.  I think I am schizophrenic.  Am I now a process?  A wrapper point for Transactions?  I swear to you, there are days when I even think I am a stored procedure!”

“Ohhhh, noooo!”

“But I know who I am. I am a Service.  I like working with DomainObjects.  They need me.  Somethimes they can’t do everything on their own, so I help out.  I complete their world and it makes me feel like I belong in the right place.  One guy even refactored me from the domain package to a services package.  Can you believe that!  Actually, now that I’ve said it aloud, I am not surprised that I feel more and more disoriented.”

“Disoriented?  Oh boy, we really can’t help you here.  You see we’re recovering from abusive Controllers and Views and when you told us the story in the beginning, we thought we could help you but you’re just a …”

“But that story is true.  Apparently there was this Process on other side of  this MessageBus that needed something and he asked the Fat Controller who got involved with View and then I got sent this message and …”

“WTF?!!  The Process asked the Controller…”

“Yeah!  You won’t understand … it’s after your time.  It’s really a WebServices Controller but I hear that he also feels abused and just wants some REST.  You think there’s a support group for Controllers?”

“Wait.  I remember this other crowd … uhhmmmm … sorry – those weren’t Controllers, those were Presenters working with Models and Views, but I used to be confused as a Delegator and I can …”

“You’re NOT HELPING!!!  I told you I am a Service!  And I am confused!  I want to be part of the domain again!  I feel disoriented!  And I …”

“And I told you that we can’t help you.  The truth is that you are DOA.”

“Dead on Arrival?”

“No! DisOriented Architecture.  You know … they don’t know where to put that, so it becomes a service.”

“But, but … I am a Service … <sniff>”

“YOU WERE NEVER A SERVICE!  YOU’RE ONLY HOPE!!!”