What is the relation and differences between the following terms?
- Enterprise Messaging System (EMS)
- Enterprise Service Bus (ESB)
- Message Oriented Middleware (MOM)
- Java Messaging Service (JMS)
Answer
Good question - the crucial difference between a service bus and messaging system is the data convention on your messaging system. A messaging system typically let's you sent everything: binary blobs, XML, comma-separated lists, etc. So application A can send a comma-separated string to application B and B sends some XML to application C and C sends some other XML to app D. That's messaging, but not a 'service bus'. You could say a messaging system is 'untyped' (dynamic structure) while an ESB is 'typed' (static structure).
In a 'service bus' you have a common data definition for all applications and adapters on that bus (could be XML with a shared XSD). Common data objects (CDOs). Anything that connects MUST send it's information adhering to this data definition. The ESB should support loading, sharing and versioning this common data definition. The big advantage is that you can connect a component (e.g. a Message Broker) and it does it's thing without having to know which application sent this data and where this data is going to.
The trade-offs of Messaging vs. ESB are similar to other untyped/typed choices: REST vs. SOAP, unvalidated XML vs. XML with XSD, Groovy vs. Java, ... Some people will enjoy the additional structure (looks good on paper - managers like it) - some will hate it (stuff breaks when versions change, for a little addition you have to update everything - hackers don't like it so much ;-)
Coming back to your questions (reordered)
Message Oriented Middleware (MOM): software libraries for various languages with a broker (or not) to communicate 'messages' between applications. One step up from TCP/IP communication. 'messages' are structured objects or text strings or binary data. Usually you have additional reliability over TCP/IP or UDP. Some examples: TIBCO RV and EMS, IBM MQ, Apache ActiveMQ, ZeroMQ, ...
Java Messaging Service (JMS): the definition of a common API for MOM's - people complained that when your application switches from MOM 'X' to MOM 'Y' you need to rewrite the messaging code. If you code against JMS, you can just switch the libraries and the same application that used to work with TIBCO EMS suddenly works with ActiveMQ (or vice versa)
Enterprise Messaging System (EMS): TIBCO's implementation of JMS (product name: TIBCO EMS)
Enterprise Service Bus (ESB): an ESB uses a message oriented middleware to integrate applications, databases, brokers, etc. An ESB is a MOM with added data structure and structure definition management. When connecting a new component to an ESB, you can expect more 'compatibility' out of the box than when connecting it to a MOM. In an ESB there are higher standards on what a component must do to connect. TIBCO's ESB is called ActiveMatrix, I think.