Configuring TIBCO EMS for Fault-Tolerance on Linux

TIBCO Enterprise Message Service provides Java Message Service (JMS) compliant communications across a wide range of platforms and application technologies. I will describe below the main configuration steps required to configure two Linux machines with EMS installed to work together in an shared-state fault-tolerance configuration. This configuration was made with TIBCO EMS 8.5.0

EMS shared state Fault-tolerance configuration is where the configuration and message store is shared between the two EMS servers. One EMS server is in active state; writing and reading from the message store and configuration files. Once the active server is down, the ems passive server will take over and start reading the configuration files and message store and continue accepting connections from clients.

The connection string will be in the form tcp://server1:port1,tcp://server2:port2
port1 may be the same value as port2. 7222 is the default value for EMS port.

Installation

You would install EMS normally. Using TIBCO universal installer, or by just unziping the file TIB_ems_8.5.0_linux_x86_64.zip. I was not able to find TIBCOuniversalinstaller binary for EMS 8.5.0 so I just unzipped the files into some folder let's say /var/tibco/ems/8.5

Configuration

To configure EMS you just take a copy from sample configuration files and edit them. The main configuration file tibemsd.conf, determines things like listen ip address and port, EMS servername, location of other configuration files and so on.

For a standalone EMS server I just set the following parameters:
server this is the name of the server
users path of users.conf
groups path of groups.conf
topics path of topics.conf
queues path of queues.conf
acl_list path of acl.conf
factories path of factories.conf
routes path of routes.conf
bridges path of bridges.conf
transports path of transports.conf
tibrvcm path of tibrvcm.conf
durables path of durables.conf
channels path of channels.conf
stores path of stores.conf
store path where store files will be created
listen listen address and port, ex. tcp://10.10.101.22:8222 or tcp://8222 to listen on the default IP address.
authorization should be set to enabled to verify users' passwords

For EMS Fault-Tolerance configuration, the two EMS machines share the configuration files and datastore. EMS server will try to lock the datastore on the shared file system in order to ensure it is the active server. If one EMS server is able to have a file lock on the datastore and it is unable to access the other EMS server, it will be the active server.

The file system shared between the two machines must support the following criteria:

1. Write Order: The storage solution must write data blocks to shared storage in the same order as they occur in the data buffer.
   (Solutions that write data blocks in any other order (for example, to enhance disk efficiency) do not satisfy this requirement.)

2. Synchronous Write Persistence: Upon return from a synchronous write call, the storage solution guarantees that all the data have been written to durable, persistent storage.

3. Distributed File Locking: The EMS servers must be able to request and obtain an exclusive lock on the shared storage. The storage solution must not assign the locks to two servers simultaneously. EMS servers use this lock to determine the primary server.

4. Unique Write Ownership: The EMS server process that has the file lock must be the only server process that can write to the file. Once the system transfers the lock to another server, pending writes queued by the previous owner must fail.

More details about the file system that is required is mentioned in the user guide TIBCO documentation for EMS. Further info about the file system is mentioned in this TIBCO support link:
KB Article Shared-state requirements
I've tried cifs file system for the shared state and I did not encounter any issues.

The configuration is done on each server for the main config file tibemsd.conf. The rest of the configuration files and datastore location must be on the shared folder.

Each EMS server have its own copy of tibemsd.conf file. The configuration parameters related to fault tolerance setup are as follows:
server: The server name. The two servers must have the same server name.
ft_server: the url of the other server in the fault tolerant setup.
password: the password that is used by each server to connect the other.
authorization: must be set to enabled.
Configuration files path should be changed to point to the shared state folder.
datastore: should point to the datastore folder on the shared state folder.
ft_heartbeat: Specifies the interval (in seconds) the server has to send a heartbeat signal to other server to indicate that it is still operating.
ft_activation: maximum length of time between heartbeat signals. Default is 60 seconds.
ft_reconnection_timeout:If a client does not reconnect within this time period, the server removes its state from the shared state files. The ft_reconnect_timeout time starts once the server has fully recovered the shared state, so this value does not account for the time it takes to recover the store files. Default is 60 seconds.

To start the EMS server you will use the command tibemsd -conf tibemsd.conf

Summary

Steps

1. Install EMS Server on machine1.
2. Install TEA server to use ems-agent on machine1.(Optional)
3. Configure EMS Server on machine1 for fault-tolerance as active server.
4. Configure TEA ems-agent and register the EMS server to one of the TEA servers.(Optional)
5. Repeat the above steps on the machine2
6. Start EMS server on machine 1
7. Start EMS server on machine 2
8. Connect to each TEA server to make sure that you are able to see the configured EMS servers.(Optional)
9. Test the EMS FT setup using the EMS client sample java application tibjmsMsgProducerPerf from EMS's samples/java/ folder.