Configuring a Heterogeneous Multi-Node Domain

White Paper Sections

• Overview
• Application Model
• Administration Role
• Tuxedo Configuration File
• Configuration Highlights
• Tuxedo Configuration File (cont)
• Configuration Highlights (cont)
• Node Propagation
• NT Node
• UNIX Node
• AS/400 Node
• Tlisten Process
• Transaction Logs
• Boot the Domain
• Conclusion

Overview

The Tuxedo system allows a system architect to construct a networked application that includes diverse of operating systems. The MP (multi-process) domain model provides for the inclusion of heterogeneous systems to be joined to function as a single Tuxedo application. The MP model allows for the distribution of logic among domain nodes (machines) most suitable to carry out a unit of work.

A highly distributed Travel Services application might have one node dedicated to providing Hotel accommodation services while another node provides Airline reservations and a third might be the repository of Automobile rental services. In such an application the distribution of data and services might reflect the geographic or structural layout of the overall business as being divided into distinct business segments that individually contribute to the business unit as a whole.

Alternatively an MP model might be the basis for a system that prioritizes the high availability of services. High availability is often achieved by providing service, server and node redundancy. If the original Travel Service example was designed to consist of three nodes. High availability might call for the replication of each node onto additional machines. The additional machines could be strategically or geographically located whereby they are maintained independently yet operate in concert to provide continuous service availability.

A multi-node application can be designed to address many business scenarios that might benefit from the distribution of services. This design might also consider the benefits that various platforms might offer. Once again, in reference to the Travel Services application, the hardware choices may have been influenced by various factors. Hotel services might be the foundation of the business and originally written and deployed on a UNIX system. The Automobile portion of the business might deal with exclusively with a local rental fleet whereby an NT system is found to satisfy the business needs. The Airline reservations system might be an existing component in a federated reservations system used by Travel agencies worldwide, its platform, an AS/400.

This document considers the configuration of a system that includes the NT, UNIX and AS/400 platforms in an MP modeled Tuxedo environment. The hypothetical business model is given as a theme to provide focus. The intent of this paper is to illustrate the steps to successfully deploy an MP modeled application across a heterogeneous environment.

Return to Top of Page

Application Model

The target application is illustrated in the included image. The nodes illustrated represent the business segments provided for by the component machines:

The NT node represents the Car Rental business segment. Its services will provide the ability to retrieve fleet listings and to make reservations.

The UNIX node represents the Hotel business. The services listed will provide room listings; check in/check out services as well as reservations.

The AS/400 node represents the Airline segment of the business. It enables a client to view inbound and outbound flights to the Island. Oh, did I forget to mention that this Travel Service is based on an Island? Not to worry, it’s a small point. The services offered by this node are the appropriately scaled down to show flight availability and of course to make reservations. Being a small Island, we can assume that many of the actual airline services are not local to this node. They are imported from the much larger federation of reservations systems that this node utilizes.

Clients are not expressly illustrated in the application model. However, the configuration file to be built will make provisions for the NT node to accommodate up to 50 clients. Additionally, the domain gateway defined on the AS/400 node will provide remote access to the MultiDomain services by Federated client requests via the Domain Gateway. Lastly, any node within the domain can accommodate native clients.

Return to Top of Page

Administration Role

The first step we will take in constructing this domain model is to assume the role of Systems administrator. This loosely means that someone else has coded, compiled and tested all of the application code and that it is ready to go. As administrator, network addresses must be retrieved, user accounts created on the participant machines and other necessary installations of code or support mechanisms. We will jump to the part where the configuration file has been built.

The Tuxedo configuration file will be the basis for representing the desired application model. The configuration file is built to represent the desired layout of a domain.

The Tuxedo configuration file is built on the master node. It can be arbitrarily or strategically selected. In this example it will be the NT node that will serve as the master node. As such it is on this node that the configuration file will be constructed and compiled to produce the "tuxconfig" file.

Information to take note of prior to the propagation phase is the network addresses and port numbers selected for each of the participating nodes:

The NT node will need the network address of the NT machine and 12 port assignments. Two will be used in the Tuxedo configuration file to define the NLSADDR and NADDR network address information. Additionally the NLSADDR address will be used to start the tlisten process on the same machine. The Workstation Handlers that support workstation clients will use the other 10. The first of the 10 will be listed in the Tuxedo configuration file in the *SERVERS section.

The UNIX node will need the network address of the UNIX machine and 2 port assignments. They will be used in the Tuxedo configuration file to define the NLSADDR and NADDR network address information. The NLSADDR address will be used on the UNIX machine to start the tlisten process.

The AS/400 node will need the network address of the AS/400 and 3 port assignments. Two will be used in the Tuxedo configuration file to define the NLSADDR and NADDR network address information. The NLSADDR address will be used on the AS/400 to start the tlisten job. The third will be used to define the NWADDR of the domain gateway that will run on this node.

The configuration details of the Domain Gateway to be run on the AS/400 node will not be reviewed in detail.

Return to Top of Page

Tuxedo Configuration File

# UBBCONFIG file for the MultiDomain Application.


*RESOURCES
IPCKEY           123456

DOMAINID         multidomain
MASTER           rental
MAXACCESSERS     200
MAXSERVERS       100
MAXSERVICES      300
MODEL            MP
OPTIONS          LAN,migrate
LDBAL            Y

*MACHINES
NT_WS           LMID=rental
                APPDIR="c:\rentalapp\bin"
                TUXCONFIG="c:\rentalapp\cfg\tuxconfig"
                TUXDIR="c:\Tuxedo"
                TLOGDEVICE="c:\rentalapp\etc\TLOGD"
                TLOGNAME="TLOG"
                TLOGSIZE=500
                MAXWSCLIENTS=50
                TYPE="NT"


UNIX_BOX        LMID=hotel
                TUXDIR="/appl/sw/tuxedo"
                APPDIR="/appl/sw/hotel/bin"
                TUXCONFIG="/appl/sw/hotel/cfg/tuxconfig"
                ENVFILE="/appl/sw/hotel/env/hotel.env"
                TLOGDEVICE="/appl/sw/hotel/etc/TDEVICE"
                TLOGNAME="TLOG"
                TLOGSIZE=500
                GID=100
                UID=200
                TYPE="UNIX"

AS400_BOX       LMID=airline
                TUXDIR="/tuxdir"
                APPDIR="/apphome/tux/bin"
                TUXCONFIG="/apphome/tux/cfg/tuxconfig"
                ENVFILE="/applhome/tux/cfg/airline.env"
                GID=150
                UID=250
                TYPE="AS400"

*GROUPS
GROUP1  LMID=rental	GRPNO=10  TMSNAME=TMS  OPENINFO=NONE
WSLGRP  LMID=rental	GRPNO=20  TMLNAME=TMS  OPENINFO=NONE
GROUP2  LMID=hotel    	GRPNO=30  TMSNAME=TMS  OPENINFO=NONE
GROUP3  LMID=airline  	GRPNO=40 
DOMGRP  LMID=airline 	GRPNO=50
AIR_GRP LMID=airline 	GRPNO=60

Return to Top of Page

Configuration Highlights

*RESOURCES
The MP model, specifies a multi-machine configuration; the options identifier LAN indicates that this is a networked application.

MAXACCESSERS, MAXSERVERS and MAXSERVICES must be sized to accommodate the sum of all clients, servers and services offered on all three nodes.

The domain name to which all three nodes belong shall be identified as "multidomain". The master node of this domain is the machine identified by Logical Machine ID (LMID) "rental".

*MACHINES
This section lists all machines to be included in the domain. Each machine will act as a member node in the "multidomain" domain.

The TYPE parameter is used as an indication that the application involves a heterogeneous network of machines and that different compilers were used on the machines in the network. The value specified is arbitrary; its significance here indicates that all three machines are different from each other.

The ENVFILE parameter for the Unix and AS/400 boxes indicates that the contents of the specified environment file should be evaluated prior to booting the Tuxedo services.

*GROUPS
The groups’ definitions lend some insight into the organization structure of the server distribution. Ultimately servers will be assigned to groups.

The conclusions to be drawn upon the group definitions are:

• All servers belonging to groups "rental" and "hotel" may participate in global transactions.
• The Tuxedo provided server WSL will run on the "rental" node in its own server group (WSLGRP).
• The Tuxedo provided server DMADM will run on the "airline" node in its own server group (DOMGRP).
• The Tuxedo provided servers GWADM and GWTDOMAIN will run on the "airline" node in their own server group (GROUP3). Services imported through the Domain Gateway will appear (to Tuxedo) as belonging to this group.

Return to Top of Page

Tuxedo Configuration File (cont)

*SERVERS
DEFAULT:	CLOPT="-A"

# Define the Rental node servers
"WSL"   	SRVGRP="WSLGRP" SRVID=100
		CLOPT="-A -- -n //NT_NAME:55323 -m2 -M2 -x 10 -w WSH"
	    	RQPERM=0666 REPLYQ=N    RPPERM=0666 MIN=1   MAX=1   CONV=N
	    	SYSTEM_ACCESS=FASTPATH
	    	MAXGEN=1    GRACE=0 RESTART=N

"NTSRV1"      	SRVGRP=GROUP1 SRVID=105
"NTSRV2"      	SRVGRP=GROUP1 SRVID=110

# Define the Hotel node servers
"UNIXSRV1"	SRVGRP=GROUP2 SRVID=200
"UNIXSRV2"	SRVGRP=GROUP2 SRVID=205
"UNIXSRV3"	SRVGRP=GROUP2 SRVID=210

# Define the Airline node servers
"DMADM"		SRVGRP=DOMGRP SRVID=300
		RQPERM=0666 REPLYQ=N RPPERM=0666 MIN=1 MAX=1 CONV=N
		SYSTEM_ACCESS=FASTPATH
		MAXGEN=1 GRACE=86400 RESTART=N

"GWADM"		SRVGRP= GROUP3 SRVID=305
		RQPERM=0666 REPLYQ=N RPPERM=0666 MIN=1 MAX=1 CONV=N
		SYSTEM_ACCESS=FASTPATH
		MAXGEN=1 GRACE=86400 RESTART=N

"GWTDOMAIN"	SRVGRP=GROUP3 SRVID=310
		RQPERM=0666 REPLYQ=N RPPERM=0666 MIN=1 MAX=1 CONV=N
		SYSTEM_ACCESS=FASTPATH
		MAXGEN=1 GRACE=86400 RESTART=N

"AS400SRV"	SRVGRP=AIR_GRP SRVID=320


*SERVICES

Getlisting     ROUTING=TravelType BUFTYPE="FML32"


*NETGROUPS
DEFAULTNET      NETGRPNO=0      NETPRIO=100

*NETWORK
rental	NETGROUP=DEFAULTNET
	NADDR="//NT_NAME:55322"
	NLSADDR="//NT_NAME:55321"

hotel   NETGROUP=DEFAULTNET 
        NADDR="//UNIX_NAME:54322" 
        NLSADDR="//UNIX_NAME:54321"

as400   NETGROUP=DEFAULTNET 
        NADDR="//12.34.5.6:53322"
        NLSADDR="//12.34.5.6:53321"

*ROUTING
Multiplication
	FIELD=agency
	RANGES="'Rental':GROUP1,'Lodging':GROUP2"
	BUFTYPE="FML32"


	

Return to Top of Page

Configuration Highlights (cont)

*SERVERS
Within the server section, the group association points to the node where the server is expected to reside and run.

The WSL is a Tuxedo provided server. It is an entry point for workstation clients. It listens to the specified network address and port and is capable of spawning up to 10 workstation handlers, which would utilize the next 10 available port numbers. This server will be boot on the Rental node, as indicated by its group assignment.

The DMADM, GWADM and GWTDOMAIN are Tuxedo provided which facilitates Domain gateway communications. It is assigned to listen to a specified network address and port number for gateway-to-gateway connectivity. This server will be boot on the Airline node.

*SERVICES
It is not a requirement to list services. However the "Getlisting" service has special routing needs. This service exists in two different servers that provide different functionality. The TravelType routing criteria will determine the data dependent server group routing.

*NETGROUPS
This section is required for the MP model.
The NETGROUPS section describes the network groups available to the application in the LAN environment. Any pair of machines may be in any number of network groups. Two communicating nodes use the priority mechanism in order to determine how to communicate between elements of its group.
Every LMID will be a member of the default netgroup.

*NETWORK
This section is required for the MP model with the OPTIONS value of LAN.
The NETWORK section describes the network configuration for a LAN environment. For each processor on which a bridge server is located, an entry must be placed in the NETWORK section giving the network address of the bridge process.

The NADDR field parameter specifies the complete network address to be used by the bridge process placed on the LMID as its listening address. The listening address for a bridge is the means by which it is contacted by other bridge processes in the domain. The NLSADDR specifies the network address used by the tlisten process servicing the network on the node identified by the LMID.

*ROUTING
The routing criterion specified will evaluate the agency field element of the received FML32 buffer. If the value matches the "Rental" routing variable, the request is directed to the server group whose servers are running on the Rental node. Alternatively a "Lodging" routing resolution would direct a given request to the server running on the Hotel node.

Return to Top of Page

Node Propagation

Prior to booting the domain application servers, fielded tables, view files and other support information will need to be propagated to or built on the non-master nodes. The tlisten process can be started now that the TUXDIR and APPDIR have been determined.

The propagation should be specific to the functionality provided by each node in the domain. (i.e. Support files such as the Domain Gateway configuration file need only exist on the "airline" node. Gateway communication will be available to every node in the domain, however the servers that supply it will only run on the "airline" node)

Tuxedo provided servers will pre-exist on each node. The configuration file once it is propagated will trigger their use if applicable. Application servers however should be pre-built and placed in the designated ${APPDIR} directory.

Return to Top of Page

NT Node

Servers NTSRV1 and NTSRV2 will be installed to directory "c:\rentalapp\bin".

The client/server communications medium will be by FML buffer. Requests for this nodes instance of the "GetListing" service will be resolved by data dependent routing. The FML field "agency" having the value "Rental" will determine the routing to this server.

The environment variables TUXDIR and APPDIR must be appropriately set (to c:\Tuxedo and c:\rentalapp respectively) prior to starting the tlisten process.

The tlisten process is started via the BEA Tuxedo tool. It is possible to use the tool auto-start the tlisten process when the workstation is reboot. See the section on tlisten.

For transactional control for transactions involving the "rental" node a transaction log must be created. The following: "tmadmin -c crdl -z c:\rentalapp\etc\TLOGD -b 700" will create the transaction log for the NT machine. See the section on Transaction Logs for more detail.

Return to Top of Page

UNIX Node

The servers UNIXSRV1, UNIXSRV2 and UNIXSRV2 will be installed to directory "/appl/sw/hotel/bin".

All services will communicate via FML buffers. Requests for this nodes instance of the "GetListing" service will be resolved by data dependent routing. The FML field "agency" having the value "Lodging" will determine the routing to this server.

The environment variables TUXDIR, LD_LIBRARY_PATH and APPDIR must be appropriately set (/appl/sw/tuxedo, /appl/sw/tuxedo/lib and /appl/sw/hotel/bin respectively) prior to starting the tlisten process.

A separate environment file should exist for non-master nodes. It is an environment that all clients and serves on the machine are to be executed upon. Lines must be of the form ident=value.

When the master node (rental) boots the "hotel" node, the Tuxedo servers will inherit the environment of tlisten. For this reason TUXDIR and APPDIR are defined prior to starting the tlisten process. When the Tuxedo bbl and its servers are booted, it will assume environment definitions specified in the MACHINES section of the configuration file that is automatically copied from the master node. Within this file is an ENVFILE parameter that indicates the environment file to load.

The environment file on the Hotel node, will contain entries like the following:

PATH=/appl/sw/tuxedo:/appl/sw/hotel/apps                       
VIEWDIR32=/appl/sw/tuxedo/udataobj:/appl/sw/hotel/include         
VIEWFILES32=mib_views.V,tmib_views.V,Hotel.V,FMLVIEW.V    
FLDTBLDIR32=/appl/sw/tuxedo/udataobj:/appl/sw/hotel/include
FIELDTBLS32=Usysfl32,dmadmin,evt_mib,rental_flds,FMLVIEW 

An environment file can also be specified within the definition of a server. In respect to the two environment files - with the exception of TUXCONFIG, TUXDIR and APPDIR, if a variable is set in both the machine and server ENVFILE, the value in the server ENVFILE will override the value in the machine ENVFILE.

After the application software, environment file, View and/or FML files have been manually propagated to the UNIX machine. The tlisten process can be started manually as follows:
"tlisten -l //unix_name:54321"
This command can be entered into the system crontab for auto start or manually executed. See the section on tlisten.

Transactions running on the "hotel" node will require a transaction log. The following: "tmadmin -c crdl -z /appl/sw/hotel/etc/TLOGDEVICE -b 700"
will create the transaction log for the UNIX machine. See the section on Transaction Logs for more detail.

Return to Top of Page

AS/400 Node

The server AS400SRV will be installed to directory "/apphome/tux/bin".

All services will communicate via FML buffers. The environment variables TUXDIR and APPDIR must be appropriately set (to /tuxdir and /apphome/tux/bin respectively) prior to starting the tlisten job.

The tlisten process can be started manually by submitting it as a background job: "sbmjob cmd(tlisten ipaddr('//12.34.5.6:53321')) job(tlisten) jobd(tuxedo) jobq(tuxedo) user(tuxedo) cpyenvvar(*YES)".

Following the activation of this job "wrksbsjob Tuxedo" will display a list of background jobs, which will help confirm that the tlisten job was properly started. See the section on tlisten.

A separate environment file should exist for non-master nodes. It is an environment that all clients and serves on the machine are to be executed upon. Lines must be of the form ident=value.

When the master node (rental) boots the "airline" node, the Tuxedo servers will inherit the environment of tlisten. For this reason TUXDIR and APPDIR are defined prior to starting the tlisten process. When the Tuxedo bbl and its servers are booted, it will assume environment definitions specified in the *MACHINES section of the configuration file that is automatically copied from the master node. Within this file is an ENVFILE parameter that indicates the environment file to load.

The environment file on the "airline" node, will contain entries like the following:

PATH=/tuxdir/bin:/apphome/tux:/home/tux:/usr/bin WSTYPE=as400 BDMCONFIG=/apphome/tux/bdmconfig DMCONFIG=/apphome/tux/dmconfig QMCONFIG=/apphome/tux/QUE FIELDTBLS32=FLDTBL FLDTBLDIR32=/buffers/fml32 VIEWDIR32=/buffers/view32 VIEWFILES32=ukemp.VV,FLDTYPES.VV,ALLTYPES.VV,VIEWS.VV

An environment file can also be specified within the definition of a server. In respect to the two environment files - with the exception of TUXCONFIG, TUXDIR and APPDIR, if a variable is set in both the machine and server ENVFILE, the value in the server ENVFILE will override the value in the machine ENVFILE.

The domain configuration file can also be defined at this time. It specifies the configuration information that will be in effect when the Tuxedo provided Gateway servers are boot on the "airline" node. The domain configuration file cannot be compiled (via dmloadcf()) until the tuxconfig file is propagated to this node. The tuxconfig file is automatically transferred to all nodes in the domain when the master attempts to boot the domain.

The compiled domain configuration file "bdmconfig" contains information providing connectivity via the Domain Gateway server.

Similar to the TLOGDEVICE, the domain transaction log facilitates the processing of requests in transaction mode. The Domain transaction device is defined by the domain configuration entry "DMTLOGDEV" and the name of the transaction log is specified by the log name entry "DMTLOGNAME".

Return to Top of Page

Tlisten Process

The tlisten process is a network independent listener that runs as a daemon process on BEA Tuxedo application processors and provides remote service connections for other BEA Tuxedo processes.

Each node must have the tlisten process properly started and running on an established network address and port.

The master node upon startup will communicate to the domain member nodes via the tlisten process. In order for member node startup, which is directed by the tlisten process it must have environment information and rights to locate and execute Tuxedo programs. Using this it boots the bbl and initializes it to the proper environment.

Since the tlisten process provides an open network connection, by which it facilitates access and control over a nodes bbl, password security is provided. The security measure is one, which validates access to the tlisten process.

During the installation process, the administrative password file is created. When accessing tlisten, BEA Tuxedo searches for this file in two places:

• ${APPDIR}/.adm/tlisten.pw
• ${TUXDIR}/udataobj/tlisten.pw

Return to Top of Page

Transaction Logs

For transactional control between nodes a transaction log (TLOG) needs to exist on all machines that will participate in global transactions. The TLOG refers to a log in which information on the transactions is kept until the transaction is completed. A TLOG entry is created in the TLOGDEVICE.

The TLOG device is created using tmadmin. The command:
"crdl -z config -b blocks" creates the device.

The config argument should be the full path name for the device to be created, it must match the TLOGDEVICE entry in the ubbconfig file for the machine. This should be done for each machine in the domain that will participate in transactions.

The blocks argument should be greater than the log size specified in the ubbconfig file. The device is created to accommodate logs. The size specified in the ubbconfig file reflects the portion of the device size that should be dedicated to the transaction log. Appropriately there should be more space in the device, than is used by a log residing in the device.

Return to Top of Page

Boot the Domain

The BEA Tuxedo bulletin board command interpreter, tmadmin, run on the master node provides the ability to activate all machines in the domain.

The command "boot -A" when run from within tmadmin, starts all administrative servers for all machines in the *MACHINES section. Starting with the master node, the DBBL, BBL and BRIDGE processes are activated. Using the "-A" option confirms connectivity to the listeners on the respective machines. Using these listeners, as a conduit for instructions, the BBL and BRIDGE processes are started on the non-master machines. A significant part of this boot process is that the Tuxedo configuration file, tuxconfig, is transferred to each of the non-master nodes.

During the propagation step, the transaction log devices were created on each node, but the TLOG was not. This log could not be created on the non-master nodes until the tuxconfig file was present on the non-master nodes. While still in the tmadmin session on the master node, verify that the transaction log has been created before starting the servers:

"livtoc -z <device path>" - This command will print the VTOC contents of the specified device. Use this to verify that the transaction log named in the configuration file (TLOGNAME) exists.

If this log file is not present on each of the nodes, the TLOG entries can be created as follows:

"crlog -m rental" - This command will create the transaction log on the NT machine. This command references the tuxconfig file thereby retrieving information regarding the name and location of the TLOGDEVICE, as well as the size and name of the log file to create.

"crlog -m hotel" - This command will create the transaction log on the UNIX machine. This command references the tuxconfig file thereby retrieving information regarding the name and location of the TLOGDEVICE, as well as the size and name of the log file to create.

Now that the configuration file, tuxconfig, is now present on each node of the domain, the Airline node (AS/400) can now compile its domain configuration file (dmconfig). Using the Tuxedo utility command, dmloadcf(), the resulting file "bdmconfig" will be created. This command must be executed on the AS/400.

The last step in this process is to boot the non-administrative servers. The boot command: "boot -S" will start all servers in the *SERVERS section of the configuration file.

Return to Top of Page

Conclusion

A premise for the construction of the preceding MP configuration was the architectural need to construct a Travel Services business unit. The domain servers included in the AS/400 node services indicates connectivity to some "other" domain. The implication was that this "other" domain represented a different business unit. (An Airline industry)

These suppositions illustrate one of the fundamental reasons of when to use an MP configuration and when to use multiple domains communicating via gateway - business constraints.

Contributing factors and situations, which influence the use of MP configurations or Domains, are unique to each application. Therefore hard and fast rules might take the back seat to flexible and thoughtful reflection in regard to past, present and future application needs. These include – maintenance, application turnover, service availability, expansion, redundancy, network availability, network and service reliability and application longevity.

A single Tuxedo Configuration file represents the application configuration, multi-node or single. This fact translates to a single point of administration. The administrator defines and controls the behavior and resources to be allotted to the application as a whole from one place. This administrative detail should factor into the decision making process.

~end of Aurora Information Systems White Paper Series #11
  "Configuring a Heterogeneous Multi-Node Domain"~

Return to Top of Page

Do you find this information interesting? Consider employment with Aurora!

Do you need help implementing your Middleware app? Aurora can make your life easier.