NAME

mbx - DMH Mailbox message system commands for Tcl/Tk scripts

SYNOPSIS

dmh::mbx clientID
dmh::mbx close boxname
dmh::mbx count boxname
dmh::mbx count boxname_pattern reset
dmh::mbx disarm boxname
dmh::mbx end
dmh::mbx flush boxname
dmh::mbx get boxname
dmh::mbx help
dmh::mbx init groupname ?retryCount?
dmh::mbx open boxname
dmh::mbx product
dmh::mbx put mailbox message
dmh::mbx putr destBox replyBox message
dmh::mbx server groupname ?retryCount?
dmh::mbx whenmsg boxname code
dmh::mbx whenmsg again
dmh::mbx whenmsg dump


DESCRIPTION

This collection of commands is used to send and receive messages in a Tcl/Tk application using the Distributed Message Hub (DMH) from Hume Integration Software.  These commands are available in the extended Tcl/Tk shell, dmh_wish or after executing package require dmh.  As of Tcl version 8.4, the dmh package commands are in their own namespace ::dmh.  For convenient use, and for compatibility with pre-Tcl 8.4 applications, the command dmh_import can be executed to import these commands into the global namespace so that they can be called without using the ::dmh:: namespace prefix.

Interprocess communication using messages is a desirable alternative to Remote Procedure Calls (RPC) in typical CIM applications. A message interface is easier to develop, debug, and test. Also, the message system provides queuing. Queuing allows a slower process such as storage into a persistent database, to be used with fast processes such as test floor data collection, without being a bottleneck.

The concept is simple. Messages are sent to named queues called mailboxes. Each cooperating application reads messages from a set of mailboxes and sends messages to other mailboxes. Usually a message is a command, such as an SQL statement to insert a new row of data into a database table. A message may also carry with it the name of a mailbox where reply messages are to be sent.

Cooperating applications belong to named messaging groups. Typically, the first message system command used is mbx init. The groupname argument specifies the name of the message group. The mbx init command must be called before attempting message exchange. The mbx init command is the usual way for a client application to connect to an existing DMH message group server. A DMH message group server is created by executing a server application such as the Hume Datahub, or by using the mbx server groupname, command instead of the mbx init groupname command.

As of Tcl version 8.2, both the mbx init and mbx server commands accept an optional retryCount argument to specify an integer number of times that establishing the underlying TCP/IP socket connection should be attempted if failure is encountered. The default value is 1. The value of 6 can be used to specify the behavior that existed with the Tcl 8.0 version.

There is no fixed limit to the size of a message that can be sent or received using the DMH message system. If a DMH application regularly sends messages larger than 64k, the send and receive buffers can be reconfigured for both the server and client components by using the fconfigure command.

SENDING MESSAGES

The commands mbx put and mbx putr are used to send messages. The mbx put command sends a message with the attribute that no reply is expected. The mbx putr command is used when a mailbox is specified as the destination for reply messages. There is no requirement that your application has opened or reads from the specified reply mailbox. You may direct replies to a mailbox used by another process. Mailboxes are specified using their text names.
    mbx put DBM "insert into process_run (run_id, ts_start) \ 
      values ('$run_id', '[localtime 9]')"

    mbx putr DBM MYREPLY "select * from process_run where\
 ts_start>='1998-12-01 00:00:00'"

RECEIVING MESSAGES

The command mbx whenmsg boxname code is used to register Tcl code that gets executed when a message is received in the boxname mailbox. By default the code is executed only one time when the next message is received in the specified mailbox. The command mbx whenmsg again is executed when handling a message, to indicate that the same handling code should be invoked when the next message is received in the same mailbox. The code may constitute any valid Tcl code string - procedure calls, etc.

When a message is received into a mailbox where there is registered code, the whenmsg code begins execution at the global level of the interpreter, in the namespace ::dmh. This means that variables that are used in the code outside of a procedure call are accessible as global or namespace variables without being declared global, or being declared as namespace variables. Because of the danger of inadvertently using the same data instance in more than one active dispatch, it is highly recommended that you call a procedure in your whenmsg code to perform most of your logic. The following example is a recommended model:

mbx whenmsg MACHMGR { 
    process_MACHMGR_msg $mbxmsg $mbxdest 
    mbx whenmsg again
    }
Writing assignment statements directly from the whenmsg clause, as in:
mbx whenmsg MACHMGR {
    set i 0
    set token1  [lindex $mbxmsg 0]
    ...
may be problematic since variables like i and token1 can have their values changed when message exchanges are nested.

Before the code is evaluated, the following ::dmh namespace data items are set from the received message:

mbxmsg
The data content (text) of the message.
mbxdest
The mailbox name that the message was sent to.
mbxreply
The mailbox name where reply messages should be sent. An empty string signifies that reply messages should not be sent.


Note that the above variables are updated whenever a new message is received using mbx whenmsg.  If an application needs to preserve these data items across the reception of an additional message, they should be copied. It is recommended to the use these variables as arguments to a procedure, in which case local copies are made by the Tcl interpreter. A common method to pass data from one message to the next receiving context, is to make the data part of the code that is dynamically registered by the mbx whenmsg command.

The following command prints a message to the console whenever a message is received in the mailbox MBTEST.

mbx whenmsg MBTEST {puts "msg=$mbxmsg (reply to $mbxreply)";
       mbx whenmsg again}
The next example, demonstrates passing data from a previous message. In the mbx whenmsg code, brackets are used with the list command so that the current value of mbxmsg is evaluated and substituted as part of the command string. The example passes the entire message, but it is more common to pass selected data items such as index keys.
proc domsg {priormsg} {
    puts "prior message=$priormsg"
    puts "latest message=$::dmh::mbxmsg"
    # the next line is substituted now and saved
    mbx whenmsg MBTEST [list domsg $::dmh::mbxmsg]
    }
# initialize to receive the first message: 
mbx whenmsg MBTEST {domsg "there is no prior message"}

Only one code string can be registered for a specific mailbox. Newer registrations replace any existing registration for the specific mailbox. A process may wait on as many mailboxes as it wants.

When you write code for the whenmsg command, it can be tricky to combine variable values that are substituted at the time the whenmsg command is used, with variable values such as $mbxmsg that are to be substituted at the time the message is received.  The Tcl list command is the safest way to properly format values that are immediately substituted.  However, it will surround a value such as \$mbxmsg with braces, which will prevent it from being substituted.  Here is an example where both list and quoted string substitution are used to combine immediate and deferred substitution:


    proc handle_reply {cdata timer_h reply destbox} { ... }

    mbx whenmsg $replybox \
 "[list handle_reply $clientdata $timer_handle] \$mbxmsg \$mbxdest"

The command mbx whenmsg dump is used to display all of the currently registered handlers.

During the execution of a handler, it is not included in the set returned by the mbx whenmsg dump command. This is because the whenmsg code for the active handler is deactivated so that another message does not cause re-entrant usage of the whenmsg logic. Usually this behavior is highly desirable because it simplifies matters for the application developer. However, there are rare circumstances where re-entrant usage is needed. An example: a programmable database A initializes in response to a message. As part of the initialization, the database A sends a message to a server B, and waits for the reply from B to formulate the reply to the original initialization message. However, server B needs to query database A in order to process the request from database A. Database A needs to re-entrantly process commands from server B or he will be deadlocked. To activate re-entrant execution, it is not sufficient to call mbx whenmsg again. The full mbx whenmsg boxname code must be executed to rearm the handler.

A server that has been developed with the SEMATECH generic server framework or a compatible server, such as a Hume Datahub, may be rearmed for re-entrant commands using

mbx whenmsg server_command_mailbox {
    mbx whenmsg again; gs_execute $mbxdest $mbxmsg $mbxreply}

As we said above, it is only in rare circumstances where you want to re-arm a whenmsg handler for re-entrant execution.

The command mbx disarm is used to remove handler code that was registered using the mbx whenmsg command.

The command mbx get can be used to read the oldest pending message from a mailbox. If no messages are pending, an empty message with a null reply is received. If the command succeeds, the message and the reply mailbox are returned formatted as a two element list. Do not poll for messages using the get sub-command. Use the mbx whenmsg command instead.

MAILBOX MANIPULATION AND MISCELLANEOUS

In general you do not need to open a mailbox to use it; just start sending messages to it, or setting up reception of messages from it. The command mbx open can be used to create a mailbox if it does not already exist. The command does not affect an existing mailbox.

The command mbx flush is used to empty a mailbox of all pending messages. A pending message is one that has been sent to the mailbox but has not been consumed. In other words, a pending message is waiting in a queue associated with the mailbox. Messages are consumed by reading (mbx get or mbx whenmsg) or flushing. The mbx flush command opens a mailbox if the mailbox does not exist. If you want to insure that a mailbox exists and is empty, you can use mbx flush.

Calling mbx close disarms the mailbox for the caller and removes it from existence only if it is empty. If you intend to delete the mailbox when you close it, use mbx flush first. Any subsequent use of the mailbox will automatically reopen it.

The command mbx count mailbox returns a three element list giving the total count of messages that have been sent to the mailbox, the total count of messages that have been consumed from the mailbox, and last, the current count of pending messages. 

The alternate syntax mbx count name_pattern reset is used to reset the count for mailboxes that match the name_pattern argument.  The name_pattern argument is used with the Tcl string match command to determine which mailboxes are affected.  A mailbox name may be used as name_pattern argument value to match the single named mailbox.  The return value is a list of two element pairs. The first element in each pair is a mailbox name, and the second element is the three element list of counts after the values have been reset by subtracting the number of consumed messages from the total sent, and setting the consumed count to 0.  The current count of pending messages does not change.  The return value usually shows in-use mailbox names that match the pattern and have been affected by the reset.  If no in-use mailboxes match the pattern, the return value shows the name_pattern as a mailbox name with the reset counts of 0 0 0.

The command mbx product returns the string DMH. The reply indicates that the mbx Tcl command interface is supported by the Distributed Message Hub software product.

The command mbx end is used to free resources and discontinue using mbx commands. It is not proper to subsequently use most of the mbx commands without first calling mbx init.

The DMH server uses a unique identifier for each client connection. The client connection may be obtained using the mbx clientID command. The identifier is useful in conjunction with the mh_app_lostclient procedure. The Hume Data Collection Component uses this identifier in its API in order to name subscriptions which should be closed when a client disconnects from the DMH group.

The command mbx help returns a text synopsis of the mbx commands.

The Datahub user interface implements the procedure ::dmh::mh_statuswin which creates the DMH status window. This procedure can be called by your applications to create the same window.  The data displayed in the status window is obtained by calling the procedure ::dmh::mh_server_status using the global variable tcl_version as the single argument, as in "::dmh::mh_server_status $::tcl_version". The data returned by the ::dmh::mh_server_status is structured as follows:

The first element of the list is a list of 5 elements: {hostname:port messages_received messages_sent messages_queued tcl_version}

Subsequent elements in the list are lists of four or five elements:
{mailboxname count_in count_out count_pending [reader_handle]}

Additional elements may exist in the list if there are DMH clients that are not currently waiting to receive messages. These elements are formatted as:
{{{no whenmsg pending}} - - - reader_clientID}

The DMH message system must be initialized as a client or server to use the ::dmh::mh_server_status procedure.

AUTOLINKING

The message system supports using multiple independent named groups of communicating processes. This feature is important in larger deployments to distribute the processing load and to manage risk by reducing the dependence of the system on any single workstation or software process. Ordinarily, messages are sent to named mailboxes within a group. Communication across groups is easily and transparently accomplished by specifying an optional group designation in the mailbox name.

Mailbox names of the form <boxname>@<groupname> such as DBM@spc2 can be used to designate both the boxname and the message group. Message passing across separate message groups is automatically established when a compound name for a box in an external message group is encountered. Also, the trailing @<group> is properly ignored when it specifies the group that the application is a member of. The messaging support code takes care of setting up the return path for reply mailboxes as well as the forward direction for sending message. You never have to explicitly add the groupname of your own group to a local mailbox such as a reply mailbox.

The DMH software launches a single gateway process on demand per message group to connect to other message groups. This process is launched as a child of the message server on the workstation where the message server runs. When the server process is terminated, the gateway process is also terminated because of the Unix parent process - child process relationship. A gateway process is able to re-establish communication with external message groups if they are terminated and restarted. If only one named message group is used, there are no DMH demon processes, even if there are multiple connected workstations in the group. The Distributed Message Hub relies on the robust services of the operating system's TCP/IP support for name resolution, connection management, and message distribution.

ALIAS SUPPORT

The use of aliases is provided to support the creation and mapping of virtual message groups to physical message groups. This feature is used in more sophisticated applications where separate message groups such as SFC, FRONT_END, BACK_END, etc., are defined as a logical means of partioning the messaging and processing load. At installation time, the application can be readily configured to run in a single physical message group for a small deployment, or into several physical message groups (possibly on several workstations) for a large deployment.

When aliases are used, the ::dmh namespace array mh_alias holds the mapping of alias names to physical names. The alias array can also be used to specify TCP/IP hostnames of the workstation where a physical message group is "homed". Hostnames (or IP addresses) are needed when the indicated physical message group is not present on the current workstation. The supported syntax is that array element mh_alias(logical_groupname) is given the alias value [hostname:]physical_groupname.

Alias names are only effective if defined prior to sending messages to the boxname@group the first time. Also, they need to be known by the message server process, not the client process.

The DMH uses TCP/IP to connect multiple clients to message server hubs, any of which may execute on different hosts. At any time, a process can be started on any host and connect with any of the existing message hubs. Hostnames are specified as part of the message group name or in the mh_alias array whenever an application process is on a different workstation host than the message server process.

MAILBOX NAMES

Mailbox names should begin with alphanumeric characters and or the underscore. Subsequent characters may include ., @, or !. The @ character should only be used when specifying a two part name@group designation. The : character should only be used if a hostname or IP address is specified as part of the group as in RTDH@baloo:mbx or DATABASE@192.168.2.8:sfc. Older servers that use the SEMATECH generic server framework have a length limit of 32 characters in a mailbox name. This limit does not apply to the Hume Datahub. The mailbox name NULL may be used to indicate a "throwaway" destination. Command messages or reply messages sent to mailbox NULL are not actually sent, they are discarded. The most common use for this feature is as the REPLYTO parameter for a Hume Datahub subscription.

SEE ALSO

Higher level Tcl commands are available to perform transactions such as obtaining a reply or timing out mbx_timed_get), obtaining a series of replies ending with a certain pattern or timing out (mbx_xact_pat), or obtaining a known number of replies or timing out (mbx_do_xact).

The Hume SECS/GEM applications include the enhanced send-and-reply command ::humelib::mbx_xact which was originally developed for the Hume Component applications.

The mbx_RPC command is used to execute received DMH messages as Tcl commands. Using this command allows a DMH client to be inspected remotely using the inspect application.

You are also able to install procedures that are executed if the DMH message system connection is lost. See mh_app_lostclient and mh_app_lostserver.

AUTHOR

This is licensed and supported software available from Hume Integration Software.
 

KEYWORDS

DMH, mailbox, mbx, IPC, message