The Hume OEM Data Collection & SPC Application Component

Licensed and Supported Software
(C) 2004 Hume Integration Software
All Rights Reserved

Introduction

Hume Integration has developed a series of high-level application components which build upon the high performance and portable architectural framework provided by the Hume Datahub and the DMH message system.  The first such component, the Data Collection Component (DCC) features comprehensive Statistical Process Control (SPC) analysis capabilities including all traditional Shewhart and Attribute charts, along with cusum and Exponentially Weighted Moving Average (EWMA) control charts.  The SPC capability is offered as part of a data collection system.  The data collection system accommodates capturing non-SPC data such as context items (lot, wafer ID, material, process step, equipment, etc.), qualitative observations, pareto lists, selection choices, as well as structured text input.  Manual data entry can easily be combined with semi-automated, or fully automated collection of data from production and test equipment.  The system also includes configurable, table-driven security services.  The security services make it easy to password protect privileged application features such as the configuration of SPC run rules.

The DCC is typically used in a client-server, distributed application.  The DCC can be deployed as an application by itself, or as a component in a larger system.  It is also possible to use the DCC server package imbedded in an application process, perhaps for in-situ SPC analysis.  In the usual client-server deployment, the DCC server package executes in a separate process,  integrated with User Interface processes,  equipment interface processes, and java-enabled web clients, all communicating using the DMH message system.  The DCC is also capable sending e-mail messages, or placing telephone calls to notify recipients when SPC alarms occur.   The DCC includes a Graphical User Interface (GUI) client application that can be used to collect and edit data, to manage the configuration of data collection items,  to view charted data, or to view tabular reports.  The GUI also features the ability to manage SECS Trace Report and Event Report data collection by communicating with instances of the Hume SECS/GEM applications.  The GUI can be used as a ready-to-run solution for managing collected data, or as a starting point or supplemental tool for the OEM who chooses to build his own GUI.

The DCC server package consists of a set of SQL tables running in a Tcl/Tk datahub, a handful of Tcl procedures that are registered as table subscribers and executed in response to changes in the table data, and a family of Tcl procedures that are provided as an Application Programming Interface (API).   The DCC server package also includes the capability to create, populate, and use tables in an SQL relational database such as an Oracle,  Sybase or DB2 instance.  When a persistent database is used, the in-memory SQL tables are populated at startup using data from the persistent database.  As the system runs,  the logic uses the in-memory tables for higher performance.  Connectivity is provided to the persistent database using the DMH message system so that clients are able to query and use historic data from the persistent database which contains older and more voluminous data than the in-memory database.

The DCC component uses an open, extensible architecture.  The table schema is described in detail in this document which enables the OEM or enduser to create his own reports, custom logic, or custom applications.  The schema is designed to support using multiple instances of the DCC system, as peers or within a hierarchy, where there may be a need to combine data from multiple instances in common tables.  The server API procedures are also described, and the source code is provided for inspection or customization.  It is completely feasible that the OEM could use the DCC component and/or the extensible Tcl/Tk environment for exciting new applications such as an XML-RPC server without requiring source code or cooperation from Hume Integration.  The DCC can be used in the middle of a distributed application, with the DMH message system software being used to span Visual Basic and Active-X platforms,  Java, Visual C++, and POSIX C platforms, in order to integrate data collection from multiple sources and to share results with multiple consumers.  The DCC server can also function as a web server using the popular Tcl httpd extension.

Feature Summary

Data Collection & SPC Component - Server Installation and Startup

The DCC server component executes in an extended Tcl/Tk process such as the Hume Integration Datahub.  It is common for the DCC server process to also function as a DMH message system server so that client processes can easily and efficiently interact with the DCC server using DMH messages.  The DMH clients may be other extended Tcl/Tk processes, or they may be DMH client processes in other programming environments such as Java, Visual Basic, or C/C++.

The DCC server component requires installation of the Tcl Byte Code Loader extension (package tbcload) in order to use byte code compiled password encoding/decoding logic.  The DCC User Interface requires the tbcload package also.  The DCC User Interface also requires installation of [incr Tcl], [incr Tk], the Iwidgets packages, and the packages in the Hume Component Libraries.  All of these required packages can be installed from the Setup program that is included with the DCC software.

The DCC software is usually installed as a subdirectory, dcc1.0, of the main Tcl/Tk library directory.  The DCC server logic implements the package named HumeDCC, and the installation path can be discovered and the server software made useable by executing "package require HumeDCC", if the software is installed under the Tcl/Tk library directory.  Developers should deploy their custom application software in their own, separate directories.  By keeping the DCC in a separate directory, you can keep it understood by the developers that the DCC files are provided as a product by Hume Integration, and other considerations apply as far as customization.

Hume Integration provides an example DCC server startup batch file.  For Windows, there is also a Start menu program item that starts the DCC server as a standalone process without a persistent database interface.  When an external persistent database is not in use, the DCC saves configuration data and current collected data in files of SQL statements.  When running as a standalone process, the DCC does not save older data that is beyond the time range or group size limits of the configured collection items.  By using an external persistent database, the older data is saved and available using tools that directly interface to the persistent database such as reporting packages.

The following notes are of benefit when deploying a custom system.

The usual DCC server initialization flow is:

  1. Your application processes start and establish DMH communication to each other.  The datahub Tcl application is commonly used to provide the DMH server function.   If you are running your application as NT services, it is common just to start the DMH server process as a service, and then have the server start the other application processes.
  2. If you are using a persistent database, you establish a DMH mailbox for SQL commands directed to the persistent database.  The dmh_SQLsrv Tcl application is commonly used to provide this function.
  3. If you have custom application software, you make it known to your server process by requiring packages or adding your directories to the global auto_path variable.
  4. The DCC server software is made ready for use in the DMH server process by executing "package require HumeDCC".
  5. The DMH server process executes the dc_hub_init procedure.  Optional arguments to the dc_hub_init procedure specify the persistent database mailbox name, and the type of persistent database in use, such as DB2, ORACLE, or SYBASE.  For the most part the DCC logic uses portable SQL but there are some differences between database field types and error replies that the software must accommodate.
  6. The DCC server logic will create and initialize its own persistent database tables if they do not exist, and a functional connection to the persistent database exists.
  7. The DCC server logic will automatically read and initialize its in-memory tables from the saved data in the persistent database.  A subset of collected data is read into memory so that SPC analysis and viewing of the current control data sets, can be performed very rapidly.
Example initialization code may look something like this.  The following lines have been added to the main DMH server initialization:

       # initialize the HUME DCC module to provide SPC analysis
       catch {spc_HUME_init} reply
       mbx put TRACE "HUME SPC init: $reply"

and the spc_HUME_init procedure is defined as:

#
# called by the main datahub during startup
#
proc spc_HUME_init {} {
    global debug
    # initialize the HUME Data Collection Component package
    # for persistent database mailbox DB,database type ORACLE
    set rc [catch {
        package require HumeDCC
        dc_hub_init DB ORACLE
        } text]
    if { $rc != 0 } {
        set debug(hume_spc_status) "init error: $text"
        return "ERROR: Could not initialize HumeDCC package: $text"
        }
    # set up a separate command mailbox to process Tcl commands
    # sent to the HUME SPC package by distributed clients
    mbx whenmsg HUMESPC_RPC mbx_RPC
    set debug(hume_spc_status) "successful initialization"
    return "HUME SPC interface successfully initialized"
    }

The DCC server logic can also be used without the DMH message system or a persistent database.  In this situation, steps 1, 3, 4 and 5 above are executed.  If you are starting with a Tcl/Tk process that has not been extended with the DMH software, you will need to execute "package require dmh" to bring the Hume provided extensions into your Tcl/Tk process.
 

Security Services Integration

An OEM application can choose to utilize Security Services provided by the Hume DCC software.  The Security Services use a proven conceptual model.  A user belongs to any number of security groups.  A security group is associated with privileges.  At login, the user is given the union of all privileges associated with the groups that he is a member of.

The application developers assign privilege names and descriptions.  In the application logic, they enable access to features based on the possession of specific privilege names by the user.

The productization team defines a starting set of groups, and at least a single highly-privileged user.  The application enduser logs on as the privileged user and can create new users, and revise and change the group configurations.

A User Interface should call the dc_user_session procedure at startup to announce its existence.  It should call the dc_user_logout, or the dc_user_session procedure at exit, with appropriate arguments to indicate that it is exiting and that existing logins for the UI are no longer valid.  During the application lifetime, the dc_user_login procedure is used to validate a user login and obtain the user's privileges.  The dc_user_logout procedure is called to update the dc_user_session status table when the user logs out.  Other procedures exist to simplify the configuration management of users, security groups, and privileges.

Data Collection & SPC Component - Server API

The DCC server logic provides the package named HumeDCC which can be used to dynamically discover and use the latest DCC package version, or to discover and use a particular version.  The server API commands are implemented in the usual global Tcl namespace.

In the previous section, the initialization of the server component using the dc_hub_init command was presented.  This command needs only to be called once at application startup.  A data collection application will use the dc_data_insert command as the usual means of adding new data to the system.  To setup the configuration of new items, or to revise the configuration of existing items, involves using more of the API such as the dc_item_read and the dc_item_save procedures.

Hume Integration also provides a working user interface application which can be used to collect data, to configure data collection items, or to view collected data.  This UI application, dc_ui (dc_ui.bat on windows), can serve as a ready-to-run solution or as a useful starting point for the OEM who integrates the configuration, viewing, and management of collected data into the user interface of his product.  The UI software can be discovered and used in a custom application  by executing package require HumeDCCgui, and then executing the command dc_ui.  The dc_ui command assumes that a DMH message system connection already exists to the server, or that the UI is being run in the same process as the DCC server logic.  Note that the same name, dc_ui, is implemented both as an application script and as a Tcl command.

No calls are needed at server shutdown other than the usual Tc/Tk exit.  If a persistent database is in use, the collected data has been posted to the persistent tables as it was collected.  If a persistent database is not in use, during the normal exit, the system calls the dc_save_data command to save the in-memory table images as .tab files.

In a distributed system, the DMH messages system can be used to transport the procedure invocations from the client to the server process.   These invocations are text strings that are formatted as Tcl procedure calls with their arguments.  It is important to format the command invocations properly.   In general, it is better practice to use Tcl list commands that are provided in the various client programming environments than to hardcode the use of brace delimiters for arguments.  The reason for this is the Tcl list commands perform correctly for all possible data arguments, but the hardcoded use of braces is incorrect for data containing unmatched braces or other problematic constructs.  If the commands are sent to a DMH mailbox that is configured to process SQL commands, such as a Datahub mailbox, the message commands should be started with eval as the first command word, and then continue with the usual Tcl command and its arguments.  For example:
 

// Java example
// mailbox HUB processes SQL commands so start with eval
String module_id="APC9";
String item_id="RF_FORWARD";
TclList cmd=new TclList("");
String reply="";
try {
    cmd.lappend("eval"); cmd.lappend("dc_item_read");
    cmd.lappend(module_id); cmd.lappend(item_id);
    cmd.lappend(""); cmd.lappend("1"); cmd.lappend("0");
    reply=dmh.DoXact("HUB", cmd.join());
    } catch (Exception e) { // exception handling... }
 

# Tcl example
# mailbox HUB_RPC processes Tcl commands so eval is not needed
set module_id APC9
set item_id RF_FORWARD
set cmd [list dc_item_read $module_id $item_id {} 1 0]
set reply [mbx_do_xact HUB_RPC $cmd APC9_PID343_REPLY]
if { $reply == "TIMEOUT" } { ;# exception handling.... }

Some of the API commands return data in a format that is convenient to work with in Tcl but less convenient in other environments.  Some thoughts on this issue:


In the procedure descriptions that follow, mandatory data arguments to the procedures are shown as single names.  When an argument is shown as a pair, the argument is optional; it does not need to be supplied to invoke the procedure.  For optional arguments, the left element of the pair is the argument name, and the right element is the default argument value.

dc_alarm_ack - Acknowledge an SPC alarm..
dc_cfg_export - Obtain configuration records in import/export format
dc_config - Read a configured value.
dc_config_write - Save a configured value.
dc_data_insert - Add new collected data.
dc_data_sub_add - Read configuration, data, and subscribe to an item.
dc_data_sub_filtered - Open a filtered, shared dc_data table subscription
dc_data_sub_mod - Modify a filtered, shared dc_data table subscription
dc_data_sub_open - Open a dc_data table subscription.
dc_drop_tables - Erases all of the DCC tables in the persistent database.
dc_hub_init - Initializes the DCC system, must be called at startup.
dc_item_delete - Deletes the configuration records of a data collection item.
dc_item_read - Read the configuration and/or data of an item.
dc_item_save - Write the configuration records of a data collection item to database tables.
dc_item_saveAs - Copy the configuration records of a data collection item to a new name.
dc_items_view - Return a table view of the data collection items.
dc_save_data - Write the in-memory tables to the file system as .tab SQL files.
dc_schema - Obtain SQL statements representing the database table schema.
dc_schema_write - Write SQL statements representing the database table schema to a file.
dc_set_auto_path - Initialize the Tcl auto_path variable.
dc_statistics - Compute control limits and other SPC statistics from collected data.

The following API procedures have been implemented to support Security Services:

dc_encode - Convert a plain text string to an encoded representation.
dc_user_access_check - Validation of a User for a Privilege.
dc_user_all_groups - List Configured Security Groups.
dc_user_all_privileges - List Configured Privileges.
dc_user_all_user_ids - Obtain a list of Configured Users.
dc_user_assoc_groups - Obtain the Security Groups Associated with A User.
dc_user_assoc_privileges - List the Privileges Associated with Specified Groups.
dc_user_delete, _read, _write - Configure Users for Security Services.
dc_user_group_delete, _read, _write - Configure Security Groups.
dc_user_login, _logout - Security Services Login and Logout.
dc_user_password_update - Update a User's Password.
dc_user_privilege_delete, _read, _write - Configure Security Privileges.
dc_user_session - Announce the startup or shutdown of a User Session.
 



dc_alarm_ack - Acknowledge an SPC alarm.

Tcl Command Syntax

dc_alarm_ack module_id item_id user_id comment {annotate 1} {ts_pt_utc {}} {sequence_pt {}} {ts_format 9}
Return Value
0 DC_ALARM_ACK_OK
1 DC_ALARM_NO_CONFIG
2 DC_ALARM_NOT_ACTIVE
3 DC_ALARM_NOT_ANNOTATED
Desciption
The action of acknowledging an SPC alarm clears the alarm status information in the dc_item_use table for an item, and optionally adds an alarm signoff record to the notes field value of the dc_data observation record.  A particular alarmed point can be optionally identified by it's UTC timestamp value, ts_pt_utc, and its sequence number, sequence_pt.  The default behavior is to work with the latest alarmed point for the item.  The user_id value identifies the person who is initiating the signoff action.  The comment argument is text which becomes part of the alarm signoff data.  The optional argument annotate may be supplied as 0 to prevent annotating the dc_data record.  The optional ts_format argument may be used to select the localtime command format for the timestamp of alarm signoff.  Since the alarm signoff timestamp is not a key value, the default value of 9 yields a timestamp for the local timezone.

The return values are literally the two element lists of codes and tokens written above.  The return code of 3 indicates that the alarm condition has been cleared for the item, however the annotation could not be saved.  This situation can occur when the alarmed point is older than the current in-memory data set for the item.  The return value of 1 means that a configuration record for the item identified by module_id and item_id could not be found.  The return value of 2 indicates that the alarm status information for the item does not indicate an unacknowledged alarm.  The return value of 0 is the usual successful case and it implies that the alarmed condition has been cleared for the item, and if point annotation was requested, it was completed successfully.



dc_cfg_export - Obtain configuration records in import/export format

Tcl Command Syntax

dc_cfg_export  export_list  {user_id  {}} {display_id  {}}
Return Values
0 import_code
error_code error_message
Description
The export_list  argument is a Tcl list of pairs.  Each pair is the module_id and item_id of a data collection item.  The identity of the user and the display are optionally specified and added to the exported data.  The return value is a list of two elements, an integer return code, and a second argument.   The return code value of 0 indicates success in which case the second argument is Tcl code that can be executed to import the specified items.  A non-zero return code indicates failure, in which case the second return value element is an error message.  The return code value of 1 indicates that database records for one or more of the items was not found.


dc_config - Read a configured value.

Tcl Command Syntax

dc_config domain name
Return Value
value
Description
The system has a named value dictionary table that can be used to hold configuration values.  The values are stored in the dc_custom table.  This call is simply a convenience function to read a value from this table.  If an entry does not exist, an empty string is returned.  The domain argument is intended to support organizing the stored data in distinct sets so that the same name values can be used without conflict.


dc_config_write - Save a configured value.

Tcl Command Syntax

dc_config_write domain name value
Return Value
SQL_reply
Description
The system has a named value dictionary table that can be used to hold configuration values.  The values are stored in the dc_custom table.  This call is simply a convenience function to write a value into this table.  The reply value is the string that the Tcl SQL command returns for an update or insert command.  The domain argument is intended to support organizing the stored data in distinct sets so that the same name values can be used without conflict.  If you are using a persistent database, a subscription to the in memory table posts the newly inserted or updated data to the persistent database.  All of the data in the dc_custom table is loaded into the in-memory table from the persistent database during initialization of the system.


dc_data_insert - Add new collected data.

Tcl Command Syntax

dc_data_insert module_id  item_id  {prime_num  0} {second_num  -1} {text_value ""} {item_type ""} {ts ""} {sequence ""} {user_id ""} {notes ""} {spc_alarm 0} {spc_excluded 0} {is_partial 0} {discard_flag 0} {equipment_id ""} {track_id ""} {datahub -}
Return Value
return_code ts highest_sequence spc_alarm alarm_text reply
Description
This procedure is the usual means to add new item data to the system.   The procedure supports the addition of single data item records as well as aggregate entries such as pareto lists.   The module_id and item_id arguments identify the data collection item and must be supplied.  The remaining arguments are optional, and the caller supplies only those values that are appropriate for the kind of data collection item (item_type), or appropriate for how the system is being used.  For example, user_id and equipment_id may or may not be used at your site.  Since the arguments are positional,  the caller may need to specify default values shown above to get to the correct position for an argument that he chooses to explicitly provide.   The procedure arguments correspond to the fields in the dc_data table.  The description for this table can be read for details. Also, the data in the table dc_item_type provides an approximate guide to how the use of the arguments prime_num, second_num, and text_value which varies depending on the kind of data collection item.  For most numeric data the caller is expected to supply the X or XBAR value as the prime_num argument.  For MOVING RANGE, cusum, or EWMA data, the caller defaults or provides the value -1 for the second_num argument, and the system computes the desired secondary statistic by analyzing the current point along with the most recent data for the item.   For Standard Deviation, S, or ordinary RANGE values, where the secondary statistic is computed based only on the current input, the caller is expected to supply the computed value as the second_num argument.  For item types using XBAR data, the individual X observations that were averaged to create the XBAR value are usually supplied as a space delimited list in the text_value argument.  The system will then store the raw values in the dc_data table record, and the data will be available for viewing and reporting.  In many cases the procedure calling logic does know the item_type of the data collection item, and it saves the server a lookup if it is supplied as an argument.  The ts argument is a high resolution timestamp for the data collection action, and it defaults to the current localtime 16 result.  The UTC timezone is used to avoid issues with Daylight Saving Time, and multiple clients straddling different local timezones.  The sequence field is usual left to default.  It is used to distinguish and sequence multiple records that are added as a single data collection action, such as a pareto lists, as described in the dc_data schema.

Other than the convenience, an important reason for calling this procedure in the server process instead of manipulating the dc_data table directly from a client, is so a unique timestamp value can be assigned.

The return value is a six element list.  The first element, the return_code, is usually 0 signifying success.  A non-zero return_code indicates an error condition, in which case the reply element will be an error description and the other fields may not be meaningful.  For a successful result, the data collection event timestamp, ts, and the highest sequence number assigned, highest_sequence, are the next elements in the return value.  A non-zero value of the spc_alarm element indicates that the addition of the data caused an SPC alarm.  The spc_alarm element indicates 1 for an alarm on the primary value, 2 for an alarm on the secondary value, or 3 for alarms on both values; and the alarm_text item includes descriptions of the run rule violation(s).

The logic which calls the dc_data_insert procedure knows from the return value whether an SPC alarm occurred as a result of entering data. Other software in the distributed system can learn of SPC alarms in real-time by opening subscriptions to the dc_item_use table. Table subscriptions are explained in the Datahub document.



dc_data_sub_add - Read item configuration, current data, and open a shared subscription for future values.

Tcl Command Syntax

dc_data_sub_open clientID destbox new_id_pair combined_pair_list  {reply_format 0} {array_name {}} {extra_clause {is_partial=0}}
Return Value
returns the dc_item_read result
see dc_data_sub_filtered regarding future subscription notifications
Description
 This procedure is called to return the configuration and existing data for a DCC item, and to revise a shared, filtered data subscription to include the new item along with other subscribed items.  You want to do this in a single procedure like this one so there is no chance for a data change to be unobserved between reading the current data and opening a subscription.

The subscription is opened using the dc_data_sub_filtered procedure, so the subscription feed has the DCC analysis changes merged into the insert notifications.  Also, the same hub_safe_sub_set call is made to register cleanup code for the subscription if the client should go away ungracefully.

The clientID argument is the DMH server's identification for the client connection which is obtained by executing mbx clientID.  The value is used in the subscription name opened for the client so that the DCC cleanup logic tied to mh_app_lostclient will find the subscription and close it if the client process disconnects.

The destbox argument is the DMH mailbox name that the caller is using to receive subscription notifications.  The value should be unique for each client instance, so it is often based on the client's display and/or process ID.

The new_id_pair argument is a two element Tcl list containing the module_id and item_id whose current data and configuration are to be read and returned as the immediate procedure call result.  The reply_format and array_name arguments are used when invoking the dc_item_read procedure and they control the format of the result.

The combined_pair_list argument is a Tcl list of two element lists.  Each pair contains the module_id and item_id of a DCC item that is to be included in a shared subscription to the dc_data table.  The new_id_pair value should be listed in the combined list if it is to be included in the shared subscription.

The extra_clause argument is used to specify selection criteria for the subscription in addition to the implied criteria of the combined_pair_list argument.



dc_data_sub_mod - Modify or close a shared, filtered dc_data subscription.

Tcl Command Syntax

dc_data_sub_mod clientID destbox combined_pair_list  {extra_clause {is_partial=0}}
Return Value
None.
Description
 This procedure is called to revise a shared, filtered data subscription.  If the combined_pair_list argument value is an empty list, the shared subscription is closed.  This procedure is usually called to remove DCC items from the shared subscription list after the user has dismissed plots or similar objects and no longer has an interest in future data of the item.


dc_data_sub_filtered - Open a filtered subscription to table dc_data.

Tcl Command Syntax

dc_data_sub_filtered clientID  destbox {where_clause {where is_partial=0}} {max_msgs 100}
Return Value
returns the result of executing the SQL open command

future subscription notifications are formatted as the Datahub Tcl Reply Format

Description
The DCC analysis subscription may change the data in table dc_data as it is entered.  If a dc_data subscription is used for a live, updating plot, it is more efficient to see a subscription feed that somehow combines the initial insert with the possible analysis updates, and then just sends the final result.  Here is a procedure, that opens a subscription that just sends the post analysis data.  The data feed is ordinarily within a couple of milliseconds of the original insert.  The original ordering of the inserts is faithfully preserved.  The technique works well, and is a neat trick to deliver a simpler subscription feed.

The clientID argument is the DMH server's identification for the client connection which is obtained by executing mbx clientID.  The value is used in the subscription name opened for the client so that the DCC cleanup logic tied to mh_app_lostclient will find the subscription and close it if the client process disconnects.

The destbox argument is the DMH mailbox name that the caller is using to receive subscription notifications.  The value should be unique for each client instance, so it is often based on the client's display and/or process ID.

The where_clause argument is used to specify the selection criteria for the subscription.

The max_msgs argument is used with the hub_safe_sub_set call as a criteria for triggering cleanup logic even if the client process remains connected but stops reading subscription notification messages.

You can keep calling this procedure, changing the where_clause as the viewer changes which plots are displayed.  You really should not subscribe to the whole dc_data table unless you are interested in all of it.

Two higher level procedures, dc_data_sub_add and dc_data_sub_mod are available to obtain the existing configuration and data for newly displayed charts,  and to call this procedure for a combined feed of future changes.



dc_data_sub_open - Open a subscription to table dc_data.

Tcl Command Syntax

dc_data_sub_open open_statement
Return Value
returns [SQL $open_statement]
Description
The DCC analysis subscription changes the data that is inserted into the dc_data table.  These changes may be seen out of order by a subscription unless the DCC analysis subscription is the last one on the dc_data table.  Accordingly, this procedure should be used to open subscriptions to the dc_data table.  It insures that the DCC analysis subscription is the last one.


dc_drop_tables - Erases all of the DCC tables in the persistent database.

Tcl Command Syntax

dc_drop_tables
Return Value
None
Description
Invoking this command deletes all of the DCC tables in the persistent database using the SQL drop command.  All configuration and collected data is erased.  This command is not usually exposed to the system enduser by the OEM application.

dc_encode - Convert a plain text string to an encoded representation.

Tcl Command Syntax

dc_encode string
Java Syntax
import com.hume.DMH.*;
String DCC.encode(String password);
Return Value
encoded_string
Description
The dc_encode procedure is used in a User Session to convert an entered password into a coded representation that is passed through the network for the dc_user_login call.  Hume is supplying the equivalent encoding logic for other programming language environments.  The Java equivalent is the static class method, Dcc.encode, which is part of the com.hume.DMH package.


dc_group_insert - Save the entered data of a Group.

Tcl Command Syntax

dc_group_insert group_module_id group_item_id dc_data_inserts \
 {user_id {}} {equipment_id {}} {track_id {}} {datahub -}
Return Value
0 ts alarm_list
return_code module_id item_id error_reply
Description
A dc_data  table record is made for the group, and the ts timestamp is obtained.   The ts value is substituted into the dc_data_insert statements for the group member data entries.  Similarly, the provided datahub value or its default value are substituted in the dc_data_insert statements. You can propagate your own values of user_id, equipment_id, and track_id into the dc_data_insert statements since you might be varying these values by item.  If the dc_data_insert statements do not provide values for these items,  the values from the dc_group_insert call are used.

If an error occurs when processing any of the dc_data_insert statements,  partially entered data, if any, will be deleted.

The usual successful return value is a 3 element list, 0 timestamp alarm_list, where the first element is the success return code of 0, the second element is the entry timestamp, and the third element is a list of SPC alarm records, possibly empty, that were obtained by executing the dc_data_insert statements.  Each alarm record is structured as a four part list,  module_id item_id spc_alarm alarm_text.

An error return is based on a failed dc_data_insert call.  A non-zero return code is provided with the module_id, item_id, and error_reply of the underlying dc_data_insert call.



dc_group_save - Save the configuration of a Group.

Tcl Command Syntax

dc_group_save group_module_id group_item_id description options members_list
Return Value
0 DC_GC_SAVED_OK
4 DC_GC_MEMBER_UNKNOWN module_id item_id
5 DC_GC_MEMBER_DUPLICATE module_id item_id
6 DC_GC_MEMBER_BAD sql_stmt sql_reply
dc_item_save_error_reply
Description
This procedure is used to save the configuration of a data entry Group.  The members_list argument is one or more elements structured as {module_id item_id item_type is_optional calc_type calc_context calculation}; ie., rows for the dc_group_config table.   The options argument is saved in the prime_runrules field in the dc_item table row that is created for the group.  The options argument is a string of the form "-comment b -compact b" where the b value is 1 or 0.   This call is used by the Hume DCC UI to save a group configuration.


dc_hub_init - Initializes the DCC system, must be called at startup.

Tcl Command Syntax

dc_hub_init {MB_DB DB} {DBtype SYBASE}
Return Value
"Data Collection Component Initialized Successfully." is returned for the usual successful case.

The procedure will return a Tcl error if the system cannot be initialized.

Description
See the description of server initialization given above.


dc_input_config - Obtain configuration data needed to conduct Data Entry.

Tcl Command Syntax

dc_input_config module_id item_id {ts {}}
Return Value
return_code columns rows group_entry_options data_cols data_rows alarm_inhibit
2 DC_DE_GROUP_INPUT_ONLY
3 DC_DE_NO_CONFIG $module_id $item_id
4 DC_DE_NO_CONFIG $member_module_id $member_item_id

columns := {sequence item_type module_id item_id is_optional calc_type calc_context calculation item_type description units formats subgroup_ct input_type save_options prime_runrules high_screen has_high_screen high_spec has_high_spec low_spec has_low_spec low_screen has_low_screen input_is_inhibited spc_alarm selection}

rows := one or more rows of values corresponding to the columns

data_cols := {dc_data table columns}

data_rows := {dc_data table rows where ts='$ts'}

alarm_inhibit := the value of the SPC, alarm_inhibits_entry dc_custom table record

Description
This procedure returns the configuration data needed to perform data entry or data editing.  The result is essentially a table join connecting tables dc_group_config, dc_item, dc_item_use, and possibly dc_list_text.

For editing usage, the timestamp of the existing entry, ts, is provided, and the reply data contains the existing data records from the dc_data table.

If the item specified by module_id and item_id is not a Group, the procedure returns return code 2 if the item is configured to only allow group data collection (dc_item.group_input_only is 1).  If the item allows data collection outside of a group, the procedure returns the data of a Group definition that includes the item as its only member.

For items with selection lists, there is a virtual column selection which is a list of sorted selection choices.



dc_item_delete - Deletes the configuration records of a data collection item.

Tcl Command Syntax

dc_item_delete module_id  item_id {delete_data 1}
Return Value
None
Description
This procedure deletes configuration records for a data collection item.  The default value of the delete_data argument also causes collected data for the item to be deleted.  If a persistent database is being used, and the data is being deleted, only data records in the dc_data table whose datahub field values match the server's value, or whose datahub field values are blank, are deleted.


dc_item_read - Read the configuration and/or data of an item.

Tcl Command Syntax

dc_item_read module_id item_id  {config_flag 1} {data_flag 1} {ts ""} {reply_format 0} {array ""}
Return Values
0 NameValueList
0 Tcl_statements
error_code error_message
Description
The procedure is an efficient and convenient means of obtaining the configuration and/or current data of a collection item in a single invocation.  The module_id and item_id arguments specify the data collection of interest.  The config_flag is used as a boolean, the value 1 specifies that all configuration data is desired.
The data_flag is used as an integer as follows:
 
0   No observation data is returned.
1   All current data is returned.
2   All data for the timestamp value ts is returned.
-N   A negative integer value obtains the records for the newest preceding N timestamp values relative to the value of the timestamp argument.  If the timestamp argument is defaulted, the latest N values relative to the current time are obtained.
The reply_format argument is used as an integer to choose the desired successful reply format.  The default value of 0 specifies that a list of alternating names and values, NameValueList,  is returned in addition to the 0 success return code.  The NameValueList is typically used with the array set Tcl command to populate an array, or with the foreach command to instantiate scalar variables.  The returned items in the list are described in the table below.

If the reply_format value of 1 is specified,  the array argument is used to specify an array name which is used to build the return value Tcl_statements.  The Tcl_statements element in the return value is a list of Tcl assignment statements.  Each assignment statement is of the form "set ${array}($name) $value".  The Tcl_statements can be executed in a Tcl interpreter to instantiate an array.  The items returned in this array include the following:

NameValueList Item Name   or
Array Subscript Name
When Present Element Value(s)
dc_item_cols if configuration requested A list of the table column names for table dc_item
each table column from the dc_item table record if configuration requested Each corresponding table column field value
dc_item_use_cols if configuration requested The value is a list of column names from the dc_item_use table including the values {datahub ts_groupstart spc_alarm ts_alarm
 alarm_user_id alarm_text input_is_inhibited}
each table column named in the dc_item_use_cols list if configuration requested Each corresponding column value.  If a row in the dc_item_use table does not exist for the item, default values are supplied.
dc_group_config_cols if configuration requested and only for item_type G The value is the column names for the dc_group_config table.  If group data is requested, the column names includes the additional column name, formats
dc_group_config if configuration requested and only for item_type G The value is a list of data rows from the dc_group_config table.  In each row the values correspond to the names in the  dc_group_config_cols array element.  If group data is requested, each row contains an additional value, formats, whose value is found for each group member by joining to the dc_item table.  The addition of this element is useful so that the application knows the configured presentation format of the group data.
dc_list_text_cols if configuration requested and only for item_types DI, DL, LL, LI, PL, and PI (items with list selections) The value is the column names for the dc_list_text table.  For dynamically computed selection lists, types DI and DL, the lists are computed at the server as if they exist as rows in the dc_list_text table.
dc_list_text if configuration requested and only for item_types DI, DL, LL, LI, PL, and PI (items with list selections) The value is a list of data rows from the dc_list_text table.  In each row, the values correspond to the names in the dc_list_text_cols array element.
dc_data_cols if data requested The value is a list of the column names from the dc_data table
dc_data if data requested The value is a list of data rows from the dc_data table.  In each row, the values correspond to the names in the dc_data_cols array element.
dc_recipient_cols if configuration requested The value is a list of the column names from the dc_recipient table
dc_recipient if configuration requested The value is a single row from the dc_recipient table for the configured alarm_recipient_id.  A 0 value for the alarm_recipient_id indicates a value has not been configured, and the recipient data is not valid.



dc_item_save - Write the configuration records of a data collection item to database tables.

Tcl Command Syntax

dc_item_save NameValueList
Return Values
0 {}
return_code error_message
Description
This procedure is used to create or update the configuration data of a data collection item.  The procedure takes a single argument, NameValueList, which is either formatted as a Tcl list of name, value pairs or as a Tcl list of alternating names and values.  Here we are experimenting with formats that are easier to use than a long positional argument list,  simpler than the style of  arguments passed to Tcl/Tk window objects, and somewhat neutral to the choice of programming language.  The name value items can be in any order.  One of the more significant input parameters is mode which sets the expectation as to whether the item is being newly created, revised, or for the case of import, either created or revised.  The remainder of the names that are possible in the NameValueList list include the fieldnames in table dc_item specified in lowercase.  The procedure logic validates the consistency of the argument data, and supplies appropriate default values, so it is used by most applications rather than writing directly to the dc_item table.  The schema descriptions for the tables dc_item, dc_item_use and dc_list_text can be reviewed for more detail on the arguments.  An example command invocation looks something like the following; braces or quote delimiters can be used per the conventions of Tcl:

dc_item_save {{mode insert} {module_id test} {item_id indxtest} {description "only a test"}}

The return value is always a two element list.  The first element, the return_code, is 0 for success in which case the message element is empty.  A non-zero return code is accompanied by an error message.
 
Parameter Name Default Value  Description
module_id   Must be supplied to identify the item.  See the dc_item schema.
item_id   Must be supplied to identify the item.  See the dc_item schema.
mode   Must be supplied.  Allowed values are insert, update, or import.   The value of mode sets the expectation as to whether the item is being newly created, revised, or for the case of import, either created or revised.  For insert mode an error is returned if the item already exists.  For update mode, an error is returned if the item does not exist.  With import mode, the item is created if it does not exist, and updated if it does exist.
alarm_recipient_id 0 Optionally used to identify a recipient or list of recipients for SPC alarm notifications.  See the dc_recipient schema.
description {} See the dc_item schema.
discard_flag 0 See the dc_item schema.
display_id   A value to help identify the workstation or display of the agent who initiated the configuration change. This value can be passed to support future audit and event logging.
ewma_weight {} See the dc_item schema.
formats {%9.2f  %9.2f  %s} See the dc_item schema.
group_ct 25 See the dc_item schema.
has_high_screen 0 See the dc_item schema.
high_screen 100 See the dc_item schema.
has_high_spec 0 See the dc_item schema.
high_spec 0 See the dc_item schema.
input_type 0 See the dc_item schema..
item_type XM See the dc_item schema.
limit_prime_high 100 See the dc_item schema.
limit_prime_low 70 See the dc_item schema.
limit_second_high 15 See the dc_item schema.
limit_second_baseline 7.5 See the dc_item schema.
limit_second_low 0 See the dc_item schema.
list_selection {} For the item types that possess predefined selection lists, this value is passed as an ordered Tcl list of the selectable values.  The values are stored in the dc_list_text table.
has_low_spec 0 See the dc_item schema.
low_spec 0 See the dc_item schema.
prime_runrules {} See the dc_item schema.
save_options 0 See the dc_item schema.
second_runrules {} See the dc_item schema.
spc_is_enabled 0 See the dc_item schema.
ts_groupstart {} See the dc_item_use schema.  A non-blank value of this item causes an insert or update to the dc_item_use table.  In the case of an insert, the calling logic can also specify a value for the dc_item_use datahub field which ordinarily gets the value of the global array item dc_hub(datahub).
units {} See the dc_item schema.
user_id {} A value to help identify the agent who initiated the configuration change.  This value can be passed to support future audit and event logging.



dc_item_saveAs - Copy the configuration records of a data collection item to a new name.

Tcl Command Syntax

dc_item_saveAs oldmodule_id olditem_id module_id item_id
Return Value
return_code message


Description

This procedure will validate the destination names, validate that the destination item does not already exist, and then copy the source item definition to the destination.  It does not delete the source item.  The return value is always a two element list.  The first element, the return_code, is 0 for success in which case the message element is empty.  A non-zero return code is accompanied by an error message.


dc_items_view - Return a table view of the data collection items.

Tcl Command Syntax

dc_items_view
Return Value
An SQL selection result.
Description
The return value of this procedure is formatted as an SQL selection result that joins the data collection items, their type names, and their current alarm status.  The procedure obtains data from three tables to return the view.  The column names returned in the result are ModuleName, ItemName, DataType, AlarmStatus, and Description.  The column names are the result of calling the msgcat::mc command and mapping SQL table column names to names that have been installed for display to the user.


dc_save_data - Write the in-memory tables to the file system as .tab SQL files.

Tcl Command Syntax

dc_save_data
Return Value
None
Description
If you are not using a persistent database, this procedure gets called at exit to save your changed data.  Only relying on this mechanism to save your changes is not recommended.   Depending on your requirements, running from the memory image may be feasible if you use an Uninterruptible Power Supply, periodically call dc_save_data, and in between calls, have subscriptions that write and flush SQL statements representing data changes to the file system.


dc_schema - Obtain SQL statements representing the database table schema.

Tcl Command Syntax

dc_schema {DBtype hub}
Return Value
a Tcl list of {tablename create_statement} pairs
Description
This procedure is used by the DCC to obtain SQL table create statements for creating the table schema in the Tcl/Tk process or in the persistent database.  The DBtype argument defaults to hub which returns statements for the Tcl/Tk datahub.  Other supported values include db2, oracle, or sybase.  The DBtype argument can be supplied in either alphabetic case.  The database types vary in the field types that need to be declared to support numeric data or timestamp data.  Also the use of the keywords NOT NULL varies between the hub and its persistent counterparts.  There is a feature to support customization of the table schema.  The dc_schema procedure attempts to execute a user-written procedure dc_schema_custom, calling it with the standard schema as an argument.  The dc_schema_custom procedure is able to supply a revised schema as its return value.  This feature can be used to add more columns to the DCC tables.


dc_schema_add_cols - Add columns to DCC Tables.

Tcl Command Syntax

dc_schema_add_cols schema_list table added_cols
Return Value
a revised Tcl list of {tablename create_statement} pairs
Description
A customer who has written his own dc_schema_custom procedure is able to supply a revised schema for the DCC system.  The procedure, dc_schema_add_cols has been provided for use as a convenient way to revise the schema_list argument for the purpose of adding columns to tables.  The added_cols argument is the text to be inserted into the table SQL create statement, just before the final primary key ( ) phrase.  Here are example invocations:

set schema [dc_schema_add_cols $schema dc_item "max_days_age int"]
set schema [dc_schema_add_cols $schema dc_data "chamber int, run_id varchar(32)"]
return $schema



dc_schema_custom - Optionally, Customize the DCC Table Schema.

Tcl Command Syntax

dc_schema_custom schema_list DBtype
Return Value
a Tcl list of {tablename create_statement} pairs
Description
This is a procedure that is optionally written by the customer, and installed in his own application directory prior to initializing the DCC.   See the file dc_schema_custom.tcl for a model implementation and additional comments.  The dc_schema procedure attempts to execute dc_schema_custom, calling it with the standard schema as an argument.  The dc_schema_custom procedure is able to supply a revised schema as its return value.  The procedure, dc_schema_add_cols has been provided as a convenient way to add table columns.


dc_schema_write - Write SQL statements representing the database table schema to a file.

Tcl Command Syntax

dc_schema_write {filename dc_schema.sql} {DBtype hub}
Return Value
None
Description
This procedure calls dc_schema and writes the result to a file with each create statement being written as a new line of text.


dc_secs_discovery - Discover Event Report Configurations.

Tcl Command Syntax

dc_secs_discovery {spname *}
Return Value
0 DC_ASYNC_RESULT
1 DC_NO_SECS_HOSTS
Description
This command will refresh the DCC tables dc_secs_event and dc_secs_var_map with the current status of event reports available for a given SECS Host interface, spname.  The default value of spname causes the event report data to be refreshed for all online SECS Host interfaces.


dc_secs_ei_hosts - Discover SECS Host Interfaces.

Tcl Command Syntax

dc_secs_ei_hosts
Return Value
a Tcl list of spname (SECS Host interface names) values
Description
Use this command to retrieve a list of possible SECS Host interface names that are discovered in the DMH group based on the DMH mailbox names that are in use.  The list may be empty.


dc_secs_event_report_save - Validate and Save a configuration to collect an Event Report

Tcl Command Syntax

dc_secs_event_report_save spname CEID description varID_dcc_items \
   {auto_setup 1} {datahub -}
Return Value
0 DC_EI_ER_SAVED CEID_is_configured CEID_is_enabled CEID_is_collected \
  varID_thought_not_configured_list
1 DC_EI_ER_BAD_SPNAME
2 DC_EI_ER_BAD_CEID
8 DC_EI_ER_BAD_DCC_ITEM module_id item_id
9 DC_EI_DB_ERROR
10 DC_EI_DB_ERROR
Description
This command is used to validate and save the DCC collection configuration data for an event report.  The varID_dcc_items argument is structured as a list of one or more varID module_id item_id is_collected sequences.  So it is a single list where the number of elements is a multiple of 4.  The is_collected item in the varID_dcc_items list is used as a boolean flag to state whether collection of a particular variable value is desired.  If the is_collected value is true for any variable in the report, then the is_collected value for the event record in the dc_secs_event table will be saved as true.  The varID values are the low-level SECS variable identifiers; typically, cryptic integer codes.  The auto_setup argument is used as a boolean.  If the value is true,  and the saving action causes an event report to be collected when it previously was not being collected, the dc_secs_event_report_setup procedure call is made for the particular SECS Host interface spname so that the interface will send the event report messages to the DCC.  The error return value of 8 occurs when the configuration indicates saving a report variable is desired but the specified DCC item for the variable does not exist.  The successful return value of 0 also supplies status and warning information.  There are three values to indicate whether the event is believed to be configured at the Host interface, and whether the event report is believed to be enabled for reporting at the equipment, and whether the event is now being collected.  The possibly empty list varID_thought_not_configured_list contains the varID values for report items that are believed to not be part of the configuration of the event report as known to the DCC.  The DCC does not synchronize with the SECS Host in performing this procedure so the DCC's knowledge may not be current and accurate.


dc_secs_event_report_setup - Open or revise subscriptions to collect SECS Event Reports.

Tcl Command Syntax

dc_secs_event_report_setup {spname *}
Return Value
0 DC_ASYNC_RESULT
Description
This command should be called after the configuration controlling the collection of event reports is updated.  The command causes table subscriptions to be opened or revised so that only the collected Event Reports are sent to the DCC server.


dc_secs_insert - Add SECS Trace and Event Report Data.

Tcl Command Syntax

dc_secs_insert spname CEID name_value_pairs {ts {}}
Return Value
list [list varID module_id item_id dc_data_insert_reply]*
Description
This command is used by the Event and Trace report collection logic.  It basically takes the varID-value pairs in a report, and adds them to their respective DCC items.  The mapping is configured in the dc_secs_var_map table.  The procedure returns a synchronous result indicating the results of calling the dc_data_insert procedure.  If collection of a varID-value is not configured, the dc_data_reply value for that item is an empty string.  The execution of this procedure also causes status fields in the dc_secs_event table to be updated.


dc_secs_trace_action - Start, Restart, or Stop Trace Report Collection.

Tcl Command Syntax

dc_secs_trace_action spname trace_number start
Return Value
0 DC_ASYNC_RESULT
1 DC_SECS_TRACE_NO_CONFIG
2 DC_SECS_TRACE_NO_VARS
Description
This command is used to start or stop trace report collection.  Set the start argument to 1 to start or restart a trace.  Set the start argument to 0 to stop it.  The procedure causes a DMH message to be sent to the spname SECS host interface.  The caller will typically have a subscription to the dc_secs_trace table, and see the success or failure of the command as signaled by the asynchronous updates to the table.


dc_secs_trace_report_save - Validate and Save A Trace Report Configuration.

Tcl Command Syntax

dc_secs_trace_report_save spname trace_number description period_secs \
  total_samples  report_sample_ct track_id varID_DCC_items {datahub -}
Return Value
0 DC_EI_TR_SAVED
1 DC_EI_TR_BAD_SPNAME
2 DC_EI_TR_BAD_TRACE_NUMBER
3 DC_EI_TR_BAD_TOTAL_SAMPLES
4 DC_EI_TR_BAD_SAMPLE_CT
5 DC_EI_TR_BAD_PERIOD_SECS
6 DC_EI_TR_BAD_SAMPLING
7 DC_EI_TR_NO_VARS
8 DC_EI_TR_BAD_DCC_ITEM
9 DC_EI_DB_ERROR
10 DC_EI_DB_ERROR
Description
This command is used to validate and save the configuration data of a trace report.  The varID_DCC_items argument is structured as a list of one or more varID module_id item_id sequences.  So it is a single list where the number of elements is a multiple of 3.  The varID values are the low-level SECS variable identifiers; typically, cryptic integer codes.


dc_set_auto_path - Initialize the Tcl auto_path variable.

Tcl Command Syntax

dc_set_auto_path
Return Value
None
Description
This procedure is called by both the DCC server and the example DCC UI to set the Tcl global variable auto_path which is a  sequenced list of file system directories that are searched for Tcl procedures.  The logic insures that if the peer directory named custom exists, and it has a tclIndex file, then it is added to the auto_path before the usual dcc directory so that the user's custom procedure versions are found and used before the Hume distributed versions.


dc_statistics - Compute control limits and other SPC statistics from collected data.

Tcl Command Syntax

dc_statistics array_name {global 1}
Return Value
The return values are added elements in the array supplied by the caller.
Description
This procedure is used to calculate the standard deviation, Cp, Cpk, and control limit values for SPC data items.  The caller passes in the name of an array which has been populated with configuration and collected data for the item using the dc_item_read procedure.  By default, the array is assumed to be a global data item.  However, the optional global procedure argument can be set to 0 to indicate that the array should be accessed in the caller's stack frame using upvar.  The calculations use the sum of square differences from the mean techniques to avoid round-off error for large or ill-conditioned data sets.  Note that the dc_item_read procedure must be evaluated at the server where the data is accessible, but the dc_analysis procedure can be executed by any Tcl process.

The following array entries are added:
 
Array Subscript Description
!analysis A substituted text message of the form:

"Calculated SPC values are based on $count points dating from
 $ts_min to $ts_max.

 Primary value average=$prime_avg
 Primary value standard deviation=$prime_sigma"

Or of the form:

"Insufficient data exists for SPC calculations."

!count The integer number of points included in the SPC analysis.  The value may be less than the total available because of the dc_data field spc_excluded values.
!cp When both an upper and lower specification limit are defined, Cp is the ratio of the distance between the specification limits, to the calculated six sigma distance separating the calculated control limits. 
!cpk The Cpk statistic is the smaller value based on either the upper or lower specification limit, taking the ratio of:
  (spec limit -  centerline)/(control limit - centerline)
It is a measure of process control that is also sensitive to the possibility that the control limits are not necessarily centered in the specification limits.
!prime_avg An empty string if !count < 1.  If !count > 1, the average of the included primary number values.  This is usually presented to the user as the centerline of the computed control limits for the primary variable control chart.
!prime_sigma An empty string if !count < 2.  If !count > 2, the standard deviation of the included primary values.  For most SPC charts, the calculated chart limits are +/- 3 sigma around the primary average.  For P, IP, U, and IU item_types, the primary limits are for the actual average sample size found in the data.
!prime_lcl The calculated lower control limit for the primary value.  An empty string if not applicable or available. 
!prime_ucl The calculated upper control limit for the primary value.  An empty string if not applicable or available. 
!second_avg The average value of the included dc_data.second_num values.  For P, IP, U, and IU data, it is the average sample size.  For other item types, the value has an intermediate role in computing control limits, and may not be exposed to the user.
!second_lcl The secondary statistic lower control limit such as the lower limit on a RANGE or S chart. Second chart control limits are applicable to item_types XM, XR, XS, XC, and XE.   The value is an empty string if not applicable or available.  For a cusum item, this value is half the total V mask angle expressed in degrees.
!second_bsl An empty string if not applicable or available.  The computed baseline value for the second chart control limits.  Also the computed target for EWMA or cusum charts (the same as !prime_avg).
!second_ucl An empty string if not applicable or available. The secondary statistic upper control limit such as the upper limit on a RANGE or S chart.  For a cusum item, this value is the "d" value and it is calculated assuming alpha risk is .0027, beta risk is 0.01, and a cusum half angle of 14.0 degrees.
!ts_min An empty string if not available. The earliest timestamp of the data included for analysis.
!ts_max An empty string if not available. The latest timestamp of the data included for analysis.



dc_user_access_check - Validation of a User for a Privilege.

Tcl Command Syntax

dc_user_access_check user_id encoded_password privilege_id
Return Value
0 success_message
failure_code failure_message
Description
This command does the direct validation of a user and password for a particular privilege without performing a login.  It can be useful for features such as a Supervisor sign-off that may occur during another User's login session.  The encoded_password is prepared using the dc_encode command or its equivalent for other programming languages.


dc_user_all_groups - List Configured Security Groups.

Tcl Command Syntax

dc_user_all_groups {want_descriptions 0}
Return Value
grouplist
Description
This call is a convenient means of querying the dc_user_grouptable.  The result is a list sorted in dictionary order, where each element in the list is either the group_id by itself, or the group_id paired with the group description.


dc_user_all_privileges - List Configured Privileges.

Tcl Command Syntax

dc_user_all_privileges {want_descriptions 0}
Return Value
privilegelist
Description
This call is a convenient means of querying the dc_user_privilege table.  The result is a list sorted in dictionary order, where each element in the list is either the privilege_id by itself, or the privilege_id paired with the privilege description.


dc_user_all_user_ids - Obtain a list of Configured Users.

Tcl Command Syntax

dc_user_all_user_ids {want_names 0}
Return Value
userlist
Description
This call is a convenient means of querying the dc_user table.  The result is a list sorted in dictionary order, where each element in the list is either the user_id by itself, or the user_id paired with the user_name field value.


dc_user_assoc_groups - Obtain the Security Groups Associated with A User.

Tcl Command Syntax

dc_user_assoc_groups user_id
Return Value
grouplist
Description
This call is a convenient means of querying the dc_user_grp_assoc table.  The result is a list of associated group_ids sorted in dictionary order.  The result of the call can be used as an input argument to dc_user_assoc_privileges, described next.


dc_user_assoc_privileges - List the Privileges Associated with Specified Groups.

Tcl Command Syntax

dc_user_assoc_privileges grouplist
Return Value
privilegelist
Description
This call is a convenient means of querying the dc_user_priv_assoc table.  The result is a list of associated privilege_ids sorted in dictionary order, with duplicates removed.


dc_user_delete, _read, _write - Configure Users for Security Services.

Tcl Command Syntax

dc_user_delete user_id
dc_user_read user_id {groups_or_privs groups}
dc_user_write user_id user_name timeout_secs encoded_password grouplist editor {clientdata {}}
Delete Return Values
0 SS_user_delete_ok
1 SS_unknown_user_id
Read Return Values
user_id user_name timeout_secs encoded_password chosenlist clientdata
user_id {} {} {} {} {}
Write Return Values
0 SS_user_write_ok
1 improper_user_id_message
2 SS_unknown_group
3 SS_invalid_password
4 SS_short_password
5 SS_unexpected_SQL_error
6 SS_invalid_timeout
Description
These commands provide for configuring users.  The second form of the Read Return Value is returned for improper or unknown user_id values.   The dc_user_read method has an optional argument groups_or_privs which defaults to the value groups.  The value groups is used to obtain a list of group_id's representing the Security Groups that the user is a member of.  The value privs is used to obtain the list of privilege_id's implied by the user's Security Group memberships.  In this case, the return value of the dc_user_read method corresponds to the user_data structure returned by dc_user_login method.  The Write result of return code 4 is controlled by the business rule value MinPasswordLength.


dc_user_group_delete, _read, _write - Configure Security Groups.

Tcl Command Syntax

dc_user_group_delete group_id
dc_user_group_read group_id
dc_user_group_write group_id description privilege_ids
Delete Return Values
0 SS_group_delete_ok
1 SS_unknown_group
Read Return Value
group_id description privilegelist
Write Return Values
0 SS_group_write_ok
1 improper_group_id_message
2 SS_unknown_privilege
5 SS_unexpected_SQL_error
6 SS_no_description
Description
These commands provide for configuring Security Groups.  If the Read method returns a blank description it signifies that the group was not found.


dc_user_login, _logout - Security Services Login and Logout.

Tcl Command Syntax

dc_user_login display user_id encoded_password clientID
dc_user_logout display user_id clientID atexit
Login Return Values
0 user_data ts_login
user_data := user_id user_name timeout_secs encoded_password privileges clientdata
1 SS_unknown_user_id
2 SS_incorrect_password
3 SS_display_missing
8 SS_max_login_exceeded
9 SS_password_expired
5 SS_unexpected_SQL_error
Logout Return Value
0 SS_logout
1 SS_logout_who
Description
The dc_user_login command validates the user's password and if successful, returns a list of three items, the 0 return code, a user_data structure, and the UTC timestamp of login.  The user_data structure is a Tcl list of the elements as indicated above.  Within the user_data structure the privileges item is a list of all of the privileges granted to the user through his membership in any number of Security Groups.  The clientID argument is the DMH Server's identification for the DMH client.  This value is obtained by using the mbx clientID command or the equivalent method using the DMH client libraries.  By using the clientID value, the DCC can use the mh_app_lostclient DMH feature to know when a User Session has disconnected.  The encoded_password is prepared using the dc_encode command or its equivalent for other programming languages.  If the dc_user_login command is not successful, a non-zero return code and substituted text message is returned.  The various SS_* tokens above are the particular message ID values used with the msgcat::mc call to prepare the reply text.  There are three business rule values that affect login results, PasswordExpirationDays, MaxLoginPerUser, and MaxUsersPerDisplay.  As a side effect of the login call, the row in the dc_user_session status table representing the User Session is updated.

The dc_user_logout command has more flexibility than is first apparent.  If the user_id is supplied as an empty string, the logout intention is applied to any existing login for a specified display.  If no user_id and no display are specified, the logout intention is applied to any login using the same DMH connection identified by the clientID value.  If the atexit argument is set to 1, the call also signifies that the application is exiting; and the logic initiates the same cleanup as if the dc_user_session call was made with the atexit flag set to 1.  A User Interface will typically send the logout command asynchronously to the DCC server at exit, before the DMH connection is closed.



dc_user_password_update - Update a User's Password.

Tcl Command Syntax

dc_user_password_update user_id encoded_new encoded_old
Return Values
0 SS_password_update_ok
3 SS_invalid_password
4 SS_short_password
5 SS_password_update_failure
7 SS_same_password
Description
If the encoded form of the new password implies that the new password length is less than the business rule value MinPasswordLength, the return code value of 4 is obtained.  If the encoded form of the old password is not appropriate for the user_id record, or if the user_id record does not exist, the return code value is 5.  If the new password is the same as the old password, the return code is 7.  If the call succeeds, the ts_lastedit field value in the dc_user table is updated to the current time (UTC), and the lasteditor field value is set to the user_id value.  So the ts_lastedit field can be used to support password expiration logic.  The encoded passwords are prepared using the dc_encode command or its equivalent for other programming languages.


dc_user_privilege_delete, _read, _write - Configure Security Privileges.

Tcl Command Syntax

dc_user_privilege_delete privilege_id
dc_user_privilege_read privilege_id
dc_user_privilege_write privilege_id description
Delete Return Values
0 SS_privilege_delete_ok
1 SS_unknown_privilege
Read Return Value
privilege_id description
Write Return Values
0 SS_privilege_write_ok
1 improper_privilege_id_message
5 SS_unexpected_SQL_error
6 SS_no_description
Description
These commands provide for configuring security privileges.  When a privilege is defined, the logic insures that a description is provided.  When the Read method returns an empty string for a description, it implies that the specified privilege_id was not found in the dc_user_privilege table.


dc_user_session - Announce the startup or shutdown of a User Session.

Tcl Command Syntax

dc_user_session display clientID atexit
Return Value
0 {}
3 SS_display_missing
Description
A User Interface should call this procedure at startup and shutdown to help maintain the status information in table dc_user_session  The display argument should be the X-Windows DISPLAY value or the Windows hostname where the User Session is running.  A Tcl application can determine the latter by calling dp_hostname.  The atexit argument should be set to 1 to indicate User Session shutdown, otherwise 0.   The clientID argument is the DMH Server's identification for the DMH client.  This value is obtained by using the mbx clientID command or the equivalent method using the DMH client libraries.  By using the clientID value, the DCC can use the mh_app_lostclient DMH feature to know when a User Session has disconnected.



Data Collection & SPC Component - GUI

Hume Integration provides a complete user interface, that can be started as a separate process by executing the dc_ui.bat file on Windows platforms, or the dc_ui script on POSIX platforms.  The dc_ui procedure can also be run in the same process that executes the DCC server logic.  In a newly installed system, you can login using the user name of admin and the password admin.  The content of this document is aimed at the needs of the system integrator.  Contact Hume Integration for training and User Guide materials that are aimed at the needs of the system enduser.




Data Collection & SPC Component - Table Schema

General Schema Comments

The conceptual framework for data collection is the notion of data collection items and groups of items.  An item combines closely related data inputs and analysis.   There are approximately two dozen types of items defined by the DCC system.  An example of an item type is the XBAR, S traditional Shewhart SPC analysis.  With this example, the item definition includes parameters specifying input of sample group data combined with control limits and run rules for both the XBAR and S charts, which are populated by the input of the sample group data.

Collected item data is saved in the dc_data table.  Configuration data for the data collection items is found in the dc_item table.  Per-item data that varies with the usage of an item is found in the dc_item_use table.  The fact that the per-item configuration data is cleanly separated from the per-item usage data means that activities such as re-importing configuration data do not disrupt current use of an item definition.  Also, the dc_item records can be shared across multiple DCC servers.

If you are using a persistent database the startup logic creates nearly the same table schema in your persistent database as is used for the in-memory tables.  The only difference between the two sets of tables is some minor variation in the primary table keys to support shared use of a persistent database with multiple DCC instances.

The in-memory dc_data table only holds the most recent subset of data as established by the group_ct field of the dc_item table.  Older data as well as current data can be found in the persistent database dc_data table until it is archived and deleted by the user's custom logic.  The system design relies on the use of high resolution timestamps to key and organize collected data.  To avoid problems with Daylight Saving Time and deployment across multiple timezones, the timestamps are stored in the database for the UTC timezone.  As appropriate, the application logic converts the database timestamps to the user's local timezone for presentation.  Sets of dc_data records that are created as the result of a single data entry action share the same high resolution timestamp value as part of their data.   Some data collection items such as pareto lists provide for the input of multiple dc_data records in a single data entry action.  In addition to the common timestamp, the integer sequence field in the dc_data table is used to uniquely key the similar records as well as show the order in which the observations were entered.

dc_data Table -   Collected Data

Data collection items can be a data input item such as an XBAR,S item, or a grouping of data input items.  The idea of Groups is that a dialog is presented to the user to obtain sets of related data items in one action.  Also, when data is collected in a Group, there is provision to customize the appearance of the data collection dialog, and to customize its logic.  In contrast, the standard data collection dialogs for the input of individual data items are not customizable.

For the usual case of Group data collection, one or more dc_data table records are created.  Each record corresponds to a value of an entered data input item as indicated by the module_id and item_id fields.  In most cases, an additional dc_data record is created  with the module_id and item_id of the Group name in order to record that the data collection event occurred for the Group.  All of these records are given the same ts value to indicate that they were collected as part of the same data collection event.  A record for the Group name is not written if the data collection action is managed by the Process Flow Component.  In this case, the data collection event is recorded in other tables associated with the entity being tracked.  The distinction becomes important when the user wants to display past entries, or re-visit them for editing.  Data that is collected by the Data Collection system is efficiently viewed and managed separately from data that is collected for process flow tracking.

The SPC analysis logic subscribes to the insert of new data into this table.  Entered data should have a corresponding record for the {module_id item_id} in the dc_item table.  The analysis logic will create a corresponding row in the dc_item_use table if one does not already exist.  The analysis logic also subscribes to the deletion of rows to this table, and will automatically dismiss an SPC alarm if the record causing the alarm is deleted.  The analysis logic does not listen to the update of data values.  If the user interface permits the editing or revision of collected data, the user interface should delete and re-insert the revised record for the point.  The SPC analysis will be performed correctly if it is always the latest point that is being deleted.  Therefore, if the application allows revision of a point in the past, and the SPC analysis is to be re-done, the application should delete the revised point and all newer points in the order of newest to oldest, and then re-insert the revised point and the newer points in the order of oldest to newest.  The standard application allows a user to revise the latest point of a data item if he is the person that entered the point.  A higher level of privilege is needed to revise data that has been entered by someone else, or data that has been entered in the past.  The standard application does not attempt to delete and re-insert the newer data to recalculate the SPC analysis.  The standard application also logs the event of the data revision.  A site can withhold the data editing privilege from all users to prevent data revision using the standard application.

A cautionary note on subscriptions.  The SPC analysis logic changes the data that is inserted into the dc_data table.   For EWMA, Individual X, and cusum data items, the second_num field is updated to contain the moving average or a similar statistic.  If an SPC alarm occurs, the notes and spc_alarm fields are updated.  Because of the way subscriptions work, these updates are seen in the correct order by table subscribers only if the SPC analysis subscription is the last subscription on the table.  A procedure is provided, dc_data_sub_open, which should be used to open subscriptions to the dc_data table.  The procedure makes sure that the analysis subscription is always the last subscription.  Another procedure, dc_data_sub_filtered, is provided which is able to combine the analysis updates with the original insert subscription notifications, so that the subscriber just sees the simpler subscription feed.  Still higher level procedures, dc_data_sub_add, and dc_data_sub_mod are provided to combine the filtered subscription with the reading of current item data and configuration.
 
Column Name Key Type Description
module_id PCK varchar(32) Data input items and Group items are given two part names.  This field is used to hold the less specific part of the item name.  For example, the same item_id, GAS_FLOW might be collected for different furnace systems specified by module_id.  Mandatory input.
item_id PCK varchar(32)  The item_id field holds the more specific part of the item name.  Mandatory input.
ts PCK varchar(26) The timestamp of the data collection event formatted as returned by the "localtime 16" command; for example, 1998-10-06 09:18:47.230000.  All of the records for a specific timestamp value are associated with the same data collection event.  In order to guarantee unique values for each data collection event at a specific datahub, the timestamp value is almost always generated in the datahub process.  Mandatory input. 

We are storing the timestamp as UTC values so that there is no issue with usage across timezones or problems with Daylight Saving Time.  To convert into the local timezone, use the Tcl command "mktime $ts 15".

sequence PCK integer Certain data types such pareto lists can have more than one value per data collection event.  The sequence field is used to distinguish and order multiple records of a variable collected in a single data collection event.  When used, sequence values ordinarily start from 100 and are incremented by one.  If not applicable, the value can default to 0.  The starting value of 100 results in a properly sorted list whether numeric or text comparison is used.
prime_num   float For numeric data items, the primary numeric value.  For example, the sample value of an Individual X data item. 
second_num   float For numeric data items, the secondary numeric value such as the sample size, or sample standard deviation.  For system calculated moving ranges, the value is set to -1 when the record is created, and updated to the proper value by the analysis logic.
text_value   varchar(200) This field is used to store collected data that is text such as a selection from a list.  Also, the field is used to hold raw input values to support later review or revision. 
spc_excluded   integer Set to 1 if this record should be ignored by the SPC analysis logic or the logic that accumulates raw individual inputs into subgroup samples.
is_partial   integer Set to 1 if this record is for a raw partial value that the system should collect multiple instances of to form a complete subgroup sample before analysis.
notes   varchar(1000) This text field is formatted as a Tcl list.  The first item of each list element is a tag indicating the nature of the element.  The following tags are used by the system software.  There can be more than one instance of certain tags such as the annotation tag. You are free to develop and use additional tags. 
ec  - Entry comment.  The element is a pair.  The second item of the pair is the free form text entry made by the user during the data collection event. 
oos - Out of specification.  The element is a pair.  The second item of the pair is a text string indicating the specification exceeded by the input value. 
am - SPC alarm message.  The element is a pair.  The second item of the pair is a text message indicating the type of SPC alarm caused by the data input. 
as - Alarm signoff.  The element is a 4 element list.  The second item is the timestamp of SPC alarm signoff.  The third element is the name of the person who performed the signoff.  The fourth element is the text entered at the time of alarm signoff. 
ann - Annotation.  The element is a 4 element list.  The second item is the timestamp of the annotation.  The third element is the name of the person who added the annotation.  The fourth element is the text of the annotation.
spc_alarm   integer Data is ordinarily entered with this field value set or defaulting to 0.  When applicable, the SPC analysis logic will update this field to indicate the occurrence of one or more alarms.  The field is treated as a bit string.  The 0x01 bit indicates an alarm on the primary variable; and the 0x02 bit indicates an alarm on the secondary variable.  For example, the value 3 indicates alarms on both the primary and secondary variables.
user_id   varchar(32) The identifier of the user who input the data in the case of manual input.  Automated equipment interfaces or similar data collection programs should write a software component name into this field.
equipment_id   varchar(32) The equipment identifier in the case of data that is strongly associated with an equipment item.
track_id   varchar(32) This field is used to hold the tracking group identifier when data input is associated with the Process Flow Component.  The timestamp field is used to correlate the process step in the process flow.
discard_flag   integer Ordinarily, data that is written to this table is also saved to the persistent storage system.  Setting this flag to 1 when the record is first created in the datahub indicates that the record should not be saved by the persistent storage system.  If the discard_flag is updated to 1 from 0, the update is passed on to the persistent storage system and it signifies that the record has been removed from the datahub's in-memory database.  The record will not be reloaded into the datahub if the system is shutdown and restarted.  The updated record can be deleted from the datahub without being deleted from the persistent storage system.  The persistent storage system only sees the insertion and deletion of data where the discard_flag is 0. 

Persistent Storage System subscription interface: 

open ... insert, update, delete * where discard_flag=0 
open ... update discard_flag where discard_flag=1 

To remove from datahub but leave in persistent storage: 
update discard_flag to 1 
delete

datahub PCK* varchar(80) In a distributed system with multiple datahubs feeding a single persistent storage system, this field is needed in case of the unlikely event of multiple entries at the same instant at different datahubs.  The field is given the value of datahub's command mailbox, mailbox@group.  The message group alias feature is used as necessary to keep specific computer hostnames out of the message group name.  The field is made part of the primary composite key in the persistent storage system, but it is not part of the primary composite key in the datahub. 

If a value for this field is not supplied when a new record is inserted, and a persistent storage system is in use, and the discard flag is 0, the analysis logic updates the record to have an appropriate value. 

dc_dmh_lostclient Table -   Lost Client Notification

When a client process disconnects from the DMH message system, if the application has defined an mh_app_lostclient procedure, the procedure is called to notify the application. The DCC application uses these notifications to help manage the dc_user_session table data, and to trigger the cleanup of table subscriptions.  So that other application logic can also use these notifications, the notification events are inserted into this table to enable sharing by table subscriptions.  The rows are automatically deleted after a timed period.
Column Name Key Type Description
dmh_clientID PCK varchar(32) This field holds the identification of the client connection which is assigned by the DMH server when the connection is established. 
ts_lost PCK varchar(26) The timestamp of the lost client notification as returned by the "localtime 16" command. We are consistently using UTC timezone values.
superceded   int The value of the superceded parameter passed from the mh_app_lostclient call.

dc_item Table -   Configuration of Data Items

This table is used to hold the configuration records of data collection items such as SPC variables, or data collection groups.  The records hold only fields related to the configuration of the items, and not to any fields related to the usage of items.  Therefore the records in this table can be used to configure multiple instances of item usage throughout the enterprise.
 
Column Name Key Type Description
module_id PCK varchar(32) This field is used to hold the less specific part of the item name.  The value must contain only alphanumeric characters or the underscore. 
item_id PCK varchar(32) This field is used to hold the more specific part of the item name.  The value must contain only alphanumeric characters or the underscore. 
description   varchar(80) The description assists selection by the user.
units   varchar(12) Unit of measure displayed by user interface prompts.
item_type   varchar(2) A  code indicating the type of data item, such as XM for an Individual X and Moving Range item.
formats   varchar(20) A three element list with format command specifications for the prime_num, second_num, and text_value of the data item.  The format used for the prime_num is also used for specification limits, control limits, and plot axis labels.  The usual default is "%g  %g  %s".  See the format command for details on controlling decimal places, etc.
group_input_only   integer Setting this flag prevents data collection for the individual data item by itself.  The data item can always be used as configured in data collection groups.
subgroup_ct   integer The sample size; the count of individual values that contribute to a complete sample reading.  For an SPC item type with a variable sample size, this value is the minimum sample size or the minimum interval duration that can be input using the standard user interface..
group_ct   integer The maximum count of subgroups that (1) are kept in the in-memory datahub table, (2) are used for SPC group statistic calculations, and (3) can be seen in a report or control chart based on the datahub data.  The in-memory  limit does not apply to data that is collected in the context of the Process Flow Component.  The latter is kept in-memory until the tracked entity is removed.
spc_is_enabled   integer SPC  analysis can be configured on or off. 
 input_type   integer This field is used for item types having an XBAR prime value where there is leeway in how the data is input to the system.  0 means all subgroup values are entered in event, 1 means individual values are entered with the system performing subgroup rollup, 2 means subgroup statistics are entered directly
discard_flag   integer This field is not being used by the current version of the Hume written GUI.  It is intended for use by a custom GUI or a future Hume version. The field value is configured to control whether the persistent storage interface saves collected data of this item.  The field value is copied to the dc_data record for data entered through the standard user interface.  0 == save, 1 == discard.
save_options   integer This field is not being used by the current version of the Hume written GUI.  It is intended for use by a custom GUI or a future Hume version. The configured value is intended to control additional saving of item_id values during data collection through the standard user interface.  (used as bitfield:  0x01 = also save as lot attribute, 0x02 also save as equipment attribute, 0x04 also save as user interface attribute, 0x08 also save as a datahub instance attribute).  The item_id portion of the name is used as the name of the attribute - the module_id portion of the name is ignored.
high_screen   float An upper screening limit used to guard against entry error.
has_high_screen   integer Since the datahub does not support the notion of null data, this boolean flag indicates whether the corresponding value is set and in use.
high_spec   float An optional high specification value used for Cp and Cpk calculations, and as input to plot scaling.
has_high_spec   integer Since the datahub does not support the notion of null data, this boolean flag indicates whether the corresponding value is set and in use.
low_spec   float An optional low specification value used for Cp and Cpk calculations, and as input to plot scaling.
has_low_spec   integer Since the datahub does not support the notion of null data, this boolean flag indicates whether the corresponding value is set and in use.
low_screen   float A lower screening limit used to guard against entry error.
has_low_screen   integer Since the datahub does not support the notion of null data, this boolean flag indicates whether the corresponding value is set and in use.
prime_runrules   varchar(200) Use of this field depends on the item type.  A list of run rule names for SPC variables.  A regular expression for text pattern matching for text entry variables.  Tcl code to evaluate in the server process to provide a selection list for dynamically computed list selections.  For item_type "G", the field is used for data entry screen options.
second_runrules   varchar(120) A list of run rule names for the second statistic for certain SPC variables such as XBAR,S.
limit_prime_high   float upper 3 sigma limit for the specific subgroup size
limit_prime_low   float lower 3 sigma limit for the specific subgroup size
limit_second_high   float The upper 3 sigma limit for the range or stddev chart for the subgroup size, also d value for cusum.  For EWMA charts the second limits are for infinitely many points, and the analysis adjusts for the actual number of points.
limit_second_baseline   float Baseline of the range or stddev chart for the subgroup size,  also target value for EWMA or cusum.  This value is not always the mean of the upper and lower limits because of the asymmetry caused by 0.
limit_second_low   float Lower 3 sigma limit for the range or stddev chart, also the cusum theta angle in degrees. 
ewma_weight   float EWMA weighting factor, also known as lambda.
alarm_recipient_id   int Optionally, a non-zero address_id value from a dc_recipient table row specifying a recipient or recipient list for SPC alarm notifications.
#   int Maximum number of dc_data entries - provides guidance for  maintenance of the persistent database.   0 means no limit.
#   int Maximum age of dc_data entries in days - provides guidance for maintenance of the persistent database.  0 means no maximum.

 

dc_item_use  Table - Usage Records for Data Collection Items

This table is used to hold fields related to the usage of data items.  An application can subscribe to the alarm related fields of this table in order to receive notification of SPC alarms.  Fields marked (private) are for internal use and are subject to change.  Also, their values may not be saved to the persistent storage system.

The datahub field is not part of the primary composite key for the table in the datahub, but it is part of the primary composite key in the persistent storage system.

When new data items are created, new rows are inserted into the dc_item table.  The application software does not need to create a corresponding row in this table.  The system software will recognize that a usage record does not exist when the item is first used by the insert of a dc_data record, and a row will be created.
 
 
Column Name Key Type Description
module_id PCK varchar(32) The less specific part of the item name, see dc_data description.
item_id PCK varchar(32) The more specific part of the item name, see dc_data description.
datahub PCK* varchar(80) Used to support multiple server instances sharing tables.
ts_groupstart   varchar(26) A timestamp value that is older than the data records that are in the datahub.  The ts_groupstart is updated by the removal logic to keep the amount of data  in the datahub to its configured size.

We are storing the timestamp as UTC values so that there is no issue with usage across timezones or problems with Daylight Saving Time.  To convert into the local timezone, use the Tcl command "mktime $ts 15".

spc_alarm   integer The field is treated as a bit string to represent the latest unacknowledged alarm.  The 0x01 bit indicates an alarm on the primary variable; and the 0x02 bit indicates an alarm on the secondary variable.  For example, the value 3 indicates alarms on both the primary and secondary variables. 

Updated by newer alarms, cleared by alarm acknowledgment actions.

ts_alarm   varchar(26) If non-blank, the timestamp of the latest data entry which has an SPC alarm.  Updated by newer alarms, cleared by alarm acknowledgment actions.

We are storing the timestamp as UTC values so that there is no issue with usage across timezones or problems with Daylight Saving Time.  To convert into the local timezone, use the Tcl command "mktime $ts 15".

alarm_user_id   varchar(32) If non-blank the user_id of the latest record with an SPC alarm.  Updated by newer alarms, cleared by alarm acknowledgment actions.
alarm_text   varchar(256) If non-blank the description of the latest SPC alarm.  Updated by newer alarms, cleared by alarm acknowledgment actions.
input_is_inhibited   integer Set true on a variable that has an SPC alarm at the time the alarm is detected if consistent with the business rule configuration.  Subsequent input of the variable is blocked until the alarm is signed off.  Data collection in groups where the variable is a mandatory input is also blocked.  This field is not maintained for group data items - its easier to query the included variables in a group at the time of data collection to decide if input is allowed.
partial_data_list   varchar(2000) Used to manage rolling up individual raw data entries into complete sample groups.  A new dc_data record is created after enough inputs have been accumulated.  Each list item contains the ts and prime_value of a raw input. (private)

 

dc_item_type Table -   Data Item Types

To support input prompting, online help, and reporting, the following table is defined and populated with information on the various item types implemented by the Data Collection and SPC Component.
 
 
item_type
varchar(2)
description
varchar(80)
prime_num
varchar(30)
second_num
varchar(30)
text_value
varchar(30)
notes
varchar(400)
XM Individual X and Moving Range X range NA This type is used for single numeric entries that are optionally analyzed using SPC.  For new data, -1 is entered for the second_num and if SPC is enabled, the system logic updates the value to the computed moving range.   Use type NI for single numeric items that are not subject to SPC.
XS XBAR,S XBAR S x1 x2 ... xN Preferred over XBAR,R because the standard deviation is a superior characterization of the variation. 
XR XBAR,R XBAR range x1 x2 ... xN The classical mean and range SPC type.  The most popular Shewhart chart because its easy to do manually. 
XC XBAR and cusum XBAR cusum x1 x2 ... xN The cusum statistic is preferred by many SPC professionals to XBAR,S but it is not familiar to the lay person.
XE XBAR and Exponentially Weighted Moving Range XBAR EWMA x1 x2 ... xN This type can also be used for individual X values.  For new data, -1 is entered for the second_value and if SPC is enabled, the system logic updates the value to the computed EWMA.
C c chart - the number of defects in a fixed sized sample count sample size NA
NP np chart - the number of defective units in a fixed size sample of units. count  sample size NA
P p chart - the proportion of defective units for a variable size sample proportion sample size defective_units sample_size
U u chart - the number of defects in a variable size sample defects/unit sample_size defects_count sample_size
LI List item - a single item selected from a predefined list NA NA selected_text
DI Dynamic list item - a single item selected from a dynamically computed list NA NA selected_text This type includes selections from system provided lists such as users, equipment, and active lots.
TI A single text entry NA NA entered_text The entered text can be subjected to screening by regular expression pattern matching.
LL One or more items selected from a predefined list NA NA selected_text Each selection becomes an additional dc_data record.
DL One or more items selected from a dynamically computed list NA NA selected_text Each selection becomes an additional dc_data record.
TL One or more text entry items. NA NA entered_text Each entry becomes an additional dc_data record.
PI Pareto item - an entered quantity and a selection from a predefined list quantity NA selected_text
PL Pareto List - one or more pareto items quantity NA selected_text This type is the most common means of capturing yield and yield loss data.  Each entry becomes an additional dc_data record.
G Specifies a Group of data items NA NA NA An empty dc_data record is created when data collection for the group occurs using the Data Collection and SPC Component.
IC Interval c - the number of events in a fixed duration interval count hours_duration ts_stop On these interval data items, the time units are not consistent between types in order to have statistics that are intuitive in scale.  The IC statistic is used to monitor alarm occurrences and similar events.
IN Interval np - the number of seconds spent in a state for a fixed duration interval seconds secs_duration ts_stop
IP Interval p - the proportion of time in a state for a variable length interval proportion hours_duration ts_stop This statistic can be used to monitor utilization, downtime, or assist time in relation to usage time or available time for equipment.
IU Interval u - the number of events in a variable length interval events_per_hour hours_duration ts_stop count This statistic is used to monitor throughput, alarm occurrences and similar events.
NI Numeric Item - a simple numeric input without SPC X NA NA This type is suitable for any type of numeric input, floating point or integer, such as a quantity.
LB Label - displayed text that optionally is created from dynamic content NA NA NA This type and the next type are defined for use in data collection dialogs, but there is no data collected for these types. 
HC Hidden Calculation - Custom Tcl code that is executed during data collection NA NA NA

dc_runrules Table -   Data Collection SPC Analysis Run Rules

To support input prompting, online help, and reporting, the following table is defined and populated with information on the SPC run rules implemented by the Data Collection and SPC Component.

Run rules are listed in severity order.  Only the most severe run rule violated is reported for a given input point.
 
 
name
varchar(10) PK
description
varchar(80)
3sigma Western Electric #1 - one point outside of +/- 3 sigma.
3sigmaH One point higher than + 3 sigma
3sigmaL One point lower than -3 sigma
2sigma Western Electric #2 - 2 of 3 points above +2 sigma or below -2 sigma
2sigmaH 2 out of 3 points higher than +2 sigma
2sigmaL 2 out of 3 points lower than -2 sigma
1sigma Western Electric #3 - 4 of 5 points above + 1 sigma or below -1 sigma
1sigmaH 4 of  5 points higher than +1 sigma
1sigmaL 4 of 5 points lower than -1 sigma
Biased Western Electric #4 - 8 successive points above CL or below CL
BiasedH 8 successive points above CL
BiasedL 8 successive points below CL
Vmask Cusum statistic V-mask
Trend 7 successive points all increasing or all decreasing
TrendH 7 successive points all increasing
TrendL 7 successive points all decreasing

 

dc_custom Table -   Storage for User Defined Items and Configuration Values

This table provides a handy place to store data values for sharing in a distributed application.  If a persistent storage system is in use, all table data is saved and restored at application startup.

The table is available for use by user defined custom logic.  Other application components such as the Process Flow Component make use of this table as well.  Values that are configured for other components are documented with the specific component.

When a data collection item is saved as a datahub instance attribute, its entered value is written to this table using the mailbox@groupname of the datahub (truncated to the field length) as the module_id and the data item item_id as the name.

In a distributed system with multiple datahubs and a single persistent store, it is assumed that all rows of this table are loaded at initialization by all of the datahubs.  Application use of the table and naming conventions should insure that conflicting changes to the same rows at different datahubs do not occur.
 
Column Name Key Type Description
domain PCK varchar(32) The domain or module of application for the configured entry.
name PCK varchar(32) A specific name for the configured entry. 
value   varchar(2000) Application data formatted as text.

There are certain values that are used by the Data Collection and SPC Component software that are defined during site configuration.  These values include:
 
domain name value description
DCC DataEntryRepeat 1 A boolean - if true, Data Entry dialogs refresh for another use instead of closing after the user has completed data entry.
DCC LostClientCleanupAllTables 1 A boolean - if true, subscription and mailbox cleanup triggered by the DMH LostClient notification  is applied to all tables, and not just DCC tables.  The logic finds and closes subscription names that have the DMH clientID string isolated by underscore characters on both sides.  The logic also flushes and closes DMH mailboxes that were used as the destination mailboxes of the closed subscriptions.
DCC MB_DB_SUBFEED_max 50 When using a persistent database, the DCC periodically checks the number of messages queued for the persistent database to detect a problem.  This integer value is the threshold number of queued messages which indicates a problem passing data to the persistent database.
DCC MailboxPollSeconds 1800 When using a persistent database, an integer value for the polling interval in seconds used to monitor the persistent database feed.  The minimum value allowed is 1000.
DCC MaxLoginPerUser 0 An integer value for the maximum number of login sessions per user, 0 means no limit.
DCC MaxUsersPerDisplay 1 An integer value for the maximum number of user sessions expected per display, 0 means no limit.
DCC MinPasswordLength 4 An integer value for the minimum characters allowed for the length of a password.
DCC PasswordExpirationDays 0 If greater than zero, logins are rejected if the password has not been changed within this many days.  If your application is using password expirations, and the user interface does not allow the user to change his password without login, a user with an expired password will not be able to access the system. 
DCC distribution_list {} A list of Data Collection Component server addresses used for the distribution of configuration data.
DCC email_sender {} A value used in the From: e-mail address for e-mail sent by the DCC system.  If the value is blank, the e-mail is sent without an originator.  This may cause the e-mail to be rejected by SMTP servers that  filter spam.
DCC email_SMTP_servers smtp mail One or more SMTP server hostnames to support sending e-mail SPC alarm notifications.  Usually includes the domain name as in smtp.sematech.org.
DCC email_template_pager   A template for the format of e-mail notifications.  The pager format is intended to be a short format for SPC alarm notifications sent to pager devices.  The value is a list of two elements; templates for the e-mail Subject and the Body of the message.
DCC email_template_report   A template for the format of e-mail notifications.  The report format is intended to be a longer format for SPC alarm notifications sent to people as ordinary e-mail.  The value is a list of two elements; templates for the e-mail Subject and the Body of the message. 

Note: a site can use additional e-mail message formats by installing additional rows in this table where the name value is of the form email_template_message_format and the domain value is DCC.

DCC ERROR_LOG {} If background Tcl runtime errors occur either in the DCC server, a persistent database interface, or a Tcl GUI processes, the error information is trapped, forwarded to the DCC server, and appended to a common log file.  This table entry can hold a custom pathname for the error log file.  The default value is the filename error.log in the DCC server Tcl library directory.  The System Administrator should periodically see if this file exists and if so, examine the unexpected errors that have occurred. 
DCC modem_device {} This optionally configured value supports SPC alarm notification using phone calls.  The value is a device name such as COM3 or /dev/tty01 where there is a modem that uses AT dialing commands.
DCC print_template_unix exec lpr {$pathname} A template for printing postscript files when running on UNIX systems.
DCC print_template_windows {} A template for printing files on Windows systems, such as 'exec print {/d:\\server\share} {$pathname}'
SPC alarm_inhibits_entry 1 Set to 0 to override the default behavior of inhibiting manual data entry for a variable that has an unacknowledged SPC alarm.
SPC show_item_save_cmd 1 A boolean - if true, the command used to create or update an item configuration is displayed on the console and to TRACE as an aid to development.

dc_group_config Table -   Data Collection Group Configuration

Data collection items can be assigned to any number of groups.  The group definitions are used for data collection operations, and subsequently for reporting and plotting.  A group is a set of data items as configured individually in table dc_item plus other items such labels and calculation formulas that control dialog appearance and customize the logic of the data collection actions.  The framework logic that manages the dialog-user interaction incorporates the execution of user-defined custom code during dialog creation, data entry, data validation, and data acceptance.  It is possible to do sophisticated actions such as coordinated communication with equipment, or user interaction with custom child dialog windows using the data collection component as the organizing framework.  The custom logic has read/write access to the data that the user enters or sees on the dialog window.

Calculation code can use the dcData( ) global array variable for read/write access to current entry data, and read access to historic collected data.

dcData(module_id,item_id[,instance[,subfield])
The subscript is specified as 2 to 4 values separated by commas.  The instance value defaults to 0.  The instance value 0 refers to data currently being entered.  The instance value -1 is used to specify data that was entered previously, the value -2 is used to specify data that was entered the time before the previous entry, etc.  The subfield value defaults to 0.  During data entry, the subfield value is used as an index, starting from 0, to specify an entry field by its relative position among the entry fields for an item.  So a value such as $dcData(EdsCar,Odometer) can refer to an Odometer reading currently being entered.  For historic data, the subfield value is specified as P, S, or T to specify the prime_num, second_num, or text_value of a dc_data table row.  So a value such as $dcData(EdsCar,Odometer,-1,P) can refer to the Odometer reading that was entered last time.  For historic data, the default subfield value specifies the prime_num dc_data table row value, so the previous example can be shortened to $dcData(EdsCar,Odometer,-1).

Continuing with the Odometer example, you can have a data entry group where the gas mileage entry is computed using a formula.  You would configure the mileage item to have a calculation such as the one shown below, which supplies a read-only value for gas mileage that is calculated when the user presses the Calculate button or presses the Ok button to complete the data collection action.

expr {($dcData(EdsCar,Odometer) - $dcData(EdsCar,Odometer,-1))/$dcData(EdsCar,GasGallons)}
There is another global array, dcAT( ) which is provided for read access to attributes of the current data collection context including items such as the logged in user.
dcAT(attributeName)
In sequence, the logic that supports this array attempts to find values in (1) the global array that is used to manage the data collection dialog, (2) the global array dc_ui( ) that has DCC User Interface data, and (3) the global array env( ) that accesses environment variables.  Some of the interesting attributeName values available include: You can use the inspect application to examine the global data arrays of the DCC User Interface.

The calculation feature can be used to great advantage for interacting with equipment and combining data collected from equipment with manually collected data.  You can define your own procedures in your own Tcl code files and have them executed when the user presses an appropriate button on a data collection dialog.  The procedures can write results into visible fields that the user can review and comment on.  There is nothing stopping you from creating additional windows, for example to provide operator instructions or for complex interaction with attached equipment.  The displayed text of the existing buttons and labels can also be customized in your calculation logic.  Think of the Data Collection Component as an organization framework for integrating the factory floor.
 
Column Name Key Type Description
group_module_id PCK varchar(32) The module_id of the group.
group_item_id PCK varchar(32) The item_id of the group.
sequence PCK integer Members of the group are ordered in drawing and/or calculation sequence starting from 100 and incrementing by 1.  The starting value of 100 results in a properly sorted list whether numeric or text comparison is used.
item_type   varchar(2) The type of the item that is a member of the group.  Types LB (label) and HC (hidden calculation) represent items that do not have data input.
module_id   varchar(32) If the item_type indicates a data input item, the module_id name.
item_id   varchar(32) If the item_type indicates a data input item, the item_id name.
is_optional   integer If the item_type indicates a data input item, this boolean indicates whether input of the item is optional and may be skipped.
calc_type   integer (0) No calculation.
(1) The calculation supplies a value for the data item that the user sees but cannot alter.
(2) The calculation supplies a default value for the data item that the user can overwrite.
(3) The calculation value is a string that optionally imbeds procedure calls or variable substitutions.  This calculation type is used for labels.
calc_context   integer Organized as a bitfield to describe under what contexts the calculation is evaluated 
0x01 - initial creation of the dialog for a new entry 
0x02 - initial creation of the dialog for editing an existing entry 
0x04 - refresh of the dialog for another entry 
0x08 - A button has been pressed to request the calculation. 
0x10 - The user has pressed the Ok button and validation will occur after this calculation pass.  Last chance for calculations to affect values.  This pass may occur more than once if the data fails validation. 
0x20 - The data has been validated and is being committed.  Calculations should not change any values. 
0x40 - The dialog is being destroyed.
calculation   varchar(240) If applicable, the value of this field is Tcl code that is executed directly in a global context.  It can be one or more statements.  The result of the calculation is usually assigned to a displayed entryfield or label. 

dc_list_text Table -   Preconfigured selection list values

This table is used to store the predefined selection lists corresponding to data collection item_types "LI", "LL", "PI", and "PL".  A sequence field is used to order the list for selection when presented to the user.
 
Column Name Key Type Description
module_id PCK varchar(32) see dc_data description.
item_id PCK varchar(32) see dc_data description.
text_value PCK varchar(80) The text selected by the user and recorded in dc_data. 
sequence   integer The selection list is ordinarily presented to the user sorted in ascending sequence by this field.

dc_recipient Table -   E-mail and Paging Recipient Data

The DCC is able to send e-mail notifications to individual addresses or to lists of addresses when SPC alarms occur.  The format of the email notifications is configurable; there can be any number of named formats.  This feature accommodates the different needs of numeric pagers, alphanumeric pagers, and human e-mail recipients.  The notification system can also dial phone numbers using standard, platform independent AT modem commands.  This feature can be customized for interacting with phone system menu commands.  An example script is available for issuing a numeric page to a Sprint PCS cellular phone.

This table holds notification recipient data.  A table record can represent either a recipient or a list of recipients.  In the case of a list, the addr_type field value is list, and the addr_data value is a list of address_id key values for the members of the list.  The DCC GUI and logic do not support lists containing lists; each member of a list is expected to be a recipient.

To support email and paging, additional entries are configured in the dc_custom table.
 
Column Name Key Type Description
datahub *PCK varchar(80) Used as a primary key in the persistent database, but not for the in-memory tables.  You may have a persistent database which is shared among DCC instances.
address_name PCK varchar(64) A user defined description for the record - usually a person's name, or a name given to a recipient list.  A GUI will typically display the other record values along with the description so the person does not need to indicate address type data in the name.  This field is made part of the table primary key so the records are sorted by name, but the address_id value by itself is useable as a unique identifier for the record.
address_id PCK int A unique integer key value that is system generated when a record is created. 
sending_is_disabled   int Used as a boolean to disable use of the record.  You are able to disable and enable sending e-mail or dialup notifications to a recipient or recipient list without having to delete and recreate configuration records.  The Hume Scheduling Component can be used to update the table according to changing work shifts.
address_type   varchar(20) Indicates the type of recipient table record.  Current supported values are list, email, and modem.
message_format   varchar(32) The use of this field depends on the address_type value.  For email records, the value in this field is used to look up a template in the dc_custom table for the format of the e-mail message.  For example, if the message_format value is pager, the DCC logic looks in the dc_custom table for a template value entry where domain='DCC' and name='email_template_pager'.
address_data   varchar(2000) For a record of address_type email, a valid email address such as jsmith@company.com.

You can also use standard RFC formatting and include a "friendly name" with the actual address delimited with angle brackets, as in the example:
Stepper 12 Engineer <jsmith@company.com>.

For email addresses, at the time of sending e-mail, if the address_data value does not contain angle brackets, the DCC logic uses a longer address format that is formed using the address_name value as the "friendly name" and the address_data value as the data placed within angle brackets - "address_name" <address_data>.  This feature makes it easier for human readers to understand the recipient list of an e-mail message.

For a record of address_type list, this field value is a list of one or more address_id values.


 

dc_secs_event Table -   For SECS Trace Report and Event Report Data Collection

This table is used to list the Event Reports which SECS host interfaces in the local DMH message group have configured, and are possible sources of input data to the DCC.  The table also holds status information related to the receiving and processing of the Event Reports and Trace Reports.
 
Column Name Key Type Description
datahub *PCK varchar(80) Used as a primary key in the persistent database, but not for the in-memory tables.  You may have a persistent database which is shared among DCC instances.
spname PCK varchar(32) The name of the SECS host interface connection.  The connection is assumed to be using the Hume GEM Host application software.  Connections are discovered in the DMH message group.
CEID PCK varchar(32) The SECS Event Report Collection Event ID.  For Event Report types that are discovered from the SECS Host instances, this is the "VFEI" Event_ID seen in the GEM Host user interface list of configured reports.

For Trace Reports that managed by the DCC, the CEID values are patterned as "TRACE_REPORT.$n" where $n is an integer value used as the "trace_number" identifier.

description   varchar(80) A description for the event.  SECS equipment that uses the Hume software usually provide descriptions of events as a Status Variable value, EventDescriptions.  The Hume Host software is able to automatically recognize and use the descriptions.  The DCC is able to collect event descriptions that are automatically acquired by the Host interfaces, or that are manually entered.
is_collected   int Used as a boolean to indicate which Event Reports the user would like the DCC to receive and collect data from.  Set to 0 for DCC managed Trace Reports because they use different collection logic than the Event Reports.
EI_configured   varchar(1) Used as a boolean with values 1 or 0, or an empty string when the value is not known or not relevant.  This field is set to 1 for Event Reports that the DCC determines are configured at the SECS host interface.  This field is set to 0 if the DCC determines that the configuration of the report no longer exists.

This value, and similar values in this table are updated by the dc_secs_discovery logic which is called when initiated from the UI.
The value is not used for DCC managed Trace Reports.

EI_enabled   varchar(1) Used as a boolean with values 1 or 0, or an empty string when the value is not known or not relevant.  This field is set to 1 for Event Reports that the DCC determines are defined and setup with the SECS equipment.  This field is set to 0 if the DCC determines that the report is not presently setup with the equipment.  Typically, the value reflects whether the SECS Host has made it through the initialization logic and has successfully enabled and defined the configured event reports with the attached equipment.
The value is not used for DCC managed Trace Reports.
ts_received   varchar(26) Used to record the local timezone timestamp for when the Event Report or Trace Report was last received by the DCC, as a call to the procedure dc_secs_insert.   The value can be used for status and diagnostics.
last_result   varchar(2000) Used to record the result of executing dc_secs_insert for the event.   The value can be used for status and diagnostics.

dc_secs_host Table -   SECS Host and Event Report Discovery

This table is used to record which SECS Host interface instances are found as participants in the DMH message group, and to show the asynchronous results from contacting each instance to discover Event report configurations.
 
Column Name Key Type Description
datahub *PCK varchar(80) Used as a primary key in the persistent database, but not for the in-memory tables.  You may have a persistent database which is shared among DCC instances.
spname PCK varchar(32) The name of the SECS host interface connection.  The connection is assumed to be using the Hume GEM Host application software. 
ts_command   varchar(26) The local timezone timestamp when the latest command was sent to the SECS host using the DMH message system.
last_command   varchar(32) A short, readable description such as "event report discovery" for the last command message sent to the SECS host interface.
last_result   varchar(80) A short, readable description for the Host's response (or lack of response) such as "Event report data obtained." or "TIMEOUT". 

dc_secs_trace Table -   Configuration and Status of SECS Trace Reports

This table is used to record trace report configurations defined by the system user.
 
Column Name Key Type Description
datahub *PCK varchar(80) Used as a primary key in the persistent database, but not for the in-memory tables.  You may have a persistent database which is shared among DCC instances.
spname PCK varchar(32) The name of the SECS host interface connection. 
trace_number PCK int For maximum compatibility with SECS equipment, the DCC forces the choice of positive integers for trace request identifiers (SEMI "TRID"). 
description   varchar(80) Comments defined by the user, displayed when viewing the list of configured traces.
period_secs   float The sampling period in seconds.  The DCC will allow the period to be configured as short as 0.1 seconds.  Most equipment does not accept the newer format of S2F23 and cannot be told to use non-integer values, or values less than 1 second.

Even for integer values, the user's desired value may be unacceptable to the equipment, in which case the numeric error code of 3 is seen in the last_result value when the trace is started. 

total_samples   int A positive value for the total number of samples to be acquired by the trace.  The total samples must be evenly divisible by the report_sample_ct value.
report_sample_ct   int The number of sample values to be reported in each Trace Report SECS message.  Defaults to 1. 
ts_started   varchar(26) A timestamp in the local timezone when the request to start tracing was replied to with success.
ts_stopped   varchar(26) A timestamp in the local timezone for the projected completion time of a trace.  The value is set when the start command has been replied to with success.
ts_command   varchar(26) A timestamp in the local timezone when the latest trace report related command was sent to the SECS host interface.
last_command   varchar(32) A short, readable description such as "start/restart" for the last trace report related command sent to the SECS host interface.
last_result   varchar(80) The reply message received from the latest trace report related command sent to the SECS host interface.  A typical success value is "0 0" which shows that a Tcl procedure was executed without error, and the return code of the procedure was a 0 indicating success.  A typical failure is signaled with the literal text TIMEOUT which is seen if the SECS host interface, or SECS equipment is not communicating.

dc_secs_var_map Table -   Mapping of SECS Trace Report and Event Report Data Items

This table holds configuration data describing the mapping of SECS variable values to DCC items.  The table is consulted by the logic of the dc_secs_insert command to determine which data items in an Event Report or Trace Report are to be added to which DCC item.  The dc_secs_discovery logic adds records to the table that reflect the Event Report configurations that it finds.  The user can then update the table to determine which report values are saved, and to determine which DCC items the values are saved under.  The latter choice can vary by event type.
 
Column Name Key Type Description
datahub *PCK varchar(80) Used as a primary key in the persistent database, but not for the in-memory tables.  You may have a persistent database which is shared among DCC instances.
spname PCK varchar(32) The name of the SECS host interface connection. 
CEID PCK varchar(32) The SECS Event Report Collection Event ID.  For Event Report types that are discovered from the SECS Host instances, this is the "VFEI" Event_ID seen in the GEM Host user interface list of configured reports.

For Trace Reports that managed by the DCC, the CEID values are patterned as "TRACE_REPORT.$n" where $n is an integer value used as the "trace_number" identifier.

varID PCK varchar(32) The low-level SECS variable identifier such as an integer code.
module_id   varchar(32) Part of the target DCC item identifier.  Defaults to the spname value when discovering new Event Reports or defining new Trace Reports.
item_id   varchar(32) Part of the target DCC item identifier.  Defaults to the variable name value when discovering new Event Reports or defining new Trace Reports.
EI_configured   varchar Used as a boolean with values 1 or 0, or an empty string when the value is not known or not relevant.  This field is set to 1 for Event Report variables that the DCC determines are configured at the SECS host interface.  This field is set to 0 if the DCC determines that the variable is no longer configured as part of the Event Report. 

The value is not used for DCC managed Trace Reports.

is_collected   int This value is used as a boolean to indicate whether values of this variable, when received in the CEID report, should be collected by the DCC.  More than one variable can be received in a report.  The user can defined which variables in a report are to saved or ignored. 
track_id   varchar(32) The configured value of the track_id gets copied into the dc_data records.  The track_id value is typically a Lot identifier or similar tracking ID.  This is a workable scheme for manually initiated Trace Reports.  For automatically collected Event Reports, the track_id value can be dynamically updated by custom logic since it is not practical to expect the Event Report saving to be manually reconfigured with every lot.

dc_user Table -   User Profile Data

This table is used to help provide application level security. A user belongs to one or more security groups. A group is associated with privileges. At login, the user is given the union of all privileges associated with the groups that he is a member of. The application logic is coded to enable buttons and menu actions depending on the possession of specific privileges by the user.
 
Column Name Key Type Description
user_id PK varchar(32) A shorthand identification of the user. Consistent with other names in the application, the user_id should not contain any embedded spaces.
user_name   varchar(60) Usually used to hold the firstname and lastname of the user.
password   varchar(60) An encoded representation of the user's password created using the Tcl dc_encode procedure or its equivalent. 
timeout_secs   int Some user interfaces support automatically logging out a session where the application has been left idle for a number of seconds in excess of this value.  Hume provided versions of Tcl provide a winfo timelastevent command which can be polled to determine if the application is idle.
ts_lastedit   varchar(26) A "localtime 16" timestamp for the last time that the User's record or associated security group membership data was updated using the DCC API. We are consistently using UTC timezone values. 
lasteditor   varchar(32) The user_id of the person who last updated the User's record using the DCC API.
clientdata   varchar(1000) A field for applications to store miscellaneous per-User data such as badge numbers, desired window placements, etc.

dc_user_group Table -   Security Groups

A User is associated with any number of Security Groups. This table is used to identify and describe the groups.
 
Column Name Key Type Description
group_id PK varchar(32) The group_id is a shorthand name for the security group. It obeys the same naming conventions of other names in the DCC system. 
description   varchar(80) A description for the security group assigned when the group is created or updated. The DCC API does not allow saving a group configuration with a blank description.

dc_user_grp_assoc Table -   User - Security Group Associations

This table holds the association of Users to Security Groups. A User can belong to any number of groups.
 
Column Name Key Type Description
user_id PCK varchar(32) The User key value.
group_id PCK varchar(32) The Security Group key value.

dc_user_priv_assoc Table -   Security Group Privilege Associations

This table holds the associations of Privileges to Security Groups.  A privilege can be assigned to any number of Security Groups.
 
Column Name Key Type Description
group_id PCK varchar(32) The Security Group key value.
privilege_id PCK varchar(32) The Privilege key value.

dc_user_privilege Table -   Security Privileges

A protected application feature such as entering data, is assigned a Privilege name. The application logic is coded to check if the user has the assigned privilege before allowing access to the feature.
 
Column Name Key Type Description
privilege_id PK varchar(32) A shorthand name for the Privilege.  Obeys the same naming conventions as other names in the DCC system.
description   varchar(80) A description provided when saving the Privilege.  The DCC API does not allow saving a blank description.

dc_user_session Table -   User Session Status

This table is used to track the existence of User Sessions and logins. The table only exists in the DCC server, and not in the persistent database. By tracking the DMH Server's identification for a client connection, the logic can know if a User Session has disconnected, and if so, the logic performs an automatic logout. Fields such as the user_id or ts_login are empty strings until the associated action is performed.
 
Column Name Key Type Description
display PCK varchar(80) This field is the X-Windows DISPLAY value for the user session, or the Windows hostname.
dmh_clientID PCK varchar(32) The DCC logic is able to determine the DMH server's identification of the client from the global variable dp_rpcFile which is set to the client's identification during command message processing.
logged_in   int Used as a boolean to indicate whether the user is currently logged in.
user_id   varchar(32) The key value for the User who last logged in.
user_name   varchar(60) The User name copied from the dc_user table field at the time of login. 
ts_login   varchar(26) This field is updated to show the timestamp of the latest login for the User Session. We consistently use the UTC timezone values. 
ts_logout   varchar(26) The timestamp of the last logout for the User Session.
ts_startup   varchar(26) The timestamp when the User Session is first known to exist. A proper UI application calls the dc_user_session procedure at application to announce its existence. The existence of a User Session is also inferred from calls to the dc_user_login procedure.
privileges   varchar(2000) The union of privileges for the logged in user as returned by the dc_user_login procedure. 
clientdata   varchar(1000) Copied from the dc_user table at the time of the last login. 

dmh_trace Table -   DMH message system trace messages

The DCC application has customized the Trace feature of the Hume DMH message system.  Messages that are sent the the DMH mailbox TRACE are appended to this table where they can be queried or subscribed to.  During startup with a persistent database, messages are sent to TRACE to show the progress of reading table data.  While running the system, there can be other messages sent to TRACE if Tcl programming errors occur.  The System Administrator should display the data of this table to see if there are diagnostic messages in the event of abnormal behavior.  By having the data in a table, the hubclient application can be used from another workstation to display the data.  There is automatic trimming logic to keep the number of rows to less than 1000.  This table only exists as an in-memory table in the DCC server.
 
Column Name Key Type Description
id PK varchar(6) An increasing sequence number padded and formatted as a character string so that the messages are sorted from oldest to newest.
timestamp   varchar(26) A high resolution local timezone timestamp YYYY-MM-DD HH:MM:SS.mmmmmm showing the instant that the message was appended to the table.
tracemsg   varchar(15000) The text of the trace message.


License Terms

The Data Collection Component software is not part of the Hume Datahub SDK; it is available as a separate licensed product. Contact Hume Integration for current terms.

Document Version

Date of last revision: $Date: 2006/01/17 16:33:03 $

This document covers the Hume Integration Software developed Data Collection Component which is available for the Tcl 8.4/Tk 8.4 environment on the Windows XP/2000/NT and UNIX/POSIX platforms.