createAndStartDataChangeStreams

Initialize one or more data change streams

The "createAndStartDataChangeStreams" action starts processing one or more data change streams for the specified streaming connection. If a stream includes existing records, this action adds the stream to a queue where the server loads existing records from tables in the order you start them. 

You can call "createAndStartDataChangeStreams" after it has already started to resume a paused stream.

This action returns the unique ID of the streams along with the stream properties. You use the unique ID to pause, list, and describe streams.

It uses the "streamingConnectionName" property to connect the stream to one table in a FairCom DB or RTG server. You must use the "sourceTableName", "sourceOwnerName", and "sourceDatabaseName" properties to specify the table. You may optionally use the "localTableName", "localOwnerName", and "localDatabaseName" properties to specify your own name for the table in FairCom MQ; otherwise the action uses the source name, owner, and database.

At any time, you can use the "tableName" property of MQ API's "configureTopic" action to assign or change the topic assigned to the local table created by this action. 

You can set the "includeExistingRecords" property to true to start a data change stream from the first record in the table or set the "includeExistingRecords" property to false to start the stream from the next data change event. When a stream starts from the first record, FairCom MQ loads all existing data into the target servers as data change events. The time to synchronize existing records depends on the number of existing records.

You may optionally use the "recordFilter" property to include matching records in the stream. The filter uses FairCom's expression language.

You may optionally use the "includedFields" property to include specific fields in the stream.

 

Request examples

Minimal

{
  "action": "createAndSstartDataChangeStreams",
  "params": {
    "streamingConnectionName": "db1",
    "streams": [
      {
        "description": "new records from ctreesql_admin_custmast to custmast",
        "topic": "custmast",
    
        "sourceTableName": "custmast",    
        "sourceOwnerName": "admin",
        "sourceDatabaseName": "ctreeSQL",

        "includeExistingRecords": false
      }
    ]
  },
  "authToken": "replaceWithAuthTokenFromCreateSession"
}
 
 

Maximal

This example returns all information about each affected data change stream.

{
  "api": "admin",
  "action": "createAndSstartDataChangeStreams",
  "params": {
    "streamingConnectionName": "db1",
    "streams": [
      {
        "description": "existing & new CA records from ctreesql_admin_custmast",

        "topic": "custmast",

        "localTableName": "custmast",
        "localOwnerName": "admin",
        "localDatabaseName": "faircom",
        "localDataFilePath": "",
    
        "sourceTableName": "custmast",
        "sourceOwnerName": "admin",
        "sourceDatabaseName": "ctreeSQL",
        "sourceDataFilePath": "",
    
        "includeExistingRecords": true,

        "recordFilter": "!stricmp(cm_custstat,\"CA\")",
        "includedFields": [
          "cm_custnumb",
          "cm_custzipc",
          "cm_custcity"
        ],
    
        "triggers": [ "insert", "update", "delete" ],
        "recordFormat": "fields",
    
        "tagChanges": "forEachField",
        "includePrimaryKey": "forPrimaryKeyFields",
        "includeMetadata": [],
        "revealAfterValueOnFilteredDelete": false,
        "revealBeforeValueOnFilteredInsert": false,
        "fixedOutput": false
      }
    ]
  },
  "authToken": "replaceWithAuthTokenFromCreateSession"
}
 
 

 

Response examples

Maximal

{
  "api": "admin",
  "action": "createASstartDataChangeStreams",
  "params": {
    "streams": [
      {
        "description": "existing & new CA records from db1 ctreesql_admin_custmast",
    
        "dataChangeStreamStatus": "starting",
        "dataChangeStreamFirstStartTimestamp": "2025-06-07T12:23:19.275",
        "dataChangeStreamLastStartTimestamp": "2025-06-07T12:23:19.275",
        "dataChangeStreamLastPausedTimestamp": null,
    
        "streamingConnectionName": "db1",
        "streamingConnection": {
          "sourceServerName": "FAIRCOMS",
          "sourceHostname": "10.70.13.112",
          "sourcePort": 5597,
          "sourceUsername": "ADMIN",
          "sourcePassword": "ADMIN",
          "tls": {
            "enabled": true,
            "caCertificateFilename": "ca.crt",
            "allowedCipherSuites": "",
            "clientCertificateEnabled": true,
            "clientCertificateFilename": "admin_client.crt",
            "clientPrivateKeyFilename": "admin_client.key"
          },
          "metadata": {
          }
        },
    
        "topic": "custmast",
    
        "localTableName": "custmast",
        "localOwnerName": "admin",
        "localDatabaseName": "faircom",
        "localDataFilePath": "",
    
        "sourceTableName": "custmast",
        "sourceOwnerName": "admin",
        "sourceDatabaseName": "ctreeSQL",
        "sourceDataFilePath": "",
    
        "includeExistingRecords": true,
        "recordFilter": "!stricmp(cm_custstat,\"CA\")",
        "includedFields": [
          "cm_custnumb",
          "cm_custzipc",
          "cm_custcity"
        ],
        "triggers": [
          "insert",
          "update",
          "delete"
        ],
    
        "recordFormat": "fields",
    
        "tagChanges": "forEachField",
        "includePrimaryKey": "forPrimaryKeyFields",
        "includeMetadata": [
        ],
        "revealAfterValueOnFilteredDelete": false,
        "revealBeforeValueOnFilteredInsert": false,
        "fixedOutput": false
      }
    ]
  },
  "authToken": "replaceWithAuthTokenFromCreateSession"
}
 
 

 

Properties

Request properties ("params")

Property Description Default Type Limits (inclusive)

streamingConnectionName

The "streamingConnectionName" property specifies a unique, user-defined name for a streaming connection. The API uses it to identify streaming connections and to connect a data change stream to a FairCom DB or RTG server. Required - No default value string 1 to 64 characters

streams

The "streams" property specifies an array of stream definitions. Required - No default value Array of stream definitions 1 or more stream definitions

streams

.description

The "description" property describes objects such as code packages, labels, or things. The server indexes this field with a full-text index so that you can search for any word or phrase. You cannot use this property for filtering in the Thing API.

Optional with default of "".

"unknown" for Thing API 

string

0 to 65,500 bytes

1 to 512 bytes for the Thing API 

streams

.fixedOutput

The "fixedOutput" property includes all properties in a data change event when true.

Optional with default of false Boolean

true

false

streams

.includedFields

The "includedFields" property determines which source table fields in the data change event will be included. When empty, this property includes all fields.

Optional with default of [] array of strings 0 or more strings

streams

.includeExistingRecords

The "includeExistingRecords" property returns streams that synchronize existing records if true.

Optional with default of null Boolean

true

false

streams

.includeMetadata

The "includeMetadata" property specifies whether to include the "metadata" property in the response.

Optional with default of true

Boolean

 

true

false

null

 

streams

.includePrimaryKey

The "includePrimaryKey" property specifies when to add the "pk" property to the data change event's "fields" object to indicate the field's position in the primary key. Optional with default of "forPrimaryKeyFields" string enum

"forEachField"

"forPrimaryKeyFields"

"never"

streams

.localDatabaseName

The "localDatabaseName" property specifies the database name of the table on the FairCom MQ server that stores the stream's data change events.

Optional with default of the "defaultDatabaseName" value that is set during "createSession". If no default is set during "createSession", then "faircom" is used. string 1 to 64 bytes

streams

.localDataFilePath

The "localDataFilePath" property specifies the data file path of the table on the FairCom MQ server that stores the stream's data change events. It can be a full path or a relative path from the server's data directory.

Required if "localTableName" is not specified. string No limits

streams

.localOwnerName

The "localOwnerName" property specifies the account that owns the table on the FairCom MQ server that stores the stream's data change events.

Optional with default of the "defaultOwnerName" value that is set during "createSession". If no default is set during "createSession", then "admin" is used. string 1 to 64 bytes

streams

.localTableName

The "localTableName" property specifies the name of the table on the FairCom MQ server that stores the stream's data change events.

Required if "localDataFilePath" is not specified string 1 to 64 bytes

streams

.recordFilter

The "recordFilter" property specifies a FairCom expression that must match a record's field values before the record is included as a data change event.

Optional with default of "" string 1 to 65,000 bytes

streams

.recordFormat

The "recordFormat" property includes the record's value in the data change event as a binary-encoded string or individual field values.

Optional with default of "fields" string enum

"fields"

"buffer"

streams

.revealAfterValueOnFilteredDelete

The "revealAfterValueOnFilteredDelete" property includes the "afterValue" property in the notification message when set to true. When "recordFilter" filters out a record based on field values, an update causes a deleted change event to occur when its field values no longer match the filter. The "afterValue" property, when present, reveals the new field values and leaks information.

Optional with default of false Boolean

true

false

streams

.revealBeforeValueOnFilteredInsert

The "revealBeforeValueOnFilteredInsert" property includes the "beforeValue" property in the notification message when set to true. When "recordFilter" filters out a record based on field values, an update causes an insert change event when previous field values did not match the filter but now match it. The "beforeValue" property, when present, reveals the old field values and leaks information.

Optional with default of false Boolean

true

false

streams

.sourceDatabaseName

The "sourceDatabaseName" property specifies the database name of the table on the FairCom DB or RTG server that generates the stream's data change events.

Optional with default of the "defaultDatabaseName" value that is set during "createSession". If no default is set during "createSession", then "faircom" is used. string 1 to 64 bytes

streams

.sourceDataFilePath

The "sourceDataFilePath" property specifies the data file path of the table on the FairCom DB or RTG server that generates the stream's data change events. It can be a full path or a relative path from the server's data directory.

Required - if "sourceTableName" is not specified string No limits

streams

.sourceOwnerName

The "sourceOwnerName" property specifies the account that owns the table on the FairCom DB or RTG server that generates the stream's data change events.

Optional with default of the "defaultOwnerName" value that is set during "createSession". If no default is set during "createSession", then "admin" is used. string 1 to 64 bytes

streams

.sourceTableName

The "sourceTableName" property specifies the name of the table on the FairCom DB or RTG server that generates the stream's data change events.

Required if "sourceDataFilePath" is not specified string 1 to 64 bytes

streams

.tagChanges

The "tagChanges" property specifies when to add "changed": true to field objects in the data change event to indicate when a field changed value.

Optional with default of "forEachField" string enum

"forEachField"

"forPrimaryKeyFields"

"never"

streams

.topic

The "topic" property specifies the unique, user-defined name for the topic.

  • The "topic" name is typically less than 150 characters.
  • The "topic" name will be used in all subsequent references to the topic, such as when re-configuring or deleting the topic.

 

Optional with default of "" string 1 to 65,500 bytes

streams

.triggers

The "triggers" property specifies a list of events on a table that create data change events.

Optional with default of [ "insert", "update", "delete" ] array of enum strings

"delete"

"insert"

"update"

 

Response properties ("result")

Property Description Type Limits (inclusive)

streams

The "streams" property specifies an array of stream definitions. Array of stream definitions 1 or more stream definitions

streams

.dataChangeStreamFirstStartTimestamp

The "dataChangeStreamFirstStartTimestamp" property specifies the UTC date and time in ISO-8601 format when the stream first started.

 

string timestamp "2025-06-07T12:23:19.275"

streams

.dataChangeStreamLastPausedTimestamp

The "dataChangeStreamLastPausedTimestamp" property specifies the UTC date and time in ISO-8601 format when the stream last paused.

 

string timestamp "2025-06-07T12:23:19.275"

streams

.dataChangeStreamLastStartTimestamp

The "dataChangeStreamLastStartTimestamp" property specifies the UTC date and time in ISO-8601 format when the stream last restarted.

 

string timestamp "2025-06-07T12:23:19.275"

streams

.dataChangeStreamStatus

The "dataChangeStreamStatus" property specifies the status of the data change stream. It may specify any of the following states: string enum

"failed"

"initializing"

"jumpstarting"

"paused"

"pausing"

"running"

"scheduled"

streams

.description

The "description" property provides additional information about an object, such as a label or thing. You can use it as internal or external documentation of a label's meaning, purpose, and usage.

Markdown is a good language for formatting description text. You must ensure the text is compatible with a JSON string. For example, you must escape a double quote character using the backslash character:  \".

 

In the Thing API, It defaults to "unknown" and is a string from 1 to 512 bytes. You cannot use it for lookups and filtering.

 

string 1 to 65,500 bytes

streams

.fixedOutput

"fixedOutput" includes all properties in a data change event when true. Boolean

true

false

streams

.includedFields

The "includedFields" property specifies the source table fields in the data change event or all fields when empty.

array of strings 0 or more strings

streams

.includeExistingRecordsFilter

The "includeExistingRecordsFilter" property returns streams that synchronize existing records if true.

Boolean

true

false

streams

.includeMetadata

The "includeMetadata" property adds user-defined properties to each data change event.

array of metadata objects
[
  {
    "propertyPath": "myPath",
    "propertyValue": "myValue"
  }
]

streams

.includePrimaryKey

The "includePrimaryKey" property specifies when to add the "pk" property to the data change event's "fields" object to indicate the field's position in the primary key.

string

"forEachField"

"forPrimaryKeyFields"

"never"

streams

.localDatabaseName

The "localDatabaseName" property specifies the database name of the table on the FairCom MQ server that stores the stream's data change events.

string 1 to 64 bytes

streams

.localDataFilePath

The "localDataFilePath" property specifies the data file path of the table on the FairCom MQ server that stores the stream's data change events. It can be a full path or a relative path from the server's data directory.

string No limits

streams

.localOwnerName

The "localOwnerName" property specifies the account that owns the table on the FairCom MQ server that stores the stream's data change events.

string 1 to 64 bytes

streams

.localTableName

The "localTableName" specifies the name of the table on the FairCom MQ server that stores the stream's data change events.

string 1 to 64 bytes

streams

.recordFilter

The "recordFilter" property specifies a FairCom expression that must match a record's field values before the record is included as a data change event.

string 1 to 65,000 bytes

streams

.recordFormat

The "recordFormat" property includes the record's value in the data change event as a binary-encoded string or individual field values.

string

"fields"

"buffer"

streams

.revealAfterValueOnFilteredDelete

The "revealAfterValueOnFilteredDelete" property includes the "afterValue" property in the notification message when true. When "recordFilter" filters out a record based on field values, an update causes a deleted change event to occur when its field values no longer match the filter. The "afterValue" property, when present, reveals the new field values and leaks information.

Boolean

true

false

streams

.revealBeforeValueOnFilteredInsert

The "revealBeforeValueOnFilteredInsert" property includes the "beforeValue" property in the notification message when true. When "recordFilter" filters out a record based on field values, an update causes an insert change event when previous field values did not match the filter but now match it. The "beforeValue" property, when present, reveals the old field values and leaks information.

Boolean

true

false

streams

.sourceDatabaseName

The "sourceDatabaseName" property specifies the database name of the table on the FairCom DB or RTG server that generates the stream's data change events.

string 1 to 64 bytes

streams

.sourceDataFilePath

The "sourceDataFilePath" property specifies the data file path of the table on the FairCom DB or RTG server that generates the stream's data change events. It can be a full path or a relative path from the server's data directory.

string No limits

streams

.sourceOwnerName

The "sourceOwnerName" property specifies the account that owns the table on the FairCom DB or RTG server that generates the stream's data change events.

string 1 to 64 bytes

streams

.sourceTableName

The "sourceTableName" property specifies the name of the table on the FairCom DB or RTG server that generates the stream's data change events.

string 1 to 64 bytes

streams

.streamingConnection

The "streamingConnection" property is an object that contains the current values of the streaming connection object that connects FairCom MQ to the source FairCom DB or RTG server.

object
"streamingConnection": {
  "sourceServerName": "FAIRCOMS",
  "sourceHostname": "10.70.13.112",
  "sourcePort": 5597,
  "sourceUsername": "ADMIN",
  "sourcePassword": "ADMIN",
  "maxSecondsBeforeConnectingToNextServer": 15,
  "tls": {
    "enabled": true,
    "caCertificateFilename": "ca.crt",
    "allowedCipherSuites": "",
    "clientCertificateEnabled": true,
    "clientCertificateFilename": "admin_client.crt",
    "clientPrivateKeyFilename": "admin_client.key"
  },
  "metadata": {}
}

streams

streamingConnection

.maxSecondsBeforeConnectingToNextServer

The "maxSecondsBeforeConnectingToNextServer" property specifies the maximum number of seconds the server will attempt to reconnect to the existing server before it attempts to connect to the next server in the "sourceServers" list. int32 1 to 65535

streams

streamingConnection

.metadata

The "metadata" field allows a customer to add a JSON value to an object such as a label or thing. It is typically a JSON object, but may be any JSON value. Its purpose is to provide additional metadata about the label, such as translations in multiple languages, historical value changes, etc. The "alterLabel" action can only replace this value with a new JSON value.

JSON 0 to 65,500 bytes

streams

streamingConnection

.sourceHostname

The "sourceHostname" property specifies a unique host name or TCP/IP address of a FairCom DB or RTG server. string 1 to 255 bytes

streams

streamingConnection

.sourcePassword

The "sourcePassword" property specifies the login password of a FairCom DB or RTG server. string 1 to 128 bytes

streams

streamingConnection

.sourcePort

The "sourcePort" property specifies the ISAM TCP/IP port of a FairCom DB or RTG server. int16 1 to 65535

streams

streamingConnection

.sourceServerName

The "sourceServerName" property specifies the server name of a FairCom DB or RTG server. It is the name specified by the SERVER_NAME keyword defined in the target server's configuration file, ctsrvr.cfg. The server name used by most FairCom DB and RTG servers is "FAIRCOMS". This property is required if the "sourceHostname" is not defined. string 1 to 255 bytes

streams

streamingConnection

.sourceUsername

The "sourceUsername" property specifies the name of a FairCom DB or RTG server. string 1 to 64 bytes

streams

streamingConnection

.tls

The "tls" property is a JSON object that defines the public server certificate filename, the private key filename, the certificate authority filename, the allowed cipher suites, and whether the client certificate is required. 

 

"tls": { "serverCertificateFilename": "server.crt", "privateKeyFilename": "server.key", "caCertificateFilename": "ca.crt", "allowedCipherSuites": "AES256-SHA256" "requireClientCertificate": "true"
}
object

"allowedCipherSuites"

"caCertificateFilename"

"certificateFilename"

"privateKeyFilename"

streams

streamingConnection

tls

.allowedCipherSuites

The "allowedCiperSuites" property specifies an array of ciphers that the server will accept for communications with clients. It defaults to an empty string.

  • It specifies the encryption ciphers that are allowed to be used for encrypting a TLS (SSL) connection.
  • A client is allowed to connect to the server only if it uses one of the ciphers in this list.
  • The default setting of an empty string supports industry-standard secure connections.
  • The default value requires clients to use full AES 256-bit encryption when they talk to the server.
  • If a client cannot support AES 256-bit encryption, a lower encryption level should be added to the list.
    • This is undesirable because malicious users will attempt to connect at the lowest possible level so they can harm the system (for more information, see ciphers main page at OPENSSL.org.
  • Example settings:
    • Maximally secure example:
      • This example only allows clients to connect securely.
      • ["AES256-SHA256", "AES256-GCM-SHA384", "DHE-RSA-AES256-SHA256"]
        
    • Minimally secure example with the broadest client support:
      • ["!aNULL", "!eNULL", "!SSLv2", "!LOW", "!EXP", "!RC4", "!MD5", "@STRENGTH"]
        
    • Insecure example allowing clients to connect using any level of security from none to the maximal:
      • ["ALL", "!aNULL", "!eNULL", "!SSLv2", "!LOW", "!EXP", "!RC4", "!MD5", "@STRENGTH"]
        
  • Add @STRENGTH to the end of the list to force the server to prioritize the strongest algorithms first. 
  • Place an exclamation point before a cipher to disable it. 
string No limits

streams

streamingConnection

tls

.caCertificateFilename

The "caCertificateFilename" property specifies the name and optional path of the CA certificate file (such as "ca.pem").

  • You must include "caCertificateFilename" to allow clients to use X509 certificates to authenticate with the server.
  • The certificate authorities file contains the list of certificate authorities the server uses to validate X509 certificates that clients present as authentication credentials.
  • In order for an X509 certificate to be accepted by the server, the certificate must be signed by a certificate authority that is present in the certificate authorities certificate file.
string No limits

streams

streamingConnection

tls

.clientCertificateEnabled

The "clientCertificateEnabled" property enables client certificate authentication if true. The target FairCom DB or RTG server must be configured to accept client certificates. boolean

true

false

streams

streamingConnection

tls

.clientCertificateFilename

The "clientCertificateFilename" property specifies the name of the client certificate file. 

string No limits

streams

streamingConnection

tls

.clientPrivateKeyFilename

The "clientPrivateKeyFilename" property specifies the name of the client private key file.

string No limits

streams

streamingConnection

tls

.enabled

The "enabled" property determines whether or not the specified feature is enabled. The example below enables the TLS feature.

        "tls": {
          "enabled": true,
          "caCertificateFilename": "ca.crt",
          "allowedCipherSuites": "",
          
          "clientCertificateEnabled": true,
          "clientCertificateFilename": "admin_client.crt",
          "clientPrivateKeyFilename": "admin_client.key"
        }
Boolean

true

false

streams

.streamingConnectionName

The "streamingConnectionName" property specifies a unique, user-defined name for a streaming connection. The API uses it to identify streaming connections and to connect a data change stream to a FairCom DB or RTG server. string 1 to 64 characters

streams

.tagChanges

The "tagChanges" property specifies when to add "changed": true to field objects in the data change event to indicate when a field changed value.

string

"forEachField"

"forPrimaryKeyFields"

"never"

streams

.topic

The "topic" property specifies the MQTT topic on the external broker to which messages will be forwarded.

string A topic name between 1 and 65,500 bytes

streams

.triggers

The "triggers" property specifies a list of events on a table that create data change events. array of enum strings

"delete"

"insert"

"update"