Event Logging

What is Event Logging?

Event logging is the process of communicating the actions your Connector performs to the SyncHive platform. This may involve simple actions such as receiving data from an external system and preparing it to send to SyncHive, or more complex actions like sending bulk records to an external system and notifying SyncHive once the process is completed.

Why Should You Event Log?

Event logging ensures that the SyncHive platform is aware of the work your Connector is performing. This information provides end-users with accurate and up-to-date visibility of their SyncHive instance, enabling them to monitor activities and resolve issues quickly when they arise.

When Should You Event Log?

We expect Connector developers to provide accurate event logs to SyncHive in the following scenarios:

Receiving data from an external system.
Successfully saving data to an external system in asynchronous scenarios.
Any time an error or pertinent warning occurs.

How to Event Log

There are two primary scenarios to consider when event logging to SyncHive: synchronous processing and asynchronous processing.

Synchronous Processing

In synchronous processing, work is predictably done in sequence from start to finish. Consider the diagrams below for guidance on when to provide event logs.

Inbound Example

Using the example of receiving a webhook from an external system, a Connector is expected to provide an event log when it begins to process a message and if it encounters any errors or warnings while doing so. Since the final step is for a Connector to send a message to SyncHive, an additional event log is not required.

Outbound Example

When receiving a message from SyncHive, a Connector is not required to provide additional event logs. The Connector's response to SyncHive upon receiving a message provides all the necessary information. For more details, see the section on Receiving Messages from SyncHive: Responding to SyncHive.

Asynchronous Processing

In asynchronous processing, a Connector does not immediately process a message received from SyncHive. Instead, it processes the message later. During this time, the Connector must maintain communication with SyncHive and notify it once the message has been processed.

Batch Processing Example

For batch processing multiple messages received individually from SyncHive, a Connector should generate event logs for each message. These logs should indicate whether each message was successfully or unsuccessfully processed.

Event Log Key Concepts

When constructing an event log payload, there are several key concepts to learn:

Lifecycle Stage

A lifecycle stage indicates a particular moment in the SyncHive message lifecycle. When sending an event log, you must specify the corresponding stage within the message lifecycle.

Stage	Key	Notes
1. Connector collect inbound data	INBOUND_1
2. Connector assemble inbound message	INBOUND_2
3. Connector transfer inbound message	INBOUND_3
4. Platform receive inbound message	INBOUND_4	For platform use only
5. Platform store inbound message	INBOUND_5	For platform use only
6. Platform generate outbound message(s)	OUTBOUND_1	For platform use only
7. Connector receive outbound message	OUTBOUND_2
8. Connector assemble outbound data	OUTBOUND_3
9. Connector deliver outbound data	OUTBOUND_4
10. Platform finalize outbound delivery	OUTBOUND_5	For platform use only

Event Code

Event codes are alphanumeric representations of the content within an event log. These codes are logically organized into distinct ranges.

Event Code Range

Code Range	Category	Description
1000-1999	Data	Retrieving or storing data
2000-2999	Logic	Business logic or data transformation
3000-3999	Filter	Filtering messages
4000-4999	Contact	Communicating with an external system
5000-5999	Exchange	Sending or receiving messages
0000	Internal Server Issue	Fallback "catch-all" codes for unexpected issues

Event Code Glossary

Refer to the glossary below to select an appropriate event code when creating your event log. Note that a suffix of the event level is appended to the code.

Code	Level	Description
1000N	INFO	Data saved
1000W	WARNING	Could not save data
1001E	ERROR	Data rejected by external system
1001W	WARNING	Key data is missing
1002E	ERROR	Key data is missing
1003E	ERROR	Duplicate data ID
1004E	ERROR	Invalid message
1004W	WARNING	Invalid message
1005E	ERROR	Data is missing
2000N	INFO	Message created
2001E	ERROR	Bad data received from external system
2002E	ERROR	Message transformation failed
3000N	INFO	Message filtered
3001N	INFO	Message skipped
3002E	ERROR	Invalid filter
4000N	INFO	Authenticated with SyncHive
4001N	INFO	Authenticated with external system
4002E	ERROR	Could not reach SyncHive
4003W	WARNING	Could not reach external system
4003E	ERROR	Could not reach external system
4004E	ERROR	Authentication to SyncHive failed
4004W	WARNING	Rate Limit Exceeded
4005E	ERROR	Authentication to external system failed
5000N	INFO	Message sent
5001N	INFO	Message received
5002E	ERROR	SyncHive rejected the message
5003E	ERROR	Data rejected by external system
5003N	INFO	Data polled
5004N	INFO	Data received
5005N	INFO	Data sent
5006W	WARNING	Unauthorized
5006E	ERROR	Unauthorized
5007W	WARNING	Connector unavailable
5007E	ERROR	Connector unavailable
5008W	WARNING	Platform unavailable
5008E	ERROR	Platform unavailable
0000E	ERROR	Error
0000W	WARNING	Warning

Correlation ID

The correlation ID is crucial for maintaining traceability of event logs and messages in SyncHive. The platform uses the correlation ID on messages received for event logging. To ensure that the event logs are linked together, the same correlation ID must be used when event logging in the connector.

See more: Correlation ID.

Writing a Clear and Informative Event Log

When composing the contents of an event log, prioritize clarity and relevance for the end-user. To ensure readability and prompt action, we recommend structuring your event log detail field as follows:

What state are we in?
How did it happen?
What can the end-user do about it?

Example of an Informative Event Log

Failed to send message to ShopifyNZ.

The HTTP API Key was not accepted.

Please provide a valid HTTP API Key to the ShopifyNZ Integration and try again.

Understanding Error Codes

When event logging with an error-level code to SyncHive, the associated message is marked as a terminal failure and will be displayed to end-users in the "Stuck" messages UI. This message must not be processed further by the Connector, so avoid providing any further event logs with new codes for that message.

For example, if a Connector is processing an event from an external system and encounters an issue that it will retry, it should use a warning-level event code. Only when the Connector has stopped retrying and reaches a point of terminal failure should it use an error code.

Event Log API

To send an event log to SyncHive, create a payload according to the schema below and call the following API:

PUThttps://{synchive_url}/data-service/v1/eventLog

Headers
"X-Shopify-Connector-Key": "9XnZr4JmPWTVHuyFBheg",
"limber-connector": "shopify_connector",
"Content-Type": "application/json"

Body
{
  "eventCode": "4005E",
  "direction": "OUTBOUND",
  "lifeCycleStage": "OUTBOUND_4",
  "outcome": "FAILED",
  "generatedAt": "2024-05-03 17:30:00.895 +1200",
  "detail":
    "Failed to send message to ShopifyNZ.

    The HTTP API Key was not accepted.

    Please provide a valid HTTP API Key to the ShopifyNZ Integration and try again.",
  "correlationId": "14cac731-6dd7-424c-bfbf-f0be0b07f8cf",
  "dataId": {
    "@type": "DataReference",
    "schemaName": "limber",
    "shapeName": "Product",
    "externalIdentities": [
      {
        "integrationKey": "ShopifyNZ",
        "externalId": "9217354452632"
      }
    ]
  },
  "integrationKey": "ShopifyNZ",
  "shapeName": "Product",
  "mode": "LIVE",
  "schemaName": "limber",
  "schemaVersion": "2.1.0",
}

Note

Event logs that have a "direction": "INBOUND" may use a "dataId" value that contains either an "externalIdentities" array (pictured above) or an "iri" value (pictured below).

Event logs that have a "direction": "OUTBOUND" must use a "dataId" value that contains an "iri". For example:

"dataId": {
  "@type": "DataReference",
  "schemaName": "limber",
  "shapeName": "Product",
  "iri": "https://schema.limbergraph.com/limber/Product#366821d6-a049-46e8-b73a-ea9ee7faea0d"
}

See more ID Resolution: Data References.

Schema
{
  eventCode: string,
    // Required
    // An event code that represents the content of this event log
    // Enum (see Event Code documentation below)

  direction: string,
    // Required
    // Direction of data flow. INBOUND for data flowing IN to SyncHive. OUTBOUND for data flowing OUT of SyncHive
    // Enum: [ INBOUND, OUTBOUND ]

  lifeCycleStage: string,
    // Required
    // The current stage of processing this event log relates to
    // Enum (see Lifecycle Stage documentation below)

  outcome: string,
    // Required
    // The outcome of this event
    // Enum: [ WAITING, FILTERED, RETRYING, FAILED, MESSAGE_STORED, MESSAGE_DELIVERED, PROCESSING ]

  generatedAt: string,
    // Optional
    // The date and time with an offset from UTC, represented as a string in the ISO 8601 format
    // Example: "2024-05-03 17:30:00.895 +1200"

  detail: string,
    // Optional
    // maxLength: 3000
    // Human readable data event log detail. Stored as Markdown

  technicalInfo: string,
    // Optional
    // maxLength: 2000
    // A technical message to help end-users resolve issues, typically specific to external systems (e.g. SAP S4/HANA), when more information is needed beyond the "detail" string above

  correlationId: string,
    // Required
    // Correlation ID for related message tracing

  dataId: string,
    // Required
    // A data ID block that associates this event log with data in SyncHive (see Data ID documentation below)

  integrationKey: string,
    // Required
    // Source Integration for the event log

  shapeName: string,
    // Required
    // The type of data this event log relates to (see Shape documentation below)

  mode: string,
    // Required
    // The mode of operation
    // Enum: [ LIVE, SANDBOX ]

  stream: string,
    // Optional
    // Advanced: used to let SyncHive know that this event is split into multiple parts.
    // Use a unique stream value for each part and ensure that you use the same dataId value for both

  schemaName: string,
    // Required
    // The name of the schema for the event log

  schemaVersion: string,
    // Optional
    // The version of the schema for the event log

  customData: JSON,
    // Optional
    // Custom data to attach to this event log

  linkIdentities: string,
    // Optional
    // Used to link identities in asynchronous scenarios (see Link Identities documentation below)
}

References: Event Code, Lifecycle Stage, Data ID, Shape, Link Identities.

What is Event Logging?​

Why Should You Event Log?​

When Should You Event Log?​

How to Event Log​

Synchronous Processing​

Inbound Example​

Outbound Example​

Asynchronous Processing​

Batch Processing Example​

Event Log Key Concepts​

Lifecycle Stage​

Event Code​

Event Code Range​

Event Code Glossary​

Correlation ID​

Writing a Clear and Informative Event Log​

Example of an Informative Event Log​

Understanding Error Codes​

Event Log API​

Note​