Data Stream
  • 04 Oct 2023
  • 5 Minutes to read

    Data Stream


      Article Summary

      Insider Data Stream allows you to listen to real-time default and custom event occurrences, and user attribute changes. This capability may serve you in different use cases as follows:

      • It feeds your analytics tools as it gets the event data when Insider collects it. 
      • You can call your users via your call centers when their attribute value changes from a specific value to another specific value.
      • You can reach out your customers via any channel when they return an item.

      Insider can also send the wanted user attributes in both event and attribute streams. This means that you can receive an attribute value along with an event in the hook, or other attributes of a user along with the given attribute. For example, you can get the subscription type attribute along with the newsletter subscription event.

      With Insider's Data Stream, you can utilize webhooks to receive real-time data for the following two subscriptions:

      A data stream can be set up for all events and attribute changes on the Event and Attributes page. However, we do not suggest installing it for all of them because the data stream sends almost real-time data. When installed for all values or too many values, your endpoint may not be able to handle it. Therefore, when you set up a data stream for the attributes or events you genuinely want to follow, it will create a more manageable structure.

      What is required for the Data Stream?

      To start receiving data stream for attributes and events, you should share the following details with your Insider Technical Account Manager:

      A webhook URL

      You should provide a webhook URL to receive the data stream. You can provide the same webhook URL or separate URLs for both events and attributes. It is recommended to use separate webhook URLs for events and attributes for easier operations.

      Header(s)

      You can also set your webhook URL(s) with header(s). In this case, you should share the header(s) with your Insider Technical Account Manager as well.

      Attribute Stream

      The given webhook URL will receive the data stream of the attribute(s) that you subscribe to in the following cases:

      • When an attribute gets any value: E.g. the membership attribute gets the loyal value.
      • When an attribute gets a specific value: E.g. the membership attribute gets the previously specified VIP value.
      • When the value of an attribute changes from a given value to another given value: E.g. the membership attribute value changes from loyal to VIP.

      In any of the given cases above, Insider sends a payload to the given webhook URL with the following details:

      ParameterDefinition
      attrsThe attributes object that contains the wanted attributes along with the subscribed attribute
      hook_idThe ID of the subscribed webhook
      insider_idThe Insider ID/Profile ID of the user that is shared in the payload
      nameThe webhook name if applicable. You can name your webhooks, and share this name with Insider.
      partnerThis is your partner name. Navigate to Inone > Inone Settings > Account Preferences to copy your partner name. The partner name should be lowercase.
      timestampThe time when Insider receives the respective data that triggers the webhook (epoch time)
      triggersThe array that contains the case(s) that triggers the attribute stream

      Webhook Sample

      {
        "attrs": {
          "age": 20,
          "email_optin": false,
          "name": "John",
          "surname": "Doe"
        },
        "hook_id": 3,
        "insider_id": "a1b2c3d4e5f6",
        "name": "My attribute webhook",
        "partner": "poshstreet",
        "timestamp": 1623182670,
        "triggers": [
          {
            "current": {
              "value": "loyal"
            },
            "key": "membership",
            "next": {
              "value": "VIP"
            }
          }
        ]
      }
      

      Event Stream

      Insider can listen to any default or custom user event, and can send the corresponding webhook to the corresponding hook address when a user performs an event. Some event examples are purchase, product detail page view, form submit, etc. 

      You can also receive webhooks with a filter applied on the events. For example, you may want to listen to the purchase event whose product name event parameter has jeans as the value. In this case, Insider will not send webhooks when the purchase event is performed unless it matches the value criteria.

      For the event webhooks, you can also define the parameters to be sent within the webhook. For example, the form_submit event may include 5 event parameters, but you may want only 3 of them. You can specify the other 2 to be excluded from the webhook.

      With or without the filters applied, Insider sends a payload to the given webhook URL with the following details:

      ParameterDefinition
      attrsThe attributes object that contains the wanted attributes along with the subscribed event
      event_nameThe name of the subscribed event
      event_parametersThe event parameters of the subscribed event
      hook_idThe ID of the subscribed webhook
      insider_idThe Insider ID/Profile ID of the user that is shared in the payload
      nameThe webhook name if applicable. You can name your webhooks, and share this name with Insider.
      partnerThis is your partner name. Navigate to Inone > Inone Settings > Account Preferences to copy your partner name. The partner name should be lowercase.
      timestampThe time when Insider receives the respective data that triggers the webhook (epoch time)
      triggersThe array that contains the case(s) that triggers the event stream

      Webhook Sample

      {
        "attrs": {
          "age": 25,
          "email_optin": true,
          "name": "John",
          "sms_optin": false
        },
        "event_name": "purchase",
        "event_parameters": {
          "unit_price": 10,
          "url": "https://mybrand.com/blue-jeans"
        },
        "hook_id": 12,
        "insider_id": "1a2b3c4d5e6f",
        "name": "My event webhook",
        "partner": "mybrand",
        "timestamp": 1623187855,
        "triggers": [
          {
            "key": "url",
            "next": {
              "value": "https://mybrand.com/blue-jeans"
            }
          }
        ]
      }
      

      Managing Data Stream Transmission

      Insider has a retry mechanism designed to prevent data streams from being transmitted in the event of errors or instantaneous problems with the endpoint. If a request encounters a timeout (set at 15 seconds), Insider triggers a retry process for the specific data stream. However, if a response is received, no further retries are attempted.

      Insider's retry strategy includes a maximum of 10 attempts, following an exponential backoff duration pattern:

      • First Retry: 2 minutes after the initial attempt
      • Second Retry: 4 minutes after the first retry
      • Third Retry: 8 minutes after the second retry
      • Subsequent Retries: The waiting time between retries continues to double, adhering to the exponential backoff strategy.

      After the 10th retry, the system ceases any further attempts to re-send the request. 

      It's important to note that Insider applies this retry mechanism not only for timeouts but also for other error codes such as 429, 5xx, and more, ensuring a comprehensive and resilient approach to data stream transmission.

      Your title goes here
      You can apply IP Restriction for data stream payloads arriving at your endpoint. As Insider, we send data streams from certain IP addresses. You can reach out to your Technical Account Manager to learn these IP addresses.
      Your title goes here
      More than one attribute change trigger can be added to a data stream payload. However, only one event trigger can be added to a payload for event data stream.

      Was this article helpful?

      ESC

      Eddy, a super-smart generative AI, opening up ways to have tailored queries and responses