Build a Web Connector service

Web Connectors allow ConnectReport templates and report tasks to stream data from on-premises, public and private cloud data sources, allowing report authors to work with data from nearly any source.

To do this, a Web Connector interfaces with a Web Connector service, a self-hosted service responsible for handling data and visualization retrieval in the target data source in response to HTTP requests made by the ConnectReport report processing service.

Architecture

ConnectReport Web connector architecture

A web connector is comprised of a web connector service and the web connector itself, a point of integration between the web connector service and ConnectReport’s data interface.

The web connector service supporting a web connector should be solely accessible to ConnectReport - you should restrict it in the same way as you would a database. When ConnectReport passes along user context and filter criteria to the web connector endpoints, it is the responsibility of the web connector to account for these and enforce access control.

Routes

A web connector service responds to a documented set of API requests that come from the ConnectReport rendering engine. These routes are detailed below.

POST /getMetadata

Used to deliver metadata to the UI to support report authoring. Receives a User object indicating the current user. It should respond with a MetaDataResponse

Payload (JSON) 

{
  "user": {
    "username": "jsmith",
    "id": "848b53aa-258d-432f-82db-bbf63ee2c81b",
    "firstName": "John",
    "lastName": "Smith",
    "email": "john.smith@example.com"
  }
}

Response (JSON) 

{
  // Visualizations available to render
  "visualizations": [
    {
      // The name of the visualization
      "name": "Sales by Year",
      // An optional thumbnail image displayed in the insert data sidebar
      "thumbnail": "https://example.com/sales-by-year-thumbnail.png",
      // The identifier of the visualization
      "id": "sales"
    },
    {
      "name": "Revenue vs Profit",
      "thumbnail": "https://example.com/revenue-vs-profit-thumbnail.png",
      "id": "revenue"
    }
  ],
  // Dimensions available for analysis
  "dimensions": [
    {
      // Display name for the field
      "fieldName": "Year",
      // Field identifier or definition
      "fieldDef": "year"
    },
    {
      "fieldName": "Quarter",
      "fieldDef": "quarter"
    }
  ],
  // Measures available for analysis
  "measures": [
    {
      // Display name for the field
      "fieldName": "Sales",
      // Field identifier or definition
      "fieldDef": "sales"
    },
    {
      "fieldName": "Profit",
      "fieldDef": "profit"
    }
  ],
  // Tables available for analysis. 
  // Table fields cannot be combined with  dimensions or measures by users in the template editor
  // You should choose whether to return top level dimensions and measures or tables 
  "tables": [
    {
      "name": "Sales",
      "fields": [
        {
          // Display name for the field
          "fieldName": "Year",
          // Type of the field, "dimension" or "measure"
          "fieldType": "dimension",
          // Field identifier or definition
          "fieldDef": "Year"
        },
        {
          "fieldName": "Quarter",
          "fieldType": "dimension",
          "fieldDef": "Quarter"
        }
      ]
    },
    {
      "name": "Customers",
      "fields": [
        {
          // Display name for the field
          "fieldName": "Customer",
          // Type of the field, "dimension" or "measure"
          "fieldType": "dimension",
          // Field identifier or definition
          "fieldDef": "Customer"
        },
        {
          "fieldName": "State",
          "fieldType": "dimension",
          "fieldDef": "State"
        }
      ]
    }
  ],
  // Fields to filter on, made available in the filters UI 
  "filterFields": [
    {
      // Field identifier or definition. This is later used to retrieve field values
      "fieldDef": "Sale.Year",
      // Display name for the field
      "fieldName": "Sale Year"
    },
    {
      "fieldDef": "Customer.Region",
      "fieldName": "Customer Region"
    }
  ]
}

POST /getFieldValues

Used to retrieve list of fields values to filter on in UI. Receives a User object indicating the user running the report and a FieldValuesRequest options object. It should respond with a FieldValuesResponse

Payload (JSON) 

{
  "user": {
    "username": "jsmith",
    "id": "848b53aa-258d-432f-82db-bbf63ee2c81b",
    "firstName": "John",
    "lastName": "Smith",
    "email": "john.smith@example.com"
  },
  "options": {
    // The target field 
    "field": {
      // Field identifier or definition
      "fieldDef": "Sale.Year",
      // Display name for the field
      "fieldName": "Sale Year"
    };
    // The number of field values that should be returned (maximum)
    "height": 100,
    // The result row offset, used by the UI for lazy loading the fields
    "top": 50,
    // An optional string that may be sent indicating a field value search string
    "search": "2008"
  }
}

Response (JSON) 

{
  // List of values available to filter on 
  "fieldValues": [
    {
      // The text representation of the field value
      "text": "2020",
      // Optional numerical representation of field value
      "number": 2020
    },
    {
      "text": "2021",
      "number": 2021
    }
  ],
  "size": {
    "width": 1,
    "height": 2
  }
}

POST /getTable

Used to fulfill tabular data requests. Receives a User object indicating the user running the report and a TableRequest options object. It should respond with a TableResponse

Payload (JSON) 

{
  "user": {
    "username": "jsmith",
    "id": "848b53aa-258d-432f-82db-bbf63ee2c81b",
    "firstName": "John",
    "lastName": "Smith",
    "email": "john.smith@example.com"
  },
  "options": {
    // Optional table name metadata configured in the template
    "tableName": "Sales by Average Sale Amount",
    // The fields in the table request
    "fields": [
      {
        // Field identifier or definition
        "fieldDef": "Sale.Year",
        // Display name for the field
        "fieldName": "Sale Year"
        "fieldType": "dimension",
        "columnIndex": 0
      },
      {
        "fieldDef": "Avg(Sale.Amount)",
        "fieldName": "Average Sale Amount",
        "fieldType": "measure",
        "columnIndex": 1
      },
    ],
    // The expected height of the response (maximum)
    "height": 1000,
    // The result row offset
    "top": 500,
    // The filter criteria for the table request 
    "selections": [
      {
        // Field identifier or definition
        "fieldDef": "Sale.Year",
        // Display name for the field
        "fieldName": "Sale Year"
        // The values of the field to filter on
        "fieldValues": [
          {
            "text": "2022"
          },
          {
            "text": "2021"
          },
          {
            "text": "2019"
          }
        ]
      }
    ]
  }
}

Response (JSON) 

{
  // Matrix of table rows. Each table row is an array of values.
  "table": [
    [
      {
        // The text representation of the value
        "text": "2019",
        // The numerical representation of the value (used in certain export formats)
        "number": 2019
      },
      {
        "text": "$140.30",
        "number": 140.30
      }
    ],
    [
      {
        "text": "2021",
        "number": 2021
      },
      {
        "text": "$234.11",
        "number": 234.11
      }
    ],
    [
      {
        "text": "2022",
        "number": 2022
      },
      {
        "text": "$310.43",
        "number": 310.43
      }
    ]
  ],
  // The size of the response
  "size": {
    // Number of columns
    "width": 2,
    // Number of rows
    "height": 3
  }
}

GET /loadVisualization

Used to load external visualizations. Upon rendering a visualization associated with a web connector, ConnectReport opens a headless browser session to an HTML page hosted by the web connector that is responsible for rendering visualizations. ConnectReport passes along several query string parameters documented below which may be used to identify the appropriate visualization to load and the filters that must be applied, along with other useful metadata. To support this, the web connector service must handle a request to the path /loadVisualization and return an HTML response. See example loadVisualization handler.

The loadVisualization page receives the following URL parameters, which can be accessed from the page’s JavaScript context:

  • docId: the ID of the web connector
  • vizId: the visualization ID configured in the template. You can use this to determine the appropriate visualization to render.
  • selections: the effective selections for the visualization. This should be handled to appropriately filter the visualization data
  • width: the intended width of the visualization. The browser context will also have this viewport width
  • height: the intended height of the visualization. The browser context will also have this viewport width

Additionally, the loadVisualization page is automatically injected with two callback functions - One of these functions must be called before the visualization timeout for the visualization to succeed or fail. These functions are available in the loadVisualization page’s JavaScript context:

  • completedVizLoad(): used to indicate to ConnectReport that the visualization is completely rendered
  • vizLoadError({ message: string, disableRetry?: boolean }): used to ConnectReport that the visualization has encountered an error. The error argument is surfaced in the final output. disableRetry is false by default, which enables the visualization processing service to retry rendering the visualization using exponential backoff. If it is set to true, retries are disabled.

The loadVisualization page is also automatically configured to attach an X-CR-USER header to every request it makes within the page context. This allows you to identify the user in upstream requests. Services the loadVisualization page accesses that rely on this header should be appropriately restricted.

Errors

Each Web Connector HTTP route can respond with errors that are surfaced to report authors in the Template Editor, content administrators through the Management Console’s report task view, and administrators through the server logs.

To respond with an error, the HTTP route handler may return a valid HTTP 4xx or 5xx status code and JSON response indicating error details:

{
  // The error message
  "message": "Example error message",
  // Optional. Default false. Indicates whether or not the 
  // report processing service should retry this request with exponential backoff
  "disableRetry": false
}

Note that the GET /loadVisualization front-end may call the injected vizLoadError({ message: string, disableRetry?: boolean }) callback with an error object of the same shape as the above.

Example Web Connector

An example Web Connector is made available on GitHub.

Run the example Web Connector

  1. Run the server
git clone https://github.com/connectreport/web-connector-example.git
cd ./web-connector-example
npm i 
npm run verbose
  1. Create a web connector in ConnectReport
  2. The ConnectReport backend must be able to address your web connector service instance. You can use a tool like ngrok to quickly host your web connector service instance from your local machine for development or deploy it using your preferred cloud provider. Once hosted or tunneled via ngrok, update the base URL of your web connector.
  3. Create a template that utilizes the web connector and observe the requests logged by the example connector.

Deployment

Once developed, the Web Connector must be deployed and accessible over HTTP from the ConnectReport server backend. Information on configuring the Web Connector is available in Configuring Web Connector connections.