How to integrate an application with Habit Analytics

Connecting people and machines

Habit Analytics' platform (muzzley) aims at efficiently and effectively connect, in real-time, people and machines, allowing end-users and software programs to establish communication channels and exchange information or actionable requests. It's stated software programs and not devices, because Habit doesn't restrict the ontology used within the established communication channels, although IoT devices are widely supported and the most common integration within Habit.

In order to be able to act as an intervenient in the Habit Analytics platform, 2 things are required:

• intervenient credentials, generated by the Habit Analytics platform

• the hability to interact through one of the communication protocols:

  • HTTP

  • MQTT

Once these requirements are established, the intervenient will be able to (Figure 1):

1. interact with a RESTful API to setup, manage and access it's own and other intervenients data.

2. use an MQTT infrastructure to establish real-time communication with other intervenients

Figure 1 - interacting with Muzzley - zoom in

There are the following types of intervenients, each with a set of permissions and allowed types of interactions:

• user: represents the end-user of the platform

• application: represents the software program that interacts with the platform on behalf of the user

• device: represents things that the user wishes to interact with and connects directly with the platform

• manager: represents the software program that interacts with the platform on behalf of devices hosted in and proxied by the manufacturer cloud

In this tutorial we will focus, mainly, on application.

Meaningful communication (Specification)

Having meaningful communication between intervenients means to empower interactions with the type of structured data that allows efficient and effective data analyses.

In order to be able to achieve this level of analysis without restricting the structure of data used, the specification of ontology schemas is mandatory and each communication channel must relate to and uphold a given schema.

This allows data observers to process the metadata associated with the ontology schema, understand the communication between intervenients and produce relevant additional knowledge and higher levels of data abstraction.

Each low-level communication channel is managed by an entity denoted channel. A channel is composed of components and/or properties (Figure 2).

Figure 2 - channel structure - zoom in

Channels are entities that represent the communication channels opened between intervenients. They are unique and allow the setup, configuration and management of low-level communication channels, regarding access permissions, message schema validation and message routing and broadcasting.

Components are containers that will hold other components and/or properties, with no special characterisation regarding data, just the specification of usage categories. Components are crucial to contextualize and organize properties (like folders in a filesystem).

Properties are the entities that represent data being sent over the communication channels, between intervenients. Properties may represent actions or just storage records. Properties are characterised by a data type, unit system, unit type, usage categories and permission set.

In order to characterize channels, components and properties, tools to specify ontology schemas, denoted specs, are provided.

Specs are schema specifications for channels, components and properties, There are 3 types of specs channel specs, component specs and property specs each one related to each level of the communication channel hierarchy (Figure 3).

Figure 3 - specs and channels - zoom in

Each spec schema may be associated with one or more templates, characterising a given brand, model and manufacturer, denoted channel template (Figure 4).

Figure 4 - channel templates - zoom in

The channel template entity is composed by the brand name, model, manufacturer, descriptive texts and images. Each channel must be associate to a channel template (Figure 5). The channel is created as an instance of a channel template.

Figure 5 - channels as template instances - zoom in

After specs and templates are in place, a channel may be generated based on a channel template. The generated channel will have a hierarchy mirrored from the underlying spec hierarchy. Each channel part may be configured and accessed via the respective topic:

• /v3/channels/{channel_id} to access the channel configuration

• /v3/channels/{channel_id}/components/{id or namespace} to access the component configuration

• /v3/channels/{channel_id}/components/{id or namespace}/property/{id or namespace} to access the property configuration or for engaging in interactions

For instance, given the examples in the above figures, one could send a message to another intervenient by publishing to /v3/channels/be5d4510-94a7-11e7-b1bf-c33cf9be2246/components/thermostat/properties/mode

Terminology

• intervenient: every software program that hold a set of credentials and is able to interact through one of the allowed protocols

• user: intervenient that represents the end-user of the platform

• application: intervenient that represents the software program that interacts with the platform on behalf of the user

• device: intervenient that represents things that the user wishes to interact with and connects directly with the platform

• manager: intervenient that represents the software program that interacts with the platform on behalf of devices hosted in and proxied by the manufacturer cloud

• channel: representation on the communication channel between two or more intervenients, used to exchange messages matching the underlying ontology schema

• component: parts of the intervenients channels, used to organize sets of properties

• property: entities that represent data being sent over the communication channels

• channel spec: schema specification for a channel

• component spec: schema specification for a component

• property spec: schema specification for a property

• channel template: template for characterising a communication channel for a brand / model / manufacturer

Accessing the API

Authentication with OAuth 2.0

In order to interact with the Habit Analytics API, clients must authenticate using the OAuth 2.0 protocol, in particular the Client credentials flow. This call must be made using the /v3/auth/authorize endpoint, passing pre-generated credentials, called client_id and client_secret.

Other parameters may / must also be provided, being:

• response_type, indicating the flow type, in this case client_credentials

• scope, indicating the requested permissions, in this case application

• redirect_uri, in case the clients needs to be redirect to a hosted callback

• state, a value to be passed as part of the authorization result

The response will be populated with, at least, 4 main attributes:

• access_token, the actual token to be used in subsequent calls

• expires, the validity of the returned tokens

• refresh_token, for refreshing this information, before expires is reached

• endpoints, an object containing the HTTP and MQTT base URLs assigned to the requesting client

All subsequent calls must be directed to the base URL provided in endpoints.http and an Authorization header must be added to every API call, in the form:

    Authorization: Bearer {access_token}

Request parameters

PARAMETER	TYPE	DESCRIPTION
client_id	uuid	the client unique identifier, provided by the Habit Analytics Selfcare
client_secret	string	automatically generated hash, provide by the Habit Analytics Selfcare
redirect_uri	uri	URI to be provided to the user-agent as part of a 303 or 307 HTTP response
response_type	string	the OAuth 2.0 flow to be used
scope	string	a comma separated string of permissions to be associated with the token
state	string	a general purpose string that will be forwarded and added to the flow final result

Response fields

FIELD	TYPE	DESCRIPTION
access_token	string	the token to be used in subsequent calls to the API as part of an HTTP Authorization header
client_id	uuid	the client unique identifier
code	string	an exchange code, for usage with code flow requests
endpoints	object	HTTP and MQTT base URLs assigned to the requesting client
expires	timestamp	the expiration date for the provided tokens
grant_type	string	the type of grant flow used to generate the provided tokens
refresh_token	string	a token used with /v3/auth/exchange to retrieve fresh tokens, without the re-executing the entire flow
scope	array	granted permissions associated with the provided tokens
state	string	general purpose string forwarded by the client in the first flow call

Call using POST method

• Request (application/json)

  • Body

      {
              "client_id": "bef41164-90c6-11e7-a25d-97266664a105",
              "client_secret": "CLIENT_SECRET",
              "response_type": "client_credentials",
              "scope" : "application",
              "state":"active"
      }

• Response 200 (application/json)

    {
            "access_token":"{access_token}",
            "client_id": "{client-id}",
            "code":"{code}",
            "endpoints":{
                    "http":"https:\/\/api.platform.muzzley.com",
                    "mqtt":"mqtts:\/\/api.platform.muzzley.com:8881"
            },
            "expires":"2017-12-03T02:08:01.582+0000",
            "grant_type":"client_credentials",
            "refresh_token":"{refresh_token}",
            "scope":[
                    "application"
            ]
    }

Call using GET method

• Parameters

  • client_id (required, string)

  • client_secret (required, string)

  • response_type (required, string)

  • scope (required, string)

  • redirect_uri (optional, string)

  • state (optional, string)

• Request (application/json)

• Response 307

  • Headers

      Location: https://mydomain.com/drop_credentials?expires=2017-12-03T02:08:01.582+0000&refresh_token=l8puku66syp5llktgrtrrxgw6vqc8hm109yfsf3wcm1t2dpkzoxs6na3na0zzc2q&access_token=pi34fshyb19niugymus2skd174nmp2z0j1nvmegaxz4ji0zfwjxolee74ydrv2x5s8zfly81hzh26v6295e0wxystsrb936l2xyr8w4t2grdxxo0pvz7l1lzz5298k5j

Renew authorization

Before the expiration timestamp returned in expires, the intervenient should renew the access token by calling /v3/auth/exchange endpoint, passing the refresh_token return by the call to /v3/auth/authorize.

The return will be exactly the same as return by /v3/auth/authorize.

Request parameters

PARAMETER	TYPE	DESCRIPTION
client_id	uuid	the client unique identifier, provided by the Habit Analytics Selfcare
redirect_uri	uri	URI to be provided to the user-agent as part of a 303 or 307 HTTP response
refresh_token	string	the token provided as refresh_token in a previous call to /v3/auth/authorize
grant_type	string	the type of grant flow used to generate the provided tokens

Response fields

FIELD	TYPE	DESCRIPTION
access_token	string	the token to be used in subsequent calls to the API as part of an HTTP Authorization header
client_id	uuid	the client unique identifier
code	string	an exchange code, for usage with code flow requests
endpoints	object	HTTP and MQTT base URLs assigned to the requesting client
expires	timestamp	the expiration date for the provided tokens
grant_type	string	the type of grant flow used to generate the provided tokens
refresh_token	string	a token used with /v3/auth/exchange to retrieve fresh tokens, without executing the entire flow
scope	array	granted permissions associated with the provided tokens

Call using POST method

• Request (application/json)

  • Body

      {
              "client_id": "bef41164-90c6-11e7-a25d-97266664a105",
              "refresh_token": "l8puku66syp5llktgrtrrxgw6vqc8hm109yfsf3wcm1t2dpkzoxs6na3na0zzc2q",
              "grant_type": "client_credentials"
      }

• Response 200 (application/json)

    {
            "access_token": "7enfl95a36qa72go4cdgd7usvln0riijh19a1bmbtjdkkoltn5l5fcfth397snao5ut79gageyv41zfoiourpkt3a1b7p6bzr9jwwiqi26emu33qdabq1vufw627oetv",
            "client_id": "bef41164-90c6-11e7-a25d-97266664a105",
            "code": "cl98bv8kkvcu4ele2pnmsau2befvnuamni3eq6wu6oi4o410bwhn9sfalk9vds9q",
            "endpoints":{
                    "http":"https:\/\/api.platform.muzzley.com",
                    "mqtt":"mqtts:\/\/api.platform.muzzley.com:8881"
            },
            "expires":"2017-12-03T04:08:01.582+0000",
            "grant_type":"client_credentials",
            "refresh_token":"39cyl6fjcotgvkbc2cf3583r8xf2ezai37wknrijplenjb9ngt492ojww4zhffut",
            "scope":[
                    "application"
            ]
    }

Call using GET method

• Parameters

  • client_id (required, string)

  • refresh_token (required, string)

  • grant_type (required, string)

  • redirect_uri (optional, string)

• Request (application/json)

• Response 307

  • Headers

      Location: https://mydomain.com/drop_credentials?expires=2017-12-03T04:08:01.582+0000&refresh_token=39cyl6fjcotgvkbc2cf3583r8xf2ezai37wknrijplenjb9ngt492ojww4zhffut&access_token=7enfl95a36qa72go4cdgd7usvln0riijh19a1bmbtjdkkoltn5l5fcfth397snao5ut79gageyv41zfoiourpkt3a1b7p6bzr9jwwiqi26emu33qdabq1vufw627oetv

API Usage

Authentication with OAuth 2.0

In order to interact with the Habit Analytics API, clients must authenticate using the OAuth 2.0 protocol, in particular the Client credentials flow. This call must be made using the /v3/auth/authorize endpoint, passing pre-generated credentials, called client_id and client_secret.

All subsequent calls must be directed to the base URL provided in endpoints.http and an Authorization header must be added to every API call, in the form:

    Authorization: Bearer {access_token}

Retrieve Application

The response will be populated with:

• id, application ID

• name, application name

• namespace, application namespace

• state, the current state of the application

Call GET /v3/applications/[me|self|:application-id]

• Response 200 (application/json)

{
    "_id":"\/v3\/applications\/{application-id}",
    "_rev":"3-94c1450541b80ec43c7503d7b2ec8e6c",
    "active_ts":"2019-05-24T09:58:16.824+0000",
    "cdata":null,
    "confirmation_hash":"qweqweqwe",
    "created":"2019-05-24T09:58:15.841+0000",
    "href":"\/v3\/applications\/{application-id}",
    "id":"{application-id}",
    "inactive_ts":"1970-01-01T00:00:00.000+0000",
    "name":"{application-name}",
    "namespace":"qweqweqwe",
    "state":"active",
    "type":"applications",
    "updated":"2019-05-24T09:58:16.825+0000"
}

List Application Users

The response will be populated with, at least, 5 main attributes:

• id, the user ID

• name, the user full name

• namespace, unique user namespace

• state, user state

• target_id, application ID

Call GET /v3/applications/[me|self|:application-id]/users

• Parameters

  • fields (optional, string) # ex: id,name,email,mobile,state

  • page_size (optional, integer) # ex: 20

  • state (optional, string) # ex: active

  • order_by (optional, string) # ex: -created

• Response 200 (application/json)

{
    "elements": [
        {
            "_id":"\/v3\/users\/{user-id}",
            "_rev":"3-94c1450541b80ec43c7503d7b2ec8e6c",
            "active_ts":"2019-05-24T09:58:16.824+0000",
            "confirmation_hash":"qweqweqwe",
            "created":"2019-05-24T09:58:15.841+0000",
            "href":"\/v3\/users\/{user-id}",
            "id":"{user-id}",
            "inactive_ts":"1970-01-01T00:00:00.000+0000",
            "name":"{user-name}",
            "namespace":"qweqweqwe",
            "state":"active",
            "type":"users",
            "updated":"2019-05-24T09:58:16.825+0000"
        }
    ],
    "size: 1
}

Retrieve Application User

The response will be populated with, at least, 5 main attributes:

• id, the user ID

• name, the user full name

• namespace, unique user namespace

• state, user state

• target_id, application ID

Call GET /v3/applications/[me|self|:application-id]/users/:user-id

• Response 200 (application/json)

{
    "_id":"\/v3\/users\/{user-id}",
    "_rev":"3-94c1450541b80ec43c7503d7b2ec8e6c",
    "active_ts":"2019-05-24T09:58:16.824+0000",
    "confirmation_hash":"qweqweqwe",
    "created":"2019-05-24T09:58:15.841+0000",
    "href":"\/v3\/users\/{user-id}",
    "id":"{user-id}",
    "inactive_ts":"1970-01-01T00:00:00.000+0000",
    "name":"{user-name}",
    "namespace":"qweqweqwe",
    "state":"active",
    "type":"users",
    "updated":"2019-05-24T09:58:16.825+0000"
}

List Application Enabled Channel Templates

The response will be populated with, at least, 5 main attributes:

• id, the channel template ID

• template.name, the channel template name

• template.channelspec_id, the channel spec ID

• access, user access mode

• channeltemplate_id, channel template ID

Call GET /v3/applications/[me|self|:application-id]/channel-templates

• Parameters

  • fields (optional, string) # ex: id,template.name

  • page_size (optional, integer) # ex: 20

  • template.state (optional, string) # ex: active

  • access (optional, string) # ex: public

  • order_by (optional, string) # ex: -created

• Response 200 (application/json)

{
    "elements": [
        {
            "_id" : "\/v3\/data-layer\/enabled-channel-templates\/{enabled-channeltemplate-id}",
            "_rev" : "1-4483b02a73d10bd267c82cda1f7213eb",
            "access" : "public",
            "access_list" : [],
            "channeltemplate_id" : "{channeltemplate-id}",
            "client_id" : "{application-id}",
            "created" : "2020-04-29T13:56:19.242+0000",
            "href" : "\/v3\/data-layer\/enabled-channel-templates\/{enabled-channeltemplate-id}",
            "id" : "{enabled-channeltemplate-id}",
            "order_index" : 1,
            "overlay" : null,
            "role" : {
                    "type" : "application"
            },
            "template" : {
                    "_id" : "\/v3\/channel-templates\/{channeltemplate-id}",
                    "_rev" : "4-a5a5c02486ed19ea0d9a26505570fa52",
                    "active_ts" : "2018-11-08T15:45:53.790+0000",
                    "additionable" : true,
                    "brand" : "Netatmo",
                    "channelspec_id" : "{channelspec-id}",
                    "created" : "2018-11-08T15:45:53.791+0000",
                    "href" : "\/v3\/channel-templates\/{enabled-channeltemplate-id}",
                    "icon" : "https:\/\/cdn\/things\/profiles\/netatmo\/netatmo_smoke_centered_510x510.jpg",
                    "id" : "{channeltemplate-id}",
                    "image" : "https:\/\/cdn\/things\/profiles\/netatmo\/netatmo_smoke_centered_510x510.jpg",
                    "inactive_ts" : "2018-11-08T15:45:53.790+0000",
                    "manufacturer" : "Netatmo",
                    "multiple" : true,
                    "name" : "Netatmo Smart Smoke Alarm",
                    "required_capability" : "discovery-webview",
                    "state" : "active",
                    "type" : "device",
                    "updated" : "2019-06-19T18:47:35.572+0000"
            },
            "type" : "application",
            "updated" : "2020-04-29T13:56:19.242+0000"
        }
    ],
    "size: 1
}

Retrieve Application Enabled Channel Template

The response will be populated with, at least, 5 main attributes:

• id, the channel template ID

• template.name, the channel template name

• template.channelspec_id, the channel spec ID

• access, user access mode

• channeltemplate_id, channel template ID

Call GET /v3/applications/[me|self|:application-id]/channel-templates/:enabled-channeltemplate-id

• Response 200 (application/json)

{
    "_id" : "\/v3\/data-layer\/enabled-channel-templates\/{enabled-channeltemplate-id}",
    "_rev" : "1-4483b02a73d10bd267c82cda1f7213eb",
    "access" : "public",
    "access_list" : [],
    "channeltemplate_id" : "{channeltemplate-id}",
    "client_id" : "{application-id}",
    "created" : "2020-04-29T13:56:19.242+0000",
    "href" : "\/v3\/data-layer\/enabled-channel-templates\/{enabled-channeltemplate-id}",
    "id" : "{enabled-channeltemplate-id}",
    "order_index" : 1,
    "overlay" : null,
    "role" : {
            "type" : "application"
    },
    "template" : {
            "_id" : "\/v3\/channel-templates\/{channeltemplate-id}",
            "_rev" : "4-a5a5c02486ed19ea0d9a26505570fa52",
            "active_ts" : "2018-11-08T15:45:53.790+0000",
            "additionable" : true,
            "brand" : "Netatmo",
            "channelspec_id" : "{channelspec-id}",
            "created" : "2018-11-08T15:45:53.791+0000",
            "href" : "\/v3\/channel-templates\/{enabled-channeltemplate-id}",
            "icon" : "https:\/\/cdn\/things\/profiles\/netatmo\/netatmo_smoke_centered_510x510.jpg",
            "id" : "{channeltemplate-id}",
            "image" : "https:\/\/cdn\/things\/profiles\/netatmo\/netatmo_smoke_centered_510x510.jpg",
            "inactive_ts" : "2018-11-08T15:45:53.790+0000",
            "manufacturer" : "Netatmo",
            "multiple" : true,
            "name" : "Netatmo Smart Smoke Alarm",
            "required_capability" : "discovery-webview",
            "state" : "active",
            "type" : "device",
            "updated" : "2019-06-19T18:47:35.572+0000"
    },
    "type" : "application",
    "updated" : "2020-04-29T13:56:19.242+0000"
}

Accessing channels with MQTT

For reading about the MQTT protocol, please, refer to the official documentation

Each intervenient should listen for MQTT messages by subscribing its channels topic. For instance:

if it's an application: /v3/applications/{application_id}/channels/#

Each intervenient should send MQTT messages by publishing to the channel topic, /v3/channels/{channel_id}/components/{namespace}/properties/{namespace}/value

Each intervenient must provide a unique identifier (client_id) as MQTT user and the previously retrieved access_token as a password.

Message structure

Each message received or sent to the MQTT topic associated with a given property must state the underlying performative. The performative contextualises the receiving intervenient on what to proceed.

Allowed performatives are:

• w, that asks the receiving intervenient to change the property value

• r, that ask the receiving intervenient for the property value

• iw, that informs the receiving intervenient of the property value, as a result of a w performative message

• ir, that informs the receiving intervenient of the property value, as a result of a r performative message

Every time the application receives a request to read or write and it is not possible to execute it due to a problem with an external API, a "iw" should be sent back, to the property "access", with the following data: "unreachable".

Message parameters

PARAMETER	TYPE	DESCRIPTION
io	string	The performative, stating what to do with this message
value	any	The property value, structured as specified in the underlying property spec
sender	uuid	The requesting intervenient unique identifier (read-only, no need to instantiate and if instantiate, overwritten by the platform) Same as the X-Client-Id from the webhook calls
on_behalf_of	uuid	The intervenient unique identifier on behalf of the requesting intervenient is sending this message. Typically for access tokens that were acquired through the password flow, that usually apply to the application -> user relation (read-only, no need to instantiate and if instantiate, overwritten by the platform) Same as the X-Owner-Id from the webhook calls
timestamp	timestamp	Timestamp for the moment the message was sent (read-only, no need to instantiate and if instantiate, overwritten by the platform)

Example

{
    "io" : "w",
    "sender" : "c9a78b2c-94a9-11e7-a76d-4b1c78a6ed17",
    "on_behalf_of" : "18f52e1a-9ebd-11e7-8613-e70c15c3f16f",
    "timestamp" : "2017-08-28T00:34::40.000+0000"
    "data" : {
        "location" : [ -34.9823479, 12.445989 ]
        "velocity" : 38.5587
    }
}

MQTT clients

A list of MQTT clients support several developing languages:

• Eclipse Paho

• Mosquitto

Mosquitto Client Sub Example (Linux):

$ mosquitto_sub -h :endpoint-mqtt-host -p :endpoint-mqtt-port -u :application-id -P <token> -t /v3/applications/:application-id/channels/# --capath /etc/ssl/certs -d

Sample Project

An SDK and python project that can be used as an example of how to make an integration with Habit is available and open source: https://github.com/habitio/manager-sdk-python