AQW - Publish data to Goliath & visualise in a Grafana Dashboard

zpm1066

Hi Jared,
I’m trying to setup a Grafana Dashboard on Grafana Cloud for AQW using the ethernet_hub_ble example.

So far, I’ve successfully done the following:

Created a project in Goliath, provisioned my AQW device.
Published AQW data to Golioth. I’m able to view sensor data updates in Golioth LightDB State.
Setup Grafana Cloud instance.
Created an API Key.
Setup a data source with JSON API using the X-API Key and the value to the API key. Save & test was successful.
Created a dashboard using your Air Quality Wing-1633657750477.json.
I’m able to view the dashboard but no data appears in the dashboard for any of the AQW sensors. I do observe data updates in Golioth LightDB State.

For some reason, Grafana Cloud isn’t picking up the data from Golioth.

Any suggestions as to what may be going on? Thanks.

jaredwolff

zpm1066 that .json configuration may be out of date. I believe they built a Websocket based plugin that talks directly to their API:

https://blog.golioth.io/using-websockets-with-grafana-golioth-demo/

zpm1066

jaredwolff

Thanks. I’ll review the link.

btw - Should I use LightDB stream?

jaredwolff

Yes, that’s the only way to send real time data and retrieve it.

zpm1066

jaredwolff

From the link you shared above and ethernet_hub_ble code unchanged, I should be able use Golioth's WebSocket API to graph the LightDB State data.

Steps:

Install the Golioth’s WebSocket API in the Grafana Cloud instance
Create a WebSocket API data source
Fill in the WebSocket API info:
Host: wss://api.golioth.io/v1/projects/{Project ID}, add Custom Headers
Header: X-API-Key and the value, then Save & test
Grafana displays that the “Data source is working”.
Create a dashboard with the following.
Set Data source to WebSocket API
Add Path and Fields for LightDB State and transform the JSON result into something that the dashboard can display, for example
Path: /devices/{Device Id}
Field $..env.hum
Field $..env.temp

LightDB State
{1 item
     "env":{4 items
       "hum":30
       "pm25":2
       "temp":26
       "voc":52
      }
}

I tried creating a dashboard but got error Some queries returned an error: Query A - undefined.

I don’t see anything wrong with Data source configuration but there may be some other configuration issue.

I’ll open a ticket with Grafana Support.

zpm1066

jaredwolff

I’ve confirmed that the data source is returning data correctly using websocat from a macOS terminal session.

Below, I’ve replaced actual “Device Id” and “APIkey” by “mydeviceid” and “myapikey”

$ websocat wss://api.golioth.io/v1/ws/projects/aqw1066/devices/mydeviceid/data?x-api-key=myapikey
{"result":{"data":{"env":{"hum":33,"pm25":0,"temp":29,"voc":157}}}}
{"result":{"data":{"env":{"hum":32,"pm25":0,"temp":29,"voc":153}}}}
{"result":{"data":{"env":{"hum":32,"pm25":0,"temp":29,"voc":153}}}}

Next step, check why Grafana isn’t getting the data.

zpm1066

I’ve also ran curl against the device’s JSON endpoint to confirm that JSON data source returns the expected sensor data. Below, I’ve replaced the actual API key by myapikey.

$ curl -X 'GET' \

'https://api.golioth.io/v1/projects/aqw1066/devices/643c620df272169b4557ee61/data/env' \
-H 'accept: application/json' \
-H 'x-api-key: myapikey' \
-H 'Authorization: myapikey'
{"data":{"hum":20,"pm25":0,"temp":41,"voc":265}

jaredwolff

@zpm1066 playing around with this now. I’m not even getting stream data from the wss endpoint. Stay tuned on that

zpm1066

jaredwolff

Hello Jared,
It looks like the WebSocket API plugin through Grafana has a few issues. Grafana support informed me that I need to approach Goliath support as it’s a third party plugin. I had the same issue with JSON API with Grafana.

A resource in Grafana has kindly offered to assist. So, hopefully will get a resolution soon.

jaredwolff

Sound good. I know they worked on it a while back so it’s likely it could be broken especially if there’s not a ton of people using it.

zpm1066

jaredwolff

Hi Jared,

Success!! I had call with Golioth and was recommended to use the JSON APIfor both LightDb State and LightDB Stream, as the sensors publish data at relatively slow interval of per minute.

The resolution turned out to be a minor change in the Grafana Dashboard Path, plus an addition of a query in the Body of the Dashboard configuration.

I used the version of JSON API plugin that is available in Grafana Cloud, the same one that I tried using earlier.

Golioth weren’t sure why the WebSocket API didn’t work and will get back to me.

Later I’ll setup separate charts for each of the sensors, as you had in your JSON.

What’s the best to generate what you had in the Air Quality Wing-1633657750477.json? Thanks.

Data source - JSON API
https://api.golioth.io/v1/projects/aqw1066


Dashboard Configuration

Path
POST
/devices/mydevicei/stream

Fields
$..time
$..env.hum
$..env.voc
$..env.temp
$..env.p25

Body
{
	"start" : "${__from:date}",
        "end" : "${__to:date}",
	"perPage" : 99999,
	"query": {
		"fields" : [
			{ "path": "time" },
			{ "path": "deviceId"},
			{ "path": "*"}
		],
		"timeBucket" : "5s"
	}
}

zpm1066

@jaredwolff

btw - Currently, I’m running with Segger RTT output enabled and noticed that if I close the J-Link RTT viewer the sensor data stops going to Golioth.

In your ethernet_hub_ble code, you have Segger RTT disabled. I’ll check what’s going on.

jaredwolff

Btw Chris posted a video on how to set up Websockets: https://www.loom.com/share/803b2894b0f548979d8314df5e6c45cb

zpm1066

jaredwolff

Thanks. There seems to be an issue with the Golioth WebSocket plugin for Grafana Cloud. Another user had the same issue as me. Hopefully we’ll see someone from on Golioth Forum can assist.