Frequently Asked Questions (FAQ)

Q1. Which data formats are supported?

- The platform supports both JSON and XML formats.

Q2. How do you create the sensor (data) schema?

- It is crucial that you know the exact format of the data document (i.e. the data "output") prior to creating the data schema. When you have this document in hand, you can either use the Windows desktop application configuration tool, or you can do this manually by using the appropriate REST endpoint directly. The latter case requires some basic knowledge of the Avro specification to map the data attributes correctly. You can also look at the configuration examples to get a better understanding of how this is done.

Q3. What are the key steps for pushing data to the data platform?

#1. A data provider must be registered to obtain valid access token(s). These tokens must be used when interacting with REST endpoints that requires authentication.

#2. A description of the expected format (partial Avro schema) of the data to send must be defined (please refer to Q2).

#3. Use the sensor endpoint URL for data transfer (push).

#4. Implement push logic, either directly in sensor software, or indirectly by using a data transfer mechanism (if sensors are pushing internally to a cloud system) to transfer data.

NOTE: when schema is successfully configured, the endpoint URL for sensor can be found by using the atom URL for sensor given in the HTTP created response.

Q4. Are nested data structures also supported?

- Yes.

Q5. Is it always required to use converters?

- Not always. Converters must be used if the value of the JSON/XML attribute (that will be bound to one component) is not numeric. If this value already is numeric, then no converter is required.

Q6. Is it always required to use timestamp mapping?

- Not always. Mapping of the timestamp attribute must be used if you want to use your own timestamp and not the default system Unix Epoch time. If your date attribute is already given as number of seconds since 1970-01-01 UTC (long datatype), then no "StringEpochTime" converter is required

Q7. What happens in cases when no timestamp mapping exists?

- The system time (Unix Epoch UTC) will be used.

Q8. What if one of the attributes that are bound to a component is missing a value?

- In these cases, the NaN value must represent the missing value.

Q9. What if the sensor schema changes?

- In cases when one (or more) attribute that is bound to an inventory component needs to be removed, the sensor schema must be changed (or reconfigured). Please read more here.

Q10. What is the usage of the binding path?

- The binding path is the JSON path (JPath) starting from root of the document to a target JSON attribute node. This binding path needs to be used to tell the system which JSON attribute that should be bound to a component and unit.

- By using this path, it will be possible to extract the attribute (i.e. component) value.

Q11. What is the usage of the target path?

- The target path is (like the binding path) the JSON path (JPath) starting from root of the document to a target JSON attribute node. The target path is used by a converter or field mapper when converting or mapping a JSON attribute value.

Q12. What is a data schema?

- A type of document that describes the structure/nature and the data types of a second document, in this context the data packet generated, either directly or indirectly, by a sensor.

Q13. Is "raw" data also stored?

- Yes, the complete data packet is stored as Avro record in backend system. REST endpoint(s) for retrieving data has not yet been implemented.

Q14. How is quality control (QC) managed?

- The quality control on sensor data is managed by one of the Spark applications in backend system. Trained machine learning (ML) models are also used in the outlier detection algorithms.

Note: not all components are configured for quality control. The exceptions are the humidity, temperature, CO, NO2, PM2.5 and PM10 components. More components may be added for QC in future releases. ML models for humidity and temperature have not yet been established.

Q15. Why is the Windows utility application being removed by antivirus software?

- This is a false positive. A potential reason for this is that the application has not yet been digitally signed by a third party. You can therefore safely ignore this warning from the software.

- Please make sure to "whitelist" the desktop application to avoid this problem.