HowTo setup Fluentd on Sensor

Agenda

  • Supply fluentd software on Sensor
  • Learn configuration parameters for fluentd
  • Request fluentd instance for Sensor

 

This tutorial teaches how to setup fluentd on IOT Gateway. 

Before doing this tutorial make sure you read and completed the following tutorials

You will also need to configure slapos client. For that read SlapOS HowTo Use SlapOS Client tutorial from Get SlapOS Master Token chapter.

 

Supply Software

First find the reference of your Streaming Device.

It is located in your Slapos Dashboard in Servers tab.

Then using supply command of Slapos Client install fluentd software release on that computer.


supply("https://lab.nexedi.com/nexedi/slapos/raw/master/software/fluentd/software.cfg", "YOUR_COMPUTER_ID")

Configuration Parameters

<source>
  @type exec
  tag YOUR_TAG_NAME
  command  /usr/local/bin/custom_read_bme280.py
  run_interval 1m
  <parse>
    keys pressure, humidity, temperature
  </parse>
</source>
<match YOUR_TAG_NAME>
  @type forward
  <server>
   name myserver1
   host IPV6_ADDRESS_OF_YOUR_IOT_GATEWAY
  </server>
</match>

About Fluentd configuration file you can learn from here.

For input plugin we use Fluentd's standard exec plugin

The plugin will execute the python script we wrote in HowTo Setup Sensor and IOT Gateway to read sensor data every minute. 

For output plugin we use Fluentd's standard forward plugin.

YOUR_TAG_NAME - it must consist of 2 part separated by .(dot). For example sensor_1.sample-environment-raw-data. First part (sensor_1) is the name of the sensor and must be the same as the reference of the Data Supply we created in HowTo Create Data Supply tutorial. The second part (sample-environment-raw-data) is the type of data (Data Product  -  HowTo Create Data Product).

host - IPV6 address of your server where data will be send. In our case it will be the address of  IOT Gateway. 

 

Request Instance

config_params = """<source> 
@type exec
tag YOUR_TAG_NAME
command python /usr/local/bin/custom_read_bme280.py 
run_interval 1m 
<parse> 
keys pressure, humidity, temperature 
</parse> 
</source> 
<match YOUR_TAG_NAME> 
@type forward 
<server>
name myserver1 
host IPV6_ADDRESS_OF_YOUR_IOT_GATEWAY 
</server> 
</match>"""

request("https://lab.nexedi.com/nexedi/slapos/raw/master/software/fluentd/software.cfg", "Sensor-Tutorial-Fluentd",filter_kw={"computer_guid": "YOUR_COMPUTER_ID"}, partition_parameter_kw={"conf_text":config_params})

As the fluentd configuration has many parameters for convenience we will write a short python script which will request the instance, instead of calling request function from Slapos Client.  

Create a python script and name it request_fluentd.py and copy/paste the presented code. 

** Don't forget to correctly fill in your computer reference, Instance name, url etc. 

As we are requesting instance of fluentd for the same computer the reference is the same as when we were supplying the software.

"YOUR_INSTANCE_NAME" -  give your instance a distinguishing name so you can later easily find it. For example Sensor-Tutorial-Fluentd.

Once it's ready run it by calling following command : 

slapos console request_fluentd.py

Fluentd Instance

This step can take some time.  

In your Slapos dashboard you can see you newly requested instance.

Instance is ready when the the Monitoring Status is green.

Note the Parameters section - there are the flunetd configuration parameters we defined earlier.

Sensor

Login to your  Sensor and run the following command : 

sudo slapos node status

You see that a new service for fluentd is added.

And the generated flunted configuration file is located here :

/srv/slapgrid/slappartX/etc/fluentd-agent.conf

 

To enable slapuser read sensor data, change the group of i2c-3 (this is where the sensor is attached) : 

chgrp slapuser8 /dev/i2c-3 

The number of slapuser depends on which slappart is fluentd installed. 

Now your Sensor is ready to retrieve data and send to, in our case, IOT Gateway.

Thank You

  • Nexedi SA
  • 147 Rue du Ballon
  • 59110 La Madeleine
  • France
  • +33629024425

For more information, please contact Jean-Paul, CEO of Nexedi (+33 629 02 44 25).