Open Source Exanalytics
HowTo setup Fluentd on Streaming Device
© 2024  Nexedi GmbH
Eteri Sokhoyan, Ekaterina Belyaeva, Levin Eric Zimmermann, Paul Graydon, Ivan Tyagov  |

HowTo setup Fluentd on Sensor

Agenda

  • Request Fluentd instance for IOT Sensor
  • Learn configuration parameters for Fluentd

This tutorial teaches how to setup Fluentd on IOT Sensor. 

Prerequisites

Before doing this tutorial make sure you have read and completed the following tutorials:

IMPORTANT: In order to successfully request a Fluentd instance, you will need to have a validated Allocation Supply in your project with the following configuration:

  • Software Product: Fluentd 1.0.359 software release with the default software type
  • Compute Node: the Node corresponding to your Olimex Sensor

The Software Product should already be ready for you to use, no need to create it.

If you wish to know more about the configuration of Fluentd, you can find the official documentation here: Fluentd Configuration File.

Request Fluentd instance for IOT Sensor

Step 1: Request instance

Please click on "Request" menu at left side of the page as shown on image below.

This will redirect you to your project page where you will have to select the appropriate project.

After selecting the project will will be given option to choose the product you want to install. In this case it is fluentd.

Finally you will be redirected to your new instance where you will need to specify the needed instance's parameters.

Following actions are needed:

  1. Set a proper instance name (in the case it is "t20.sensor1")
  2. Click on "Raw XML" button which will enable you to edit directly the instance's XML parameters
  3. Insert and adjust for your case the instance XML (the parameters ware wrapped in "<" and ">"). The XML itself you can find here. Please note that it is intentionally escaped XML!
  4. Click on "Request" button. Your instance will be created in background and its status will be visible from your "Service" tabs on your left side.

The instance itself

Description of parameters

FLUENTD-GATEWAY-IPv4 The IPv4 address which fluentd gateway will bind to. Usually something like 192.168.x.x

Step 2: Monitor instantiation of your instance

You can monitor the process by running

tail -f /opt/slapos/log/slapos-node-instance.log

on the respective machine.

Fluentd Instance

This step can take some time.

In your SlapOS dashboard you can see your newly requested instance.

The instance is ready when the the Monitoring Status is green.

Note that the Fluentd configuration parameters we defined earlier can be found and edited in the Update parameter form .

Step 3: Setup additional packages and adjust permissions

You need to further install additional packets and adjust security permissions for your sensor to work, see below

# add Python's packages for reading environment data from MOD-ENV
root@t20:~# pip install bme280 smbus

# download script to read environment data from MOD-ENV (Note: please place it in "/srv/slapgrid"as this is where fluentd expects it!)
root@t20:/srv/slapgrid# wget https://lab.nexedi.com/nexedi/wendelin/-/raw/master/tutorial/kbme280-sim.py

# make sure that downloaded script is executable by all system users
root@t20:/srv/slapgrid# chmod a+x kbme280-sim.py

# make sure that the user running fluentd service on your IOT sensor can access I2C bus to read data from MOD-ENV
# you can see the user running the fluentd by executing "slapos node" command. If your fluentd's instance is "slappart4:fluentd-service-on-watch"
# then it is owned and run by "slapuser4". Thus adjust accordingly command below.
root@t20:/srv/slapgrid# usermod -a -G i2c slapuserX

# test that script is executable for slapuserX (adjust accordingly!)
root@t20:/srv/slapgrid# su slapuserX
$ /usr/bin/python3 /srv/slapgrid/kbme280-sim.py
998.8597649764879    33.679160017867545    21.115451616437735

Step 4: Control (start / stop) IOT Sensor

Login to your IOT Sensor and run the following command :

sudo slapos node status

You see that a new service for Fluentd is added.

And the generated fluentd configuration file is located here :

/srv/slapgrid/slappartX/etc/fluentd-agent.conf

Now your IOT Sensor is ready to read data and to forward it over fluentd IOT Gateway to Wendelin.

Thank You

  • Nexedi SA
  • 147 Rue du Ballon
  • 59110 La Madeleine
  • France
  • +33629024425

For more information, please contact Jean-Paul, CEO of Nexedi (+33 629 02 44 25).