Wendelin Big Data Learning Track

This learning track contains a sequence of lectures teaching how to setup and use Wendelin.  After finishing this learning track, you should have a ready-to-use Wendelin system and be familiar with setting up a sensor, retrieve, analyse and visualize data. 

Lecture 1: Wendelin Introduction

This lecture will introduce Wendelin and underlying concepts.

• Presentation: Introduction to Wendelin

Lecture 2: Setup

There are three ways to setup Wendelin - requesting a Wendelin in the SlapOS Panel, Wendelin standalone on a VM and Wendelin provisioned with SlapOS.
Wendelin Standalone is a quick way to test Wendelin and to do the tutorials in Lecture 3.
Wendelin provisioned with SlapOS is recommended for production.
If you are planning to do all the tutorials Setup A or Setup C is advised to use.

Setup A: Wendelin Instance via SlapOS Panel

• HowTo Request Wendelin in the SlapOS Panel
• HowTo Configure Wendelin Site (Presentation)

Setup B: Wendelin Standalone on a VM

• HowTo Install Wendelin Standalone

Setup C: Wendelin with SlapOS

These tutorials describe the installation process of Wendelin on SlapOS. You can either install it on Theia (new) or on Webrunner (old).

• Create An Account On Rapid.Space  OR  install your own SlapoOS Master
• HowTo Request A Theia.Runner (new) OR HowTo Request Webrunner (Presentation) (old)
• HowTo Supply A Software And Request An Instance On Theia (new) OR HowTo Install Wendelin on Webrunner (Presentation) (old)
• HowTo Request ipv4 frontend with Ciphers (Presentation)
• HowTo Configure Wendelin Site (Presentation)

Lecture 3: Wendelin for Data Scientists

This section teaches how to use Data Lakes in Wendelin, how to easily upload and download data using ebulk. 

Lecture 3.1: Data Lake Basics

• HowTo Enable User Registration (Presentation)
• HowTo Ingest Data in Wendelin with ebulk (Presentation) 

Lecture 3.2: Big Data Collaboration

• Data Lake Security Model (Presentation)
• HowTo Publish Data
• HowTo Use Wendelin for Big Data Collaboration (Presentation)

Lecture 4: Wendelin for Data Science Industrialisation

This section shows on a specific example how to configure wendelin, receive streaming data using fluentd and batch data using embulk, as well as how to create simple notebook and visualise data.

• Introduction to Wendelin for Data Science Industrialisation (Presentation)

Lecture 4.1: Dynamic Ingestion Policies

• HowTo Create Data Product (Presentation)
• HowTo Create Data Acquisition Unit (Presentation)
• HowTo Create Data Supply (Presentation)

Lecture 4.2: Automated Batch Ingestion

• How to Request embulk on SlapOS

Lecture 4.3: Automated Data Streaming

• HowTo setup Sensor and IOT Gateway (Presentation)
• HowTo setup Fluentd on IOT Gateway (Presentation) 
• HowTo setup Fluentd on Sensor (Presentation)
• HowTo check data arrival (Presentation)
• HowTo setup fluent-bit on Sensor (Presentation)

Lecture 4.4: Data Processing Workflow

• HowTo Transform Data (Presentation)
• HowTo Resample Data (Presentation)

Lecture 4.5: Data Visualisation

• HowTo Create Notebook with Graphs (Presentation)
• HowTo Create Notebook with Plotly Chart Editor

Lecture 4.6: Industrialize Machine Learning

• HowTo Create Basic Prediction with scikit-learn (Presentation)