Introduction to Wendelin

Agenda

  • Nexedi
  • Success Cases
  • Wendelin Big Data Analytics
  • Batch Transfer To Data Lake
  • Buffered Streaming To Data Lake
  • Data Transformation
  • Data Visualisation
 

Nexedi

 

Nexedi - Profile

Nexedi World Map
  • Largest Free Software Publisher in Europe
  • Founded in 2001 in Lille (France) - 35 engineers worldwide
  • Enterprise Software for mission critical applications
  • Build, deploy, train and run services
  • Profitable since day 1, long term organic growth
Nexedi is probably the largest Free Software publisher with more than 10 products and 15 million lines of code. Nexedi does not depend on any investor and is a profitable company since day 1.

Nexedi - Clients

Nexedi References and Map

nexedi.com/success

Nexedi clients are mainly large companies and governments looking for scalable enterprise solutions such as ERP, CRM, DMS, data lake, big data cloud, etc.  

Nexedi - Free Software Stack

Nexedi Software Stack

stack.nexedi.com

Nexedi software is primarily developed in Python, with some parts in Javascript.

Success Cases

 

Wendelin: Wölfel Wind Systems, City of Munich

 

SlapOS: Rapid.Space & Teralab AI

https://rapid.space

https://beta.rapid.space/ is a high performance, low cost cloud infrastructure that provides:

  • big servers;
  • CDN;
  • IoT buffering.

It is available in China  in addition to Europe. It is based on SlapOS and Open Compute Project (OCP) hardware, the same as the one used by Facebook.

Introduction to Wendelin

 

Global Picture

 

Batch Transfer to Data Lake

 

Buffered Streaming to Data lake

 

Data Transformation

 

Data Visualisation

 

Highlights

 

re6st: resilient IPv6 everywhere

re6st was created to fix problems of current Internet through an IPv6 overlay network.

Current problems:

  • sub-optimal latency and unreliable transit
  • network cuts and data corruption in basic protocols (e.g. TCP)
  • government added censorships and bogus routing policies (e.g. China)

The probability of connectivity fault is about 1% in Europe/USA and 10% inside China - too much for industrial applications.

Without re6st, SlapOS (or any distributed container system) can not work. If one has to deploy 100 orchestrated services over a network of edge nodes with a 1% probability of faulty routes, the overall probability of failure quickly becomes too close to 100%. There is therefore no way to deploy edge without fixing the Internet first.

re6st routing provides one solution to that. re6st is available in China (license: 中华人民共和国增值电信业务经营许可证:沪A1-20140091). Nexedi has the right to provide global low latency high resiliency IPv6 network for IoT (http://www.grandnet.cn/).

In addition to re6st, we use buffering to that we do not lose data sent by edge nodes (gateways or sensors) in case of application server failure for example fluentd.

Both re6st and fluentd are used in all IoT deployments done by Nexedi and based on SlapOS.

SlapOS everywhere: sensors, gateway, data lake...

  • Master: ERP5 (promise definition, ordering, provisioning, accounting, billing, issue tracking)
  • Slave: buildout (promise execution, build, instantiation, configuration, monitoring)

So, we used buildout (http://docs.buildout.org/en/latest/) as the base for our service descriptor language and ERP5 to keep track of "service lifecycle" after we found out that any edge or cloud system can be made of two components: a devops and an ERP (see "SlapOS: SlapOS: A Multi-Purpose Distributed Cloud Operating System Based on an ERP Billing Model" https://ieeexplore.ieee.org/document/6009348). 

For resiliency, we based all our design on the idea that resiliency must be implemented with software and should rely on redundant infrastructure on redundant sites with redundant suppliers. However each site or hardware does not need to be redundant.

This approach was quite successful. By sticking to a very simple and minimal architecture, we could achieve with a small budget what huge community projects such as OpenStack still fail to achieve after 10 years. And we could do much more, because our architecture was more generic.

Batch Transfer to Data Lake

 

Ingesting Mongodb Data via Ebulk to Wendelin

 

Buffered Streaming to Data Lake

 

Olimex Sensor and IoT Gateway

 

SlapOS Dashboard

 

Wendelin Home Page

 

Data Transformation

 

Data Resampling Definition

 

Data Resampling Code

 

Environmental Data Array Preview

 

Data Visualisation

 

Notebook with Time Series Visualisation

 

Notebook with Plotly Chart Editor

 

Thank You

  • Nexedi SA
  • 147 Rue du Ballon
  • 59110 La Madeleine
  • France
  • +33629024425

For more information, please contact Jean-Paul, CEO of Nexedi (+33 629 02 44 25).