Documentation

# Installation

eKuiper provides docker image, binary packages and helm chart to install.

# Running eKuiper in Docker

Please make sure docker has installed before running.

  1. Get docker image.

    docker pull lfedge/ekuiper:x.x.x
    1

  2. Start docker container.

    docker run -p 9081:9081 -d --name kuiper -e MQTT_SOURCE__DEFAULT__SERVER=tcp://broker.emqx.io:1883 lfedge/ekuiper:xxx
    1

In this example, we specify the default MQTT broker via environment variable to broker.emqx.io, which is a public MQTT test server hosted by EMQ (opens new window).

For more configuration and docker image tags, please check lfedge/ekuiper in docker hub (opens new window).

# Running eKuiper with management console

eKuiper manager is a free eKuiper management web console which is provided as a docker image. We can use docker compose to run both eKuiper and eKuiper manager at once to ease the usage.

Please make sure docker compose has installed before running.

  1. Create docker-compose.yaml file.

    version: '3.4'

services:
    manager:
       image: emqx/ekuiper-manager:x.x.x
       container_name: ekuiper-manager
       ports:
       - "9082:9082"
       restart: unless-stopped
       environment:
         # setting default eKuiper service, works since 1.8.0
         DEFAULT_EKUIPER_ENDPOINT: "http://ekuiper:9081"
    ekuiper:
       image: lfedge/ekuiper:x.x.x
       ports:
         - "9081:9081"
         - "127.0.0.1:20498:20498"
       container_name: ekuiper
       hostname: ekuiper
       restart: unless-stopped
       user: root
       volumes:
         - /tmp/data:/kuiper/data
         - /tmp/log:/kuiper/log
       environment:
         MQTT_SOURCE__DEFAULT__SERVER: "tcp://broker.emqx.io:1883"
         KUIPER__BASIC__CONSOLELOG: "true"
         KUIPER__BASIC__IGNORECASE: "false"
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28

  2. Start docker-compose cluster.

    $ docker-compose -p my_ekuiper up -d
    1

  3. Check docker images running status, make sure two containers are started.

    $ docker ps
CONTAINER ID   IMAGE                         COMMAND                  CREATED              STATUS                  PORTS                                                NAMES
e2dbcd4c1f92   lfedge/ekuiper:latest          "/usr/bin/docker-ent…"   7 seconds ago        Up Less than a second   0.0.0.0:9081->9081/tcp, 127.0.0.1:20498->20498/tcp   ekuiper
fa7c33b3e114   emqx/ekuiper-manager:latest   "/usr/bin/docker-ent…"   About a minute ago   Up 59 seconds           0.0.0.0:9082->9082/tcp                               manager
    1
    2
    3
    4

Please check use of eKuiper management console to set up and configure the eKuiper manager.

# Install From Zip

eKuiper binary packages are released on below operating systems with AMD64, ARM and ARM64 support:

  • CentOS 7 (EL7)
  • CentOS 8 (EL8)
  • Raspbian 10
  • Debian 9
  • Debian 10
  • Ubuntu 16.04
  • Ubuntu 18.04
  • Ubuntu 20.04
  • macOS

For other operating systems such as Windows, users can compile from source code manually.

  1. Download eKuiper zip or tar for your CPU architecture from ekuiper.org (opens new window) or Github (opens new window).

  2. Unzip the installation file:

    unzip kuiper-x.x.x-linux-amd64.zip
    1

  3. Start eKuiper.

    $ bin/kuiperd
    1

  4. Remove eKuiper. Simply delete the eKuiper directory.

After installation, all the files are inside the unzipped directory. Please check installed directory structure for detail.

# Install from package

  1. Download eKuiper package for your CPU architecture from ekuiper.org (opens new window) or Github (opens new window).

  2. Install eKuiper.

    • DEB package:

      # for debian/ubuntu
$ sudo apt install ./kuiper-x.x.x-linux-amd64.deb
      1
      2

    • RPM package:

      # for CentOS
$ sudo rpm -ivh kuiper-x.x.x-linux-amd64.rpm
      1
      2

  3. Start eKuiper.

    • quick start

      $ sudo kuiperd
      1

    • systemctl

      sudo systemctl start kuiper
      1

  4. Remove eKuiper.

    • DEB:

      sudo apt remove --purge kuiper
      1

    • RPM:

      sudo yum remove kuiper
      1

When installing by package, the eKuiper folders are not in the same directory. The installation structure is as below:

/usr/lib/kuiper/bin
  kuiperd
  kuiper
/etc/kuiper
  ...
/var/lib/kuiper/data
  ...
/var/lib/kuiper/plugins
  ...
/var/log/kuiper
   ...
1
2
3
4
5
6
7
8
9
10
11

# Install via Helm (K8S、K3S)

  1. Add helm repository.

     $ helm repo add emqx https://repos.emqx.io/charts
 $ helm repo update
    1
    2

  2. Query eKuiper.

     $ helm search repo emqx
 NAME         CHART VERSION APP VERSION DESCRIPTION
 emqx/emqx    v4.0.0        v4.0.0      A Helm chart for EMQX
 emqx/emqx-ee v4.0.0        v4.0.0      A Helm chart for EMQX
 emqx/ekuiper  0.1.1         0.1.1       A lightweight IoT edge analytic software
    1
    2
    3
    4
    5

  3. Start eKuiper.

     $ helm install my-ekuiper emqx/ekuiper
    1

  4. View eKuiper status.

    $ kubectl get pods
NAME         READY  STATUS    RESTARTS  AGE
my-ekuiper-0 1/1    Running   0         56s
    1
    2
    3

# Compile from source code

  1. Get the source code.

    $ git clone https://github.com/lf-edge/ekuiper.git
    1

  2. Compile.

    $ make
    1

  3. Start eKuiper.

    $ cd _build/kuiper-x.x.x-linux-amd64/
$ bin/kuiperd
    1
    2

# Compile Packages and Docker Images

  • Packages: $ make pkg

    • Packages files that support EdgeX: $ make pkg_with_edgex

  • Docker images: $ make docker

    Notice that: Docker images support EdgeX by default

# Cross-compile binaries

Go supports cross-compiling binaries for multiple platforms which applies to eKuiper as well. Because eKuiper depends on sqlite, CGO_ENABLED must be set to 1 which requires to install and specify the gcc of the target system.

  • Install the GNU toolchain/gcc of the target system.
  • Modify the Makefile to specify GOOS, GOARCH and CC and then build.

For example, to cross build ARM64 binaries in AMD64 ubuntu/debian machine, do these steps:

  1. Install the GNU toolchain/gcc of the target system ARM64

    apt-get install gcc-aarch64-linux-gnu
    1

  2. Update the Makefile in the build command. Examples:

    GO111MODULE=on CGO_ENABLED=1 GOOS=linux GOARCH=arm64 CC=aarch64-linux-gnu-gcc go build -trimpath -ldflags="-s -w -X main.Version=$(VERSION) -X main.LoadFileType=relative" -o kuiperd cmd/kuiperd/main.go
    1

  3. Run make

# Compile with Selected Features

eKuiper allows tailoring the binary in compilation to get a customized feature set. As written by go, it also allows cross compilation. Except core runtime and REST api, there are some features that are allowed to be enabled or disabled during compilation by go build constraints (opens new window). Uses can customize the built binary to include only the desired features to reduce the binary size according to the limit of the target environment.

FeatureBuild TagDescription
CorecoreThe core of eKuiper. It contains the processor and REST API for stream/table/rule, the configuration processing, the SQL parser, the rule runtime etc.
CLIrpcThe cli server
EdgeX Foundry integrationedgexThe built-in edgeX source, sink and connection
Native pluginpluginThe native plugin runtime, REST API, CLI API etc.
Portable pluginportableThe portable plugin runtime, REST API, CLI API etc.
External serviceserviceThe external service runtime, REST API, CLI API etc.
Msgpack-rpc External servicemsgpackSupport msgpack-rpc protocol in external service
UI Meta APIuiThe REST API of the metadata which is usually consumed by the ui
Prometheus MetricsprometheusSupport to send metrics to prometheus
Extended template functionstemplateSupport additional data template function from sprig besides default go text/template functions
Codecs with schemaschemaSupport schema registry and codecs with schema such as protobuf

In makefile, we already provide three feature sets: standard, edgeX and core. The standard feature set include all features in the list except edgeX; edgeX feature set include all features; And the core feature set is the minimal which only has core feature. Build these feature sets with default makefile:

# standard
make
# EdgeX
make build_with_edgex
# core
make build_core
1
2
3
4
5
6

Feature selection is useful in a limited resource target which is unlikely to run as docker container. So we only provide standard feature set in the docker images.

And users need to build from source to customize the feature sets. To build with the desired features:

go build --tags "<FEATURE>"
1

For example, to build with core and native plugin support:

go build --tags "core plugin"
1

Recommend updating the build command in the Makefile with tags and building from make.

# Installation structure

Below is the directory structure after installation.

bin
  kuiperd
  kuiper
etc
  ...
data
  ...
plugins
  ...
log
  ...
1
2
3
4
5
6
7
8
9
10
11

# bin

The bin directory includes all of executable files. Such as the kuiper server kuiperd and the cli client kuiper.

# etc

The etc directory contains the default configuration files of eKuiper. Such as the global configuration file kuiper.yaml and all the source configuration files such as mqtt_source.yaml.

# data

This folder saves the persisted definitions of streams and rules. It also contains any user defined configurations.

# plugins

eKuiper allows users to develop your own plugins, and put these plugins into this folder. See extension for more info for how to extend the eKuiper.

# log

All the log files are under this folder. The default log file name is stream.log.

← Stream: unbounded series of events →

Edit this page
What’s on this page