Skip to content

Latest commit

 

History

History
310 lines (215 loc) · 14.5 KB

README.md

File metadata and controls

310 lines (215 loc) · 14.5 KB
Raw

EdgeX ObjectBox Snap

snap store badge

This folder contains snap packaging for the EdgeX Foundry reference implementation.

The snap contains Consul, ObjectBox all of the EdgeX Go-based micro services from this repository, device-virtual, as well as Vault, Kong, Cassandra, and the two go-based security micro services. The snap also contains a single OpenJDK JRE used to run device-virtual.

Installation

Installing snapd

The snap can be installed on any system that supports snaps. You can see how to install snaps on your system here.

However for full security confinement, the snap should be installed on an Ubuntu 16.04 LTS or later Desktop or Server, or a system running Ubuntu Core 16 or later.

Installing EdgeX Foundry as a snap

The snap is published in the snap store at https://snapcraft.io/edgex-objectbox. You can see the current revisions available for your machine's architecture by running the command:

$ snap info edgex-objectbox

The snap can be installed using snap install. To install the snap from the edge channel:

$ sudo snap install edgex-objectbox --edge

Lastly, on a system supporting it, the snap may be installed using GNOME (or Ubuntu) Software Center by searching for edgex-objectbox.

Using the EdgeX snap

Upon installation, the following EdgeX services are automatically and immediately started:

  • consul
  • core-data
  • core-command
  • core-metadata
  • core-config-seed
  • security-services (see note below)

The following services are disabled by default:

  • support-notifications
  • support-logging
  • support-scheduler
  • export-client
  • export-distro
  • device-virtual

Any disabled services can be enabled and started up using snap set:

$ sudo snap set edgex-objectbox support-notifications=on

To turn a service off (thereby disabling and immediately stopping it) set the service to off:

$ sudo snap set edgex-objectbox support-notifications=off

All services which are installed on the system as systemd units, which if enabled will automatically start running when the system boots or reboots.

Configuring individual services

All default configuration files are shipped with the snap inside $SNAP/config, however because $SNAP isn't writable, all of the config files are copied during snap installation (specifically during the install hook, see snap/hooks/install in this repository) to $SNAP_DATA/config. The configuration files in $SNAP_DATA may then be modified. You may wish to restart the snap's services to take configuration into account with:

$ sudo snap restart edgex-objectbox

Viewing logs

Currently, all log files for the snap's can be found inside $SNAP_COMMON, which is usually /var/snap/edgex-objectbox/common. Once all the services are supported as daemons, you can also use sudo snap logs edgex-objectbox to view logs.

Additionally, logs can be viewed using the system journal or snap logs. To view the logs for all services in the edgex-objectbox snap use:

$ sudo snap logs edgex-objectbox

Individual service logs may be viewed by specifying the service name:

$ sudo snap logs edgex-objectbox.consul

Or by using the systemd unit name and journalctl:

$ journalctl -u snap.edgex-objectbox.consul

Security services

Currently, the security services are enabled by default. The security services consitute the following components:

All services are currently bundled in the singular service, security-services (see issue #485 for more details on why).

When security is enabled, Consul is secured using Vault for secret management, and Kong is used as an HTTPS proxy for all the services. The HTTPS keys for Kong and Vault are placed in $SNAP_DATA/vault/pki. Kong needs a database to manage itself, and can use either Postgres or Cassandra. Because Postgres cannot run inside of a snap due to issues running as root (currently all snap services must run as root, see this post for details), we use Cassandra.

To turn off security, use snap set:

$ sudo snap set edgex-objectbox security-services=off

Limitations

See the GitHub issues with label snap for current issues.

Building

The snap is built with snapcraft, and the snapcraft.yaml recipe is located within edgex-go, so the first step for all build methods involves cloning this repository:

$ git clone https://github.com/objectbox/edgex-objectbox
$ cd edgex-objectbox

Installing snapcraft

There are a few different ways to install snapcraft and use it, depending on what OS you are building on. However after building, the snap can only be run on a Linux machine (either a VM or natively). To install snapcraft on a Linux distro, first install support for snaps, then install snapcraft as a snap with:

$ sudo snap install snapcraft

(note you will be promted to acknowledge you are installing a classic snap - use the --classic flag to acknowledge this)

Note - currently the snap doesn't support cross-compilation, and must be built natively on the target architecture. Specifically, to support cross-compilation the kong/lua parts must be modified to support cross-compilation. The openresty part uses non-standard flags for handling cross-compiling so all the flags would have to manually passed to build that part. Also luarocks doesn't seem to easily support cross-compilation, so that would need to be figured out as well.

Running snapcraft on MacOS

To install snapcraft on MacOS, see this link. After doing so, follow in the below build instructions for "Building with multipass"

Running snapcraft on Windows

To install snapcraft on Windows, you will need to run a Linux VM and follow the above instructions to install snapcraft as a snap. Note that if you are using WSL, only WSL2 with full Linux kernel support will work - you cannot use WSL with snapcraft and snaps. If you like, you can install multipass to launch a Linux VM if your Windows machine has Windows 10 Pro or Enterprise with Hyper-V support. See this forum post for more details.

Building with multipass

The easiest way to build the snap is using the multipass VM tool that snapcraft knows to use directly. After installing multipass, just run

$ snapcraft

Building with LXD containers

Alternatively, you can instruct snapcraft to use LXD containers instead of multipass VM's. This requires installing LXD as documented here.

$ snapcraft --use-lxd

Note that if you are building on non-amd64 hardware, snapcraft won't be able to use it's default LXD container image, so you can follow the next section to create an LXD container to run snapcraft in destructive-mode natively in the container.

Building inside external container/VM using native snapcraft

Finally, snapcraft can be run inside a VM, container or other similar build environment to build the snap without having snapcraft manage the environment (such as in a docker container where snaps are not available, or inside a VM launched from a build-farm without using nested VM's).

This requires creating an Ubuntu 18.04 environment and running snapcraft (from the snap) inside the environment with --destructive-mode.

LXD

Snaps run inside LXD containers just like they do outside the container, so all you need to do is launch an Ubuntu 18.04 container, install snapcraft and run snapcraft like follows:

$ lxc launch ubuntu:18.04 edgex
Creating edgex
Starting edgex
$ lxc exec edgex /bin/bash
root@edgex:~# sudo apt update && sudo apt install snapd squashfuse git -y
root@edgex:~# sudo snap install snapcraft --classic
root@edgex:~# git clone https://github.com/objectbox/edgex-objectbox
root@edgex:~# cd edgex-objectbox && snapcraft --destructive-mode

Building ARMv6hf on Raspberry Pi

The following steps apply when running Raspbian. With Ubuntu Core, you may have luck trying one of the generic build options described in folowing chapters.

Adapted from https://snapcraft.io/docs/build-on-lxd

  1. Make sure you're running an armhf OS version

    dpkg --print-architecture
uname -a

  2. Install snapd & reboot

    sudo apt-get update
sudo apt-get install snapd
sudo reboot

  3. Install LXD and initialize it, accepting all the default options (maybe except for size, 10 GB should be enough). You may need to log out and back in before running lxd init, if it says "command not found".

    sudo snap install lxd
sudo lxd init
sudo usermod -a -G lxd ${USER}
newgrp lxd

  4. Create a new container, connect to it, install snapcraft:

    lxc launch ubuntu:18.04 mysnapcraft
lxc exec mysnapcraft -- /bin/bash
snap install snapcraft --classic

    In case you get an error: cannot communicate with server: ..., just wait a few seconds and try again.

  5. Clone the repo and build the snap. APT update is necessary for part dependencies package discovery.

    apt update
git clone -b fuji --single-branch https://github.com/objectbox/edgex-objectbox.git
cd edgex-objectbox
snapcraft --destructive-mode

  6. In the host environment, get the snap:

    lxc file pull mysnapcraft/root/edgex-objectbox/edgex*.snap .

Building with multipass

The easiest way to build the snap is using the multipass VM tool that snapcraft knows to use directly. After installing multipass, just run

$ snapcraft

Docker

Snapcraft is smart enough to detect when it is running inside a docker container specifically, to the point where no additional arguments are need to snapcraft when it is run inside the container. For example, the upstream snapcraft docker image can be used (only on x86_64 architectures unfortunately) like so:

$ docker run -it -v"$PWD":/build snapcore/snapcraft:stable bash -c "apt update && cd /build && snapcraft"

Note that if you are building your own docker image, you can't run snapd inside the container, and so to install snapcraft, the docker image must download the snapcraft snap and extract it as if it was installed normally inside /snap (same goes for the core and core18 snaps). This is done by the Linux Foundation Jenkins server for the project's CI and you can see an example of that here. The upstream docker image also does this, but only for x86_64 architectures.

Multipass / generic VM

To use multipass to create an Ubuntu 18.04 environment suitable for building the snap (i.e. when running natively on windows):

$ multipass launch bionic -n edgex-snap-build
$ multipass shell edgex-snap-build
multipass@ubuntu:~$ git clone https://github.com/objectbox/edgex-objectbox
multipass@ubuntu:~$ cd edgex-objectbox
multipass~ubuntu:~$ sudo snap install snapcraft --classic
multipass~ubuntu:~$ snapcraft --destructive-mode

The process should be similar for other VM's such as kvm, VirtualBox, etc. where you create the VM, clone the git repository, then install snapcraft as a snap and run with --destructive-mode.

Developing the snap

After building the snap from one of the above methods, you will have a binary snap package called edgex-objectbox_<latest version>_<arch>.snap, which can be installed locally with the --devmode flag:

$ sudo snap install --devmode edgex*.snap

Open Consul - http://127.0.0.1:8500

Note You can try installing a locally built snap with the --dangerous flag (instead of the --devmode flag), but there is a race condition with this method. Specifically Cassandra, and other services require accesses not provided by default to the snap, and these are provided by connecting the interfaces detailed below. The race condition occurs because if the services fail to start because the accesses were denied (because the interfaces weren't connected soon enough), the installation may be entirely aborted by snapd. If you do install with --dangerous, it is recommended to perform the connections detailed below in the same shell command to minimize the time between the installation (and hence service startup) and granting of accesses from interface connection. Note this race condition doesn't happen when installing the snap from the store because the interface connection automatically happens before starting the services.

In addition, if you are using snapcraft with multipass VM's, you can speedup development by not creating a *.snap file and instead running in "try" mode . This is done by running snapcraft try which results in a prime folder placed in the root project directory that can then be "installed" using snap try. For example:

$ snapcraft try # produces prime dir instead of *.snap file
...
You can now run `snap try /home/ubuntu/go/src/github.com/objectbox/edgex-objectbox/prime`.
$ sudo snap try --devmode prime # snap try works the same as snap install, but expects a directory
edgex-objectbox 1.0.0-20190513+0620a8d1 mounted from /home/ubuntu/go/src/github.com/objectbox/edgex-objectbox/prime
$

Interfaces

After installing the snap, you will need to connect interfaces and restart the snap. The snap needs the hardware-observe, mount-observe, and system-observe interfaces connected. These are automatically connected using snap store assertions when installing from the store, but when developing the snap and installing a revision locally, use the following commands to connect the interfaces:

$ sudo snap connect edgex-objectbox:hardware-observe
$ sudo snap connect edgex-objectbox:system-observe
$ sudo snap connect edgex-objectbox:mount-observe

After connecting these restart the services in the snap with:

$ sudo snap restart edgex-objectbox