Skip to content

Commit

Permalink
Merge pull request #46 from theelims/eventsocket
Browse files Browse the repository at this point in the history
Release Event Socket as 0.4.0
  • Loading branch information
theelims authored Apr 21, 2024
2 parents fd5bfb0 + 70a9f91 commit 1622a40
Show file tree
Hide file tree
Showing 55 changed files with 2,086 additions and 1,153 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,4 @@ node_modules
*WWWData.h
lib/framework/WWWData.h
ssl_certs/cacert.pem
/logs
43 changes: 29 additions & 14 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,37 +2,52 @@

All notable changes to this project will be documented in this file.

## [WIP] - Work in Progress
## [0.4.0] - 2024-04-21

This upgrade might require one minor change as `MqttPubSub.h` and its class had been renamed to `MqttEndpoint.h` and `MqttEndoint` respectively. However, it is strongly advised, that you change all existing WebSocketServer endpoints to the new event socket system.

> [!NOTE]
> The new Event Socket system is likely to change with coming updates.
### Added

- Added build flag `-D SERIAL_INFO` to platformio.ini to enable / disable all `Serial.print()` statements. On some boards with native USB those Serial prints have been reported to block and make the server unresponsive.
- Added a hook handler to StatefulService. Unlike an UPDATE a hook is called every time a state receives and updated, even if the result is UNCHANGED or ERROR.
- Added missing include for S2 in SystemStatus.cpp (#23)
- Added awareness of front end build script for all 3 major JS package managers. The script will auto-identify the package manager by the lock-file. (#40)
- Added a hook handler to StatefulService. Unlike an UPDATE a hook is called every time a state receives an updated, even if the result is UNCHANGED or ERROR.
- Added missing include for S2 in SystemStatus.cpp [#23](https://github.com/theelims/ESP32-sveltekit/issues/23)
- Added awareness of front end build script for all 3 major JS package managers. The script will auto-identify the package manager by the lock-file. [#40](https://github.com/theelims/ESP32-sveltekit/pull/40)
- Added a new event socket to bundle the websocket server and the notifications events. This saves on open sockets and allows for concurrent visitors of the internal website. The normal websocket server endpoint remains as an option, should a pure websocket connection be desired. An EventEndpoint was added to use this with Stateful Services. [#29](https://github.com/theelims/ESP32-sveltekit/issues/29) and [#43](https://github.com/theelims/ESP32-sveltekit/pull/43)
- TS Types definition in one central place for the frontend.

### Changed

- more generic board definition in platformio.ini (#20)
- refactored MqttPubSub.h into a single class to improve readability
- Moves appName and copyright to `layout.ts` to keep customization in one place (#31)
- Make eventSource use timeout for reconnect (#34)
- Make each toasts disappear after timeout (#35)
- more generic board definition in platformio.ini [#20](https://github.com/theelims/ESP32-sveltekit/pull/20)
- Renamed `MqttPubSub.h` and class to `MqttEndpoint.h` and class.
- refactored MqttEndpoint.h into a single class to improve readability
- Moves appName and copyright to `layout.ts` to keep customization in one place [#31](https://github.com/theelims/ESP32-sveltekit/pull/31)
- Make event source use timeout for reconnect [#34](https://github.com/theelims/ESP32-sveltekit/pull/34)
- Make each toasts disappear after timeout [#35](https://github.com/theelims/ESP32-sveltekit/pull/35)
- Fixed version `platform = espressif32 @ 6.6.0` in platformio.ini
- Analytics data limited to 1000 data points (roughly 33 minutes).
- postcss.config.cjs as ESM module [#24](https://github.com/theelims/ESP32-sveltekit/issues/24)

### Fixed

- Duplicate method in FeatureService (#18)
- Fixed compile error with FLAG `-D SERVE_CONFIG_FILES`
- Fixed typo in telemetry.ts (#38)
- Fixed the development warning: `Loading /rest/features using `window.fetch`. For best results, use the `fetch`that is passed to your`load` function:`
- Fixed typo in telemetry.ts [#38](https://github.com/theelims/ESP32-sveltekit/pull/38)
- Fixed the development warning: `Loading /rest/features using 'window.fetch'. For best results, use the 'fetch' that is passed to your 'load' function:`

### Removed

- Duplicate method in FeatureService [#18](https://github.com/theelims/ESP32-sveltekit/pull/18)
- Duplicate lines in Systems Settings view.
- Removes duplicate begin (#36)
- Removes duplicate begin [#36](https://github.com/theelims/ESP32-sveltekit/pull/36)
- Temporary disabled OTA progress update due to crash with PsychicHttp [#32](https://github.com/theelims/ESP32-sveltekit/issues/32) until a fix is found.

### Known Issues

- On ESP32-C3 the security features should be disabled in features.ini: `-D FT_SECURITY=0`. If enabled the ESP32-C3 becomes extremely sluggish with frequent connection drops.

## [0.3.0] - 2023-02-05
## [0.3.0] - 2024-02-05

> [!CAUTION]
> This update has breaking changes!
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ SvelteKit is ideally suited to be served from constrained devices like an ESP32.

### :telephone: Rich Communication Interfaces

Comes with a rich set of communication interfaces to cover most standard needs of an IoT application. Like MQTT client, HTTP RESTful API or WebSocket Server. All communication channels are stateful and fully synchronized. Changes propagate and are communicated to all other participants. The states can be persisted on the file system as well. For accurate time keeping time can by synchronized over NTP.
Comes with a rich set of communication interfaces to cover most standard needs of an IoT application. Like MQTT client, HTTP RESTful API, a WebSocket based Event Socket and a classic Websocket Server. All communication channels are stateful and fully synchronized. Changes propagate and are communicated to all other participants. The states can be persisted on the file system as well. For accurate time keeping time can by synchronized over NTP.

### :file_cabinet: WiFi Provisioning and Management

Expand Down
10 changes: 10 additions & 0 deletions docs/buildprocess.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,16 @@ build_flags =
-D SERVE_CONFIG_FILES
```

### Serial Info

In some circumstances it might be beneficial to not print any information on the serial consol (Serial1 or USB CDC). By commenting out the following build flag ESP32-Sveltekit will not print any information on the serial console.

```ini
build_flags =
...
-D SERIAL_INFO
```

## SSL Root Certificate Store

Some features like firmware download or the MQTT client require a SSL connection. For that the SSL Root CA certificate must be known to the ESP32. The build system contains a python script derived from Espressif ESP-IDF building a certificate store containing one or more certificates. In order to create the store you must uncomment the three lines below in `platformio.ini`.
Expand Down
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ SvelteKit is ideally suited to be served from constrained devices like an ESP32.

### :telephone: Rich Communication Interfaces

Comes with a rich set of communication interfaces to cover most standard needs of an IoT application. Like MQTT client, HTTP RESTful API or WebSocket Server. All communication channels are stateful and fully synchronized. Changes propagate and are communicated to all other participants. The states can be persisted on the file system as well. For accurate time keeping time can by synchronized over NTP.
Comes with a rich set of communication interfaces to cover most standard needs of an IoT application. Like MQTT client, HTTP RESTful API, a WebSocket based Event Socket and a classic Websocket Server. All communication channels are stateful and fully synchronized. Changes propagate and are communicated to all other participants. The states can be persisted on the file system as well. For accurate time keeping time can by synchronized over NTP.

### :file_cabinet: WiFi Provisioning and Management

Expand Down
120 changes: 91 additions & 29 deletions docs/statefulservice.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ class LightStateService : public StatefulService<LightState> {
};
```
### Update Handler
You may listen for changes to state by registering an update handler callback. It is possible to remove an update handler later if required.
```cpp
Expand All @@ -74,9 +76,11 @@ An "originId" is passed to the update handler which may be used to identify the
| Origin | Description |
| -------------------------- | ----------------------------------------------- |
| http | An update sent over REST (HttpEndpoint) |
| mqtt | An update sent over MQTT (MqttPubSub) |
| mqtt | An update sent over MQTT (MqttEndpoint) |
| websocketserver:{clientId} | An update sent over WebSocket (WebSocketServer) |

### Hook Handler

Sometimes if can be desired to hook into every update of an state, even if the StateUpdateResult is `StateUpdateResult::UNCHANGED` and the update handler isn't called. In such cases you can use the hook handler. Similarly it can be removed later.

```cpp
Expand Down Expand Up @@ -119,7 +123,7 @@ There are three possible return values for an update function which are as follo
| StateUpdateResult::UNCHANGED | The state was unchanged, propagation should not take place |
| StateUpdateResult::ERROR | There was an error updating the state, propagation should not take place |

### Serialization
### JSON Serialization

When reading or updating state from an external source (HTTP, WebSockets, or MQTT for example) the state must be marshalled into a serializable form (JSON). SettingsService provides two callback patterns which facilitate this internally:

Expand Down Expand Up @@ -165,7 +169,7 @@ JsonObject jsonObject = jsonDocument.as<JsonObject>();
lightStateService->update(jsonObject, LightState::update, "timer");
```
### Endpoints
### HTTP RESTful Endpoint
The framework provides an [HttpEndpoint.h](https://github.com/theelims/ESP32-sveltekit/blob/main/lib/framework/HttpEndpoint.h) class which may be used to register GET and POST handlers to read and update the state over HTTP. You may construct an HttpEndpoint as a part of the StatefulService or separately if you prefer.
Expand All @@ -191,7 +195,7 @@ Endpoint security is provided by authentication predicates which are [documented

To register the HTTP endpoints with the web server the function `_httpEndpoint.begin()` must be called in the custom StatefulService Class' own `void begin()` function.

### Persistence
### File System Persistence

[FSPersistence.h](https://github.com/theelims/ESP32-sveltekit/blob/main/lib/framework/FSPersistence.h) allows you to save state to the filesystem. FSPersistence automatically writes changes to the file system when state is updated. This feature can be disabled by calling `disableUpdateHandler()` if manual control of persistence is required.

Expand All @@ -209,7 +213,33 @@ class LightStateService : public StatefulService<LightState> {
};
```
### WebSockets
### Event Socket Endpoint
[EventEndpoint.h](https://github.com/theelims/ESP32-sveltekit/blob/main/lib/framework/EventEndpoint.h) wraps the [Event Socket](#event-socket) into an endpoint compatible with a stateful service. The client may subscribe and unsubscribe to this event to receive updates or push updates to the ESP32. The current state is synchronized upon subscription.
The code below demonstrates how to extend the LightStateService class to provide an WebSocket:
```cpp
class LightStateService : public StatefulService<LightState> {
public:
LightStateService(EventSocket *socket) :
_eventEndpoint(LightState::read, LightState::update, this, socket, "led") {}
void begin()
{
_eventEndpoint.begin();
}
private:
EventEndpoint<LightState> _eventEndpoint;
};
```

To register the event endpoint with the event socket the function `_eventEndpoint.begin()` must be called in the custom StatefulService Class' own `void begin()` function.

Since all events run through one websocket connection it is not possible to use the [securityManager](#security-features) to limit access to individual events. The security defaults to `AuthenticationPredicates::IS_AUTHENTICATED`.

### WebSocket Server

[WebSocketServer.h](https://github.com/theelims/ESP32-sveltekit/blob/main/lib/framework/WebSocketServer.h) allows you to read and update state over a WebSocket connection. WebSocketServer automatically pushes changes to all connected clients when state is updated.

Expand All @@ -235,11 +265,11 @@ WebSocket security is provided by authentication predicates which are [documente
To register the WS endpoint with the web server the function `_webSocketServer.begin()` must be called in the custom StatefulService Class' own `void begin()` function.
### MQTT
### MQTT Client
The framework includes an MQTT client which can be configured via the UI. MQTT requirements will differ from project to project so the framework exposes the client for you to use as you see fit. The framework does however provide a utility to interface StatefulService to a pair of pub/sub (state/set) topics. This utility can be used to synchronize state with software such as Home Assistant.
[MqttPubSub.h](https://github.com/theelims/ESP32-sveltekit/blob/main/lib/framework/MqttPubSub.h) allows you to publish and subscribe to synchronize state over a pair of MQTT topics. MqttPubSub automatically pushes changes to the "pub" topic and reads updates from the "sub" topic.
[MqttEndpoint.h](https://github.com/theelims/ESP32-sveltekit/blob/main/lib/framework/MqttEndpoint.h) allows you to publish and subscribe to synchronize state over a pair of MQTT topics. MqttEndpoint automatically pushes changes to the "pub" topic and reads updates from the "sub" topic.
The code below demonstrates how to extend the LightStateService class to interface with MQTT:
Expand All @@ -248,7 +278,7 @@ The code below demonstrates how to extend the LightStateService class to interfa
class LightStateService : public StatefulService<LightState> {
public:
LightStateService(AsyncMqttClient* mqttClient) :
_mqttPubSub(LightState::read,
_mqttEndpoint(LightState::read,
LightState::update,
this,
mqttClient,
Expand All @@ -257,18 +287,68 @@ class LightStateService : public StatefulService<LightState> {
}
private:
MqttPubSub<LightState> _mqttPubSub;
MqttEndpoint<LightState> _mqttEndpoint;
};
```

You can re-configure the pub/sub topics at runtime as required:

```cpp
_mqttPubSub.configureBroker("homeassistant/light/desk_lamp/set", "homeassistant/light/desk_lamp/state");
_mqttEndpoint.configureBroker("homeassistant/light/desk_lamp/set", "homeassistant/light/desk_lamp/state");
```

The demo project allows the user to modify the MQTT topics via the UI so they can be changed without re-flashing the firmware.

## Event Socket

Beside RESTful HTTP Endpoints the Event Socket System provides a convenient communication path between the client and the ESP32. It uses a single WebSocket connection to synchronize state and to push realtime data to the client. The client needs to subscribe to the topics he is interested. Only clients who have an active subscription will receive data. Every authenticated client may make use of this system as the security settings are set to `AuthenticationPredicates::IS_AUTHENTICATED`.

### Emit an Event

The Event Socket provides an overloaded `emit()` function to push data to all subscribed clients. This is used by various esp32sveltekit classes to push real time data to the client. First an event must be registered with the Event Socket by calling `_socket.registerEvent("CustomEvent");`. Only then clients may subscribe to this custom event and you're entitled to emit event data:

```cpp
void emit(String event, String payload);
void emit(const char *event, const char *payload);
void emit(const char *event, const char *payload, const char *originId, bool onlyToSameOrigin = false);
```
The latter function allowing a selection of the recipient. If `onlyToSameOrigin = false` the payload is distributed to all subscribed clients, except the `originId`. If `onlyToSameOrigin = true` only the client with `originId` will receive the payload. This is used by the [EventEndpoint](#event-socket-endpoint) to sync the initial state when a new client subscribes.
### Receive an Event
A callback or lambda function can be registered to receive an ArduinoJSON object and the originId of the client sending the data:
```cpp
_socket.onEvent("CostumEvent",[&](JsonObject &root, int originId)
{
bool ledState = root["led_on"];
});
```

### Get Notified on Subscriptions

Similarly a callback or lambda function may be registered to get notified when a client subscribes to an event:

```cpp
_socket.onEvent("CostumEvent",[&](const String &originId, bool sync)
{
Serial.println("New Client subscribed: " + originId);
});
```

The boolean parameter provided will always be `true`.

### Push Notifications to All Clients

It is possibly to send push notifications to all clients by using the Event Socket. These will be displayed as toasts an the client side. Either directly call

```cpp
esp32sveltekit.getSocket()->pushNotification("Pushed a message!", PUSHINFO);
```
or keep a local pointer to the `EventSocket` instance. It is possible to send `PUSHINFO`, `PUSHWARNING`, `PUSHERROR` and `PUSHSUCCESS` events to all clients.
## Security features
The framework has security features to prevent unauthorized use of the device. This is driven by [SecurityManager.h](https://github.com/theelims/ESP32-sveltekit/blob/main/lib/framework/SecurityManager.h).
Expand Down Expand Up @@ -397,24 +477,6 @@ esp32sveltekit.recoveryMode();

will force a start of the AP regardless of the AP settings. It will not change the the AP settings. To exit the recovery mode restart the device or change the AP settings in the UI.

### Push Notifications to All Clients

It is possibly to send push notifications to all clients by using Server Side Events. These will be displayed as toasts an the client side. Either directly call

```cpp
esp32sveltekit.getNotificationEvents()->pushNotification("Pushed a message!", PUSHINFO, millis());
```
or keep a local pointer to the `NotificationEvents` instance. It is possible to send `PUSHINFO`, `PUSHWARNING`, `PUSHERROR` and `PUSHSUCCESS` events to all clients. The HTTP endpoint for this service is at `/events/notifications`.
In addition the raw `send()` function is mapped out as well:
```cpp
esp32sveltekit.getNotificationEvents()->send("Pushed a message!", "event", millis());
```

This allows you to send your own Server-Sent Events without opening a new HTTP connection.

### Power Down with Deep Sleep

This API service can place the ESP32 in the lowest power deep sleep mode consuming only a few µA. It uses the EXT1 wakeup source, so the ESP32 can be woken up with a button or from a peripherals interrupt. Consult the [ESP-IDF Api Reference](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/system/sleep_modes.html#_CPPv428esp_sleep_enable_ext1_wakeup8uint64_t28esp_sleep_ext1_wakeup_mode_t) which GPIOs can be used for this. The RTC will also be powered down, so an external pull-up or pull-down resistor is required. It is not possible to persist variable state through the deep sleep. To optimize the deep sleep power consumption it is advisable to use the callback function to put pins with external pull-up's or pull-down's in a special isolated state to prevent current leakage. Please consult the [ESP-IDF Api Reference](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/system/sleep_modes.html#configuring-ios-deep-sleep-only) for this.
Expand All @@ -441,7 +503,7 @@ esp32sveltekit.getSleepService()->sleepNow();

### Battery State of Charge

A small helper class let's you update the battery icon in the status bar. This is useful if you have a battery operated IoT device. It must be enabled in [features.ini](https://github.com/theelims/ESP32-sveltekit/blob/main/features.ini). It uses Server-sent events and exposes two functions that can be used to update the clients.
A small helper class let's you update the battery icon in the status bar. This is useful if you have a battery operated IoT device. It must be enabled in [features.ini](https://github.com/theelims/ESP32-sveltekit/blob/main/features.ini). It uses the [Event Socket](#event-socket) and exposes two functions that can be used to update the clients.

```cpp
esp32sveltekit.getBatteryService()->updateSOC(float stateOfCharge); // update state of charge in percent (0 - 100%)
Expand Down
Loading

0 comments on commit 1622a40

Please sign in to comment.