xoss_sync

Python (CPython and Micropython) codes to fetch FIT files from XOSS G+ cyclo-computer over bluetooth (BLE) for you.

(C) 2024 ekspla

A quick/preliminary version of code to use with XOSS G+ GPS cyclo-computer, inspired by f-xoss project. The code is a modified version of cycsync.py for Cycplus M2, which does not work for my use case as is.

The PC version (xoss_sync.py) was tested with XOSS G+ (Gen1), Windows10/11/Linux(BlueZ 5.56), TPLink USB BT dongle (UB400, v4.0, CSR8510 chip)/Intel Wireless (v5.1), Python-3.8.6/3.12.6 and Bleak-0.22.2.

The Micropython (MPY) version (mpy_xoss_sync.py) was tested with MPY-1.23.0/1.24.0-preview on ESP32-WROOM-32E/ESP32-S3-WROOM-1, SD card, and aioble. After a bit of modification (changes in the path to /sd), this code was also tested with a unix-port of MPY-1.23.0(+ PR#14006)/aioble on the same PC-Linux-x64 and TPLink UB400.

Features

These scripts allow you to:

• Obtain a list of data files on your device
• Download data (in FIT fromat) from your device
• See free/usage of storage in your device

Usage (PC version)

1. Install bluetooth low energy interface/driver software on your PC.

2. Check if your device and the PC are paired.

3. Install python (of course).

4. Install bleak:

pip install bleak

5. Download and run the script python xoss_sync.py:

D:\backup\Bicycle\XOSS\python>python xoss_sync.py
Scanning for Bluetooth devices...
Found device: XOSS G-040989 - EC:37:9F:xx:yy:zz
Found target device: XOSS G-040989 - EC:37:9F:xx:yy:zz
Connected to XOSS G-040989
Notifications started
Free Diskspace: 684/8104kb
Successfully wrote combined data to filelist.txt
Skip: 20240713062144.fit
Skip: 20240609141047.fit
Skip: 20240504063456.fit
Skip: 20240525060956.fit
Skip: 20240511055922.fit
Skip: 20240609130215.fit
Skip: 20240608063049.fit
Skip: 20240615062615.fit
Retrieving 20240715062336.fit
Successfully wrote combined data to 20240715062336.fit
Skip: 20240420055014.fit
Skip: 20240427061131.fit
Skip: 20240502061739.fit
Skip: 20240518060936.fit
Skip: 20240707055411.fit
Skip: 20240429060242.fit
Skip: 20240622055813.fit
Skip: 20240629073413.fit
Skip: 20240506053748.fit
Skip: 20240706062605.fit
Skip: 20240601060515.fit

D:\backup\Bicycle\XOSS\python>

Though I tested this only with XOSS G+ (Gen1) and Windows10/11/Linux(BlueZ 5.56), combinations of the other XOSS device/OS may work. C.f. Bleak supports Android, MacOS, Windows, and Linux.

Usage (Micropython version)

1. Install SD card/interface on your ESP32 board.

2. Install Micropython (of course).

3. Install aioble:

mpremote mip install aioble

4. Download/install and run the script

>>> import mpy_xoss_sync
>>> mpy_xoss_sync.start()

Though it works very well as PC version, this is an ad hoc implementation to MPY/aioble. The code was also tested with MPY-1.24.0-preview/aioble on ESP32-S3 and with unix-port of MPY-1.23.0/aioble on PC-Linux-x64 (Core-i5).

5. Optional

Throughput (see Note 3) can be increased by specifying the optional connection parameters of scan_duration_ms, min_conn_interval_us and max_conn_interval_us as described here. These intervals can be reduced to the minimum value of 7_500 (7.5 ms) on ESP32-S3 and on PC-Linux-x64 using unix port.

Modify async def _connect() in aioble/central.py:

-           ble.gap_connect(device.addr_type, device.addr)
+           ble.gap_connect(device.addr_type, device.addr, 5_000, 11_500, 11_500)

Alternatively, if you have installed the latest aioble after commit 68e3e07,

modify async def run() in mpy_xoss_sync.py:

-               connection = await device.connect(timeout_ms=60_000)
+               connection = await device.connect(timeout_ms=60_000, scan_duration_ms=5_000, min_conn_interval_us=11_500, max_conn_interval_us=11_500)

Limitation

The script seems to work perfectly for my use case as shown above, but there are possible limitations due mainly to the implementation of YMODEM in part as followings.

• The script expects a transport with MTU of 23, 128-byte data per block, and CRC16/ARC (not CRC16/XMODEM). I am not sure if the SoC(seems to be nRF52832)/software in the XOSS device supports larger MTU or 1024-byte data in YMODEM (see, Notes 1 & 2).

Notes

1. My XOSS-G+ (Gen1) was found to be not changing MTU(23)/block data size(128) with Win11 and Bluetooth 5.1 interface, which always requests MTU of 527, while f-xoss project for XOSS-NAV used MTU of 206.

2. The proprietary XOSS App on mobile phone itself seems to support larger MTU/block data size by DLE (data length extension) and STX. See, for example this Xingzhe's web site.

3. Sync times (throughputs in parentheses) using my FIT file of 235,723 bytes were as followings (as of 31 OCT 2024). The connection intervals were measured by using nRF Sniffer for BLE (nRF52840 dongle) and Wireshark.

• Proprietary XOSS App
  • Android-x86 and TPLink UB400, 00:07:27 (4.2 kbps).
    • 50.0 ms connection interval (measured); this could not be changed (see Note 4).
• PC/Bleak version
  • Windows10 and TPLink UB400, 00:03:45 (8.4 kbps).
    • 15.0 ms connection interval (measured).
  • Windows11 and Intel wireless, 00:08:41 (3.6 kbps).
    • 60.0 ms connection interval (measured).
  • Linux (BlueZ 5.56) and TPLink UB400, 00:07:08 (4.4 kbps).
    • 50.0 ms connection interval (measured).
• MPY/aioble version (hereafter: ESP32 = ESP32-WROOM-32E; ESP32-S3 = ESP32-S3-WROOM-1-N16R8)
  • MPY/aioble, ESP32, 00:07:11 (4.4 kbps).
    • 50.0 ms connection interval (measured).
  • MPY/modified aioble(conn_intervals=11.5 ms), ESP32, 00:04:04 (7.7 kbps).
    • 11.25 ms connection interval (measured).
  • MPY/modified aioble(conn_intervals=11.5 ms), ESP32-S3, 00:03:46 (8.3 kbps).
  • MPY/modified aioble(conn_intervals=7.5 ms), reduced NAK/ACK delays and no garbage-collection, ESP32-S3, 00:02:42 (11.6 kbps).
    • Further optimization requires a modified firmware with increased tick-rate in FreeRTOS ; change CONFIG_FREERTOS_HZ from default value of 100 Hz [10 ms] to 1000 Hz [1 ms].
  • MPY(CONFIG_FREERTOS_HZ=1000)/modified aioble(conn_intervals=7.5 ms), optimized delays and no garbage-collection, ESP32-S3, 00:02:05 (15.0 kbps).
    • 7.5 ms connection interval (measured).
    • While the client (mpy_xoss_sync.py) using _thread in file.write() improves a little, 00:02:00 (15.7 kbps), the throughput is determined by the unresponsive peripheral to the ACKs in YMODEM (i.e. no packets sent from XOSS-G+). Typically, 2-4 ACKs (using 2-4 connection events) are necessary irrespective of connection intervals. The theoretical limit of 3 connections/block, as shown below, does not occur because of the unresponsiveness. See example sniffer logs of 7.5 and 50 ms for details. This strange issue, irrespective of the intervals, may be caused by Nordic's SoftDevice in XOSS-G+.
  • MPY(ports/unix)/modified aioble(conn_intervals=7.5 ms), optimized delays and no garbage-collection, PC-Linux-x64 and TPLink UB400, 00:02:25 (13.0 kbps).
    • 7.5 ms connection interval (measured).
    • The throughput was a bit less than those of ESP32-S3, probably because of the difference in bluetooth stacks; BTstack vs. NimBLE. Prior to the YMODEM-ACK an unnecessary empty packet is always sent from BTstack to XOSS-G+, while this is not the case in ESP32s (using NimBLE).
  • Using a pair of test codes (nus_modem_server.py, nus_modem_client.py): MPY/PC-Linux-x64 (server) --> MPY/ESP32-S3 (client), 00:01:08 (27.7 kbps).
    • 7.5 ms connection interval (measured).
    • The throughput was significantly faster without the strange unresponsive delays caused by XOSS-G+.

(c.f.) Theoretical limit using 11.5 ms connection interval on MPY/aioble:

1 s / 11.5 ms = 87 connections; 1 connection = 6 packets * 20 bytes (mtu=23); so, 133 bytes (data/block [128], header [3] and CRC [2]) == 2 connections + 1 connection for ACK in YMODEM.

87 connections/s * (128 data bytes / 3 connections) * 8 bits/byte = 29.7 kbps [this would be 45.5 kbps for 7.5 ms interval].

On Win11, the limits are 1.9, 5.7 and 22.8 kbps for PowerOptimized (180 ms), Balanced (60 ms) and ThroughputOptimized (15 ms) BLE settings, respectively. There is no API in Bleak on Windows to change this setting though. The measured throughput of 3.6 kbps on Win11 using Intel wireless adaptor (as shown above) suggests Balanced setting, which agrees well with those of the measured value using the sniffer. On Linux, the min/max connection intervals may be specified by the user (see below).

4. Conn_min_interval/conn_max_interval on Linux kernels.

Unfortunately, changing the parameters did not work for the XOSS App/Android-x86 in my case.

x86:/ $ su
x86:/ # cat /sys/kernel/debug/bluetooth/hci0/conn_min_interval
40                                                                      # 40 * 1.25  = 50 ms
x86:/ # cat /sys/kernel/debug/bluetooth/hci0/conn_max_interval
56                                                                      # 56 * 1.25 = 70 ms
x86:/ # echo 9 > /sys/kernel/debug/bluetooth/hci0/conn_min_interval     # 9 * 1.25 = 11.25 ms
x86:/ # echo 20 > /sys/kernel/debug/bluetooth/hci0/conn_max_interval    # 20 * 1.25 = 25 ms

5. LE 2M PHY support of XOSS-G+.

Although my XOSS-G+ shows LE_2M_PHY = True (BLE 5.0) in the feature response packet, it stops communication silently and starts advertising again after receiving a LL_PHYS_REQ (preference of 2M PHY) packet. It seems that the client's request of changing from 1M to 2M is not handled appropriately in the XOSS-G+ software as specified in the Bluetooth Core Spec. This is similar to the case of unfunctional Data Length Extension (DLE) = True (BLE 4.2) as described in Notes 1 & 2.

I am not sure if these problems are solved in the latest models.