Multi stream applications (like torrent) can benefits from multiple available network. Some of them even have support for multiple NICs yet we have to setup iptables rules to redirect their flows to the right direction (gw) based on the oif, which is decided by the application.
This approach is reusing the generic cgroup v2 eBPF hook support therefore completely agnostic wether the application supports load balancing or not. The eBPF prog attached to a given cgroup (or the cgroup root to system wide operation) and randomly select a VRF for the socket.
For eBPF compiling and handling (bpftool):
sudo apt install linux-tools-common clang llvm
The libbpf also required, installation guide here.
Lets assume a test setup with a wired network interface usb0 which is actually a shared mobile data and the wireless wlp1s0 interface which is the built in notebook wifi. The implicit default VRF contains the main routing table, where the network of the usb0 used as the default route and the wifi as a backup (default linux behavior when both interface active). We have to create a VRF for the wifi where the network of the wlp1s0 used as default. If we enslave an interface to a VRF, that will override the routing lookups according to the VRF's routing table.
# VRF setup with routing table id 10
sudo ip link add wifi-vrf type vrf table 10
sudo ip link set dev wifi-vrf up
# Put the wlp1s0 interface into the VRF
sudo ip link set wlp1s0 master wifi-vrf
sudo ip route add 10.0.0.0/24 dev wlp1s0 scope link table 10
sudo ip route add default via 10.0.0.1 table 10
The last two commands are not required in theory, because when we set the VRF as master for the interface, all routes are associated with that interface will be automatically moved to the VRF's table. However for some reasons the scope link route didn't moved for me, and we didn't have default route either because the main table use the usb0 route for that.
The next step is to create a cgroup for the applications where we want to use the socket loadbalancing. Then we load the eBPF prog, attach it to the cgroup and put a bash into that cgroup. Therefore every application started from that bash will be loadbalanced between the default and the wifi VRFs. Now we do simple round-robing loadbalancing based on a random value but more sofisticated logic could be relaized in the eBPF prog.
sudo mkdir /sys/fs/cgroup/unified/user.slice/loadbalance
bpftool prog load vrf_setsockopt.o /sys/fs/bpf/binder
bpftool cgroup attach /sys/fs/cgroup/unified/user.slice/loadbalance connect4 pinned /sys/fs/bpf/binder
sudo -i
echo $$ >> /sys/fs/cgroup/unified/user.slice/loadbalance/cgroup.procs
To verify the operation we can use the iperf3 with paralell (-P) mode. In the end of the test the [SUM] will show the aggreagated througput of the networks. Also, the local addresses will be different per connection. In the example below the IP address of the usb0 is 192.168.42.164 and the wlp1s0 is 10.0.0.3:
# Without loadbalacing:
iperf3 -c 77.120.3.236 -R -P10
Connecting to host 77.120.3.236, port 5201
Reverse mode, remote host 77.120.3.236 is sending
[ 5] local 192.168.42.164 port 58586 connected to 77.120.3.236 port 5201
[ 7] local 192.168.42.164 port 58588 connected to 77.120.3.236 port 5201
[ 9] local 192.168.42.164 port 58590 connected to 77.120.3.236 port 5201
[ 11] local 192.168.42.164 port 58592 connected to 77.120.3.236 port 5201
[ 13] local 192.168.42.164 port 58594 connected to 77.120.3.236 port 5201
[ 15] local 192.168.42.164 port 58596 connected to 77.120.3.236 port 5201
[ 17] local 192.168.42.164 port 58598 connected to 77.120.3.236 port 5201
[ 19] local 192.168.42.164 port 58600 connected to 77.120.3.236 port 5201
[ 21] local 192.168.42.164 port 58602 connected to 77.120.3.236 port 5201
[ 23] local 192.168.42.164 port 58604 connected to 77.120.3.236 port 5201
# With loadbalancing:
iperf3 -c 77.120.3.236 -R -P10
Connecting to host 77.120.3.236, port 5201
Reverse mode, remote host 77.120.3.236 is sending
[ 5] local 192.168.42.164 port 58634 connected to 77.120.3.236 port 5201
[ 7] local 10.0.0.3 port 51998 connected to 77.120.3.236 port 5201
[ 9] local 192.168.42.164 port 58638 connected to 77.120.3.236 port 5201
[ 11] local 192.168.42.164 port 58640 connected to 77.120.3.236 port 5201
[ 13] local 192.168.42.164 port 58642 connected to 77.120.3.236 port 5201
[ 15] local 10.0.0.3 port 52006 connected to 77.120.3.236 port 5201
[ 17] local 10.0.0.3 port 52008 connected to 77.120.3.236 port 5201
[ 19] local 10.0.0.3 port 52010 connected to 77.120.3.236 port 5201
[ 21] local 192.168.42.164 port 58650 connected to 77.120.3.236 port 5201
[ 23] local 10.0.0.3 port 52014 connected to 77.120.3.236 port 5201
The minimum requirement is Linux 5.8 (with bpf_setsockopt SO_BINDTODEVICE option). Also DNS lookup inside VRFs currently broken on systemd based distros like Ubuntu, more on that here.