This project is an extention on Pub-sub distributed system part1.
You can find part 1 here: https://github.com/CS682-S22/distributed-system-part1-Jennytang1224
I'm adding two additional features in this project:
1. Bully Election
2. Heartbeat message
3. Membership table
4. Replication: both asynchronous and synchronous replications
5. Push-based broker
6. Performance analysis
The existing program is a simple version of Kafka pub-sub system:
- Producers: will send data to all brokers through load balancer.
- Load Balancer: will calculate the partition number based the "key" inside of data and pass data to the brokers.
- Brokers: maintain their own topic map that stores data based on assigned partition number.
- consumers: requesting data from brokers given the topic and starting position.
Currently I have pull-based broker with partition implemented.
Variables:
- pull-based broker and push-based broker
- number of partitions
- number of producers
- number of brokers
- number of topics
- randomly-generate partitionID VS. mod-generated partitionID
Measurement:
will use runtime as measurement to compare result by testing on different variables
Paper:
will producer a short paper to descirbe the result and analyze the data with graphs
By May 10
- implement push-based brokers
- set breakpoints for runtime measurement
- experiment on pull-based broker and push-based broker
By May 12
- experiment on number of data
- experiment on number of partitions
- experiment on number of brokers
- experiment on randomly-generate partitionID VS. mod-generated partitionID
- experiment on consumer runtime
- experiment on loadbalancer runtime
By May 15
- generate graphs
- write short paper