Alpha (#236)

* 109 to alpha (#189) * 1st impl for #109 Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * Arch refactor alpha (#190) * add crpto interface as repare for #127 Signed-off-by: Sam Yuan <yy19902439@126.com> * adding Worker interface for #56 and decouple Assembler and Integrator Signed-off-by: Sam Yuan <yy19902439@126.com> * refactor for worker interface Signed-off-by: Sam Yuan <yy19902439@126.com> * package structure Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * package refactor Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * remove Envelope from elements Signed-off-by: Sam Yuan <yy19902439@126.com> * add memeory free Signed-off-by: Sam Yuan <yy19902439@126.com> * remove Proposal from Elements Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * move start time to ctx Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up for time setting (#191) * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * impl for #173 supports for configurable signers in parallel (#194) Signed-off-by: Sam Yuan <yy19902439@126.com> * adding for commit phase only dummy impl (#195) * adding for commit phase only dummy impl Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * impl for endorsement only Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * update document and prepare for release (#196) * update document and prepare for release Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up in readme (#197) Signed-off-by: Sam Yuan <yy19902439@126.com> * impl for #200 (#207) Signed-off-by: Sam Yuan <yy19902439@126.com> * Impl173 (#208) * adjust code Signed-off-by: Sam Yuan <yy19902439@126.com> * code refactor Signed-off-by: Sam Yuan <yy19902439@126.com> * code refactor Signed-off-by: Sam Yuan <yy19902439@126.com> * use const instead Signed-off-by: Sam Yuan <yy19902439@126.com> * adding parallel flag Signed-off-by: Sam Yuan <yy19902439@126.com> * try to check escapes in ci (#209) * try to check escapes in ci Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * add benchmark test and adjust regex impl solution Signed-off-by: Sam Yuan <yy19902439@126.com> * attempt Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * 174 (#210) * use kingpin global flag and adding test case with timeout command Signed-off-by: Sam Yuan <yy19902439@126.com> * impl for #174 Signed-off-by: Sam Yuan <yy19902439@126.com> * adding test case Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * add signal listening Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * adding patch for #174 (#211) * adding patch for #174 Signed-off-by: Sam Yuan <yy19902439@126.com> * bug fix Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * impl for #213 (#214) * impl for #213 Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * add orderer observer into full workflow process (#216) * add orderer observer into full workflow process Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * impl for txid tracing at traffic generate (#218) * impl for txid tracing at traffic generate Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * try to add tx tracing for observer (#219) Signed-off-by: Sam Yuan <yy19902439@126.com> * nit fix (#220) Signed-off-by: Sam Yuan <yy19902439@126.com> * Nitfix for date format (#221) * use RFC3339Nano Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * impl for #223 (#225) * impl for #223 Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up (#226) * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up (#227) * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up delete map to avoid OOM (#228) Signed-off-by: Sam Yuan <yy19902439@126.com> * update for git doc (#235) Signed-off-by: Sam Yuan <yy19902439@126.com> * impl for policy support (#237) * add plan Signed-off-by: Sam Yuan <yy19902439@126.com> * update ginkgo and add test for policy impl Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * try to impl with policy support Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com> * fix up Signed-off-by: Sam Yuan <yy19902439@126.com>
Hyperledger-TWGC · Mar 23, 2022 · 02bde3a · 02bde3a
1 parent 1195da7
commit 02bde3a
Show file tree

Hide file tree

Showing 77 changed files with 4,098 additions and 1,093 deletions.
diff --git a/.gitignore b/.gitignore
@@ -14,3 +14,6 @@
 
 # Files from IDE
 *.idea/
+*.vscode/
+
+e2e/Tape.log
diff --git a/Makefile b/Makefile
@@ -39,6 +39,10 @@ tape:
 	@echo "Building tape program......"
 	go build -tags "$(GO_TAGS)" -ldflags "$(GO_LDFLAGS)" ./cmd/tape
 
+escapes:
+	@echo "go build check for escapes"
+	go build -gcflags="-m -l" ./... | grep "escapes to heap" || true
+
 .PHONY: docker
 docker:
 	@echo "Building tape docker......"

diff --git a/README.md b/README.md
@@ -1,84 +1,84 @@
-# Tape
-A light-weight tool to test performance of Hyperledger Fabric
+# Tape : Aplha
 
-English/[中文](README-zh.md)
-
-[![Build Status](https://dev.azure.com/Hyperledger/TWGC/_apis/build/status/Hyperledger-TWGC.tape?branchName=master)](https://dev.azure.com/Hyperledger/TWGC/_build/latest?definitionId=135&branchName=master)
-[![Build Status](https://dev.azure.com/Hyperledger/TWGC/_apis/build/status/Hyperledger-TWGC.tape?branchName=alpha)](https://dev.azure.com/Hyperledger/TWGC/_build/latest?definitionId=135&branchName=alpha)
-
-<img src="logo.svg" width="100">
-
-## Why Tape
-
-Sometimes we need to test performance of a deployed Fabric network with ease. There are many excellent projects out there, i.e. Hyperledger Caliper. However, we sometimes just need a tiny, handy tool, like `tape`.
+## A light-weight tool to test performance of Hyperledger Fabric
 
-## What is it
-This includes a very simple traffic generator:
-- it does not use any SDK
-- it does not attempt to deploy Fabric
-- it does not rely on connection profile
-- it does not discover nodes, chaincodes, or policies
-- it does not monitor resource utilization
+English/[中文](README-zh.md)
 
-It is used to perform super simple performance test:
-- it directly establishes number of gRPC connections
-- it sends signed proposals to peers via number of gRPC clients
-- it assembles endorsed responses into envelopes
-- it sends envelopes to orderer
-- it observes transaction commitment
+---
+[**Sample run of Tape**](https://www.bilibili.com/video/BV1k5411L79)
 
-Our main focus is to make sure that *tape will not be the bottleneck of performance test*
+---
+## Table Of Content
 
-## Usage
+* [Prerequisites](#prerequisites)
+* [Configure](docs/configfile.md)
+* [Usage](#usage)
+* [Contributing](#contributing)
+* [License](#license)
+* [Contact](#contact)
+* [Regards](#thanks-for-choosing)
 
-### Install
+---
+## Prerequisites
 
 You could get `tape` in three ways:
 1. Download binary: get release tar from [release page](https://github.com/hyperledger-twgc/tape/releases), and extract `tape` binary from it
 2. Build from source: clone this repo and run `make tape` at root dir. Go1.14 or higher is required. `tape` binary will be available at project root directory.
-3. Pull docker image: `docker pull ghcr.io/hyperledger-twgc/tape`
+3. Pull docker image: `docker pull ghcr.io/hyperledger-twgc/tape` or `docker pull ghcr.io/hyperledger-twgc/tape:alpha`
+---
 
-### [Configure](docs/configfile.md)
+## [Configure](docs/configfile.md)
 
-### Run
+## Usage
 
-#### Binary
+### Binary
 
 Execute `./tape -c config.yaml -n 40000` to generate 40000 transactions to Fabric.
 
-#### Docker
 
-```
+### Docker
 
+```
 docker run -v $PWD:/tmp ghcr.io/hyperledger-twgc/tape tape -c $CONFIG_FILE -n 40000
 ```
 
 *Set this to integer times of batchsize, so that last block is not cut due to timeout*. For example, if you have batch size of 500, set this to 500, 1000, 40000, 100000, etc.
 
-## Tips
-
-- We use [logrus](https://github.com/sirupsen/logrus) for logging, which can be set with env var `export TAPE_LOGLEVEL=debug`.
-Here are possbile values (warn by default)
-`"panic", "fatal", "error", "warn", "warning", "info", "debug", "trace"`
+### CommitOnly
+```
 
-- Put this generator closer to Fabric, or even on the same machine. This is to prevent network bandwidth from being the bottleneck.
+docker run -v $PWD:/tmp guoger/tape tape commitOnly -c $CONFIG_FILE -n 40000
 
-- Increase number of messages per block in your channel configuration may help
-- [Workflow](docs/workflow.md)
+```
 
 
+### EndorsementOnly
+```
 
-## [How to Contribute](CONTRIBUTING.md)
+docker run -v $PWD:/tmp guoger/tape tape endorsementOnly -c $CONFIG_FILE -n 40000
 
-## [Maintainers](MAINTAINERS.md)
+```
 
-## Recent Updates
-Recently, we opened some features at alpha branch, you can take a look at [here](https://github.com/Hyperledger-TWGC/tape/tree/alpha).
+---
+## Contributing
+Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are **greatly appreciated**.
 
-## LICENSE
+1. Fork the Project
+2. Create your Feature Branch (`git checkout -b feature/AmazingFeature`)
+3. Commit your Changes (`git commit -s`)
+4. Push to the Branch (`git push origin feature/AmazingFeature`)
+5. Open a Pull Request
+6. [How to Contribute](CONTRIBUTING.md)
 
+---
+## License
 Hyperledger Project source code files are made available under the Apache License, Version 2.0 (Apache-2.0), located in the [LICENSE](LICENSE) file.
 
-## Credits
+---
+## Contact
+
+* [Maintainers](MAINTAINERS.md)
+---
+
+### THANKS FOR CHOOSING
 
-Icons made by <a href="https://www.flaticon.com/authors/good-ware" title="Good Ware">Good Ware</a> from <a href="https://www.flaticon.com/" title="Flaticon">www.flaticon.com</a>
diff --git a/azure-pipelines.yml b/azure-pipelines.yml
@@ -44,6 +44,8 @@ jobs:
         displayName: run basic checks like linter
       - script: make tape
         displayName: build tape binary program
+      - script: make escapes
+        displayName: run escapes check for tape
       - script: make install
         displayName: install tape to GOBIN
       - script: make unit-test
@@ -85,13 +87,24 @@ jobs:
           FABRIC_VERSION: "1_4"
           INTERGATION_CASE: "ANDLogic"
         LATEST:
-          FABRIC_VERSION: "latest"
-          INTERGATION_CASE: "ANDLogic"
+          FABRIC_VERSION: 'latest'
+          INTERGATION_CASE: 'ANDLogic'
+        LATEST_ORLogic:
+          FABRIC_VERSION: 'latest'
+          INTERGATION_CASE: 'ORLogic'
+        LATEST_COMMITONLY:
+          FABRIC_VERSION: 'latest'
+          INTERGATION_CASE: 'COMMITONLY'
+        LATEST_ENODRSEMENTONLY:
+          FABRIC_VERSION: 'latest'
+          INTERGATION_CASE: 'ENDORSEMNTONLY'
     steps:
       - script: make integration-test FABRIC_VERSION=$(FABRIC_VERSION) INTERGATION_CASE=$(INTERGATION_CASE)
 
   - job: BuildAndReleaseBinaries
     displayName: BuildAndReleaseBinaries
+    pool:
+      vmImage: "ubuntu-latest"
     dependsOn:
       - integrationTest
     steps:

diff --git a/cmd/tape/main.go b/cmd/tape/main.go
@@ -4,33 +4,53 @@ import (
 	"fmt"
 	"os"
 
-	"github.com/hyperledger-twgc/tape/pkg/infra"
+	"github.com/Hyperledger-TWGC/tape/pkg/infra"
+	"github.com/Hyperledger-TWGC/tape/pkg/infra/cmdImpl"
 
 	"github.com/pkg/errors"
 	log "github.com/sirupsen/logrus"
 	"gopkg.in/alecthomas/kingpin.v2"
 )
 
 const (
-	loglevel = "TAPE_LOGLEVEL"
+	loglevel    = "TAPE_LOGLEVEL"
+	logfilename = "Tape.log"
 )
 
 var (
 	app = kingpin.New("tape", "A performance test tool for Hyperledger Fabric")
 
-	run     = app.Command("run", "Start the tape program").Default()
-	con     = run.Flag("config", "Path to config file").Required().Short('c').String()
-	num     = run.Flag("number", "Number of tx for shot").Required().Short('n').Int()
-	rate    = run.Flag("rate", "[Optional] Creates tx rate, default 0 as unlimited").Default("0").Float64()
-	burst   = run.Flag("burst", "[Optional] Burst size for Tape, should bigger than rate").Default("1000").Int()
+	con            = app.Flag("config", "Path to config file").Short('c').String()
+	num            = app.Flag("number", "Number of tx for shot").Short('n').Int()
+	rate           = app.Flag("rate", "[Optional] Creates tx rate, default 0 as unlimited").Default("0").Float64()
+	burst          = app.Flag("burst", "[Optional] Burst size for Tape, should bigger than rate").Default("1000").Int()
+	signerNumber   = app.Flag("signers", "[Optional] signer parallel Number for Tape, default as 5").Default("5").Int()
+	parallelNumber = app.Flag("parallel", "[Optional] parallel Number for Tape, default as 1").Default("1").Int()
+
+	run = app.Command("run", "Start the tape program").Default()
+
 	version = app.Command("version", "Show version information")
+
+	commitOnly = app.Command("commitOnly", "Start tape with commitOnly mode, starts dummy envelop for test orderer only")
+
+	endorsementOnly = app.Command("endorsementOnly", "Start tape with endorsementOnly mode, starts endorsement and end")
+
+	trafficOnly = app.Command("traffic", "Start tape with traffic mode")
+
+	observerOnly = app.Command("observer", "Start tape with observer mode")
 )
 
 func main() {
 	var err error
 
 	logger := log.New()
 	logger.SetLevel(log.WarnLevel)
+	file, err := os.OpenFile(logfilename, os.O_CREATE|os.O_WRONLY, 0755)
+	if err != nil {
+		panic(err)
+	}
+	defer file.Close()
+	logger.SetOutput(file)
 	if customerLevel, customerSet := os.LookupEnv(loglevel); customerSet {
 		if lvl, err := log.ParseLevel(customerLevel); err == nil {
 			logger.SetLevel(lvl)
@@ -40,23 +60,39 @@ func main() {
 	fullCmd := kingpin.MustParse(app.Parse(os.Args[1:]))
 	switch fullCmd {
 	case version.FullCommand():
-		fmt.Printf(infra.GetVersionInfo())
+		fmt.Printf(cmdImpl.GetVersionInfo())
+	case commitOnly.FullCommand():
+		checkArgs(rate, burst, signerNumber, parallelNumber, *con, logger)
+		err = cmdImpl.Process(*con, *num, *burst, *signerNumber, *parallelNumber, *rate, logger, infra.COMMIT)
+	case endorsementOnly.FullCommand():
+		checkArgs(rate, burst, signerNumber, parallelNumber, *con, logger)
+		err = cmdImpl.Process(*con, *num, *burst, *signerNumber, *parallelNumber, *rate, logger, infra.ENDORSEMENT)
 	case run.FullCommand():
-		checkArgs(rate, burst, logger)
-		err = infra.Process(*con, *num, *burst, *rate, logger)
+		checkArgs(rate, burst, signerNumber, parallelNumber, *con, logger)
+		err = cmdImpl.Process(*con, *num, *burst, *signerNumber, *parallelNumber, *rate, logger, infra.FULLPROCESS)
+	case trafficOnly.FullCommand():
+		checkArgs(rate, burst, signerNumber, parallelNumber, *con, logger)
+		err = cmdImpl.Process(*con, *num, *burst, *signerNumber, *parallelNumber, *rate, logger, infra.TRAFFIC)
+	case observerOnly.FullCommand():
+		checkArgs(rate, burst, signerNumber, parallelNumber, *con, logger)
+		err = cmdImpl.Process(*con, *num, *burst, *signerNumber, *parallelNumber, *rate, logger, infra.OBSERVER)
 	default:
 		err = errors.Errorf("invalid command: %s", fullCmd)
 	}
 
 	if err != nil {
 		logger.Error(err)
-		logger.Error("Please go to https://github.com/Hyperledger-TWGC/tape/wiki/FAQ find FAQ")
+		fmt.Fprint(os.Stderr, err)
 		os.Exit(1)
 	}
 	os.Exit(0)
 }
 
-func checkArgs(rate *float64, burst *int, logger *log.Logger) {
+func checkArgs(rate *float64, burst, signerNumber, parallel *int, con string, logger *log.Logger) {
+	if len(con) == 0 {
+		os.Stderr.WriteString("tape: error: required flag --config not provided, try --help")
+		os.Exit(1)
+	}
 	if *rate < 0 {
 		os.Stderr.WriteString("tape: error: rate must be zero (unlimited) or positive number\n")
 		os.Exit(1)
@@ -65,6 +101,14 @@ func checkArgs(rate *float64, burst *int, logger *log.Logger) {
 		os.Stderr.WriteString("tape: error: burst at least 1\n")
 		os.Exit(1)
 	}
+	if *signerNumber < 1 {
+		os.Stderr.WriteString("tape: error: signerNumber at least 1\n")
+		os.Exit(1)
+	}
+	if *parallel < 1 {
+		os.Stderr.WriteString("tape: error: parallel at least 1\n")
+		os.Exit(1)
+	}
 
 	if int64(*rate) > int64(*burst) {
 		fmt.Printf("As rate %d is bigger than burst %d, real rate is burst\n", int64(*rate), int64(*burst))

diff --git a/config.yaml b/config.yaml
@@ -1,4 +1,6 @@
 # Definition of nodes
+# addr address for node
+# tls_ca_cert tls cert
 peer1: &peer1
   addr: localhost:7051
   tls_ca_cert: ./organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/msp/tlscacerts/tlsca.org1.example.com-cert.pem
@@ -11,26 +13,42 @@ orderer1: &orderer1
   addr: localhost:7050
   tls_ca_cert: ./organizations/ordererOrganizations/example.com/msp/tlscacerts/tlsca.example.com-cert.pem
 
-# Nodes to interact with
+# Peer Nodes to interact with as endorsement Peers
 endorsers:
   - *peer1
   - *peer2
-# we might support multi-committer in the future for more complex test scenario,
-# i.e. consider tx committed only if it's done on >50% of nodes. But for now,
-# it seems sufficient to support single committer.
+
+# Peer Nodes to interact with as Commit Peers as listening
 committers: 
   - *peer1
   - *peer2
+# we might support multi-committer in the future for more complex test scenario.
+# i.e. consider tx committed only if it's done on >50% of nodes. 
+# Give your commit Threshold as numbers for peers here.
 commitThreshold: 2
+
+# orderer Nodes to interact with
 orderer: *orderer1
 
 # Invocation configs
 channel: mychannel
 chaincode: basic
+# chain code args below, in a list of str
+# we provides 3 kinds of randmon
+# uuid
+# randomString$length
+# randomNumber$min_$max
 args:
-  - GetAllAssets
+  - CreateAsset
+  - uuid
+  - randomString8
+  - randomNumber0_50
+  - randomString8
+  - randomNumber0_50
+# Tx submiter information
 mspid: Org1MSP
 private_key: ./organizations/peerOrganizations/org1.example.com/users/User1@org1.example.com/msp/keystore/priv_sk
 sign_cert: ./organizations/peerOrganizations/org1.example.com/users/User1@org1.example.com/msp/signcerts/User1@org1.example.com-cert.pem
+# network traffic control
 num_of_conn: 10
 client_per_conn: 10