Private transactions on Besu+Tessera

[mfolnovic]

I'm running Besu, Tessera and Firefly.

When I run:

curl -X POST http://firefly-ethconnect.local/ \
-H 'Content-type: application/x-yaml' \
--data-binary @- << EOF
headers:
  type: SendTransaction
from: 0xf2B91C1B59dcb57846A5965d2102dCf5e73864A1
to: 0x1292640599689A9f0DD7eCfD9DFAa30333625C9c
gas: 1000000
privateFrom: 'wMshvELKv1dDK58dfs4GEv/eqfJ9GTL/7XQmE40KzSU='
privateFor: ['ccwFw3t7LYW7jZunO2XuROiDxORHE/Uau1M9ULIA6jI=']
params:
  - value: 123
    type: uint256
  - value: test
    type: string
methodName: create

I get an error in logs:

[2022-05-26T10:07:32.168Z]  INFO --> POST /
[2022-05-26T10:07:32.168Z]  INFO Received message 'Content-Type: application/x-yaml' Length: 356
[2022-05-26T10:07:32.168Z]  INFO Webhook accepted message. MsgID: a6774c5e-bf11-494c-5fe0-9581fa22e0d3 Type: SendTransaction
[2022-05-26T10:07:32.168Z]  INFO In-flight 1000003 added. nonce=0 addr=0xf2b91c1b59dcb57846a5965d2102dcf5e73864a1 before=0 (node=false)
[2022-05-26T10:07:32.700Z]  WARN TX: Failed to send: 500 Internal Server Error: {"jsonrpc":"2.0","id":754141,"error":{"code":-32604,"message":"Method not enabled"}} [0.53s]
[2022-05-26T10:07:32.700Z]  INFO In-flight 1000003 complete. nonce=0 addr=0xf2b91c1b59dcb57846a5965d2102dcf5e73864a1 nan=true sub=false before=1 after=0 highest=-1
[2022-05-26T10:07:32.700Z]  WARN Failed to process message MsgContext[SendTransaction/a6774c5e-bf11-494c-5fe0-9581fa22e0d3]: 500 Internal Server Error: {"jsonrpc":"2.0","id":754141,"error":{"code":-32604,"message":"Method not enabled"}}
[2022-05-26T10:07:32.700Z]  INFO Received reply message. requestId='a6774c5e-bf11-494c-5fe0-9581fa22e0d3' reqOffset='' type='Error': 500 Internal Server Error: {"jsonrpc":"2.0","id":754141,"error":{"code":-32604,"message":"Method not enabled"}}
[2022-05-26T10:07:32.700Z]  INFO a6774c5e-bf11-494c-5fe0-9581fa22e0d3: Inserted receipt into receipt store
[2022-05-26T10:07:32.700Z]  INFO <-- POST / [200]: Webhook RequestID=a6774c5e-bf11-494c-5fe0-9581fa22e0d3

From what I understand, private transactions should be done with eea_sendTransaction / eea_sendRawTransaction:

https://besu.hyperledger.org/en/stable/Reference/API-Methods/#eea_sendrawtransaction
https://docs.ethsigner.consensys.net/en/latest/Using-EthSigner/Using-EthSigner/#eea_sendtransaction

But through reading code, it seems that if I only send privateFor/privateTo without privacyGroupId, Firefly will send eth_sendTransaction instead: https://github.com/hyperledger/firefly-ethconnect/blob/main/internal/eth/send.go#L238. There's missing jsonRPCMethod = "eea_sendTransaction".

[peterbroadhurst]

Hi @mfolnovic,

A slightly incomplete, and potted history, of the off-chain EVM based private transactions might help here:

1. Quorum implemented it with Constellation as extra fields in eth_sendTransaction

• The group was inferred by the list of private participants
• There was significant adoption, and EthConnect evolved its initial support in this era

2. Quorum began replacing the Haskel Constellation codebase with Java Tessera
   • The API was unchanged
   • All production environments of Quorum migrated over time from Quorum+Constellation to Quorum+Tessera
   • To my personal knowledge, this is where the majority of production adoption of these off-chain transaction resides
   • EthConnect continues to work very well with this proven+hardened configuration
3. The team behind Pantheon, which became Hyperledger Besu (ConsenSys PegaSys) began development of a separate Orion codebase

• A set of eea_ prefix APIs were proposed
• These separate APIs encouraged new lifecycle to create a Privacy Group prior to submission of the transaction
• EthConnect was updated to support this model with the privacyGroupId - the support in EthConnect of the original Quorum private transaction eth_sendTransaction APIs remained unchanged, due to production usage
• These eea_ APIs were adopted at this point by only within Pantheon/Besu, not within Quorum

5. ConsenSys PegaSys donated the Pantheon codebase to become Hyperledger Besu
   • There was no change to the Orion codebase ownership
   • Hyperledger Besu only integrated with Orion, not Tessera
6. Custodianship of Quorum and Tessera transitioned to the same core team as had developed Orion (JPMC -> ConsenSys)
7. Orion was deprecated
8. Support for using Tessera with Hyperledger Besu was developed
   • Provides a path forwards for Besu+Orion users
   • Some reconciliation of the APIs was performed, but still (to my knowledge)
     • Besu: Only supports eea_* send transaction JSON/RPCs
     • Quorum: Only supports eth_* send transaction JSON/RPCs

So EthConnect as it stands supports:

1. Besu+Tessera: Using an explicitly created privacyGroupId:

• See https://docs.tessera.consensys.net/en/stable/Concepts/Privacy-Groups/

2. Quorum+Tessera (the most commonly adopted combo): Using the original API:

• See https://consensys.net/docs/goquorum/en/latest/reference/api-methods/#eth_sendtransaction

Very happy to see a contribution if you would like to make one @mfolnovic to reconcile this support in EthConnect further, if you prefer the Besu+Tessera option over the Quorum+Tessera option, but do not want to use the privacy group API.

However, it is important that none of the existing working combinations are broken by any changes.

[peterbroadhurst]

fyi for anyone looking to submit Tessera style EVM off-chain transactions via FireFly APIs, you will need hyperledger/firefly#844 to be able to specify the privateFrom+privateFor or privacyGroupId on your blockchain transactions.

Not currently back-ported to v1.0 (as of Jun 1st 2022)

[mfolnovic]

Thank you from the bottom of my heart for really detailed explanation!

I had a feeling it's something about previous solutions, though I also wasn't sure how big and important https://entethalliance.org/ is.

However, it is important that none of the existing working combinations are broken by any changes.

I'm interested in creating a PR, but this was also an important feedback before I even tried.

I was also aware that Firefly itself didn't support this, didn't know of hyperledger/firefly#844 so it's nice to hear that's already been worked on! This further motives me to work on this issue and plan a place for Firefly in my architecture. :)

Do you think it's a good direction to add in config something like:

rest:
  rest-gateway:
    privacy-type: tessera

By default, it would be orion - so it doesn't brake any existing deployment.

[peterbroadhurst]

I think privacyType: besu # or quorum would be better. It's the Tessera side-car with either Enterprise Ethereum runtime, it's actually the JSON/RPC interface of the blockchain node itself that differs.

The internal/eth package doesn't directly read configuration - so you'll need to update

firefly-ethconnect/internal/eth/send.go

Line 217 in a2b26d6

func (tx *Txn) submitTXtoNode(ctx context.Context, rpc RPCClient, txArgs *SendTXArgs) (string, error) {

to either:

1. Pass in the mode switch
2. Have a config option on the Txn object that is set

Then the most appropriate config block to update is here:

firefly-ethconnect/internal/tx/txnprocessor.go

Lines 79 to 89 in a2b26d6

	type TxnProcessorConf struct {
	eth.EthCommonConf
	AlwaysManageNonce bool `json:"alwaysManageNonce"`
	AttemptGapFill bool `json:"attemptGapFill"`
	MaxTXWaitTime int `json:"maxTXWaitTime"`
	SendConcurrency int `json:"sendConcurrency"`
	OrionPrivateAPIS bool `json:"orionPrivateAPIs"`
	HexValuesInReceipt bool `json:"hexValuesInReceipt"`
	AddressBookConf AddressBookConf `json:"addressBook"`
	HDWalletConf HDWalletConf `json:"hdWallet"`
	}

[mfolnovic]

awesome, tnx for feedback and pointing me to right direction! 😊

[mfolnovic]

Okay this is interesting 🙈

Looking at txnprocessor.go, I saw OrionPrivateAPIS and saw this logic:

firefly-ethconnect/internal/tx/txnprocessor.go

Lines 232 to 243 in a2b26d6

	if p.conf.OrionPrivateAPIS {
	if msg.PrivacyGroupID != "" && len(msg.PrivateFor) > 0 {
	err = errors.Errorf(errors.TransactionSendPrivateForAndPrivacyGroup)
	return nil, err
	} else if msg.PrivacyGroupID != "" {
	inflight.privacyGroupID = msg.PrivacyGroupID
	} else if len(msg.PrivateFor) > 0 {
	if inflight.privacyGroupID, err = eth.GetOrionPrivacyGroup(txnContext.Context(), p.rpc, &from, msg.PrivateFrom, msg.PrivateFor); err != nil {
	return nil, err
	}
	}
	}

Which looked similar to what I was doing client-side. So I've tried setting OrionPrivateAPIS to true and well... it seems it works.