Skip to content

Adding Per‐Wallet Application to RP2

Jump to bottom
eprbell edited this page Dec 5, 2024 · 42 revisions

Introduction

This design document focuses on how to add per-wallet application support to the RP2 tax engine, which currently only supports universal application.

For a definition of per-wallet vs universal application check this article. The issue is discussed in the context of RP2 at #135.

Requirements

The following features are needed:

  • the tax engine should support both universal and per-wallet application;
  • different countries can support one or both application types. The country plugin should express this;
  • some countries change the type of application they allow over time. E.g. the US supported both universal and per-wallet application up to 2024, but only per-wallet application after that. The country plugin should express this as well;
  • the user should be allowed to change the application type and/or the accounting method year over year, in any combination that is supported by the country plugin. Note that RP2 already allows users to change accounting method year over year.

High-Level Design

At high level adding per-wallet semantics changes the compute_tax function as follows:

  • the function receives in input an object of type InputData, which is the result of parsing the ODS file. This object will be used for universal application (similar to what happens in the current version of RP2). Let's call it universal InputData;
  • the universal InputData is processed by a transfer analysis algorithm, to understand which wallet each lot ends up at after transfers. This results in as many new instances of InputData as there are wallets. The new set of InputData objects will be used for per-wallet application. Let's call it per-wallet InputData set;
  • the tax engine can now run using either universal application (via the universal InputData) or per-wallet application (via the per-wallet InputData set). Note that computing per-wallet taxes means running the tax computation algorithm once per element of the per-wallet InputData set;
  • at the end of the computation, if universal application was selected for the last year of taxes, return the universal ComputedData (similar to what happens in the current version of RP2). If per-wallet application was selected for the last year of taxes, there are as many ComputedData objects as elements of the per-wallet InputData set: normalize/unify the per-wallet ComputedData objects and return the result.

This approach strives to minimize changes to the tax engine and report generators (which have been thoroughly tested): it just adds an extra layer between input parser and tax engine (transfer analysis), and then another layer between tax engine and report generators (for result unification).

Note that:

  • transfer analysis isn't simply about tracking lots and where they go: transferring funds can split a lot into fractions. E.g. if one buys 1 BTC on Coinbase and then sends 0.5 BTC to a hardware wallet, there was one lot before the transfer, and, after transferring, the lots became two (0.5 on Coinbase and 0.5 on the hardware wallet).
  • the logic to select which lot to transfer is very similar to the logic of existing accounting methods (FIFO, LIFO, etc.) to pair taxable events to acquired lots: existing accounting methods can be reused as-is in the context of transferred lot selection and continue to be used as accounting methods in the context of acquired lot selection. Which selection algorithm to use in transfers has been discussed here. Some possible options:
    • always use FIFO;
    • Same as the accounting method;
    • Let user select a method that may be different than the accounting method.

Detailed Design

Transfer Analysis and Per-wallet InputData Creation

The _transfer_analysis function (see pseudocode):

  • Receives an InputData object containing in, out and intra-transactions, and reflecting what the user entered in the ODS file. This object is used by universal application and we'll refer to it as universal InputData;
  • Outputs a dictionary of 1 or more InputData objects (one per wallet). Each of these objects contains the in, out and intra transactions that originate from the given wallet. These InputData objects are used by per-wallet application and we'll refer to them as the per-wallet InputData set;

This function creates artificial InTransactions to capture the receive side of intra transactions in per-wallet application. These artificial transactions are added to their respective per-wallet InputData (they are not used in universal application).

Two new fields need to be added to InTransaction to support per-wallet application:

  • __parent is populated only in the artificial InTransactions and set to the InTransaction that represents the origin of the funds (i.e. the send side of the current IntraTransaction being processed).
  • __per_wallet is a dictionary of wallets and artificial InTransactions representing how and when the funds in the transaction have been sent a given wallet (after per-wallet analysis).

The original InTransactions coming from ODS parsing always has __parent set to None. Artificial InTransactions always have __parent set to not-None.

For example, let's consider this data in the user ODS input file:

  • 1/1: InTransaction of 10 BTC on Coinbase
  • 2/1: IntraTransaction of 4 BTC from Coinbase to Kraken
  • 3/1: IntraTransaction of 2 BTC from Kraken to Trezor
  • 4/1: IntraTransaction of 5 BTC from Coinbase to Trezor
  • 5/1: OutTransaction of 6 BTC from Trezor

Results of transfer analysis are:

  • Coinbase:
    • InTransaction(10, Coinbase), with __parent set to None and __per_wallet set to: {Kraken: [InTransaction(4, Kraken)], Trezor: [InTransaction(2, Trezor), InTransaction(5, Trezor)]}
    • IntraTransaction(4, Coinbase -> Kraken)
    • IntraTransaction(5, Coinbase -> Trezor)
  • Kraken:
    • artificial InTransaction(4, Kraken), with __parent set to InTransaction(10, Coinbase), __per_wallet set to: "Trezor" -> [InTransaction(2, Trezor)]
    • IntraTransaction(2, Kraken -> Trezor)
  • Trezor:
    • artificial InTransaction(2, Trezor), with __parent set to InTransaction(4, Kraken) and __per_wallet set to None
    • artificial InTransaction(5, Trezor), with __parent set to InTransaction(10, Coinbase) and __per_wallet set to: None
    • OutTransaction(6, Trezor)

Note that the artificial InTransactions represent a fraction of funds that are already modeled by their __parent transaction: e.g. in the example above, the Coinbase wallet has InTransaction(10, Coinbase) and the Kraken wallet has InTransaction(4, Kraken). The Kraken funds represent a fraction of the Coinbase ones: the two transactions overlap.

Enabling Switching from Universal to Per-wallet and Viceversa

After getting per-wallet InputData the tax engine can be invoked. The user can move back and forth from universal to per-wallet application over the years (similar to what they can already do with accounting methods). The current version of the accounting engine handles accounting method changes year over year as follows:

  • create one LotCandidates object per year and add it to the AccountingEngine.__years_2_lot_candidates AVL tree;
  • each LotCandidates constructor receives two parameters:
    • acquired_lot_list: full list of all acquired lots as a list (in_transactions from universal InputData), so each LotCandidates object can see all acquired lots, regardless of which exchange they sit on (this is in line with the definition of universal application, which has only one global queue);
    • acquired_lot_2_partial_amount: the same dictionary instance is passed to all of the LotCandidates via this parameter: this way when accounting methods change from one year to the next, acquired lots that have been already used (or partially used) are not double-counted, regardless of what the accounting methods were used in previous years.

Implementing the ability to switch from one application type to another year over year requires the following changes:

  • the parameters passed to LotCandidates constructor change as follows:
    • acquired_lot_list: if the current year uses universal application, pass in_transactions from universal InputData (same as discussed above). If it uses per-wallet application, pass the in_transactions from the per-wallet InputData for the wallet that taxes are being generated for;
    • acquired_lot_2_partial_amount: similar to above, pass the same object to all LotCandidates, regardless of accounting method and universal or per-wallet application: this way when application type or accounting methods change from one year to the next, acquired lots that have been already used (or partially used) are not double-counted.

This enables both universal and per-wallet application in the tax engine, including the ability to switch between them over the years.

Tax Computation, Setting Partial Amounts and Two-Model Syncing

During the actual tax computation, the new InTransaction fields are used to keep partial amounts in sync between universal and per-wallet InputData, regardless of which application type is selected in any given year.

Note that:

  • application type can change year over year (same as accounting method);
  • since the funds in the artificial InTransactions overlap with those in their parents, computing universal application taxes requires updating partial amounts in both the universal and per-wallet models, regardless of which application is being used in the current year: keeping both models updated is important when the application type changes year over year. As an optimization, if the application type doesn't change updating both models isn't necessary, but this design considers the more general case in which it does change.

Universal Application

If the current year uses universal application, when setting a partial amount on a InTransaction from the universal InputData object, the accounting engine must also set partial amounts for the __per_wallet artificial transactions associated to the same exchange where the current taxable event occurred (_per_wallet artificial transactions are selected using FIFO semantics). See pseudocode.

Then for each of the __per_wallet transactions for which the partial amount was set it also follows the parent link all the way to the original InTransaction and sets the partial amount in all the intermediate InTransactions.

In the example above, when processing the OutTransaction the accounting engine sets partial amount to 6 for InTransaction(10, Coinbase) (since there is only one InTransaction in this example it doesn't matter which accounting method is used). Then it increases the partial amount by 2 for InTransaction(2, Trezor) and by 4 for InTransaction(5, Trezor). Then it sets the partial amounts following the __parent links for each of the __per_wallet transactions it touched. For the first one, the parent is InTransaction(4, Kraken), so increase its partial amount by 2, and its parent is the original InTransaction in which the partial amount was already set. For the second one, again, the parent is the original InTransaction which was already set.

Per-wallet Application

If the current year uses per-wallet application, when setting a partial amount on a InTransaction from a per-wallet InputData object, the accounting engine also follows the __parent link all the way to the original InTransaction and sets the partial amounts in all the intermediate InTransactions. See pseudocode.

In the example above, when processing the OutTransaction (and assuming FIFO is the accounting method) the accounting engine sets partial amount to 2 for InTransaction(2, Trezor) and to 4 for InTransaction(5, Trezor) in the Trezor per-wallet InputData. Then it sets the partial amounts following the __parent links for each of the two __per_wallet InTransactions. For the first one, the parent is InTransaction(4, Kraken), so increase its partial amount by 2, and its parent is the original InTransaction, so increase its partial amount by 2. For the second one, the parent is the original InTransaction, so increase its partial amount by 4.

Unifying the output of the tax engine

The tax engine outputs a TransactionSet and a GainLossSet for each wallet. These objects are easy to join before they are passed to generators.

Changes to Config File

The config file needs two new sections (each of which is similar to the existing accounting_methods one):

  • application_methods: year -> universal/per-wallet. This dictionary models changes in the application method over the years. It is translated to an AVL tree in the code (similar to how accounting_methods is processed).
  • transfer_methods: year -> accounting method. This dictionary models changes in the accounting method that is used specifically for transfers in per-wallet accounting: i.e. when funds get transferred from wallet A to wallet B, which InTransaction do they come from? It is translated to an AVL tree in the code (similar to how accounting_methods is processed).

Changes to Country Plugins

Add a method returning a structured value describing which application types are valid in which year.

Pseudocode

Transfer Analysis

class InTransaction(AbstractTransaction):
  __parent: InTransaction
  __per_wallet: Dict[str,List[InTransaction]]
  ...

# Private class used to build a per-wallet set of transactions.
class _PerWalletTransactions:
  def __init__(asset: str, lot_selection_method: AbstractAccountingMethod):
    self.__asset = asset
    self.__lot_selection_method = lot_selection_method # to decide which lot to pick when transferring funds.
    self.__in_transactions: AbstractAcquiredLotCandidates = lot_selection_method.create_lot_candidates([], {})
    self.__out_transactions: List[OutTransactions] = []
    self.__intra_transactions: List[IntraTransactions] = []

# Utility function to create an artificial InTransaction modeling the "to" side of an IntraTransaction
def _create_to_in_transaction(from_in_transaction: InTransaction, transfer_transaction: IntraTransaction) -> Intransaction:
  artificial_id = get_artificial_id(configuration),
  result = InTransaction(
    timestamp=transfer_transaction.timestamp,
    exchange=transfer_transaction.to_exchange,
    asset=transfer_transaction.asset,
    holder=transfer_transaction.to_holder,
    transaction_type=from_in_transaction.transaction_type,
    crypto_in=transfer_transaction.crypto_received,
    spot_price=from_in_transaction.spot_price,
    crypto_fee=0,
    # same thing for fiat fields...
    row=artificial_id
    unique_id=f"{transfer_transaction.unique_id}_{artificial_id}",
    notes="Artificial transaction modeling the reception of <amount> <asset> from <from_exchange> to <to_exchange>",
  )
  result.set_parent(from_in_transaction)
  current_transaction = from_in_transaction
  while True:
    current_transaction = current_transaction.parent
    if current_transaction== None:
      break
    in_transactions = current_transaction.per_wallet.set_default(transfer_transaction.to_exchange, [])
    in_transactions.append(result)
 
def _transfer_analysis(input_data: InputData, transferred_lot_selection_method: AbstractAccountingMethod) -> Dict[str, InputData]:
  # Transfer analysis
  all_transactions: List[AbstractTransaction] = input_data.input_transactions + input_data.out_transactions + input_data.intra_transaction
  all_transactions = sorted(all_transactions, key=lambda t: t.timestamp) # sort chronologically
  wallet_2_per_wallet_transactions: Dict[str, _PerWalletTransactions] = {}
  for transaction in all_transactions:
    if isinstance(transaction, InTransaction):
      per_wallet_transactions = wallet_2_per_wallet_transactions.set_default(transaction.exchange, _PerWalletTransactions(input_data.asset, transferred_lot_selection_method))
      per_wallet_transactions.in_transactions.add_acquired_lot(transaction)
      per_wallet_transactions.in_transactions.set_to_index(len(per_wallet_transactions.in_transactions.acquired_lot_list)) 
    elif isinstance(transaction, OutTransaction):
      per_wallet_transactions = wallet_2_per_wallet_transactions[transaction.exchange] # The wallet transactions object must have been already created when processing a previous InTransaction.
      per_wallet_transactions.out_transactions.append(transaction)
    elif isinstance(transaction, IntraTransaction):
      # IntraTransactions are added to from_per_wallet_transactions.
      from_per_wallet_transactions = wallet_2_per_wallet_transactions[transaction.from_exchange] # The wallet transactions object must have been already created when processing a previous InTransaction.
      from_per_wallet_transactions.intra_transactions.append(transaction)
      # Add one or more artificial InTransaction to to_per_wallet_transactions.
      to_per_wallet_transactions = wallet_2_per_wallet_transactions.set_default(transaction.to_exchange, _PerWalletTransactions(input_data.asset, transferred_lot_selection_method))
      amount = transaction.crypto_sent
      while True:
        result = transferred_lot_selection_method.seek_non_exhausted_acquired_lot(from_per_wallet_transactions.in_transactions, transaction.crypto_sent)
        if result is None:
          raise RP2Error(f"Insufficient balance on {transaction.from_exchange} to send funds: {transaction}")
        if result.amount >= amount:
          to_in_transaction = _create_to_in_transaction(result.acquired_lot, transaction)
          to_per_wallet_transactions.in_transactions.add_acquired_lot(to_in_transaction)
          from_per_wallet_transactions.in_transactions.set_partial_amount(result.acquired_lot, result.amount - amount)
          break
        to_in_transaction = _create_to_in_transaction(result.acquired_lot, transaction)
        to_per_wallet_transactions.in_transactions.add_acquired_lot(to_in_transaction)
        from_per_wallet_transactions.in_transactions.clear_partial_amount(result.acquired_lot)
        amount -= result.amount
    else:
      raise RP2ValueError(f"Internal error: invalid transaction class: {transaction}")

  # Convert per-wallet transactions to input_data and call the tax engine.
  result: Dict[str, InputData] = {}
  for wallet, per_wallet_transactions in wallet_2_per_wallet_transactions:
    per_wallet_input_data = _convert_per_wallet_transactions_to_input_data(wallet, per_wallet_transactions)
    result[wallet][per_wallet_input_data]
  return result

Updating Partial Amounts Across the Two Models

def _update_parent_partial_amount(lot_candidates: AbstractAcquiredLotCandidates, in_transaction: InTransaction, amount RP2Decimal) -> None:
  current_parent = in_transaction.parent
  while True:
    if current_parent is None:
      break
    lot_candidates.increment_partial_amount(current_parent, amount)

def _update_partial_amounts_across_applications(lot_candidates: AbstractAcquiredLotCandidates, in_transactions: List[InTransaction], amount: RP2Decimal) -> None:
  total_amount = amount
  for in_transaction in in_transactions:
    transaction_amount = in_transaction.crypto_in
    if lot_candidates.has_partial_amount(in_transaction):
      transaction_amount = lot_candidates.get_partial_amount(in_transaction)
    # This could be sped up by keeping track of the lots that have been already used (so we don't restart everytime from the first lot).
    if transaction_amount = ZERO: 
      continue
    if total_amount <= transaction_amount:
      lot_candidates.increment_partial_amount(in_transaction, total_amount)
      _update_parent_partial_amount(lot_candidates, in_transaction, total_amount)
      break
    lot_candidates.increment_partial_amount(in_transaction, transaction_amount)
    _update_parent_partial_amount(lot_candidates, in_transaction, transaction_amount)
    total_amount -= transaction_amount:
Clone this wiki locally