npm run bitcoin cannot start for testnet on versions above v0.20.1

[coffnix]

After testing v22.x and v23.x versions of bitcoind I got errors loading the wallet using npm run bitcoin command.
we need bitcoin-core v23.x version support.

ion only works using v0.20.1

[LiranCohen]

Newer versions of bitcoin core create a descriptor wallet by default, i've started working on this with this PR:
decentralized-identity/sidetree#1192

Works good, but core still needs to be compiled with Berkeley DB, so I need to do some testing to make sure the pre-built binaries are compiled with that included. In my testing setup currently I compile core from source.

I will also test against older versions of core pre-descriptors to see if core complains about the additional RPC parameter so that it can be properly documented in the ION Installation Guide.

[coffnix]

@LiranCohen good work! Bitcoin core v0.20.1 no longer compiles on newer Linux distributions using newer boost libraries

[LiranCohen]

@coffnix that's good to know too.

I've opened an Issue to investigate using a descriptor wallet in sidetree/ion libraries:
#304

Will spend some time over the next couple of weeks to see what it would take to accomplish that and if there are any good Node libraries that would support descriptors or if we'd need to write one specifically for ion to use.

Would be good to get everything up to date for the latest versions of core.

[thehenrytsai]

@LiranCohen and @coffnix, thanks for your investigation and contribution here. I'd like to understand better regarding the issue:

1. Is this issue only isolated to testnet?
2. Do you know if this is OS dependent?
3. Does this apply only to completely fresh setups (not reusing existing Bitcoin Core DB)?

I ask because I spun up an ION node against Bitcoin Core 22 last week and it is running fine. The setup is:

1. mainnet
2. Windows
3. I copied the entire bitcoin data folder from another Windows machine with v0.20.1 I believe (mainly to save me from resync everything again!!)

[LiranCohen]

@thehenrytsai I did not try v22 on either mainnet nor testnet with the current release of ION, but I'm pretty sure it should work.

Maybe @coffnix' issue with v22 was a fluke due to some other scenario caused by previously trying v23?

I know that the 'createwallet' RPC for v23 creates a descriptor wallet by default, and the 'loadwallet' RPC does not complete in time (300ms timeout) which causes the ion bitcoin service it to fail and restart.

When I raised the timeout it created and loaded a new descriptor wallet(the new default) but the 'importpubkey' is not supported by descriptor wallets so the service failed.

At some point we would probably want to consider supporting descriptor wallets and using the 'importdescriptors' RPC i believe. We have until v26 before the legacy wallets will be deprecated completely.

I'm thinking that if v23 was run first as a fresh installation it probably created a descriptor wallet (default), then when downgrading the descriptor wallet was already created as such and just failed with the 'importpubkey' command? That' just speculation though.

This shouldn't be OS specific, rather just the gradual deprecation of the BerkleyDB wallets that core has been using up until v 0.21 when they introduced descriptors as an optional parameter, now default in v23.

The change in: decentralized-identity/sidetree#1192 would fix this, but would not be compatible with anything before v 0.21, so the installation instructions as well as the bitcoin core install script should be updated to reflect this.

I'll spend some time this week running an updated installation script with the newer binary on a Linux x86_64, to make sure that script will work as expected and add that to the PR as well as open a PR for the ION Install Guide to reflect the minimal requirements.

[coffnix]

@LiranCohen try compile your own bitcoind binary:

su - bitcoin

git clone https://github.com/bitcoin/bitcoin.git

cd bitcoin

git checkout v23.0

./autogen.sh

./contrib/install_db4.sh `pwd`

export BDB_PREFIX='/home/bitcoin/bitcoin/db4'

./configure BDB_LIBS="-L${BDB_PREFIX}/lib -ldb_cxx-4.8" BDB_CFLAGS="-I${BDB_PREFIX}/include"

make -j $(nproc)

[LiranCohen]

@coffnix Yeah I normally do this and this was actually the first way I tested the change. I also tested against the pre-built binary for aarch614 to make sure that BDB was being included in those.

I just want to update the script:
https://github.com/decentralized-identity/sidetree/blob/master/lib/bitcoin/setup.sh

Which is included in the install guide to reflect the new version, that way new users will have a compliant version since that script install v 0.18 and that will not support the descriptors parameter in the RPC request.

I'll test that on a spare Linux x86_64 box I have. I normally run ION on my raspberry Pi 4s.