v0.32.0: Registry whitelisting & staking nodes registration, improved nomination & enrollment

Major changes since v0.31.1:

• The registry will now longer reject requests for AAAA records;
• The optional node.commons_budget_address config has been removed,
  as it was overwritten by the selected chain;
• All configuration errors now include the field and YAML file position;
• Agora will now be more responsive when trying to connect to unresponsive peers;
• Fixed a bug where whitelisting was not working for caught-up blocks;
• Received Enrollments are now provided and signed with an explicit height,
  solving issues that would arise when an Enrollment wasn't included in the next block;
• Config fields validator.addresses_to_register and flash.addresses_to_register
  now needs to be fully specified (including the scheme, and port, if applicable);
• The registry will now refuse any AXFR request that doesn't come from a whitelisted IP,
  see config option registry.$ZONE.allow_transfer which is an array of IPs.
• Validators will now limit the size of transaction sets they propose for externalization;
• Nodes will now accept any fee level when their transaction pool is sparsely populated;
• RPC connections will now reject banned peers immediately;
• The registry will now accept registrations from staking keys, not only enrolled ones;
• Nodes will now remember what peers they previously sent pre-images to,
  and avoid flooding them with the same data again;
• Ledger.acceptBlock will now be a no-op when the block is the latest one;
• A bug where two addresses with different ports were considered the same address was fixed;

From this release, in addition to the Docker image available under bosagora/agora:$TAG,
native builds for x86_64 ubuntu-20.04 and macOS 11 will be available on the Github release page.

v0.31.1: Improve registry support for NS records

Major bug fixes since v0.31.1:

• ResourceRecords (RRs) are now printed more accurately in logs, instead of just "XX byte(s)";
• The registry now sends NS records for the zones on which it is a primary;
• The default TTL is now 600, up from 0, which is what nodes use when they register;
• Agora can be started with -c /dev/null to get a zeroconf CoinNet full node;
• Registry will now longer perform UDP truncation when contacted via TCP;
• Fixed a bug where NS records were not properly deserialized when some labels ere compressed;

Additionally, the Ledger class has been moved to its own module,
and stripped of many dependencies, to simplify using it in clients.

v0.31.0: Many DNS improvements and usability fixes

Major changes since v0.30.0:

• Flash: Fixed a crash that would happen with high closing TX fee;
• Reading Agora's state DB while Agora is running is now handled better;
• Various improvements have been made w.r.t. database consistencies in case of crash;
• The registry will no longer store signatures for addresses;
• Configuration: SOA-related fields have been moved to a 'soa' field of the zone;
• Bech32 addresses can now be provided in mixed cases and will be interpreted as lowercase;
• Fixed a bug where DNS queries were case sensitive;
• Fixed various issues with domain name validation, leading to invalid names not being properly reported;
• Handlers now use exponential backoff, so long downtime will be handled more gracefully,
  but recovery time will be slower.
• Correctly validate received block headers during catchup;
• DNS: Fixed a bug where a query with multiple references to the same name could be wrongly deserialized;

v0.30.0

What's Changed

• Actions: checkout faucet and agora recursively by @omerfirmak in #2737
• Actions: Properly update agora submodules in faucet by @omerfirmak in #2738
• Retry delay fix in attemptRequest by @hewison-chris in #2739
• Trivial: Fix quotes around date in CI config by @Geod24 in #2741
• Implement Chain ID by @omerfirmak in #2736
• Unit test: Add the missing seeds for pre-images of coinnet by @hewison-chris in #2751
• Set confirmed height earlier by @hewison-chris in #2744
• Remove signConfirmBallot function by @hewison-chris in #2745
• Use and adjust for tests the pre-image catchup interval by @hewison-chris in #2747
• Issue2653 debug log by @hewison-chris in #2743
• Make clone() available to all agora code by @hewison-chris in #2752
• Allow setting quorums while nomination flag is true by @linked0 in #2740
• Flash: Add onchain fees to settlement TX by @omerfirmak in #2733
• Refactor the wait for more than half signed last block by @hewison-chris in #2750
• Do not throw away commit envelopes when missing tx by @hewison-chris in #2748
• Extend timer interface by @mkykadir in #2730
• Fix LocalRest commit hash by @omerfirmak in #2756
• Manager: Update required peer list when updating peer stake by @omerfirmak in #2755
• Reduce the frequency of refreshing connections by @hewison-chris in #2746

v0.29.1: More TestNet validators

Adds frozen funds for 6 more validators #2732

v0.29.0: Freezing fees, better signature aggregation and better network reliability

Major improvements since v0.28.0:

Introduce Freezing fee: #2597

Freezing TXs now have to include SlashAmountPenatly amount of fee
for each Freeze output they have. This amount will be refunded to them
if they are not slashed for the duration of their validation cycle.

Add missing signatures if possible on fetched blocks and headers: #2727

The missing signatures that are known are added and gossiped to peers. This will help ensure that we have all the available signatures stored in the block header as soon as possible.

Add TransactionResult struct for result of posting transaction: #2723

A struct is now returned with a status of Accepted, 'Rejected' or Duplicated and a reason if rejected.

Fix utility retry logic to also wait when exception occurs: #2713

This prevents many successive failures without delay between retries.

Only use unidentified connections for address discovery: #2710

This helps to prevent edge cases that are difficult to handle.

Deserialize RDATA for supported types: #2711

Deserialize Resource Records fetched through AXFR query (Domain requires special deserializer context).

Registry: Only answer with records matching query type: #2715

DNS: Merge two definitions of RDATA: #2719

Manager: Try to connect to last known Validators: #2722

This will enable a validator to catchup after a restart even when it is not enrolled.

Validator: Push BlockHeader on new signatures received: #2725

This will ensure the listeners are kept up to date with latest header updates.

v0.28.0: Respect RPC timeouts, better configuration and interoperability for DNS

Major improvements since v0.27.1:

Foreign key support is now enabled on SQLite

This fixes a minor bug which prevented the name registry from removing stale addresses.

DNS zones are no longer updated when not changed

Previously, calling the name registry's registration function, either for flash or validators,
would always result in a zone update (as long as the payload was valid).
However most update were just setting the same value. The registry now detects this
and will not trigger an update if nothing has changed, reducing network loads
for downstream registry.

DNS: Now answer to SOA requests

SOA were already sent as part of AXFR query, but not when directly asked for.
Some clients will ask for an SOA record before triggering an AXFR,
hence this improves compatibility with external DNS servers.

DNS: Added ability to configure the realm zone directly

Under the DNS scheme used in Agora, there is a realm (e.g. testnet.bosagora.io),
and two sub-zones: validators and flash. While the registry had the ability
to configure those two sub-zones, it didn't reply to queries for the realm itself,
which complicated the upstream configuration, as the primary server for the realm
needed to be another server. It is now possible to use the name registry as a primary
for the realm itself.

DNS: The primary server now needs to be supplied in the zone configuration

SOA records include a field, mname, which indicates which server is the primary for a zone.
While the primary might not be public, the previous implementation of the registry
simply used ns1 appended to the zone name. This now needs to be supplied in the configuration.
Note that non-primary servers do not need to specify it, as it will be fetched.

RPC: Timeouts are now handled correctly while waiting for response

When the RPC interface was waiting for a response, it was not respecting the timeout
set in the config, and could freeze. This has now been fixed.

v0.27.1: Reduce risk of validator being slashed

Major improvements since v0.27.0

Reduce risk of Slashing Validators

By revealing pre-images on new block events.

Cache the Coinbase Transaction

To reduce processing and logging only calculate the payouts to the Validators and Commons Budget once per payout block.

Several Registry improvements

• DNS: Respond CNAME RR with registered address
• Refactor zone finding
• Refactor ZoneData
• Move supported query types into central place

Prevent DOS attack via pre-image reveal

By rejecting pre-images that are too far into the future excessive hashing can be prevented.

Flash fee calculation fixes

Correct handling of update and funding transaction fees.

v0.27.0: No time_offset in block header, getTransactions(Hash from)

Major bug fixes since v0.26.1:

The block header no longer has a time offset included

To help ensure blocks are externalized at the correct interval the time offset is not included.
block time is Genesis start time + height of the block x block interval in seconds.
This block time is the time that the Validators will start to nominate a block and once they agree
on the block contents will externalize (add it to the blockchain).

Agora API added that returns pending transactions

The new API returns:

public Transaction[] getTransactions (Hash from)

To enable paging each request returns up to 128 pending transactions that have a hash higher than the provided hash.

Prefer blocks with more Enrollments

Help ensure we enroll as many validators who are wanting to enroll as soon as possible.

Sign blocks that are fetched

If the Validator was online during nomination it still signs a block that it receives during block catchup.

DNS: Fix Header flag bitwise operations

A fix and code improvement.

Encourage more signatures on previous blocks

When a block is externalized it can be signed by less than half of the Validators.
To ensure we have over half the Validators signing the blocks in the blockchain
we only start nominating the next block if half have signed the previous block.

v0.26.1: EDNS(0) support, better error message, AXFR fixes

Major bug fixes since v0.26.0:

A node can now start with no configured peer if its registry is enabled

This allows to run a name registry without having to input a dummy peer
into the configuration file.

Default interface settings have been changed

The default interface setting, used when no interfaces section is present
in the config file, has been changed from one HTTP endpoint on port 2826
and one RPC endpoint on port 2571 to one RPC endpoint on port 2826
(the default RPC endpoint from now on) and one HTTP endpoint on 8080.

DNS: Fixed AXFR regression from v0.26.0

v0.26.0 inadvertently introduced a regression which made the node
always reject AXFR requests even for zones they know about.

DNS: AXFR requires to use a subdomain

The AXFR query for zone a.b needed to be performed on foo.a.b
(where foo is any valid string). This has been fixed and the node
will now answer AXFR properly.

Improved network-sent error messages

In the recent releases, Agora has made more use of reusable exception.
This correlated with a decrease in the quality of error messages,
as the HTTP framework didn't understand them.
This has been changed, and error message will now be properly sent,
instead of the unhelpful "And Exception has occured".

EDNS(0) support was added

The DNS server will now include an OPT record if one was sent in the query,
as well as honor the requested payload size, and set the TC flag when needed.

Do not add empty addresses to the list of banned/whitelisted nodes

This would happen when the RPC interface was in use, leading to nodes failing
to load their banlist / whitelist on restart.