Komodo Core and Bitcoin v0.20.0 refactoring

Note: The following are my thoughts as a developer of Komodo Core. This is not a guarantee the work will be done in this manner, or even be done at all. This is me “typing out loud”.

The Komodo Core code has diverged quite a bit from the Bitcoin codebase. Of course, Komodo is not a clone. There is quite a bit of functionality added to Komodo which requires the code to be different.

However, there are areas where Komodo could benefit from merging in the changes that Bitcoin has since made to their code base. Some of the biggest benefits are:

  • Modularization – Functionality within the code base is now more modular. Interfaces are better defined, and some of the ties between components have been eliminated.
  • Reduction in global state – Global state makes certain development tasks difficult. Modularization, testing, and general maintainability are increased when state is pushed down into the components that use them instead of exposed for application-wide modification.
  • Testability – When large processes are broken into smaller functions, testing individual circumstances (i.e. edge cases) becomes less cumbersome, and sometimes trivial.
  • Maintainability – With improvements in the points above, modifications to the code base are often more limited in scope and easier to test. This improves developer productivity and code base stability.

Plan of Attack

“Upgrading” Komodo to implement the changes to the Bitcoin code base sounds great. But to do a simple git merge will not work. The extent of the changes is too great.

My idea is to merge in these changes in smaller chunks. Looking at the NodeContext object (a basic building block of a Bitcoin application), we can divide functionality into 3 large pieces.

  • P2P – The address manager, connection manager, ban manager, and peer manager
  • Chain – The chain state, persistence, and mempool
  • Glue – Smaller components that wire things together. Examples are the fee estimator, task scheduler, config file and command line argument processing, client (i.e. wallet) connectivity and functionality.

Building a NodeContext object will be the first step. Each piece will have access to an object of this type. This will provide the state of the system, as well as be where components update state when necessary.

The “glue” components often require resources from both “p2p” and “chain” components. Hence they should probably be upgraded last.

The task of upgrading the “chain” piece is probably smaller in scope than upgrading “p2p”. I will attempt to attack that first.

The “p2p” pieces do not seem to be too difficult. Most of the work seems to be in wrapping much of the functionality of main.cpp and bitcoind.cpp into appropriate classes. The communication between components are now better defined behind interfaces. The pimpl idiom is also in use in a few places.

Note: Bitcoin 0.22 requires C++17. For more information, click here.

Branching Strategy for Komodo Core

A bit of background…

The current branching strategy used in Komodo Core is somewhat informal. The idea is to formalize ideas so that everyone agrees on the procedures to follow, and those coming after us know what should be followed.

This is a living document. If the procedures below do not serve their intended purpose, they (or portions of them) should be replaced.

Why this and not that?

There are a number of branching strategies available. This Komodo Core branching strategy is heavily based on Git Flow. This was chosen because the existing strategies match the style of the code base (long lived branches, multiple versions deployed and supported, etc.).

How it works

There are a number of long-lived branches within the Komodo Core repository. The primary ones are:

  • master – The main production branch. This should remain stable and is where code for official releases are built.
  • dev – The development branch. This is where work is done between branches. Non-hotfix bug fixes and new features should branch off this branch.
  • test – The branch for the testnet. As the codebase stabilizes before a release, the code is merged from dev to test.
  • hotfix – After a release, a hotfix branch may need to be created to fix a critical bug. Once merged into test and master, it should also be merged into dev. These branches should not be long-lived.

Tags

Each release for testnet and production should be tagged. The versioning strategy is not currently part of this document.

Reviewing / Merging

Before a feature or fix is merged into a parent branch, reviews must be approved by repository maintainers. Anyone can review, but approvals from 2 repository maintainers must be completed.

Once approved, it is best (if possible) that the author of the feature or fix merges their branch into the parent branch. Once the merge is completed locally and all unit and CI tests pass, the code is pushed to the parent branch.

Note: It is best to not use the merge feature of the GitHub web interface. Perform the merge locally and push to the server.