Boost Test Command Line Favorites

The BitShares-Core project uses Boost Test features to exercise the product. When starting the test, here are some interesting command line options:


This will run a specific test. We organize them into modules, that usually match the filename for more easily finding a failing test.


This provides more detail about a test


If you have a lot of output, this will give you a report at the end that details passing and failing tests. It can be parsed to find the failing tests, and associate that failure with the test that caused it.

BitShares Testnet Notes

Here are some notes about the BitShares testnet. I just needed a single place to keep these:

When syncing from scratch, it will seem to hang with the following message in the default log:

1753845ms th_a application.cpp:569 handle_block ] Got block: #6690000 006614d0b54c9b1d1aa4f3e9e4af9fbdf55d5b26 time: 2017-03-07T12:32:40 transaction(s): 4 latency: 76463793845 ms from: f0x irreversible: 6689981 (-19)

This is due to the testing that took place in March of 2017. Be patient. It will go through eventually.

Also, the chain id of the public testnet is 39f5e2ede1f8bc1a3a54a7914414e3779e33193f1f5693510e73cb7a87617447

If there is a problem with network latency, it can flounder when trying to catch up on blocks, especially around March of 2017. To increase the timeout, adjust this line.

HTLC and Wallet Security

As Hash and Time Lock Contracts become more popular, they begin to be seen by the end user. For instance, wallets are now being built with this functionality. And that exposes the user to some unscrupulous actors that could cause financial harm.

This is intended to be a document aimed at software developers. What an HTLC is and how an atomic swap is conducted will not be discussed. But interested end-users may also gain some knowledge within.

Hashlock attacks

A message digest looks very random, and it is. An important item to remember is that it is a one-way hash. The hash itself tells you nothing about the original preimage (a.k.a seed). We do not know what it was, how large it was, or when the hash was generated, or even if it was generated.

Only if we have the preimage and algorithm can we verify that the hash was generated by that preimage. In other words: if we apply a message digest algorithm to a given preimage, it will always generate the same output.

Note in the statement above that there are no guarantees about a different preimage not generating the same output as well. Therefore, using a strong algorithm is important. At the time of this writing, SHA256 is the most common, and is considered secure.

Most hashing algorithms have a fixed size output. For instance, SHA256 generates 256 bits (32 bytes) of output. No matter the input, the output will be at least and at most 32 bytes.

An Oversized Preimage Attack

There are limits to the size of a preimage that can be used. That limit is set by the blockchain itself. But when dealing with atomic swaps across blockchains, an attacker could use a preimage that is too large for one side of the 2 sided transaction. That means that an HTLC can be created on both chains, but only redeemed on one chain, and not the other.

This can be mitigated by specifying the preimage size as part of the contract. This feature is available on many bitcoin-based chains among others, but is not standard. Should the preimage size be used with an atomic swap, both sides of the transaction should include the preimage size in their contract.

Timelock Attacks

It is standard practice for HTLCs that the timelock should be long enough so that the block that contains the contract can be considered irreversible. This makes it possible for the redeemer to expose the preimage and still be guaranteed that the contract itself will not be reversed.

This becomes even more important with atomic swaps. The shorter duration (a.k.a “inner”) contract should allow time to achieve its own irreversibility. And the longer duration (a.k.a. “outer”) contract must allow time for irreversibility of both.

In addition, with an atomic swap both sides must consider the redemption time necessary. The creator of the inner contract must decide that should the redeemer redeem at the last moment, is there enough time to redeem the outer contract before the timelock expires.

One final consideration of the timelock portion of the contract is the use of capital. Should the other party not accept, the funds in the contract are held until the timelock expires. Long expiration times could result in missed opportunity costs.

The Technical Hurdle for Wallets and Users

Hash and Time Lock Contracts are implemented on many chains. Following the general guidelines of BIP199 is not the hard part. Providing an interface where an end-user is as protected as possible will require effort. With the current wallet user base, such protections for HTLC creation, verification, and execution will be the responsibility of the wallet developer.

Other notes on HTLCs

It is generally recommended that the “outer” contract be on the slower chain. This can minimize the length of the timelock, should there be the desire to use the shortest possible time.

While it may be obvious, the “inner” contract must expire before the “outer” contract.

While the Bitcoin blockchain provides the functionality for specifying the preimage length, BIP199 does not include that logic in its proposal. It is my opinion that if both chains support it, it should be used. Most chains that derive from Bitcoin should support it. BitShares also supports it.

BitShares TESTNET Binaries

These are version 3.1.0 binaries that connect to the BitShares TESTNET. I compiled these mainly for use by the BUIDL Boston hackathon, but there is nothing special about them. You can build them yourself from source by building the tags/test-3.1.0 tag from

The file cli_wallet_mac.tar.gz has the sha 256 checksum hash of 5e0d89576115e4ece3ec2fa99221a2a9ebcadfdd0d22e309f2739ba38259a787

The file BitShares-Core-test-3.1.0-Linux-cli-tools.tar.gz has the sha 256 checksum hash of bdc9159a617b56e560e565a633012f739d20170f68aebd79744be1655a5acb02

The file has the sha256 checksum hash of 3b8c26ce8955bcd81b716ff4f6236a5123648a3eb0009010ac5c9f5aeb0b69a8

Note: The Windows version of cli_wallet.exe must know where your certificates are stored if you wish to connect to secure websockets. The environment variable SSL_CERT_FILE should point to a .pem file that contains those certificates. If you need a certificate file, I recommend you download it from the curl website by clicking here.

If you place the above cacert.pem file in the same directory as cli_wallet.exe, you can easily set the environment variable and start the cli_wallet.

set SSL_CERT_FILE=./cacert.pem
cli_wallet.exe -s wss://

Below are some URLs to BitShares Testnet nodes:

  • wss://
  • wss://
  • wss://

Bitshares and Hashed Time Lock Contracts (HTLC)

It was certainly a privilege to participate in Bitfest Amsterdam (22-September-2018). Here are the slides of the presentation I gave there.

Bitshares will soon support Hashed Time-Lock Contracts (HTLC). The idea of HTLCs has its basis in a secret (called a “preimage”) and an algorithm that creates a message digest (often called a hash algorithm). There are a variety of hash algorithms available. The most common are SHA256, SHA1, and RIPEMD160. Bitshares will support those three.

A hash algorithm takes a preimage and turns it into a binary string of a certain length. For instance, if you ran the words “My Secret” through the algorithm SHA256, you would get:

If you change just one small thing in your preimage, the hash changes entirely. Another property of these algorithms is that the length of the hash does not change. The hash length of “My Secret” will be the same as the hash length of the bytes of an image, or a PDF document that contains the text of the book War and Peace.

Such algorithms are key to many interesting possibilities, including Hashed Time-Lock Contracts. The most popular use of HTLCs are atomic swaps. If two chains support HTLC, and share at least one hash algorithm, you can do cross-chain atomic swaps.

This means that if you can find someone to trade with, you can exchange one coin on one chain for another coin on another chain. Lets walk through this together with an example:

Here we have Alice and Bob. Alice has 1 bitBTC on the Bitshares blockchain. But she needs to pay her rent. Her landlord is old-school, and only accepts Bitcoin. But Bob is willing to receive her 1 bitBTC in exchange for 1 BTC. Alice and Bob agree to swap. Alice creates an HTLC on the Bitshares blockchain which will transfer 1 bitBTC to Bob’s account if he presents the preimage that matches this hash before a certain time. 24 hours for instance.

If Bob presents the preimage, he gets 1 bitBTC in his Bitshares wallet. If he does not, Alice gets her 1 bitBTC back in her account. But how does he get Alice’s secret preimage? Hold on, it is coming.

Alice shares the identifier of her contract with Bob. Bob can examine it to see if it matches the terms of the deal. He also can see the hash, and will use the same hash to create another HTLC on the Bitcoin blockchain.

Bob’s HTLC is very similar to Alice’s contract. He is agreeing to put 1 BTC in Alice’s Bitcoin account, if she can present a preimage that can be run through the hashing algorithm and return the correct hash. There is also a timeout. If she does not do this with say 12 hours, the 1 BTC again belongs to Bob.

Bob then shares the Bitcoin hash for this contract with Alice. She can inspect it, and if she likes, she can present her preimage and the 1 BTC is in her Bitcoin wallet. She can now pay her rent. At the same time, her preimage is now exposed on the blockchain. Bob can see it, and can use it to “unlock” the 1 bitBTC on the Bitshares blockchain. Both contracts are satisfied, and the cross-chain atomic swap is complete.

Here are some things you can do with HTLCs:

They may be used for regular purchases. It is not necessary that they be used for only cross-chain atomic swaps. It could be used to prove that Alice has the funds, and is willing to set the funds aside for a certain amount of time.

HTLCs could also be used to swap assets without using order books and online exchanges. Perhaps Alice and Bob agree to swap a large quantity of assets at a certain rate, but do not want to use the exchanges available to them. This is similar to “dark pools” that some US securities brokers have.

But of course, the most popular use-case is the scenario we just walked through, cross-chain atomic swaps.

Which foreign chains support HTLCs? The website keeps track of them. Here is an image of that website as of today:

As you can see, HTLC support has been implemented across many blockchains. Red ones are not quite there, orange ones can be used but are missing some features, and green ones have fully implemented it.

So what is left to do to make HTLCs available in Bitshares? Wallets that support creation and review of these contracts would certainly be useful. Communities of “mini-gateways” are needed so that users can find others that are willing to swap with them. And note that these “gateways” do not need to issue their own coins, as is often done today.

The ability to create these contracts in Bitshares Core is actively being worked on, and it is my hope that it is included in the next hardfork release. The BSIP is almost complete, and should be available for public review in a few days. After that, the community can build on top of it.

This is an important feature that will soon be included in Bitshares. Learn about it. Build on top of it. There are several opportunities here that you can capitalize on. I am excited to see different ways this feature can be used to help you and your business. If I can help, or if you have questions, contact me.

Bloom Filter Types

In the implementation I am working with, a bloom filter allows us to determine if a particular item has been selected from a set. Both the set and subset can be large.

While speed is a benefit, the real benefit for us is the memory footprint. We can quickly determine if the item is (probably) selected, without maintaining a list of (large) keys.

I am researching various types of bloom filters that must fit within the following parameters:

  • Low false positive
  • Memory efficient
  • Deletable
  • No false negative

The last two pose the challenge. A standard implementation of a bloom filter does not have false negatives, but items cannot be deleted. Deletions without knowing if an item was previously inserted can cause false negatives in the following implementations:

  • counting bloom filter
  • cuckoo filter
  • dlbf (possibly, not tested)

Research links:

Git Tricks

Here are some tricks that I use. They’re here because I need a place to quickly refer to them.

  • Updating a fork with upstream changes:
  • Interactive rebasing:
    • git rebase -i <branch>
  • To avoid messing up a PR, you can do things like:
    • git reset –hard HEAD-1 # or HEAD-2, etc, or directly to a commit hash
    • git push –force origin branch_name
  • Always do a backup before reset or rebase

C++ Publish Subscribe

I had the need to make a publish-subscribe mechanism in C++. So as is common, I looked around at other examples to see how others have done it and found a simple example courtesy of GitHubGist user makomweb.

His didn’t include comments, so I’m commenting it on my own copy here.

As programmers, we learn from each other. Each piece of code we write or see helps us to reason, adjust, and form opinions.

Getting our hands dirty and coding is a great learning experience. But reviewing the code of others helps us to build skills.

Why did makomweb not just have the Subscriber class conform to an interface? The entire class could have been passed to the Producer instead of just saving off the method. Think about it.

Look at the Publisher.Publish method. Elegant. One line of code to call Event(). The magic happens in Event.operator(). It calls the method that was registered in the Event.Subscribe call.

Had we stored an object that complies to an interface, we could do the same thing. We store the object in a map. When a “Publish” happens, we iterate the map, calling the method from the interface.