Nov 2018 Stress Test Report
After the successful creation of the first blocks larger than 32MB, the stress test team identified a number of changes to the architecture of the Satoshi Shotgun to better enable the successful generation of a longer string of big blocks on the BCH-SV network. These changes were implemented, and from the 17th to 19th of November a sustained, high volume stress test was conducted that demonstrated the network’s ability to sustain high volume traffic and to mine big block, as well as demonstrating its robust nature when faced with volume well outside its normal usage.
Miners, users and services were all impacted however the Stress Test team believe that performing this nature of work is necessary to proving not just the validity of the scaling path laid out in Satoshi’s Vision, but the long term viability of services which need to scale with the network.
Over 24 million transactions were sent over a 5 day period leading to the mining of the largest block ever on a public blockchain. Despite the impacts to businesses and services, real world testing has proven again to be one of the best ways of generating feedback on system performance. We believe this demonstrates how useful stress tests can be when properly scheduled, advertised and conducted in a transparent and thoughtful manner.
The report goes through the design of the Satoshi Shotgun and covers three separate testing periods:
1) A 1.4 million tx blast immediately after the fork at around 7pm UTC on the 15th of November.
2) 10 million txs sent as part of the main test on the 17th and 18th of November.
3) A further 16 million transactions sent as a follow up test on the 19th and 20th of November.
The Satoshi Shotgun
For this test, we decided to go all out. In the event of a split, we wanted a way to ensure that transactions would be replayed on both chains so we set up numerous nodes with different BCH implementations to ensure our testing would cover all potential outcomes of the split.
For this test our main node, which generates all the addresses, transactions and handles the dispatching process, was configured as a 64-core machine with 240GB of RAM. This was to handle a transaction load in excess of our 1 million per hour target. The main node was running Bitcoin Unlimited at the time of the fork, however some early testing indicated that it was having issues making P2P connections with the Bitcoin SV nodes in our cluster so we transitioned to a Bitcoin SV node for the main test.
NOTE: See the final page of the report for a diagram detailing the workflow of the Shotgun from when funds are received to when transactions are sent.
We included nodes using Bitcoin SV, Bitcoin BAB and Bitcoin Unlimited, configured to operate under both the Bitcoin SV and Bitcoin BAB rulesets.
The nodes were distributed as shown in the following table:
Satoshi Shotgun Software
Notable improvements include:
- On-demand scaling system to allow rapid assembly of a shotgun of any size, using nodes of any client or setting
- Automatic distribution of funds from collection accounts in packages for tx creation and sending
- Automatic connection of nodes within the Shotgun cluster to each other after establishment of connections to the P2P network
- Automated recovery system to re-start transaction batches which have failed (work in progress)
We believe that the Satoshi Shotgun is a potent tool that can be used to demonstrate the performance of the Bitcoin SV network in a highly visible and public way and plan to continue building on the platform to make it easier to use and more capable. So far, we have identified the following features in the development roadmap towards the next iteration of the Shotgun, scaling its capability and utility in line with that of the Bitcoin SV network:
- Ability to accurately calibrate the rate at which transactions are sent
- Plugins to change the type of transaction
- Transactions that use only confirmed funds
- Transactions with randomized numbers of inputs/outputs and values
- Transactions with data carrier (OP_RETURN) included
- Transactions including tokens and bitcoin script
- Ability to support any node client that supports the required RPC calls (including other coins)
- Web facing panel for simple operation
- Simple and fast consolidation of funds post test
Test period 1: Post upgrade blast
In the hours prior to the network upgrade, YoBits a member of the Stress Test Team donated 4 BCH into our collector wallet and requested we use the funds in a blast immediately following the first block following the new consensus rules. We did this and created some of the first big blocks seen on the BCH-SV network. These were as follows:
Table 1: Post Fork Blast Results
This blast was successful in sending over 1,400,000 transactions onto the network over a duration of 3 and a half hours. At this point, nodes the network was already split in two, with nodes following the BAB ruleset banning nodes following the SV ruleset leading to slower transaction propagation. The result of this was that it took approximately 16 hours for all transactions to be mined into blocks.
While the network was not performing as well as it should have, the test demonstrated the performance of the Shotgun delivering similar performance metrics to what we had seen with the pre-test in the previous week and giving us confidence that we would be able to demonstrate large block capacity at the time of the main stress test.
It seemed at this time that our transactions were being replayed on the BAB network however none of the miners on the BAB network were allowing blocks of greater than 8MB to be mined, with the vast majority using block sizes configured to 4MB and below. We are disappointed to see that more than 6 months after the upgrade to 32MB blocks that miners would still refuse to enable the network to perform at its true limits. We also encountered the first instance of transaction malleation in the electron cash wallet, resulting in some funding transactions playing with different txids on each network. This was caused by our wallet sending transactions twice with disparate nLockTime parameters resulting in different TXIDs for spending the same funds on each network. In short, our coins were split, and the blaster was only configured to detect the funds arriving on the SV network. This was an oversight on our part and we then began work on a separate shotgun to for the BAB network rather than a single system for both.
Once this coin split occurred the blaster stopped playing transactions on the BAB network, resulting in just 337,690 transactions being mined during the test.
Test period 2: November 17th Stress Test
While it was our intention to play transactions onto both the Bitcoin BAB and Bitcoin SV networks in the event of a split, the fund split made this more difficult and the decision was made to focus on testing the Bitcoin SV network only rather than both networks at once. There had already been news reports calling our pre-test blast on the BAB network a ‘Spam Attack’, so rather than fanning the flames of discontent, the team discussed options and decided the more responsible decision was to focus on testing the Bitcoin SV network only. We knew we had the support of miners and businesses 5 | P a g e on the network to attempt to generate big blocks, and that the BAB network miners would not support any attempt to create big blocks so we powered down the BAB shotgun.
The official test was kicked off at Midday on the 17th of November and ran for a full 24 hours. We quickly saw large numbers of transactions arriving in mempools and some large blocks mined almost straight away, however we did not see any blocks bigger than 32MB. Highlights included the following blocks:
Table 2: Big blocks mined between midday 17/11 and midday 18/11
Early on we realized very quickly that the BCH-SV network was still operating in a highly fractured state and that transactions were not being propagated in an efficient manner. We also encountered some issues with the Shotgun that led to our master node not detecting incoming transactions, leading to the shotgun running out of funds several times, requiring time intensive re-scanning of the blockchain to recover funds each time.
After 24 hours we decided to continue the test beyond the original 24 hour timeframe, sending a further 5,152,218 transactions onto the network over the next 13 hours. We successfully generated more large blocks but still none over 32MB. Notable blocks included:
Table 3:Large blocks mined during extended test
After over 36 hours of continuous testing, we decided that the network was not going to be able to keep up with the tests we were performing so we decided to wait another day, and to use the time to perform some modifications to the shotgun to improve its reliability.
Test period 3: 19th Nov, first 32MB+ blocks
Starting from around 10am on the 19 th of November, we began testing again, with an updated and improved Shotgun cluster. We added more nodes, changing the 42 unused Bitcoin ABC nodes over to Bitcoin SV, giving us a total of 84 nodes on the Bitcoin SV network.
Using this setup we were able to reliably send onto the network for over 18 hours without stop, generating and transmitting 15,463,369 transactions onto the network.
The analysis of the data that we have conducted shows that while transactions were being pushed onto the network at an average rate of almost 1 million per hour (sometimes hitting up to 2000 per second) the mining nodes were only able to capture and mine around 25% of them.
During this run, we saw a larger number of big blocks than in previous tests, but only 2 that were over the 32MB limit.
At 11:01PM, block 557,333 was mined by SVPool, which included 198,430 transactions and measured just over 38.0MB in size. This was the first ever block over 32MB and included 0.38526938BSV in fees.
Just 2 blocks later, at 11:54pm, block number 557,335 was mined by Coingeek, coming in at 63,999,195 bytes. This was the largest block ever mined on a public blockchain and included 334,074 transactions. This earned Coingeek 0.65140036 in fees and demonstrated the capacity of the network to break through the 32MB threshold.
We received reports that these two blocks caused the most issues, especially for non-mining nodes on the network, with some users reporting validation timeframes well over 1 hour for the 64MB block. This clearly highlights the need for a new software implementation that provides faster validation and better handling of large transaction volumes. This issue will only be further exacerbated as blocks grow to 512MB and beyond with the next Bitcoin SV upgrades planned for May and November 2019.
The breakdown is as follows:
At the conclusion of the test, we had numerous Bitcoin SV application developers contact us thanking us for conducting the test and letting us know that we had shown them the limitations of their own systems, which they are now working hard to improve.
This tells us that the Stress Test has proven very valuable to the network and is a worthwhile and useful service. As a public demonstration these tests serve to show businesses and users where we are in terms of the network’s evolution into the ledger for all.
The Satoshi Shotgun team thank all Bitcoiners for your support during this test and for the kind words and useful tips given by many in the community. We hope to conduct a new test each time the Bitcoin SV network is upgraded to demonstrate its capacity to scale in-line with the original Satoshi Vision and look forward to the excitement of generating bigger and bigger blocks.