Sunday, March 23, 2014

Arthur C. Clarke on economic meltdowns and network congestion

Here is a fun passage from Arthur C. Clarke's Rama II fictional novel, published in 1989.  It presents an arguably realistic crash scenario for our [real world] financial systems.  Food for thought in the bitcoin economy or fiat economy both. We are incredibly dependent on computer financial databases.

In contrast, terrestrial affairs were dominated by the emerging world economic crisis. On May 1, 2134, three of the largest international banks announced that they were insolvent because of bad loans. Within two days a panic had spread around the world. The more than one billion home terminals with access to the global financial markets were used to dump individual portfolios of stocks and bonds. The communications load on the Global Network System (GNS) was immense. The data transfer machines were stretched far beyond their capabilities and design specifications. Data gridlock delayed transactions for minutes, then hours, contributing additional momentum to the panic.

By the end of a week two things were apparent—that over half of the world's stock value had been obliterated and that many individuals, large and small investors alike, who had used their credit options to the maximum, were now virtually penniless. The supporting data bases that kept track of personal bank accounts and automatically transferred money to cover margin calls were flashing disaster messages in almost 20 percent of the houses in the world.

In truth, however, the situation was much much worse. Only a small percentage of the transactions were actually clearing through all the supporting computers because the data rates in all directions were far beyond anything that had ever been anticipated. In computer language, the entire global financial system went into the "cycle slip" mode. Billions and billions of information transfers at lower priorities were postponed by the network of computers while the higher priority tasks were being serviced first.

The net result of these data delays was that in most cases individual electronic bank accounts were not properly debited, for hours or even days, to account for the mounting stock market losses, Once the individual investors realized what was occurring, they rushed to spend whatever was still showing in their balances before the computers completed all the transactions. By the time governments and financial institutions understood fully what was going on and acted to stop all this frenetic activity, it was too late. The confused system had crashed completely. To reconstruct what had happened required carefully dumping and interleaving the backup checkpoint files stored at a hundred or so remote centers around the world.

For over three weeks the electronic financial management system that governed all money transactions was inaccessible to everybody. Nobody knew how much money he had—or how much anyone else had. Since cash had long ago become obsolete, only eccentrics and collectors had enough bank notes to buy even a week's groceries. People began to barter for necessities. Pledges based on friendship and personal acquaintance enabled many people to survive temporarily. But the pain had only begun. Every time the international management organization that oversaw the global financial system would announce that they were going to try to come back on-line and would plead with people to stay off their terminals except for emergencies, their pleas would be ignored, processing requests would flood the system, and the computers would crash again.

It was only two more weeks before the scientists of the world agreed on an explanation for the additional brightness in the apparition of Halley's Comet. But it was over four months before people could count again on reliable data base information from the GNS. The cost to human society of the enduring chaos was incalculable. By the time normal electronic economic activity had been restored, the world was in a violent financial down-spin that would not bottom out until twelve years later. It would be well over fifty years before the Gross World Product would return to the heights reached before the Crash of 2134.

Wednesday, March 19, 2014

Bitcoin Core v0.9.0 release overview

The Bitcoin open source project has released a major new version, 0.9.0.  This update brings the usual basket of fixes, performance improvements, security enhancements and new features.  The Bitcoin Core Wallet (formerly Bitcoin-Qt) introduces a new direct-to-merchant payment method with the BIP 70 payment protocol and the workflow for receiving coins has been notably improved.  Under the hood, several improvements should reduce P2P network spam, enhance privacy and security, and improve performance.  For enterprise users, the wallet is made optional, introducing a blockchain-only "border router" mode.

Initial support for the BIP 70 payment protocol was added to the Bitcoin Core Wallet.  The payment protocol is an optional feature designed to upgrade the security and reliability of the customer-merchant payment process.  For the case where a direct connection between the customer and merchant already exists, such as when customer is shopping on a merchant's website, the payment protocol may be used to send payments directly from the customer's bitcoin wallet to the merchant.  This has several advantages: added security, speed, and the ability of a merchant to assist the customer in getting their transaction relayed and confirmed on the P2P network. Refund ability is also included. The goal is to make the payment process smooth and secure, and lessen dependence on everyday use of the now-familiar bitcoin addresses.

A recent trend has been the use of Bitcoin Core Server (bitcoind) in a new role.  Organizations with large bitcoin architectures will use bitcoind as a "border router" while developing their own custom wallet solutions.  This role involves bitcoind running the P2P network and distributed consensus services, providing a high quality payment firewall, while the organization manages their own keys and transactions.  The wallet, in this mode, may be disabled at compile time or runtime.  Some sites have seen memory savings of 40-200MB when running in router mode.

Many minor improvements have been made to payment network operation. Transaction relay rules have been tightened further, reducing several types of spam or malicious traffic, including the recent transaction malleability issues.  The anti-spam minimum fees have been reduced, as they were set at a time when bitcoin's price was much lower. Historically, these fees are only reduced.  Future "smart fee" work is intended to eliminate the hardcoded minimums completely.  A dynamic system and open, free fee market are the desired long term goals. Unrelated to fees, a new "reject" P2P message should provide useful feedback to nodes submitting invalid transactions.  Additional DoS and privacy protections were added.

It should be noted that this fee reduction only applies to the suggested anti-spam minimums.  The actual fee paid by users is determined by what miners are willing to include in a block.  Reducing the anti-spam minimum fee should increase the chance of a transaction being relayed across the network to various miners.  This does not imply a global network fee reduction.  Transaction fees are and continue to be set by the mining market.

Automated services and websites make use of Bitcoin Core Server's programmable RPC API.  Many new, minor features were added to RPC to facilitate blockchain queries, validate blockchain data and bitcoin addresses, and other utility features that bitcoin websites find useful.

New features for bitcoin developers include a new autotools-based build system, harmonizing bitcoin development with many other packages in the open source world.  A regression test mode has been introduced to facilitate automated bitcoin testing.  This "regtest" mode features near-zero block generation times, permitting a site to simulate thousands of test blockchain scenarios in a short amount of time.

This update also includes platform-specific improvements.  Windows client is now 64-bit, enabling better performance and stabilty.  GCC's stack smashing feature is enabled on Windows, as it already was on other platforms.  Apple OSX integration was improved.  Finally, support for the older 10.5 OSX/32-bit platform was removed.

Outside of new features, this update includes several fixes and tools designed to avoid problems with zero-confirmation malleable transactions.  (Confirmed transactions are, by definition, not malleable)

Read the full release notes at