The entire blockchain ecosystem has evolved a lot over the last few years. However, for widespread adoption of existing public blockchain systems, scalability is the next challenge. In this article, we'll discuss the pressing need for scalability and the common challenges faced in scaling the blockchain. We'll then dive into the measures being taken for tackling these challenges, including potential scaling solutions (like sharding, lightning network, etc.) which are currently being implemented in the existing blockchain systems.
Costs of a Blockchain System
A blockchain is essentially a linked-list of blocks containing ordered-data, with some constraints around it:
- blocks can't be modified once added, in other words, it's "append-only",
- it's distributed, or rather decentralized, and
- there are specific rules (for consensus) for putting data inside a blockchain.
These constraints, when implemented in a form of a protocol, yield a data-recording mechanism with some highly desirable characteristics:
- no single point of control or failure,
- immutability of data, and
- a verifiable audit trail of changes.
Due to these advantages, blockchain systems are suited for the scenarios where mutually mistrusting parties have to exchange value digitally. However, these advantages don't come for free. If we look at it from a different angle, a blockchain is a slow, immutable database with very high redundancy, which means it is expensive to maintain and also hard to scale.
The Need for Scaling Blockchain
Over the past few years, cryptocurrencies (especially Bitcoin and Ethereum) have attracted a lot of interest. For instance, the number of transactions in Ethereum increased from just 3,000 in Oct 2015 to over one million at some point in time in Jan 2018 (that's a whopping 300,000% increase). The following graphs show the number of daily transactions on Bitcoin and Ethereum networks:
Daily transactions on Bitcoin since July 2011
Daily transactions on Ethereum since mid-2015
As we can see, the number of transactions has grown (and hopefully will continue to grow) over time. Moreover, there are newer infrastructures like EOS that are also gaining popularity.
Daily EOS transactions since July 2017
These trends show an increasing adoption of the blockchain systems, but the problem is, over time, a few issues have popped up which call for a better design.
Consequences of the Growing Number of Daily Transactions
Due to the growing number of transactions, a few things have occurred:
- The average confirmation time for a transaction increased (At one point, there were 200,000 unconfirmed transactions and confirmation time of more than a day).
- The network transaction fee increased. (It went up to $60 per transaction in January 2018).
- The block difficulty increased, and hence the average computation power consumed when mining a block increased, leading to higher electricity consumption. (High enough to power entire country, like Switzerland).
- The blockchain size also increased massively, hence making it difficult to set up new full-nodes which have to maintain the complete blockchain state for transaction processing and verification.
As the number of transactions grows, these systems tend to become slower, expensive, and unsustainable for a use-case like payment. That's why scaling is becoming an inhibiting factor in a wider acceptance of blockchain systems. Also, scaling the systems isn't as straightforward as deploying more resources (more nodes slow down the network because there's additional time is required for the information to reach new node via gossip method, and more time required for the computations that the new node performs), so there's a need to come up with better designs for scalability.
In the rest of the post, we'll discuss what is being done (or can be done) to overcome the hindrances in the adoption of blockchain systems en masse.
Potential Solutions for Blockchain Scalabilitiy
Keeping the blockchain architecture in mind, let's look at some of the focus areas (in an architectural context) where we can make changes to scale the system for the masses.
We'll take Bitcoin (and Ethereum) as a reference here, but these areas generally apply to most blockchain systems. For the sake of clarity, the key metric which we'd like to improve is transaction rate (number of transactions processed per second) without compromising other desirable characteristics (security, usability, etc.) of the blockchain system.
Bitcoin processes 7 transactions per second on average, and Ethereum processes 20. The transaction-per-second rates of centralized payment methods like VISA are 1700 (PayPal is 200), so blockchain technologies need to get close, if not better, to be a viable means for a use case like payment at scale.
Apart from transaction rate, we'd also like to focus on the sustainability of the solution. For example, PoW systems are unsustainable due to the linear relationship of electricity consumption on a growing scale.
Number of Transactions in a Block
One idea for scaling is to include more transactions in a block. This can be done in two ways,
Increasing the Block Size
In a blockchain system, the block size is restricted to prevent spamming and clogging of the network by malicious parties. In Bitcoin, the block size is 1MB. In Ethereum, there's an equivalent concept of gas limit (gas is a measure of computation) which induces an upper-cap on the amount of gas that all transactions in the block can consume. We can increase the block size so that more transactions can fit into it.
Using Less Bytes to Represent Information In the Block
There is information related to each transaction that needs to be maintained in each block for security and verifiability. If we can somehow reduce the amount of information required, we can achieve better throughput. Some of the ideas could be"
- Using more efficient hashing algorithm that can generate short signatures (Schnorr signature is one such approach).
- Using alternative data structures other than merkle trees to organize transactions inside a block. (Merkelized Abstract Syntax Trees (MAST) is one such approach that reduces the size, and increases their privacy.)
Frequency of Block Addition
How often a new block is added to the blockchain also impacts the transaction rate. The time is a function of the block difficulty level in PoW systems. In Bitcoin, the average block-creation time is 10 minutes, and in Ethereum, it's 7 seconds. Reducing this time would mean more transactions going through at a faster rate.
The number of transactions in a block multiplied with the frequency of block addition gives us the upper-limit for the transaction rate.
You might wonder, if increasing the transaction rate is as simple as increasing the block size and frequency of block addition, why is there all that fuss about scaling? It's tricky, and let's go over why.
Reducing block creation time will:
- increase the orphan rate (mined blocks not making it to the main chain due to competition), and hence more waste of computations.
- result in reduced effective hash rate, greater centralization pressure, and hence the reduced security of the network.
- require increased network bandwidth (since more internode communication will happen).
Increasing block size will:
- increase the propagation time of the blocks (since it's more data)
- mean that the blocks would take longer to verify.
This is worse than it appears because the node finding a legit block would get a head start on the next block and so on. It will tend to favor the large miner groups (and hence centralization).
Also, both of the changes would require hard forks (non-backward compatible upgrade to the chain function).
Communication Between Nodes
Depending upon the protocol implementation, there's an exchange of information between the nodes in every transaction confirmation (or block added). For example, in Bitcoin, the transaction is propagated twice, first in the transaction broadcasting phase, and then after being mined in a block.
Since every over-the-network communication costs some time, utilizing the network bandwidth more efficiently is one area we can focus at for improving the throughput. Increasing the connectivity between the nodes can also reduce the propagation delay to some extent.
Consensus and Verification
The speed of consensus would also affect the throughput of the blockchain system. Proof of Work (PoW) involves a lot of computation, and the block difficulty only gets higher with scale, which means more time and resources to process a transaction. And not to mention that PoW is unsustainable.
Alternative Proofs
Just like Altcoin means coin other than BTC, alternate proofs are any other consensus techniques than PoW. The PoW works on a fair assumption that "it's practically infeasible to get computation power more than 51% of the network".
One of the famous consensus technique is Proof of Stake (PoS). In PoS, the miners (or rather validators) will have to put their tokens at stake to validate a transaction. The validators will get the reward (usually proportional to their stake, but other variations also exist) for validating and processing the transaction, and will lose their stake in case of failure to validate a transaction.
Compared to PoW, PoS is sustainable in terms of electricity consumption. PoS will also result in an improved frequency of block addition since miners don't have to spend time solving the cryptographic puzzles. Also, Ethereum tokens are more evenly distributed than the current hash-rate distribution of the miners, so it provides more finality and decentralization than PoW. There are still some caveats with PoS, and there are many variations of it being worked upon currently.
Storage architecture
In Bitcoin, we store all the blocks across all the nodes, which requires a lot of storage space. A miner node needs to have the entire blockchain state to be able to verify new blocks. The size of this storage is only going to increase over time. This will rule small individual miners out of business (who can't afford to have too much storage capacity) and favor large groups of miners (and hence centralization). The problem becomes worse if we increase the transaction rate (since it means data is getting into the chain at a faster rate).
So the question is, do we really need to replicate everything? Is it possible to prune the data needed to be maintained by the nodes without compromising with the existing functionality?
Off-chaining
The off-chaining concept, as the name suggests, involves getting some of the processing done off the main-chain. This would decrease the load on the blockchain network and the transaction processing cost.
In off-chaining, not every, but only a few nodes are allowed to compute the complex proof. The participant nodes have to submit a deposit before computing the proof. When the proof is obtained, it is verified by a group of nodes. If the proof is correct, the submitter gets the deposit back along with the reward. Otherwise, the deposit is fortified.
State Channels
The idea of state channels is to establish a communication channel between participants so that state updates and transaction processing can be offloaded to them. A smart contract (an implementation of Hashed Timelock contracts) locks a part of the blockchain state enforcing the business logic among the participants. The participants perform multi-signed transactions amongst each other (without the involvement of blockchain). Finally, the resultant state can be submitted directly to the blockchain without the involvement of miners (which reduces the time taken). The smart contract makes sure that the transactions are legitimate, updates the state, and the locked portion of blockchain state is now unlocked.
Sharding
The idea here is similar to how sharding happens in typical databases. The blockchain state is divided into different shards, and a node doesn't need to depend on the entire data to verify and process a transaction. The transactions will be directed to the nodes based on the shards they affect, and all the nodes work in parallel, thereby reducing the overall time taken. How shards communicate with each other and arrive on consensus is an active area of research.
Also, sharding is more secure if applied in a PoS based system instead of PoW. The validators for a shard are chosen randomly. The holy grail of PoW security is the amount of computations required to outperform the majority, which would be reduced in the case of sharding, making it easier to attack shards with a lower hash rate.
Sharding in Ethereum has been discussed in detail in this post.
Subchains
Subchain architecture has a tree-like hierarchical structure, having a root chain and a bunch of child chains. It works on the concept of strong blocks and weak blocks. Weak blocks are the blocks that meet a weaker difficulty target. The weak block is broadcasted to other miners as soon as it is found. Miners pile up weak blocks until a block with the defined difficulty target is found (strong block). Once the target is met, the child chain is closed and linked to the root chain, reducing the average first confirmation time of transactions.
Subchains were proposed as a layered solution above the existing Bitcoin architecture, and more details can be found in this paper.
Why Making the Change Isn't Easy
Making the change is not as easy as it sounds. In a decentralized system, we cannot force anyone to do anything. For a change to be implemented, the consent of all the stakeholders (miners, developers, and users) needs to be established, otherwise, community splits happen —like what happened in the case of Bitcoin Cash and Ethereum Classic. Not to mention, there are big corporations, politicians, and governments who make the process tougher through regulations (especially if the change conflicts with their interests).
Even if everyone favors the change, it's not an easy process for the developers, since all upgrades have to be backward-compatible (as there's no such thing called forced-updates in a system that's decentralized). Any sort of failing in these systems is absolutely unacceptable, and hence rigorous testing needs to be performed before going live with any changes (a corrupted blockchain is of no use).
Community splits due to an opinionated change are very costly to the entire blockchain ecosystem, as maintaining a blockchain system requires a lot of effort, and hence requires a passionate community.
What has the Bitcoin Community Done so Far?
The Bitcoin community has been actively working on improving the system. Every proposed change for improvement is documented in a Bitcoin Improvement Proposal and only implemented after the consensus of Bitcoin users. Bitcoin is an open source project, but funding can also play a role in prioritizing the changes to implement.
Here are some of the notable changes in the context of scalability, which we'll go into more detail on:
- Bitcoin Cash fork (the block-size change)
- Segwit (improving the transactions storing capability in the block)
- The Lightning Network
- Bitcoin Next-Generation (a different consensus architecture)
Bitcoin Cash (BCH)
There have been many hard forks within Bitcoin's journey. The BTC-BCH hard fork was probably one of the most debated hard forks. Despite the downsides of increasing the block size, a part of community favored higher block size, and the BCH protocol ended up supporting an 8MB block size.
Very recently, BCH has further experienced a fork into Bitcoin Cash ABC and Bitcoin Cash SV.
Segwit (Segregated Witness)
Segwit was a very substantial backward-compatible protocol upgrade in Bitcoin to account for scalability.
Before Segwit, the blocks contained signature information (required only for initial verification of transaction) which accounted for two-thirds of the stored data inside a block. Removing this data would allow more transactions to be included in a block, and hence, increasing the overall throughput.
After Segwit, the signature data is stored in separated extended blocks in a chain parallel to the main blockchain. The signature data is also called the "witness", hence the name of the event, "segregated witness." The concept of "block weight" was enabled to include more transactions in the blocks following the new protocol while still being compatible with the old protocol, hence preventing a hard fork.
Lightning Network
Segwit also fixed the problem of transaction malleability, which made a layer-2 upgrade possible, known as the Lightning Network. Lightning Network is an implementation of state channel architecture, as we have already discussed, wherein all the participants of a transaction sign it to be valid. This allows for instant settlements and micropayments in a channel since no fee is involved. Once the channel is closed, the consolidated transactions can then be broadcasted to the network.
Next-Generation Bitcoin (BTC-NG)
The Next-Generation Bitcoin paper proposes to give a "leader" node the responsibility of including the transaction. This means:
- A leader is elected for every block-creation cycle.
- The transactions are included in the micro-blocks, which are later signed with a leader's private key.
- Leader blocks are responsible for Proof of Work.
The paper included some very interesting concepts, which were later implemented in new projects like Waves and Aeternity.
What is Ethereum Doing?
Just like Bitcoin Improvement Proposals, Ethereum has an Ethereum Improvement Proposal for proposing new changes. On a broad level, here are some Ethereum projects that are being implemented, keeping scalability in mind. Here are a few things that are under development in Ethereum and related projects:
- Plasma and Loom (subchain architecture)
- Raiden (a lightning network)
- Casper (PoS implementation for Ethereum)
- Sharding
- eWASM (a better instruction set and execution environment)
Introducing Proof of Stake (PoS) is going to make the blockchain a lot faster because it is much more simple to check who has the most stake than to see who has the most hashing power. This makes coming to a consensus much more simple. Plus, implementing a “Proof of Stake blockchain” is an integral part of Serenity, the 4th and final form of Ethereum (more on this in a bit.)
At the same time, PoS makes the implementation of sharding easier. In a Proof of Work system, it will be easier for an attacker to attack individual shards which may not have high hash rates.
Also, in PoS miners, won’t be getting a block fee, and they can only earn transaction fees. This incentivizes them to increase the block size to get in more transactions (via gas management).
Loom, Plasma, and Raiden
There are a lot of second layer solutions being worked upon currently. Plasma is a subchain like the Layer-2 approach where a series of contracts run on top of the root chain (the main Ethereum blockchain). The child chains process a high number of transactions and regularly feedbacks a compiled inalterable version of its ledger to verify the authenticity. Ethereum based project, OmiseGo, has already implemented an MVP for Plasma. Loom is another project working on the Plasma implementation. Ethereum is also planning to activate something like the Lightning Network, which is called “Raiden.”
Casper
Casper is the flavor of the Proof of Stake implementation in Ethereum. Initially, the plan was to roll out Casper FFG (a hybrid of PoW/PoS where in one in every 50 transactions is PoS to smoothen transition to PoS) in the Constantinople upgrade this year. However, there was a delay due to the developments not aligning to the direction of sharding. Here's an interesting infographic about that from Vitalik:
It's expected to roll out sometime next year.
Sharding
One blocker for scalability was that all the processing happens on a single main chain. Sharding enables parallelization of processing (in the form of multiple mini node-clusters) which would eventually increase the transaction per second rate. The main chain which keeps track of the entire blockchain state would be called the "beacon chain".
Sharding with PoW is much more tricky than PoS. PoS implementation in Casper would lay a much better foundation for implementation of sharding, ultimately, addressing the scalability problem.
eWASM (Ethereum Web Assembly)
EVM (Ethereum Virtual Machine) instructions are ideal for running in Ethereum but not efficient for execution on normal CPUs. Ethereum is going to improve the efficiency of EVM by implementing eWASM instructions (derived from WASM), which are ideal for running on CPUs (as WASM opcodes are more similar to opcodes in common CPUs). eWASM instruction implementation would result in faster execution of smart contracts and effectively more instructions per Ethereum transaction.
Also, currently, deployed contracts are automatically compiled and executed. The web assembly upgrade, on the other hand, plans to transition to on-demand compilation and execution, further improving the EVM efficiency.
Serenity (Ethereum 2.0)
According to Vitalik Buterin (the co-founder of Ethereum),
“Serenity is ‘the world computer’ as it’s really meant to be, not a smartphone from 1999 that can play snake.”
Serenity upgrade is where years of effort in different areas of research in blockchain and Ethereum would come together into one picture. It involves the realization of pure PoS, sharding, eWASM, and other actively researched ideas in the past few years.
Other Noteworthy Projects and Research Frontiers
DCS theorem, probably a blockchain version of the CAP theorem, points out that we can have only (at most) 2 properties out of the 3 desired properties in a decentralized consensus mechanism: Decentralization (D), Consensus (C), and Scale (S).
An advancement in one of the property comes at the cost of the other two. The paper suggests to combine multiple DC solutions or DC Layer 1 solutions (e.g. the existing Bitcoin chain) with SD Layer 2 solutions (e.g. state channels and Lightning Network) for scaling, and this is exactly the direction in which most of the upgrades are being rolled out.
Here's a couple more projects to mention before we conclude:
IOTA
IOTA is a blockchain designed for IoT (Internet of Things) that doesn't use blocks. Instead, IOTA transactions are stored in a distributed ledger named structure in the form of a Directed Acyclic Graph. In IOTA, there are no miners, and newer transactions are responsible for confirming earlier transactions. This parallelized validation by participants results in high throughput.
The absence of miners results in a greater degree of decentralization and no transaction fees. IOTA makes use of a quantum-immune hash function called Curl-p, which makes the ledger resistant to quantum attacks (another area where existing blockchains may suffer in future).
EOS
EOS in another interesting project that's gaining popularity. EOS uses delegated PoS as its consensus mechanism. There are 21 delegated nodes (elected by the stakeholders every few blocks) called block producers, which are responsible for the validation of new transactions and putting them into the chain.
EOS has no fee, instant transactions, is high scalable (block creation time is just about 0.5 seconds), but it all comes at the cost of decentralization — what stops the block producers to function in synergy against a participant? The EOS architecture is defended by the argument that no stakeholder would vote for a block producer that's untrustworthy, and the block producers themselves have a high incentive to function honestly.
Trusted Execution Environments
Recently, Intel came up with a Software Guard Extension (SGX) architecture, which allows allocation of private hardware environments for the execution of a program called enclaves. After the code runs in an isolated enclave, a proof (called quote) is generated, which is verifiable by Intel's cloud server. So the node executing the smart contract can provide proof, and other nodes just have to verify it (instead of executing the code). However, the dependence on Intel's cloud servers raises concerns of centralization.
Conclusion
In this post, we came across the challenges to scaling a blockchain system to meet the ultimate goal of widespread adoption of public blockchain systems. We saw how far we've come with existing projects like Bitcoin and Ethereum, and also looked through the other innovations happening in the industry. We have high ambitions for blockchain systems changing payments, asset management, etc., but there's still some distance to cover to reach a system that's sustainable, efficient, secure, and scalable. However, if we maintain our efforts, we'll reach there soon!
