Internet Computer Wiki - User contributions [en]

L1 comparison

2024-11-15T14:00:49Z

Yap: Updated finality number to 0.64 s according to https://victoria.ch1-obs1.dfinity.network/select/0/vmui/#/?g0.expr=avg%28quantile+by%28ic_subnet%29+%28%0A++++++++.5%2C%0A++++++++++++avg_over_time%28%0A++++++++++++++++%28%0A++++++++++++++++++++artifact_pool

The promise of a [[World Computer]] and the emergence of a Blockchain Singularity has far reaching consequences in technology, sociology, economics, politics, communication, entertainment, and most aspects of our digital lives. As the industry is one of rapid innovation and progress, and as projects constantly and dynamically change, it's important to take stock, every now and then, to note how the industry is doing, and to check if the Internet Computer is on track to achieve the goals of decentralization, scalability, usability, and functionality.

The industry is now moving out of its infancy, which is seen by the increasing number of smart contract developers, rather than core protocol developers, and users wanting to fully engage with a platform, rather than simply sending transactions back and forth. The shift away from simple payment systems, towards Web3 is well on its way, and it's within this scope that this page attempts to map the blockchain landscape.

Top performing blockchain projects are compared across a number of metrics that are expected to yield a 'good' Web3 experience under the categories of core protocol, developer experience, and user experience.

Unless otherwise stated, all data is correct as of February 7th 2024. Metrics are explained and references are given below.

== Base comparisons ==
This section compares standard metrics that are used to measure performance of the core protocol of popular blockchain projects. Note that these metrics should not always be taken at face value. While references are listed below to note where the figures can be found, it's not always clear how these figures are computed. Additionally, parts of different projects may have the same name, but often are constructed differently (most notably, transactions), and so should not be compared blindly like-for-like. The [https://a16zcrypto.com/why-blockchain-performance-is-hard-to-measure/ a16z blog] has a nice article describing how the industry should think about metrics.

{| class="wikitable sortable"
|-
! Metrics / L1 !! Average MIEPs !! Average TPS !! Average finality !! Average block time (seconds) !! Average tx Cost !! Average energy consumption per transaction (wh/tx) !! Size of network (nodes) !! On-Chain storage cost (1GB p/a)
|-
| ICP ||20000||[https://dashboard.internetcomputer.org/ 3200 (update calls)]||0.64 secs||0.48||$0.0012||0.003||[https://dashboard.internetcomputer.org/ 559]||[https://internetcomputer.org/docs/current/developer-docs/gas-cost#storage $5.35]
|-
| Avalanche || [https://stats.avax.network/dashboard/overview/ 46]||[https://subnets.avax.network/stats/network?selectedChains=mainnet&selectedChart=tx_count&timespan=MAX 14]||unclear||2.3||[https://docs.avax.network/reference/standards/guides/txn-fees $0.035]||2.395||[https://snowtrace.io/validators 1752]||unclear
|-
| Cardano || 2||[https://cexplorer.io/tps 3]|| [https://docs.cardano.org/explore-cardano/time/#:~:text=Transaction%20finality%20can%20be%20achieved,according%20to%20Ouroboros%20consensus%20design. 1 day]||20||[https://docs.cardano.org/explore-cardano/fee-structure/ $0.1]|| 41.27 || [https://cardanoscan.io/ 2876] || [https://docs.cardano.org/cardano-testnet/tools/plutus-fee-estimator/ $1,480]
|-
| Ethereum || 5* || [https://blockchair.com/ethereum/charts/transactions-per-second 12] || [https://ethereum.org/fil/roadmap/single-slot-finality 15min] || 12 || [https://www.theblock.co/data/on-chain-metrics/comparison-bitcoin-ethereum/average-transaction-fee $8.4] || 9.956 || [https://etherscan.io/nodetracker 6395] || $2,439,827
|-
| Near || [https://nearblocks.io/charts 1600]|| [https://pikespeak.ai/near-world/overview 40] || [https://docs.near.org/concepts/advanced/near-indexer-framework#limitations 3.3secs] || 1.1 || [https://docs.near.org/concepts/basics/transactions/gas#the-cost-of-common-actions $0.00013] || 0.602 || [https://nearblocks.io/node-explorer 204] || [https://docs.near.org/concepts/storage/storage-staking#how-much-does-it-cost $1,468]
|-
| Solana || [https://solanacompass.com/ 90]|| [https://solanacompass.com 700 (non-voting)] || [https://www.tbstat.com/wp/uploads/2022/02/20220222_FinalityReport_TheBlockResearch.pdf 12secs] || 0.54 || [https://solanacompass.com/statistics/fees $0.005] || 0.517 || [https://solanacompass.com/statistics/decentralization 1617] || [https://solana.com/docs/intro/rent $35,984]
|}

* ''' Average MIEPs''' measures millions of executed instructions per second which is an approximation of useful work performed. For ICP, Avalanche and Solana the calculation follows from the reported cycles / gas / compute units used in execution. For Near, we approximate by assuming 1 Tgas corresponds to 1ms of CPU time at 2B instructions / 1s of CPU. For Cardano we give the maximum capacity corresponding to 20ms of CPU time per block at 2B instructions / 1s of CPU. For Ethereum we go by the block gas limit. (*However, the EVM is a 32-byte stack machine, so we count 1 gas as 4 CPU instructions to be generous). Further remarks can be found here: [[Not all transactions are equal|"Not all transactions are equal"]].
* '''Average TPS''' measures the transactions processed per second - note that the interval over which these are measured does vary across chains.
* '''Average finality''' refers to the amount of time that passes between the proposal of a new valid block containing transactions until the block has been finalized and its content is guaranteed to not be reversed or modified (for some blockchains, e.g., Bitcoin, this guarantee can only be probabilistic). For ICP, the reported value is the average over all subnets of their node’s average time between starting a round until a valid finalization for this round is available.
*'''Average block time''' refers to the amount of time between blocks (per subnet on the IC)
* '''Average tx Cost''' measures the cost of a transaction. Note that the definition of 'transaction' varies widely across chains. The dollar amounts are computed by converting the native token cost cycles/gas/fee needed per transaction, to USD given the exchange rate.
* '''Average energy consumption per transaction''' measures the network energy consumption to process a transaction (measured in watt hours). Figures true as of December 2023. Source: [https://assets.carboncrowd.io/reports/ICF2023.pdf Carbon Crowd Sustainability Report 2023].
* '''Size of network (nodes)''' notes the number of nodes currently validating the blockchain.
* '''On-chain storage cost''' gives the dollar cost of storing 1GB of data per year on chain. For Near and Solana, to store data one needs to maintain a specified token balance. We convert this balance to USD and annualise by multiplying by 5%. For Cardano and Ethereum, the user pays to store the data "forever", and again we annualise by multiplying this cost by 5%.

== Comparing developer experience ==
Whether they were writing games, operating systems or text editing applications, in the 70s, 80s and early 90s, developers always had to face limitations imposed by hardware. Applications were constrained to accessing a few kilobytes of memory through small stacks and heaps, using limited (and constantly changing) instruction sets, and using significant amounts of power to run instructions. The history repeats itself in the blockchain landscape these days. Application developers are limited to stack sizes of a few kilobytes to several megabytes at best. Persistent storage is expensive and limited. Programmers are bound to using cumbersome APIs that make hidden assumptions in terms of numbers of executed instructions. And, moreover, most chains operate inefficiently, burning too much power per executed transaction. This not only limits the types of applications that can be deployed on chain, but also increases development and testing time (and cost).

As opposed to all existing blockchains, the IC brings modern programming to on-chain developers, allowing them to use time for creativity rather than fixing memory packing issues or spreading computation in small iterations that do not hit instruction limits. The IC programming model offers orthogonal persistence, large stack and heap spaces (4 GiB), stable storage of 400 GiB in mainstream languages, such as Rust, JavaScript, or even Python.

{| class="wikitable sortable"
! Metrics / L1 !! Stable tx cost !! HTTPs outcalls !! Smart contract language support !! Max stack size !! Max persisted memory (per smart-contract) !! Active developers (full-time / monthly) !! Active repositories
|-
| ICP || ✅ || ✅ || Motoko (native), Rust, TypeScript, Python ||data-sort-value="4294967296"| 4 GiB ||data-sort-value="55834574848"| 400 GiB || 161/ 644 || 3973
|-
| Avalanche || ❌ || ❌ || Solidity || || || 455 / 1485 || 1901
|-
| Cardano || ❌ || ❌ || Plutus (native), Haskell || || || 170 / 490 || 1252
|-
| Ethereum || ❌ || ❌ || Solidity (native), Vyper, Yul, FE || data-sort-value="32768" | 32 KiB || data-sort-value="3705346855594118253554271520278013051304639509300498049262642688253220148477952" | 2^261 B || 2392 / 7864 || 29117
|-
| Near || ❌ || ❌ || Rust, Javascript || data-sort-value="262144" | 256 KiB || data-sort-value="32768" | 32 KiB || 282 / 1137 || 5352
|-
| Solana || ❌ || ❌ || Rust C, C++ || || || 436 / 1615 || 6137
|-
|}

* '''Stable tx cost''' provides the ability to have predictable costs for computation
* '''HTTPs outcalls''' is the ability to communicate directly with Web2 services (outside of the network)
* '''Max stack size''' is the maximum size the stack can grow for smart contracts and serves as a measure for the complexity of code that is supported by each platform
* '''Max persisted memory''' is the maximum size of persisted memory supported by each platform. Persisted memory is preserved across individual function calls
* '''Active developers''' counts the number of developers who made commits on more than 10 days in a month (full-time) or original code authors who made commits in a given month. Source [https://www.developerreport.com/ Electric Capital] (31/12/23).
* '''Active repositories''' are sourced from the [https://www.electriccapital.com/ Electric Capital] [https://github.com/electric-capital/crypto-ecosystems crypto ecosystems list]. Figures true as of 15/12/22
== Comparing user experience ==
It is widely accepted that the Web3 user experience needs massive development before mainstream adoption is likely. This sections starts to map out key metrics for Web3 usability. First and foremost is privacy, identity management and authentication. On many projects, every interaction that a user ever makes can be traced and monitored. While transparency is good for some things, it's argued that this is a severe hindrance to adoption. Financial privacy and the freedom to interact should be paramount.
The tools needed to interact with a project are also noted. These can be seen as a measure of accessibility and openness to onboarding.
Finally, metrics about participation in the network are included. A large draw of Web3 is the fact that users can become owners and drivers of the platform. Here the percentage of native tokens staked as a measure of user confidence and participation in the project are included.

{| class="wikitable sortable"
|-
! Metrics / L1 !! Privacy-preserving authentication !! Prerequisites to use !! Staking ratio !! Monthly active wallets
|-
| ICP || ✅ || Browser || 73.89% || 93k
|-
| Cardano || ❌ || Browser, browser extension, tokens || 71.58%||
|-
|Avalanche || ❌ || Browser, browser extension, tokens || 61.78% || [https://stats.avax.network/dashboard/overview/ 390k]
|-
| Algorand || ❌ || Browser, browser extension, tokens || 51.17% ||
|-
| Ethereum || ❌ || Browser, browser extension, tokens || 13.57% || [https://www.theblock.co/data/on-chain-metrics/ethereum/number-of-active-addresses-on-the-ethereum-network-monthly 15m]
|-
| Near || ❌ || Browser, browser extension, tokens || 43.19% ||
|-
| Solana || ❌ || Browser, browser extension, tokens || 68.59% || [https://dune.com/queries/829800 655k]
|}

* '''Privacy-preserving authentication''' notes whether a project allows privacy-preserving interactions with the blockchain.
* '''Prerequisites to use''' lists what is needed to interact with the project
* '''Staking ratio''' gives the percentage of native tokens that are staked in the protocol. The staking ratio metrics are from [https://www.stakingrewards.com/cryptoassets/ Staking Rewards] and are correct as of 19.12.2022
* ''' Monthly active wallets''' counts the wallet addresses that sent or received native currency in a given month (December 2022)

=== A note about decentralization ===
Decentralization is key to make web3 dapps run in a trustless manner. However, decentralization has many dimensions and cannot be understood and quantified using a single number or coefficient. One can distinguish between a) the decentralization of the entities running the machines on top of which a protocol runs, b) the decentralization of the consensus and sharding mechanism, c) the governance system, the owners of liquid tokens etc. In this case, the whole is greater than the sum of its parts and one cannot understand decentralization as a single discussion on each of these topics.

See [[Decentralization|Decentralization Wiki Page]] for more details.
== References ==
* '''ICP''' : [https://dashboard.internetcomputer.org IC Dashboard]
* '''ADA''' : [https://explorer.cardano.org/en Cardano explorer] and [https://cexplorer.io/ cexplorer]
* '''AVAX''' : [https://snowtrace.io/ Snowtrace] and [https://subnets.avax.network/ Avalanche explorer]
* '''ETH''' : [https://etherscan.io/ Etherscan]
* '''NEAR''' : [https://explorer.near.org/ Near explorer] and [https://docs.near.org/ Near docs]
* '''SOL''' : [https://solana.com/ Solana website] and [https://solanabeach.io/ Solana beach]

L1 comparison

2024-10-10T11:10:17Z

Yap: Update numbers for finality and block time to today's measurements

The promise of a [[World Computer]] and the emergence of a Blockchain Singularity has far reaching consequences in technology, sociology, economics, politics, communication, entertainment, and most aspects of our digital lives. As the industry is one of rapid innovation and progress, and as projects constantly and dynamically change, it's important to take stock, every now and then, to note how the industry is doing, and to check if the Internet Computer is on track to achieve the goals of decentralization, scalability, usability, and functionality.

The industry is now moving out of its infancy, which is seen by the increasing number of smart contract developers, rather than core protocol developers, and users wanting to fully engage with a platform, rather than simply sending transactions back and forth. The shift away from simple payment systems, towards Web3 is well on its way, and it's within this scope that this page attempts to map the blockchain landscape.

Top performing blockchain projects are compared across a number of metrics that are expected to yield a 'good' Web3 experience under the categories of core protocol, developer experience, and user experience.

Unless otherwise stated, all data is correct as of February 7th 2024. Metrics are explained and references are given below.

== Base comparisons ==
This section compares standard metrics that are used to measure performance of the core protocol of popular blockchain projects. Note that these metrics should not always be taken at face value. While references are listed below to note where the figures can be found, it's not always clear how these figures are computed. Additionally, parts of different projects may have the same name, but often are constructed differently (most notably, transactions), and so should not be compared blindly like-for-like. The [https://a16zcrypto.com/why-blockchain-performance-is-hard-to-measure/ a16z blog] has a nice article describing how the industry should think about metrics.

{| class="wikitable sortable"
|-
! Metrics / L1 !! Average MIEPs !! Average TPS !! Average finality !! Average block time (seconds) !! Average tx Cost !! Average energy consumption per transaction (wh/tx) !! Size of network (nodes) !! On-Chain storage cost (1GB p/a)
|-
| ICP ||20000||[https://dashboard.internetcomputer.org/ 3200 (update calls)]||0.75secs||0.48||$0.0012||0.003||[https://dashboard.internetcomputer.org/ 559]||[https://internetcomputer.org/docs/current/developer-docs/gas-cost#storage $5.35]
|-
| Avalanche || [https://stats.avax.network/dashboard/overview/ 46]||[https://subnets.avax.network/stats/network?selectedChains=mainnet&selectedChart=tx_count&timespan=MAX 14]||unclear||2.3||[https://docs.avax.network/reference/standards/guides/txn-fees $0.035]||2.395||[https://snowtrace.io/validators 1752]||unclear
|-
| Cardano || 2||[https://cexplorer.io/tps 3]|| [https://docs.cardano.org/explore-cardano/time/#:~:text=Transaction%20finality%20can%20be%20achieved,according%20to%20Ouroboros%20consensus%20design. 1 day]||20||[https://docs.cardano.org/explore-cardano/fee-structure/ $0.1]|| 41.27 || [https://cardanoscan.io/ 2876] || [https://docs.cardano.org/cardano-testnet/tools/plutus-fee-estimator/ $1,480]
|-
| Ethereum || 5* || [https://blockchair.com/ethereum/charts/transactions-per-second 12] || [https://ethereum.org/fil/roadmap/single-slot-finality 15min] || 12 || [https://www.theblock.co/data/on-chain-metrics/comparison-bitcoin-ethereum/average-transaction-fee $8.4] || 9.956 || [https://etherscan.io/nodetracker 6395] || $2,439,827
|-
| Near || [https://nearblocks.io/charts 1600]|| [https://pikespeak.ai/near-world/overview 40] || [https://docs.near.org/concepts/advanced/near-indexer-framework#limitations 3.3secs] || 1.1 || [https://docs.near.org/concepts/basics/transactions/gas#the-cost-of-common-actions $0.00013] || 0.602 || [https://nearblocks.io/node-explorer 204] || [https://docs.near.org/concepts/storage/storage-staking#how-much-does-it-cost $1,468]
|-
| Solana || [https://solanacompass.com/ 90]|| [https://solanacompass.com 700 (non-voting)] || [https://www.tbstat.com/wp/uploads/2022/02/20220222_FinalityReport_TheBlockResearch.pdf 12secs] || 0.54 || [https://solanacompass.com/statistics/fees $0.005] || 0.517 || [https://solanacompass.com/statistics/decentralization 1617] || [https://solana.com/docs/intro/rent $35,984]
|}

* ''' Average MIEPs''' measures millions of executed instructions per second which is an approximation of useful work performed. For ICP, Avalanche and Solana the calculation follows from the reported cycles / gas / compute units used in execution. For Near, we approximate by assuming 1 Tgas corresponds to 1ms of CPU time at 2B instructions / 1s of CPU. For Cardano we give the maximum capacity corresponding to 20ms of CPU time per block at 2B instructions / 1s of CPU. For Ethereum we go by the block gas limit. (*However, the EVM is a 32-byte stack machine, so we count 1 gas as 4 CPU instructions to be generous). Further remarks can be found here: [[Not all transactions are equal|"Not all transactions are equal"]].
* '''Average TPS''' measures the transactions processed per second - note that the interval over which these are measured does vary across chains.
* '''Average finality''' refers to the amount of time that passes between the proposal of a new valid block containing transactions until the block has been finalized and its content is guaranteed to not be reversed or modified (for some blockchains, e.g., Bitcoin, this guarantee can only be probabilistic). For ICP, the reported value is the average over all subnets of their node’s average time between starting a round until a valid finalization for this round is available.
*'''Average block time''' refers to the amount of time between blocks (per subnet on the IC)
* '''Average tx Cost''' measures the cost of a transaction. Note that the definition of 'transaction' varies widely across chains. The dollar amounts are computed by converting the native token cost cycles/gas/fee needed per transaction, to USD given the exchange rate.
* '''Average energy consumption per transaction''' measures the network energy consumption to process a transaction (measured in watt hours). Figures true as of December 2023. Source: [https://assets.carboncrowd.io/reports/ICF2023.pdf Carbon Crowd Sustainability Report 2023].
* '''Size of network (nodes)''' notes the number of nodes currently validating the blockchain.
* '''On-chain storage cost''' gives the dollar cost of storing 1GB of data per year on chain. For Near and Solana, to store data one needs to maintain a specified token balance. We convert this balance to USD and annualise by multiplying by 5%. For Cardano and Ethereum, the user pays to store the data "forever", and again we annualise by multiplying this cost by 5%.

== Comparing developer experience ==
Whether they were writing games, operating systems or text editing applications, in the 70s, 80s and early 90s, developers always had to face limitations imposed by hardware. Applications were constrained to accessing a few kilobytes of memory through small stacks and heaps, using limited (and constantly changing) instruction sets, and using significant amounts of power to run instructions. The history repeats itself in the blockchain landscape these days. Application developers are limited to stack sizes of a few kilobytes to several megabytes at best. Persistent storage is expensive and limited. Programmers are bound to using cumbersome APIs that make hidden assumptions in terms of numbers of executed instructions. And, moreover, most chains operate inefficiently, burning too much power per executed transaction. This not only limits the types of applications that can be deployed on chain, but also increases development and testing time (and cost).

As opposed to all existing blockchains, the IC brings modern programming to on-chain developers, allowing them to use time for creativity rather than fixing memory packing issues or spreading computation in small iterations that do not hit instruction limits. The IC programming model offers orthogonal persistence, large stack and heap spaces (4 GiB), stable storage of 400 GiB in mainstream languages, such as Rust, JavaScript, or even Python.

{| class="wikitable sortable"
! Metrics / L1 !! Stable tx cost !! HTTPs outcalls !! Smart contract language support !! Max stack size !! Max persisted memory (per smart-contract) !! Active developers (full-time / monthly) !! Active repositories
|-
| ICP || ✅ || ✅ || Motoko (native), Rust, TypeScript, Python ||data-sort-value="4294967296"| 4 GiB ||data-sort-value="55834574848"| 400 GiB || 161/ 644 || 3973
|-
| Avalanche || ❌ || ❌ || Solidity || || || 455 / 1485 || 1901
|-
| Cardano || ❌ || ❌ || Plutus (native), Haskell || || || 170 / 490 || 1252
|-
| Ethereum || ❌ || ❌ || Solidity (native), Vyper, Yul, FE || data-sort-value="32768" | 32 KiB || data-sort-value="3705346855594118253554271520278013051304639509300498049262642688253220148477952" | 2^261 B || 2392 / 7864 || 29117
|-
| Near || ❌ || ❌ || Rust, Javascript || data-sort-value="262144" | 256 KiB || data-sort-value="32768" | 32 KiB || 282 / 1137 || 5352
|-
| Solana || ❌ || ❌ || Rust C, C++ || || || 436 / 1615 || 6137
|-
|}

* '''Stable tx cost''' provides the ability to have predictable costs for computation
* '''HTTPs outcalls''' is the ability to communicate directly with Web2 services (outside of the network)
* '''Max stack size''' is the maximum size the stack can grow for smart contracts and serves as a measure for the complexity of code that is supported by each platform
* '''Max persisted memory''' is the maximum size of persisted memory supported by each platform. Persisted memory is preserved across individual function calls
* '''Active developers''' counts the number of developers who made commits on more than 10 days in a month (full-time) or original code authors who made commits in a given month. Source [https://www.developerreport.com/ Electric Capital] (31/12/23).
* '''Active repositories''' are sourced from the [https://www.electriccapital.com/ Electric Capital] [https://github.com/electric-capital/crypto-ecosystems crypto ecosystems list]. Figures true as of 15/12/22
== Comparing user experience ==
It is widely accepted that the Web3 user experience needs massive development before mainstream adoption is likely. This sections starts to map out key metrics for Web3 usability. First and foremost is privacy, identity management and authentication. On many projects, every interaction that a user ever makes can be traced and monitored. While transparency is good for some things, it's argued that this is a severe hindrance to adoption. Financial privacy and the freedom to interact should be paramount.
The tools needed to interact with a project are also noted. These can be seen as a measure of accessibility and openness to onboarding.
Finally, metrics about participation in the network are included. A large draw of Web3 is the fact that users can become owners and drivers of the platform. Here the percentage of native tokens staked as a measure of user confidence and participation in the project are included.

{| class="wikitable sortable"
|-
! Metrics / L1 !! Privacy-preserving authentication !! Prerequisites to use !! Staking ratio !! Monthly active wallets
|-
| ICP || ✅ || Browser || 73.89% || 93k
|-
| Cardano || ❌ || Browser, browser extension, tokens || 71.58%||
|-
|Avalanche || ❌ || Browser, browser extension, tokens || 61.78% || [https://stats.avax.network/dashboard/overview/ 390k]
|-
| Algorand || ❌ || Browser, browser extension, tokens || 51.17% ||
|-
| Ethereum || ❌ || Browser, browser extension, tokens || 13.57% || [https://www.theblock.co/data/on-chain-metrics/ethereum/number-of-active-addresses-on-the-ethereum-network-monthly 15m]
|-
| Near || ❌ || Browser, browser extension, tokens || 43.19% ||
|-
| Solana || ❌ || Browser, browser extension, tokens || 68.59% || [https://dune.com/queries/829800 655k]
|}

* '''Privacy-preserving authentication''' notes whether a project allows privacy-preserving interactions with the blockchain.
* '''Prerequisites to use''' lists what is needed to interact with the project
* '''Staking ratio''' gives the percentage of native tokens that are staked in the protocol. The staking ratio metrics are from [https://www.stakingrewards.com/cryptoassets/ Staking Rewards] and are correct as of 19.12.2022
* ''' Monthly active wallets''' counts the wallet addresses that sent or received native currency in a given month (December 2022)

=== A note about decentralization ===
Decentralization is key to make web3 dapps run in a trustless manner. However, decentralization has many dimensions and cannot be understood and quantified using a single number or coefficient. One can distinguish between a) the decentralization of the entities running the machines on top of which a protocol runs, b) the decentralization of the consensus and sharding mechanism, c) the governance system, the owners of liquid tokens etc. In this case, the whole is greater than the sum of its parts and one cannot understand decentralization as a single discussion on each of these topics.

See [[Decentralization|Decentralization Wiki Page]] for more details.
== References ==
* '''ICP''' : [https://dashboard.internetcomputer.org IC Dashboard]
* '''ADA''' : [https://explorer.cardano.org/en Cardano explorer] and [https://cexplorer.io/ cexplorer]
* '''AVAX''' : [https://snowtrace.io/ Snowtrace] and [https://subnets.avax.network/ Avalanche explorer]
* '''ETH''' : [https://etherscan.io/ Etherscan]
* '''NEAR''' : [https://explorer.near.org/ Near explorer] and [https://docs.near.org/ Near docs]
* '''SOL''' : [https://solana.com/ Solana website] and [https://solanabeach.io/ Solana beach]

Not all transactions are equal

2024-09-17T17:54:08Z

Yap: Update the numbers and add links as references

Whilst it’s typical for blockchains to flaunt metrics around transactions per second (TX/s) or transactions per day (TX/d), comparisons between blockchains only make sense when those transactions are roughly equivalent i.e. TX/s comparisons only make sense to compare within a single problem domain.

The ICP is a blockchain which aims to provide a general purpose world computer, capable of serving real web services directly to user's browsers without any need for web2 cloud providers. Individual transactions on the ICP can be considered to be richer and typically do more computation than most other blockchains.

This page aims to explain the differences between the work performed by transactions on the ICP vs those on Ethereum.

== ETH vs. ICP execution throughput ==
Both ETH and ICP are able to run (general-purpose) smart contracts. At the execution layer, contracts are translated to a lower-level virtual machine interpretable language. These are EVM in the case of ETH and a Wasm-compatible runtime in the ICP case. Both EVM and Wasm instructions include arithmetic instructions (e.g., add, mul, div), but also more smart-contract specific instructions (e.g., reading and writing memory). The latter are in general more expensive operations in terms of consumed resources, which is then translated to the amount of gas used for each opcode of ETH and cycles used for ICP.

To compare the overall throughput of the two blockchains (i.e., how many ops per second can be handled), one needs to make several assumptions. First, it is assumed that the simpler EVM instructions (e.g., add, mul, div etc.) are roughly equivalent to the Wasm instructions of the same type, both kinds being translated to a similar x86 instruction executed by the hardware. The comparison is much more complex and not apples-to-apples for the more complex operations. For a proper comparison here one would need to either (1) thoroughly understand the design of both execution layers or (2) run a similar program/benchmark on both blockchains and compare their overall performance. These two options are time-consuming and would lead to longer-term research efforts.
For a quicker comparison, it can be assumed that all EVM instructions are equal in terms of gas cost (and also assume no fees are involved). Since ETH is currently burning approximately '''108.3B gas units per day''' (https://ycharts.com/indicators/ethereum_gas_used_per_day, on Sept 16 2024), and assuming each instruction costs 1 gas unit (which vastly ''underestimates'' the costs of memory access operations), it is clear that the ETH blockchain is running less than 109B instructions per day.

The IC executed more than 110 billion replicated Wasm instructions per second on Sept 16 2024. Under the simplifying assumption that all instructions are comparable, this means the IC runs the daily number of ETH instructions in less than 1 second.

Ethereum executed about about 1.102M transactions on September 16, 2024 ([https://ycharts.com/indicators/ethereum_transactions_per_day#:~:text=Ethereum%20Transactions%20Per%20Day%20is,completed%20on%20the%20Ethereum%20network. https://ycharts.com/indicators/ethereum_transactions_per_day.] ), which means that there are on average 0.098M instructions per transaction that day. On the IC, the dashboard shows that 110 billion instructions/s were executed for about 8500 replicated calls/s, amounting to an average of more than 12.9M instructions per call. Comparing the work intensity of the two blockchains, taking the ratio of instructions per transaction ICP performs around 130x the amount of computational work of Ethereum per transaction. It's important to note that the multiplier is calculated only considering replicated calls as these are the interactions that carry out ETH equivalent work.

To compare the two networks in terms of efficiency, one also needs to consider the replication factor. In ICP the typical replication factor is 13 versus approximately 1.5M for Ethereum (a number that is steadily increasing https://beaconscan.com/stat/validator). Concluding, ICP is at least 15.5 million times more efficient than Ethereum.

== ETH vs ICP EdDSA verification ==
To get a view on the validity of the above calculations in a real-world setting, comparisons can be made by running a given function. A realistic function that is used often in the blockchain setting is signature verification.

Previous work from the Ethereum Foundation estimates the validation of an EdDSA signature to cost ~500k in Gas [[https://ethresear.ch/t/verify-ed25519-signatures-cheaply-on-eth-using-zk-snarks/13139 source]]. One way to get a comparison on the IC is to create a canister, import the [https://docs.rs/ed25519/latest/ed25519/ Rust ed25519 library] and test verification by creating a signature on a hash of an arbitrary message and using that for verification. Counting cycles burned before and after this call, discounting the base cost (i.e., cycles charged for ingress and for running an update call) results in a cycle cost of 4,211,120.

Putting a dollar cost on this comparison, with a conservative assumption that 1 Gas costs 40GWEI and 1 ETH being ~USD1800 means the cost of an EdDSA verification on Ethereum currently costs USD36. Considering the cycle cost (4,211,120) on the IC with an XDR rate of USD1.3476 yields a cost of USD0.00000567490 to run an EdDSA verification on the IC. Overall, this suggests that the IC is 6,343,718 times less costly for a standard computation.

[[L1 comparison]]

L1 comparison

2024-08-27T09:42:04Z

Yap: Update the average time to finality and its explanation for ICP.

The promise of a [[World Computer]] and the emergence of a Blockchain Singularity has far reaching consequences in technology, sociology, economics, politics, communication, entertainment, and most aspects of our digital lives. As the industry is one of rapid innovation and progress, and as projects constantly and dynamically change, it's important to take stock, every now and then, to note how the industry is doing, and to check if the Internet Computer is on track to achieve the goals of decentralization, scalability, usability, and functionality.

The industry is now moving out of its infancy, which is seen by the increasing number of smart contract developers, rather than core protocol developers, and users wanting to fully engage with a platform, rather than simply sending transactions back and forth. The shift away from simple payment systems, towards Web3 is well on its way, and it's within this scope that this page attempts to map the blockchain landscape.

Top performing blockchain projects are compared across a number of metrics that are expected to yield a 'good' Web3 experience under the categories of core protocol, developer experience, and user experience.

Unless otherwise stated, all data is correct as of February 7th 2024. Metrics are explained and references are given below.

== Base comparisons ==
This section compares standard metrics that are used to measure performance of the core protocol of popular blockchain projects. Note that these metrics should not always be taken at face value. While references are listed below to note where the figures can be found, it's not always clear how these figures are computed. Additionally, parts of different projects may have the same name, but often are constructed differently (most notably, transactions), and so should not be compared blindly like-for-like. The [https://a16zcrypto.com/why-blockchain-performance-is-hard-to-measure/ a16z blog] has a nice article describing how the industry should think about metrics.

{| class="wikitable sortable"
|-
! Metrics / L1 !! Average MIEPs !! Average TPS !! Average finality !! Average block time (seconds) !! Average tx Cost !! Average energy consumption per transaction (wh/tx) !! Size of network (nodes) !! On-Chain storage cost (1GB p/a)
|-
| ICP ||20000||[https://dashboard.internetcomputer.org/ 3200 (update calls)]||0.95secs||0.9||$0.0012||0.003||[https://dashboard.internetcomputer.org/ 559]||[https://internetcomputer.org/docs/current/developer-docs/gas-cost#storage $5.35]
|-
| Avalanche || [https://stats.avax.network/dashboard/overview/ 46]||[https://subnets.avax.network/stats/network?selectedChains=mainnet&selectedChart=tx_count&timespan=MAX 14]||unclear||2.3||[https://docs.avax.network/reference/standards/guides/txn-fees $0.035]||2.395||[https://snowtrace.io/validators 1752]||unclear
|-
| Cardano || 2||[https://cexplorer.io/tps 3]|| [https://docs.cardano.org/explore-cardano/time/#:~:text=Transaction%20finality%20can%20be%20achieved,according%20to%20Ouroboros%20consensus%20design. 1 day]||20||[https://docs.cardano.org/explore-cardano/fee-structure/ $0.1]|| 41.27 || [https://cardanoscan.io/ 2876] || [https://docs.cardano.org/cardano-testnet/tools/plutus-fee-estimator/ $1,480]
|-
| Ethereum || 5* || [https://blockchair.com/ethereum/charts/transactions-per-second 12] || [https://ethereum.org/fil/roadmap/single-slot-finality 15min] || 12 || [https://www.theblock.co/data/on-chain-metrics/comparison-bitcoin-ethereum/average-transaction-fee $8.4] || 9.956 || [https://etherscan.io/nodetracker 6395] || $2,439,827
|-
| Near || [https://nearblocks.io/charts 1600]|| [https://pikespeak.ai/near-world/overview 40] || [https://docs.near.org/concepts/advanced/near-indexer-framework#limitations 3.3secs] || 1.1 || [https://docs.near.org/concepts/basics/transactions/gas#the-cost-of-common-actions $0.00013] || 0.602 || [https://nearblocks.io/node-explorer 204] || [https://docs.near.org/concepts/storage/storage-staking#how-much-does-it-cost $1,468]
|-
| Solana || [https://solanacompass.com/ 90]|| [https://solanacompass.com 700 (non-voting)] || [https://www.tbstat.com/wp/uploads/2022/02/20220222_FinalityReport_TheBlockResearch.pdf 12secs] || 0.54 || [https://solanacompass.com/statistics/fees $0.005] || 0.517 || [https://solanacompass.com/statistics/decentralization 1617] || [https://solana.com/docs/intro/rent $35,984]
|}

* ''' Average MIEPs''' measures millions of executed instructions per second which is an approximation of useful work performed. For ICP, Avalanche and Solana the calculation follows from the reported cycles / gas / compute units used in execution. For Near, we approximate by assuming 1 Tgas corresponds to 1ms of CPU time at 2B instructions / 1s of CPU. For Cardano we give the maximum capacity corresponding to 20ms of CPU time per block at 2B instructions / 1s of CPU. For Ethereum we go by the block gas limit. (*However, the EVM is a 32-byte stack machine, so we count 1 gas as 4 CPU instructions to be generous). Further remarks can be found here: [[Not all transactions are equal|"Not all transactions are equal"]].
* '''Average TPS''' measures the transactions processed per second - note that the interval over which these are measured does vary across chains.
* '''Average finality''' refers to the amount of time that passes between the proposal of a new valid block containing transactions until the block has been finalized and its content is guaranteed to not be reversed or modified (for some blockchains, e.g., Bitcoin, this guarantee can only be probabilistic). For ICP, the reported value is the average over all subnets of their node’s average time between starting a round until a valid finalization for this round is available.
*'''Average block time''' refers to the amount of time between blocks (per subnet on the IC)
* '''Average tx Cost''' measures the cost of a transaction. Note that the definition of 'transaction' varies widely across chains. The dollar amounts are computed by converting the native token cost cycles/gas/fee needed per transaction, to USD given the exchange rate.
* '''Average energy consumption per transaction''' measures the network energy consumption to process a transaction (measured in watt hours). Figures true as of December 2023. Source: [https://assets.carboncrowd.io/reports/ICF2023.pdf Carbon Crowd Sustainability Report 2023].
* '''Size of network (nodes)''' notes the number of nodes currently validating the blockchain.
* '''On-chain storage cost''' gives the dollar cost of storing 1GB of data per year on chain. For Near and Solana, to store data one needs to maintain a specified token balance. We convert this balance to USD and annualise by multiplying by 5%. For Cardano and Ethereum, the user pays to store the data "forever", and again we annualise by multiplying this cost by 5%.

== Comparing developer experience ==
Whether they were writing games, operating systems or text editing applications, in the 70s, 80s and early 90s, developers always had to face limitations imposed by hardware. Applications were constrained to accessing a few kilobytes of memory through small stacks and heaps, using limited (and constantly changing) instruction sets, and using significant amounts of power to run instructions. The history repeats itself in the blockchain landscape these days. Application developers are limited to stack sizes of a few kilobytes to several megabytes at best. Persistent storage is expensive and limited. Programmers are bound to using cumbersome APIs that make hidden assumptions in terms of numbers of executed instructions. And, moreover, most chains operate inefficiently, burning too much power per executed transaction. This not only limits the types of applications that can be deployed on chain, but also increases development and testing time (and cost).

As opposed to all existing blockchains, the IC brings modern programming to on-chain developers, allowing them to use time for creativity rather than fixing memory packing issues or spreading computation in small iterations that do not hit instruction limits. The IC programming model offers orthogonal persistence, large stack and heap spaces (4 GiB), stable storage of 400 GiB in mainstream languages, such as Rust, JavaScript, or even Python.

{| class="wikitable sortable"
! Metrics / L1 !! Stable tx cost !! HTTPs outcalls !! Smart contract language support !! Max stack size !! Max persisted memory (per smart-contract) !! Active developers (full-time / monthly) !! Active repositories
|-
| ICP || ✅ || ✅ || Motoko (native), Rust, TypeScript, Python ||data-sort-value="4294967296"| 4 GiB ||data-sort-value="55834574848"| 400 GiB || 161/ 644 || 3973
|-
| Avalanche || ❌ || ❌ || Solidity || || || 455 / 1485 || 1901
|-
| Cardano || ❌ || ❌ || Plutus (native), Haskell || || || 170 / 490 || 1252
|-
| Ethereum || ❌ || ❌ || Solidity (native), Vyper, Yul, FE || data-sort-value="32768" | 32 KiB || data-sort-value="3705346855594118253554271520278013051304639509300498049262642688253220148477952" | 2^261 B || 2392 / 7864 || 29117
|-
| Near || ❌ || ❌ || Rust, Javascript || data-sort-value="262144" | 256 KiB || data-sort-value="32768" | 32 KiB || 282 / 1137 || 5352
|-
| Solana || ❌ || ❌ || Rust C, C++ || || || 436 / 1615 || 6137
|-
|}

* '''Stable tx cost''' provides the ability to have predictable costs for computation
* '''HTTPs outcalls''' is the ability to communicate directly with Web2 services (outside of the network)
* '''Max stack size''' is the maximum size the stack can grow for smart contracts and serves as a measure for the complexity of code that is supported by each platform
* '''Max persisted memory''' is the maximum size of persisted memory supported by each platform. Persisted memory is preserved across individual function calls
* '''Active developers''' counts the number of developers who made commits on more than 10 days in a month (full-time) or original code authors who made commits in a given month. Source [https://www.developerreport.com/ Electric Capital] (31/12/23).
* '''Active repositories''' are sourced from the [https://www.electriccapital.com/ Electric Capital] [https://github.com/electric-capital/crypto-ecosystems crypto ecosystems list]. Figures true as of 15/12/22
== Comparing user experience ==
It is widely accepted that the Web3 user experience needs massive development before mainstream adoption is likely. This sections starts to map out key metrics for Web3 usability. First and foremost is privacy, identity management and authentication. On many projects, every interaction that a user ever makes can be traced and monitored. While transparency is good for some things, it's argued that this is a severe hindrance to adoption. Financial privacy and the freedom to interact should be paramount.
The tools needed to interact with a project are also noted. These can be seen as a measure of accessibility and openness to onboarding.
Finally, metrics about participation in the network are included. A large draw of Web3 is the fact that users can become owners and drivers of the platform. Here the percentage of native tokens staked as a measure of user confidence and participation in the project are included.

{| class="wikitable sortable"
|-
! Metrics / L1 !! Privacy-preserving authentication !! Prerequisites to use !! Staking ratio !! Monthly active wallets
|-
| ICP || ✅ || Browser || 73.89% || 93k
|-
| Cardano || ❌ || Browser, browser extension, tokens || 71.58%||
|-
|Avalanche || ❌ || Browser, browser extension, tokens || 61.78% || [https://stats.avax.network/dashboard/overview/ 390k]
|-
| Algorand || ❌ || Browser, browser extension, tokens || 51.17% ||
|-
| Ethereum || ❌ || Browser, browser extension, tokens || 13.57% || [https://www.theblock.co/data/on-chain-metrics/ethereum/number-of-active-addresses-on-the-ethereum-network-monthly 15m]
|-
| Near || ❌ || Browser, browser extension, tokens || 43.19% ||
|-
| Solana || ❌ || Browser, browser extension, tokens || 68.59% || [https://dune.com/queries/829800 655k]
|}

* '''Privacy-preserving authentication''' notes whether a project allows privacy-preserving interactions with the blockchain.
* '''Prerequisites to use''' lists what is needed to interact with the project
* '''Staking ratio''' gives the percentage of native tokens that are staked in the protocol. The staking ratio metrics are from [https://www.stakingrewards.com/cryptoassets/ Staking Rewards] and are correct as of 19.12.2022
* ''' Monthly active wallets''' counts the wallet addresses that sent or received native currency in a given month (December 2022)

=== A note about decentralization ===
Decentralization is key to make web3 dapps run in a trustless manner. However, decentralization has many dimensions and cannot be understood and quantified using a single number or coefficient. One can distinguish between a) the decentralization of the entities running the machines on top of which a protocol runs, b) the decentralization of the consensus and sharding mechanism, c) the governance system, the owners of liquid tokens etc. In this case, the whole is greater than the sum of its parts and one cannot understand decentralization as a single discussion on each of these topics.

See [[Decentralization|Decentralization Wiki Page]] for more details.
== References ==
* '''ICP''' : [https://dashboard.internetcomputer.org IC Dashboard]
* '''ADA''' : [https://explorer.cardano.org/en Cardano explorer] and [https://cexplorer.io/ cexplorer]
* '''AVAX''' : [https://snowtrace.io/ Snowtrace] and [https://subnets.avax.network/ Avalanche explorer]
* '''ETH''' : [https://etherscan.io/ Etherscan]
* '''NEAR''' : [https://explorer.near.org/ Near explorer] and [https://docs.near.org/ Near docs]
* '''SOL''' : [https://solana.com/ Solana website] and [https://solanabeach.io/ Solana beach]

IC Smart Contract Memory

2023-12-19T10:00:28Z

Yap: Stress the size difference in first part and make it more obvious in the figure. Remove concrete numbers from figure to avoid having to update them.

==Overall Architecture==
[[File:Memory types.png|alt=|thumb|512x512px|Figure 1. The two memories that can be accessed by the canister smart contracts.]]
Canister smart contracts running on the Internet Computer (IC) store data just like most other programs would. To this end, the IC offers developers two types of memory where data can be stored, as depicted in Figure 1. The first is the regular '''heap memory''' that is exposed as the Web Assembly virtual machine heap. This should be used as a scratch, temporary memory that will be cleared after any canister upgrade. The second type of memory is the '''stable memory''', which is a larger memory (several orders of magnitude larger than the heap) used for permanent data storage.

==Orthogonal Persistence==

<syntaxhighlight lang="rust">
use ic_cdk_macros::{query, update};
use std::{cell::RefCell, collections::HashMap};

thread_local! {
static STORE: RefCell<HashMap<String, u64>> = RefCell::default();
}

#[update]
fn insert(key: String, value: u64) {
STORE.with(|store| store.borrow_mut().insert(key, value));
}

#[query]
fn lookup(key: String) -> u64 {
STORE.with(|store| *store.borrow().get(&key).unwrap_or(&0))
}
</syntaxhighlight>

The IC offers orthogonal persistence, an illusion given to programs to run forever: the heap of each canister is automatically preserved and restored the next time it is called. For that, the execution environment needs to determine efficiently which memory pages have been dirtied during message execution so that the modified pages are tracked and periodically persisted to disk. The listing above shows an example key-value store that illustrates how easy it is to use orthogonal persistence. The key-value store in this case is backed by a simple Rust HashMap stored on the heap by means of a thread-local variable. A RefCell is used to provide interior mutability. The example would also be possible without it, but mutating the thread-local variable would be unsafe in that case, as the Rust compiler cannot guarantee exclusive access to it.

==Main Memory==
Canisters running on the IC are programmed either in Rust or Motoko. The canisters are then compiled down to web assembly (Wasm). All the variables and data structures defined in these higher-level languages are then stored in the Wasm heap. All accesses to data structures and variables defined in the higher-level languages are then translated to memory copy operations in Wasm (e.g., load, store, copy, grow). The Wasm main memory (also known as heap memory) has a maximum size of 4GiB, due to the 32-bit address space that backs the Wasm programs. The memory pages are persistent between calls to a canister (changes made by calls that throw exceptions are reverted, so these pages never enter an inconsistent state). However, they are reset when the canister's software bytecode is upgraded. Typically, canisters that need to be upgraded, serialize data in main memory to stable memory to perform upgrades. More precisely, because possible changes in data structures and in Wasm (and high-level language) compilers, the heap layout might change (i.e., data structure layouts) which could leave the canister in an unusable state when a canister is upgraded. Thus, the heap should not be used as a permanent memory, but rather as a (faster) scratch, temporary memory.

==Stable Memory==
Next to the heap memory, canister developers can make use of the stable memory. This is an additional 64-bit addressable memory, which is currently 96GiB in size, with plans to increase it further in the future. Programs written in either Rust or Motoko need to explicitly use stable memory by using the API. This API offers primitives to copy memory back and forth between the Wasm heap and the stable memory. An alternative to using this lower level API directly is to use the stable structures API, which offers developers a collection of Rust data structures (e.g., B-trees) that operate directly in stable memory. Next to using the stable memory through stable data structures, a pattern often used by developers is to persist heap state between canister upgrades. This is achieved via serializing heap memory (or data structures), saving it to stable memory and applying the opposite operations (copying back and deserializing) when the upgrade is done.

==Behind the scenes: Implementation==
To serve memory contents to canister smart contracts, the IC software stack has the following design. First, it is important to mention that every N (consensus) rounds, canister state (heap, stable memory and other data structures) are checkpointed on disk. This is called a checkpoint file. Whenever a canister executes messages after a checkpoint, all its memory resides in the checkpoint file. Therefore, all memory requested will be served from the checkpoint file. Memory modifications (i.e., dirtied pages in terms of operating systems) are saved in a data structure called the heap delta. The following paragraphs describe how this design enables orthogonal persistence.

[[File:Screen_Shot_2022-12-01_at_14.30.58.png|512px|thumb|Figure 2. The memory faulting architecture which encompasses the checkpoint file and the heap delta.
.]]

Any implementation of orthogonal persistence has to solve two problems: (1) How to map the persisted memory into the Wasm memory?; and (2) How to keep track of all modifications in the Wasm memory so that they can be persisted later. Page protection is used to solve both problems.The entire address space of the Wasm memory is divided into 4KiB pages. All pages are initially marked as inaccessible using the page protection flags of the operating system.

The first memory access triggers a page fault, pauses the execution, and invokes a signal handler. The signal handler then fetches the corresponding page from persisted memory and marks the page as read-only. Subsequent read accesses to that page will succeed without any help from the signal handler. The first write access will trigger another page fault, however, and allow the signal handler to remember the page as modified and mark the page as readable and writable. All subsequent accesses to that page (both r/w) will succeed without invoking the signal handler.

Invoking a signal handler and changing page protection flags are expensive operations. Messages that read or write large chunks of memory cause a storm of such operations, degrading performance of the whole system. This can cause severe slowdowns under heavy load.

==Versioning: Heap Delta and Checkpoint Files==

A canister executes update messages sequentially, one by one. Queries, in contrast, can run concurrently to each other and to update messages. The support for concurrent execution makes the memory implementation much more challenging. Imagine that a canister is executing an update message at (blockchain) block height H. At the same time, there could still be a previous long-running query that started earlier, at block height H-K. This means the same canister can have multiple versions of its memory active at the same time; this is used for the parallel execution of queries and update calls.

A naive solution to this problem would be to copy the entire memory after each update message. That would be slow and use too much storage. Thus, our implementation takes a different route. It keeps track of the modified memory pages in a persistent tree data-structure called Heap Delta that is based on Fast Mergeable Integer Maps. At a regular interval (i.e., every N rounds), there is a checkpoint event that commits the modified pages into the checkpoint file after cloning the file to preserve its previous version. Figure 2 shows how the Wasm memory is constructed from Heap Delta and the checkpoint file.

====Memory-related performance optimizations====
'''Optimization 1:''' Memory mapping the checkpoint file pages.
This reduces the memory usage by sharing the pages between multiple calls being executed concurrently. This optimization also improves performance by avoiding page copying on read accesses. The number of signal handler invocations remains the same as before, so the issue of signal storms is still open after this optimization.

'''Optimization 2:''' Page Tracking in Queries
All pages dirtied by a query are discarded after execution. This means that the signal handler does not have to keep track of modified pages for query calls. As opposed to update calls, queries saw the introduction of a fast path that marks pages as readable and writable on the first access. This low-hanging fruit optimization made queries 1.5x-2x faster on average.

'''Optimization 3:''' Amortized Prefetching of Pages
The idea behind the most impactful optimization is simple: to reduce the number of page faults, more work is needed per signal handler invocation. Instead of fetching a single page at a time, the signal handler tries to speculatively prefetch pages. The right balance is required here because prefetching too many pages may degrade performance of small messages that access only a few pages. The optimization computes the largest contiguous range of accessed pages immediately preceding the current page. It uses the size of the range as a hint for prefetching more pages. This way the cost of prefetching is amortized by previously accessed pages. As a result, the optimization reduces the number of page faults in memory intensive messages by an order of magnitude.

A downside of this approach is that prefetched page content needs to be compared with previous content after message execution to determine if a page was modified instead of relying on tracking write accesses via signal handlers.

These optimizations bring substantial benefits for the performance of the memory faulting component of the execution environment. The optimizations allow the IC to improve its throughput for memory-intensive workloads.

==See Also==
* '''The Internet Computer project website (hosted on the IC): [https://internetcomputer.org/ internetcomputer.org]'''

File:Screenshot 2023-12-19 at 10.56.52.png

2023-12-19T09:57:20Z

Yap:

Source https://docs.google.com/presentation/d/1HmUx5Ook019X66AOPx9FMfOu5QC9HhMsXvQ3yFmgQLg/edit#slide=id.g2a849d7da05_1_0

File:Screenshot 2023-12-19 at 10.47.13.png

2023-12-19T09:48:10Z

Yap:

Source https://docs.google.com/presentation/d/1HmUx5Ook019X66AOPx9FMfOu5QC9HhMsXvQ3yFmgQLg/

Current limitations of the Internet Computer

2023-11-16T07:48:01Z

Yap: Remove mention of team

This article contains a list of the common limitations that anyone developing on the IC should be familiar with. In some cases, it is not clear what is the best way to address the limitations and further design discussions are needed before a protocol or tool improvement can be developed.

== Bugs in pre_upgrade hooks ==
If there is a bug in your pre_upgrade hook that causes it to panic, then the canister can no longer be upgraded. This is because the pre_upgrade hook is part of the currently deployed wasm module and the system will always execute it before deploying the new wasm module and if the pre_upgrade hook fails, then the system will fail the whole upgrade.

Currently there is no good mitigation around this issue other than urging developers to make sure that their pre_upgrade is bug free by doing a lot of testing.

== Long running upgrades ==
Generally speaking, when a canister is being upgraded, the logic in the pre_upgrade hook serialises state from the wasm heap to stable memory and the logic in the post_upgrade hook deserialises it from stable memory back to wasm heap. There is an instructions bound on how long the upgrade process can run for. So it is possible that if the canister has too much state or the [de]serialising logic is not very efficient, then the whole process does not finish in time.

The recommended mitigation here is to ensure that the state that needs to be persisted across upgrades does not exceed what the canister can [de]serialise during the upgrade process.

== [de]serialiser requiring additional wasm memory==
[https://github.com/dfinity/motoko/issues/2909 Related issue in Motoko]. Generally speaking, it is possible that the serialising logic requires some additional wasm heap to run. Let’s say that the canister has 3.5GiB of wasm heap and the serialising logic requires an additional 600MiB to serialise the data, given that the wasm heap is limited to 4GiB, the upgrade process will again fail. Note that this issue will also be present for canisters written in Rust.

The recommended mitigation here is to again ensure that the state that needs to be persisted across upgrades does not exceed what the canister can [de]serialise during the upgrade process.

== Only upgrading stopped canisters ==
Generally speaking, it is only safe to upgrade stopped canisters i.e. canisters that do not have any outstanding responses. This is because it is possible that the new wasm module will not be compatible with the response that refers to an earlier state of the canister and could corrupt the current state of the canister. The current implementation of the IC allows canisters that are not stopped to be upgraded (with the assumption being that the canister developer has taken sufficient precautions to ensure the above mentioned corruption cannot happen).

== Calling potentially malicious or buggy canisters can prevent canisters from upgrading ==
If a canister can only be safely upgraded when it is stopped, it can run into the follow issue. If a canister `A` sends a `Request` to another canister `B` and `B` is buggy or malicious, it can create a situation where it does not send a `Response` back to `A` for arbitrarily long time. `B` can keep the call context alive if it keeps sending out `Request`s of its own. As long as `A` has an outstanding `Response`, upgrading it may not be safe.

== Loops in call graphs ==
The current implementation of the IC allows canisters to be called in a loop. For example, A can call B, B can call C, and C can call A. There can be subtle issues when one attempts to do canister management in a loop though. If A sends a request (Req1) to B, while processing Req1 from A, B sends a StopCanister message to stop A, the canisters will effectively deadlock. When the StopCanister message is processed, A will be put in the Stopping state. It will only fully stop, when it has received a response for Req1. It will not receive a response for Req1 till B gets a response for its StopCanister message, and B will not get that response till A is stopped.

IC execution layer

2023-11-16T07:16:33Z

Yap:

==== Overview ====
The execution layer is in charge of executing canister smart contracts. The Internet Computer works in rounds, which are triggered by a consensus layer agreeing on a block of messages. During each round, the replicated subnetwork state is processed, and execution ends once a certain instruction limit has been reached.

At the start of a round, the messages in a block are placed into input queues for their respective destination canisters. Messages directed to the subnetwork (these are messages to the management canister) are placed into a subnetwork input queue. The scheduler then orders these messages for execution.

To optimize throughput, the scheduler uses canister-level scheduling. When a canister is scheduled for execution, an available CPU core is allocated, and the messages in the canister's input queue are executed one by one until all messages have been processed. The scheduler then selects the next canister for execution, continuing this process until the instruction round limit is reached, or no more canisters can be scheduled. Each canister executes in isolation, enabling parallel processing.

To cover resource usage costs, each canister holds a local balance of tokens called cycles. The execution environment monitors resource usage and deducts the corresponding cycles from the canister's balance.

=== Scheduler ===
The main requirements of the scheduler are (1) it must be deterministic (2) it should distribute workloads fairly among canisters (3) optimizing for throughput over latency.

To ensure responsiveness under heavy load, canisters have the option of paying upfront for a compute allocation. Since canisters are single threaded, a compute allocation is a fraction of one CPU core, expressed in percentage points. Only part of a subnet’s compute capacity can be allocated, ensuring progress for canisters with zero compute allocation, i.e. best effort canisters. Fairness is defined as guaranteeing canister compute allocations (i.e., a backlogged canister with compute allocation A executing at least A full rounds out of every 100) and evenly distributing the remaining capacity ("free compute") across all canisters. Given a deterministic state machine with N CPU cores (and N × 100 compute capacity), the scheduler selects (at least) N canisters to execute a full round: a round, in which a canister either exhaust the instruction limit or completes the execution of all their enqueued messages. The scheduling algorithm uses credits accumulated across rounds as priority: an amount of credits equal to the canister’s compute allocation plus a uniform share of the free compute is credited to every canister at the beginning of every round; canisters in the priority queue are assigned round-robin to CPU cores (each of the first N canisters are scheduled first on a CPU core), and 100 credits are debited from each canister that executes a full round.

=== Message execution ===

==== Single canister message execution ====
For security and reliability, each canister is executed within a sandboxed environment that isolates it from other canisters and the rest of the data and computation taking place on the node. For executing each individual message, the scheduler brings up the sandbox process hosting the canister and executes the canister on the provided message. If the sandbox does not yet exist, it compiles the canister, creates the sandbox which contains a Wasm runtime and loads the canister compiled code. The message is then executed by invoking the runtime. Each message execution can lead to new messages to other canisters, memory pages of the canister's state being modified, and/or a response to be generated. If the response corresponds to an ingress message then the response is written to the ingress history from where it can be read by the sender of the ingress message. This information, together with the number of instructions consumed are returned to the execution environment for bookkeeping purposes.

Bookkeeping associated with requests is maintained in a call context associated with that request. This is a data structure which keeps track of information related to the call, among other things, where it originates, whether the call has been answered, etc. Calls to other canisters are recorded in the call context and placed in the output queues of the canister. If a response is generated this is placed in the ingress history if it is a response to an ingress message; the call context records that the call had been answered and the response is stored in the ingress history for some fixed amount of time.

Executing a response is similar to executing a request: the difference is that the response is executed within the call context of the request that triggered it. The other steps are as above.

==== Heartbeat and timer ====
Canisters have the ability to schedule tasks at regular intervals by setting up heartbeat or time methods. Internally, this is implemented by placing messages in a canister specific task queue. When a canister is scheduled for execution the scheduler selects in a round robin fashion if it should execute a task or a message.

==== Management canister messages ====
Several types of operations on the Internet Computer are sent to and executed by the so-called management canister. This is not a canister per se: the messages sent to the management canister are directed to the relevant subnetwork (see below for some examples) and are intercepted by the execution environment which triggers their execution.

Technically this is implemented via two subnetworks specific queues, one for input messages and another for output messages. Messages in these queues are prioritized for execution at the beginning of each round.

===== Managing canisters =====
For some of these messages, the execution is entirely confined to the execution environment. This is the case of messages that manage canisters (e.g. messages for creation, updating settings, starting, stopping canisters). When issued, these messages are routed to the subnetwork hosting the corresponding canister (for canister creation, the correct subnetwork is decided out of band) and included in the subnetwork queue. At the beginning of each round, the scheduler selects and invokes, sequentially, the execution environment on a number of these messages. The execution results in either global replicated state changes (i.e. when a new canister is created) or changes local to some canister (i.e. installing code, or updating the canister settings). A response, e.g. indicating the status of the execution result is produced and routed back to the issuer of the canister management message.

===== ECDSA messages =====
Other management canister messages, like those for ECDSA operations are executed by involving other components of the Internet Computer software stack. Such messages are routed to a special subnetwork (the subnetwork that holds, in a threshold manner, the ECDSA master key) and enqueued on that subnetworks queue. When scheduled for execution, they are picked up by the execution environment which in turns involves the consensus layer which then directs the replicas in a subnetwork to generate the signature shares, gathers the shares and constructs the relevant signature which is then returned to the subnetwork outqueue.

===== Bitcoin messages =====
These messages are addressed to the bitcoin canister; they are rerouted to the subnetwork hosting the bitcoin canister and enqueued in the input queue of that canister. The response provided is routed directly to the canister that requested that made the request.

===== Randomness generation =====
Canisters can send requests to the management request to get random bits from the IC. To answer these requests, the execution environment uses random bits generated by the IC, every round via a digital signature scheme. Specifically, every round each subnetwork on the IC produces a BLS signature. Although deterministically generated, the bits of the signature are unpredictable until the signature is produced. The entropy in this signature is extracted and used as a seed to a pseudorandom generator to generate pseudorandom bits fed as answers to randomness requests issued by canisters in the previous round.

=== Execution features and bounds ===

==== Fast-tracked executions ====
Messages between canisters that live on different subnetworks require at least two sequential rounds of consensus: one is the round which includes the messages into blocks on the destination subnetwork, and one which includes replies into blocks on the source subnetwork.

The execution environment implements fast tracking for messages between canisters that are hosted by the same subnetwork. A new canister message targeted at a canister in the local subnet, is queued up directly in the input queue of the target canister and scheduled in the same round or an upcoming round. This message does not need to go through consensus since the generation and enqueuing of the new message is completely deterministic and thus happens in exactly the same way on all the nodes of the subnet. If the instruction limit on the round permits, the message may get to be executed in the same round.

==== Deterministic time slicing ====
While the execution of a round is bounded by a fixed number of instructions, the execution of any individual call can span multiple rounds. Technically, this is implemented by pausing the execution in each round if the round instructions have been used, and picking up the execution from where it left off whenever the canister is scheduled next (typically in the next round). To avoid that a canister takes over a CPU core there is a bound on the maximum number of instructions any single call to a canister can use. If a canister attempts to execute more instructions than permitted, the runtime stops the execution; the canister state is rolled back and the cycles charged for execution are forfeited.

==== Heap deltas bound ====
The execution environment also imposes a limit on the number of heap pages that a canister can change during a single round. This bound is soft, in that if a canister goes above the limit, the result of the execution is still committed to the state. However, future executions of the canister are skipped until the amortized number of heap changes across the rounds when the canister is scheduled dips below the limit.

=== Cycle Charging ===
During execution, canisters consume resources such as CPU, network bandwidth, and memory usage, which are charged for with cycles. Each canister holds a local cycles account to pay for its resource usage on the Internet Computer. It is important to note that costs of resources scale with the replication factor of the subnet, which is the number of nodes powering the subnet.

''Execution Charging'': When installed, the Wasm code in a canister gets instrumented with code that counts the executed instructions for smart contract messages. This allows for the exact amount of cycles to be charged for each executed message to be deterministically computed.

Right before a message is scheduled for execution, an amount of cycles that corresponds to running a maximal number of instructions is withdrawn from the cycle balance of the canister. Execution proceeds – the instrumentation returns the total number of instructions executed. If this goes above the maximum instructions permitted, the canister traps and the cycles withdrawn are forfeited. If the canister executes fewer instructions than maximum available, the cycles corresponding to the instructions that were not used are returned to the canister balance.

''Call Charging'': Canisters are charged for the bandwidth consumed when sending messages to other canisters. The cost of message transmission is proportional with the size of the message that is sent, and is therefore capped, since messages on the Internet Computer are themselves capped in size.

When a canister makes a call to another canister, the execution environment deducts cycles from the caller's account to cover both the cost of the outgoing call and that of the potential reply that the callee will send. Since the size of the reply is not known a priori, cycles deducted cover a maximal size reply, with excess cycles returned to the calling canister if the reply is shorter.

''Memory Charging'': The memory used by a canister, including its Wasm code and state, is also charged for with cycles. The execution environment tracks the memory usage across multiple rounds and charges periodically for this usage.

''Freezing threshold'': To prevent a canister suddenly running out of cycles and losing all of its stored data, the Internet Computer allows the canister owner to define a so-called freezing threshold. When the cycle account of a canister dips below this threshold, the canister stops performing any new computation, that is it rejects all incoming requests. It still executes replies (whenever a request is made, cycles for the execution of the corresponding reply are set aside) and only pays for memory consumption.

Furthermore, withdrawing cycles for execution or for message transmission does not occur if withdrawing would take the cycle account below the freezing threshold. In this case the canister would not be trying to perform the corresponding action.

Roll-over of NNS voting rewards

2023-11-16T07:08:28Z

Yap:

==Background & goal==
* The automation of exchange rate proposals is planned to be released in May ‘23.
* Post this release, on some days (in particular on weekends) no NNS proposals might be submitted. Previously this was not possible as exchange rate proposals were submitted every 10 minutes.
* As a consequence, there might be days during which no NNS proposals will settle. Since the distribution of NNS rewards is linked to the ballots of settled proposals, on those days no rewards are distributed. Rather, rewards will roll over to the following day.
* On this page the roll-over mechanism is explained.

==Reward roll-over mechanism==
* For any given day t_0 the NNS determines the total amount of available rewards R(t_0) for that day as a function of total supply and the voting reward function. For further background information on voting rewards see [https://internetcomputer.org/docs/current/tokenomics/nns/nns-staking-voting-rewards/#voting-rewards here].
* Now assume that on day t_0, no proposal settled and thus no rewards could be distributed.
* As a consequence, the NNS will roll over the rewards to the following day t_1. This means that on day t_1 a total amount of R(t_0) + R(t_1) is available for distribution.
* If at least one proposal settled on day t_1 then this triggers a reward distribution. Otherwise rewards will again roll over to the following day and so on.

==How this will be displayed in the NNS dapp==
The NNS dapp will provide users information about the last time voting rewards were distributed, as you can see from the below screenshot.

[[File:NNS dapp roll-over.png|800px|center]]

If the indicated date is more than one day in the past, then NNS rewards are currently rolling over.

Daemon canisters

2023-11-16T07:06:26Z

Yap: Add links

'''On the Internet Computer blockchain, you can create canister smart contracts that run like daemon processes — that is, you can configure them so that they are automatically activated by the network itself at specified block intervals.'''

When smart contracts are hosted on traditional blockchain networks, computations can be only be invoked by submitting a new transaction to their networks. This means that if, say, a DeFi (decentralized finance) smart contract needs to periodically perform some action, such as recording the latest asset prices published by [[DEX]]s (decentralized exchanges), a traditional off-chain system such as software running on a centralized [[cloud service]] must be configured to periodically submit transactions.

The approach that must be used with traditional blockchains is complex, fault prone, and introduces several problems native to centralization. For example, who will be responsible for running the centralized infrastructure, and would such a person become a de facto "controller" or "owner" of an otherwise decentralized financial system in the eyes of financial regulator?

The Internet Computer provides a means to avoid such problems, by allowing canister smart contracts to be configured so that they aer invoked by the blockchain itself, at some specified block interval.

Learn more here:

https://internetcomputer.org/docs/current/developer-docs/backend/periodic-tasks/

https://medium.com/dfinity/internet-computer-canister-timers-21cd3201b831

Not all transactions are equal

2023-11-16T07:02:55Z

Yap: clarification

Whilst it’s typical for blockchains to flaunt metrics around transactions per second (TX/s) or transactions per day (TX/d), comparisons between blockchains only make sense when those transactions are roughly equivalent i.e. TX/s comparisons only make sense to compare within a single problem domain.

The ICP is a blockchain which aims to provide a general purpose world computer, capable of serving real web services directly to user's browsers without any need for web2 cloud providers. Individual transactions on the ICP can be considered to be richer and typically do more computation than most other blockchains.

This page aims to explain the differences between the work performed by transactions on the ICP vs those on Ethereum.

== ETH vs. ICP execution throughput ==
Both ETH and ICP are able to run (general-purpose) smart contracts. At the execution layer, contracts are translated to a lower-level virtual machine interpretable language. These are EVM in the case of ETH and a Wasm-compatible runtime in the ICP case. Both EVM and Wasm instructions include arithmetic instructions (e.g., add, mul, div), but also more smart-contract specific instructions (e.g., reading and writing memory). The latter are in general more expensive operations in terms of consumed resources, which is then translated to the amount of gas used for each opcode of ETH and cycles used for ICP.

To compare the overall throughput of the two blockchains (i.e., how many ops per second can be handled), one needs to make several assumptions. First, it is assumed that the simpler EVM instructions (e.g., add, mul, div etc.) are roughly equivalent to the Wasm instructions of the same type, both kinds being translated to a similar x86 instruction executed by the hardware. The comparison is much more complex and not apples-to-apples for the more complex operations. For a proper comparison here one would need to either (1) thoroughly understand the design of both execution layers or (2) run a similar program/benchmark on both blockchains and compare their overall performance. These two options are time-consuming and would lead to longer-term research efforts.
For a quicker comparison, it can be assumed that all EVM instructions are equal in terms of gas cost (and also assume no fees are involved). Since ETH is currently burning approximately '''108B gas units per day''', and assuming each instruction costs 1 gas unit (which vastly ''underestimates'' the costs of memory access operations), it is clear that the ETH blockchain is running less than 108B instructions per day.

The IC executes more than 20 billion replicated Wasm instructions per second. Under the simplifying assumption that all instructions are comparable, this means the IC runs the daily number of ETH instructions in less than 6 seconds.

To see this, consider that Ethereum executes 1,250,000 instructions and 15 transactions per second, which means that there are on average 83,333 instructions per transaction. While the IC executes 20 Billion instructions and 3,000 update calls per second, there are on average 6,666,667 instructions per call. Comparing the work intensity of the two blockchains, taking the ratio of instructions per transaction (6,666,667/83,333) ICP performs 80x the amount of computational work of Ethereum per transaction. It's important to note that the multiplier is calculated only considering update calls as these are the interactions that carry out ETH equivalent work.

To compare the two networks in terms of efficiency, one also needs to consider the replication factor. In ICP the typical replication factor is 13 versus approximately 550'000 for Ethereum (a number that is steadily increasing and rose by approximately 100'000 over the last 6 months).
Concluding, ICP is at least 3.4 million times more efficient than Ethereum.

== ETH vs ICP EdDSA verification ==
To get a view on the validity of the above calculations in a real-world setting, comparisons can be made by running a given function. A realistic function that is used often in the blockchain setting is signature verification.

Previous work from the Ethereum Foundation estimates the validation of an EdDSA signature to cost ~500k in Gas [[https://ethresear.ch/t/verify-ed25519-signatures-cheaply-on-eth-using-zk-snarks/13139 source]]. One way to get a comparison on the IC is to create a canister, import the [https://docs.rs/ed25519/latest/ed25519/ Rust ed25519 library] and test verification by creating a signature on a hash of an arbitrary message and using that for verification. Counting cycles burned before and after this call, discounting the base cost (i.e., cycles charged for ingress and for running an update call) results in a cycle cost of 4,211,120.

Putting a dollar cost on this comparison, with a conservative assumption that 1 Gas costs 40GWEI and 1 ETH being ~USD1800 means the cost of an EdDSA verification on Ethereum currently costs USD36. Considering the cycle cost (4,211,120) on the IC with an XDR rate of USD1.3476 yields a cost of USD0.00000567490 to run an EdDSA verification on the IC. Overall, this suggests that the IC is 6,343,718 times less costly for a standard computation.

[[L1 comparison]]

Bitcoin Integration

2023-11-16T06:59:24Z

Yap: remove jargon

== Overview ==

[https://en.wikipedia.org/wiki/Bitcoin Bitcoin] is a decentralized digital currency based on an open-source [https://en.wikipedia.org/wiki/Bitcoin_network P2P protocol]. Bitcoin uses the [https://en.wikipedia.org/wiki/Unspent_transaction_output unspent transaction output] (UTXO) accounting model, i.e., transactions create outputs which are fully consumed when used as inputs for other transactions that create new transaction outputs.

Bitcoin is a payment network without support for [https://en.wikipedia.org/wiki/Smart_contract smart contracts]. Smart contract support for Bitcoin through the Internet Computer adds tremendous value: It leverages the combined strength of the Bitcoin network as the world's digital gold reserve and the Internet Computer as a platform for securely and efficiently executing smart contracts. An example class of applications is decentralized finance (DeFi) built around Bitcoin, which can currently be implemented only with wrapped Bitcoin requiring additional trusted entities. Moreover, Bitcoin can be used to pay for any kind of services on the Internet Computer, which opens up a sheer endless number of application scenarios.

The Internet Computer is integrated with the Bitcoin blockchain to make powerful smart contract functionality available for Bitcoin through a direct, bridgeless, integration of the two blockchains. "Direct" in this context means that no trust assumptions are required other than trust in the correct functioning of the Bitcoin network and the Internet Computer. In other words, there are no additional parties required, such as bridges or other types of intermediaries, resulting in a much cleaner and more secure integration.

This direct integration has the following main features:

* Canisters can have Bitcoin addresses (and therefore receive and hold bitcoin directly on the Bitcoin blockchain).
* Canisters can access the balance and UTXO set of Bitcoin addresses.
* Canisters can securely sign Bitcoin transactions.
* Canisters can submit Bitcoin transactions to the Bitcoin network.

The direct, bridgeless, integration relies on a novel [https://internetcomputer.org/docs/current/developer-docs/integrations/t-ecdsa threshold ECDSA protocol] that enables a subnet to compute ECDSA signatures based on a secret-shared private key upon a canister's request. With this protocol, each canister can "control" a vast number of derivable ECDSA keys and obtain signatures for them, making it possible for canisters to receive, hold, and transfer bitcoin directly on the Bitcoin blockchain in a secure manner. Naturally, a canister must be able to retrieve the UTXOs associated with its Bitcoin addresses. To this end, the Internet Computer pulls in blocks directly from the Bitcoin network. In the following, the underlying architecture of this integration is explored in more detail.

== Architecture ==

This section presents the architecture for the direct integration with the Bitcoin blockchain on a high level. The integration changes and adds components at every layer of the Internet Computer (IC) protocol stack.

[[File:Updated Bitcoin integration architecture overview.png|thumb|center|upright=3|A high-level overview of the Bitcoin integration architecture.]]

The main component is the Bitcoin canister, which runs on a system subnet and exposes the Bitcoin integration endpoints through the [https://internetcomputer.org/docs/current/references/ic-interface-spec/#ic-management-canister virtual management canister] interface.
The API offers functions to query Bitcoin state (UTXOs, balances, and current transaction fees) and to submit transactions.

In order to serve up-to-date information about the Bitcoin state, Bitcoin blocks must be pulled continuously into the Internet Computer from the Bitcoin network and the set of all UTXOs (referred to as the "UTXO set") updated based on the transaction inputs and outputs contained in the blocks.

The Bitcoin canister maintains a set of recent Bitcoin blocks and the entire UTXO set, which are used together to respond to UTXO and balance queries of canisters.

The ''Bitcoin adapter'', an OS-level process external to the replica, connects to multiple Bitcoin nodes chosen randomly using Bitcoin's peer discovery protocol. The Bitcoin adapter maintains a local copy of the whole block header chain as well as a small cache of blocks that it provides to the replica upon request.

In every IC round, the block maker, a core component of the consensus layer, sends a request from the Bitcoin canister to the Bitcoin adapter and gets the response, which is then included in the IC block's payload. More precisely, the Bitcoin canister makes a call to access internal replica APIs to store the request in the replicated state, where it is picked up by the payload builder in the Consensus layer. The payload builder uses interprocess communication to relay the request to the external Bitcoin adapter, which handles the request and returns the result to the payload builder. The payload builder then incorporates the result in the IC block.
A request contains hashes of recent block headers for which the Bitcoin canister has the block of transactions as well as the outgoing transactions. The Bitcoin adapter compares the received block hashes against its view of the Bitcoin network: if its cache contains one or more successors of the blocks that the Bitcoin canister has, it returns one or more of these blocks. Furthermore, it adds the transactions received in the request to a queue for outgoing transactions to be submitted asynchronously to the Bitcoin network.

Every IC block is broadcast to the subnet nodes and needs to go through the notarization and finalization process. The notarization process is extended for the Bitcoin integration: Each replica performs a deterministic validity check of the Bitcoin payload contained in the IC block. It is crucial that the block maker propose a block only if it is guaranteed that the block will be successfully validated by all honest replicas as otherwise consensus may not be reached, causing the entire IC block to be dropped.

Once the IC block with the Bitcoin payload has been validated successfully, finalization proceeds without changes. Once the block is finalized, the Bitcoin payload needs to be extracted in the message routing layer and posted to the correct subnet queue for execution. When the Bitcoin payload reaches the execution layer, it gets sent to the Bitcoin canister where the payload is validated and the state of the Bitcoin canister is updated accordingly.

Creating a Bitcoin transaction requires computing one ECDSA signature per UTXO used as transaction input. Canisters can request ECDSA signatures through the threshold ECDSA API that is implemented as part of dedicated chain-key signing subnets. There is one such subnet deployed and, if demand increases, multiple signing subnets may be made available in the future. Figure 1 shows the threshold ECDSA functionality in a simplified manner as part of the Bitcoin-enabled subnet instead of being located on a separate subnet. The threshold ECDSA API makes it possible for a canister to request an ECDSA signature computed jointly by the replicas of the ECDSA subnet based on a secret-shared private key.

== Technical Details ==

As described in the previous section, canisters interact with the Bitcoin canister to retrieve information about the Bitcoin state and send transactions. The Bitcoin canister in turn depends on the Bitcoin adapter, which is the component that interacts with the Bitcoin network. In this section, technical details about the individual components are provided, starting bottom-up with the Bitcoin adapter.

=== Bitcoin Adapter ===

The Bitcoin adapter interacts with the Bitcoin network to obtain block headers and blocks, and publish Bitcoin transactions issued by canisters.

==== Connecting to Bitcoin Nodes ====

By default, the Bitcoin adapter connects to 5 randomly chosen Bitcoin nodes but the number of connections is configurable. In order to ensure that the Bitcoin adapter of each replica connects to a different random set with high probability, the Bitcoin adapter queries the Bitcoin nodes for addresses until it has received 2000 addresses and randomly chooses nodes from these addresses until 5 connections have been established. Experiments showed that this process results in the Bitcoin adapter of each replica connecting to mostly different addresses.

==== State ====

The Bitcoin adapter maintains the following state:
* All Bitcoin block headers.
* A cache for Bitcoin blocks, which it expects the Bitcoin canister to request next.
* A cache for outgoing transactions that are advertised but not transmitted to Bitcoin nodes yet.

Initially, the adapter only has the hard-coded genesis block header and the caches are empty.

==== Bitcoin Adapter in Operation ====

When the Bitcoin adapter is started, it connects to Bitcoin nodes and starts pulling in block headers until it is fully synced. For each downloaded block header, the Bitcoin adapter performs the following checks:

* The block header is well-formed, i.e., it can be parsed as a correct block header.
* The previous block field points to a locally available block header.
* The hash work in the block header is sufficient based on the difficulty target.
* The timestamp in the block header is greater than the [https://en.bitcoin.it/wiki/Block_timestamp median of the 11 previous blocks].

The caches remain empty until requests from the Bitcoin canister are received. Periodically, the Bitcoin canister sends a list of block header hashes of the recent block headers for which it already has the full blocks (containing all transactions). Details on which block header hashes are sent are provided in the section about the Bitcoin canister.

The Bitcoin adapter checks if it has any blocks that the Bitcoin canister is missing and responds with a message containing the missing blocks, prioritizing blocks with a lower height. Multiple blocks can be returned in a single message up to a soft cap of 2 MB, ensuring that at least one Bitcoin block can be returned even if its size exceeds 2 MB. This upper bound implies that only one block is returned for most of the recent blocks whose size is typically over 1 MB. The ability to send multiple blocks in one message is advantageous mainly for the Bitcoin testnet where blocks are usually significantly smaller. In addition to the Bitcoin block(s), the adapter also appends up to 100 block headers of subsequent blocks to its response. The purpose of these block headers is explained in the section about the Bitcoin canister.

If the Bitcoin adapter does not have the missing blocks in its cache, it immediately returns an empty message to prevent an unnecessary delay in the Consensus layer. Having an accurate view on the state of the Bitcoin canister, the Bitcoin adapter then downloads the next missing blocks so that it can put them into a return message in a future request from the Bitcoin canister. Since the Bitcoin adapter does not keep track of transactions, it cannot verify the validity of transactions in received blocks. However, in order to prevent spamming the Bitcoin canister with invalid blocks, it performs a few basic checks:
* The block is well-formed, i.e., it can be parsed as a correct Bitcoin block.
* The Merkle tree root hash corresponds to the hash in the corresponding block header.

Blocks are dropped from the cache as soon as the received set of block header hashes indicates that the Bitcoin canister has received the blocks.

When the Bitcoin adapter receives outbound transactions, they are placed in the transaction cache and advertised to the Bitcoin network. The transactions are transmitted to the connected Bitcoin peers upon request and removed from the cache after 10 minutes. There is no guarantee that the transactions are ever requested by connected peers.

=== Bitcoin Canister ===

As mentioned above, the Bitcoin canister is the main component as it provides access to the Bitcoin integration API.

==== State ====

For a certain block height h, defined below, the Bitcoin canister stores the following state:
* The full UTXO set from genesis up to block height h.
* The balances of all addresses in the UTXO set.
* Blocks including their block headers starting from block height h+1.
* The full history of block headers of ''stable blocks'' (the notion of stability is defined below).

Since the Bitcoin canister does not store the full history of transactions, it must decide when it is safe to drop a block, relying only on the information in the UTXO set that it maintains.

==== Fork Resolution ====

The Bitcoin canister uses [https://en.bitcoinwiki.org/wiki/Difficulty_in_Mining block difficulty] for fork resolution to determine the "correct" chain just like Bitcoin. Concretely, the chain with the largest sum of block difficulties is considered the "correct" chain.

Due to the risk of long-running forks, the Bitcoin canister further uses a concept called ''stability'' to determine which blocks can be dropped and to count confirmations for transactions. Simply put, a block is considered ''stable'' if there is a certain number of subsequent blocks, reducing the risk that the block will be discarded in the future, and the tip of any fork is also at least the same number of blocks behind the tip of the chain containing the block.

More formally, if <code>h(b)</code> denotes the number of predecessor blocks of block <code>b</code> since genesis and <code>d(b)</code> denotes the length of the longest successor path starting at and including block <code>b</code>, stability is defined as follows.

Definition (𝜹-stability): Let B denote the set of locally available blocks.
For a parameter 𝜹>0, it is said that a block b is 𝜹-stable if the following conditions hold:
* d(b) ≥ 𝜹
* ∀ b’ ∈ B \ {b}, h(b’) = h(b): d(b) - d(b’) ≥ 𝜹

Note that the genesis block is always considered stable. Given the latest stable block and a set of unstable blocks, a generalization of 𝜹-stability, dubbed difficulty-based 𝜹-stability, is used to determine when an unstable block becomes stable. Thus, difficulty-based stability is used to define the cut-off block height h mentioned in the description of the state above: h is the largest height with a difficulty-based 𝜹-stable block.

The difference to 𝜹-stability is its use of a difficulty-based depth function <code>dh(b)</code>: The depth <code>dh(b)</code> is the sum of the difficulty of block <code>b</code> itself and all successor blocks, divided by the difficulty of the latest stable block.

Note that if all blocks have the same difficulty, then <code>d(b) = dh(b)</code> for every block <code>b</code>.

The Bitcoin canister is configured with a ''difficulty-based stability threshold'' 𝜹 of 144, i.e., if there are no forks, the Bitcoin canister keeps all blocks around for about one day (at one block every 10 minutes on average). Once a block becomes "difficulty-based 144-stable", the transactions are applied to the UTXO set and the block is discarded.

==== Bitcoin Canister in Operation ====

When requesting an update from the Bitcoin adapter, the Bitcoin canister sends a message containing the hashes of all ''unstable blocks'' to the Bitcoin adapter, which uses the hashes to determine which blocks the Bitcoin canister is missing, if any. When receiving blocks and block headers from the Bitcoin adapter, the following validity checks are performed for each block/block header pair:
* The block header is well-formed, i.e., it can be parsed as a correct block header.
* The hash work in the block header is sufficient based on the difficulty target.
* The block header points to a known predecessor.
* The timestamp in the block header is greater than the median of the 11 previous blocks.
* The timestamp in the block header is at most two hours in the future with respect to IC time.
* The block is well-formed, i.e., it can be parsed as a correct Bitcoin block.
* The Merkle tree root hash corresponds to the hash in the corresponding block header.

It is important to note that the validity of transactions is '''not''' verified in the Bitcoin canister. The Bitcoin canister relies on the proof of work that goes into the blocks and the verification of the blocks in the Bitcoin network. For a newly discovered block, a regular Bitcoin (full) node therefore provides a higher level of security than the Bitcoin canister, which implies that it is advisable to set the number of confirmations to a reasonably large value to gain confidence in the correctness of the information provided by the Bitcoin canister.

If the verification is successful, the block is added to the list of unstable blocks. Additionally, if a block becomes (difficulty-based 144-)stable as a result, the UTXO set is updated based on the transaction in this block and the block is discarded, keeping only the corresponding block header.

As mentioned above, the Bitcoin canister may also receive block headers of missing blocks from the Bitcoin adapter. The Bitcoin canister uses this information to determine if it is currently in syncing mode or fully synced with respect to the Bitcoin adapters' views of the Bitcoin blockchain. Concretely, the Bitcoin canister is in the state “synced” if the difference between the maximum height of all block headers and the maximum height of all unstable blocks is at most 2. Otherwise, the canister is in the state “syncing”. The Bitcoin canister only acts on requests if it is in the state “synced”, i.e., all requests are rejected in the state "syncing". This mechanism is meant to ensure that the Bitcoin canister only provides up-to-date information about the Bitcoin blockchain state to prevent Bitcoin dapps from making decisions based on outdated information.

The (virtual) management canister exposes the following Bitcoin integration API:
* bitcoin_get_utxos: The function returns the unspent transaction outputs (UTXOs) of a given Bitcoin address.
* bitcoin_get_balance: The function returns the balance of a given Bitcoin address.
* bitcoin_send_transaction: The function sends the given transaction to the Bitcoin network.
* bitcoin_get_current_fee_percentiles: The function returns the percentiles of the fees, in satoshi per virtual byte, for the last 10,000 transactions.

The following address formats are supported:

* Pay to public key hash (P2PKH)
* Pay to script hash (P2SH)
* Pay to witness public key hash (P2WPKH)
* Pay to witness script hash (P2WSH)
* Pay to taproot (P2TR)

Details about the Bitcoin integration API can be found in the [https://internetcomputer.org/docs/current/references/ic-interface-spec/#ic-bitcoin-api Internet Computer specification].

In order to reduce the risk of inconsistencies due to forks, confirmations are counted conservatively, using 𝜹-stability (and not difficulty-based 𝜹-stability). The ''stability count'' of a block is defined as the largest
𝜹 so that the block is 𝜹-stable. The number of confirmations for a transaction in a block is defined as the stability count of the block.

While this rule may seem complicated, it has several nice properties: If there are no forks, the number of confirmations for each transaction is exactly what one would expect: Once a transaction is in a block, it has one confirmation, if there is a subsequent block, it has two confirmations and so forth. However, if there are multiple competing forks with similar heights, the number of confirmations will remain low for blocks on the competing forks until one fork prevails, at which point the number of confirmations starts to increase.
In short, the rule is meant to ensure that a large number of confirmations, based on the stability count, implies a large probability that the transaction will not be undone even in the presence of competing forks.

The following figure shows an example of a chain with two forks. The numbers inside the blocks are the stability counts, which correspond to the number of confirmations for transactions in the blocks. Unlike regular confirmation counts, the stability counts can be negative. Only blocks with a positive stability counts are considered when answering requests.

[[File:Forks and stability.png|thumb|center|upright=3|A chain with two forks and the resulting stability counts (inside the blocks) is shown.]]

When the Bitcoin canister receives an outbound transaction and the transaction passes a basic verification that it can be decoded, the Bitcoin canister forwards the transaction to the Bitcoin adapter as described above.
There is no guarantee that the transaction will ever be included in a block. It is up to the issuer of the transaction to maximize the chances to get the transaction into a block by making sure that the transaction is valid and including an appropriate miner fee.

=== Bitcoin Testnet Release ===

In summer 2022, the Bitcoin testnet integration was released on the IC. The Testnet Integration is intended to allow engineers to interact with the Bitcoin API on the IC, but only with Bitcoin Testnet and a threshold ECDSA test key. The goal is to allow engineers to build and test their Bitcoin-integrated dApps using a beta running on the IC. This is the next step after the previously-launched developer preview that facilitates dApp development through the community.
It is strongly advised that engineers do not associate any value with the threshold ECDSA test key as it may be deleted at some point in time and it is running only on regular-size subnet.

=== Bitcoin Mainnet Release ===

The Bitcoin integration was released on the IC in the beginning of December 2022, making all Bitcoin integration features available for Bitcoin mainnet.

==See Also==
* '''Code Bitcoin on the Internet Computer: [https://internetcomputer.org/bitcoin-integration/ https://internetcomputer.org/bitcoin-integration]'''

NNS Canisters

2023-11-16T06:52:51Z

Yap: Add intro sentence

The [[Network Nervous System|Network Nervous System (NNS)]] is the decentralized autonomous organization (DAO) that governs the Internet Computer by proposals on which ICP neuron owners can vote. Once such a proposal is accepted, it is autonomously executed.

The NNS of the Internet Computer is realized by a set of ''[[glossary#canister|canisters]]''. NNS canisters include:

#'''Ledger canister:''' The ledger canister stores the ICP utility ''token balance'' of each principal and the history of ICP ''transactions''.
#'''Governance canister:''' The governance canister receives and stores ''Proposals'', which are suggestions for how the Internet Computer should be changed. These proposals can then be voted on. The governance canister also tracks ''Neurons'', which determine who is allowed to participate in governance.
#'''Registry canister:''' The registry canister stores the configuration of the whole Internet Computer, e.g., which nodes belong to a certain subnet and the software each node should run.
#'''Cycles minting canister''': This canister is responsible for minting ''cycles'', the fuel for canisters for computation, communication and storage. New cycles can be minted when a new canister is newly created or when an existing canister is topped up with additional cycles.
#'''Root canister''': The root canister is the controller of all other NNS canisters and responsible for upgrading them.
#'''Lifeline canister''': The lifeline canister is the controller of the root canister and responsible for upgrading it.
#'''Archive canisters''': The canisters that store the history of the ledger transactions once there are too many transactions to keep in a single canister.
#'''Genesis token canister:''' The canister that initialized the neurons that already existed during genesis.

The canisters that users of the Internet Computer are interacting with the most are the first two: the ledger canister for making transactions, and the governance canister for staking tokens and submitting and voting on proposals.

==Ledger canister & ICP utility tokens in the NNS==
[[ICP token|ICP utility tokens]] are managed by the ledger canister, which stores two things: accounts and transactions. An account record keeps track of how many tokens are in the possession of a given [[principal]] (i.e., an identity by which a user is authenticated on the Internet Computer). Tokens can then be sent from one account to another, and this is recorded in the transactions of the ledger canister.

In the NNS, ICP utility tokens are used for three different things:

# ICP tokens facilitate '''participation in governance''' (more below on how the neurons are connected with the tokens).
# Those who participate in governance and those who provide compute capacity by operating note machines are '''rewarded''' in ICP tokens.
# ICP tokens are used for '''conversion into cycles''', which are fuel for canisters for computation, communication and storage.

==Governance Canister==
The governance canister is responsible for holding ''neurons'', determining who is allowed to participate in governance. Moreover, it stores proposals, which are suggestions for changes to the Internet Computer, and the information associated with proposals that decides if these suggestions should be implemented. If a proposal is adopted, the governance canister automatically executes the decision. Finally, the governance canister distributes rewards to those neurons who participated in voting and contributed to the decision making.

==Neurons==
Neurons contain ''locked'' ICP utility tokens. These staked tokens are not liquid and cannot be transferred freely to others.

===Neuron Attributes===
Each neuron stores a number of neuron attributes. Some of the most important ones are the following:

* How many ICP tokens are ''locked'' in the neuron. This information is accessible both by a neuron referencing an account on the ledger canister that stores the neuron's balance and by a cached stake on the governance canister.
* A ''controller'' principal, which identifies who manages the neuron's actions.
* A unique ''neuron ID''
*The neuron's ''dissolve delay.'' Intuitively, the dissolve delay defines the earliest time when the locked ICP tokens can be unlocked. This time can be increased to up to 8 years but it can only be decreased by waiting for time to pass. A neuron can either be ''non-dissolving'', ''dissolving'', or ''dissolved.'' If a neuron is non-dissolving, its set dissolve delay is kept stable and the timer is not running down. The neuron can then be set to dissolving, which means that the dissolve delay is decreasing with the time. Finally, if a the dissolve delay reaches 0, the neuron is dissolved and the locked tokens can be transferred out of the neuron.
*The ''age'', which is between 0 and 4 years. A neuron's age determines the time since when then neuron has last entered the non-dissolving state.
A neuron's dissolve delay determines a neuron's ''eligibility'' to participate in voting. Namely, only those neurons with tokens locked for at least six months are eligible to participate in governance. This incentivizes neuron holders to vote such that the value of their tokens is maximized for a future date. Assuming the value of the tokens is a rough proxy for the network’s success, this incentivizes neuron holders to vote in the long-term interest of the Internet Computer.

A neuron's ''voting power'' depends on how many tokens a neuron has locked, as well as the neuron's dissolve delay and age. Intuitively, those who are more committed to the Internet Computer, as they have locked their tokens for longer or have already staked them for a long time, have more voting power. The voting power increases with the dissolve delay and the age.

Finally, the number of rewards that each neuron receives depends on the number of votes that the neuron participated in as well as the neuron's voting power.

===How to lock tokens in a neuron<ref>https://medium.com/dfinity/the-network-nervous-system-governing-the-internet-computer-1d176605d66a</ref>===

As a user, there are different [[ICP staking options]] to stake ICP utility tokens in a neuron. The effect of such an operation is that these ICP tokens are transferred to a ledger account that is associated with a newly created neuron. The tokens are thus locked and cannot be used freely by the neuron holder.

'''Example.''' ''Let’s assume that User B, who has an account (A1) on the ledger canister, would like to lock a hundred tokens in a neuron. To do so, User B sends a command to the NNS specifying the number of tokens and User B’s corresponding principal ID.''

''A transaction is then recorded on the ledger, which specifies that some tokens are sent from the original account (A1) of the user to a new account (A2), which also creates the new account (A2) that holds the locked tokens. A new neuron is created in the governance canister that specifies that User B is the one controlling this neuron and that specifies that the amount of locked tokens is defined by the new ledger account A2.''

''Externally, it’s not visible that the new account (A2) holds locked tokens or is in any way related to the original account (A1). Nevertheless, this account is in fact controlled by the neuron, which means that the tokens are not liquid and that User B cannot transfer the tokens or convert the tokens into cycles.''

The main reason for a user to lock tokens in a neuron is to be able to participate in voting and get voting rewards. Both are described in more detail below.

==Proposals==
A proposal is a suggestion for a change to the Internet Computer. More technically, a proposal describes a ''method'' in a canister that is called if the proposal is accepted. Moreover, it describes the ''parameters'' with which the method will be called.

The Internet Computer supports a variety of different proposal ''topics''. Here are some examples of topics that are supported in the NNS governance canister:
* #SubnetManagement Proposals: This considers topology changes. The example proposal above about whether a node should be added to a subnet falls into this category.
* #NodeAdmin Proposals: This concerns the administration of node machines. An example of a proposal could specify that all of the nodes in a subnet should be updated.
* #NetworkEconomics Proposals: This concerns the administration of network economics. For example: what rewards should be paid to the node machine providers?
*#Motion Proposals: These proposals do not have a direct execution of a method as a consequence but are merely there to record the opinion of the community on a specified matter.

==Voting and proposal lifecycle==

=== Submitting a proposal ===
Any eligible neuron can make and [https://wiki.internetcomputer.org/index.php?title=Neuron_Attributes_and_Commands#Make_Proposal submit a proposal]. To avoid being inundated by useless proposals, a user submitting a proposal has to pay a fee of 1 ICP when submitting a proposal, that they will receive back if the proposal is adopted (but that they will not receive back if the proposal is rejected).

'''Example.''' ''Let’s consider a User C who would like to suggest that a new subnet is created that initially consists of two nodes: Node 1 and Node 2. Once User C controls a neuron, they can submit a proposal by specifying their neuron ID, the type of proposal that they would like to submit, and the proposal’s parameters. In our example, the proposal specifies that a new subnet should be created and the proposal's parameters consist of the initial nodes Node 1 and Node 2. Upon the receipt of this proposal, the governance canister first checks that this user is indeed the one controlling the neuron with the given ID and that this neuron is eligible to vote. If the requirements are met, the proposal is added to the governance canister.''

After a proposal is submitted by an eligible neuron, the proposal is created and stored in the governance canister. Moreover, the governance canister computes and stores additional information with each proposal. First, the [[voting power]] of each neuron is computed and stored together with the proposal. The sum of all of these voting powers also determines the ''total voting power'' associated with a given proposal.

When a new proposal is created, the number of “yes” votes associated with the proposal is already increased by the proposer’s voting power. This reflects that the proposal already has the support of the user submitting it.

Moreover, each proposal has an associated ''voting period,'' which determines the period of time over which votes for this proposal are accepted.

===Viewing NNS Proposals===
You can see all the NNS proposals on the Internet Computer dashboard: https://dashboard.internetcomputer.org/governance

===Discussing NNS Proposals===
Voters can freely discuss proposal anywhere they like. A lot of NNS proposals are discussed on the developer forum: https://forum.dfinity.org/c/roadmap/29.

===Voting on a proposal===
After a proposal is submitted and added to the governance canister, other users who control neurons can [[ICP voting options|vote on the proposal]]. Currently, the most user-friendly way to vote on NNS proposals is via the NNS Frontend dapp: https://nns.ic0.app/. For voting, users would first learn which are the open proposals on the governance canister that they can actually vote on. This information is available, for example on Internet Computer dashboard: https://dashboard.internetcomputer.org/governance or in the [[NNS frontend dapp]].

If a neuron votes in favor of a proposal, the governance canister adds this neuron’s voting power, as stored with the proposal, to the “Yes”-votes associated with the proposal. Likewise, if a neuron votes against a proposal, the governance canister adds the neuron’s voting power to the “No”-votes of the proposal.

'''Example.''' ''Assume that a given User D would like to reject the proposal that was just added by User C. To do so, User D would send their neuron ID and a “no” vote to the governance canister. The governance canister would check that the vote is coming from the correct user who controls the neuron and confirm that the neuron is eligible to vote. If the conditions are met, the governance canister would then add the voting power of User D to the “no” votes.''

====Neuron following and liquid democracy====
Users may not have the time or knowledge to participate in all voting decisions. Therefore, instead of directly voting on proposals, neuron holders may choose to delegate their vote to other neurons that they trust with certain decisions. This concept of delegating the right to vote to other voters who then effectively vote with more voting power is called ''liquid democracy.''

Concretely, for each proposal topic a neuron can [[How to follow neurons|specify a set of other neurons that it would like to ''follow'']] ''(so-called followees)''. In addition, a neuron can specify a set of followees for "all other topics" that are not covered by specific rules. The governance canister keeps track of this relation of follower and followee neurons. It then automatically casts a vote for a follower neuron based on the decision of the followees. In particular, if more than 50% of the followees vote "yes", then a "yes" vote is cast for the follower and if at least 50% of the followees vote "no", then a "no" vote is cast for the follower.

'''Example'''. ''Consider neuron N1 that follows the set of neurons {N2, N3, N4, N5} on all proposal topics. Consider now that a proposal is submitted by another neuron N6. Assume in a first scenario that first N2 votes "No" and then N3 votes "No" on the proposal. In that case, the governance canister will also send a "No" vote for N1 two out of four followees voted "No" (which also means that it is not possible anymore to get more than 50% "Yes" votes). Assume a second scenario where N2 votes "Yes" and then N3 votes "Yes" on the proposal. In that case, no vote is sent yet for N1. However, if either N4 and N5 also send a "Yes" vote, a "Yes" vote is also cast for N1.''

This liquid democracy has great advantages. First, it allows even neurons that do not have enough knowledge of a certain topic to nevertheless participate in governance by choosing the neurons that they trust with certain decisions and by delegating their vote to them. In particular, a neuron can choose a different set of followees for different topics. Moreover, this mechanism allows neuron holders to get voting rewards from voting participation even if they do not have time to actively participate in all voting decision.

=== Proposal decision and wait-for-quiet ===
A proposal can be ''decided'' in two ways:
# '''Absolute Majority before the voting period ends''': At any point, even before the voting period ends, if an absolute majority (more than half of the total voting power stored in the proposal) has voted Yes, then the proposal is adopted and if an absolute majority has voted No, then the proposal is rejected.
# '''Simple Majority at the voting period’s end''': When the voting period ends, if a simple majority (more than half of the cast votes) has voted Yes and the number of these Yes-votes constitute at least 3% of the total voting power, then the proposal is adopted. Otherwise, the proposal is rejected.

What also plays into this is the fact that the governance voting algorithm also applies ''wait-for-quiet''. The idea of wait-for-quiet is to decide proposals quickly when all voters agree, but increase the time that neurons can vote for proposals that are controversial. That means that the voting period can be dynamically increased, depending on the neurons’ votes. In particular, each time a proposal’s outcome is turned (either a Yes-majority is turned to a No-majority or vice versa), the proposal’s deadline is increased. Currently, a proposal's initial voting period is 4 days and can be increased by at most another 4 days. That is, the voting period that is taken into account for the above rules can be between 4 and 8 days, depending on the voting activity.

=== Proposal execution ===
Recall that a proposal defines a method, a canister, and some parameters. As soon as a proposal is adopted, the defined method on the specified canister is called with the given parameters. This is done fully automatically by the governance canister.

== Voting Rewards ==
Contributing to decision-making is one incentive for voters to lock their neurons, but they are also ''rewarded'' for participating in network governance.

Specifically, each day the governance canister considers which proposal can be settled. It is then considered for each neuron in how many proposals they participated and with which voting power. Depending on this, the neurons get rewarded. Concretely, rewards are added to the neuron's attribute that is called ''maturity.''

A neuron holder can then profit from the maturity in two ways:

# '''Spawn maturity:''' A neuron holder can choose to [[How to spawn a neuron|spawn a new neuron]] to get liquid ICP utility tokens worth the voting rewards. Spawning a new neuron has the effect that a new neuron with a new associated account on the ledger will be created, which contains the amount of staked ICP utility token equivalent to the maturity of the original neuron. In fact, the new tokens are minted and transferred to the new account, which is also recorded in the ledger canister. The new neuron has a small dissolve delay of 7 days. This means that users only need to wait a short amount of time to be able to unlock the tokens and use them freely, no matter what dissolve delay the original neuron has.
# '''Stake maturity''': A user can stake maturity which locks the maturity until the neuron is dissolved. Once a neuron is dissolved, staked maturity is unlocked and becomes normal maturity. Staked maturity contributes to the voting power of the neuron as determined by Voting Power = (ICP staked + maturity staked) x Dissolve Delay Bonus x Age Bonus. No ICP is (or can be) produced from staked maturity until the neuron is dissolved and the maturity is spawned (as described in the section above), at which point it is subject to maturity modulation.

==Cycles Minting Canister and Cycles<ref>https://medium.com/dfinity/the-network-nervous-system-governing-the-internet-computer-1d176605d66a</ref>==
Besides governance participation and voting rewards, tokens can also be converted into cycles, which fuel computation and storage on the Internet Computer. Each canister on the Internet Computer, except for those on the NNS, uses cycles for computations and has some cycles stored within it. While the token price may vary over time, the goal of the cycles is to keep the price of computation roughly consistent over time.

The canister responsible for converting ICP utility tokens into cycles is the ''Cycles Minting Canister.'' To convert cycles for creating or topping-up a canister, a user needs to send ICP utility tokens to the Cycles Minting Canister. This canister then burns the tokens and mints the cycles.

The Cycles Minting Canister only facilitates the conversion of ICP utility tokens to cycles but not the other way around. Cycles are burned in canisters when they use computation and storage over time.

'''Example.''' ''Consider User E who runs a canister on the Internet Computer and would like to top up the cycles of this canister so that it can perform even more computations. Also assume that this canister currently has 700 trillion cycles and User E would like to increase this number by 200 trillion. To do so, the user would send a command to the NNS that specifies the action of topping up their canister. Upon receiving this command, a transaction is made from the user’s account to the Cycles Minting Canister. As a result of this transaction, the Cycles Minting Canister would burn the tokens, mint new cycles, and send these freshly minted cycles to the user’s canister, meaning the canister balance is now 900 trillion cycles.''

==See also==

* [https://medium.com/dfinity/the-network-nervous-system-governing-the-internet-computer-1d176605d66a Medium article]
* [https://dfinity.org/howitworks/network-nervous-system-nns Video]

=References=

Node providers

2023-11-16T06:35:09Z

Yap: There are a lot places in the wiki where the NP is a node owner, including many instructions. Referring to it as the principal is a special case, imho

A '''node provider''' (NP) is the owner of one or several [[Glossary#node|node]] machines, and may also be involved in node operation and related tasks. Often, NP is also used to refer to a non-canister [[Glossary#principal|principal]] that receives the rewards stemming from node participation to the [[Glossary#Internet Computer (IC)|IC]] (aka “payout principal”). A node provider may receive rewards from multiple nodes in multiple [[Glossary#data center|data centers]].

Extend Bitcoin, Ethereum and other blockchains

2023-11-16T06:21:17Z

Yap: Typos and clarifications

'''The Internet Computer blockchain is the first [[World Computer]] blockchain that provides an alternative to traditional IT. It is a blockchain with new capabilities, such as [[Web Serving|web serving]], unlimited scaling, web speed, and orders of magnitude efficiency gains. All of this has been made possible by the new [[chain key cryptography]] framework. Now this same framework makes the capabilities of the Internet Computer available within ''other blockchain ecosystems'', by allowing smart contracts on the Internet Computer to initiate transactions and computations directly on other blockchains...'''

[https://internetcomputer.org/bitcoin-integration Quick link: Dive deeper into the workings of smart contracts that directly process bitcoin on the Bitcoin ledger].

Essentially, the Internet Computer network can sign arbitrary transactions for execution on other blockchains. It does this using the [[chain key cryptography]] framework embedded within its decentralized protocols. For example, it can sign transactions to transfer cryptocurrency, or invoke smart contract code, ''on other blockchains''.

This means you can extend other ecosystems with Internet Computer capabilities, or use its smart contracts as glue to construct fully decentralized Web3 and DeFi services that span multiple blockchains.

Let's look at some examples...

=== Extending Bitcoin or working with bitcoin ===

Bitcoin was the blockchain that started it all. Currently the network hosts vast liquidity (monetary value) in the form of bitcoin. However, the Bitcoin network is simple by design and does not host smart contracts.

As a consequence, Bitcoin does not have a native DeFi ecosystem of its own. Instead, Bitcoin's liquidity is "ported" to other blockchains that do have smart contract capabilities using insecure centralized [[blockchain bridges]], which take custody of bitcoin for users, and create equivalent amounts of wrapped bitcoin on a destination blockchain, such as Ethereum.

Using native Bitcoin DeFi that operates directly on real bitcoin can provide major advantages over using wrapped bitcoin. In practice, blockchain bridges are cumbersome to use, can act as censors, or can get shutdown or hacked causing the loss of bitcoin backing the wrapped bitcoin, completely breaking services using the wrapped bitcoin, and potentially causing rippling systemic black swan events.

On the Internet Computer, smart contracts can process bitcoins ''directly on the Bitcoin ledger'', without need for any kind of intermediary. They can create new bitcoin addresses, and send and receive bitcoin — as though they are in fact running on the Bitcoin network itself.

'''This means the Bitcoin community can now finally create what amounts to native Bitcoin DeFi using smart contracts hosted on the Internet Computer. These smart contracts can even serve frontends, allowing user experiences to be created for DeFi users that are also fully decentralized, and can provide alternative means to sign Bitcoin transactions without browser extensions or similar, leveraging [[Internet Identity]].'''

''Moreover, Internet Computer smart contracts can enable services from other blockchain ecosystems, such as Ethereum, to leverage native bitcoin too. For example, native bitcoin processing might be used by on-chain DEXs (decentralized exchanges), decentralized fundraising schemes, Web3 services, and more.''

[https://internetcomputer.org/bitcoin-integration Quick link: Dive deeper into the workings of smart contracts that directly process bitcoin on the Bitcoin ledger].

=== Extending Ethereum or working with Ethereum smart contracts ===

Ethereum was the next major blockchain to launch after Bitcoin. It introduced [[smart contract]] functionality, which made it possible to upload "blockchain code" that would thereafter process and store data on the blockchain itself.

This was new, because on Bitcoin it's only possible to attach basic access control code to the bitcoin tokens themselves, which disappear when they are moved (transferred). Ethereum by contrast can host [https://en.wikipedia.org/wiki/Turing_completeness Turing-complete] [[smart contracts]] at permanent addresses. ETH tokens move between these stable smart contracts, such that the blockchain is code-centric.

'''When considering the implications of blockchain hosting code, and related computation and data, in 2014 a member of the early Ethereum community mooted the concept of a [[World Computer]]. Ethereum's technological underpinnings, which will remain after [https://ethereum.org/en/upgrades/merge/ The Merge], prevent it from making that leap. However, from 2015, Dfinity worked on delivery of the [[World Computer]] concept, finally launching the Internet Computer May 2021.'''

The incredible community and DeFi ecosystem that Ethereum built in the years from launch, can now be combined with the technological capabilities of the Internet Computer — without the use of bridges or other intermediaries.

'''This advance has come at a crucial time, because increasingly the Ethereum ecosystem is being held back and harmed by a lack of decentralization.'''

For example, today, although Ethereum smart contracts can be used to create a DeFi service, they are not capable of [[Web Serving|serving]] interactive web experiences that enable end users to interact with them. Typically, [[cloud computing]] infrastructure is used to provide the user experience, and often also to perform the vast majority of data processing and storage — especially where Web3 services and dapps are involved. This exposes them to all kinds of vulnerabilities, including being censored by the cloud operator, getting hacked, and the transfer of legal liability for the otherwise decentralized service to the developers operating the cloud accounts involved.

'''For all the reasons, it has become very clear that DeFi and Web3 services built using Ethereum need to fully decentralize and replace centralized IT such as cloud services, and go fully ''on-chain''.'''

Thankfully, the Internet Computer provides an easy solution. For example, a front-end user experience built using Internet Computer smart contracts, can respond to end user inputs in a stock browser without extensions by initiating arbitrary transactions on Ethereum, and can display Ethereum data and Ethereum transaction results to them.

Here's how a DEX (decentralized exchange) running on Ethereum could be improved:

* The interactive web experience, through which users place orders and manage their accounts, can be created using smart contracts on the Internet Computer, which can process HTTP requests.

* Expensive data processing and storage can be offloaded to Internet Computer smart contracts. For example, the Internet Computer can be used to manage user profile information, and log all their trades — it would even be possible to use them to create a continuous double auction order book.

* [[Internet Identity]] can be leveraged. This is an anonymizing blockchain authentication framework the Internet Computer provides to allow end users to securely create sessions with Web3 services using special hardware, such as the fingerprint sensor on their laptop, or Face ID on their phone. For example, a front-end built on the Internet Computer can map Internet Identity anchors to Ethereum public keys, and then allow end users to securely and conveniently authenticate themselves to the DeFi service using their fingerprint sensor ([https://medium.com/dfinity/internet-identity-the-end-of-usernames-and-passwords-ff45e4861bf7 read a friendly Internet Identity introduction]).

Ethereum integration, and Internet Computer integrations with other blockchains, works slightly differently than with Bitcoin, which involves UTXOs. Internet Computer smart contracts can create arbitrary Ethereum transactions, then query the results using [[HTTPS outcalls]].

NOTE: At the time of writing, the full end-to-end framework for direct integration with Ethereum and other blockchains is still in development, although individual Internet Computer technology features can be combined to interact with Ethereum. You can proceed now using its [https://internetcomputer.org/docs/current/developer-docs/functionality/t-ecdsa/ Threshold ECDSA capabilities] — watch how other community members are proceeding.

==See Also==
* '''The Internet Computer project website (hosted on the IC): [https://internetcomputer.org/ internetcomputer.org]'''

How-to: Interact with SNS canisters

2023-11-16T05:58:18Z

Yap: typos

This article explains how to interact with an SNS and the dapp canisters it controls. There are two ways to interact with canisters - via the [https://dashboard.internetcomputer.org/ dashboard], where you can click through a friendly user interface, or via the command line using [https://internetcomputer.org/docs/current/developer-docs/build/install-upgrade-remove DFX].

For more information on the SNS, see the [https://internetcomputer.org/sns/faq SNS FAQ].

'''Disclaimer: participation in any SNS is at your own risk.'''

== Finding all SNS and dapp canisters ==

If you want to interact with a dapp and the SNS DAO that governs the dapp, you likely know either the dapp canister ID or the SNS root canister ID by which an SNS is identified.
This article first describes how, starting from SNS root canister ID, you can find and interact with all SNS and dapp canisters.

== Finding all SNS and dapp canisters - starting from SNS ==

==== Get the SNS root canister ====
The SNS root canister is the SNS canister that knows about and controls all the other SNS canisters and the dapp canisters. Therefore it is the canister by which an SNS can be identified. Given this canister ID you can verify it is indeed an “official SNS” that runs an SNS wasm that has been vetted by the NNS.
All SNS root canisters, and thus all SNSs, are listed in the NNS canister called SNS wasm modules canister, or SNS-W for short. SNS-W is a fixed canister ID in the NNS subnet and its canister ID is <syntaxhighlight inline>qaa6y-5yaaa-aaaaa-aaafa-cai</syntaxhighlight>. Call the method <syntaxhighlight inline>list_deployed_snses</syntaxhighlight> and find all root canister IDs in the returned list. You can now check that the SNS root canister ID you know is one of them.

''' DFX.'''
<nowiki> $ dfx canister --network ic call qaa6y-5yaaa-aaaaa-aaafa-cai list_deployed_snses '(record {})' </nowiki>

'''Dashboard.'''
In the SNS-W canister interface, navigate to the SNS-W canister, for example by entering its ID in the search bar.
You will get to a [https://dashboard.internetcomputer.org/canister/qaa6y-5yaaa-aaaaa-aaafa-cai page] that summarises this canister and all its methods.
Click ‘+’ next to <syntaxhighlight inline>list_deployed_snses</syntaxhighlight>, and then click “Call”.

====Get all SNS and dapp canister IDs====
From the last step, you know that the root canister ID has been verified as a “true” SNS root canister ID deployed through the blessed canister wasm path. You can next get the rest of the canister IDs associated with that SNS by calling <syntaxhighlight inline>list_sns_canisters</syntaxhighlight> on the SNS root canister.
This list contains a list of all canisters that collectively make up the SNS.
The response from this method also has a “dapps” entry that lists the dapp canisters that are governed by the SNS.

'''DFX.'''
<nowiki> $ dfx canister --network ic call zxeu2-7aaaa-aaaaq-aaafa-cai list_sns_canisters '(record {} )' </nowiki>

''' Dashboard.'''
On the dashboard, enter the canister ID of SNS root in the search tab. You should see all the methods of SNS root now. Similarly to the last step, click ‘+’ next to <syntaxhighlight inline>list_sns_canisters</syntaxhighlight>, and then click “Call”.
This will return the canister IDs of all SNS canisters and the dapp canisters.

====Get the summary of all SNS and dapp canisters====
Finally, you can get the summary and ownership graph of the SNS including the dapps. To do so, call <syntaxhighlight inline>get_sns_canisters_summary</syntaxhighlight> on the SNS root canister. This will return the status of each canister in the SNS, including information such as the canister’s controller and cycles balance.

''' DFX. '''
<nowiki> $ dfx canister --network ic call zxeu2-7aaaa-aaaaq-aaafa-cai get_sns_canisters_summary '(record {} )' </nowiki>

'''Dashboard.'''
On the dashboard, where you should still be in the SNS root canister interface from the last Step, click ‘+’ next to <syntaxhighlight inline>get_sns_canisters_summary</syntaxhighlight>. You see the option to choose an argument, but you don't have to select this. Click “Call”.

====Verify the controller hierarchy ====
In an SNS, the control hierarchy is as follows.
The SNS root canister controls all SNS canisters, except for the swap canister that implements the decentralization swap and which is controlled by the NNS root canister. The SNS root canister also controls the dapp canisters that are governed by the NNS.
Thus you can check that all canister's controller except swap is SNS root and that the swap canister's controller is NNS root <syntaxhighlight inline>r7inp-6aaaa-aaaaa-aaabq-cai</syntaxhighlight>. The swap canister also has itself as a controller for technical reasons.
Finally, the SNS root canister itself is controlled by the SNS governance canister.

'''Dashboard.'''
You can verify all of this by inspecting what you received in <syntaxhighlight inline>get_sns_canisters_summary</syntaxhighlight> in the last step or you can search for the individual canister IDs, which will lead you to an overview page for each canister, where the controllers are summarised on top.

== Finding all SNS and dapp canisters - starting from the dapp ==
If you have a dapp canister ID, you can first query the canister to find its controller. If the dapp is controlled by an SNS, then this will return you the SNS root canister ID and you can then proceed as explained in the section above.
Often you might just have the URL to interact with the dapp. To follow the above approach to find the underlying canister's controller, you need the dapp canister's ID, which you can get by removing <syntaxhighlight inline>.ic0.app</syntaxhighlight> from the URL.

== Inspecting the SNS canisters ==

'''DFX Preparation. '''
Download these snippets into an empty repo so that dfx can point at the correct network and correct canisters. This allows to call the canisters and read the output in a human friendly way.

Snippet 1: [https://gitlab.com/-/snippets/2431348 dfx.json and candid files]

====List the neurons that exist in SNS Governance====
Before the decentralization swap finishes, the only neurons are the developer and airdrop neurons. Thus, during this time, you can inspect all neurons and see with which neurons the governance canister was initialized.
In SNS governance, <syntaxhighlight inline>list_neurons</syntaxhighlight> returns the full neurons.
This method is paginated: As it is possible that there are more neurons than fit into one message, <syntaxhighlight inline>list_neurons</syntaxhighlight> just returns a number of neurons that you can enter by choosing the parameter <syntaxhighlight inline>limit</syntaxhighlight>. The neurons have a deterministic order so you can get the "next portion" or next "page" by entering in <syntaxhighlight inline>start_page_at</syntaxhighlight> the last neuron of the previously returned page (which will not be included in the next page). Proceeding this way allows you to get the full list of neurons over multiple calls.

''' DFX. '''
<syntaxhighlight> $ dfx canister --network ic call zqfso-syaaa-aaaaq-aaafq-cai list_neurons '(record { of_principal=null; limit=100: nat32; start_page_at=null } )' </syntaxhighlight>

'''Dashboard.''' On the dashboard, navigate to the SNS governance canister by entering the canister ID in the search bar.
Then, click ‘+’ next to <syntaxhighlight inline>list_neurons</syntaxhighlight>. You don't need to choose anything for <syntaxhighlight inline>of_principle</syntaxhighlight> if you want to look at all neurons. Choose a limit as described above. For the first page you don't need to select a <syntaxhighlight inline>start_page_at</syntaxhighlight> and can click “Call”. For the subsequent calls, adjust <syntaxhighlight inline>start_page_at</syntaxhighlight> as described above.

==== Check out how many tokens are sold in the decentralization swap. ====
As long as the SNS has not launched yet, there are some tokens reserved for the SNS decentralization swap. They are stored in the SNS ledger account that is owned by the SNS swap canister (previously called swap). Thus, to learn how many tokens are in the decentralization swap, call <syntaxhighlight inline>icrc1_balance_of</syntaxhighlight> to get the balance of the Swap canister. The returned value is given in fractions of 10E-8 of an SNS token.

'''(Future) Dashboard.''' ''For this to work, the SNS ledger canister needs to be upgraded a new version.''


'''DFX'''
<syntaxhighlight>
$ dfx canister --network https://ic0.app call zfcdd-tqaaa-aaaaq-aaaga-cai --query icrc1_balance_of '(record {owner = principal "zcdfx-6iaaa-aaaaq-aaagq-cai" })'
</syntaxhighlight>



====Learn how many tokens are in the SNS treasury.====
The SNS treasury is a SNS ledger account that is owned by SNS governance. Specifically, it is on a defined subaccount of the SNS governance’s ledger account. To learn it, use <syntaxhighlight inline>icrc1_balance_of</syntaxhighlight> to get the balance of the SNS ledger account with the following arguments:

'''DFX'''
<syntaxhighlight>
dfx canister --network https://ic0.app call --query zfcdd-tqaaa-aaaaq-aaaga-cai icrc1_balance_of '(record {owner = principal "zqfso-syaaa-aaaaq-aaafq-cai"; subaccount = blob "\ce\c2\14\5a\75\0f\d4\a4\72\b2\a0\21\f6\df\f1\e4\a9\07\29\97\00\06\2a\0a\02\bc\1e\a6\5c\a2\92\58" })'
</syntaxhighlight>



'''(Future) Dashboard.''' ''For this to work, the SNS ledger canister needs to be upgraded to a new version.''

Node Provider Data Center and ISP Guide

2023-11-15T19:57:55Z

Yap: typo

For ensuring decentralization, it is highly beneficial to distribute nodes across multiple countries and data centers. Although not as stringent, a similar principle applies to the selection of ISPs.

At present, there are no mandatory technical certifications required for data centers, but this might change in the future. The sole requirement is verifiability of the data center's location.

Suggested approach:

# Visit the [https://dashboard.internetcomputer.org/ world map on the homepage of the IC dashboard]. Identify countries where there are no nodes in operation. For decentralization purposes, it is highly recommended to set up nodes in these countries.
# Check the "[https://dashboard.internetcomputer.org/centers Data centers]" page of the IC dashboard for the country of interest. Verify if there are any data centers in that region with IC nodes present. For better network decentralization, opt for data center companies different from those already hosting IC nodes.
# Contact data centers and request a quote. This [https://docs.google.com/document/d/1-9rj9cEgOqMM3ZzXcniGfAyYkdN9oI8PngflokGazrY/edit template] is an example request. Feel free to adapt it to your (local) needs.
#* The template is quite comprehensive but the key information that should be included in the data center’s quote are:
#*# Monthly Recurring Costs (MRC) of colocation (renting rack space within the data center)
#*# Non-Recurring Costs (NRC), like one-off setup fees
#*# Cost structure for the services provided by their hands & eyes team (if present)
# Contact ISPs and request a quote. This [https://docs.google.com/document/d/17b-OCFMitFGvCzcHX3Qopjyag2cB0GaZkv6wrIQEtZk/edit template] is an example request. Feel free to adapt it.
#* Similar to the data centers, at this stage, you are only interested in the MRC and NRC.
# If possible, visit the sites of the most promising data centers to assess suitability and professionalism of their staff.
# Utilize the received quotes to make a financial analysis according to the [[Node Provider Remuneration|Node Provider Remuneration Model]]. This will aid in determining the optimal number of nodes to host.
# Finalize a provisional contract with a selected data center. This contract can serve as supporting documentation for your declaration as a Node Provider.

To assist with any follow-up questions from data centers regarding the technical aspects of your project, refer to the documentation provided below.

# [[Node Provider Networking Guide]]
# [[Node Provider Machine Hardware Guide]]

#

Node Provider Decentralization and Security Guide

2023-11-15T19:55:35Z

Yap:

===Decentralization ===
As a Node Provider, you are a defender of the Internet Computer's decentralization. Here are some recommendation to maximize your decentralization contribution:
*Be as independent as possible from other Node Providers
**Do not own shares in multiple Node Provider organizations
**Have only a single Node Provider identity
**When seeking support/discussion, use public channels so that Node Provider interaction is transparent
**While other Node Providers may offer advice, you're fully responsible for and in charge of your own nodes
*Be vigilant
**Be independent minded, make your own decisions and do not blindly trust 3rd party advice (DFINITY is also a 3rd party that should not be blindly trusted and that holds no special authority over the Internet Computer)
**Be aware that misinformation can be used as an attack vector, and therefore, it is important to verify information from multiple sources (preferably public and authenticated)
**If you suspect somebody is trying to deceive you, it may be helpful to other Node Providers and other members of the IC community if you use public channels to warn them
*Restrict access to node machines
**Whenever possible, it is best to perform all node-maintenance yourself and to avoid 3rd-party support all together
**When 3rd-party servicing is necessary, use a local service (preferably somebody you know and trust) rather than a global one and carefully monitor their work
*Use local and trusted supply chains
**If possible, purchase hardware locally from a trusted vendor to avoid global single points of failure and to reduce the risk that somebody tampers with your hardware during delivery
*Avoid single points of failure in Node Provider organizations with multiple people
**When possible, use the four-eyes principle
**For transparency about your decentralization contribution, you may provide a description of your internal security controls against single-person access in your [[Node Provider Self-declaration|self-declaration]] (discussed in milestone three)
**Restrict access to trusted employees and collaborators and vet new personnel and collaborators
*Set up your Node Provider service with a local mindset
**Choose a local data center that you can easily reach and inspect
**Choose a data center that is not part of a global business to reduce risk from extra-territorial influence
**Operate your nodes in the same country as you/your organization reside
**Use local employees who live in the same area and that you know yourself
*Keep information relating to decentralization up-to-date
**For example, if you relocate, you should report this to the NNS
===Security ===
*Lock up your hardware
**Understand and verify the physical access control to your node machines in the data centers
**Determine who should be authorized to use the devices that comes in contact with the node machines (USB sticks, HSMs, network cables, laptops, etc.) and prevent unauthorized physical access through safe storage and alarm systems
**Establish safe work practices, e.g., four-eyes principle when accessing node machines or using other devices to avoid tampering with them
**Determine who should have physical keys to access hardware and instruct them to keep the keys safe
**Keep devices disconnected from the Internet except when strictly necessary to be online
*Store passwords and secret keys safely
**Use key splitting to back them up in a way that no single-point compromise will cause loss

Node Deployment Guide (with an HSM)

2023-11-15T19:28:16Z

Yap: typos

This runbook covers all steps necessary to install the Internet Computer Operating System (IC-OS) using the legacy NitroKey HSM instructions. To use the non-HSM onboarding instructions, follow the [[IC-OS Installation Runbook]].

The physical machine is expected to be racked and stacked according to its respective manual.

To complete these steps, you are expected to be physically present in the data center your machine(s) reside(s). Once you successfully onboard your first node, the others can the brought up in parallel.

If you encounter issues through any of these steps, check the [[Node Provider Troubleshooting]] page. If that does not solve your problem, you are encouraged to ask for assistance in the [[Node Provider Matrix channel]].

❗️❗️❗️DFINITY does ''not'' provide live support for Node Providers attempting to onboard nodes.

==1. Choose onboarding path (HSM vs no HSM)==
If you chose the [[Node Provider Onboarding#5. Choose onboarding path .28HSM vs no HSM.29|HSM Node Provider Onboarding Path]], '''continue to the next step.'''

If you chose to onboard '''without''' a Nitrokey HSM, follow the [[IC-OS Installation Runbook]] to onboard your nodes.

==2. Obtain requirements==
*A USB (3.0 speed that can hold at least 4GB) to put the image file on.
** Faster USBs will allow the process to go much faster.
*The NitroKey HSM for your data center.
*[Optional] A USB hub
** This is helpful at some data centers for simultaneously connecting keyboard, mouse, Nitrokey, etc.
*It is recommended that each server has a label with the BMC's MAC address for ease of identification in future dashboard upgrades.

==3. Download installation image==
Download the latest release of the '''IC-OS USB Installer Image''' and the '''corresponding checksum''' from the [https://dashboard.internetcomputer.org/releases Internet Computer Dashboard Releases].
*Note that you should always use a release that is less than 6 weeks old in order to ensure that your node can keep up with the blockchain.

==4. Verify checksum and unarchive file==
===Mac OS X===
#Open the Terminal and type:
#:<syntaxhighlight lang="shell">shasum -a 256 ~/Downloads/disk-img.tar.gz</syntaxhighlight>
#Compare the calculated checksum with the '''IC-OS installation image checksum''' file downloaded in the previous step. '''Warning:''' Only continue if they are identical, otherwise please post your issue in the [[Node Provider Matrix channel]].
#:Open the Terminal and type: <syntaxhighlight lang="shell">tar xzvf ~/Downloads/disk-img.tar.gz</syntaxhighlight>

===Linux / Ubuntu===
#Open the Terminal and type:
#:<syntaxhighlight lang="shell">sha256sum ~/Downloads/disk-img.tar.gz</syntaxhighlight>
#Compare the calculated checksum with the '''IC-OS installation image checksum''' file downloaded in the previous step. '''Warning:''' Only continue if they are identical, otherwise please post your issue in the [[Node Provider Matrix channel]].
#:Open the Terminal and type: <syntaxhighlight lang="shell">tar xzvf ~/Downloads/disk-img.tar.gz</syntaxhighlight>

===Windows===
#Open PowerShell and type:
#:<syntaxhighlight lang="shell">Get-FileHash -Algorithm SHA256 .\Downloads\disk-img.tar.gz</syntaxhighlight>
#Compare the calculated checksum with the '''IC-OS installation image checksum''' file downloaded in the previous step. '''Warning:''' Only continue if they are identical, otherwise please post your issue in the [[Node Provider Matrix channel]].
#:Open PowerShell and type: <syntaxhighlight lang="shell">tar xzvf .\Downloads\disk-img.tar.gz</syntaxhighlight>

==5. Create Bootable USB Stick==
===Mac OS X===
#Open the Terminal and type:
#:<syntaxhighlight lang="shell">diskutil list</syntaxhighlight>
#All available drives should be shown. Identify which device corresponds to your USB stick. You may need to unmount the USB drive:
#:<syntaxhighlight lang="shell">sudo diskutil unmount /dev/YOUR_USB_DEVICE_MOUNTED_PARTITION # E.g. /dev/disk4s1</syntaxhighlight>
#The file path is an example. Use the absolute path to the downloaded image. '''Warning:''' You risk losing your own data if you specify a wrong device.
#:<syntaxhighlight lang="shell">sudo dd if=/Users/YOUR_USER_NAME/Downloads/disk.img of=/dev/YOUR_USB_DEVICE bs=1M</syntaxhighlight>If you get a “device is busy” error from the dd command, you can try running the following command to unmount all of the partitions on the disk, then re-run the dd command:
#:<syntaxhighlight lang="shell">sudo diskutil unmountDisk /dev/YOUR_USB_DEVICE # E.g. /dev/disk4</syntaxhighlight>

===Linux / Ubuntu===
#Open the Terminal and type
#:<syntaxhighlight lang="shell">blkid</syntaxhighlight>
#All available drives should be shown. Identify which device corresponds to your USB stick. You may need to unmount the USB drive:
#:<syntaxhighlight lang="shell">sudo diskutil unmount /dev/YOUR_USB_DEVICE_MOUNTED_PARTITION # E.g. /dev/sdb1</syntaxhighlight>
#Replace ''/dev/YOUR_USB_DEVICE'' with the device that corresponds to your USB stick. '''Warning:''' You risk losing your own data if you specify a wrong drive.
#:<syntaxhighlight lang="shell">sudo dd if=~/Downloads/disk.img of=/dev/YOUR_USB_DEVICE bs=1M</syntaxhighlight>

===Windows===
#Download and install [https://rufus.ie/en/ Rufus Portable]
#Start Rufus
#Select the USB stick under device and select the previously downloaded IC-OS disk image and press start
#:[[File:05.png|480px|screenshot]]
#You may see some warnings. Make sure you don't have any other USBs in your computer and chose OK
#:[[File:06.png|480px|screenshot]]
#:[[File:07.png|480px|screenshot]]
#The "Ready" bar will go from left to right as it completes.

==6. Add configuration ==

===A. Open Config.ini in a text editor===

===='''Mac OS X'''====

#Open Finder. You should now be able to see the CONFIG partition. If it's not visible, remove the USB and insert it again.
#:[[File:mac_01.png|580px|screenshot]]
#Double-click <code>config.ini</code> to open it in TextEdit.

===='''Linux'''====

#Open the File Manager. You should now be able to see the CONFIG partition. If it's not visible, remove the USB and insert it again.
#:[[File:linux_01.png|580px|screenshot]]
#Double-click <code>config.ini</code> to open it in KWrite.

===='''Windows'''====

#Open the Disk Management utility with a right click on the Start menu
#:[[File:09-b.png|300px|screenshot]]#:
#Right click the CONFIG partition
#Select Change drive letter or paths...
#:[[File:10-b.png|780px|screenshot]]
#Select any letter from the drop-down list
#:[[File:11-b.png|480px|screenshot]]
# Click OK.
# You should now be able to see the CONFIG partition in your Windows Explorer. Select the <code>config.ini</code> configuration file
#:[[File:12-b.png|780px|screenshot]]
#Click on Edit to open it.

===B. Edit Config.ini===

#Insert your IPv6 prefix, subnet and gateway.
#:[[File:Edit config ini.png|580px|screenshot]]
#:*The IPv6 prefix should consist of four groups of hexadecimal digits, separated by colons (':'). Each group can contain up to four hex digits.
#:* For example, a valid prefix could look like this: <code>2a00:fb01:400:100</code>
#:*'''Important:'''
#:** The prefix should not have a trailing ':'
#:**IPv6 CIDR notation allows for a double colon ('::') to represent consecutive groups of zeroes in an address. However, the prefix configuration in this context does '''not''' support '::'. The '::' shorthand should '''not''' be used. Even if some groups are all zeros, they must be explicitly written out.
#Save the changes.
#:*If you have trouble saving this file directly, you may need to save to a known location first, then copy the file into place.
#:*If you need help, please do not hesitate to post your issue in the [[Node Provider Matrix channel]].
#:*:[[File:mac_03.png|580px|screenshot]]

==7. Connect Crash Cart==
#In order to configure the UEFI and initiate the installation of the IC-OS, please connect a crash cart to the physical machine.
#Plug-in the VGA/Video, keyboard and IC-OS USB stick
#:[[File:08.png|580px|screenshot]]

==8. UEFI Setup and Boot Menu==
Use the related page below to set up the BIOS/UEFI according to your hardware vendor.

*[[Node Provider Machine Hardware Guide#Gen 2 Node Machine requirements|Gen2 hardware]]
**[[IC-OS Installation - UEFI Configuration - Gen2 Dell]]
**[[IC-OS Installation - UEFI Configuration - Gen2 Supermicro]]
**[[IC-OS Installation - UEFI Configuration - Gen2 Gigabyte]]
**[[IC-OS Installation - UEFI Configuration - Gen2 ASUS]]
*[[Node Provider Machine Hardware Guide#Gen 1 Node Machine requirements|Gen1 hardware]]
**[[IC-OS Installation - UEFI Configuration - Gen1 Dell|IC-OS Installation - UEFI Configuration - Gen1 Dell (Poweredge R6525)]]
**[[IC-OS Installation - UEFI Configuration - Gen1 Supermicro]]
***
'''Important:''' Do NOT enable the RAID bios setting. Doing so will cause issues with the IC-OS installation.

Resume from this point when you are finished configuring the BIOS.

==9. IC-OS Installation ==
#Please wait while the USB Installer is booting up. This process can take up to 3 minutes.
#:[[File:35-sm.png|580px|screenshot]]
#The IC-OS installation starts. Please keep an eye on the progress. This part can take up to 10 minutes. Please remember to check the [[Possible Node Onboarding Errors]] page if you encounter any errors.
#:[[File:36-sm.png|580px|screenshot]]
#Once you get asked to insert the HSM, please remove the keyboard and instead insert the HSM USB device.
#:[[File:37-sm.png|580px|screenshot]]
#If the installation finished successfully, it will initiate a reboot. '''Please do not unplug the USB stick or HSM USB''' device at this point.
#:[[File:38-sm.png|580px|screenshot]]

==10. First Boot==
Please remember to check the [[Possible Node Onboarding Errors]] page if you encounter any errors onboarding.

🚨 '''Do NOT re-try the IC-OS installation after completing this section, as this can cause duplication within the registry.'''
#The first boot of the IC-OS still requires the HSM USB device. Please wait until further instructions. This step can take up to 2 minutes.
#:[[File:39-sm.png|580px|screenshot]]
#Once you see this message, you may unplug the HSM USB device, USB stick and VGA/Video. Your machine successfully joined the Internet Computer! **'''Label the server with the node ID for easy future identification in the dashboard (at least the first 10 characters). ***'''
#:[[File:Node join message.png|580px|screenshot]]

Congratulations! Your machine successfully joined the Internet Computer! The machine has joined the IC and the Node Provider will start receiving rewards!

🚨 Again: Once you reach this stage and see this message, '''do not attempt to restart the onboarding process.''' Doing so may cause duplicate entries in the registry.
==11. Verify node onboarding ==

#Verify that your node was successfully onboarded by checking its status on the [https://dashboard.internetcomputer.org/ dashboard] is set to either “Awaiting Subnet” or “Active in Subnet”.
#*The dashboard can be searched by your Node Provider principal. There, you should see the Node ID of your node (Node ID outputted in step 10).
#*If the status of your node is not either “Awaiting Subnet” or “Active in Subnet”, or if it is not listed under your Node Provider principal, you should contact the [[Node Provider Matrix channel]] for assistance.
#*:[[File:Dashboard-node-verification.png|thumb|998x998px]]

L1 comparison

2022-12-19T14:43:20Z

Yap: replace avg time between blocks 0.96s with average finality (1.4s)

To get a view of the top performers in the blockchain industry, it's useful to compare across common metrics. Here we build a table that does such a comparison. All data correct as of December 9th 2022.

Metrics explanations and references are given below.

== Base comparisons ==

{| class="wikitable"
|-
! Metrics / L1 !! ICP !! Cardano !! Avalanche !! Algorand !! Ethereum !! Near !! Solana
|-
| Average TPS || 9’720 || 2.37 || 49.52 || 15.5 || 11.1 || 8.25 || 286
|-
| Average finality || 1.4secs || || 2.3secs || 3.5secs || 15mins || 2.4secs ||
|-
| Average tx Cost || $0.0000022 || $0.1 || $0.0066 (C-Chain only)|| $0.00025 || $2.39 || $0.0031 || $0.000026
|-
| Average energy consumption wh/tx || 0.008 || 51.59 || 4.76 || 2.7 || 6.29 || || 0.166
|-
| Size of network (nodes) || 823 || 1050 || 1195 || 1530 || || 798 || 1872
|-
| On-Chain storage || $5 (3.95T cycles x 1XDR) || $17,035 - $113,507 (53,236 – 354708ADA) || $206,875 (15,62 5AVAX) || ||
$15,494,409 (12,643.75 ETH) || || $48,625 (3,477.69 SOL) ||
|}

* '''TPS''' measures the transactions processed per second - note that the interval over which these are measured does vary across chains. The dollar amounts are computed by converting the native token cost cycles/gas/fee needed per transaction, to USD given the exchange rate on December 9th 2022.
* '''Finality''' refers to the amount of time that passes between the proposal of a new valid block containing transactions until the block has been finalized and its content is guaranteed to not be reversed or modified (for some blockchains, e.g., Bitcoin, this guarantee can only be probabilistic).
* '''Tx Cost''' measures the cost of a transaction. Note that the definition of 'transaction' varies widely across chains, where some are described below. The dollar amounts are computed by converting the native token cost cycles/gas/fee needed per transaction, to USD given the exchange rate on December 9th 2022. (Cardano and Ethereum figures found in [https://messari.io/asset/cardano/chart/txn-fee-avg Messari dashboard].)
* '''Energy Consumption''' measures the energy consumption
* '''Nodes/Validators''' measures the number of nodes

== Comparing developer experience ==
Whether they were writing games, operating systems or text editing applications, in the 70s, 80s and early 90s, developers always had to face limitations imposed by hardware. Applications were constrained to accessing a few kilobytes of memory through small stacks and heaps, using limited (and constantly changing) instruction sets, and using significant amounts of power to run instructions. The history repeats itself in the blockchain landscape these days. Application developers are limited to stack sizes of a few kilobytes to several megabytes at best. Persistent storage is expensive and limited. Programmers are bound to using cumbersome APIs that make hidden assumptions in terms of numbers of executed instructions. And, moreover, most chains operate inefficiently, burning too much power per executed transaction. This not only limits the types of applications that can be deployed on chain, but also increases development and testing time (and cost).

As opposed to all existing blockchains, the IC brings modern programming to on-chain developers, allowing them to use time for creativity rather than fixing memory packing issues or spreading computation in small iterations that do not hit instruction limits. The IC programming model offers orthogonal persistence, large stack and heap spaces (4GB), stable storage of 48GB (with plans for increase) in mainstream languages, such as Rust, or even Python.

{| class="wikitable"
|-
! Metrics / L1 !! ICP !! Cardano !! Avalanche !! Algorand !! Ethereum !! Near !! Solana
|-
| Fixed tx cost || ✅ || ❌ || ❌ || ❌ || ❌ || ❌ || ❌
|-
| HTTPs outcalls || ✅ || ❌ || ❌ || ❌ || ❌ || ❌ || ❌
|-
| Smart contract language support || Motoko (native), Rust, TypeScript, Python || Plutus (native), Haskell || Solidity || Teal (native), Python ||Solidity (native), Vyper, Yul, FE || Rust, Javascript || Rust C, C++
|-
| Max stack size || 4 GiB || || || 4 MB || 32 KiB || 256 KiB ||
|-
| Max persisted memory (per smart-contract) || 52 GiB || || || 1 MB || 2^261 B (however, 15,494,409$ per GiB) || 32 KiB ||
|-
| On-Chain storage || $5 (3.95T cycles x 1XDR) || $17,035 - $113,507 (53,236 – 354708ADA) || $206,875 (15,62 5AVAX) || ||
$15,494,409 (12,643.75 ETH) || || $48,625 (3,477.69 SOL) ||
|}

* ''' Fixed tx cost''' provides the ability to have predictable costs for computation.
* ''' HTTPs Outcalls''' is the ability to communitcate with Web2 services (outside of the network)
* '''Max stack size''' is the maximum size the stack can grow for smart contracts and serves as a measure for the complexity of code that is supported by each platform
* '''Max persisted memory''' is the maximum size of persisted memory supported by each platform. Persisted memory is preserved across individual function calls
* '''On-Chain Storage''' measures the cost of storing data on-chain

== Comparing user experience ==

{| class="wikitable"
|-
! Metrics / L1 !! ICP !! Cardano !! Avalanche !! Algorand !! Ethereum !! Near !! Solana
|-
| Privacy-preserving identity management || ✅ || ❌ || ❌ || ❌ || ❌ || ❌ || ❌
|-
| Prerequisites to use || Browser || Browser, Extension, Gas || Browser, Extension, Gas || Browser, Extension, Fees || Browser, Extension, Gas || ||
|-
| Staking ratio || 73.89% || 71.58% || 61.78% || 51.17% || 13.57% || 43.19% || 68.59%
|}

== Metrics ==
* '''TPS''' measures the transactions processed per second - note that the interval over which these are measured does vary across chains. The dollar amounts are computed by converting the native token cost cycles/gas/fee needed per transaction, to USD given the exchange rate on December 9th 2022.
* '''Finality''' refers to the amount of time that passes between the proposal of a new valid block containing transactions until the block has been finalized and its content is guaranteed to not be reversed or modified (for some blockchains, e.g., Bitcoin, this guarantee can only be probabilistic).
* '''Tx Cost''' measures the cost of a transaction. Note that the definition of 'transaction' varies widely across chains, where some are described below. The dollar amounts are computed by converting the native token cost cycles/gas/fee needed per transaction, to USD given the exchange rate on December 9th 2022. (Cardano and Ethereum figures found in [https://messari.io/asset/cardano/chart/txn-fee-avg Messari dashboard].)
* '''Energy Consumption''' measures the energy consumption
* '''On-Chain Storage''' measures the cost of storing data on-chain
* '''Nodes/Validators''' measures the number of nodes
* '''Repos, DAOs, Dapps, Tokens''' showcases the leading applications supported by the network
* '''Max stack size''' is the maximum size the stack can grow for smart contracts and serves as a measure for the complexity of code that is supported by each platform
* '''Max persisted memory''' is the maximum size of persisted memory supported by each platform. Persisted memory is preserved across individual function calls

=== A note about average transactions cost ===
* Algorand: https://metrics.algorand.org/#/protocol/, explanation: Average transaction fee of all transactions in the selected time period. [https://developer.algorand.org/docs/get-details/transactions/#fees Algorand fees]
* Cardano: [https://docs.cardano.org/explore-cardano/fee-structure Cardano fees] Fees are constructed around two constants (a and b). The formula for calculating minimal fees for a transaction (tx) is '''a''' times '''size(tx) + b''', where:
** a/b are protocol parameters
** size(tx) is the transaction size in bytes
* Solana: [https://docs.solana.com/transaction_fees Solana fees]

== References ==
* '''ICP''' : [https://dashboard.internetcomputer.org IC Dashboard]
* '''ADA''' : [https://explorer.cardano.org/en Cardano explorer] and [https://cexplorer.io/ cexplorer]
* '''AVAX''' : [https://snowtrace.io/ Snowtrace] and [https://subnets.avax.network/ Avalanche explorer]
* '''ALGO''' : [https://www.algorand.com/ Algorand website] and [https://metrics.algorand.org/ Algorand metrics site]
* '''ETH''' : [https://etherscan.io/ Etherscan]
* '''NEAR''' : [https://explorer.near.org/ Near explorer] and [https://docs.near.org/ Near docs]
* '''SOL''' : [https://solana.com/ Solana website] and [https://solanabeach.io/ Solana beach]

IC P2P (peer to peer) layer

2022-11-10T09:42:54Z

Yap: Revise caption of figure

The peer-to-peer layer of the IC is responsible for delivering messages between peers within subnets. It is designed to ensure efficient delivery of messages even in the presence of malicious activity.

== Artifacts ==
Each client of the P2P layer, i.e., components in the layers above it like Consensus, maintains a collection of artifacts (artifact pool). Artifacts are messages that are exchanged between peers within a subnet to create, validate and agree on blocks. These can be, for example, consensus block proposals, ingress messages from users, or signature shares of responses for the HTTPS outcalls feature. Nodes distribute these artifacts to their peers so all peers have the same view of the state of the subnet. This allows nodes to include ingress messages sent to other peers when they create block proposals and to catch up efficiently if they have been disconnected or joining a new subnet.

The P2P layer is invoked by the artifact pools when an artifact needs to be delivered to peers. This can happen when the node itself generates a new artifact (e.g., a new block proposal), or when it wants to ''relay'' an artifact it received from another peer (for example, we would want to relay block proposals even when all peers are connected to all peers, since a malicious peer can send one block proposal to a subset of peers, and another (or none) to another subset. This can slow down a subnet. Relaying proposals prevents this). Relay also helps in cases when some nodes are only connected to some peers. This can happen when a node’s ISP changes its BGP peering settings.

== Adverts ==
In order to save bandwidth, which would be wasted by sending an artifact to a peer that already has the artifact in its pool, the P2P layer does not send out artifacts immediately to all peers when requested by a client to broadcast an artifact.
Instead, it creates a message called ''advert'', which is a very small message with the hash of the artifact and some more metadata, and broadcasts this advert to all peers. Other peers then see the advert and decide whether they would like to download the corresponding artifact from that peer, and if so, they send an explicit ''request'' message to that peer, which will respond with the artifact itself.
[[File:P2p.jpg|thumb|400px|Instead of receiving an artifact from multiple peers (left), peers send small adverts) and the recipient requests the artifact from a small subset of its peers.]]
This approach clearly trades throughput for latency, but for the requirements of the IC’s consensus protocol this is desired.

To share the bandwidth fairly among all peers and to avoid running out of memory, the number of in-flight requests is limited.

== Chunks ==
Some artifacts, like state sync artifacts, are too big to be sent as a single chunk. For these artifacts, a single advert represents a set of ''chunks'' that together make up a complete artifact. When a downloaded artifact is ''chunkable'', the P2P layer will attempt to download the corresponding chunks from multiple peers which advertised the corresponding artifact. This speeds up the download and better utilizes the bandwidth.

== Verification of Artifacts ==
Each advert contains an ''integrity hash'' of the corresponding artifact. The integrity hash is the result of applying a cryptographic hash function over the content of the artifact. When an artifact download is complete, the same function is applied locally on the downloaded content, and only if the result matches the advertised hash value, the artifact is processed and provided to the corresponding artifact pool of the corresponding client.

It is important to note that this is only a first layer of defense: a malicious node can send an advert with some integrity hash, and then an artifact that matches the advertised hash. This will pass all checks in the P2P layer, and the client will then have to notice that the artifact does not meet its requirements, e.g., is not signed correctly. The clients in the IC implementation always perform validation checks to catch such attacks before processing artifacts further or relaying them to other peers.

Chunkable artifacts are validated both per-chunk and as a whole, where the corresponding client is responsible for per-chunk validation.

== Retransmission Requests ==
In some cases, for example when a node notices an error in receiving adverts, or when it joins a new subnet, it sends out a ''retransmission request'' to its peers. The request can go to one specific peer (e.g., when a connectivity issue is detected with that peer), or to all peers (e.g., when the node joins). The request contains information about the node’s current state, and peers can respond with adverts that can help the node make progress from its current state, as reported in the retransmission request.

While retransmission requests are meant for helping nodes to catch up on missed adverts, they may also issue them periodically to make sure they haven’t missed anything.

== Transport ==
The underlying Transport component of the P2P layer is responsible for maintaining the actual connections between peers in a subnet. It creates TLS over TCP connections between the peers, where both sides authenticate using their private keys.

Each peer has its public key stored in the registry canister of the Network Nervous System (NNS). The registry canister also contains the most up-to-date subnet membership information (which nodes belong to which subnet), as well as historic information. Each node learns its own membership, as well as who its peers are, what their IP addresses are, and what their public keys are, by querying the registry canister. Nodes always make sure they only connect to peers in their subnet by enforcing two-way authentication when establishing the TLS connections.

Subnet memberships change over time, with nodes being added and removed from subnets regularly based on NNS accepted proposals. The Transport component continuously tracks these changes and adjusts the connections accordingly. In some cases, nodes need to maintain connections to peers that are no longer in the subnet record of the newest registry canister entries, while consensus still considers them to be part of the subnet, and thus the Transport component allows for such connections until they are no longer required.

The Transport component also provides mechanisms for keeping the connection alive, quickly identifies connection issues (using a heartbeat mechanism), and automatically reconnects when the connection breaks. Whenever a TCP connection is idle for more than 200ms, a heartbeat message is sent over. On the receive side, when no data (including heartbeats) is received for more than 5s, the connection is being dropped and reconnected. This is done to avoid waiting for a possibly very long timeout duration of the TCP protocol, which sometimes happens upon Internet routing change events. After a connection is re-established, a retransmission request is sent to the corresponding peer.

Limitless Scaling

2022-10-27T12:52:26Z

Yap: Typos and clarifications

'''The Internet Computer (IC) can scale its capacity without limit, simply by adding additional nodes to fuel new subnets. Nodes are added via the Network Nervous System (NNS) which forms new subnets almost on a weekly basis since genesis. In contrast, most other blockchains have transaction limits baked into the protocol (e.g. adding more servers to Bitcoin does not increase the throughput) and need cumbersome layers to address scaling.

See Internet Computer Dashboard for current scale and performance numbers: https://dashboard.internetcomputer.org/'''

=== Subnet Architecture ===

The Internet Computer's scalability is enabled by sharding canisters and their state into subnets. Each subnet runs its own instance of state machine replication and can therefore process update and query calls independently from other subnets.
While running independently, it is still possible for canister on different subnetworks to communicate by sending messages to each other.

It's therefore possible to scale out by adding more IC nodes to the Internet Computer.
* If new nodes are added as *new subnetworks*, more update calls can be executed per time unit since the new subnetwork can process update calls independently from the rest of the system.
* If new nodes are added to *existing subnetworks*, more queries can be executed per time unit. That is because query calls, in contrast to update calls, are executed on individual machines, rather than replicated on all nodes of a subnetwork as for update calls.

However, the finalization rate of the consensus layer and therefore the rate of update calls per second, might be negatively affected by a larger number of nodes in a subnetwork. That is because more data has to be exchanged between the larger number of nodes before consensus on the order of the input messages can be reached.

Further, having more subnetworks tends to decrease locality: the probability of needing to communicate across subnetworks increases.

Consequently, the IC also must scale up by increasing performance on individual IC node machines. Example are scaling the IC runtime to efficiently utilize resources such as CPU and memory and achieving highly concurrent execution of independent canister calls.

Motoko

2022-09-20T05:59:17Z

Yap: Correct link to motoko docs

==Motoko Programming Language==

The Motoko programming language is a new, modern and type safe language for developers who want to build the next generation of distributed applications to run on the Internet Computer blockchain network. Motoko is specifically designed to support the unique features of the Internet Computer and to provide a familiar yet robust programming environment. As a new language, Motoko is constantly evolving with support for new features and other improvements.<ref name="motoko">https://internetcomputer.org/docs/current/developer-docs/build/cdks/motoko-dfinity/motoko/</ref>

==References==

{{reflist}}