EOS Misled the Public on the Number of Transactions it Can Handle Says New Study

9

“The performance metrics observed when factors such as real world network latency and packet loss are implemented indicate that TPS in EOS approaches levels comparable to Ethereum.”

So says a new study that claims to have completed the first independent benchmark testing of the EOS software.

While previously eos blockchain explorers have shown the network can handle more than 1,200 transactions a second, the study says it can actually handle only 250 transactions a second even under unrealistic conditions of no latency. They say:

“High levels of TPS are sustained at 0 ms round trip latency. When 50 ms of RTT is added into the system, performance immediately falls below 50 TPS indicating that latency has a significant effect on throughput. Once RTT is increased to 100 ms, performance drops even further.”

Round trip time (RTT) is the length of time it takes for a signal to be sent plus the length of time it takes for an acknowledgement of that signal to be received. That’s obviously not possible at zero seconds under real world conditions, with the authors so stating:

“The research results prove the inaccuracies in performance claims and concluded that the foundation of the EOS system is built on a flawed model.”

The study, funded by the Bounties Network which is led by Brent Xu of ConsenSys, replicated the eos network to analyze its performance during two months of testing. The conclusions it reached are fairly damning. They say, for example:

“There is no formal verification protocol that occurs to validate transactions beyond the fact that that block producers are including them within transactions.

The configuration of this system introduces further suspicion as mentioned in our performance testing on the fate of orphaned transactions that are not successfully processed by the network.

In Ethereum, orphaned and stale transactions are apparent for the network to see in the available mempool, while economics are used to determine whether miners or validators choose to process those transactions.

In EOS, there is no true mechanism to determine the fate of unprocessed transactions. One theory as mentioned in our performance section is that transactions are selectively chosen for the purpose of maximizing transaction throughput in the system while bypassing difficult transactions that would cause network congestion. Picking transactions in this manner does not alleviate network congestion but rather ignores it.

What this means is that unprocessed transactions are kept in the computer heap while only easy transactions are successfully processed. Unprocessed transactions are essentially left behind as there is no access to the multi- index table that houses these transactions.

This evidence further calls into question the design of EOS and how it functions as a software that does not successfully optimize the consensus around processing the transactions that are sent to the network.”

Orphaned blocks occur when two blocks are found at or near the same time. One block has to be discarded as otherwise you’re printing money out of thin air. What happens to orphaned blocks in eos, however, is apparently not very clear.

Nor is it very clear whether this is a blockchain at all. The study says  it isn’t. It’s just a database. They say:

“EOS is not a blockchain, rather a distributed homogeneous database management system, a clear distinction in that their transactions are not cryptographically validated.”

Describing what’s occurring in the above featured image, they say: “The duplicate transaction errors execute every time the block producer with faulty code processes transactions. After the duplicate transaction errors, the non faulty block producers continue to successfully process transactions.

One important point to make here is that only the block producer that is running the modified source code is able to notice the duplicate transaction errors.

Nothing in the system allows other block producers to audit the logs of the faulty block producer because the system relies on social trust that each block producer is correctly running the right software and there is no formal verification of that software.”

Eos thus runs on social “consensus” according to the study, which further says eos is not Byzantine Fault Tolerant and suffers from consensus failures. They conclude:

“The transaction throughput in the system does not exceed 250 TPS even in optimal settings with 0 ms of latency and 0% packet loss.

During tests with real world conditions of 50 ms of round trip latency and 0.01% packet loss, performance dropped below 50 TPS putting the system in close proximity to the performance that exists in Ethereum.

Consensus in EOS does not exhibit Byzantine Fault Tolerance. The use of 21 block producers in the system is arbitrary as the producers are processing transactions in a round robin manner, with no algorithmic logic when block producers behave nefariously.

Various forms of consensus failure were shown in the network testing. Consensus is entirely dependent upon the voting mechanism which is based on social consensus, not algorithmic BFT consensus.”

They say the eos set-up is not good enough even for a sidechain due to the black box nature of what block producers are doing. The authors so stating:

“EOS can potentially act as a side chain appended to other more foundationally secure networks, though the system would need to be rebuilt in order to address the problems detailed in this report.”

The tests were commissioned by ConsenSys, so one can argue there might have been some bias, but the authors state the raw data will be shared with:

“Cosmos, Ethereum Community Fund, Amis, Maker DAO, IMToken, PlatON, Status, Loom Network, Coinfund, 1KX, Transference Fund, Web 3 Foundation, Grid+, MixLabs, Ledger Capital, Enterprise Ethereum Alliance, Google, Microsoft, ConsenSys and Bo Shen – Founding Partner of Fenbushi Capital & Cofounder of Bitshares, along with academic institutions such as Duke, USC, and MIT.”

They “have committed to contributing resources to develop comprehensive reports based on Whiteblock’s raw data,” the authors say, with Google apparently having an interest in participating.

A number of problems with eos have previously been revealed, but the findings that even block producers themselves do not abide by some algorithmic logic, as in protocol rules, goes far beyond what could have previously been imagined.

Eos, in effect, is worse than Ripple. There at least there is some sort of algorithm, there are some sort of rules, albeit a consensus algorithm that requires a high level of centralization and a greater degree of trust than for proper public blockchains.

In eos, however, there appears to be no rules that have to be followed. Transactions are not cryptographically validated. Block Producers have to manually kick out misbehaving individuals, yet a bored kid could probably manually overwhelm them, let alone a kid with simple tools.

Suggesting that a word which is often incorrectly thrown around in this space, is perhaps well deserved in this case because eos appears to be a complete facade.

Copyrights Trustnodes.com

 

9
Leave a Reply

100000
5 Comment threads
4 Thread replies
0 Followers
 
Most reacted comment
Hottest comment thread
8 Comment authors
  Subscribe  
newest oldest most voted
Notify of
indykpol

This info doesn’t hold water when you look at actual numbers of the live mainnet (http://eosnetworkmonitor.io). It already peaked at 4000 tps, and performed over 115 tps throughout an entire day once (https://blocktivity.info/). It currently performs about 45 on average.

Anon Anon

Absolute nonsense. Adding on an ambiguous round-trip-time (RTT) and then claiming that is the TPS time. I wonder which blockchain project paid for this benchmark study to be conducted and what the end goal was? I doubt it was motivated toward honesty and more likely an attempt to smear. Pretty obvious it’s probably Etherium attempting to spread lies because they know they are covered in the dust of EOS disappearing in the rear-view mirror. So much has not been taken into consideration and to publish a damaging click-bait title like that shows that trustnodes can’t be trusted either. Especially the… Read more »

Mulk

Latency and RTT should not have such a dramatic effect on the number of transactions per second. The network should take care of that otherwise it is badly developed. So, if that is the case I only can agree with this writer. Just my two cents.

Robert

What a load of crap….
EOS is in a different league, the EOS ecosystem is on fire & the community is epic its unstoppable

anon

its unfortunate that Ethereum’s ConsenSys is resorting to these sorts of underhanded tactics because their platform is lagging behind. This report will confuse readers and give them the wrong impression of what’s actually going on. Did the author of this article read this? https://medium.com/@thomas.cox_39839/yes-eos-is-a-blockchain-a5921e5497d5

Why has journalism so shitty lately where basic due diligence is just complete not done?

anon

And yet the ‘real world’ confirms EOS is actually processing 10x the transactions ETH is… and isn’t even breaking a sweat. This report was paid for by ETH interests, used old crappy hardware for the testing and is completely at odds with reality. The reality is that EOS is the fastest, most scalable and most used blockchain on the planet. See for yourself:

https://blocktivity.info/

Bobby

Looks like WAX has EOS beat.

anon

WAX is an eosio software chain. So EOS continues to dominate.

Billy

doesn’t change the fact that its a glorified database and not a blockchain. No cryptographic verification either… Just swell.