Participation has dropped again now to at times 25% or so as nodes kick each other out for thinking everyone else is in the wrong fork except for them.
As it happens it is probable they’re all in the wrong fork because the network has experienced a total failure and therefore it has broken down at the protocol level with “extra-protocol measures” most likely needed now as they explain themselves in their fork-choice documentation.
As such, this insta crash on finalization should have been expected you’d think, but the eth 2 coordinator Danny Ryan appears to have thought otherwise, while Vitalik Buterin is kind enough to just sit and watch despite ostensibly designing this whole thing.
Jordan however says “there was no disagreement on the finalized block between nodes. The peer disconnections were due to a different bug at the time of finality that had to do with inefficiencies in processing at the database layer. There is a fix in progress in prysm and in teku as we speak.”
Apparently a bit of code to migrate 30k slots worth of states to the database failed.
“With the order of logic here, we had updated our finalized checkpoint in memory but not updated it on the database.
When Prysm verified status messages from peers, it chekced that the block was finalized in the database which it was not marked as such because of the above error,” Van Loon says.
So maybe this is just simple bugs instead of ‘how do nodes know what fork,’ but what it is exactly remains to be seen depending on how it is resolved.