Researchers Propose a Decentralized Search Engine Built on Ethereum and IPFS – Trustnodes

Researchers Propose a Decentralized Search Engine Built on Ethereum and IPFS


“I suggest we need a bold goal, one that is understandable and achievable. Something that we might be able to rally around, and have multiple groups contribute to, in order to build an easy to use Distributed Web.”

So said three years ago Jeff Kaplan from the Internet Archive in a call to arms to build a decentralized web (dweb). While the internet upon which the web is built is decentralized, the web itself is centralized, he says, adding:

“Any particular website has only one physical location. Therefore, if the hardware in that particular location is down then no one can see that website. In this way, the Web is centralized: if someone controls the hardware of a website or the communication line to a website, then they control all the uses of that website.”

He says a number of new technologies allow in combination the creation of a decentralized web where privacy is possible, censorship is difficult, and the modern day library of Alexandria remains open to all.

Now three years later, the idea seems to have spread, with Mozilla recently introducing the decentralized web. While researchers from the University of Hong Kong say a decentralized web needs a decentralized search engine.

Google might not directly be in sight, but many now think the global giant has far too much power over human knowledge, and as all know, power corrupts. They could for example, perhaps without anyone noticing, remove billions of pages from their index. Sending them to an unfoundable place without anyone knowing such place even exists.

They could, likewise, send highly informative but perhaps slightly edgy content to the 100th page where almost no one goes. They probably don’t, not so obviously anyway and probably not without the content being a soft target, but if they are doing so more subtly, would anyone know?

That’s just one of many problems with monopoly like centralized services, which in part are a product of a centralized web structure, starting with domain names which now can run on the Ethereum Name Service.

The Hong Kong researchers think search can run on ethereum too. In a very high level overview they outline the design as follows:

“Queen-Bee incentivizes content creators to publish (create or update) their contents via QueenBee’s smart contract to gain ‘honey’ in the form of a cryptocurrency.

Honey is also rewarded to worker bees peers that help update the index and compute the page ranks, which are hosted in a decentralized storage (e.g., IPFS).

Users submit their keyword queries via QueenBee’s HTML+Javascript frontend on the DWeb. The frontend is also responsible for composing the search results by intersecting the matched inverted lists, ranking the results, and displaying relevant ads.

QueenBee is a decentralized organization – advertisers directly make advertisements through our smart contract and the ad revenue is shared among the content creators and worker bees.

QueenBee’s decentralized nature rids itself of problematic issues (e.g., search bias) found in centralized search engines.

To our best knowledge, QueenBee is the world first initiative to build a decentralized search engine on the DWeb. Existing P2P search engines (e.g., YaCy ) only work on Web 2.0, without an incentive scheme or a security incentive that guard against practical attacks.”

Queenbee’s architecture.

They have a prototype, acting as a search engine of IPFS. Naturally we tried it, and it is a bit slow but as this is peer to peer technology speed is based on how many peers are “connected” and hold the distributed data.

It does however work, although it looks like IPFS doesn’t know about Trustnodes. They have a mirror of Wikipedia, but we’re clearly not important enough for that. So we went with “trust search” to receive these results:

IPFS search results by Queenbee.

We did try to find out more, but the Queenbee name is so generic to the point Google knew nothing. So we do not know what is the smart contract address of this search engine or how exactly it operates underneath.

The idea itself, however, is interesting at a high level. Basically the design suggests this takes Google out of the equation by having a self-organizing system where advertisers pay content producers and content curators directly.

That may have problems of its own, such as collusion attacks, whereby curators conspire. The researchers point this and a few other problems, but do not address them further.

That may be because this is very much just an introduction of the project with it at the prototype stage, but the experiment is interesting, especially if it is open source and especially if they find a way to give some of that advertising money to coders.

Then you’d effectively have a Decentralized Autonomous Organization (DAO), a collective of sorts where all are both communists and capitalist in that there is no concentrated capital in one individual, but is distributed through the code based on capitalist incentives.

Whether it will work remains to be seen, but the decentralized web is now starting to become a thing as numerous technologies allow in combination some very new inventions.



Comments (3)

  1. Presearch was the first they have a working product and a great team PRE tokens are on eth and you can earn them on their platform

    1. I just tried Presearch… it is just a tool navigating me to Google, duckduckgo… they say it is a decentralized search engine???

    2. Presearch team looks have no idea about how to realize their vision…they are just building a community and pray the community can magically implement the product for them… I would say I trust QueenBee researchers more

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>