Skip to content

Latest commit

 

History

History
203 lines (157 loc) · 17.8 KB

README.md

File metadata and controls

203 lines (157 loc) · 17.8 KB
Raw

A resilient system or network is fundamentally uncompromised by an isolated failure or network split. The system is malleable, adaptable to different conditions and capable of evolving to meet new requirements over time. That said, resilience here identifies also as a characteristic against changing system or network conditions, i.e., the system's core operating principles adjust so that performance remains when the system scales up to serve increasing demand from more users. Building resilience into foundational infrastructure is the key in building a computing and networking fabric for human knowledge.

Welcome to the ResNetLab public repository. In this place, you will be able to find information about our Open Problems, RFPs, Open Positions, Research Areas, Research Projects and much more. You can also consult the version on the Protocol Labs Research website

Table of Contents

The ResNetLab

Mission & Vision


The Resilient Networks Lab mission is to build resilient distributed systems, by creating and operating a platform where researchers can collaborate openly and asynchronously on deep technical work.


Motivation & Description

The lab's genesis comes from a need present in the IPFS and libp2p projects to amp their research efforts to tackle the critical challenges of scaling up networks to planet scale and beyond. The Lab is designed to take ownership of the earlier stages on the research pipeline, from ideas to specs and to code.

Research

Areas

Research Area Open Problem(s) Short Description
Resilliency in Adversarial Networks NEEDs OPEN PROBLEM
Networks Observability NEEDs OPEN PROBLEM
Decentralized Data Delivery Markets General open problem Content storage and delivery services have traditionally been under the ownership, control and management of centralised entities. Decentralised storage platforms are emerging and are in operation today. However, without actual delivery of content to users, these networks can be of little value, limited to serve as cold storage solutions only. With this Open Problem, we introduce the field of decentralised delivery, or decentralised CDNs to complement decentralised storage systems and come one step closer to the whole set of solutions that traditional CDNs offer today.
Heterogeneous Runtimes General open problem Making libp2p / IPFS modules and protocols the de-facto distributed substrate for connected devices in the near-future Internet. Enable the execution of libp2p nodes and its underlying protocols anywhere (any device and any runtime). Allow global connectivity between devices. Enable “offline-first” applications.
Content Addressing Routing at Scale (1M, 10M, 100M, 1B.. nodes) Content-addressable networks face the challenge of routing scalability, as the amount of addressable elements in the network rises by several orders of magnitude compared to the host-addressable Internet of today.
Preseve full users' privacy when providing and fetching Content How to ensure that the user's of the IPFS network can collect and provide information while mainting their full anonymity.
Mutability Mutable Data (Naming, Real-Time, Guarantees) Enabling a multitude of different patterns of interactions between users, machines and both. In other words, what are the essential primitives that must be provided for dynamic applications to exist, what are the guarantees they require (consistency, availability, persistancy, authenticity, etc) from the underlying layer in order create powerful and complete applications in the Distributed Web.
Human Readable Naming You can only have two of three properties for a name: Human-meaningful, Secure and/or Decentralized. This is Zooko's Trilemma. Can we have all 3, or even more? Can context related to some data help solve this problem?
PubSub at Scale (1M, 10M, 100M, 1B.. nodes) As the IPFS system is evolving and growing, communicating new entries to the IPNS is becoming an issue due to the increased network and node load requirements. The expected growth of the system to multiple millions of nodes is going to create significant performance issues, which might render the system unusable. Despite the significant amount of related literature on the topic of pub/sub, very few systems have been tested to that level of scalability, while those that have been are mostly cloud-based, managed and structured infrastructures.
Data Exchange Enhanced Bitswap/GraphSync with more Network Smarts Bitswap is a simple protocol and it generally works. However, we feel that its performance can be substantially improved. One of the main factors that hold performance back is the fact that a node cannot request a subgraph of the DAG and results in many round-trips in order to “walk down” the DAG. The current operation of bitswap is also very often linked to duplicate transmission and receipt of content which overloads both the end nodes and the network.
Distributed Type Systems Improved layouts to represent data in hash-linked graphs (using IPLD) Future™ ⚙️

Projects

  • ONGOING

    • 3DMs: Decentralised Data Delivery Markets
    • Content Routing Scale Up
    • P2P Observatory
    • ResNetLab on (Virtual) Tour: We have built a half-day tutorial to introduce the DWeb, the IPFS ecosystem, the IPFS architecture and its supporting protocols, and the high-level design decisions of the Filecoin network. In 2020, we have participated in multiple conferences and other academic events to discuss the exciting projects we're working on and invite great researchers to collaborate with us. The tutorial is primarily composed of lecture material, and many of our tutorials have been very interactive. In 2021, we are enhancing the tutorial with hands-on sessions, so it will be even more exciting for students and researchers with a passion to tinker as they learn. You can find summaries of our talks in the "Publications and Talks" section further down.

  • COMPLETED

    • Hydra Booster: The Hydra booster is a new type of DHT node designed to accelerate the Content Resolution & Content Providing on the IPFS Network. This new type of peer exists to augment the network by creating multiple distributed identities across the DHT address space, enabling it to contribute to the storage and discovery of content provider records. All of these identities are linked by the same backend datastore, which from the other peers’ perspective, creates the effect of multiple peers being present and holding a vast collection of the provider records in the network. The Hydra Booster has been instrumental for the stability and fast content resolution of the IPFS network. Hydra boosters have been designed, implemented and are in operation today in the public IPFS network as of release go-ipfs 0.5. The Hydra booster is open source, lives in this repository and can be deployed by anyone.

    • Gossipsub v1.1: Gossipsub is one of the many libp2p PubSub routers used to disseminate IPNS records in the IPFS network, enable real-time distributed applications, and much more. GossipSub was adopted as a messaging layer by Filecoin and ETH2.0, due to its functionality and fast performance on permissionless networks. This has led us to invest additional effort to protect it against sybil attacks and malicious behavior. Together with the libp2p team, we embarked on a mission to harden the protocol’s behavior. The outcome is a hardened version of the Gossipsub protocol that integrates several mitigation strategies at the protocol level. You can learn more about this project in this blogpost, this paper and this talk at the Matrix.org “Open Tech will save us all” event. Our extensive performance evaluation can be found in this report. The specification of the protocol can be found here.

    • drand: drand is an unbiasable source of randomness which other platforms and applications can publicly verify. Randomness is at the core of many of the security-critical operations we perform online every day, and until 2020, there wasn’t a single reliable and trustworthy source. That changed in 2020 with the drand project. drand is hosted by more than 15 independent members of the League of Entropy, and is available for any project that needs randomness to use. You can find more details in the platform's launch blog post. We organised a whole-day summit to talk about the technical details of drand and distributed randomness. You can watch all the recordings at randomness2020.com.

    • Beyond Bitswap: File-transfer is at the core of IPFS and every subsystem inside IPFS is built to enable it in a fast and secure way, while maintaining certain guarantees (e.g. discoverability, data integrity and so on). The aim of the project was two-fold: to drive speed-ups in file-sharing for IPFS and other P2P networks; and to enable a framework for anyone to join the quest of designing, implementing and evaluating brand new file-sharing strategies in P2P networks. The outcomes of the project are impressive: we produced 10 improvement proposals, many of them prototyped, a complete testbed to benchmark and debug file sharing in IPFS, a paper that provides further details about our design proposals and several blogposts to summarise our results.

RFPs

Collaborations

You can read more about these projects in this blogpost.

Publications, Talks & Trainings

Talks

Papers

Training & Seminars

Team

Contact

You can reach out to us anytime with your question and interest in these projects by emailing resnetlab@protocol.ai