Search 250 million research papers

Distributed Hash Table

Hao Zhang, Yonggang Wen, Haiyong Xie et al. · SpringerBriefs in computer science · 2013

Cited by 31

Distributed Hash Table

2013

Cited by 1

Stealth distributed hash table

Andrew Brampton, Andrew MacQuire, Azlina Idris et al. · Open Repository and Bibliography (University of Liège) · 2006

Most Distributed Hash Tables (DHTs) simply consider interconnecting homogeneous nodes on the same overlay. However, realistically nodes on a network are heterogeneous in terms of their capabilities. Because of this, traditional DHTs have been shown to exhibit poor performance in a real-world environment. Additionally, we believe that it is this approach that contributes to a limited exploitation of peer-to-peer technologies. Previous work on super-peers in DHTs was proposed to address these performance issues, however the strategy used is often based on locally clustering peers around individual super-peers. This method of super-peering, however, compromises fundamental features such as load-balancing, resilience and routing efficiency, which traditional DHTs originally promised to offer.

Cited by 15Open access

A distributed hash table

M. Frans Kaashoek, Robert Morris, Frank Dabek · 2006

DHash is a new system that harnesses the storage and network resources of computers distributed across the Internet by providing a wide-area storage service, DHash. DHash frees applications from re-implementing mechanisms common to any system that stores data on a collection of machines: it maintains a mapping of objects to servers, replicates data for durability, and balances load across participating servers. Applications access data stored in DHash through a familiar hash-table interface: put stores data in the system under a key; get retrieves the data. DHash has proven useful to a number of application builders and has been used to build a content-distribution system [31], a Usenet replacement [115], and new Internet naming architectures [130, 129]. These applications demand low-latency, high-throughput access to durable data. Meeting this demand is challenging in the wide-area environment. The geographic distribution of nodes means that latencies between nodes are likely to be high: to provide a low-latency get operation the system must locate a nearby copy of the data without traversing high-latency links. Also, wide-area network links are likely to be less reliable and have lower capacities than local-area network links: to provide durability efficiently the system must minimize the number of copies of data items it sends over these limited capacity links in response to node failure. This thesis describes the design and implementation of the DHash distributed hash table and presents algorithms and techniques that address these challenges. DHash provides low-latency operations by using a synthetic network coordinate system (Vivaldi) to find nearby copies of data without sending messages over high-latency links. A network transport (STP), designed for applications that contact a large number of nodes, lets DHash provide high throughput by striping a download across many servers without causing high packet loss or exhausting local resources. Sostenuto, a data maintenance algorithm, lets DHash maintain data durability while minimizing the number of copies of data that the system sends over limited-capacity links. (Copies available exclusively from MIT Libraries, Rm. 14-0551, Cambridge, MA 02139-4307. Ph. 617-253-5668; Fax 617-253-1690.)

Cited by 45

A Distributed Hash Table

Dabek, Frank (Frank Edward), 1977- · DSpace@MIT (Massachusetts Institute of Technology) · 2005

DHash is a new system that harnesses the storage and network resources of computers distributed across the Internet by providing a wide-area storage service, DHash. DHash frees applications from re-implementing mechanisms common to any system that stores data on a collection of machines: it maintains a mapping of objects to servers, replicates data for durability, and balances load across participating servers. Applications access data stored in DHash through a familiar hash-table interface: put stores data in the system under a key; get retrieves the data. DHash has proven useful to a number of application builders and has been used to build a content-distribution system [31], a Usenet replacement [115], and new Internet naming architectures [130, 129]. These applications demand low-latency, high-throughput access

Cited by 0Open access

Poster: Distributed Learned Hash Table

Shengze Wang, Yi Liu, Xinshu Zhang et al. · 2024

Distributed Hash Tables (DHTs) are pivotal in numerous high-impact key-value applications built on distributed networked systems, offering a decentralized architecture that avoids single points of failure and improves data availability. Despite their widespread utility, DHTs face substantial challenges in handling range queries, which are crucial for applications such as storage systems, decentralized databases, content distribution networks, and blockchains. To address this limitation, we present LEAD, a novel system incorporating learned models within DHT structures to significantly optimize range query performance. LEAD utilizes a recursive machine learning model to map and retrieve data across a distributed system while preserving the inherent order of data. Preliminary results indicate LEAD achieves tremendous advantages in system efficiency compared to existing range query methods in large-scale distributed systems while maintaining high scalability and resilience to network churn.

Cited by 0

Chord - A Distributed Hash Table

Kurt Tutschku, Yimei Liao, Unknown et al. · State and University Library Dresden (Saxon State Library) · 2006

Cited by 0Open access

Novel distributed Hash table——BChord

ZHENG Yan1, CHEN Hong-wei 1, WANG Ru-chuan1 et al. · SHILAP Revista de lepidopterología · 2008

A novel distributed Hash table（DHT） BChord was presented. BChord adopts bi-directional search mechanism of Chord. Though query items of finger table in BChord almost double to Chord, compared with Chord, BChord evi- dently increases speed at searching of resource and spreading of messages via theoretic proof and simulation results.

Cited by 0Open access

A Distributed Learned Hash Table

Shengze Wang, Yi Liu, Xinshu Zhang et al. · 2025

Distributed Hash Tables (DHTs) are pivotal in numerous high-impact key-value applications built on distributed networked systems, offering a decentralized architecture that avoids single points of failure and improves data availability. Despite their widespread utility, DHTs face substantial challenges in handling range queries, which are crucial for applications such as LLM serving, distributed storage, databases, content delivery networks, and blockchains. To address this limitation, we present LEAD, a novel system incorporating learned models within DHT structures to significantly optimize range query performance. LEAD utilizes a recursive machine learning model to map and retrieve data across a distributed system while preserving the inherent order of data. LEAD includes the designs to minimize range query latency and message cost while maintaining high scalability and resilience to network churn. Our comprehensive evaluations, conducted in both testbed implementation and simulations, demonstrate that LEAD achieves tremendous advantages in system efficiency compared to existing range query methods in large-scale distributed systems, reducing query latency and message cost by 80% to 90%+. Furthermore, LEAD exhibits remarkable scalability and robustness against system churn, providing a robust, scalable solution for efficient data retrieval in distributed key-value systems.

Cited by 2

A Distributed Learned Hash Table

T. Wang, Yi Liu, Xiaoxue Zhang et al. · IEEE Transactions on Networking · 2026

Distributed Hash Tables (DHTs) are pivotal in numerous high-impact key-value applications built on distributed networked systems, offering a decentralized architecture that avoids single points of failure and improves data availability. Despite their widespread utility, DHTs face substantial challenges in handling range queries, which are crucial for applications such as LLM serving, distributed storage, databases, content delivery networks, and blockchains. To address this limitation, we present LEAD, a novel system incorporating learned models within DHT structures to significantly optimize range query performance. LEAD utilizes recursive machine learning models as the Learned Hash Function to map and retrieve data across a distributed system while preserving the inherent order of data. LEAD includes the designs to minimize range query latency and message cost while maintaining high scalability and resilience to network churn. Our comprehensive evaluations, conducted in both testbed implementation and simulations, demonstrate that LEAD achieves tremendous advantages in system efficiency compared to existing range query methods in large-scale distributed systems, reducing query latency and message cost by 80% to 90%+. Furthermore, LEAD exhibits scalability and robustness against system churn, providing a robust, scalable structure for efficient data retrieval in distributed key-value systems.

Cited by 0

Chord - A Distributed Hash Table

Yimei Liao · Qucosa - Monarch (Chemnitz University of Technology) · 2006

An introduction to Chord Algorithm.

Cited by 0Open access

DIPSEA: A MODULAR DISTRIBUTED HASH TABLE

Gurmeet Singh Manku · 2004

Cited by 0

Dipsea: a modular distributed hash table

Rajeev Motwani, Gurmeet Singh Manku · 2004

Dipsea is a modular architecture for building a Distributed Hash Table (DHT). A DHT is a large hash table that is cooperatively maintained by a large number of machines communicating over the Internet. Decentralization and automatic re-configuration are two key design goals for a DHT. The architecture of Dipsea consists of three layers: ID Management, Overlay Routing and Data Management. The Overlay Routing layer consists of three modules: Emulation Engine, Ring Management and Choice of Long-Distance Links. Efficient algorithms for ID Management are designed—these algorithms require few messages, require few re-assignments of existing IDs and ensure that the hash table is divided among the participating machines as evenly as possible. Ring Management ensures that participating machines establish connections among themselves, as a function of their IDs, to form a fault-tolerant ring. The Emulation Engine is responsible for “emulation” of arbitrary families of routing networks. It handles issues arising out of dynamism (arrival and departure of participating machines), scale (variation in the average number of participating machines) and physical network proximity. Choice of Long-Distance Links allows a DHT to choose any family of routing networks (deterministic or randomized) for emulation. Several deterministic and randomized routing networks are designed and analyzed. Among these are Symphony (one of the first randomized DHT routing networks), Papillon (a deterministic routing network that guarantees asymptotically optimal route lengths with greedy routing for a fixed out-degree of nodes), and Mariposa (a randomized routing network that also guarantees optimal route lengths for a given out-degree of nodes).

Cited by 15

Distributed Hash table-based distributed subspace clustering algorithm

Yaowu Chen · 2010

A distributed subspace clustering (DISCLUS) algorithm based on distributed Hash table(DHT) was proposed.Each node executed subspace clustering on its local data.Then the clustering results of nodes were combined to form the final clustering results of the distributed system.The dataset reducing and pruning schemes were proposed to optimize the communication between nodes according to the speciality of subspace clustering.A DHT-based distributed voting (DDV) algorithm was proposed to combine the clustering results of nodes.The algorithm used the topology of the underlying overlay to hierarchically collect the voting information.All the nodes in the system can be covered without redundancy.The theoretical and experimental results show that the clustering error and the communication cost of DISCLUS algorithm are scalable to the dataset,nodes and dimensionality.

Cited by 0

Distributed hash table for BitTorrent client

David Brodský · Digital Repository (National Repository of Grey Literature) · 2007

The goal of this study is exact description of distributed hash table (DHT) that is implemented in BitTorrent client Azureus, its usage as a distributed tracker and comparison with another DHT that is used for the same purpose. This study also includes implementation of interface based on a TCP server that provides easy access to the DHT without need to reimplement it. Specification of communication protocol is included as well.

Cited by 0

A Read-Only Distributed Hash Table

Verdi March, Yong Meng Teo · Journal of Grid Computing · 2011

Cited by 4

Whanau: A Sybil-Proof Distributed Hash Table

Kaashoek, M. Frans, Lesniewski-Laas, Christopher Tur · DSpace@MIT (Massachusetts Institute of Technology) · 2010

Whānau is a novel routing protocol for distributed hash tables (DHTs) that is efficient and strongly resistant to the Sybil attack. Whānau uses the social connections between users to build routing tables that enable Sybil-resistant lookups. The number of Sybils in the social network does not affect the protocol’s performance, but links between honest users and Sybils do.When there are n well-connected honest nodes, Whānau can tolerate up to O(n/ log n) such “attack edges”. This means that an adversary must convince a large fraction of the honest users to make a social connection with the adversary’s Sybils before any lookups will fail. Whānau uses ideas from structured DHTs to build routing tables that contain O([sqrt]n log n) entries per node. It introduces the idea of layered identifiers to counter clustering attacks, a class of Sybil attacks challenging for previous DHTs to handle. Using the constructed tables, lookups provably take constant time. Simulation results, using social network graphs from LiveJournal, Flickr, YouTube, and DBLP, confirm the analytic results. Experimental results on PlanetLab confirm that the protocol can handle modest churn.

Cited by 0Open access

D1HT: a distributed one hop hash table

Luiz Monnerat, Cláudio L. Amorim · 2006

Distributed hash tables (DHTs) have been used in a variety of applications, but most DHTs so far have opted to solve lookups with multiple hops, which sacrifices performance in order to keep little routing information and minimize maintenance traffic. In this paper, we introduce D1HT, a novel single hop DHT that is able to maximize performance with reasonable maintenance traffic overhead even for huge and dynamic peer-to-peer (P2P) systems. We formally define the algorithm we propose to detect and notify any membership change in the system, prove its correctness and performance properties, and present a quarantine-like mechanism to reduce the overhead caused by volatile peers. Our analyses show that D1HT has reasonable maintenance bandwidth requirements even for very large systems, while presenting at least twice less bandwidth overhead than previous single hop DHT

Cited by 54

P2P Systems Based on Distributed Hash Table

Ming Xie · 2003

Peer-to-peer system can share the computing resources and services by communicating directly within a widely distributed network. A fundamental problem that confronts peer-to-peer applications is the efficient location of the node that stores a desired data item. Earlier P2P systems employ a single index server or floodingbased mechanism to search desired data, which are not suitable for large systems. The objective of the presentation is to explain how the latest P2P systems apply distributed hash table (DHT) to support fast data locating and system scalability.

Cited by 0

Data Recovery of Distributed Hash Table with Distributed-to-Distributed Data Copy

Yusuke Doi, Shirou Wakayama, Satoshi Ozaki · IEICE Transactions on Information and Systems · 2009

To realize huge-scale information services, many Distributed Hash Table (DHT) based systems have been proposed. For example, there are some proposals to manage item-level product traceability information with DHTs. In such an application, each entry of a huge number of item-level IDs need to be available on a DHT. To ensure data availability, the soft-state approach has been employed in previous works. However, this does not scale well against the number of entries on a DHT. As we expect 1010 products in the traceability case, the soft-state approach is unacceptable. In this paper, we propose Distributed-to-Distributed Data Copy (D3C). With D3C, users can reconstruct the data as they detect data loss, or even migrate to another DHT system. We show why it scales well against the number of entries on a DHT. We have confirmed our approach with a prototype. Evaluation shows our approach fits well on a DHT with a low rate of failure and a huge number of data entries.

Cited by 0Open access