network + dht + bitswap fixes

CONTRIBUTE.md

@@ -2,6 +2,13 @@
 
 There are many ways to contribute to IPFS development. This document will flesh those out in time. For now,
 
+### Necesary "we sadly live in a draconian world" notice:
+
+Please note that IPFS is Open Source and all idea contributions made (ideas + work) will be taken as Open Source contributions.
+
+To make sure we all abide by these rules, I've setup a [Contributor's License Agreement](http://en.wikipedia.org/wiki/Contributor_License_Agreement) that you should sign before contributing to this repository. I am not a lawyer, and am new to all the legal concerns here, so if anything rubs you the wrong way, please let me know by direct email or by opening an issue.
+
+
 ### Help with the design
 
 If you find an error or have a suggestion for the overall design of IPFS, please submit an Issue.

papers/ipfs-cap2pfs/ipfs-cap2pfs.tex

@@ -175,9 +175,33 @@ Version Control Systems provide facilities to model files changing over time and
 
 IPFS is a distributed file system which synthesizes successful ideas from previous peer-to-peer sytems, including DHTs, BitTorrent, Git, and SFS. The contribution of IPFS is simplifying, evolving, and connecting proven techniques into a single cohesive system, greater than the sum of its parts. IPFS presents a new platform for writing and deploying applications, a new system for distributing and versioning large data, and could evolve the web itself.
 
-\subsection{IPFS Nodes}
+IPFS is peer-to-peer; no nodes are privileged. IPFS nodes store IPFS objects in local storage. Nodes connect to each other and transfer objects. These objects represent files and other data structures. The IPFS Protocol is divided into a stack of sub-protocols responsible for different functionality:
 
-IPFS is peer-to-peer; no nodes are privileged. Nodes are identified by a \texttt{NodeId}, the cryptographic hash of a public-key (note that \textit{checksum} will henceforth refer specifically to cryptographic hashes of an object), created as in \cite{skademlia}. Nodes store their public and private keys. Users are free to instatiate a ``new'' node identity on every launch, though that loses accrued network benefits. Nodes are incentivized to remain the same.
+\begin{enumerate}
+  \item \textbf{Identities} - manages node identity generation and verification. Described in Section 3.1.
+
+  \item \textbf{Network} - manages connections to other peers, using various underlying network protocols. Configurable. Described in Section 3.2.
+
+  \item \textbf{Routing} - maintains information to locate specific peers and objects. Responds to both local and remote queries. Defaults to a DHT, but is swappable. Described in Section 3.3.
+
+  \item \textbf{Exchange} - a block exchange protocol (BitSwap) that governs efficient block distribution. Modelled as a market, weakly intentivizes replication. Trade Strategies swappable. Described in Section 3.4.
+
+  \item \textbf{Objects} - a Merkle DAG of content-addressed immutable objects with links. Used to represent arbitrary datastructures, e.g. file hierarchies and communication systems. Described in Section 3.5.
+
+  \item \textbf{Files} - versioned file system hierarchy, inspired by Git. Described in Section 3.6.
+
+  \item \textbf{Naming} - A self-certifying mutable name system. Described in Section 3.7.
+\end{enumerate}
+
+
+These subsystems are not independent; they are integrated and leverage
+blended properties. However, it is useful to describe them separately,
+building the protocol stack from the bottom up.
+
+
+\subsection{Identities}
+
+Nodes are identified by a \texttt{NodeId}, the cryptographic hash\footnote{throughout this document, \textit{hash} and \textit{checksum} refer specifically to cryptographic hash checksums of data} of a public-key, created as in \cite{skademlia}. Nodes store their public and private keys (encrypted with a passphrase). Users are free to instatiate a ``new'' node identity on every launch, though that loses accrued network benefits. Nodes are incentivized to remain the same.
 
 \begin{verbatim}
       type Checksum string
@@ -192,46 +216,36 @@ IPFS is peer-to-peer; no nodes are privileged. Nodes are identified by a \texttt
       }
 \end{verbatim}
 
-IPFS nodes store IPFS objects (which represent files and other data structures) in local storage. Nodes transfer objects to each other. The IPFS Protocol is divided into a stack of sub-protocols responsible for different functionality:
-
-\begin{enumerate}
-  \item \textbf{Network} - manages connections to other peers, using various underlying network protocols. Configurable. Described in Section 2.2.
-
-  \item \textbf{Routing} - maintains information to locate specific peers and objects. Responds to both local and remote queries. Defaults to a DHT, but is swappable. Described in Section 2.3.
-
-  \item \textbf{Exchange} - a block exchange protocol (BitSwap) that governs efficient block distribution. Modelled as a market, weakly intentivizes replication. Trade Strategies swappable. Described in Section 2.4.
-
-  \item \textbf{Objects} - a Merkle DAG of content-addressed immutable objects with links. Used to represent arbitrary datastructures, e.g. file hierarchies and communication systems. Described in Section 2.5.
+Upon first connecting, peers exchange public keys, and check: \texttt{hash(other.PublicKey) equals other.NodeId}. If not, the connection is terminated.
 
-  \item \textbf{Files} - a versioned file-system inspired by Git. Described in Section 2.6.
-
-  \item \textbf{Naming} - A self-certifying mutable name system. Described in Section 2.7.
-\end{enumerate}
+\subsection{Network}
 
+IPFS nodes communicate regualarly with hundreds of other nodes in the network, across the wide internet. IPFS can use any reliable transport protocol, and is best suited for WebRTC DataChannels \cite{WebRTC} (for browser connectivity) or uTP \cite{uTP} (LEDBAT) \cite{LEDBAT}. IPFS also uses the ICE NAT traversal techniques \cite{ICE} to increase connectivity between peers.
 
-These subsystems are not independent. They are well integrated and leverage
-their blended properties. However, it is useful to describe them separately,
-building the protocol stack from the bottom up.
-
-\subsection{Connectivity}
-
-IPFS Nodes communicate regualrly with hundreds of other nodes in the network across the wide internet. It can use any reliable transport protocol, and it is best suited for LEDBAT \ref{LEDBAT} (uTP) \ref{uTP}. IPFS also uses the ICE NAT traversal techniques \ref{ICE} to increase connectivity between peers.
+\begin{itemize}
+  \item \textbf{Reliability:} IPFS can provide reliability if underlying networks do not provide it, using uTP or SCTP.
+  \item \textbf{Integrity:} optionally checks integrity of messages using a hash checksum.
+  \item \textbf{Authenticity:} optionally checks authenticity of messages using HMAC with sender's public key.
+\end{itemize}
 
 \subsection{Routing}
 
-IPFS nodes maintain a DSHT based on S/Kademlia and Coral, to coordinate
-and identify which nodes can serve a particular block of data.
-
-
-\subsubsection{IPFS DSHT}
-
-Instead, IPFS stores a list of peers that can provide the data block.
+IPFS nodes require a routing system that can find (a) other peers' network addresses and (b) peers who can serve particular objects. IPFS achieves this using a DSHT based on S/Kademlia and Coral, using the properties discussed in 2.1. The size of objects and use patterns of IPFS are similar to Coral \cite{Coral} and Mainline \cite{Mainline}, so references are stored in the DHT instead of entire blocks. References are the \texttt{NodeIds} of peers who can serve the block.
 
+The interface of this DSHT is:
 
+\begin{verbatim}
+    routing.findPeer(NodeId)
+    // gets a particular peer's network address
 
-The IPFS DSHT supports four RPC calls:
+    routing.findValuePeers(Checksum, int)
+    // gets a number of peers serving a value.
 
+    routing.provideValue(Checksum)
+    // announces that this node can serve a value.
+\end{verbatim}
 
+Note: different use cases will call for substantially different routing systems (e.g. DHT in wide network, static HT in local network). Thus the IPFS routing system can be swapped for one to fit the users' needs. As long as the interface above is met, the rest of the system will continue to function.
 
 \subsection{Block Exchange - BitSwap Protocol}
 
@@ -245,24 +259,24 @@ blocks could come from completely unrelated files in the filesystem.
 But nodes come together to barter in the marketplace.
 
 While the notion of a barter system implies a virtual currency could be
-created, this would require a global ledger (blockchain) to track ownership
-and transfer of the currency. This will be explored in a future paper.
+created, this would require a global ledger to track ownership
+and transfer of the currency. This can be implemented as a BitSwap Strategy, and will be explored in a future paper.
 
-Instead, BitSwap nodes have to provide direct value to each other
+In the base case, BitSwap nodes have to provide direct value to each other
 in the form of blocks. This works fine when the distribution of blocks across
-nodes is such that they have the complements, what each other wants. This will
+nodes is such that they have complements, what each other wants. This will
 seldom be the case. Instead, it is more likely that nodes must \textit{work}
 for their blocks. In the case that a node has nothing that its peers want (or
-nothing at all), it seeks the pieces its peers might want, with lower
-priority. This incentivizes nodes to cache and disseminate rare pieces, even
-if they are not interested in them directly.
+nothing at all), it seeks the pieces its peers want, with lower
+priority than what the node wants itself. This incentivizes nodes to cache and
+disseminate rare pieces, even if they are not interested in them directly.
 
 \subsubsection{BitSwap Credit}
 
 The protocol must also incentivize nodes to seed when they do not need
 anything in particular, as they might have the blocks others want. Thus,
-BitSwap nodes send blocks to their peers, optimistically expecting the debt to
-be repaid. But, leeches (free-loading nodes that never share) must be avoided. A simple credit-like system solves the problem:
+BitSwap nodes send blocks to their peers optimistically, expecting the debt to
+be repaid. But, leeches (free-loading nodes that never share) must be protected against. A simple credit-like system solves the problem:
 
 \begin{enumerate}
   \item Peers track their balance (in bytes verified) with other nodes.
@@ -277,24 +291,18 @@ from trying to game the probability by just causing more dice-rolls.
 
 \subsubsection{BitSwap Strategy}
 
-The differing strategies that BitSwap peers might employ have wildly different
-effects on the performance of the exchange as a whole. In BitTorrent,
-while a standard strategy is specified (tit-for-tat), a variety of others have
-been implemented, ranging from BitTyrant (sharing the least-possible),
-to BitThief (exploiting a vulnerability and never share), to PropShare
-(sharing proportionally). A range of strategies (good and malicious) could
-similarly be implemented by BitSwap peers. The choice of function, then, should
-aim to:
+The differing strategies that BitSwap peers might employ have wildly different effects on the performance of the exchange as a whole. In BitTorrent, while a standard strategy is specified (tit-for-tat), a variety of others have been implemented, ranging from BitTyrant \cite{BitTyrant} (sharing the least-possible), to BitThief \cite{BitThief} (exploiting a vulnerability and never share), to PropShare \cite{PropShare} (sharing proportionally). A range of strategies (good and malicious) could similarly be implemented by BitSwap peers. The choice of function, then, should aim to:
 
 \begin{enumerate}
   \item maximize the trade performance for the node, and the whole exchange
   \item prevent freeloaders from exploiting and degrading the exchange
-  \item be effective with and resistant to other, unknown strategies
+  \item be effective with and resistant to other, unknown
+  strategies
   \item be lenient to trusted peers
 \end{enumerate}
 
 The exploration of the space of such strategies is future work.
-One choice of function that works in practice is the sigmoid, scaled by a
+One choice of function that works in practice is a sigmoid, scaled by a
 \textit{debt retio}:
 
 Let the \textit{debt ratio} $ r $ between a node and its peer be:
@@ -363,21 +371,10 @@ improve.
 
 \subsubsection{BitSwap Ledger}
 
-BitSwap nodes keep ledgers accounting the transfers with other nodes.
-A ledger snapshot contains a pointer to the previous snapshot (its checksum),
-forming a hash-chain. This allows nodes to keep track of history, and to avoid
-tampering. When activating a connection, BitSwap nodes exchange their ledger
-information.
-If it does not match exactly, the ledger is reinitialized from scratch,
-loosing the accrued credit or debt.  It is possible for malicious nodes to
-purposefully ``loose'' the Ledger, hoping the erase debts. It is unlikely that
-nodes will have accrued enough debt to warrant also losing the accrued trust,
-however the partner node is free to count it as \textit{misconduct} (discussed
-later).
+BitSwap nodes keep ledgers accounting the transfers with other nodes. This allows nodes to keep track of history, and to avoid tampering. When activating a connection, BitSwap nodes exchange their ledger information. If it does not match exactly, the ledger is reinitialized from scratch, loosing the accrued credit or debt.  It is possible for malicious nodes to purposefully ``loose'' the Ledger, hoping the erase debts. It is unlikely that nodes will have accrued enough debt to warrant also losing the accrued trust, however the partner node is free to count it as \textit{misconduct} (discussed later).
 
 \begin{verbatim}
       type Ledger struct {
-        parent     Checksum
         owner      NodeId
         partner    NodeId
         bytes_sent int
@@ -396,41 +393,41 @@ useful ledgers: the old (peers may not exist anymore) and small.
 BitSwap nodes follow a simple protocol.
 
 \begin{verbatim}
-      # Additional state kept:
-      type BitSwap struct {
-        ledgers map[NodeId]Ledger
-        // Ledgers known to this node, inc inactive
-
-        active map[NodeId]Peer
-        // currently open connections to other nodes
-
-        need_list []Checksum
-        // checksums of blocks this node needs
-
-        have_list []Checksum
-        // checksums of blocks this node has
-      }
-
-      type Peer struct {
-        nodeid NodeId
-        ledger Ledger
-        // Ledger between the node and this peer
-
-        last_seen Timestamp
-        // timestamp of last received message
-
-        want_list []Checksum
-        // checksums of all blocks wanted by peer
-        // includes blocks wanted by peer's peers
-      }
-
-      # Protocol interface:
-      interface Peer {
-        open (nodeid :NodeId, ledger :Ledger);
-        send_want_list (want_list :WantList);
-        send_block (block :Block) -> (complete :Bool);
-        close (final :Bool);
-      }
+    // Additional state kept
+    type BitSwap struct {
+      ledgers map[NodeId]Ledger
+      // Ledgers known to this node, inc inactive
+
+      active map[NodeId]Peer
+      // currently open connections to other nodes
+
+      need_list []Checksum
+      // checksums of blocks this node needs
+
+      have_list []Checksum
+      // checksums of blocks this node has
+    }
+
+    type Peer struct {
+      nodeid NodeId
+      ledger Ledger
+      // Ledger between the node and this peer
+
+      last_seen Timestamp
+      // timestamp of last received message
+
+      want_list []Checksum
+      // checksums of all blocks wanted by peer
+      // includes blocks wanted by peer's peers
+    }
+
+    // Protocol interface:
+    interface Peer {
+      open (nodeid :NodeId, ledger :Ledger);
+      send_want_list (want_list :WantList);
+      send_block (block :Block) -> (complete :Bool);
+      close (final :Bool);
+    }
 \end{verbatim}
 
 
@@ -505,7 +502,8 @@ A peer connection should be closed under two conditions:
 \begin{itemize}
   \item a \texttt{silence\_wait} timeout has expired without receiving any
         messages from the peer (default BitSwap uses 30 seconds).
-        In this case, the node issues a \texttt{Peer.close(false)} message.
+        In this case, the node issues a \\
+        \texttt{Peer.close(false)} message.
   \item the node is exiting and BitSwap is being shut down.
         In this case, the node issues a \texttt{Peer.close(true)} message.
 \end{itemize}