All you need to know before starting your private IPFS network.
Pre-requirements
– Download:
– Install tools:
Exctract files somewhere in SYSTEM PATH depending on your OS.
Init and Run IPFS client
Open your terminal window and type:
ipfs init
This will create your initial configuration including private and public keys, files will be located $USERPROFILE/.ipfs/ this path can be changed using env variable IPFS_PATH=/someother/path
Start Daemon
ipfs daemon &
The “&” symbol put our daemon in background processes, so we can still work in the same terminal window. IPFS daemon command will connect us with public IPFS network, and we can start using it.
Since our node now is taking part of a global IPFS network we can interact with it in a few different ways:
ipfs
This command will show us all basic, data structure, advanced and network commands.
https://medium.com/mvp-workshop/a-closer-look-to-the-inter-planetary-file-system-b3f3af31a3c7
IPFS add and ipfs cat
When we are connected to global ipfs network we can add, remove, read the data on the network.
ipfs add PATH_toFileWeWantToAdd
With this command, we can add any file to IPFS network, and the result will be the hash. Notice that if we add files smaller than 256kb the result will be single hash code. Read more about data structures in my previous article. But what happened when we want to add a bunch of files or let’s say website (directory)?
ipfs add -r PATH_toBigFile
You may notice the -r flag and which stand for recursive and allow us to add the whole directory to IPFS. But we have multiple hashes for each file. To solve this, type -Q or –silent. This will generate one hash for all files together, with which we can access it later.
Now our files are added to IPFS with its unique hash. If someone wants to access your file it can be done simply by typing:
ipfs get <your-ipfs-hash>
and it will automatically download the file to clients pc.
Files are also accessible through a browser and here are two ways: through localhost and public gateway.
When you type:
http://localhost:8080/ipfs/<your-ipfs-hash>
localhost:8080 signify that we are using 8080 local port. Ipfs means we are working with IPFS files and last is the hash of the desired file we want to open.
The second way is to access it through public gateway:
https://gateway.ipfs.io/ipfs/<your-ipfs-hash>
Anyone with the following hash can access the data through the public gateway. The IPFS gateways are maintained by IPFS core team that you can use to access any content from the IPFS network. The list of all public IPFS gateways can be found here.
Pinning
When data is published on IPFS, it becomes publicly available for at least 24h, if not interacted with. The garbage collector tool is used to clear network of unused files. This means that if a file is not used (replicated) for 24h it will be automatically deleted by a garbage collector.
Solution to keeping data in the network is pinning. When we pin a piece of data, we are only telling our node not to garbage collect it. We can pin our files by using:
ipfs pin add <your-ipfs-hash>
IPFS pin function when uploading (ipfs add) your data is by default always true. But what happens when a file that is no longer hosted (pinned) by owner?
Other nodes can download data as long as our node is online. If for any reason your node is offline, there is still a chance that data still will be available through other nodes that have your data. More popular the data is, the availability is increased because we have more seeders, and content will naturally stick around the network.
But it is not efficient to store all the data on your device, which also needs to be online all the time. There are paid pinning services like Pinbits that can be used.
IPNS
As we mentioned earlier, upon any change to file the hash changes and we get a different address, which can cause a problem. That’s where IPNS jumps in. IPNS will give us one address which will refer to other content. So let’s try it, we are going to give our previous hash that we got from IPFS add:
ipfs name publish <your-ipfs-hash>
The result will be another hash which is permanent, and it points to the latest (hash) version of our file.
Accessing IPNS is a little bit different through gateway would look like: gateway.ipfs.io/ipns/”hash”
Now when you edit and publish an old file with IPFS, we can access it thought the same IPNS address. If you like, you can edit your DNS record to anything you like:
dnslink=/ipns/<your_peer_id>
Init Cluster
IPFS cluster is a piece of software that enables coordination between IPFS daemons running on different hosts. There are two ways to connect peers in cluster, one setting a predefined peerset or bootstrapping nodes.
IPFS Cluster peers are run with the ipfs-cluster-service command, more precisely service.json, which manages the cluster behavior, along with the peerstore file which stores the peers multi addresses. Those files can be found inside ~/.ipfs-cluster. Let’s run:
ipfs-cluster-service init
This command will create cluster configuration $USERPROFILE/.ipfs-cluster/service.json
This file will be used later to start a new cluster or join existing one. Which will be explained in the next steps.
IPFS Cluster run on raft consenus algorithm.
(node1) Start your own cluster master node :
ipfs-cluster-service daemon
Important: IPFS daemon must be working also on both nodes.
(node2) Join node to existing cluster:
ipfs-cluster-service daemon — bootstrap <multiaddres>
Multiaddress of master node (node1) server which can be viewed with the following command:
ipfs-cluster-ctl id
Since IPFS cluster uses a raft consensus algorithm, there can be some unexpected behavior due to a problem to elect leader if there are less than 3 nodes in the cluster.
Confirm joining ipfs cluster
ipfs-cluster-ctl peers ls
This command will list all peers inside the cluster and all peers should see the same number of other peers.
Now we have working IPFS cluster. It runs independently from IPFS, using the go-ipfs API to control the IPFS daemon. Cluster peers communicate using libp2p, peers can control the cluster, modify the pinset and perform any operations. New peers can be added, removed so cluster can grow or decrease.
This step guide has been tested with following versions:
- ipfs version == 0.4.15 — ipfs-cluster-service version == 0.4.0 — ipfs-cluster-ctl version == 0.4.0
Feel free to leave a question or comment down below! For more about IPFS data structure check my previous article: A closer look at the Inter-planetary File System!
Hope you enjoyed!
IPFS Publishing and IPFS Cluster was originally published in MVP Workshop on Medium, where people are continuing the conversation by highlighting and responding to this story.
