Does unixfs leverage webrtc peers?

[sleep9]

After establishing a webrtc connection though a circuit relay I am trying to fetch a CID obtained via gossip. Do I need to implement cid retrieval with my own webrtc protocol? It seems unixfs does not seem to know about webrtc peers. I would prefer if the gossip was "normalized" by sending the cid of the intended message rather than the entire message itself.

`host.services.pubsub.addEventListener('message', async (message) => {

  let topic = `${message.detail.topic}`;
  if(hasTopic(topic)) {

    let messageString = uint8ArrayToString(message.detail.data);
    let parsed = null;
    try {
      parsed = JSON.parse(messageString)
    } catch(error:any) {
      console.log('msgparseerror: ',error);
      return;
    }

    const cid = CID.parse(parsed.cid)
    const decoder = new TextDecoder()    
    let text = ''
    for await (const chunk of ipfs_fs.cat(cid)) { <--- does not return
        text += decoder.decode(chunk, {
            stream: true
        })
    }

})`

[SgtPooki]

If you're using helia with libp2p, unixfs gets a {blockstore} component from helia that is networked: i.e. it's using blockBrokers (trustless gateways and bitswap) and bitswap will utilize any enabled transports.

When you say "fetch a CID obtained via gossip" do you mean that someone publishes a message with the CID string, and the then the code above is where you handle it? It seems like that's the case from your above code. In that case, you would want to call something like helia.blockstore.get(CID) to get the raw content or unixfs.(cat|ls|stat|etc) to get the blocks decoded by unixfs.

One common problem folks run into when using unixfs.cat (what you're doing above) is that cat doesn't have a timeout by default and users end up getting stuck at that async iterable. To help you figure out what is going on I would recommend the following:

1. Pass a const signal = AbortSignal.timeout(<time in milliseconds>) via ipfs_fs.cat(cid, { signal }
2. Also pass onProgress events so you can see what's happening: ipfs_fs.cat(cid, {signal, onProgress: (evt) => { myLogger(evt.type, evt?.detail?.detail ?? evt?.detail))
3. Also enable logging with localStorage.setItem('debug', 'helia*,helia*:trace') and fine tune into the areas you need to see more logs of.

The likely story though, is that the CID you're receiving is not highly available on the network. you should be able to verify this at https://pl-diagnose.on.fleek.co/

[sleep9]

Thanks for the reply. You are correct describing what my code is doing.

The node publishing the content is connected to the server which is in turn attempting to fetch the cid in return. I am using a private network so http trustless gateways are not an option. My transports are websockets and webRTC and the client is a react-native app. If I use the same code/process with a node.js client it works. Any ideas?

evt.type:  blocks:get:providers:get
onProgress:  CID(bafkreibkazdonagdhsozokwx4fw3osnst2qrfyp7gabpi6t3zubludtaoq)
evt.type:  bitswap:network:find-providers
onProgress:  CID(bafkreibkazdonagdhsozokwx4fw3osnst2qrfyp7gabpi6t3zubludtaoq)
evt.type:  bitswap:network:dial
onProgress:  PeerId(12D3KooWH1Bf3L8dXKvtCXQ8E8opAdxJSV3WZH4fwmPqickT6T63)
evt.type:  bitswap:network:send-wantlist
onProgress:  PeerId(12D3KooWH1Bf3L8dXKvtCXQ8E8opAdxJSV3WZH4fwmPqickT6T63)
evt.type:  bitswap:network:send-wantlist:error
onProgress:  {
  peer: PeerId(12D3KooWH1Bf3L8dXKvtCXQ8E8opAdxJSV3WZH4fwmPqickT6T63),
  error: Error: Cannot push value onto an ended pushable
      at QueuelessPushable._push (/opt/app/relay-test/node_modules/it-byte-stream/src/pushable.ts:98:13)
      at QueuelessPushable.push (/opt/app/relay-test/node_modules/it-byte-stream/src/pushable.ts:84:16)
      at duplex.sink (/opt/app/relay-test/node_modules/it-byte-stream/src/index.ts:87:19)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at Network._writeMessage (/opt/app/relay-test/node_modules/ipfs-bitswap/src/network.ts:293:7)
      at Network.sendMessage (/opt/app/relay-test/node_modules/ipfs-bitswap/src/network.ts:246:5)
}

[sleep9]

Issue resolved by setting up direct RTC connection for bitswap. Was previously using p2p-circuit which did not seem to work.