nftstorage / carbites Goto Github PK

View Code? Open in Web Editor NEW

24.0 2.0 7.0 330 KB

🚗 🚙 🚕 Chunking for CAR files. Split a single CAR into multiple CARs.

License: Other

JavaScript 100.00%

car cid ipld multiformats chunking splitting

carbites's People

Contributors

Stargazers

Watchers

Forkers

mehulagg i-mortals mervick fairhopeweb intenex pynchmeister aschmahmann

carbites's Issues

Add @ipld/dag-json encoder by default

Hi! It would be great if @ipld/dag-json was added by default on the encoders list, just like raw and cbor are. dag-json should be a first-class citizen in IPFS 😃

Exception: Cannot find module 'carbites/treewalk'

Os: Ubuntu 16.0.4
Nodejs version: 14.0.0

Desired behavior: I'm looking to uploading some NFT images using nft.storage client library from Nodejs.
What I have done:

// some code omitted
const uploadImageToIPFS = async (
  path,
  name,
) => {
  const API_TOKEN = process.env.API_TOKEN
  const client = new NFTStorage({ token: API_TOKEN })
  try {

    const file = fs.promises.readFile(path)
    const metadata = await client.store({
      name: 'Maasai Morans',
      description: `Hello, all the from Maasai Mara. Here come the morans`,
      image: new File(
        [file],
        `${name}.jpg`,
        { type: 'image/jpg' }
      ),
    })

    console.log(metadata)
    return {
      url: metadata.url,
      ipnft: metadata.ipnft,
      ipfsImage: metadata.data.image.href,
      hostedImage: metadata.embed().image.href
    }

  } catch (ex) {
    console.log(ex)
  }
}

Output:

events.js:292
      throw er; // Unhandled 'error' event
      ^

TypeError [ERR_INVALID_ARG_TYPE]: The first argument must be of type string or an instance of Buffer. Received an instance of Uint8Array
    at write_ (_http_outgoing.js:679:11)
    at ClientRequest.write (_http_outgoing.js:644:15)
    at Readable.ondata (_stream_readable.js:707:22)
    at Readable.emit (events.js:315:20)
    at addChunk (_stream_readable.js:296:12)
    at readableAddChunk (_stream_readable.js:272:9)
    at Readable.push (_stream_readable.js:213:10)
    at next (internal/streams/from.js:51:27)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
Emitted 'error' event on Readable instance at:
    at emitErrorNT (internal/streams/destroy.js:96:8)
    at emitErrorCloseNT (internal/streams/destroy.js:68:3)
    at processTicksAndRejections (internal/process/task_queues.js:84:21) {
  code: 'ERR_INVALID_ARG_TYPE'
}

Output: (Stepping through the code using a debuger):
Process exited with code 1 Uncaught Error: Cannot find module 'carbites/treewalk
I can't seem to get this work. Any kind of help is highly appreciated.

Thank you

Tree walk splitter can throw error getting the block links

I got this error while splitting arbitrary large car Files (over 100mb)

RangeError: Too many properties to enumerate
      at Function.entries (<anonymous>)
      at links (/Users/vsantos/gh/nft.storage/node_modules/carbites/node_modules/multiformats/cjs/src/block.js:19:37)
      at links.next (<anonymous>)
      at TreewalkCarSplitter._cars (/Users/vsantos/gh/nft.storage/node_modules/carbites/cjs/lib/treewalk/splitter.js:94:22)
      at _cars.next (<anonymous>)
      at TreewalkCarSplitter._cars (/Users/vsantos/gh/nft.storage/node_modules/carbites/cjs/lib/treewalk/splitter.js:95:24)
      at _cars.next (<anonymous>)
      at TreewalkCarSplitter.cars (/Users/vsantos/gh/nft.storage/node_modules/carbites/cjs/lib/treewalk/splitter.js:54:22)
      at cars.next (<anonymous>)

Research how to speed up gap fill

feat: Allow migration of upload history from one account to another

Getting error when trying to install carbites

codespace ➜ .../packages/client/examples/node.js (main ✗) $ npm install carbites
npm WARN deprecated [email protected]: Please see https://github.com/lydell/urix#deprecated
npm WARN deprecated [email protected]: https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated [email protected]: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated [email protected]: support for ECMAScript is superseded by `uglify-js` as of v3.13.0
npm ERR! code ENOENT
npm ERR! syscall chmod
npm ERR! path /workspaces/nft.storage/packages/client/examples/node.js/node_modules/carbites/bin/carbites.js
npm ERR! errno -2
npm ERR! enoent ENOENT: no such file or directory, chmod '/workspaces/nft.storage/packages/client/examples/node.js/node_modules/carbites/bin/carbites.js'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent 

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/codespace/.npm/_logs/2021-06-23T18_14_58_379Z-debug.log

2021-06-23T18_14_58_379Z-debug.log

Error: Cannot find module 'carbites/treewalk'

Node version 12.3.1
running on windows ...

very barebones project with just nft storage installed

Thanks for any help.

Guardrails around Pinning Service API

There needs to be guardrails around using the Pinning Service API. This epic will track what we want to implement / discussion for the holistic solution.