Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: streaming sha256 CAR hash #162

Merged
merged 1 commit into from
Jan 22, 2024
Merged

Conversation

alanshaw
Copy link
Member

In leu of multiformats/js-multiformats#261 landing, this PR uses the nodejs crypto library to create a streaming sha256 hasher.

Before:

$ ipfs-car hash snow.car 
node:internal/fs/promises:516
    throw new ERR_FS_FILE_TOO_LARGE(size);
          ^

RangeError [ERR_FS_FILE_TOO_LARGE]: File size (11998084756) is greater than 2 GiB
    at readFileHandle (node:internal/fs/promises:516:11)
    at async Module.hash (file:///usr/local/lib/node_modules/ipfs-car/cmd/hash.js:13:13) {
  code: 'ERR_FS_FILE_TOO_LARGE'
}

Node.js v20.11.0

After:

$ ipfs-car hash snow.car
bagbaierabdsb7q55nozng74qw267etolakl4jwrszyjb5elw43zercwmowva

(Note: snow.car is an 11GB CAR)

resolves #158

Copy link
Contributor

@olizilla olizilla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OH NICE YES PLZ

@alanshaw alanshaw merged commit f1601a0 into main Jan 22, 2024
8 checks passed
@alanshaw alanshaw deleted the feat/streaming-sha256-hash branch January 22, 2024 11:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ipfs-car cannot get hash of large car file it created
2 participants