Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to Upload Files Larger than 2GB to Firebase Storage #2608

Open
kevinrss01 opened this issue Jun 21, 2024 · 2 comments
Open

Unable to Upload Files Larger than 2GB to Firebase Storage #2608

kevinrss01 opened this issue Jun 21, 2024 · 2 comments

Comments

@kevinrss01
Copy link

[REQUIRED] Step 2: Describe your environment

Operating System

  • MAC OS SONOMA 14.5

Browser Version

  • ARC (CHROME)

Firebase SDK Version

  • 10.12.2

Firebase SDK Product:

  • Storage

NPM and NODE:
I'm using nestJS (@10.3.9), nodeJS (v20.11.1) and npm 10.2.4

[REQUIRED] Step 3: Describe the problem

I am experiencing an issue where attempting to upload files larger than 2GB to Firebase Storage results in a GaxiosError: request to https://storage.googleapis.com/upload/storage/v1/b/[...] failed, reason: write EPROTO.

However, smaller files (e.g., 1.8GB) upload without any issues.

Steps to reproduce:

  1. Attempt to upload a file larger than 2GB using the Firebase Storage SDK in a Node.js application.
  2. Encounter the following error:
Failed to upload file. GaxiosError: request to [https://storage.googleapis.com/upload/storage/v1/b/...] failed, reason: write EPROTO
at Gaxios._request (/path/to/project/node_modules/gaxios/src/gaxios.ts:157:13)
type: 'system',
errno: 'EPROTO',
code: 'EPROTO'

Relevant Code:

Here is a sample of the TypeScript function used for uploading:

import { Stream } from 'stream';
import { getStorageAdmin } from 'path-to-your-storage-admin-utils';

async uploadFileViaStream(fileStream: Stream, path: string): Promise<string> {
    console.debug('Uploading file to storage using stream...');
    const storage = getStorageAdmin();
    const bucket = storage.bucket('voicecheap-c1f14.appspot.com');
    const fileRef = bucket.file(path);
    const contentType =
      path.split('.').pop() === 'mp3' ? 'audio/mpeg' : 'video/mp4';

    // Verify if the file already exists
    const [exists] = await fileRef.exists();
    if (exists) {
      console.debug('File already exists. Skipping upload.');
      return fileRef.publicUrl();
    }

    return new Promise((resolve, reject) => {
      fileStream
        .pipe(
          fileRef.createWriteStream({
            contentType: contentType,
            public: true,
            metadata: {
              contentType: contentType,
              mimeType: contentType,
              type: contentType,
            },
          }),
        )
        .on('error', (error) => {
          console.error('Failed to upload file.', error);
          reject(error);
          throw new InternalServerErrorException('Failed to upload file.');
        })
        .on('finish', async () => {
          // Make the file publicly accessible
          try {
            await fileRef.makePublic();
            console.debug('File uploaded and made public.');
            resolve(fileRef.publicUrl());
          } catch (error) {
            console.error('Failed to make the file public.', error);
            reject(error);
            throw new InternalServerErrorException(
              'Failed to make the file public.',
            );
          }
        });
    });
  }
@google-oss-bot
Copy link

I found a few problems with this issue:

  • I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.
  • This issue does not seem to follow the issue template. Make sure you provide all the required information.

@aendel
Copy link

aendel commented Sep 5, 2024

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants