Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How can I extract SPS, VPS from h.264 codec? #1056

Open
eomgerm opened this issue Jul 30, 2024 · 2 comments
Open

How can I extract SPS, VPS from h.264 codec? #1056

eomgerm opened this issue Jul 30, 2024 · 2 comments
Assignees

Comments

@eomgerm
Copy link

eomgerm commented Jul 30, 2024

Hello.
I'm developing live streaming application using OAK-1 Camera.

To stream video I need to extract SPS, VPS from video data.
I successfully retrieved H264 data by referring to this article.
When I access to NAL Unit Type I only get value 9 and I think this is not a normal situation.

However, like code below, I also saved the .h264 file and when I hex dumped it all NAL Unit Types are normal.
Is my code wrong?

Please let me know how to extract SPS and VPS data using this library.
Thanks

code I tried:

#!/usr/bin/env python3

import depthai as dai

# Create pipeline
pipeline = dai.Pipeline()

# Define sources and output
camRgb = pipeline.create(dai.node.ColorCamera)
videoEnc = pipeline.create(dai.node.VideoEncoder)
xout = pipeline.create(dai.node.XLinkOut)

xout.setStreamName('h264')

# Properties
camRgb.setBoardSocket(dai.CameraBoardSocket.AUTO)
camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
videoEnc.setDefaultProfilePreset(60, dai.VideoEncoderProperties.Profile.H264_MAIN)

# Linking
camRgb.video.link(videoEnc.input)
videoEnc.bitstream.link(xout.input)

# Connect to device and start pipeline
with dai.Device(pipeline) as device:

    # Output queue will be used to get the encoded data from the output defined above
    q = device.getOutputQueue(name="h264", maxSize=30, blocking=True)

    # The .h265 file is a raw stream file (not playable yet)
    with open('video.h264', 'wb') as videoFile:
        print("Press Ctrl+C to stop encoding...")
        try:
            while True:
                h265Packet = q.get()  # Blocking call, will wait until a new data has arrived
                data = h265Packet.getData()
                print("NAL UNIT TYPE:", data[4] & 0x1F) # decoding NAL Unit Type
                h265Packet.getData().tofile(videoFile)  # Appends the packet data to the opened file
        except KeyboardInterrupt:
            # Keyboard interrupt (Ctrl + C) detected
            pass

    print("To view the encoded data, convert the stream file (.h264) into a "
          "video file (.mp4) using a command below:")
    print("ffmpeg -framerate 30 -i video.h264 -c copy video.mp4")

Output:

NAL UNIT TYPE: 9
NAL UNIT TYPE: 9
NAL UNIT TYPE: 9
NAL UNIT TYPE: 9
NAL UNIT TYPE: 9
... and so on.
@moratom
Copy link
Collaborator

moratom commented Aug 16, 2024

@asahtik could you please give this one a look?

@asahtik
Copy link
Contributor

asahtik commented Aug 16, 2024

Hi @eomgerm, in depthai an encoded frame message doesn't contain only a single NAL packet but contains many including SPS, PPS and slice packets, therefore you need to iterate over the packets in a message and extract the necessary data. You can refer to the methods in H26xParsers.cpp which find slice NAL units and extract the frame type (the NAL unit type that you're interested in is extracted in line 171).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants