nvidia xavier nx平台GStreamer pipeline挂起问题调试1

free-jdx 2021-05-06 16:03:00 7996
NVIDIA英伟达
1. 前言

在nvidia xavier nx平台使用GStreamer插件时，
出现一些异常情况，这里记录下调试过程

2.挂起问题

我设计了一个pipeline，使用两个不同的摄像头在我的基于Python/Qt的应用程序中组成输出。
一切正常，但应用程序在几个小时或几分钟后就会挂起，
使用调试级别3运行gstreamer会暴露以下警告

0:00:01.016107283 14314   0x559bdfe590 WARN          v4l2bufferpool gstv4l2bufferpool.c:790:gst_v4l2_buffer_pool_start:<source_2:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:01.923523511 14314   0x559bdfe590 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<source_2:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:01.939974401 14314   0x559bdfe590 WARN                 v4l2src gstv4l2src.c:976:gst_v4l2src_create:<source_2> lost frames detected: count = 230 - ts: 0:00:00.919337027
0:00:02.990022327 14314   0x559bdfe590 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<source_2:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:03.006565985 14314   0x559bdfe590 WARN                 v4l2src gstv4l2src.c:976:gst_v4l2src_create:<source_2> lost frames detected: count = 136 - ts: 0:00:01.985947187
0:00:04.289911178 14314   0x559bdfe590 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<source_2:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:04.306536499 14314   0x559bdfe590 WARN                 v4l2src gstv4l2src.c:976:gst_v4l2src_create:<source_2> lost frames detected: count = 205 - ts: 0:00:03.285886027
0:00:14.463057820 14314   0x559bdfe590 WARN                   queue gstqueue.c:988:gst_queue_handle_sink_event:<queue2> error: Internal data stream error.
0:00:14.463200958 14314   0x559bdfe590 WARN                   queue gstqueue.c:988:gst_queue_handle_sink_event:<queue2> error: streaming stopped, reason not-linked (-1)
0:00:14.463468224 14314   0x559bdfe540 WARN                   queue gstqueue.c:988:gst_queue_handle_sink_event:<queue0> error: Internal data stream error.
0:00:14.463529633 14314   0x559bdfe540 WARN                   queue gstqueue.c:988:gst_queue_handle_sink_event:<queue0> error: streaming stopped, reason not-linked (-1)
0:00:14.607746627 14314   0x559bdfe540 WARN          v4l2bufferpool gstv4l2bufferpool.c:790:gst_v4l2_buffer_pool_start:<source_1:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:15.610458006 14314   0x559bdfe540 WARN                   queue gstqueue.c:988:gst_queue_handle_sink_event:<queue4> error: Internal data stream error.
0:00:15.610592792 14314   0x559bdfe540 WARN                   queue gstqueue.c:988:gst_queue_handle_sink_event:<queue4> error: streaming stopped, reason not-linked (-1)
0:00:15.610765401 14314   0x559bdfe540 WARN                   queue gstqueue.c:988:gst_queue_handle_sink_event:<queue5> error: Internal data stream error.
0:00:15.610828698 14314   0x559bdfe540 WARN                   queue gstqueue.c:988:gst_queue_handle_sink_event:<queue5> error: streaming stopped, reason not-linked (-1)
0:00:15.703272958 14314   0x559bdfe540 WARN          v4l2bufferpool gstv4l2bufferpool.c:790:gst_v4l2_buffer_pool_start:<source_1:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:17.365679776 14314   0x559bdfe540 WARN                   queue gstqueue.c:988:gst_queue_handle_sink_event:<queue8> error: Internal data stream error.
0:00:17.366101444 14314   0x559bdfe540 WARN                   queue gstqueue.c:988:gst_queue_handle_sink_event:<queue8> error: streaming stopped, reason not-linked (-1)
0:00:17.366437031 14314   0x559bdfe540 WARN                   queue gstqueue.c:988:gst_queue_handle_sink_event:<queue9> error: Internal data stream error.
0:00:17.366741162 14314   0x559bdfe540 WARN                   queue gstqueue.c:988:gst_queue_handle_sink_event:<queue9> error: streaming stopped, reason not-linked (-1)
0:00:17.466034826 14314   0x559bdfe590 WARN          v4l2bufferpool gstv4l2bufferpool.c:790:gst_v4l2_buffer_pool_start:<source_2:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:01:07.267125833 14314   0x559bdfe540 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<source_1:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:01:07.320018089 14314   0x559bdfe540 WARN                 v4l2src gstv4l2src.c:976:gst_v4l2src_create:<source_1> lost frames detected: count = 11 - ts: 0:00:49.825705798
0:01:18.469308144 14314   0x559bdfe590 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<source_2:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:01:18.486142300 14314   0x559bdfe590 WARN                 v4l2src gstv4l2src.c:976:gst_v4l2src_create:<source_2> lost frames detected: count = 255 - ts: 0:01:01.014083542
0:02:35.649454180 14314   0x559bdfe590 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<source_2:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:02:35.666158884 14314   0x559bdfe590 WARN                 v4l2src gstv4l2src.c:976:gst_v4l2src_create:<source_2> lost frames detected: count = 168 - ts: 0:02:18.194158446
0:02:40.715948578 14314   0x559bdfe590 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<source_2:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:02:40.732665022 14314   0x559bdfe590 WARN                 v4l2src gstv4l2src.c:976:gst_v4l2src_create:<source_2> lost frames detected: count = 71 - ts: 0:02:23.260606902
0:03:03.117321379 14314   0x559bdfe540 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<source_1:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:03:03.157331221 14314   0x559bdfe540 WARN                 v4l2src gstv4l2src.c:976:gst_v4l2src_create:<source_1> lost frames detected: count = 1 - ts: 0:02:45.663025742
0:03:32.122298850 14314   0x559bdfe540 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<source_1:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:03:32.122536421 14314   0x559bdfe540 WARN           v4l2allocator gstv4l2allocator.c:1349:gst_v4l2_allocator_dqbuf:<source_1:pool:src:allocator> V4L2 provided buffer has bytesused 0 which is too small to include data_offset 0

gstreamer pipeline如下：

gst-launch-1.0 -e \
    v4l2src io-mode=2 device=/dev/video0 name=source_1 \
    v4l2src io-mode=2 device=/dev/video0 name=source_2 \
    nvcompositor name=comp sink_0::width=1920 sink_0::height=1080 sink_1::xpos=1280 sink_1::width=640 sink_1::height=360 \
    source_1. ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=30/1 ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! tee name=tee_source_1 \
    tee_source_1. ! queue ! comp. \
    tee_source_1. ! queue max-size-buffers=2 leaky=downstream ! nvvidconv ! video/x-raw,format=BGRx ! appsink max-buffers=10 drop=true emit-signals=true sync=false name=sink_primary \
    source_2. ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=30/1 ! nvvidconv ! 'video/x-raw(memory:NVMM),width=640,height=360' ! tee name=tee_source_2 \
    tee_source_2. ! queue ! comp. \
    tee_source_2. ! queue max-size-buffers=2 leaky=downstream ! nvvidconv ! video/x-raw,format=BGRx,width=640,height=360 ! appsink max-buffers=10 drop=true emit-signals=true sync=false name=sink_secondary \
    comp. ! 'video/x-raw(memory:NVMM)' ! nvvidconv ! omxh265enc ! tee name=recording_tee \
    recording_tee. ! queue ! rtph265pay config-interval=10 pt=96 ! udpsink host=127.0.0.1 port=5000 \
    recording_tee. ! queue ! splitmuxsink location=cam-recording-%03d.mp4 max-size-time=9000000000000 max-files=96

我认为问题是，应用接收器需要很长时间来处理，最终在管道中阻塞了一些东西，但我不确定

3. 尝试使用nvv4l2h265enc

建议尝试nvv4l2camerasrc。
您可以重新构建它以支持YUY2格式。
参考此补丁

diff --git a/gst-nvv4l2camera/gstnvv4l2camerasrc.cpp b/gst-nvv4l2camera/gstnvv4l2camerasrc.cpp
index 26ad70c..ad0a512 100644
--- a/gst-nvv4l2camera/gstnvv4l2camerasrc.cpp
+++ b/gst-nvv4l2camera/gstnvv4l2camerasrc.cpp
@@ -61,7 +61,7 @@
   "video/x-raw(memory:NVMM), " \
   "width = (int) [ 1, MAX ], " \
   "height = (int) [ 1, MAX ], " \
-  "format = (string) { UYVY }, " \
+  "format = (string) { YUY2 }, " \
   "interlace-mode = (string) { progressive, interlaced }, " \
   "framerate = (fraction) [ 0, MAX ];"

@@ -194,7 +194,7 @@ gst_nv_memory_allocator_alloc (GstAllocator * allocator,
       input_params.height = self->height;
     }
     input_params.layout = NvBufferLayout_Pitch;
-    input_params.colorFormat = NvBufferColorFormat_UYVY;
+    input_params.colorFormat = NvBufferColorFormat_YUYV;
     input_params.payloadType = NvBufferPayload_SurfArray;
     input_params.nvbuf_tag = NvBufferTag_CAMERA;

@@ -229,7 +229,7 @@ gst_nv_memory_allocator_alloc (GstAllocator * allocator,
     params.layout = NVBUF_LAYOUT_PITCH;
     params.memType = NVBUF_MEM_DEFAULT;
     params.gpuId = 0;
-    params.colorFormat = NVBUF_COLOR_FORMAT_UYVY;
+    params.colorFormat = NVBUF_COLOR_FORMAT_YUYV;

     if (self->interlaced_flag)
     {
--

编译源YUY2补丁,现在我面临的问题是：
一个或两个摄像头显示绿色混乱图像后重新连接usb电缆可能修复或可能保持绿色,
使用v4l2src没有这样的问题。

gst-launch-1.0 -e nvv4l2camerasrc device=/dev/video1 !
 'video/x-raw(memory:NVMM),format=YUY2,width=1920,height=1080,
 framerate=30/1' ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! nv3dsink

我的应用程序功有交换视频源的功能。
我送EOS到pipelien并等待它完成,然后我用交换的源启动一个全新的管道，但是在将新创建的管道状态设置为READY之后，gstreamer崩溃并产生以下输出。
我想提到的是，这种相同的方法被用于以前刷的镜像是没有问题的

0:00:03.877650773 19473      0xaf02cf0 WARN                    v4l2 gstv4l2object.c:2375:gst_v4l2_object_add_interlace_mode:0xabc1400 Failed to determine interlace mode
    0:00:03.877743286 19473      0xaf02cf0 WARN                    v4l2 gstv4l2object.c:2375:gst_v4l2_object_add_interlace_mode:0xabc1400 Failed to determine interlace mode
    0:00:03.877803766 19473      0xaf02cf0 WARN                    v4l2 gstv4l2object.c:2375:gst_v4l2_object_add_interlace_mode:0xabc1400 Failed to determine interlace mode
    0:00:03.877909238 19473      0xaf02cf0 WARN                    v4l2 gstv4l2object.c:4434:gst_v4l2_object_probe_caps:<nvv4l2h265enc1:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1

我认为问题的根源是在nvcompositor

recording_tee. ! queue ! rtph265pay config-interval=10 pt=96 !
 udpsink host=127.0.01 port=5000

然后去掉第二部分

recording_tee. ! queue ! splitmuxsink location=/mnt/storage/%02d.mp4 max-size-time=9000000000000 
max-files=96 async-finalize=true

然而，当我试图更改重新运行管道时，它失败了，只排除了整个comp.(最后3行)定义的帮助。除此之外，当我使用nvv4l2h265enc运行管道时，我经常看到以下消息

0:00:03.209421132 20591 0x7f38003850 FIXME h265parse gsth265parse.c:1798:gst_h265_parse_parse_frame: 
Implement timestamp/duration interpolation based on SEI message

这是我最后的pipeline

v4l2src io-mode=2 device=/dev/video0 name=source_1 ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=30/1 ! nvvidconv ! video/x-raw(memory:NVMM) ! tee name=tee_source_1 
v4l2src io-mode=2 device=/dev/video1 name=source_2 ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=30/1 ! nvvidconv ! video/x-raw(memory:NVMM),width=640,height=360 ! tee name=tee_source_2 
nvcompositor name=comp sink_0::width=1920 sink_0::height=1080 sink_1::xpos=1280 sink_1::width=640 sink_1::height=360 
tee_source_1. ! queue ! comp. 
tee_source_2. ! queue ! comp. 
tee_source_1. ! queue ! nvvidconv ! video/x-raw,format=BGRx ! appsink drop=true emit-signals=true sync=false name=sink_primary 
tee_source_2. ! queue ! nvvidconv ! video/x-raw,format=BGRx ! appsink drop=true emit-signals=true sync=false name=sink_secondary 
comp. ! video/x-raw(memory:NVMM) ! nvvidconv ! video/x-raw(memory:NVMM),format=NV12 ! nvv4l2h265enc bitrate=8000000 ! h265parse ! tee name=recording_tee 
recording_tee. ! queue ! rtph265pay config-interval=10 pt=96 ! udpsink host=127.0.0.1 port=5000 
recording_tee. ! queue ! splitmuxsink location=/mnt/storage/2021-04-07/021642561645/%02d.mp4 max-size-time=9000000000000 max-files=96
4. 分析相机

从截图来看，颜色看起来不太对。

（1）检测相机格式
相机是YUY2格式的吗?
似乎是在UYVY或其他格式。

（2）相机类型
USB摄像头还是YUV传感器

使用DFG/HDSDI - SDI-to-USB 3.0 converter连接相机

（3）相机参数

v4l2-ctl -d /dev/video0 --list-formats-ext
    Index       : 1
    Type        : Video Capture
    Pixel Format: 'YUYV'
    Name        : YUYV 4:2:2
        Size: Discrete 1920x1080
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.033s (29.970 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.042s (23.940 fps)
        Size: Discrete 1920x540
            Interval: Discrete 0.017s (60.000 fps)
            Interval: Discrete 0.017s (59.940 fps)
            Interval: Discrete 0.020s (50.000 fps)
        Size: Discrete 1280x720
            Interval: Discrete 0.017s (60.000 fps)
            Interval: Discrete 0.017s (59.940 fps)
            Interval: Discrete 0.020s (50.000 fps)
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.033s (29.970 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.042s (23.940 fps)

在JP4.5(r32.5)，使用nvvidconv和nvcompositor开源。
下载这个包并进一步调试:
https://developer.nvidia.com/embedded/l4t/r32_release_v5.1/r32_release_v5.1/sources/t186/public_sources.tbz2

经过进一步的调试，确定nvv4l2h265enc组件是错误的。
准备了一个简单的python应用程序，这是在我的主要应用程序中使用

import functools
import sys
import gi
from PyQt5 import QtWidgets

gi.require_version('Gst', '1.0')
from gi.repository import GObject, Gst

class MainWindow(QtWidgets.QWidget):
    def __init__(self, parent=None):
        super(MainWindow, self).__init__(parent)
        self.setWindowTitle('QHBoxLayout')
        layout = QtWidgets.QHBoxLayout()

        button1 = QtWidgets.QPushButton('/dev/viode0')
        button1.clicked.connect(functools.partial(self.create_pipeline, "/dev/video0"))
        layout.addWidget(button1)

        button2 = QtWidgets.QPushButton('/dev/video1')
        button2.clicked.connect(functools.partial(self.create_pipeline, "/dev/video1"))
        layout.addWidget(button2)

        button3 = QtWidgets.QPushButton('Stop pipeline')
        button3.clicked.connect(self.stop_pipeline)
        layout.addWidget(button3)

        self.setLayout(layout)
        self.pipeline = None
        self.bus = None
        self.width = 1920
        self.height = 1080
        self.width_descaled = 640
        self.height_descaled = 360

    def create_pipeline(self, source):
        self.stop_pipeline()

        pipeline_str = \
            f"v4l2src io-mode=2 device={source} name=source_1 ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=30/1 ! nvvidconv ! video/x-raw(memory:NVMM) ! tee name=tee_source_1 " \
            f"nvcompositor name=comp sink_0::width={self.width} sink_0::height={self.height} sink_1::xpos={self.width - self.width_descaled} sink_1::width={self.width_descaled} sink_1::height={self.height_descaled} " \
            "tee_source_1. ! queue ! comp. " \
            "tee_source_1. ! queue ! comp. " \
            f"tee_source_1. ! queue ! nvvidconv ! video/x-raw(memory:NVMM),format=NV12 ! fakesink name=sink_primary " \
            f"tee_source_1. ! queue ! nvvidconv ! video/x-raw(memory:NVMM),format=NV12,width={self.width_descaled},height={self.height_descaled} ! fakesink name=sink_secondary " \
            "comp. ! video/x-raw(memory:NVMM) ! nvvidconv ! video/x-raw(memory:NVMM),format=NV12 ! tee name=comp_tee " \
            "comp_tee. ! nv3dsink "

        # FAULTY COMPONENT IS nvv4l2h265enc
        pipeline_str += "comp_tee. ! nvv4l2h265enc maxperf-enable=1 bitrate=8000000"

        print(pipeline_str)

        self.pipeline = Gst.parse_launch(pipeline_str)
        self.bus = self.pipeline.get_bus()
        self.bus.add_signal_watch()
        self.pipeline.set_state(Gst.State.READY)
        self.pipeline.set_state(Gst.State.PLAYING)

    def stop_pipeline(self):
        timeout = 1
        if self.pipeline is not None and self.pipeline.get_state(timeout).state == Gst.State.PLAYING:
            Gst.Element.send_event(self.pipeline, Gst.Event.new_eos())
            self.bus.timed_pop_filtered(Gst.CLOCK_TIME_NONE, Gst.MessageType.EOS)
            self.pipeline.set_state(Gst.State.NULL)
            self.pipline = None
            self.bus = None
            print("Pipeline stopped")

GObject.threads_init()
Gst.init(sys.argv)
Gst.debug_set_active(False)
Gst.debug_set_default_threshold(3)

app = QtWidgets.QApplication(sys.argv)
window = MainWindow()
window.show()
sys.exit(app.exec_())
声明：本文内容由易百纳平台入驻作者撰写，文章观点仅代表作者本人，不代表易百纳立场。如有内容侵权或者其他问题，请联系本站进行删除。
free-jdx
红包 94 8 评论 打赏
  • 分享
  • 举报
评论
0个
时间排序
内容存在敏感词
手气红包
    易百纳技术社区暂无数据
相关专栏
关于作者
free-jdx

free-jdx

暂无个性签名~

原创178
阅读109w
收藏13
点赞35
评论24
TA最新文章
打赏用户 共 1 位
我要创作
分享技术经验，可获取创作收益
分类专栏
热门专栏
置顶时间设置
结束时间
删除原因
  • 广告/SPAM
  • 恶意灌水
  • 违规内容
  • 文不对题
  • 重复发帖
打赏作者
易百纳技术社区
free-jdx
您的支持将鼓励我继续创作！
打赏金额：
￥1易百纳技术社区
￥5易百纳技术社区
￥10易百纳技术社区
￥50易百纳技术社区
￥100易百纳技术社区
支付方式：
微信支付
支付宝支付
易百纳技术社区微信支付
易百纳技术社区
打赏成功!

感谢您的打赏，如若您也想被打赏，可前往 发表专栏 哦~

举报反馈

举报类型

  • 内容涉黄/赌/毒
  • 内容侵权/抄袭
  • 政治相关
  • 涉嫌广告
  • 侮辱谩骂
  • 其他

详细说明

审核成功

发布时间设置
立即发布
延迟发布
发布时间：
是否关联周任务-专栏模块
是
否

审核失败

失败原因
备注
拼手气红包 红包规则
祝福语
恭喜发财，大吉大利！
红包金额
元
红包最小金额不能低于5元
红包数量
个
红包数量范围10~50个
余额支付
当前余额：
可前往问答、专栏板块获取收益 去获取
取 消 确 定

小包子的红包

恭喜发财，大吉大利

红包规则
开

已领取20/40，共1.6元 红包规则

    易百纳技术社区