nvidia nx平台 Gstreamer tcpserverink延迟2-3秒问题调试1

free-jdx 2021-07-30 17:04:37 6436

1. 前言
2.调整nvv4l2h265enc属性
3. 尝试UDP传输
4.  更换服务器命令
1. 前言

试图发送一个视频流与tcp
但得到2-3秒的延迟。
接收到的流有时在一个灰色图像上停止,然后接收到加速帧的突发。
在Jetpack 4.5.1上有一个Jetson NX Xavier devkit。
我将它与nvpmodel - m2和jetson_clocks -fan一起使用。
使VIC在这里描述的最大时钟运行

v4l2-ctl --list-formats-ext -d /dev/video0

ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘YUYV’
Name : YUYV 4:2:2
Size: Discrete 1920x1080
Interval: Discrete 0.017s (60.000 fps)

pipeline :

std::string gst_out = "appsrc ! videoconvert ! queue ! nvvidconv ! 
nvv4l2h265enc maxperf-enable=1 ! h265parse 
! matroskamux ! tcpserversink port=AAAA 
host=XXX.XXX.XXX.XXX sync=false async=false";

codec:

cv::VideoWriter out(gst_out, cv::VideoWriter::fourcc('F', 'M', 'P', '4'), 
double(fps), cv::Size(int(w), int(h)));

尝试用video/x-raw删除视频转换,
格式=YUYV,宽度=1920,
高度=1080,帧率=60/1,
但它不工作与这个设置。

用fpsdisplaysink检查视频流

gst-launch-1.0 v4l2src ! videoconvert ! fpsdisplaysink text-overlay=0 video-sink=fakesink sink=0 -v
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = true
Setting pipeline to PLAYING …
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = true
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 32, dropped: 0, current: 61,95, average: 61,95
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 63, dropped: 0, current: 60,01, average: 60,98
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 93, dropped: 0, current: 60,00, average: 60,66
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 124, dropped: 0, current: 60,00, average: 60,49
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 154, dropped: 0, current: 60,00, average: 60,40
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 185, dropped: 0, current: 60,00, average: 60,33
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 216, dropped: 0, current: 60,01, average: 60,28
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 247, dropped: 0, current: 59,99, average: 60,25
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 278, dropped: 0, current: 60,00, average: 60,22
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 309, dropped: 0, current: 60,00, average: 60,20
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 339, dropped: 0, current: 60,00, average: 60,18
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 370, dropped: 0, current: 60,00, average: 60,16
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 401, dropped: 0, current: 60,00, average: 60,15
^Chandling interrupt.
Interrupt: Stopping pipeline …
Execution ended after 0:00:07.000306499
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …
2.调整nvv4l2h265enc属性

调整nvv4l2h265enc中的两个属性并尝试:

iframeinterval      : Encoding Intra Frame occurance frequency
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 30
  idrinterval         : Encoding IDR Frame occurance frequency
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 256

默认设置可能不适合流
可以尝试idrinterval=30或15

尝试为idrinterval(15, 30)和
iframeinterval(5, 10, 20, 50, 100, 200, 300, 600)
设置多个值,但没有任何变化

以下是我的代码:


// Convert to string
#define SSTR( x ) static_cast< std::ostringstream & >( \
( std::ostringstream() << std::dec << x ) ).str()

int main(int argc, char** argv)
{
    cv::VideoCapture invid;
    if (!invid.open(0))
    {
        std::cout << "error opening video file" << std::endl;
        return -1;
    }

    std::cout << "camera open " << std::endl;

    cv::Mat src;
    invid >> src;

    int w, h, fps, format = 0;
    w = invid.get(cv::CAP_PROP_FRAME_WIDTH);    //1920 
    h = invid.get(cv::CAP_PROP_FRAME_HEIGHT);   //1080 
    fps = invid.get(cv::CAP_PROP_FPS);          //60 
    format = invid.get(cv::CAP_PROP_FORMAT);    //0    //doesn't seems to get the right format
    /*v4l2-ctl --list-formats-ext -d /dev/video0  // Camera details
    ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture
        Pixel Format: 'YUYV'
        Name        : YUYV 4:2:2
            Size: Discrete 1920x1080
                Interval: Discrete 0.017s (60.000 fps)*/

    std::cout << "frame width : " << w << " frame height : " << h << " fps : " << fps << " format : " << format << std::endl;

    std::string gst_out = "appsrc ! videoconvert ! queue ! nvvidconv ! nvv4l2h265enc maxperf-enable=1 idrinterval=30 iframeinterval=5 ! h265parse ! matroskamux ! tcpserversink port=8554 host=192.168.1.29 sync=false async=false"; //marche --> attention 脿 l'addresse ip
    cv::VideoWriter out(gst_out, cv::VideoWriter::fourcc('F', 'M', 'P', '4'), double(fps), cv::Size(int(w), int(h)));

    using std::chrono::high_resolution_clock;                           // Calcul loop time
    using std::chrono::duration_cast;
    using std::chrono::duration;
    using std::chrono::milliseconds;
    duration<double, std::milli> time_ms;

    cv::Mat img;

    if(!out.isOpened())
    {
        std::cout << "Failed to open output" << std::endl;
        return 0;
    }

    if(invid.isOpened())
    {
        auto t1 = high_resolution_clock::now();
        //Process
        for(;;)
        {
            auto t2 = high_resolution_clock::now();
            time_ms = t2 - t1;
            //std::cout << "time_ms : " << time_ms.count() << " ms." << std::endl;  
            t1 = t2; 

            float fps_calculated = 1000/time_ms.count(); // Calculate Frames per second (FPS)

            invid >> img;
            /*if (!invid.read(img))
                return 0;*/
            if(img.empty() != 0)
                return 0;

            // Display FPS on frame
            putText(img, "FPS : " + SSTR(int(fps_calculated)), cv::Point(100,50), cv::FONT_HERSHEY_SIMPLEX, 0.75, cv::Scalar(50,170,50), 2);
            std::cout << "fps_calculated : " << fps_calculated << std::endl;
            std::cout << "time_ms.count() : " << time_ms.count() << std::endl;

            //cv::imshow("azert", img);                             

            out.write(img);
            if(cv::waitKey(10) == 27)
                break; // stop capturing by pressing ESC
        }
    }
}
3. 尝试UDP传输

尝试了UDP在TX2 NX作为服务器,
x86 PC作为客户端。
服务器命令:

gst-launch-1.0 videotestsrc is-live=1 
! video/x-raw,width=1280,height=720 
! timeoverlay valignment=4 halignment=1 
! nvvidconv ! 'video/x-raw(memory:NVMM),width=1280,height=720' 
! tee name=t ! nvv4l2h264enc insert-sps-pps=1 idrinterval=15 
! h264parse ! rtph264pay 
! udpsink host=10.19.106.10 port=5000 sync=0 t. 
! queue ! nvegltransform ! nveglglessink sync=0

客户端命令:

gst-launch-1.0 udpsrc port=5000 
! 'application/x-rtp,encoding-name=H264,payload=96' 
! rtph264depay ! avdec_h264 ! xvimagesink sync=0

结果如下
(左边是TX2 NX,右边是x86 PC):

4. 更换服务器命令

尝试以下情况下的服务器命令:

gst-launch-1.0 videotestsrc in UDP
gst-launch-1.0 videotestsrc in TCP
gst-launch-1.0 OpenCV in UDP

看看能不能通过这些弄清楚延迟的原因
运行gst-inspect-1.0 nvv4l2h265enc获取所有属性

但我的Ubuntu 18.04 x86 PC没有得到视频测试:

gst-launch-1.0 udpsrc port=5000 ! 'application/x-rtp,encoding-name=H264,payload=96' ! rtph264depay ! avdec_h264 ! xvimagesink sync=0
Définition du pipeline à PAUSED...
Le pipeline est actif et n’a pas besoin de phase PREROLL…
Passage du pipeline à la phase PLAYING…
New clock: GstSystemClock
^Chandling interrupt.
Interruption : arrêt du pipeline…
Execution ended after 0:00:17.859331593
Définition du pipeline à PAUSED...
Définition du pipeline à READY (prêt)…
Définition du pipeline à NULL…
Libération du pipeline…

没有图像显示。

还有gst-launch-1.0 nvv4l2h265enc在我的NX Xavier:

$ gst-launch-1.0 nvv4l2h265enc
Setting pipeline to PAUSED ...
Opening in BLOCKING MODE
Opening in BLOCKING MODE 
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:07.792176179
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

命令不正确
应该是gst-inspect-1.0

声明:本文内容由易百纳平台入驻作者撰写,文章观点仅代表作者本人,不代表易百纳立场。如有内容侵权或者其他问题,请联系本站进行删除。
free-jdx
红包 93 2 评论 打赏
评论
0个
内容存在敏感词
手气红包
    易百纳技术社区暂无数据
相关专栏
置顶时间设置
结束时间
删除原因
  • 广告/SPAM
  • 恶意灌水
  • 违规内容
  • 文不对题
  • 重复发帖
打赏作者
易百纳技术社区
free-jdx
您的支持将鼓励我继续创作!
打赏金额:
¥1易百纳技术社区
¥5易百纳技术社区
¥10易百纳技术社区
¥50易百纳技术社区
¥100易百纳技术社区
支付方式:
微信支付
支付宝支付
易百纳技术社区微信支付
易百纳技术社区
打赏成功!

感谢您的打赏,如若您也想被打赏,可前往 发表专栏 哦~

举报反馈

举报类型

  • 内容涉黄/赌/毒
  • 内容侵权/抄袭
  • 政治相关
  • 涉嫌广告
  • 侮辱谩骂
  • 其他

详细说明

审核成功

发布时间设置
发布时间:
是否关联周任务-专栏模块

审核失败

失败原因
备注
拼手气红包 红包规则
祝福语
恭喜发财,大吉大利!
红包金额
红包最小金额不能低于5元
红包数量
红包数量范围10~50个
余额支付
当前余额:
可前往问答、专栏板块获取收益 去获取
取 消 确 定

小包子的红包

恭喜发财,大吉大利

已领取20/40,共1.6元 红包规则

    易百纳技术社区