xavier nx建立RNN层失败问题

1. 前言

已经成功构建了X86上的TRT模型，
但在ARM64 (NX)上构建TRT模型失败。

错误如下

GRU_75: inputs to IRecurrenceLayer mismatched
Builder failed while analyzing shapes.

X86 Config:
TensorRT version: 7.2.3.4

ARM64 config:
TensorRt version: 7.1.3.4

Xavier NX 软件版本JetPack 4.5

2. 尝试其他网络、

已经使用“torch.onnx.export”导出了ONNX
已经使用PyTorch、OnnxRuntime和TensorRT完成了推理。
PyTorch和OnnxRuntime给出了相同的结果，而TensorRT则没有。

觉得ONNX的TrtEngine有问题。
我还有其他的分割模型，载入很好，但结果是不相似的。
我已经尝试了polygraphy分割模型，Pytorch和OnnRuntime给出相同的结果，
但TRT引擎不是。

3. 升级nvonnxparser

对于JetPack 4.5.1环境，可以将nvonnxparser升级到v7.2来修复这个问题。
以下是具体的操作步骤:

Install cmake-3.13.5

$ sudo apt-get install -y protobuf-compiler libprotobuf-dev openssl libssl-dev libcurl4-openssl-dev
$ wget https://github.com/Kitware/CMake/releases/download/v3.13.5/cmake-3.13.5.tar.gz
$ tar xvf cmake-3.13.5.tar.gz
$ cd cmake-3.13.5/
$ ./bootstrap --system-curl
$ make -j$(nproc)
$ echo 'export PATH='${PWD}'/bin/:$PATH' >> ~/.bashrc
$ source ~/.bashrc

Build onnx-tensorrt

$ git clone https://github.com/onnx/onnx-tensorrt.git
$ cd onnx-tensorrt/
$ git submodule update --init --recursive
$ mkdir build && cd build
$ cmake ../
$ make -j

$ sudo mv /usr/lib/aarch64-linux-gnu/libnvonnxparser.so.7.1.3 libnvonnxparser.so.7.1.3_bk
$ sudo cp libnvonnxparser.so.7.2.2 /usr/lib/aarch64-linux-gnu/libnvonnxparser.so.7.2.2

$ sudo rm /usr/lib/aarch64-linux-gnu/libnvonnxparser.so.7
$ sudo ln -s /usr/lib/aarch64-linux-gnu/libnvonnxparser.so.7.2.2 /usr/lib/aarch64-linux-gnu/libnvonnxparser.so.7
$ sudo ldconfig

实际上，在X86_64 Ubuntu上使用TensorRT 8.0+CUDA 11.3解决了所有的问题。

根源在于onnx解析器，而不是TensorRT库本身。
为了兼容性，您仍然需要使用TensorRT v7.1.3。
但是升级ONNX解析器可以解决这个问题

4.更新完测试

在ARM64-NX上更新完后解发现问题
TensorRT 7.1.3通过JetPack 4.5.1安装
Onnx-Parser按照指令安装
问题如下：

home/onnx-tensorrt/onnx2trt_utils.cpp:291: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
/home/onnx-tensorrt/ModelImporter.cpp:703: While parsing node number 37 [Resize -> "300"]:
/home/onnx-tensorrt/ModelImporter.cpp:704: --- Begin node ---
/home/onnx-tensorrt/ModelImporter.cpp:705: input: "295"
input: "299"
input: "526"
output: "300"
name: "Resize_37"
op_type: "Resize"
attribute {
  name: "coordinate_transformation_mode"
  s: "align_corners"
  type: STRING
}
attribute {
  name: "cubic_coeff_a"
  f: -0.75
  type: FLOAT
}
attribute {
  name: "mode"
  s: "linear"
  type: STRING
}
attribute {
  name: "nearest_mode"
  s: "floor"
  type: STRING
}

/home/onnx-tensorrt/ModelImporter.cpp:706: --- End node ---
/home/onnx-tensorrt/ModelImporter.cpp:709: ERROR: /home/onnx-tensorrt/builtin_op_importers.cpp:3074 In function importResize:
[8] Assertion failed: (transformationMode == "asymmetric" || transformationMode == "pytorch_half_pixel" || transformationMode == "half_pixel") && "TensorRT only supports half pixel, pytorch half_pixel, and asymmetric tranformation mode for linear resizes when scales are provided!"
kError: Assertion failed: (transformationMode == "asymmetric" || transformationMode == "pytorch_half_pixel" || transformationMode == "half_pixel") && "TensorRT only supports half pixel, pytorch half_pixel, and asymmetric tranformation mode for linear resizes when scales are provided!"
Network must have at least one output
Network validation failed.

5. 解决思路

Assertion failed: (transformationMode == “half_pixel” || transformationMode == “pytorch_half_pixel” || transformationMode == “align_corners”) && “TensorRT only supports half_pixel, pytorch_half_pixel, and align_corners transofmration modes for linear resizes when sizes are provided!”

以上错误是由一个不支持的操作引起的:调整大小+线性插值+非对称模式。

    //    alignCorners = 0: HALF_PIXEL
    //    alignCorners = 1: ASYMMETRIC
    else
    {
        if (mode == "nearest")
        {
            ASSERT(transformationMode == "asymmetric" && "TensorRT only supports asymmetric tranformation mode for nearest neighbor resizes when scales are provided!",ErrorCode::kUNSUPPORTED_NODE);
        }
        else if (mode == "linear")
        {

            ASSERT((transformationMode == "asymmetric" || transformationMode == "pytorch_half_pixel" || transformationMode == "half_pixel") && "TensorRT only supports half pixel, pytorch half_pixel, and asymmetric tranformation mode for linear resizes when scales are provided!", ErrorCode::kUNSUPPORTED_NODE);

            if (transformationMode == "asymmetric")
            {
                layer->setAlignCorners(true);
            }
        }
    }
}

// For opset 10 resize, the only supported mode is asymmetric resize with scales.

else

增加更多的插值支持的调整层。
目前可以将Resize模式更新为最接近跳过此错误的模式。

更新也可以通过ONNX graphsurgeon API完成。
增加更多的插值支持的调整层。
目前可以将Resize模式更新为最接近跳过此错误的模式。

更新也可以通过ONNX graphsurgeon API完成。

import onnx_graphsurgeon as gs
import onnx
import numpy as np

graph = gs.import_onnx(onnx.load("model.onnx"))
node = [node for node in graph.nodes if node.op == "Resize"]
for n in node:
    n.attrs['mode'] = 'nearest'

onnx.save(gs.export_onnx(graph), "updated_model.onnx")