Missing file libyolo_layer.so and serialization error

[Myron1996]

Hi @GeekAlexis

When I tried running an inference with provided yolov4_crowdhuman.onnx & yolov4_crowdhuman.trt . Everything ran successfully.

I used YOLOV4-Tiny model converted it in to yolov4-tiny.onnx & yolov4-tiny.trt and made the necessary changes as explained in the GitHub repo.

It ran successfully on my Linux 18.04 Ubuntu Machine but failed due to following error on Jetson Xavier. Its regarding some missing file but

libyolo_layer.so file is there in plugins directory. It will be very helpful if you Can you provide me any advice on this issue ?

tdc@tdc:~/Downloads/FastMOT$ python3 app.py --input_uri c1.mp4 --mot
Opening in BLOCKING MODE
Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
[ WARN:0] global /home/tdc/opencv_build/opencv/modules/videoio/src/cap_gstreamer.cpp (1044) open OpenCV | GStreamer warning: unable to query duration of stream
[ WARN:0] global /home/tdc/opencv_build/opencv/modules/videoio/src/cap_gstreamer.cpp (1081) open OpenCV | GStreamer warning: Cannot query video position: status=1, value=1, duration=-1
2021-06-14 18:01:41 [    INFO] 1280x720 stream @ 25 FPS
2021-06-14 18:01:41 [    INFO] Loading detector model...
Traceback (most recent call last):
  File "/home/tdc/Downloads/FastMOT/fastmot/utils/inference.py", line 31, in __init__
    ctypes.cdll.LoadLibrary(self.model.PLUGIN_PATH)
  File "/usr/lib/python3.6/ctypes/__init__.py", line 426, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib/python3.6/ctypes/__init__.py", line 348, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /home/tdc/Downloads/FastMOT/fastmot/plugins/libyolo_layer.so: cannot open shared object file: No such file or directory

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "app.py", line 98, in <module>
    main()
  File "app.py", line 51, in main
    draw=draw, verbose=args.verbose)
  File "/home/tdc/Downloads/FastMOT/fastmot/mot.py", line 51, in __init__
    self.detector = YOLODetector(self.size, config['yolo_detector'])
  File "/home/tdc/Downloads/FastMOT/fastmot/detector.py", line 183, in __init__
    self.backend = InferenceBackend(self.model, 1)
  File "/home/tdc/Downloads/FastMOT/fastmot/utils/inference.py", line 33, in __init__
    raise RuntimeError('Plugin not found') from err
RuntimeError: Plugin not found

[GeekAlexis]

Did you build the plugin on your Xavier NX with make?

[Myron1996]

Yes.

tdc@tdc:~/Downloads/FastMOT$ cd fastmot/plugins/
tdc@tdc:~/Downloads/FastMOT/fastmot/plugins$ make
computes: 72
NVCCFLAGS: -gencode arch=compute_72,code=[sm_72,compute_72]
make: Nothing to be done for 'all'.

[GeekAlexis]

Can you do make clean and rebuild it just in case if you copied the file from somewhere?

[Myron1996]

Oh that worked. but I got this new error with respect to engine file.

I actually generated the yolov4-tiny.onnx and yolov4-tiny.trt files on Ubuntu PC .
DO I have to build the files on Xavier ?

tdc@tdc:~/Downloads/final qfree/FastMOT$  python3 app.py --input_uri c1.mp4 --mot
Opening in BLOCKING MODE
Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
[ WARN:0] global /home/tdc/opencv_build/opencv/modules/videoio/src/cap_gstreamer.cpp (1044) open OpenCV | GStreamer warning: unable to query duration of stream
[ WARN:0] global /home/tdc/opencv_build/opencv/modules/videoio/src/cap_gstreamer.cpp (1081) open OpenCV | GStreamer warning: Cannot query video position: status=1, value=1, duration=-1
2021-06-14 20:21:56 [    INFO] 1280x720 stream @ 25 FPS
2021-06-14 20:21:56 [    INFO] Loading detector model...
[TensorRT] ERROR: INVALID_CONFIG: The engine plan file is generated on an incompatible device, expecting compute 7.2 got compute 5.2, please rebuild.
[TensorRT] ERROR: engine.cpp (1546) - Serialization Error in deserialize: 0 (Core engine deserialization failure)
[TensorRT] ERROR: INVALID_STATE: std::exception
[TensorRT] ERROR: INVALID_CONFIG: Deserialize the cuda engine failed.
Traceback (most recent call last):
  File "app.py", line 98, in <module>
    main()
  File "app.py", line 51, in main
    draw=draw, verbose=args.verbose)
  File "/home/tdc/Downloads/final qfree/FastMOT/fastmot/mot.py", line 51, in __init__
    self.detector = YOLODetector(self.size, config['yolo_detector'])
  File "/home/tdc/Downloads/final qfree/FastMOT/fastmot/detector.py", line 183, in __init__
    self.backend = InferenceBackend(self.model, 1)
  File "/home/tdc/Downloads/final qfree/FastMOT/fastmot/utils/inference.py", line 44, in __init__
    raise RuntimeError('Unable to load the engine file')
RuntimeError: Unable to load the engine file

[GeekAlexis]

Simply remove the yolov4-tiny.trt file and app.py will rebuild the engine from ONNX on your Jetson.

[GeekAlexis]

TRT engine and the plugin are platform dependent so you can't build them on another PC.

[Myron1996]

It worked. Thanks Bro. BUt my results are showing multiple boxes .
https://user-images.githubusercontent.com/72124771/121952089-b229de00-cd53-11eb-8ad6-7917a024ec6b.mp4

[GeekAlexis]

Are you sure your YOLOv4-tiny parameters are set correctly in yolo.py?

[Myron1996]

Yes I Guess so.
I changed following things

• Engine_path
• Model_Path
• Num_Classes
• Input_Shape
• ANCHORS
• LAYER_FACTORS
• SCALES

yolo.py

class YOLOv4(YOLO):
    ENGINE_PATH = Path(__file__).parent / 'yolov4-tiny.trt' #'yolov4_crowdhuman.trt'
    MODEL_PATH = Path(__file__).parent /  'yolov4-tiny.onnx' #'yolov4_crowdhuman.onnx'
    NUM_CLASSES = 80
    INPUT_SHAPE = (3, 416, 416)
    LAYER_FACTORS = [32, 16]
    SCALES = [1.05, 1.05]
    ANCHORS = [[10, 14, 23, 27, 37, 58],
               [81, 82, 135, 169, 344, 319]]

#    ANCHORS = [[11, 22, 24, 60, 37, 116],
#               [54, 186, 69, 268, 89, 369],
#               [126, 491, 194, 314, 278, 520]]

[GeekAlexis]

Your anchors look wrong. Make sure you follow #23 for filling up the anchors.

[Myron1996]

I actually took my anchors from official yolov4-tiny.cfg file as shown below

[yolo]
mask = 1,2,3
anchors = 10,14,  23,27,  37,58,  81,82,  135,169,  344,319
classes=80

As per your suggestion #23 . Do following anchors are for yolov4-tiny ?

ANCHORS = [[12, 16, 19, 36, 40, 28], [36, 75, 76, 55, 72, 146], [142, 110, 192, 243, 459, 401]]

Do these anchors are for YOLOV4-Tiny for all 80 classes

[GeekAlexis]

That is just an example for YOLOv3 but the approach doesn't change for YOLOv4-tiny. Like I said in README, your anchors are in reverse. Please follow README carefully, issues is not a place to reiterate existing instructions or past questions.

[Myron1996]

Apologies in advance if its too sily to ask this question . I am actually very new to this so . I think I got it. Just to make sure I understand it correctly.

There are two yolo layers in my yolov4-tiny.cfg file as shown below

[yolo]
mask = 3,4,5
anchors = 10,14,  23,27,  37,58,  81,82,  135,169,  344,319
classes=80
num=6
jitter=.3
scale_x_y = 1.05
cls_normalizer=1.0
iou_normalizer=0.07
iou_loss=ciou
ignore_thresh = .7
truth_thresh = 1
random=0
resize=1.5
nms_kind=greedynms
beta_nms=0.6
#new_coords=1
#scale_x_y = 2.0

[route]
layers = -4

[convolutional]
batch_normalize=1
filters=128
size=1
stride=1
pad=1
activation=leaky

[upsample]
stride=2

[route]
layers = -1, 23

[convolutional]
batch_normalize=1
filters=256
size=3
stride=1
pad=1
activation=leaky

[convolutional]
size=1
stride=1
pad=1
filters=255
activation=linear

[yolo]
mask = 1,2,3
anchors = 10,14,  23,27,  37,58,  81,82,  135,169,  344,319
classes=80
num=6
jitter=.3
scale_x_y = 1.05
cls_normalizer=1.0
iou_normalizer=0.07
iou_loss=ciou
ignore_thresh = .7
truth_thresh = 1
random=0
resize=1.5
nms_kind=greedynms
beta_nms=0.6
#new_coords=1
#scale_x_y = 2.0

So the anchors will be

    ANCHORS = [[10, 14, 23, 27, 37, 58, 81, 82, 135, 169, 344, 319],
               [10, 14, 23, 27, 37, 58, 81, 82, 135, 169, 344, 319]]

[Myron1996]

Following anchors worked for YOLOV4-tiny .

    ANCHORS = [[81, 82, 135, 169, 344, 319],
               [23, 27, 37, 58, 81, 82]]

Thank you @GeekAlexis . Really thank you for your explanation and patience.

[GeekAlexis]

@Myron1996 The solution of your issue on Jetson Nano is already covered here. Please stop duplicating issues.

[Myron1996]

Actually, I created the .onnx file and .trt file both on Jetson Nano. I didn’t copy them.It is still giving me sane error

[GeekAlexis]

Your error message clearly says the .trt file was built on a different GPU architecture.