joshdoe/gst-plugins-vision tutorial

[mgabell]

Hi,

I have just compiled and installed the gst-plugin on Ubuntu. Worked like a char,. But I have not been able to
open a stream to my Basler camera. Can anyone explain how I write the gst-launch string?
I assumed it would work like this:

gst-launch-1.0 pylonsrc ! autovideoconvert ! autovidosink

but noo.

Ideas? I want to use this in python in the end so I guess, once the gst-launcher works I can use it in Python too?

br,
M

[yair-mantis]

try
gst-launch-1.0 pylonsrc ! queue ! videoconvert ! autovidosink

note the queue , also when you say it doesnt work, can you be more specific. do you get an error ?

[mgabell]

Hi, thank you for responding...
I get WARNING: erroneous pipeline: no element "pylonsrc"

I get the pylon viewer software to recognize the camera so it is connected.
I get a feeling that gst-launch does not recognize the plugin. Perhaps I have missed some environment variables?

I noticed, when trying to rebuild that It was suggesting to install optional packages on my system and among all Basler was there. However I do have the Basler Pylon and pypylon installed. What am I missing?

My pylon files are in /opt/pylon
I have downloaded and installed not only pylon viewer but I also downloaded pylon_6.1.1.19861_x86_64_setup.tar.gz
I followed the installation instructions in the file. No errors.

Do I need to specify somewhere when building that I have pylon in /opot/pylon?

[yair-mantis]

you didnt tell gstreamer where your gst-plugins-vision plugins are located.
follow the project readme , of note
https://github.com/joshdoe/gst-plugins-vision#installation

to verify you have the plugins loaded and in order,
gst-inspect-1.0 | grep pylon

[mgabell]

Hi, nope. It does not show up. It seems its not loaded properly

[yair-mantis]

gst-inspect-1.0 -b
will output loaded plugins that have errors

[mgabell]

I get 226 blacklisted files
error.txt

[mgabell]

gst_error.txt
I get alot of these messages too

[mgabell]

I found similar issue on stackoverflow but I dont understand the solution. "Gstreamer-WARNING **: Failed to load plugins while they are existent"

The suggestiopn is to reconfigure and rebuild the libraries with the sources.
Or apt-get install packages or use symlink to the libraries due to naming and version issues. But neither I see can fit
this error I get

[joshdoe]

Seems to be a shared library problem to me. Also, looks like you included your Pylon directory in GST_PLUGIN_PATH, don't do that, only have the directory containing libgstpylon.dll be in that environment variable. I start with gst-inspect-1.0 --gst-debug=*:3 /full/path/to/libgstpylon.dll, and that is sometimes helpful. If the error shows it failed to load, then run ldd /full/path/to/libgstpylon.dll. Carefully examine this list, and make sure all the libraries are resolved. Then I'd run ldd also on your Pylon viewer, and make sure they point to the same versions of the underlying Pylon libraries.

[mgabell]

Where is the location of the libgstpylon.dll? I cant find it anywhere

[joshdoe]

Sorry libgstpylon.dll is the plugin generated by this project on Windows, on Linux it would be named with .so, maybe without the lib prefix, I can't recall.

[mgabell]

I started over with the installation and when trying to run sudo cmake .. I get: Could NOT find PYLON (missing: PYLON_INCLUDE_DIR PYLON_LIBRARIES)
But they are in /opt/pylon.
How can I get the installation to find it?

[mgabell]

cmakeError.txt
Here is the full output from cmake.

[mgabell]

I have now changed to path in FindPylon.cmake folders
The locations are:
/opt/pylon
/include
/include/pylonc
/lib.
The Development/... is not existing nor the x64 under lib.

[joshdoe]

Look at cmake/modules/FindPylon.cmake. You need to set PYLON_DIR to point to your installation, it defaults to /opt/pylon5 on non-Windows platforms, and it seems your's is in /opt/pylon.

[mgabell]

Hi, I did that but it did not help. It seems like the entire structure of the pylon lib has changed. I will see if I can revert to an older version of pylon

[mgabell]

Hi, I rolback to pylon5. Managed to compile.
I set the environment variable to my sys folder where the libgstpylon.so file is located.
I can now see the pylonsrc using gst-inspect.
I get an Error: From element /GstPipeline:pipeline0/GstPylonsrc.... Data stream error.
All in all it seems to go quite far now :D

The pylonviewer works...

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Got context from element 'autovideosink0': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"(GstGLDisplayX11)\ gldisplayx11-0";
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstPylonSrc:pylonsrc0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:pipeline0/GstPylonSrc:pylonsrc0:
streaming stopped, reason error (-5)
Execution ended after 0:00:00.727173090
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

[mgabell]

Hi. I got it working by reinstalling my VM from scratch. Installed latest G-streamer and installed version 5.20.13457-x86_64 from pylon. Then followed your instructions.

If you have the time I would appreciate if you could see it could work with pylon 6.1.1. I will try that too from a fresh installation.
I don't know if that has any impact perhaps they have updated the drivers for frame-grab. I used the pylon viewer and got alot of frame-drop. So I reduced it alot and got a stable image. Might just be my VM but you never know.

Now its time to test in python to see if I get the image to cv2

thanks so far for your support. most appreciated that you make the time!

I thought I got it working. It flashes by and I get:

Error: from element /GstPipeline:pipeline0/GstPylonSrc:pylonsrc0: Internal data stream error.
Streaming stopped, reason not-negotiated (-4)

I got an image once then it stopped working. The Pylon software works fine, though I have to pull down the bandwidth or I get to much frame drop or application freeze.
I dont know if that relates to the gstreamer though
I alsso see now that it can not pick up the OpenCV library. Dont know if that is vital but regardless of adding it to env, it ignores it. it was located in /usr/include/opencv4/opencv2

I try and try and try.. Like an idiot :D
I removed the queue and I get "not enough buffering available for..." and I see one frame in the window.
I then add the Queue again and I get back to Internal data stream error...

[mrstecklo]

Hi, @mgabell.
Those error messages are not very informative. Try setting GST_DEBUG, e.g.:
GST_DEBUG=2,pylonsrc:6 gst-launch-1.0 pylonsrc ! videoconvert ! ximagesink
There were fresh commits since you opened this issue, so also try git pull and rebuild.

Historically, plugin sets some camera properties to default values even if they are not specified.
You could try configuring camera with PylonViewer and then running
GST_DEBUG=2,pylonsrc:6 gst-launch-1.0 pylonsrc ignore-defaults ...
That would ensure properties are kept unchanged.
Or you could configure camera in PylonViewer, then go Camera->Save Features... to export .PFS file with all properties set up.
Then you run
GST_DEBUG=2,pylonsrc:6 gst-launch-1.0 pylonsrc config-file=/path/to/your.pfs ...

Since you say that PylonViewer drops a lot of frames, I can assume that you use GigE camera. In that case you could try this branch and run
GST_DEBUG=2,pylonsrc:6 gst-launch-1.0 pylonsrc config-file=/path/to/your.pfs failrate=10 grabtimeout=5000 ...
This document can help understanding other GigE transmission related properties

[mgabell]

I will go right at it. I will try with the new version 6 of pylon libs and see if that helps. I will also get the error messages.
I use a USB3 camera not the GigE.

Just to emphasize that the FindPylon.cmake has not the correct paths to Pylon6. If you update functionality it would be good if you could see how the new structure of the Pylon 6 library is. The Development/include does not seem to exist anymore, not in 5. I cant find an SDK either on Basler download that would make sense to install, I just go with the Pylon Software.

[mgabell]

Hi, Here is the full error using Pylon 6 . I changed folder in cmake from /opt/pylon5/ to /opt/pylon and /lib64 to /lib.
This compiles without error.
I set the environment variable to the GST.so path for pylon "dll" to:
/usr/lib/gstreamer-1.0/libgstpylon.se
I tried chmod it to 755 also. But gstreamer could not load the plugin so I had to set the environment variable to the compiled folder instead $home/Projects/gst-plugins-vision/build/sys/....
If i didnt gstreamer could not load the plugin.

So. I get the image to flash by and I get the output below...

error_1.txt

[mgabell]

So I went into Pylon Software and lowered bandwidth alot. Set it to 80. I saved the file but did not include it when starting gstreamer.
But when running now the g-streamer it SEEMS to be stable. I will try to include it also in the gstreamer syntax.
I still get "WARN: Not enough buffering available for the processing deadline of 0:00:00.0150000, add enough queues to buffer 0:00:00.0150000 additional data. Shorteing processing latanecy to 00:00:00.0000000.

oohh I added queue as Josh described. That worked. Also including the settings file does not produce error so I HOPE now that the gstreamer is stable.

Can you also provide a way to run this in Python3?
I will start myself to utilize the code I already have. But if you have anything top of your head :D
Is the gstr for cv2.videocapture the same but send the Gstream to appsink?
Now I am exited :D Thank you for the support so far. It has been Invaluable!

[mgabell]

So. Now I have issues in Python. I use. I have tried to find something related to this but without luck.
gst_str = ('pylonsrc ! '
'queue ! '
'videoconvert ! appsink').format(width, height)
return cv2.VideoCapture(gst_str, cv2.CAP_GSTREAMER)

---

The first error about video position can be ignored, 'cuase there is not captured image in frame0 but the rest is unexplainable.

---

I get error:
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (935) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
0:00:00.669657014 17206 0x31508f0 pylonsrc gstpylonsrc.c:3762:video_frame_free: PylonC error: PylonC error #c3000004 'Invalid stream grabber handle' (0xc3000004).

0:00:00.669712039 17206 0x31508f0 pylonsrc gstpylonsrc.c:3762:video_frame_free: PylonC error: Function 'PylonStreamGrabberQueueBuffer' failed in source file '../../../../../../../Pylon/PylonC/PylonC/PylonC.cpp' line #3012

Process finished with exit code 0

Code:
import cv2
import sys

def open_cam(width, height):
# We want to set width and height here, otherwise we could just do:
# return cv2.VideoCapture(dev)
gst_str = ('pylonsrc ! '
'queue ! '
'videoconvert ! appsink').format(width, height)
return cv2.VideoCapture(gst_str, cv2.CAP_GSTREAMER)

def main():
print(cv2.version)
cap = open_cam(2048, 2048)
if not cap.isOpened():
sys.exit("Failed to open camera!")

if name== "main":
main()
``

[mgabell]

sudo python3 main.py
I edited my response cause I had an error in my python code. No space in the code between pylon config and exclamation mark.

main.py:4: PyGIWarning: Gst was imported without specifying a version first. Use gi.require_version('Gst', '1.0') before import to ensure that the right version gets loaded.
from gi.repository import GObject, Gst
4.2.0
0:00:00.775173843 18785 0x2b0c440 WARN pylonsrc gstpylonsrc.c:1468:feature_readable: Feature is not readable: SensorReadoutMode
0:00:00.775734205 18785 0x2b0c440 WARN pylonsrc gstpylonsrc.c:1468:feature_readable: Feature is not readable: LightSourcePreset
0:00:00.775840248 18785 0x2b0c440 WARN pylonsrc gstpylonsrc.c:1468:feature_readable: Feature is not readable: BslLightSourcePreset
0:00:00.775895637 18785 0x2b0c440 WARN pylonsrc gstpylonsrc.c:1468:feature_readable: Feature is not readable: LightSourceSelector
0:00:00.778033598 18785 0x2b0c440 WARN pylonsrc gstpylonsrc.c:1468:feature_readable: Feature is not readable: BalanceWhiteAuto
0:00:00.780949416 18785 0x2b0c440 WARN pylonsrc gstpylonsrc.c:1468:feature_readable: Feature is not readable: ColorAdjustmentEnable
0:00:00.785372597 18785 0x2b0c440 WARN pylonsrc gstpylonsrc.c:1482:feature_alias_readable: Feature is not readable: ColorAdjustmentSelector or BslColorAdjustmentSelector
0:00:00.789781381 18785 0x2b0c440 WARN pylonsrc gstpylonsrc.c:1482:feature_alias_readable: Feature is not readable: ColorAdjustmentSelector or BslColorAdjustmentSelector
0:00:00.789860053 18785 0x2b0c440 WARN pylonsrc gstpylonsrc.c:1482:feature_alias_readable: Feature is not readable: ColorAdjustmentSelector or BslColorAdjustmentSelector
0:00:00.789931093 18785 0x2b0c440 WARN pylonsrc gstpylonsrc.c:1482:feature_alias_readable: Feature is not readable: ColorAdjustmentSelector or BslColorAdjustmentSelector
0:00:00.789983600 18785 0x2b0c440 WARN pylonsrc gstpylonsrc.c:1482:feature_alias_readable: Feature is not readable: ColorAdjustmentSelector or BslColorAdjustmentSelector
0:00:00.790036109 18785 0x2b0c440 WARN pylonsrc gstpylonsrc.c:1482:feature_alias_readable: Feature is not readable: ColorAdjustmentSelector or BslColorAdjustmentSelector
0:00:00.790088639 18785 0x2b0c440 WARN pylonsrc gstpylonsrc.c:1482:feature_alias_readable: Feature is not readable: ColorAdjustmentSelector or BslColorAdjustmentSelector
0:00:00.790140744 18785 0x2b0c440 WARN pylonsrc gstpylonsrc.c:1482:feature_alias_readable: Feature is not readable: ColorAdjustmentSelector or BslColorAdjustmentSelector
0:00:00.790193382 18785 0x2b0c440 WARN pylonsrc gstpylonsrc.c:1482:feature_alias_readable: Feature is not readable: ColorAdjustmentSelector or BslColorAdjustmentSelector
0:00:00.790246786 18785 0x2b0c440 WARN pylonsrc gstpylonsrc.c:1482:feature_alias_readable: Feature is not readable: ColorAdjustmentSelector or BslColorAdjustmentSelector
0:00:00.790299678 18785 0x2b0c440 WARN pylonsrc gstpylonsrc.c:1482:feature_alias_readable: Feature is not readable: ColorAdjustmentSelector or BslColorAdjustmentSelector
0:00:00.790352786 18785 0x2b0c440 WARN pylonsrc gstpylonsrc.c:1482:feature_alias_readable: Feature is not readable: ColorAdjustmentSelector or BslColorAdjustmentSelector
0:00:00.794595798 18785 0x2b0c440 WARN pylonsrc gstpylonsrc.c:1468:feature_readable: Feature is not readable: ColorTransformationSelector
0:00:00.796671401 18785 0x2b0c440 WARN pylonsrc gstpylonsrc.c:1482:feature_alias_readable: Feature is not readable: DemosaicingMode or PgiMode
0:00:00.796739760 18785 0x2b0c440 WARN pylonsrc gstpylonsrc.c:1482:feature_alias_readable: Feature is not readable: NoiseReduction or NoiseReductionAbs
0:00:00.796793924 18785 0x2b0c440 WARN pylonsrc gstpylonsrc.c:1482:feature_alias_readable: Feature is not readable: SharpnessEnhancement or SharpnessEnhancementAbs
0:00:00.806128489 18785 0x28d3460 FIXME default gstutils.c:3980:gst_pad_create_stream_id_internal:pylonsrc0:src Creating random stream-id, consider implementing a deterministic way of creating a stream-id
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (935) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
0:00:01.029586614 18785 0x2b0c440 pylonsrc gstpylonsrc.c:3762:video_frame_free: PylonC error: PylonC error #c3000004 'Invalid stream grabber handle' (0xc3000004).

0:00:01.029719083 18785 0x2b0c440 pylonsrc gstpylonsrc.c:3762:video_frame_free: PylonC error: Function 'PylonStreamGrabberQueueBuffer' failed in source file '../../../../../../../Pylon/PylonC/PylonC/PylonC.cpp' line #3012

So close :(

[mrstecklo]

gstpylonsrc.c:3762:video_frame_free: PylonC error: PylonC error #c3000004 'Invalid stream grabber handle' (0xc3000004).

This is very strange. video_frame_free is a callback which can only be called after src->streamGrabber is valid.
I'm not familiar with Python, but the first thing that comes to my mind is that pipeline is destroyed right after initialization. Maybe lifetime of the the object that manages it ends

[mrstecklo]

def main():
print(cv2.version)
cap = open_cam(2048, 2048)
if not cap.isOpened():
sys.exit("Failed to open camera!")

Don't you need some kind of busy loop to keep interpreter inside of main?

[mgabell]

def main():
print(cv2.version)
cap = open_cam(2048, 2048)
if not cap.isOpened():
sys.exit("Failed to open camera!")

Don't you need some kind of busy loop to keep interpreter inside of main?

Yes you do of course, but I cant even open the Stream so the loop would not start. The sys exit would break before coming to the loop. In a full implementation I would create a main gui thread loop that waits for exit command and the camera grabbing would be managed in a second thread that would queue to a share memory,
But since the open_cam() wont return a proper handle to the camera capture (cv2.videocapture()) It will break, hence no need to further write the code :D I was just lazy when testing the plugin