How can I get a better segmask image by using the image.py of perception packages.

[LHJ1098826475]

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04):Ubuntu 16.04
• Python version:2.7
• Installed using pip or ROS: ROS
• Camera: xtion
• Gripper: parallel jaw
• Robot: TIAGo
• GPU model (if applicable):

I want to get better segmask image from color image. I attemp to use the method of image.py in perception packages. But I don't get better segmask image, so that I can't to handle object grasp.
figure 1 and 2 is the original color and segmask image which is the example image of data/example/clutter/primesense. Then, the picture 3 that I get by using ColorImage.to_binary() method. Please tell me how to get better segmask image. Thanks.

[ltorquato4]

Hi,
this is the best approach I found using Perception.
I hope it helps you :)

# import 
import cv2 
from perception import ColorImage, BinaryImage

# parameters, change them, test and optimize for your use case
BINARY_IM_MAX_VAL = np.iinfo(np.uint8).max
BINARY_IM_DEFAULT_THRESH = BINARY_IM_MAX_VAL / 2
LOW_GRAY = 70
UPPER_GRAY = 250
AREA_THRESH_DEFAULT = 1000  
DIST_THRESH_DEFAULT = 20
FRAME = 'name_of_your_rgbd_camera_' # in my case is realsense_overhead

def binary_segmask(image, background, output_dir, filename):
    """
    Create a binary image from the color image 
    :param image: rgb image created with RGBD camera 
    :param background: path to background image to use 
    :param output_dir: path to output dir for processed images
    :param filename: name for saving data
    :return: binary_subtract_pruned
    """

    background = cv2.imread(background)
    # convert img to grayscale
    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    background_gray = cv2.cvtColor(background, cv2.COLOR_BGR2GRAY)
    # subtract foreground from background
    subtract = background_gray - gray
    # in range for grayscale values
    subtract_im_filtered = cv2.inRange(subtract, LOW_GRAY, UPPER_GRAY)
    # open as ColorImage
    subtract_im_filt = ColorImage(subtract_im_filtered, FRAME)  
    # convert to BinaryImage
    binary_subtract = subtract_im_filt.to_binary(threshold=BINARY_IM_DEFAULT_THRESH)
    # Prune contours
    binary_subtract_pruned = binary_subtract.prune_contours(area_thresh=AREA_THRESH_DEFAULT, dist_thresh=DIST_THRESH_DEFAULT)
    # save binary to npy and png format
    np.save('%s/%s_binary.npy' % (output_dir, filename), binary_subtract_pruned._image_data())
    cv2.imwrite('%s/%s_binary.png' % (output_dir, filename), binary_subtract_pruned._image_data())
    return binary_subtract_pruned

[titouanlh]

I'm using a grabcut algorithm with my realsense camera (https://dev.intelrealsense.com/docs/rs-grabcuts) which is supposed to generate better segmasks but I don't know yet if it improves the grasp-planning performances significantly.

[ericwyy]

Hi, this is the best approach I found using Perception. I hope it helps you :)

# import 
import cv2 
from perception import ColorImage, BinaryImage

# parameters, change them, test and optimize for your use case
BINARY_IM_MAX_VAL = np.iinfo(np.uint8).max
BINARY_IM_DEFAULT_THRESH = BINARY_IM_MAX_VAL / 2
LOW_GRAY = 70
UPPER_GRAY = 250
AREA_THRESH_DEFAULT = 1000  
DIST_THRESH_DEFAULT = 20
FRAME = 'name_of_your_rgbd_camera_' # in my case is realsense_overhead

def binary_segmask(image, background, output_dir, filename):
    """
    Create a binary image from the color image 
    :param image: rgb image created with RGBD camera 
    :param background: path to background image to use 
    :param output_dir: path to output dir for processed images
    :param filename: name for saving data
    :return: binary_subtract_pruned
    """

    background = cv2.imread(background)
    # convert img to grayscale
    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    background_gray = cv2.cvtColor(background, cv2.COLOR_BGR2GRAY)
    # subtract foreground from background
    subtract = background_gray - gray
    # in range for grayscale values
    subtract_im_filtered = cv2.inRange(subtract, LOW_GRAY, UPPER_GRAY)
    # open as ColorImage
    subtract_im_filt = ColorImage(subtract_im_filtered, FRAME)  
    # convert to BinaryImage
    binary_subtract = subtract_im_filt.to_binary(threshold=BINARY_IM_DEFAULT_THRESH)
    # Prune contours
    binary_subtract_pruned = binary_subtract.prune_contours(area_thresh=AREA_THRESH_DEFAULT, dist_thresh=DIST_THRESH_DEFAULT)
    # save binary to npy and png format
    np.save('%s/%s_binary.npy' % (output_dir, filename), binary_subtract_pruned._image_data())
    cv2.imwrite('%s/%s_binary.png' % (output_dir, filename), binary_subtract_pruned._image_data())
    return binary_subtract_pruned

Hi, the following error occurs when I run:
cannot import name 'ColorImage' from 'perception
cannot import name 'BinaryImage' from 'perception
Thanks.

[fzoric8]

@ericwyy you're missing Perception module which can be found here.

[ericwyy]

@ericwyy you're missing Perception module which can be found here.
@fzoric8 Thank you for your prompt reply. And I replace 'from perception import ColorImage, BinaryImag' with 'from autolab_core import ColorImage, BinaryImage' and a new error has occurred as follows:
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
cv2.error: OpenCV(4.6.0) 👎 error: (-5:Bad argument) in function 'cvtColor'
Overload resolution failed:

• src is not a numpy array, neither a scalar
• Expected Ptrcv::UMat for argument 'src'