Friday, June 4, 2010

Vision for Robots 3 of n: Hue Segmentation

Previous parts are here.

The goal for this step is to segment the image now that it has been split into hue, saturation, and value. One thing to note is that CvBridge imports the image in BGR not RGB. This may cause some confusion, but the previous information has been clarified.

The first thing is to separate the multi-channel HSV matrix into separate hue and saturation matrices for easy manipulation. Then we will iterate over each pixel to see if its hue is close to orange and if the pixel is colorful. Pixels that match our filter will be set to white and those that do not will be set to black. This information can also be used to mask the background from the output image.

void imageCallback (const sensor_msgs::ImageConstPtr & msg_ptr)
    // Convert ROS Imput Image Message to IplImage
      cv_input_ = bridge_.imgMsgToCv (msg_ptr, "bgr8");
    catch (sensor_msgs::CvBridgeException error)
      ROS_ERROR ("CvBridge Input Error");

    // Convert IplImage to cv::Mat
    img_in_ = cv::Mat (cv_input_).clone ();
    // output = input
    img_out_ = img_in_.clone ();
    // Convert Input image from BGR to HSV
    cv::cvtColor (img_in_, img_hsv_, CV_BGR2HSV);
    // Zero Matrices
    img_hue_ = cv::Mat::zeros(img_hsv_.rows, img_hsv_.cols, CV_8U);
    img_sat_ = cv::Mat::zeros(img_hsv_.rows, img_hsv_.cols, CV_8U);
    img_bin_ = cv::Mat::zeros(img_hsv_.rows, img_hsv_.cols, CV_8U);
    // HSV Channel 0 -> img_hue_ & HSV Channel 1 -> img_sat_
    int from_to[] = { 0,0, 1,1};
    cv::Mat img_split[] = { img_hue_, img_sat_};
    cv::mixChannels(&img_hsv_, 3,img_split,2,from_to,2);

    for(int i = 0; i < img_out_.rows; i++)
      for(int j = 0; j < img_out_.cols; j++)
        // The output pixel is white if the input pixel
        // hue is orange and saturation is reasonable
        if(,j) > 4 &&
 ,j) < 28 &&
 ,j) > 128) {
,j) = 255;
        } else {
,j) = 0;
          // Clear pixel blue output channel
,j*3+0) = 0;
          // Clear pixel green output channel
,j*3+1) = 0;
          // Clear pixel red output channel
,j*3+2) = 0;

    // Display Input image
    cv::imshow ("input", img_in_);
    // Display Binary Image
    cv::imshow ("binary image", img_bin_);
    // Display segmented image
    cv::imshow ("segmented output", img_out_);

    // Needed to  keep the HighGUI window open
    cv::waitKey (3);

To launch the image segmentation demo using prerecorded .bag file

roslaunch ~/ros/stacks/iheart-ros-pkg/ihr_demos/ihr_opencv/launch/demo_segment.launch

If you camera is working and you have an orange you can try the live version.

roslaunch ~/ros/stacks/iheart-ros-pkg/ihr_demos/ihr_opencv/launch/live_segment.launch

Video of Step 3 of n.

In the next step we will use morphological operations to remove noise and smooth the edge of the circles. Then we will perform circle detection to locate each object in the image.


Unknown said...

Hello! I'm trying segmental green objects. I read what you did. How you defined the interval of color orange?

Thank you!


I Heart Robotics said...

if(,j) > 4 &&,j) < 28 &&,j) > 128) {,j) = 255;
} else {

In this section of the code it check if the pixel has a hue that is between 4 and 28. If this is true and it has a saturation higher than 50% the pixel is set to white otherwise it's set to black.

Depending on the RGB to HSV conversion to exact numbers work out a little different but you can get a rough estimate using the color picker in Gimp or any other image editor.

Green should be with a hue in a range around 80-100

I will make sure to cover this in more detail in what will probably be Part 7 of n: Apples vs. Oranges

Wallace said...

Good, i will read when you to post!

This interval no stayed good! But I will try others. I'm having problems with ilumination, what do you suggest for this problem?

I Heart Robotics said...

One option is get a camera driver that allows you to turn off automatic gain control. This will keep the image brightness from constantly changing.

You may want to look at the uvc_cam driver

The easiest way of solving the problem is improving your lighting. Just add more indirect lighting.