Vision for Robots 3 of n: Hue Segmentation

Previous parts are here.

The goal for this step is to segment the image now that it has been split into hue, saturation, and value. One thing to note is that CvBridge imports the image in BGR not RGB. This may cause some confusion, but the previous information has been clarified.

The first thing is to separate the multi-channel HSV matrix into separate hue and saturation matrices for easy manipulation. Then we will iterate over each pixel to see if its hue is close to orange and if the pixel is colorful. Pixels that match our filter will be set to white and those that do not will be set to black. This information can also be used to mask the background from the output image.

void imageCallback (const sensor_msgs::ImageConstPtr & msg_ptr)
  {
    // Convert ROS Imput Image Message to IplImage
    try
    {
      cv_input_ = bridge_.imgMsgToCv (msg_ptr, "bgr8");
    }
    catch (sensor_msgs::CvBridgeException error)
    {
      ROS_ERROR ("CvBridge Input Error");
    }

    // Convert IplImage to cv::Mat
    img_in_ = cv::Mat (cv_input_).clone ();
    // output = input
    img_out_ = img_in_.clone ();
    // Convert Input image from BGR to HSV
    cv::cvtColor (img_in_, img_hsv_, CV_BGR2HSV);
    // Zero Matrices
    img_hue_ = cv::Mat::zeros(img_hsv_.rows, img_hsv_.cols, CV_8U);
    img_sat_ = cv::Mat::zeros(img_hsv_.rows, img_hsv_.cols, CV_8U);
    img_bin_ = cv::Mat::zeros(img_hsv_.rows, img_hsv_.cols, CV_8U);
    // HSV Channel 0 -> img_hue_ & HSV Channel 1 -> img_sat_
    int from_to[] = { 0,0, 1,1};
    cv::Mat img_split[] = { img_hue_, img_sat_};
    cv::mixChannels(&img_hsv_, 3,img_split,2,from_to,2);

    for(int i = 0; i < img_out_.rows; i++)
    {
      for(int j = 0; j < img_out_.cols; j++)
      {
        // The output pixel is white if the input pixel
        // hue is orange and saturation is reasonable
        if(img_hue_.at(i,j) > 4 &&
           img_hue_.at(i,j) < 28 &&
           img_sat_.at(i,j) > 128) {
          img_bin_.at(i,j) = 255;
        } else {
          img_bin_.at(i,j) = 0;
          // Clear pixel blue output channel
          img_out_.at(i,j*3+0) = 0;
          // Clear pixel green output channel
          img_out_.at(i,j*3+1) = 0;
          // Clear pixel red output channel
          img_out_.at(i,j*3+2) = 0;
        }
      }
    }

    // Display Input image
    cv::imshow ("input", img_in_);
    // Display Binary Image
    cv::imshow ("binary image", img_bin_);
    // Display segmented image
    cv::imshow ("segmented output", img_out_);

    // Needed to  keep the HighGUI window open
    cv::waitKey (3);
  }

To launch the image segmentation demo using prerecorded .bag file

roslaunch ~/ros/stacks/iheart-ros-pkg/ihr_demos/ihr_opencv/launch/demo_segment.launch


If you camera is working and you have an orange you can try the live version.

roslaunch ~/ros/stacks/iheart-ros-pkg/ihr_demos/ihr_opencv/launch/live_segment.launch



Video of Step 3 of n.

In the next step we will use morphological operations to remove noise and smooth the edge of the circles. Then we will perform circle detection to locate each object in the image.