opencv Using Cascade Classifiers In Java Detections within Detections

Example

This example uses Dice and the black spots on the dice (the pips) as our object. As the example is quite long, first explaining some key concepts is critical to understanding the example.

Understanding the first example, "Getting a static image, detecting items on it, and outputting the results." is critical to understanding this example, especially how OpenCV draws rectangles.

Take a look at the following image:

We will be using the subimaging method, where we use a detected area as our base for applying more detections. This is only possible if an object will always be within another object that we can detect, such as our pips on our dice. This method has several benefits:

• Insteading scanning the entire image, we only need to scan the area where we know the object will be in.
• Removes any chance of false positives outside the detection area.

We do this by first applying one cascade classifier scan over the the entire image to give us an MatOfRect object containing our large objects (dice, in this case). We then iterate over the Rect[] array given by the toArray() function from the MatOfRect object. This Rect object is used in creating a temporary Mat object that is "cropped" to the Rect object's properties (x, y, width, height) from the original image, where we can then perform detections on the temporary Mat object. In other words, we tell the classifier to only perform detections on the dice parts of the image instead, and we specify the position of each dice by using the Rect objects that we got from performing a detection on the entire image.

However, the Rect objects (pips) have their properties relative to their dice, and not the image itself. To solve this issue, when we want to draw rectangles to the actual image showing the pips' locations, we add both dice.x and dice.y to the start Point.

import org.opencv.core.Mat;
import org.opencv.core.MatOfRect;
import org.opencv.core.Point;
import org.opencv.core.Rect;
import org.opencv.core.Scalar;
import org.opencv.core.Size;
import org.opencv.imgcodecs.Imgcodecs;
import org.opencv.imgproc.Imgproc;
import org.opencv.objdetect.CascadeClassifier;
import org.opencv.videoio.VideoCapture;

public class Classifier {

    private CascadeClassifier diceCascade =
        new CascadeClassifier("res/newMethod/diceCascade.xml");
    private CascadeClassifier pipCascade =
        new CascadeClassifier("res/newMethod/pipCascade6.xml");
    private VideoCapture vc = new VideoCapture();
    private Mat image;

    public void openVC(int index) {
        vc.open(index);
    }

    public void closeVC() {
        vc.close();
    }

    public Mat getNextImage() {
        image = new Mat();
        vc.read(image); // Sets the matrix to the current livestream frame.
        
        MatOfRect diceDetections = new MatOfRect(); // Output container
    
        // See syntax for explainations on addition parameters
        diceCascade.detectMultiScale(image, diceDetections, 1.1, 4, 0, new Size(20, 20),
            new Size(38, 38));
        
        // Iterates for every Dice ROI
        for (int i = 0; i < diceDetections.toArray().length; i++) {
            Rect diceRect = diceDetections.toArray()[i];
            
            // Draws rectangles around our detected ROI
            Point startingPoint = new Point(diceRect.x, diceRect.y);
            Point endingPoint = new Point(diceRect.x + diceRect.width,
                diceRect.y + diceRect.height);
            Imgproc.rectangle(image, startingPoint, endingPoint, new Scalar(255, 255, 0));
            
            MatOfRect pipDetections = new MatOfRect();
            
            pipCascade.detectMultiScale(image.submat(diceRect), pipDetections, 1.01, 4, 0,
                new Size(2, 2), new Size(10, 10));
            
            // Gets the number of detected pips and draws a cricle around the ROI
            for (int y = 0; y < pipDetections.toArray().length; y++) {
                // Provides the relative position of the pips to the dice ROI
                Rect pipRect = pipDetections.toArray()[y];
                
                // See syntax explaination
                // Draws a circle around our pips
                Point center = new Point(diceRect.x + pipRect.x + pipRect.width / 2,
                    diceRect.y + pipRect.y + pipRect.height / 2);
                Imgproc.ellipse(image, center, new Size(pipRect.width / 2, pipRect.height / 2),
                     0, 0, 360, new Scalar(255, 0, 255), 1, 0, 0);
            }
        }
        
        return image;
    }
}

The getNextImage() function returns a Mat object, which with in conjunction with the other examples posted, can be called constantly and can be converted to a BufferImage, to provide a livestream displaying detections.