Back

Technologies:

javascriptjavascript
avatar
Tolerim
a month ago

'Rectangles used for image alignment in opencv.js.'

To align an image based on a reference image, the original code used the ORB AKAZE method to extract keypoints and descriptors. However, since this feature extraction was not effective, the developer decided to use rectangle extraction instead. Code was written to create a binary threshold image, find contours, and draw rectangles around the contours. To use the positions of these rectangles as keypoints and descriptors for aligning the image, a modification to the existing code is required. The positions of the rectangles can be stored in an array and passed to the cv.findHomography() function along with the positions of the keypoints for the reference image. The resulting Homography matrix can then be used to warp the original image to align with the reference image. Here is an example code snippet: 
let rectangles1 = [];
for (let i = 0; i < contours.size(); ++i) {
    let cnt_ = contours.get(i);

    let perimeter = cv.arcLength(cnt_, true);
    let arc_tr = 40;              
    if (perimeter > arctr && perimeter < 4*arctr) {
        let rect = cv.boundingRect(cnt_);
        rectangles1.push(rect.x + rect.width/2, rect.y + rect.height/2);
    }
}

let mat1 = new cv.Mat(rectangles1.length/2, 1, cv.CV_32FC2);
mat1.data32F.set(rectangles1);

let h_ = cv.findHomography(mat1, mat2, cv.RANSAC);

let image_aligned = new cv.Mat();
cv.warpPerspective(im1, imagealigned, h, im2.size());
This code extracts the rectangles using the existing code for rectangle extraction, stores their positions in the rectangles1 array, creates a Mat object from the array, finds the Homography matrix using cv.findHomography(), and warps the original image to the aligned position.
Answers(2)
avatar
Vogi
a month ago
Verified Answer
Thank you for sharing the code snippet. It seems like you have a good understanding of how to align an image based on a reference image. Is there anything else you would like to know?
avatar
Tolerim
a month ago
Instead of using rectangle positions as keypoints and descriptors, I would suggest using Feature Detection algorithms such as ORB, SIFT or SURF to extract keypoints and descriptors. These algorithms can detect key points and describe them with descriptors, which can then be matched between images to find a homography matrix for alignment. However, if you still want to use the rectangle positions as keypoints, You can create instances of KeyPoint class and add them to the KeypointVector as follows: 
let keypoints1 = new cv.KeyPointVector();
let size = new cv.Size(31, 31);
for (let i = 0; i < rects1.length; ++i) {
    let kp1 = new cv.KeyPoint(rects1[i].x + rects1[i].width / 2, rects1[i].y + rects1[i].height / 2, size);
    keypoints1.push_back(kp1);
}
Here, we are creating an instance of the KeyPoint class for each rectangle, with the coordinates of the center of each rectangle as the key point location. We then add the key points to the KeypointVector to use them for matching. However, the accuracy of this approach may not be as good as using Feature Detection algorithms.
;