In OpenCV, images can be RGB/BGR, HSV, grayscaled, black-white and so on. It is crucial to know the data type before dealing with images.
The image data types are mainly
CV_8UC3 (Matrix of uchar with 3 channels) and CV_8U (Matrix of uchar with 1 channel), however, the conversion to other types such as CV_32FC3, CV_64F are also possible. (see data types)
Consider, the image is an RGB image which is read by
Mat rgb = imread('path/to/rgb/image', CV_LOAD_IMAGE_COLOR); //to set RED pixel value of (i,j)th to X, rgb.at<Vec3b>(i,j) = X;
Similarly, if the image is grayscaled,
gray.at<uchar>(i,j) = X;
Note that, in OpenCV, Black&White images are stored as CV_8U type with the values 0 and 255. Therefore, changing BW images are same as gray images.