Artist Utility

Utility artist

kalmus.utils.artist.color_of_regions(labels, original_image)

Compute average color and brightest color of the regions and record the relative size of the regions as a ratio with respect to the size of whole image.

Parameters

• labels (numpy.ndarray) -- The labeled image. Expected shape==height x width. Integer labels

• original_image (numpy.ndarray) -- The original color image corresponding to the label image

Returns

A list of average color of the regions, a list of brightest color of the regions, and a list of sizes of the regions. The order of the regions in list is the same as they are in labeled image

Return type

(list, list, list)

kalmus.utils.artist.compute_brightest_color_and_brightness(grey_image, color_image, return_min=False, gaussian_blur=False, blur_radius=15)

Find the brightest pixel in an image and return the color and brightness at that pixel

Parameters

• grey_image (numpy.ndarray) -- The greyscale image. single channel 2D image. Expected shape==(row/height, col/width)

• color_image (numpy.ndarray) -- The corresponding color image. Expected shape==(row/height, col/width, channels)

• return_min (bool) -- If true return the darkest color and brightness (of a pixel) as well

• gaussian_blur (bool) -- Whether to apply a gaussian filter before finding the brightest point the grey image

• blur_radius (int) -- The radius of the gaussian filter

Returns

The color and brightness of the brightest pixel (, color and brightness of the darkest pixel)

Return type

(numpy.ndarray, int, tuple)

kalmus.utils.artist.compute_dominant_color(image, n_clusters=3, max_iter=10, threshold_error=1.0, attempts=10)

Compute the dominant color of an input image using the kmeans clustering. The centers of the clusters are the dominant colors of the input image

Parameters

• image (numpy.ndarray) -- input image. Either a multi-channel color image or a grayscale image (2D image) Expected shape of image is height x width x (channels)

• n_clusters (int) -- number of clusters

• max_iter (int) -- maximum iterations before terminating kmeans clustering

• threshold_error (float) -- threshold error for terminating kmeans clustering. kmean clustering terminate when the errors between the current computed and previous computed cluster center is under this threshold

• attempts (int) -- Number of attempts to rerun the Kmeans clustering with different initial cluster centers. Since kmeans clustering randomly choose n number of cluster in its initialization process.

Returns

An array of n cluster centers and an array of relative size (in percentage) of the clusters. Expected output shape: n_clusters x channels (cluster centers), n_clusters (relative size in percentage) E.g. for an input color image with 3 channels and n_clusters = 3 Output can be [[255, 255, 255], [126, 75, 198], [186, 145, 122]], [0.4, 0.5, 0.1]

Return type

(numpy.ndarray, numpy.ndarray)

kalmus.utils.artist.compute_mean_color(image)

Compute the average/mean color of the input multi-channel image or greyscale image.

Parameters

image (numpy.ndarray) -- input image. Either a multi-channel color image or a single channel greyscale image

Returns

The average color of the image (averaged across the channels). shape==channels.

Return type

numpy.ndarray

kalmus.utils.artist.compute_median_color(image)

Compute the median color of the input multi-channel color image or a single channel greyscale image.

Parameters

image (numpy.ndarray) -- input image. Either a multi-channel color image or a single channel greyscale image

Returns

The median color of the image (median values of channels), shape==channels

Return type

numpy.ndarray

kalmus.utils.artist.compute_mode_color(image, bin_size=10)

compute the mode color of an input image

Parameters

• image (numpy.ndarray) -- either a multi-channel color image or a single channel greyscale image Expected shape of image: height x width x (channels)

• bin_size (int) -- Histogramize the input image. Color/intensity of each pixel in the image will be accumulate in the bins with size==bin_size. The output mode color/intensity is always an integer multiple of the bin_size.

Returns

The mode color of the image (modes of the channels), shape=channels, and counts of the mode colors happened in the input image, shape==channels

Return type

(numpy.ndarray, numpy.ndarray)

kalmus.utils.artist.compute_percents_of_labels(label)

Compute the ratio/percentage size of the labels in an labeled image

Parameters

label (numpy.ndarray) -- the labeled 2D image

Returns

An array of relative size of the labels in the image. Indices of the sizes in the array is corresponding to the labels in the labeled image. E.g. output [0.2, 0.5, 0.3] means label 0's size is 0.2 of the labeled image, label 1' size is 0.5 of the labeled image, and label 2's size is 0.3 of the labeled image.

Return type

numpy.ndarray

kalmus.utils.artist.contrast_between_regions(region_colors, matrix, region_weights=None)

Compute the contrast between the segmented regions in image using the color of regions and adjacency matrix

Parameters

• region_colors (list) -- A list of colors of segmented regions

• matrix (numpy.ndarray) -- A 2D adjacency matrix that describe the spatial relationship between the regions Expect a binary matrix, where 1 means adjacent and 0 means non-adjacent

• region_weights (list) -- A 1D array of weights that can be applied onto the contrast calculate for each regions. e.g. Regions may have different sizes, and you can weight the computed contrast by the size of the regions. Expected shape of the regions weight is shape==number of regions

Returns

A 2D numpy matrix where each entry is the contrast between the row indexed region and column indexed region. Contrast ratio >= 1, an entry with 0 means row indexed region and column indexed region are non-adjacent

Return type

list

kalmus.utils.artist.contrast_ratio(color1, color2)

Compute the contrast ratio between two 24-bits RGB colors in range [0, 255] see https://www.w3.org/TR/2008/REC-WCAG20-20081211/#contrast-ratiodef for more references

Parameters

• color1 (numpy.ndarray) -- one of the 24-bits RGB colors in range [0, 255]

• color2 (numpy.ndarray) -- one of the 24-bits RGB colors in range [0, 255]

Returns

the contrast ratio between two 24-bits RGB colors. Contrast ratio >= 1

Return type

float

kalmus.utils.artist.find_bright_spots(image, n_clusters=3, blur_radius=21, amount_of_bright_parts=0.8, return_all_pos=False)

Find the indices location of the top-k brightest spots in an color image.

Parameters

• image (numpy.ndarray) -- input image. Must be an mutli-channel RGB color image

• n_clusters (int) -- expected number of clusters/brightest spots in the input image

• blur_radius (int) -- radius of the Gaussian blur kernel that used to smooth the image

• amount_of_bright_parts (float) -- amount of bright parts in an image. Used to find the lower bound for distinguishing the bright and non-bright part of the input image. Range of amount_of_bright_parts is in [0, 1] (all non-bright -> all bright)

Returns

The location of centers of top-3 bright spots (with irregular shape), percentage of dominance of each spot (relative size of the spot)

Return type

(numpy.ndarray, numpy.ndarray)

kalmus.utils.artist.find_letter_box_from_videos(video, num_sample=30)

Find the position of letterbox (black bars around the scene) in the input cv2 video object. The function samples out num_sample (20 by default) number of frames from cv2 video object, and find the position of letterbox in each frame. The function uses the median results from all sampled frames as the final position of video's letterbox.

Notice that the function assumes the letterbox is black or very close to black.

Parameters

• video (cv2.VideoCapture) -- Input video object captured by cv2.VideoCapture()

• num_sample (int) -- Number of frames to sample from video for finding letterbox

Returns

The smaller row index of letterbox (bound for upper horizontal letterbox), the larger row index of letterbox (bound for lower horizontal letterbox), the smaller col index of letterbox (bound for left vertical letterbox), and the larger col index of letterbox (bound for right vertical letterbox). The results are the median results on num_sample number of frames

Return type

(int, int, int, int)

kalmus.utils.artist.flatten_image(image)

Flat the input 2D image into an 1D image while preserve the channels of the input image with shape==[height x width, channels]

Parameters

image (numpy.ndarray) -- Input 2D image (either multi-channel color image or greyscale image)

Returns

The flatten 1D image. shape==(height x width, channels)

Return type

numpy.ndarry

kalmus.utils.artist.get_contrast_matrix_and_labeled_image(frame, minimum_segment_size=0.0004)

Helper function that use the watershed method to segment the input image return the matrix of the brightness contrast of each segmented region with respect to its neighbors (adjacent segmented regions)

Parameters

• frame (numpy.ndarray) -- The input frame

• minimum_segment_size (float) -- The minimum size of the segmented region in the ratio to the whole frame. Range (0, 1)

Returns

The matrix with shape (num_regions x num_regions) whose cell [i, j] represents the contrast between the region i and region j, and the corresponding labeled image (segmentation)

Return type

(list, numpy.ndarray)

kalmus.utils.artist.get_letter_box_from_frames(frame, threshold=5)

Find the position of letterbox (black bars around the scene) in the input cv2 video object. The function assumes the letter box of the frame is black (dark)

Parameters

• frame (numpy.ndarray) -- Input frame

• threshold (int) -- The brightness threshold value that distinguish the region of interest (bright) and letterbox (dark)

Returns

The smaller row index of letterbox (bound for upper horizontal letterbox), the larger row index of letterbox (bound for lower horizontal letterbox), the smaller col index of letterbox (bound for left vertical letterbox), and the larger col index of letterbox (bound for right vertical letterbox).

Return type

(int, int, int, int)

kalmus.utils.artist.get_rag(gray_image, labels)

Get the region adjacency graph using the labeled image and corresponding the edge map generated by greyscale image with sobel filter

Parameters

• gray_image (numpy.ndarray) -- The greyscale image corresponding to the labeled image

• labels (numpy.ndarray) -- a labeled segmented image, where the pixels in the image are labeled with index of the segmented regions.

Returns

The region adjacency graph in dictionary see https://scikit-image.org/docs/stable/auto_examples/segmentation/plot_rag_boundary.html for more references

Return type

skimage.future.graph.RAG

kalmus.utils.artist.grabcut_foreback_segmentation(image, start_row=0, start_col=0, row_size=- 1, col_size=- 1, num_iter=3, return_masks=False)

Perform the GrabCut segmentation over the input image with a rectangle of possible foreground specified by user. The GrabCut segment the image into two parts foreground and background, and the function return the 1D image of the foreground and background as the output

Parameters

• image (numpy.ndarray) -- The input image for GrabCut segmentation

• start_row (int) -- The starting row of the foreground rectangle of possible foreground

• start_col (int) -- The starting col of the foreground rectangle of possible foreground

• row_size (int) -- The vertical length of the rectangle

• col_size (int) -- The horizontal length of the rectangle

• num_iter (int) -- The number of iterations for GrabCut to run

• return_masks (bool) -- Return the foreground and background boolean mask if True Return the 1D image of foreground pixels and 1D image of background pixels if False False by default

Returns

If return_masks is False (default), 1D image of the foreground part of the image, and 1D image of the background part of the image Expected shape== Number of pixels x channels. If return_masks is True, return boolean masks foreground and background. Expected shape==image.shape

Return type

(numpy.ndarray, numpy.ndarray)

kalmus.utils.artist.rag_to_matrix(rag, num_regions)

Transfer the region adjacency dictionary to a more accessible region adjacency matrix where in the matrix, 1 means the region of corresponding column index is adjacent to the region of corresponding row index/ e.g. 0,1 1,0 means region 0 and region 1 are adjacent

Parameters

• rag (skimage.future.graph.RAG) -- region adjacency dictionary

• num_regions (int) -- number of regions in the region adjacency graph

Returns

An binary adjacency matrix with shape==num_regions x num_regions

Return type

numpy.ndarray

kalmus.utils.artist.random_sample_pixels(image, sample_ratio=0, mode='row-col')

Randomly sample an amount of pixels from an image based on the given sampled ratio. Two sampling mode are available. row-col: sampling first across the row and then across the column on each sampled row The output sampled image is still 2D and keep same aspect ratio as the input image, which can be used to visualize the sampling effect on the image. flat: sampling across the flat input image. The shape and aspect ratio of the input image are not preserved due to the flatten process, but it sample the pixels much faster than 'row-col' mode. The distribution of the sampling result is similar to that from the 'row-col'

Parameters

• image (numpy.ndarray) -- Input 2D image. Either a multi-channel color image or a single channel greyscale image Expected shape==height x width x (channels)

• sample_ratio (float) -- The amount of pixels sampled from the image. in range [0, 1]

• mode (str) -- two sampling mode are available. 1) 'row-col' sampling mode 2) 'flat' sampling mode

Returns

If mode="flat", return the resampled array of pixels (1-d flat array of data points) If mode="row-col", return the resampled image

Return type

numpy.ndarray

kalmus.utils.artist.watershed_segmentation(image, minimum_segment_size=0.0004, base_ratio=0.5, denoise_disk_size=5, gradiant_disk_size=5, marker_disk_size=15)

Label the connected regions in an image. Use edge detection approach with watershed algorithm. adjust the critical gradient (edge gradient) with the distribution of the input image's intensity gradient .

Parameters

• image (numpy.ndarray) -- The input color image for region segmentation using gradient-based watershed

• minimum_segment_size (float) -- The minimum size of the segments in the segmented image in percentage (size is the relative ratio of the image) The segmented regions smaller than the minimum size will be merged to its neighboring larger segmented components

• base_ratio (float) -- Amount of regions at least in the image. The image in watershed transformation is differentiated into two parts regions + boundaries. The base ratio is the ratio between the least amount of pixels that are regions and the total amount of pixels in the image.

• denoise_disk_size (int) -- the kernel size of the denoise median filter

• gradiant_disk_size (int) -- the kernel size of the gradient filter that is used for determining the amount of boundaries

• marker_disk_size (int) -- the kernel size of the gradient filter that is used for generating transformation marker

Returns

the segmented image, where shape==input_image.shape. and regions are labeled from 0 to n-1, where n is the number of regions in the labeled image. Functions also return the greyscale image corresponding to the original image

Return type

(numpy.ndarray, numpy.ndarray)