Visualizations

The vis module of pybx can be used to visualize these “stacks”

Loading from disk

Load a sample image from the data directory ../data/image.jpg and resize to (200, 200, 3)

im, ann, lgt, clr = get_example(image_sz=(200, 200, 3), pth='../data', img_fn='image.jpg')

im is the image of a clock and a frame, ann are the annotations for the image in json format, lgt are logits (activations from a layer) which can be displayed on top of the image, clr is a dict representing the colors to use for the different annotation keys.

ann, lgt, clr
([{'x_min': 100.0,
   'y_min': 46.666666666666664,
   'x_max': 180.0,
   'y_max': 146.66666666666666,
   'label': 'clock'},
  {'x_min': 6.666666666666667,
   'y_min': 120.0,
   'x_max': 76.66666666666667,
   'y_max': 173.33333333333334,
   'label': 'frame'}],
 None,
 {'frame': 'blue', 'clock': 'green'})
_ = plt.imshow(im)

Store the annotations as a MultiBx object.

get_bx(ann)
MultiBx(coords=[[100.0, 46.666666666666664, 180.0, 146.66666666666666], [6.666666666666667, 120.0, 76.66666666666667, 173.33333333333334]], label=['clock', 'frame'])

source

draw

 draw (img:numpy.ndarray, bbox:list, logits=None, alpha=0.4, **kwargs)

Method to draw an image, box and logits overlayed if passed. :param img: the image array, expects a numpy array :param bbox: list of bounding boxes in json format :param logits: activations that should be overlayed from a neural network (no checks) :param kwargs: kwargs for draw_boxes() :param alpha: same as alpha for matplotlib :return: current axis


source

draw_boxes

 draw_boxes (img:numpy.ndarray, bbox:list, title=None, ax=None,
             figsize=(5, 4), color='yellow', no_ticks=False, xo=0, yo=0,
             squeeze=False, **kwargs)

Method to draw bounding boxes in an image, can handle multiple bboxes :param figsize: sige of figure :param img: the image array, expects a numpy array :param bbox: list of bounding boxes in json format :param title: image title :param ax: which axis if already present :param yo: y offset for placement of text :param xo: x offset for placement of text :param color: text color or dict of colors for each label as a dict :param no_ticks: whether to set axis ticks off :param squeeze: squeeze axis :return: ax with image


source

draw_rectangle

 draw_rectangle (ax, coords, color='white')

Draw a rectangle using matplotlib patch. :param ax: axis :param coords: coordinates in coco format :param color: text color :return: ax object


source

draw_text

 draw_text (ax, xy:tuple, label:str, size=12, color='white', xo=0, yo=0)

Write text around boxes. :param ax: axis object :param xy: relative ax coordinates x, y to draw the text :param label: label for box :param size: font size :param yo: y offset for placement of text :param xo: x offset for placement of text :param color: text color :return: ax object


source

get_extents

 get_extents (shape)

Get extent parameter of the image.


source

get_color

 get_color (color, label=None, default_color='white')

Get colors from color dict for a given label. If label=None, return default_color. :param color: dict of key, value pairs where key is label, value is color :param label: the label for which color is needed :param default_color: :return: str that contains color


source

draw_outline

 draw_outline (obj, linewidth:int)

Make outlines around to object edges for visibility in light backgrounds :param obj: plt objects like text or rectangle :param linewidth: width of the stroke :return: plt object

ann
[{'x_min': 100.0,
  'y_min': 46.666666666666664,
  'x_max': 180.0,
  'y_max': 146.66666666666666,
  'label': 'clock'},
 {'x_min': 6.666666666666667,
  'y_min': 120.0,
  'x_max': 76.66666666666667,
  'y_max': 173.33333333333334,
  'label': 'frame'}]

Drawing random box by passing coordinates as a list.

draw(im, [[20, 20, 80, 80]])
<AxesSubplot:>

Drawing random box by passing coordinates as a list along with label.

draw(im, [[20, 20, 80, 80, 'random']], color={'random': 'blue'})
<AxesSubplot:>

draw(im, [{'x_min': 50, 'y_min': 30, 'x_max':100, 'y_max':120, 'label':'random'}], \
    color={'random': 'green'})
<AxesSubplot:>

To display the calculated anchor boxes or any other type of bounding boxes, pybx offers the vis.VisBx class. First, vis.VisBx initializes all the params for the image and loads its annotations (if available). Upon calling the show() method on the instantiated VisBx object with our predicted annotations or anchor boxes, everything gets displayed.


source

VisBx

 VisBx (image_arr=None, image_sz=None, sample=False, **kwargs)

VisBx is used to visualize the bounding boxes. The image on of which the bounding boxes are to be drawn can be instantiated with VisBx() if needed. Calling the show() method of the VisBx() instance accepts bounding box coordinates and labels that are to be shown. The boxes can be provided as any of the internal objects (MultiBx, BaseBx, …) or as any other raw format accepted by the internal objects.

Displaying image array and annotations object: This is the default approach used by VisBx(). If no arguments are passed, a tuple denoting the size for random noise random_img_sz=(100, 100, 1) is expected. Some arguments: :param image_arr: image array of shape (H, W, C). If None, it is set to a random noise image of image_sz=(100,100,3) by default. :param annots: annotations is any accepted format (see above).

Displaying from image and annotations file: To load and display the image, set sample=True. Some argmuments: :param ann_fn: annotations file name, default annots.json :param img_fn: image file name, default image.jpg :param load_ann: whether to load ann_fn or just the img_fn. If False, an empty annotations dict is returned: dict(zip(voc_keys, [0, 0, 1, 1, ''])) :param pth: path to find ann_fn and img_fn, default . :param image_sz: size to resize the loaded image a different size (annotations scaled automatically)

Common parameters: :param color: A dict of color can be passed to assign specific color to a specific label in the image: color = {'frame': 'blue', 'clock': 'green'} :param logits: Logits as ndarray that should be overlayed on top of the image or bool to generate random logits. :param feature_sz: Feature size to generate random logits if logits is not None.

Display image array (default behaviour)

To display an image array (shape of W, H, C) directly, instead of loading from disk, pass an ndarray to the image_arr argument. If available, also provide the annots in any supported format. Displaying calculated anchors with a random image:

anchor_boxes, anchor_labels = bx(200, 3, 1)
v = VisBx(image_arr=np.random.random((200, 200, 3)))  # a 200x200 image is generated
v.show(anchor_boxes, anchor_labels)
<AxesSubplot:>

v = VisBx(image_arr=np.random.random((10, 10, 3)), image_sz=(200,200, 3))  # a 10x10 will be resized to 200x200
v.show(anchor_boxes, anchor_labels)
<AxesSubplot:>

Load image and show annots

Displaying calculated anchors with a sample image.

v = VisBx(pth='../data/', img_fn='image.jpg', image_sz=(200,200, 3))
v.show(anchor_boxes, anchor_labels)
<AxesSubplot:>

Customize colors

To customize the box colors, pass a dict color with the annotation name as key and color as value. In the example below an anchor box color is changed to red using its label a_3x3_1.0_3 along with the annotations provided read from file.

v.show(anchor_boxes, anchor_labels, color={'a_3x3_1.0_3': 'red', 'frame': 'green', 'clock': 'green'})
<AxesSubplot:>