B NK€`¸÷ã~@sÂdZddlmZddlmZddlmZddlZddlZddlZe d¡ddl m Z ddl Z ddlmZddlmZddlmZddlmZddlZddlmZddlmZddlZd Zd Zd d d d ddddddddddddddddddd d!d"d#d$d%d&d'd(d)d*d+d,d-d.d/d0d1d2d3d4d5d6d7d8d9d:d;dd?d@dAdBdCdDdEdFdGdHdIdJdKdLdMdNdOdPdQdRdSdTdUdVdWdXdYdZd[d\d]d^d_d`dadbdcdddedfdgdhdidjdkdldmdndodpdqdrdsdtdudvdwdxdydzd{d|d}d~dd€dd‚dƒd„d…d†d‡g~Zdˆd‰„ZdŠd‹„Z dŒd„Z!dÉd’d“„Z"dÊd”d•„Z#dËd–d—„Z$dÌd˜d™„Z%dÍd›dœ„Z&ddž„Z'dŸd „Z(dÎd¡d¢„Z)d£d¤„Z*dÏd§d¨„Z+dÐd©dª„Z,dÑd¬d­„Z-dÒd±d²„Z.dÓd³d´„Z/dÔd¶d·„Z0dÕd¸d¹„Z1dÖd¼d½„Z2d×d¿dÀ„Z3dÁd„Z4dÃdÄ„Z5GdÅdÆ„dÆe 6ej7e8¡ƒZ9GdÇdÈ„dÈe9ƒZ:dS)ØzÐA set of functions that are used for visualization. These functions often receive an image, perform some visualization on the image. The functions do not return a value, instead they modify the image itself. é)Úabsolute_import)Údivision)Úprint_functionNÚAgg)Úrange)Úzipé Ú AliceBlueÚ ChartreuseÚAquaÚ AquamarineÚAzureÚBeigeÚBisqueÚBlanchedAlmondÚ BlueVioletÚ BurlyWoodÚ CadetBlueÚ AntiqueWhiteÚ ChocolateÚCoralÚCornflowerBlueÚCornsilkÚCrimsonÚCyanÚDarkCyanÚ DarkGoldenRodÚDarkGreyÚ DarkKhakiÚ DarkOrangeÚ DarkOrchidÚ DarkSalmonÚ DarkSeaGreenÚ DarkTurquoiseÚ DarkVioletÚDeepPinkÚ DeepSkyBlueÚ DodgerBlueÚ FireBrickÚ FloralWhiteÚ ForestGreenÚFuchsiaÚ GainsboroÚ GhostWhiteÚGoldÚ GoldenRodÚSalmonÚTanÚHoneyDewÚHotPinkÚ IndianRedÚIvoryÚKhakiÚLavenderÚ LavenderBlushÚ LawnGreenÚ LemonChiffonÚ LightBlueÚ LightCoralÚ LightCyanÚLightGoldenRodYellowÚ LightGrayÚ LightGreyÚ LightGreenÚ LightPinkÚ LightSalmonÚ LightSeaGreenÚ LightSkyBlueÚLightSlateGrayÚLightSlateGreyÚLightSteelBlueÚ LightYellowÚLimeÚ LimeGreenÚLinenÚMagentaÚMediumAquaMarineÚ MediumOrchidÚ MediumPurpleÚMediumSeaGreenÚMediumSlateBlueÚMediumSpringGreenÚMediumTurquoiseÚMediumVioletRedÚ MintCreamÚ MistyRoseÚMoccasinÚ NavajoWhiteÚOldLaceÚOliveÚ OliveDrabÚOrangeÚ OrangeRedÚOrchidÚ PaleGoldenRodÚ PaleGreenÚ PaleTurquoiseÚ PaleVioletRedÚ PapayaWhipÚ PeachPuffÚPeruÚPinkÚPlumÚ PowderBlueÚPurpleÚRedÚ RosyBrownÚ RoyalBlueÚ SaddleBrownÚGreenÚ SandyBrownÚSeaGreenÚSeaShellÚSiennaÚSilverÚSkyBlueÚ SlateBlueÚ SlateGrayÚ SlateGreyÚSnowÚ SpringGreenÚ SteelBlueÚ GreenYellowÚTealÚThistleÚTomatoÚ TurquoiseÚVioletÚWheatÚWhiteÚ WhiteSmokeÚYellowÚ YellowGreencsrttƒ‰dddddg}‡fdd„|Dƒ}|s0dS‡fd d„|Dƒ}t|ƒ}d d„tt|t|ƒƒƒDƒ}||d S) açReturns a multiplier to get semi-random colors from successive indices. This function computes a prime number, p, in the range [2, 17] that: - is closest to len(STANDARD_COLORS) / 10 - does not divide len(STANDARD_COLORS) If no prime numbers in that range satisfy the constraints, p is returned as 1. Once p is established, it can be used as a multiplier to select non-consecutive colors from STANDARD_COLORS: colors = [(p * i) % len(STANDARD_COLORS) for i in range(20)] ééé é écsg|]}ˆ|r|‘qS©rŒ)Ú.0Úp)Ú num_colorsrŒúV/root/amazon-sagemaker-tensorflow-serving-grpc/object-detection/visualization_utils.pyú Usz8_get_multiplier_for_color_randomness..écsg|]}t ˆd|¡‘qS)g$@)ÚnpÚabs)rrŽ)rrŒrr‘ZscSsg|] \}}|‘qSrŒrŒ)rÚ_ÚirŒrŒrr‘\sr)ÚlenÚSTANDARD_COLORSÚsortedrr)Zprime_candidatesZ abs_distanceZnum_candidatesÚindsrŒ)rrÚ$_get_multiplier_for_color_randomnessGs r›c Cs@t t |¡¡ d¡}tj |d¡}| |d¡WdQRXdS)z°Saves an image (represented as a numpy array) to PNG. Args: image: a numpy array with shape [height, width, 3]. output_path: path to which image should be written. ÚRGBÚwÚPNGN) ÚImageÚ fromarrayr“Úuint8ÚconvertÚtfÚgfileÚOpenÚsave)ÚimageÚ output_pathÚ image_pilÚfidrŒrŒrÚsave_image_array_as_png`sr«cCs:t t |¡¡}t ¡}|j|dd| ¡}| ¡|S)z”Encodes a numpy array into a PNG string. Args: image: a numpy array with shape [height, width, 3]. Returns: PNG encoded image string. rž)Úformat) rŸr r“r¡ÚsixÚBytesIOr¦ÚgetvalueÚclose)r§r©ÚoutputZ png_stringrŒrŒrÚencode_image_array_as_png_strks r²ÚredérŒTc CsDt t |¡¡ d¡} t| ||||||||ƒ t |t | ¡¡dS)aAdds a bounding box to an image (numpy array). Bounding box coordinates can be specified in either absolute (pixel) or normalized coordinates by setting the use_normalized_coordinates argument. Args: image: a numpy array with shape [height, width, 3]. ymin: ymin of bounding box. xmin: xmin of bounding box. ymax: ymax of bounding box. xmax: xmax of bounding box. color: color to draw bounding box. Default is red. thickness: line thickness. Default value is 4. display_str_list: list of strings to display in box (each to be shown on its own line). use_normalized_coordinates: If True (default), treat coordinates ymin, xmin, ymax, xmax as relative to the image. Otherwise treat coordinates as absolute. rœN)rŸr r“r¡r¢Údraw_bounding_box_on_imageÚcopytoÚarray) r§ÚyminÚxminÚymaxÚxmaxÚcolorÚ thicknessÚdisplay_str_listÚuse_normalized_coordinatesr©rŒrŒrÚ draw_bounding_box_on_image_arrayzs rÀc s~t |¡} |j\} } |r>|| || || || f\} } }}n||||f\} } }}|dkrˆ| j| |f| |f| |f| |f| |fg||dyt dd¡‰Wntk r´t ¡‰YnX‡fdd„|Dƒ}dt|ƒ}||krâ|}n||}xŽ|ddd …D]|}ˆ  |¡\}}t   d |¡}| j | ||d |f| ||fg|d | j | ||||f|d ˆd||d |8}qúWdS)a÷Adds a bounding box to an image. Bounding box coordinates can be specified in either absolute (pixel) or normalized coordinates by setting the use_normalized_coordinates argument. Each string in display_str_list is displayed on a separate line above the bounding box in black text on a rectangle filled with the input 'color'. If the top of the bounding box extends to the edge of the image, the strings are displayed below the bounding box. Args: image: a PIL.Image object. ymin: ymin of bounding box. xmin: xmin of bounding box. ymax: ymax of bounding box. xmax: xmax of bounding box. color: color to draw bounding box. Default is red. thickness: line thickness. Default value is 4. display_str_list: list of strings to display in box (each to be shown on its own line). use_normalized_coordinates: If True (default), treat coordinates ymin, xmin, ymax, xmax as relative to the image. Otherwise treat coordinates as absolute. r)ÚwidthÚfillz arial.ttfécsg|]}ˆ |¡d‘qS)r’)Úgetsize)rÚds)ÚfontrŒrr‘Ísz.draw_bounding_box_on_image..gš™™™™™ñ?Néÿÿÿÿgš™™™™™©?é)rÂÚblack)rÂrÆ)Ú ImageDrawÚDrawÚsizeÚlineÚ ImageFontÚtruetypeÚIOErrorZ load_defaultÚsumrÄr“ÚceilÚ rectangleÚtext)r§r¸r¹rºr»r¼r½r¾r¿ÚdrawÚim_widthÚ im_heightÚleftÚrightÚtopÚbottomZdisplay_str_heightsZtotal_display_str_heightZ text_bottomÚ display_strÚ text_widthZ text_heightÚmarginrŒ)rÆrrµ›sB    rµcCs0t |¡}t|||||ƒt |t |¡¡dS)a¶Draws bounding boxes on image (numpy array). Args: image: a numpy array object. boxes: a 2 dimensional numpy array of [N, 4]: (ymin, xmin, ymax, xmax). The coordinates are in normalized format between [0, 1]. color: color to draw bounding box. Default is red. thickness: line thickness. Default value is 4. display_str_list_list: list of list of strings. a list of strings for each bounding box. The reason to pass a list of strings for a bounding box is that it might contain multiple labels. Raises: ValueError: if boxes is not a [N, 4] array N)rŸr Údraw_bounding_boxes_on_imager“r¶r·)r§Úboxesr¼r½Údisplay_str_list_listr©rŒrŒrÚ"draw_bounding_boxes_on_image_arrayås  râc CsŽ|j}|sdSt|ƒdks&|ddkr.tdƒ‚xZt|dƒD]J}d}|rP||}t|||df||df||df||df|||ƒq.visualization_py_func_fnrŒ)ræròrñrðrórôrõrŒ)rærðrñròrórôrÚcreate_visualization_fn s1.röcCszt |¡}|jd}x`t|ƒD]T}|dd…dd…|fd}| d¡}t |d¡}| d¡|jdg|t |dqWdS) abDraws heatmaps on an image. The heatmaps are handled channel by channel and different colors are used to paint different heatmap channels. Args: image: a PIL.Image object. heatmaps: a numpy array with shape [image_height, image_width, channel]. Note that the image_height and image_width should match the size of input image. rÈNéÿr¡ÚLÚ1)rr)ÚxyÚbitmaprÂ) rÊrËrärÚastyperŸr r¢rûr˜)r§ÚheatmapsrÕÚchannelÚcZheatmaprûrŒrŒrÚdraw_heatmaps_on_image‚s     rcCsRt|tjƒs| ¡}t|tjƒs(| ¡}t t |¡¡ d¡}t||ƒt  |¡S)a÷Overlays heatmaps to an image (numpy array). The function overlays the heatmaps on top of image. The heatmap values will be painted with different colors depending on the channels. Similar to "draw_heatmaps_on_image_array" function except the inputs are numpy arrays. Args: image: a numpy array with shape [height, width, 3]. heatmaps: a numpy array with shape [height, width, channel]. Returns: An uint8 numpy array representing the input image painted with heatmap colors. rœ) Ú isinstancer“ÚndarrayÚnumpyrŸr r¡r¢rr·)r§rýr©rŒrŒrÚdraw_heatmaps_on_image_array™s   rc Cs¨|jddkr0|dd…dd…dd…dd…f}n|jddkrJtj |¡}t |¡\}}}}|rltj |¡}tjj|||gd}||g}dd„}tj ||tj dd }|S) aDraws heatmaps on batch of image tensors. Args: images: A 4D uint8 image tensor of shape [N, H, W, C]. If C > 3, additional channels will be ignored. If C = 1, then we convert the images to RGB images. heatmaps: [N, h, w, channel] float32 tensor of heatmaps. Note that the heatmaps will be resized to match the input image size before overlaying the heatmaps with input images. Theoretically the heatmap height width should have the same aspect ratio as the input image to avoid potential misalignment introduced by the image resize. apply_sigmoid: Whether to apply a sigmoid layer on top of the heatmaps. If the heatmaps come directly from the prediction logits, then we should apply the sigmoid layer to make sure the values are in between [0.0, 1.0]. Returns: 4D image tensor of type uint8, with heatmaps overlaid on top. rãNrr’)rÌcSst t|tj¡}|S)zDraws heatmaps on image.)r£Ú py_functionrr¡)Zimage_and_heatmapsZimage_with_heatmapsrŒrŒrÚ draw_heatmapsÎs z5draw_heatmaps_on_image_tensors..draw_heatmapsF)ÚdtypeÚ back_prop) rär£r§Úgrayscale_to_rgbÚ shape_utilsZ!combined_static_and_dynamic_shapeÚmathÚsigmoidÚresizeÚmap_fnr¡) ÚimagesrýZ apply_sigmoidr•ÚheightrÁZresized_heatmapsÚelemsrrŒrŒrÚdraw_heatmaps_on_image_tensors®s"  rcCs<t |d¡}tjj||tjjjdd}t t |d¡tj¡S)NrT)ÚmethodÚ align_corners) r£Ú expand_dimsr§Ú resize_imagesÚ ResizeMethodÚNEAREST_NEIGHBORÚcastÚsqueezer¡)r§Ú image_shaperŒrŒrÚ_resize_original_imageÙs réçš™™™™™É?cs`|jddkr0|dd…dd…dd…dd…f}n|jddkrJtj |¡}|| | dd| dœ}ˆdkr‚tjd|j ¡ddgd }nˆ}ˆdkr¬tjd|j ¡dd gd }nˆ}t|f|dk |dk | dk | dk d œ|—Ž‰||||||g}|dk rü| |¡|dk r| |¡| dk r$| | ¡| dk r8| | ¡‡‡‡fd d „}tj||tj dd}|S)aŒDraws bounding boxes, masks, and keypoints on batch of image tensors. Args: images: A 4D uint8 image tensor of shape [N, H, W, C]. If C > 3, additional channels will be ignored. If C = 1, then we convert the images to RGB images. boxes: [N, max_detections, 4] float32 tensor of detection boxes. classes: [N, max_detections] int tensor of detection classes. Note that classes are 1-indexed. scores: [N, max_detections] float32 tensor of detection scores. category_index: a dict that maps integer ids to category dicts. e.g. {1: {1: 'dog'}, 2: {2: 'cat'}, ...} original_image_spatial_shape: [N, 2] tensor containing the spatial size of the original image. true_image_shape: [N, 3] tensor containing the spatial size of unpadded original_image. instance_masks: A 4D uint8 tensor of shape [N, max_detection, H, W] with instance masks. keypoints: A 4D float32 tensor of shape [N, max_detection, num_keypoints, 2] with keypoints. keypoint_scores: A 3D float32 tensor of shape [N, max_detection, num_keypoints] with keypoint scores. keypoint_edges: A list of tuples with keypoint indices that specify which keypoints should be connected by an edge, e.g. [(0, 1), (2, 4)] draws edges from keypoint 0 to 1 and from keypoint 2 to 4. track_ids: [N, max_detections] int32 tensor of unique tracks ids (i.e. instance ids for each object). If provided, the color-coding of boxes is dictated by these ids, and not classes. max_boxes_to_draw: Maximum number of boxes to draw on an image. Default 20. min_score_thresh: Minimum score threshold for visualization. Default 0.2. use_normalized_coordinates: Whether to assume boxes and kepoints are in normalized coordinates (as opposed to absolute coordiantes). Default is True. Returns: 4D image tensor of type uint8, with boxes drawn on top. rãNrr’Fr´)r¿Úmax_boxes_to_drawÚmin_score_threshÚ agnostic_modeÚline_thicknessÚkeypoint_edgesrÇ)rärÈ)ròrñrðrócsh|d}|d}ˆdk r6t |d|d|ddg¡}ˆdk rLt||ƒ|d<t ˆ|dd…tj¡}|S)zDraws boxes on image.rr’NrÈrã)r Úpad_or_clip_ndrr£Úpy_funcr¡)Zimage_and_detectionsÚ true_shapeÚoriginal_shaper§Zimage_with_boxes)Úoriginal_image_spatial_shapeÚtrue_image_shapeÚvisualize_boxes_fnrŒrÚ draw_boxes=s z8draw_bounding_boxes_on_image_tensors..draw_boxes)rr) rär£r§r ÚconstantÚas_liströÚappendrr¡)rràrírîrær(r)rçrèrér#rêrr r¿Zvisualization_keyword_argsZ true_shapesZoriginal_shapesrr+rŒ)r(r)r*rÚ$draw_bounding_boxes_on_image_tensorsãsF3"         r/cCsæt ¡}t ¡}g}t||jjƒdkr^x4|D],} | |jkr.| |jkr.t  || d¡|| <q.Wx€t ||jjdƒD]f} d} |j |kr¬t  tj ||j | ddtj ¡} d} d} |j|krtj ||j| dd} |j|krütj ||j| dd} ntj t | ¡tjd} d}|j|krDt  tj ||j| ddtj ¡}d}d}|j}|j|krºtj ||j| dd}||kr¤tj tj ||| tjddd}ntj t |¡tjd}ttj ||j| ddtj ||j| ddtj ||j| ddtj ||j| dd|tj ||j| ddtj ||j| dd| | | ||||d}ttj ||j| ddtj ||j| ddtj ||j| ddtj tj||j| tjddd|tj ||j| ddtj ||j| dd||||dd|d}tj||gdd}|j|krÔttj ||j| ddtj ||j| ddtj ||j| ddtj tj||j| tjddd|tj ||j| ddtj ||j| dd|dddd|d }tj||gdd}| |¡qvW|S) aMCreates a side-by-side image with detections and groundtruth. Bounding boxes (and instance masks, if available) are visualized on both subimages. Args: eval_dict: The evaluation dictionary returned by eval_util.result_dict_for_batched_example() or eval_util.result_dict_for_single_example(). category_index: A category index (dictionary) produced from a labelmap. max_boxes_to_draw: The maximum number of boxes to draw for detections. min_score_thresh: The minimum score threshold for showing detections. use_normalized_coordinates: Whether to assume boxes and keypoints are in normalized coordinates (as opposed to absolute coordinates). Default is True. keypoint_edges: A list of tuples with keypoint indices that specify which keypoints should be connected by an edge, e.g. [(0, 1), (2, 4)] draws edges from keypoint 0 to 1 and from keypoint 2 to 4. Returns: A list of [1, H, 2 * W, C] uint8 tensor. The subimage on the left corresponds to detections, while the subimage on the right corresponds to groundtruth. r’rN)Úaxis)r) r(r)rçrèrér#rr r¿grÈ)r(r)rçrèr#rr r¿) ÚfieldsÚDetectionResultFieldsÚInputDataFieldsr—Údetection_classesräÚoriginal_imageZimage_additional_channelsr£rrÚdetection_masksrr¡Zdetection_keypointsZdetection_keypoint_scoresZ keypoint_opsZset_keypoint_visibilitiesÚfloat32Úgroundtruth_instance_masksZ!groundtruth_keypoint_visibilitiesÚgroundtruth_keypointsr/Údetection_boxesÚdetection_scoresr(r)Zgroundtruth_boxesZgroundtruth_classesÚ ones_likeÚconcatr.)Ú eval_dictrærr r¿r#Údetection_fieldsÚinput_data_fieldsÚimages_with_detections_listÚkeyÚindxrçrèrér8r9Zgroundtruth_keypoint_scoresZgt_kpt_vis_fldZimages_with_detectionsZimages_with_groundtruthZimages_to_visualizeZ+images_with_additional_channels_groundtruthrŒrŒrÚ"draw_side_by_side_evaluation_imageOsú                   rDrÃc s6ˆdkrtd ˆ¡ƒ‚t ¡}t ¡}|j|kr8tdƒ‚|j|krJtdƒ‚g}xât||jj dƒD]Ê}||j |} ||j |} ||j|} t   | | d| ddg¡} t| | ƒ} ||j|} ||j|} ||j|}‡‡‡‡fdd „}t || | || gtj¡}| |tjd d …d d …d d …f¡qdW|S) a‹Draws DensePose visualizations. Args: eval_dict: The evaluation dictionary returned by eval_util.result_dict_for_batched_example(). max_boxes_to_draw: The maximum number of boxes to draw for detections. min_score_thresh: The minimum score threshold for showing detections. num_parts: The number of different densepose parts. dp_coord_to_visualize: Whether to visualize v-coordinates (0) or u-coordinates (0) overlaid on the person masks. Returns: A list of [1, H, W, C] uint8 tensor, each element corresponding to an image in the batch. Raises: ValueError: If `dp_coord_to_visualize` is not 0 or 1. )rr’zg`dp_coord_to_visualize` must be either 0 for v coordinates), or 1 for u coordinates, but instead got {}z*Expected `detection_masks` in `eval_dict`.z3Expected `detection_surface_coords` in `eval_dict`.rr’rãc s~t |¡}x`tt|||ƒƒD]L\}\}}}|ˆkr4P|ˆkrt||ˆdt||dd…dd…ˆf|ƒqWtj||gddS)z:Overlays part masks and surface coords on original images.)Ú num_partsNr’)r0)r“ÚcopyÚ enumeraterÚdraw_part_mask_on_image_arrayÚ!draw_float_channel_on_image_arrayÚ concatenate) r§r6Úsurface_coordsrîZsurface_coord_imager–ÚscoreZ surface_coordÚmask)Údp_coord_to_visualizerr rErŒrÚdraw_densepose_py_func%s  z=draw_densepose_visualizations..draw_densepose_py_funcN)rår¬r1r2r3r6Zdetection_surface_coordsrr5rär)r(r r$rr;r£r%r¡r.Únewaxis)r>rr rErNr?r@rArCr&r'r§rîr6rKrOZimage_with_denseposerŒ)rNrr rErÚdraw_densepose_visualizationsôs8     $rQçà?rÈÚgreenc CsHt t |¡¡ d¡} t| ||||||||| d t |t | ¡¡dS)aŽDraws keypoints on an image (numpy array). Args: image: a numpy array with shape [height, width, 3]. keypoints: a numpy array with shape [num_keypoints, 2]. keypoint_scores: a numpy array with shape [num_keypoints]. If provided, only those keypoints with a score above score_threshold will be visualized. min_score_thresh: A scalar indicating the minimum keypoint score required for a keypoint to be visualized. Note that keypoint_scores must be provided for this threshold to take effect. color: color to draw the keypoints with. Default is red. radius: keypoint radius. Default value is 2. use_normalized_coordinates: if True (default), treat keypoint values as relative to the image. Otherwise treat them as absolute. keypoint_edges: A list of tuples with keypoint indices that specify which keypoints should be connected by an edge, e.g. [(0, 1), (2, 4)] draws edges from keypoint 0 to 1 and from keypoint 2 to 4. keypoint_edge_color: color to draw the keypoint edges with. Default is red. keypoint_edge_width: width of the edges drawn between keypoints. Default value is 2. rœ)rér r¼Úradiusr¿r#Úkeypoint_edge_colorÚkeypoint_edge_widthN)rŸr r“r¡r¢Údraw_keypoints_on_imager¶r·) r§rèrér r¼rTr¿r#rUrVr©rŒrŒrÚdraw_keypoints_on_image_array<srXc  sÀt |¡} |j\‰‰t |¡}dd„|Dƒ} dd„|Dƒ} |rjt‡fdd„| Dƒƒ} t‡fdd„| Dƒƒ} |dk rŠt |¡}t ||¡} nBt tjt  |¡ddt  |dd…d f¡t  |dd…d f¡¡} d d„| Dƒ} xJt | | | ƒD]:\}}}|rè| j ||||f||||fg||d qèW|dk r¼xŠ|D]‚\}}|d ks6|t|ƒks6|d ks6|t|ƒkrrq6| |r6| |sŠq6| || || || |g}| j||| d q6WdS) a²Draws keypoints on an image. Args: image: a PIL.Image object. keypoints: a numpy array with shape [num_keypoints, 2]. keypoint_scores: a numpy array with shape [num_keypoints]. min_score_thresh: a score threshold for visualizing keypoints. Only used if keypoint_scores is provided. color: color to draw the keypoints with. Default is red. radius: keypoint radius. Default value is 2. use_normalized_coordinates: if True (default), treat keypoint values as relative to the image. Otherwise treat them as absolute. keypoint_edges: A list of tuples with keypoint indices that specify which keypoints should be connected by an edge, e.g. [(0, 1), (2, 4)] draws edges from keypoint 0 to 1 and from keypoint 2 to 4. keypoint_edge_color: color to draw the keypoint edges with. Default is red. keypoint_edge_width: width of the edges drawn between keypoints. Default value is 2. cSsg|] }|d‘qS)r’rŒ)rÚkrŒrŒrr‘‡sz+draw_keypoints_on_image..cSsg|] }|d‘qS)rrŒ)rrYrŒrŒrr‘ˆscsg|] }ˆ|‘qSrŒrŒ)rÚx)rÖrŒrr‘Šscsg|] }ˆ|‘qSrŒrŒ)rÚy)r×rŒrr‘‹sNr’)r0rcSsg|]}|‘qSrŒrŒ)rÚvrŒrŒrr‘“s)ÚoutlinerÂ)rÂrÁ)rÊrËrÌr“r·ÚtupleÚgreaterÚwhereÚanyÚisnanÚ zeros_liker<rZellipser—rÍ)r§rèrér r¼rTr¿r#rUrVrÕZ keypoints_xZ keypoints_yZ valid_kptZ keypoint_xZ keypoint_yÚvalidZkeypoint_startZ keypoint_endZedge_coordinatesrŒ)r×rÖrrWhs>      rWçš™™™™™Ù?c Csø|jtjkrtdƒ‚|jtjkr(tdƒ‚|jdd…|jkrXtd|jdd…|jfƒ‚t |¡}t |¡}tj t  |¡ddt  t |ƒdddg¡}t t |¡¡  d ¡}t t d ||d k¡¡  d ¡}t |||¡}t |t |  d ¡¡¡dS)aDraws mask on an image. Args: image: uint8 numpy array with shape (img_height, img_height, 3) mask: a uint8 numpy array of shape (img_height, img_height) with values between either 0 or 1. color: color to draw the keypoints with. Default is red. alpha: transparency value between 0 and 1. (default: 0.4) Raises: ValueError: On incorrect data type for image or masks. z`image` not of type np.uint8z`mask` not of type np.uint8NrÈzBThe image has spatial dimensions %s but the mask has dimensions %s)r0r’rãÚRGBAgào@rrørœ)rr“r¡råräÚ ImageColorÚgetrgbrŸr rr<ÚreshapeÚlistr¢Ú compositer¶r·) r§rMr¼ÚalphaÚrgbÚ pil_imageZ solid_colorZpil_solid_colorÚpil_maskrŒrŒrÚdraw_mask_on_image_array©s    &"rpc Cs@|jtjkrtdƒ‚|jtjkr(tdƒ‚|jdd…|jkrXtd|jdd…|jfƒ‚t |¡}t |¡}|dd…dd…tjf}xXt t d|…ƒD]D\}}tj t   |¡tjd} |||dk| tjtjdd…f7}q–Wt t |¡¡ d¡} t t d ||d k¡¡ d ¡} t | || ¡}t |t  | d ¡¡¡dS) aÅDraws part mask on an image. Args: image: uint8 numpy array with shape (img_height, img_height, 3) mask: a uint8 numpy array of shape (img_height, img_height) with 1-indexed parts (0 for background). alpha: transparency value between 0 and 1 (default: 0.4) num_parts: the maximum number of parts that may exist in the image (default 24 for DensePose). Raises: ValueError: On incorrect data type for image or masks. z`image` not of type np.uint8z`mask` not of type np.uint8NrÈzBThe image has spatial dimensions %s but the mask has dimensions %s)rr’rfgào@rrørœ)rr“r¡rårärŸr rcrPrGr˜r·rgrhr¢rkr¶) r§rMrlrErnZ part_colorsZmask_1_channelr–r¼rmZpil_part_colorsrorŒrŒrrHÆs"    *"rHçÍÌÌÌÌÌì?ÚYlGnc Cs4|jtjkrtdƒ‚|jtjkr(tdƒ‚|jtjkr.cdf_plotN)r£r%r¡Úsummaryr§)r”rxr™rŒrŒrÚadd_cdf_image_summary¸sr›cCs.dd„}t |||gtj¡}tj ||¡dS)a'Adds a tf.summary.image for a histogram plot of the values. Plots the histogram of values and creates a tf image summary. Args: values: a 1-D float32 tensor containing the values. bins: bin edges which will be directly passed to np.histogram. name: name for the image summary. c Ssštjdd}| d¡}tj||d\}}| |dd…|¡| d¡| d¡|j  ¡|  ¡|  ¡\}}tj |j  ¡d d  d t|ƒt|ƒd ¡}|S) zNumpy function to plot hist.F)r„r…)ÚbinsNrÇÚcountÚvaluer¡)rr’rã)rsr‰rŠr“Ú histogramr‹rŒrrŽrÕrrr‘r’rir“) r”rœr—r˜r[rZrÁrr§rŒrŒrÚ hist_plotÞs     z)add_hist_image_summary..hist_plotN)r£r%r¡ršr§)r”rœrxr rŒrŒrÚadd_hist_image_summaryÕs r¡c@s@eZdZdZddd „Zd d „Zd d „Zdd„Zej dd„ƒZ dS)ÚEvalMetricOpsVisualizationaÅAbstract base class responsible for visualizations during evaluation. Currently, summary images are not run during evaluation. One way to produce evaluation images in Tensorboard is to provide tf.summary.image strings as `value_ops` in tf.estimator.EstimatorSpec's `eval_metric_ops`. This class is responsible for accruing images (with overlaid detections and groundtruth) and returning a dictionary that can be passed to `eval_metric_ops`. r‡rçš™™™™™É?TÚevaluation_imageNcCs4||_||_||_||_||_||_||_g|_dS)a7Creates an EvalMetricOpsVisualization. Args: category_index: A category index (dictionary) produced from a labelmap. max_examples_to_draw: The maximum number of example summaries to produce. max_boxes_to_draw: The maximum number of boxes to draw for detections. min_score_thresh: The minimum score threshold for showing detections. use_normalized_coordinates: Whether to assume boxes and keypoints are in normalized coordinates (as opposed to absolute coordinates). Default is True. summary_name_prefix: A string prefix for each image summary. keypoint_edges: A list of tuples with keypoint indices that specify which keypoints should be connected by an edge, e.g. [(0, 1), (2, 4)] draws edges from keypoint 0 to 1 and from keypoint 2 to 4. N)Ú_category_indexÚ_max_examples_to_drawÚ_max_boxes_to_drawÚ_min_score_threshÚ_use_normalized_coordinatesÚ_summary_name_prefixÚ_keypoint_edgesÚ_images)ÚselfræÚmax_examples_to_drawrr r¿Úsummary_name_prefixr#rŒrŒrÚ__init__ùsz#EvalMetricOpsVisualization.__init__cCs g|_dS)N)r¬)r­rŒrŒrÚclearsz EvalMetricOpsVisualization.clearcCsDt|jƒ|jkrdS|j |¡t|jƒ|jkr@g|j|jd…<dS)z5Store a list of images, each with shape [1, H, W, C].N)r—r¬r¦r~)r­rrŒrŒrÚ add_imagess  z%EvalMetricOpsVisualization.add_imagesc sĈjdkriSˆ |¡}‡fdd„}dd„}t ¡rNˆ |dgg¡}|ƒ}n0t ˆj|dggg¡}t |gtjgˆj¡}i}x.get_imagesc s0t t t t ˆ¡¡d¡‡‡fdd„dd„¡S)z0Returns image summaries for non-padded elements.r´cstj ˆˆ¡S)N)r£ršr§rŒ)r§Ú summary_namerŒrÚeózsEvalMetricOpsVisualization.get_estimator_eval_metric_ops..image_summary_or_default_string..cSs t d¡S)Nrw)r£r,rŒrŒrŒrrµfr¶)r£ÚcondÚequalrÌrä)r´r§rŒ)r§r´rÚimage_summary_or_default_stringas zaEvalMetricOpsVisualization.get_estimator_eval_metric_ops..image_summary_or_default_stringú/) r¦Úimages_from_evaluation_dictr£Úexecuting_eagerlyr²r%r¡rGrªr|) r­r>rr³r¹Ú update_opZ image_tensorsZeval_metric_opsr–r§r´Zvalue_oprŒ)r­rÚget_estimator_eval_metric_ops&s"/    z8EvalMetricOpsVisualization.get_estimator_eval_metric_opscCst‚dS)aConverts evaluation dictionary into a list of image tensors. To be overridden by implementations. Args: eval_dict: A dictionary with all the necessary information for producing visualizations. Returns: A list of [1, H, W, C] uint8 tensors. N)ÚNotImplementedError)r­r>rŒrŒrr»vs z6EvalMetricOpsVisualization.images_from_evaluation_dict)r‡rr£Tr¤N) Ú__name__Ú __module__Ú __qualname__Ú__doc__r°r±r²r¾ÚabcÚabstractmethodr»rŒrŒrŒrr¢ðs  Pr¢cs*eZdZdZd ‡fdd „ Zd d „Z‡ZS) ÚVisualizeSingleFrameDetectionszCClass responsible for single-frame object detection visualizations.r‡rçš™™™™™É?TÚ!Detections_Left_Groundtruth_RightNc s"tt|ƒj|||||||ddS)N)rær®rr r¿r¯r#)ÚsuperrÆr°)r­rær®rr r¿r¯r#)Ú __class__rŒrr°†s z'VisualizeSingleFrameDetections.__init__cCst||j|j|j|j|jƒS)N)rDr¥r§r¨r©r«)r­r>rŒrŒrr»—s z:VisualizeSingleFrameDetections.images_from_evaluation_dict)r‡rrÇTrÈN)rÀrÁrÂrÃr°r»Ú __classcell__rŒrŒ)rÊrrƃs rÆ)r³r´rŒT)r³r´rŒT)r³r´rŒ)r³r´rŒ)FFFF)F) NNNNNNNrrT)rrTN)rrrÃr)NrRr³rÈTNrSrÈ)NrRr³rÈTNrSrÈ)r³re)rerÃ)rqrr)NNNNNNFrrRFr´rÉFFFF);rÃÚ __future__rrrrÄrzÚ matplotlibÚuseÚmatplotlib.pyplotÚpyplotrsrr“Z PIL.ImagerŸZPIL.ImageColorrgZ PIL.ImageDrawrÊZ PIL.ImageFontrÎr­Ú six.movesrrÚ tensorflowr£Z_TITLE_LEFT_MARGINZ_TITLE_TOP_MARGINr˜r›r«r²rÀrµrârßrörrrrr/rDrQrXrWrprHrIrër›r¡Úwith_metaclassÚABCMetaÚobjectr¢rÆrŒrŒrŒrÚs                                D   ^ ) ` " F % 8   +