U fb7f@slddlZddlZddlZddlmZddlZddlm Z ddl m Z ddl m Z GdddZd d d ZdS)N)tqdm)iou) DriverError)CPLE_OpenFailedErrorc@seZdZdZddZddZddZd d dZdd dddgfddZdd dddgfddZ dgd ddfddZ d!ddZ d"ddZ dS)# EvaluatoraObject to test IoU for predictions and ground truth polygons. Attributes ---------- ground_truth_fname : str The filename for the ground truth CSV or JSON. ground_truth_GDF : :class:`geopandas.GeoDataFrame` A :class:`geopandas.GeoDataFrame` containing the ground truth vector labels. ground_truth_GDF_Edit : :class:`geopandas.GeoDataFrame` A copy of ``ground_truth_GDF`` which will be manipulated during processing. proposal_GDF : :class:`geopandas.GeoDataFrame` The proposal :class:`geopandas.GeoDataFrame`, added using ``load_proposal()``. Arguments --------- ground_truth_vector_file : str Path to .geojson file for ground truth. cCsz@|dr||n|dr8|j|dd||_Wn tk r`||_d|_YnX|jj|_|jjdd|_ t g|_ dS)NjsoncsvT)truthCSVzGeoDataFrame variabledeep) lowerendswith load_truthground_truth_fnameAttributeErrorground_truth_GDFsindexground_truth_sindexcopyground_truth_GDF_Editgpd GeoDataFrame proposal_GDF)selfground_truth_vector_filerg/home/ec2-user/SageMaker/vegetation-management-remars2022/remars2022-workshop/libs/solaris/eval/base.py__init__#s    zEvaluator.__init__cCsdtj|jdS)Nz Evaluator {})formatospathsplitr)rrrr__repr__3s  zEvaluator.__repr__cCs|jjdd}|S)zReturns a copy of the ground truth table, which includes a per-building IoU score column after eval_iou_spacenet_csv() has run. Tr )rr)rZoutput_ground_truth_GDFrrrget_iou_by_building7szEvaluator.get_iou_by_building? iou_scoreImageIdFrc Csg}|t|j||jjs<|t|j|tt|}|}g}d|j|<|jj|} d} |jj ddd| f} t |D]} |j|j|| kj dd|_ |j |j j |k|_ |j|j|| kj dd} | | j |k} |rt|| D]\}}|rt|j|jj dkr|j}t||j }|js|djddd}|j|}|jj || f}||}||kr||jj || f<|d|kr|d|jj|j|f<|j j|jdd |_ nd|jj|j|f<nd|jj|j|f<nd|jj|j|f<|rt|jj|jq|jjr8d}d}n|j|j|| kj dd} | | j |k} | js|| jkr| | ||kjd}| | ||kjd}ntd |d}d}ntd d}d}|j |j j dkjd}t||dkr|t||}nd}t||dkr0|t||}nd}||dkrXd||||}nd}| |||||||d }||q|S) aWEvaluate IoU between the ground truth and proposals in CSVs. Arguments --------- miniou : float , optional Minimum intersection over union score to qualify as a successful object detection event. Defaults to ``0.5``. iou_field_prefix : str , optional The name of the IoU score column in ``self.proposal_GDF``. Defaults to ``"iou_score"`` . imageIDField : str , optional The name of the column corresponding to the image IDs in the ground truth data. Defaults to ``"ImageId"``. debug : bool , optional Argument for verbose execution during debugging. Defaults to ``False`` (silent execution). min_area : float or int , optional Minimum area of a ground truth polygon to be considered during evaluation. Often set to ``20`` in SpaceNet competitions. Defaults to ``0`` (consider all ground truth polygons). Returns ------- scoring_dict_list : list list of score output dicts for each image in the ground truth and evaluated image datasets. The dicts contain the following keys: :: ('imageID', 'iou_field', 'TruePos', 'FalsePos', 'FalseNeg', 'Precision', 'Recall', 'F1Score') gNTr rr'axisskipnar+iou field {} missingzEmpty Proposal Id)imageID iou_fieldTruePosFalsePosFalseNeg PrecisionRecallF1Score)extendlistruniqueremptysetcolumnsget_locilocrrrareaprintiterrowsnamegeometryr calculate_iouidxmaxlocdropshaper floatappend)rminiouiou_field_prefixZ imageIDFielddebugmin_areaZ imageIDListr0scoring_dict_listZ iou_indexid_colsZground_truth_idsr/Zproposal_GDF_copy_pred_row pred_polyiou_GDF max_index max_iou_rowZ previous_iouZnew_iour1r2r3r4r5r6 score_calcrrreval_iou_spacenet_csv?s#               zEvaluator.eval_iou_spacenet_csvTallc Csg}|r\|std|dgkr\t|j|}|jjsP|t|jdtt|}|D]$}d||}|dkr|j|j||kj dd|_ n|jj dd|_ t |j ddD]\} } | d|ks|dkr| j } t| |j } | jsZ| j| djd dd } | d|krF| d|jj| j|f<|j j| jd d |_ nd |jj| j|f<qd |jj| j|f<q|jjrd }d }n`z8|j|j||kjd }|j|j||kjd }Wn&tk rtd d }d }YnX|j jd }t||d kr|t||}nd }t||d kr:|t||}nd }||d krbd ||||}nd }||||||||d}||q`|S)aEvaluate IoU between the ground truth and proposals. Arguments --------- miniou : float, optional Minimum intersection over union score to qualify as a successful object detection event. Defaults to ``0.5``. iou_field_prefix : str, optional The name of the IoU score column in ``self.proposal_GDF``. Defaults to ``"iou_score"``. ground_truth_class_field : str, optional The column in ``self.ground_truth_GDF`` that indicates the class of each polygon. Required if using ``calculate_class_scores``. calculate_class_scores : bool, optional Should class-by-class scores be calculated? Defaults to ``True``. class_list : list, optional List of classes to be scored. Defaults to ``['all']`` (score all classes). Returns ------- scoring_dict_list : list list of score output dicts for each image in the ground truth and evaluated image datasets. The dicts contain the following keys: :: ('class_id', 'iou_field', 'TruePos', 'FalsePos', 'FalseNeg', 'Precision', 'Recall', 'F1Score') FMust provide ground_truth_class_field if using calculate_class_scores.rZ__max_conf_class{}_{}Tr )disabler'rr*r-r.r)class_idr0r1r2r3r4r5r6 ValueErrorr8rr9rr:r7r;r rrrrArCrrDrFrErBrGrHKeyErrorr@rIrJ)rrKrLground_truth_class_fieldcalculate_class_scores class_listrOr`r0rQrRrSrTrVr1r2r3r4r5r6rWrrreval_ious"              zEvaluator.eval_iouc Csg}|r\|std|dgkr\t|j|}|jjsP|t|jdtt|}|D]^}d||}|dkr|j|j||kj dd|_ n|jj dd|_ t |j D]\} } | d|ks|dkr| j } t| |j } | jsV| j| djddd } | d|krB| d|jj| j|f<|j j| jdd |_ nd|jj| j|f<qd|jj| j|f<q|jjr~d}d}nz\|j|j||k}|jd}|dkrd }|j|j||k}|jd}|dkrd }Wn.tk rtd d}d}d }d }YnX|j }|jd}|dkr(d }t||dkrL|t||}nd}t||dkrt|t||}nd}||dkrd ||||}nd}||||||||d}||q`||||fS)aEvaluate IoU between the ground truth and proposals. Arguments --------- miniou : float, optional Minimum intersection over union score to qualify as a successful object detection event. Defaults to ``0.5``. iou_field_prefix : str, optional The name of the IoU score column in ``self.proposal_GDF``. Defaults to ``"iou_score"``. ground_truth_class_field : str, optional The column in ``self.ground_truth_GDF`` that indicates the class of each polygon. Required if using ``calculate_class_scores``. calculate_class_scores : bool, optional Should class-by-class scores be calculated? Defaults to ``True``. class_list : list, optional List of classes to be scored. Defaults to ``['all']`` (score all classes). Returns ------- scoring_dict_list : list list of score output dicts for each image in the ground truth and evaluated image datasets. The dicts contain the following keys: :: ('class_id', 'iou_field', 'TruePos', 'FalsePos', 'FalseNeg', 'Precision', 'Recall', 'F1Score') True_Pos_gdf : gdf A geodataframe containing only true positive predictions False_Neg_gdf : gdf A geodataframe containing only false negative predictions False_Pos_gdf : gdf A geodataframe containing only false positive predictions r[rZr\r]Tr r'rr*r-Nr.r)r_ra)rrKrLrdrerfrOr`r0rQrRrSrTrVr1r2Z True_Pos_gdfZ False_Pos_gdfZ False_Neg_gdfr3r4r5r6rWrrreval_iou_return_GDFsLs$                  zEvaluator.eval_iou_return_GDFsconfPolygonWKT_PixNc stj|r|r@t|}tj|fdd|Dd|_nsz+Evaluator.load_proposal..rCrr-Z __total_confr\g?Ncsg|] }|qSrr)roitem)conf_field_mappingrrrrsF)by ascending)r!r"isfilepdread_csvrrrAr read_filedropnarrmaxrEvalues sort_values)rZproposal_vector_fileZconf_field_listZ proposalCSVrqruZ pred_datar)rurqr load_proposalsJ&         zEvaluator.load_proposalc s|r2t|}tj|fdd|Dd|_n>zt||_Wn,ttfk rntgggd|_YnX|jj |_ |jj dd|_ dS)aLoad in the ground truth geometry data. Arguments --------- ground_truth_vector_file : str Path to the ground truth vector file. Must be either .geojson or .csv format. truthCSV : bool, optional Is the ground truth a CSV? Defaults to ``False``, in which case it's assumed to be a .geojson. truth_geo_value : str, optional Column of the ground truth vector file that corresponds to geometry. Returns ------- Nothing. Notes ----- Loads the ground truth vector data into the ``Evaluator`` instance. cs g|]\}}tj|qSrrk)rorpZ truth_rowtruth_geo_valuerrrr5sz(Evaluator.load_truth..rs)r conditionrCTr N) ryrzrrrArr{rrrrrr)rrr rZ truth_datarrrrs"    zEvaluator.load_truthrcCsdS)Nr)rtyperrrevalDszEvaluator.eval)r&r'r(Fr)Frj)r) __name__ __module__ __qualname____doc__rr$r%rXrgrhrrrrrrrr s<  z  K +rFrjcCst|S)z]Deprecated API to Evaluator. .. deprecated:: 0.3 Use :class:`Evaluator` instead.)r)rZcsvFilerrrr eval_baseHsr)Frj) shapely.wktrl geopandasrpandasryrr!rYr fiona.errorsr fiona._errrrrrrrrs    A