U fb(*@sdZddlZddlZddlZddlZddlmZddl Z ddl Z ddl Z ddZ ddZ dd Zdd d Zd dZdddZdS)zThe preprocessing module is used to reshape data into format suitable for training or prediction. For example cutting large tiles into smaller images. N)ImagecCs*tj||ddddd}|d}|S)zYPreprocess a single RGB numpy array as a prediction from channels last, to channels first)devicer)torchtensorpermute unsqueeze)imagerr k/home/ec2-user/SageMaker/vegetation-management-remars2022/remars2022-workshop/libs/deepforest/preprocess.pypreprocess_imagesrcCs tj|}tj|d}|S)z/Convert path to image name for use in indexing.r)ospathbasenamesplitext) image_path image_namer r r image_name_from_paths rcCs.|dkrtd|t|tjj||}|S)zCreate a sliding window object from a raster tile. Args: numpy_image (array): Raster object as numpy array to cut into crops Returns: windows (list): a sliding windows object rz&Patch overlap {} must be between 0 - 1) ValueErrorformat slidingwindowgenerateZDimOrderZHeightWidthChannel) numpy_image patch_size patch_overlapwindowsr r r compute_windows!s rFc Cs||\}}}}||}||} d} ||j|| k|j|k@|j|k@|j|| k@|j|| k@|j| k@|j|k@|j| | k@} tjd|j dd} d| || _ | j r|rt j d| |gdgd} d| d<d| d<d| d <d| d <|j d| d <nd Sn| j|| j| j| _| j|| _| j|| j| j| _| j|| _d| j| jdkdf<|| j| j|kd f<d| j| jdkdf<|| j| j|kd f<| S) aOSelect annotations that overlap with selected image crop. Args: image_name (str): Name of the image in the annotations file to lookup. annotations_file: path to annotations file in the format -> image_path, xmin, ymin, xmax, ymax, label windows: A sliding window object (see compute_windows) index: The index in the windows object to use a crop bounds allow_empty (bool): If True, allow window crops that have no annotations to be included Returns: selected_annotations: a pandas dataframe of annotations (z{}rz {}_{}.pngrcolumnsxminyminxmaxymaxlabelN)getRectr"r$r#r%rrrrruniqueemptypd DataFramer&loc) annotationsrindex allow_emptyZ window_xminZ window_yminwhZ window_xmaxZ window_ymaxoffsetZselected_annotationsimage_basenamer r r select_annotations6st       r4cCsLtj|st|t|}tj|d}d|||}|||S)zoSave window crop as image file to be read by PIL. Filename should match the image_name + window index rz {}/{}_{}.png) rrexistsmakedirsr fromarrayrrsave)base_dirrr.cropimr3filenamer r r save_crop~s    r=.皙?cs4|dk|dk@rtd|r:t|}t|dd}n|dkrJtd|jd}|dkstd |jz$|ddddddf d}Wntd ||YnX|jd} |jd } t t | | g|krt d || | gt|||} |dkrtj|}t|j|k} | jr@t d ||tfd ddDslt d jd g} t| D]Z\}}|| |}|jdkrqxt| | ||}|dk rx| |t||||qxt| dkrt d |t| } tj|d}|d}tj ||}| j!|ddd| S)aDivide a large tile into smaller arrays. Each crop will be saved to file. Args: numpy_image: a numpy object to be used as a raster, usually opened from rasterio.open.read() path_to_raster: (str): Path to a tile that can be read by rasterio on disk annotations_file (str): Path to annotations file (with column names) data in the format -> image_path, xmin, ymin, xmax, ymax, label base_dir (str): Where to save the annotations and image crops relative to current working dir patch_size (int): Maximum dimensions of square window patch_overlap (float): Percent of overlap among windows 0->1 allow_empty: If True, include images with no annotations to be included in the dataset image_name (str): If numpy_image arg is used, what name to give the raster? Returns: A pandas dataframe with annotations file for training. Nzlsupply a raster either as a path_to_raster or if ready from existing in memory numpy object, as numpy_image=rrzjIf passing an numpy_image, please also specify a image_name to match the column in the annotation.csv filezAInput rasterio had non-3 band shape of {}, ignoring alpha channeluint8a Input file {} has {} bands. DeepForest only accepts 3 band RGB rasters in the order (height, width, channels). Selecting the first three bands failed, please reshape manually.If the image was cropped and saved as a .jpg, please ensure that no alpha channel was used.rz7Patch size of {} is larger than the image dimensions {}zNo image names match between the file:{} and the image_path: {}. Reminder that image paths should be the relative path (e.g. 'image_name.tif'), not the full path (e.g. path/to/dir/image_name.tif)csg|]}|jkqSr r ).0xr-r r sz split_raster..)rr"r#r$r%r&z]Annotations file has {} columns, should have format image_path, xmin, ymin, xmax, ymax, labelz?Input file has no overlapping annotations and allow_empty is {}z.csvFT)r.header)"IOErrorrasterioopenreadnpmoveaxisshapewarningswarnrastypeanyarrayrrrrrr*read_csvrr)all enumerateindicessizer4appendr=lenconcatrjointo_csv)Zannotations_fileZpath_to_rasterrr9rrr/rbandsheightwidthrimage_annotationsZannotations_filesr.windowr:Zcrop_annotationsr3 file_pathr rEr split_rasters $             rd)F)NNr>r?r@FN)__doc__rnumpyrLpandasr*rPILrrrOrIrrrr4r=rdr r r r s*  H