sort_and_drop_duplicates

HSTB.kluster.xarray_conversion.sort_and_drop_duplicates(dset, dsetpath)

Check for duplicates and sort if necessary. We’ve picked methods here to conserve memory, using just the included is_unique property is not as efficient as doing it in numpy. The isel and sortby statements will load the lazy loaded dataset into memory to do the reindexing, so we want to avoid those statements if at all possible.

Duplicates will cause the is_monotonic_increasing to be False, so check for those first.

Parameters

• dset (Dataset) – xarray dataset to sort/drop

• dsetpath (str) – path to the xarray dataset on disk

Returns

sorted and unique dataset

Return type

xr.Dataset