1. 27 Nov, 2020 1 commit
  2. 31 Mar, 2020 2 commits
  3. 30 Mar, 2020 1 commit
  4. 22 Jul, 2019 1 commit
  5. 20 Mar, 2018 1 commit
  6. 12 Feb, 2018 1 commit
  7. 09 Feb, 2018 2 commits
  8. 08 Feb, 2018 1 commit
  9. 17 Jan, 2018 1 commit
  10. 01 Dec, 2017 4 commits
  11. 18 Oct, 2017 1 commit
  12. 28 Sep, 2017 1 commit
  13. 26 Sep, 2017 1 commit
  14. 22 Sep, 2017 1 commit
    • Daniel Scheffler's avatar
      Refactored GEOPROCESSING, METADATA, Input_reader, Output_writer and... · e057d1e1
      Daniel Scheffler authored
      Refactored GEOPROCESSING, METADATA, Input_reader, Output_writer and SpatialIndexMediator. Deleted a lot of deprecated/unused code:
      GEOPROCESSING:
      - deleted many unused functions written by Robert Behling
      L1A_P:
      - deleted decompress()
      - deleted Layerstacking()
      L1B_P:
      - deleted functions related with SIFT/Orfeo toolbox co-registration
      L2A_P:
      - deleted get_DESHIFTER_configs()
      - deleted envifilehandling.py
      - deleted SRF_DB_2_PKL.py (content moved to output_writer
      gms_object:
      - deleted calc_mask_nodataOLD()
      METADATA:
      - deleted some unused functions written by Robert Behling
      - deleted sandbox module
      Former-commit-id: 36ab28d1
      Former-commit-id: 7d82a28a
      e057d1e1
  15. 15 Sep, 2017 1 commit
  16. 30 Aug, 2017 1 commit
  17. 09 Jun, 2017 1 commit
  18. 23 May, 2017 1 commit
  19. 05 Apr, 2017 1 commit
  20. 31 Mar, 2017 1 commit
  21. 01 Dec, 2016 1 commit
    • Daniel Scheffler's avatar
      Flink compatibility update / major revision of process controller · 5138279d
      Daniel Scheffler authored
      summary:
      - revised config -> database must be queried once per machine
      - all modules of the package are now directly importable
      - process controller is now a Python class containing a couple of methods for running the job
      all modules:
      - revised import statements
      - replaced functions 'CFG.get_job()' and 'CFG.get_usecase()' by property 'CFG.job' and 'CFG.usecase'
      misc.helper_functions:
      - moved MAP() to new module processing.multiproc
      misc.SpatialIndexMediator.SpatialIndexMediatorServer:
      - bugfix for returning 'port' instead of 'process_id'
      added new package 'processing'
      - added __init__
      - added module 'multiproc'
      - added new module 'process_controller' containing new class 'process_controller' with several methods for running the GMS job
      __init__:
      - revised
      config:
      - added set_config() replacing unified_config.set_config()
      - added class GMS_configuration containing properties for 'job' and 'usecase' that replace unified_config.get_job() and unified_config.get_usecase()
      - Job:
          - added attributes 'end_time' and 'computation_time'
          - revised class structure
      - Usecase:
          - revised class structure
      - removed deprecated code
      copied old version of config to config_old (still used by deprecated process_controller_for_testing)
      added run_gms.py for running GeoMultiSens process controller from console
      - updated __version__
      5138279d
  22. 13 Oct, 2016 1 commit
    • Daniel Scheffler's avatar
      revised logging practices of whole GMS package (fixes Permission errors during... · 11e1bc1c
      Daniel Scheffler authored
      revised logging practices of whole GMS package (fixes Permission errors during logging); added direct call of external DeShifter module
      L0B_P:
      - updated calls for logger closing
      L1A_P.L1A_object:
      - converted L1A_object.logger to property 'logger'
      - added close_GMS_loggers()
      - updated __getstate__ and __setstate__
      - log_for_fullArr_or_firstTile(): added 'close logger'-calls
      - MetaObj2ODict(): bugfix for not deleting MetaObj
      - added to_GMS_file() on the basis of OUT_W.ASCII_writer() which is now deprecated
      - delete_tempFiles(): added 'close logger'-calls
      L1B_P:
      - reordered imports
      - updated logger getters and 'close logger'-statements
      - L1B_object:
          - removed its own logger setup
          - correct_spatial_shifts(): added working version
      L2A_P:
      - DESHIFTER: updated logger getters and 'close logger'-statements
      INP_R:
      - pickle_SRF_DB(): updated logger getters and 'close logger'-statements
      OUT_W:
      - ASCII_writer(): now deprecated; added 'unclosed logfile' warnings
      - Obj2ENVI: updated writer calls for GMS file
      HLP_F:
      - removed setup_logger()
      - moved close_logger() to new module 'logging'
      - moved GMS_logger() to new module 'logging'
      - failed_GMS_object: updated logger getter
      - find_nearest() is now imported from external package 'py_tools_ds'
      - removed duplicate version of corner_coord_to_minmax()
      logging:
      - added new module logging, based on previous logging functions from HLP_F
      MGRS_tile:
      - added higher timeout for getting geometry from pgSQL database
      SpatialIndexMediator:
      - SpatialIndexMediatorServer:
          - status: revised return value
          - start(): revised warnings
          - stop(): revised return value
      PC:
      - job.logger is now an instance of GMS_logger
      - added new version of L2A_map()
      - added shutdown_loggers statement
      11e1bc1c
  23. 12 Oct, 2016 1 commit
    • Daniel Scheffler's avatar
      Bugfix for not properly closed logfiles; added environment checker for Spatial... · 77a105fc
      Daniel Scheffler authored
      Bugfix for not properly closed logfiles; added environment checker for Spatial Index Mediator Server
       L1A_P:
       - L1A_Obj.__getstate__(): bugfix for not properly closed logfiles
      L1B_P:
      - get_reference_image_params_pgSQL(): bugfix for not properly closed logfile
      L2A_P:
      - correct_shifts(): bugfix for not properly closed logfile
      ENV:
      - check_dependencies(): added functionality to check current status of Spatial Index Mediator Server and to start the server if needed
      - added _log_or_print()
      HLP_F:
      - added close_logger()
      - added draft of class GMS_logger (not usable so far)
      - subcall_with_output(): added keywords no_stdout and no_stderr
      SpatialIndexMediator:
      - added class SpatialIndexMediatorServer: a class for interacting with java server (status, start, stop, restart, is_running, port, ...)
      CFG:
      - added job.path_spatIdxSrv
      PC:
      - added calls of environment checkers
      pgSQL_DB:
      - added 'path_spatial_index_mediator_server' to config table
      77a105fc
  24. 07 Oct, 2016 1 commit
    • Daniel Scheffler's avatar
      - replaced the duplicated fork of COREG within L1B_P by an imports from external packages · be17e860
      Daniel Scheffler authored
      - GeoMultiSens now depends on CoReg_Sat and py_tools_ds!
      GEOP:
      - moved warp_ndarray to external package 'py_tools_ds'
      - updated warp_ndarray calls
      L1A_P:
      - added property coreg_needed
      - added cubic resampling for MGRS tiles that have to be reprojected to the next UTM zone
      - to_MGRS_tiles(): added verbose mode
      L1B_P:
      - COREG and DESHIFTER are now imported from external package CoReg_Sat
      - deleted deprecated draft of COREG_GMS
      - added class Scene_getter(): a class used to find a proper geospatial reference scene for a given target scene
      - added class ref_Scene
      - L1B_object:
          - revised the whole class (it now generates its coreg_info and deshift_results on its own)
          - added property spatRef_available
          - added get_spatial_reference_scene()
          - added get_opt_bands4matching (based on an earlier version from COREG)
          - added coregister_spatially()
          - added correct_spatial_shifts(): not yet working
      L1C_P:
      - L1C_object: updated __init__() args of super class L1B_object
      L2A_P:
      - updated warp_ndarray calls
      L2B_P:
      - spectral homogenization is now only executed if target CWLs are different to source CWLs
      INP_R:
      - fixed an unclosed file within GMSfile2dict
      OUT_W:
      - fixed an unclosed file within Obj2ENVI
      PC:
      - added a new version of L1B_map_1()
      be17e860
  25. 23 Sep, 2016 1 commit
  26. 19 Sep, 2016 1 commit
    • Daniel Scheffler's avatar
      Bugfix for "missing L2B mask files" during continued processing that earlier... · afcb0fa5
      Daniel Scheffler authored
      Bugfix for "missing L2B mask files" during continued processing that earlier stopped in L2B processing; implemented proper MGRS tiling scheme according to project agreements
      GEOP:
      - updated some deprecated 3rd-party-library references
      - added transform_any_prj(): a function fort transforming XY-coordinates from any source to any target projection
      - warp_ndarray():
          - implemented extra option 'outExtent_within' allowing to output arrays with a larger geographical extent than the input image (needed for MGRS tiling scheme)
      - improved some docstrings
      - added prj_equal(): a function to quickly check if two projections are equal
      - added snap_bounds_to_pixGrid(): a function to snap map bounds to a given pixel grid
      - added clip_array_using_mapBounds(): a function for clipping arrays with a map info using a given bounding box
      L0A_P:
      - add_local_availability():
          - replaced a hard coded list of processing levels
          - bugfix for returning an invalid processing level if processing is already done
      L1A_P:
      - fill_from_disk(): bugfix for copying memory addresses  when fill_from_disk() is called in multiprocessing
      - log_for_fullArr_or_firstTile(): bugfix for not handling empty arr_pos
      - added get_subset_obj(), based on older code from helper_functions: a function to generate subsets of GMS objects based on bounding box that can contain image AND map coordinates whereas map coordinates can also have a different projection (needed for proper generation of MGRS tiles)
       - added to_MGRS_tiles(, based on older code from helper_functions: a function for cutting a GMS object into MGRS tiles
       L1B_P:
       - L1B_object.apply_deshift_results():
           - bugfix for not updating geoinformations of 'masks' attribute after having applied spatial shift corrections
      META:
      - added 'data ignore value' to metadata
      IO:
      - refactored module 'gms_io' to 'io'
      ARR:
      - added submodule 'GeoArray' to io:
          - added class 'GeoArray': a class for simplifying array access, regardless to their actual memory location (in memory or on disk)
          - added _clip_array_at_mapPos(): a function for clipping a geocoded array using a given bounding box in the same projection like the array itself
          - added  get_array_at_mapPos(): a function for clipping a geocoded array using a given bounding box that can have any projection
      OUT_W:
      - added 'data ignore value' to enviHdr_keyOrder
      - added 'data ignore value' to mask_to_ENVI_Classification()
      - added set_output_nodataVal(): a function for adding a data ignore value to an already written file
      - Obj2ENVI: bugfix for not handling empty arr_pos
      DB_T:
      - delete_processing_results(), GMS_JOB.__delete_procdata(), GMS_JOB.delete_procdata_of_entire_job(), GMS_JOB.delete_procdata_of_failed_sceneIDs(), added functionality to delete a specific processing level from disk
      HLP_F:
      - revised find_nearest(): it now supports an automatic rounding algorithm that rounds a value to the nearest neighbour
      - replaced a deprecated 3rd-party function reference
      - get_arrSubsetBounds_from_shapelyPolyLonLat(): added flag 'ensure_valid_coords'
      MRGS_tile:
      - added submodule "mgrs_tile" for easily retrieving informations about MGRS tiles (bounding box, projection conversions, buffering, etc.)
      SpatialIndexMediator:
      - replaced wrong host address
      PC:
      - updated calls for MGRS tiling within L2C_map_1()
      afcb0fa5
  27. 01 Sep, 2016 1 commit
    • Daniel Scheffler's avatar
      added handling of unexpected exceptions within all GMS mappers; added... · 38dad1ca
      Daniel Scheffler authored
      added handling of unexpected exceptions within all GMS mappers;  added creation of job success summary
      L1A_P:
      - removed deprecated attributes path_Outfile_L1A and path_Outfile_L1A_masks
      - calc_cloud_mask(): bugfix - wrong arguments passed to log_for_fullArr_or_firstTile
      L1B_P:
      - revised get_reference_image_params()
           -> spatial query works now but it still does not handle the case that no reference scene could be found or reference scene has a different projection
      L1C_P:
      - removed deprecated import
      INP_R:
      - get_list_GMSfiles(): bugfix for ignoring the given target processing level if job.call_type=='webapp'
      OUT_W:
      - Obj2ENVI(): bugfix for not assigning the correct output path to attribute 'mask_1bit' in case job.exec_mode=='Python'
       DB_T:
       - moved SpatialIndexMediator to a separate module within misc
       - simplified datetime reference
       - GMS_JOB: added comment keyword in order to allow short descriptions and comments for the job
      HLP_F:
      - added trace_unhandled_exceptions(): a dummy decorator function for handling unexpected exceptions
      - added class 'failed_GMS_object': a class to represent failed GMS objects (allows to get information from them - even after processing failed)
      - added log_uncaught_exceptions(): a decorator function for logging unhandled exceptions within GMS mapper functions
      - added get_job_summary(): a function that returns a detailed and a less detailed pandas dataframe summarizing job processing success
      - cut_GMS_obj_into_MGRS_tiles(): added logging
      - get_mask_classdefinition(): added mask_1bit class definition
      SpatialIndexMediator (SIM):
      - added new module SpatialIndexMediator for performing fast spatial queries targeting the 'scenes' table of postgreSQL database
      PC:
      - added decorators to all GMS mapper functions that enable handling of unexpected exceptions
      - added new list "failed_objects" in order to collect information about all GMS objects where processing failed
      - added job database entry updates regarding failed scenes and finish time
      - added creation and logging of job summary
      VC:
      - added cloud classifier objects for Landsat-5/7/8 to version control
      38dad1ca