1. 25 Sep, 2017 1 commit
  2. 22 Sep, 2017 2 commits
    • Daniel Scheffler's avatar
      Refactored GEOPROCESSING, METADATA, Input_reader, Output_writer and... · e057d1e1
      Daniel Scheffler authored
      Refactored GEOPROCESSING, METADATA, Input_reader, Output_writer and SpatialIndexMediator. Deleted a lot of deprecated/unused code:
      GEOPROCESSING:
      - deleted many unused functions written by Robert Behling
      L1A_P:
      - deleted decompress()
      - deleted Layerstacking()
      L1B_P:
      - deleted functions related with SIFT/Orfeo toolbox co-registration
      L2A_P:
      - deleted get_DESHIFTER_configs()
      - deleted envifilehandling.py
      - deleted SRF_DB_2_PKL.py (content moved to output_writer
      gms_object:
      - deleted calc_mask_nodataOLD()
      METADATA:
      - deleted some unused functions written by Robert Behling
      - deleted sandbox module
      Former-commit-id: 36ab28d1
      Former-commit-id: 7d82a28a
      e057d1e1
    • Daniel Scheffler's avatar
      PEP-8 editing. Added style checkers. · 461c4953
      Daniel Scheffler authored
      Former-commit-id: b6a2f1b6
      Former-commit-id: f867b21f
      461c4953
  3. 21 Sep, 2017 1 commit
  4. 15 Sep, 2017 1 commit
  5. 07 Sep, 2017 1 commit
  6. 04 Sep, 2017 1 commit
  7. 31 Aug, 2017 1 commit
  8. 30 Aug, 2017 1 commit
  9. 28 Jul, 2017 1 commit
    • Daniel Scheffler's avatar
      - Revised config.Job -> now features an own configuration for test mode.... · 50bf3772
      Daniel Scheffler authored
      - Revised config.Job -> now features an own configuration for test mode. Passing arguments from outside is now much easier.
      - Refactored some attributes of config.Job.
      - added cloud classifiers for the included test data
      - misc.exceptions: added GMSConfigParameterError
      - misc.path_generator: revised get_path_cloud_class_obj(): merged subfolders for cloud classifiers on disk
      - processing.pipeline: refactored 'exec__...' to 'exec_...'
      - tests.test_geomultisens: removed superfluous paths configs
      - removed cloud_classifiers from .gitignore
      Former-commit-id: 86ded16f
      Former-commit-id: 021aa530
      50bf3772
  10. 27 Jul, 2017 1 commit
  11. 22 Jul, 2017 1 commit
  12. 21 Jul, 2017 1 commit
  13. 19 Jul, 2017 1 commit
  14. 17 Jul, 2017 1 commit
  15. 14 Jul, 2017 1 commit
  16. 30 Jun, 2017 2 commits
  17. 27 Jun, 2017 1 commit
  18. 26 Jun, 2017 1 commit
  19. 09 Jun, 2017 1 commit
  20. 30 May, 2017 2 commits
  21. 23 May, 2017 2 commits
  22. 05 Apr, 2017 1 commit
  23. 30 Mar, 2017 1 commit
    • Daniel Scheffler's avatar
      fixed an issue during calculation of true data corners during L2A, Fix for... · f5eea438
      Daniel Scheffler authored
      fixed an issue during calculation of true data corners during L2A, Fix for deadlock during DEM calculation at 180 degrees meridian
      algorithms.L1A_P:
      - calc_corner_positions(): bugfix
      algorithms.L1C_P.AtmCorr:
      - _join_results_to_inObjs(): splitted into four separate functions
      - added _join_data_ac(): based on _join_results_to_inObjs(); AC nodata values within dataset are no longer overwritten with fill values but with outFill
      - added _join_data_errors(): based on _join_results_to_inObjs()
      - added _join_mask_clouds(): based on _join_results_to_inObjs()
      - added _join_mask_confidence_array(): based on _join_results_to_inObjs()
      io.Input_reader:
      - get_dem_by_extent(): fixed an issue in case of a scene directly at 180 degrees meridian
      misc.definition_dicts:
      - get_outFillZeroSaturated(): changed outZero value of uint16
      - updated __version__
      Former-commit-id: ab3aa1a5
      Former-commit-id: 487c9e2e
      f5eea438
  24. 27 Mar, 2017 1 commit
    • Daniel Scheffler's avatar
      AC is now skipped if an exception is raised there. Implemented job status... · 4146c388
      Daniel Scheffler authored
      AC is now skipped if an exception is raised there. Implemented job status updates into process controller.
      algorithms.L1C.AtmCorr:
      - run_atmospheric_correction(): AC is now skipped if an exception is raised there. NOTE: the output will be TOA reflectance instead of BOA reflectance in that case
      processing.process_controller:
      - run_all_processors():
           - added error handling for unexpected exceptions within process controller
           - added job.status updates that are also written into database in order to allow status visualization during a running job
      - stop(): added job.status updater
      - update_DB_job_record(): now 'status' column of jobs database table is also updated
      config.Job:
      - added attribute 'status'
      - updated __version__
      Former-commit-id: f71f4d01
      Former-commit-id: abe9963b
      4146c388
  25. 23 Feb, 2017 1 commit
    • Daniel Scheffler's avatar
      Bugfix for issues writing MGRS tiles in case of scenes that have an overlap to... · ba54ca0f
      Daniel Scheffler authored
      Bugfix for issues writing MGRS tiles in case of scenes that have an overlap to neighbour UTM zone; added feature for job creation from scene IDs
      algorithms.gms_object:
      - get_subset_obj(): implemented keyword out_prj
      - to_MGRS_tiles(): fix for not respecting output projection
      misc.database_tools.GMS_JOB:
      - from_dictlist(): moved parts to _get_validated_sceneInfoGDFs() and _populate_jobAttrs_from_sceneInfoGDF()
      - added from_sceneIDlist()
      - added _get_validated_sceneInfoGDFs()
      - added _populate_jobAttrs_from_sceneInfoGDF()
      misc.path_generator.path_generator:
      - added get_pathes_all_procdata(): not fully implemented yet
      processing.multiproc:
      - MAP: bugfix for ignoring CFG.job.CPUs in case CPUs is given
      - updated __version__
      Former-commit-id: 9b5925c8
      Former-commit-id: 12c9f4fe
      ba54ca0f
  26. 15 Feb, 2017 1 commit
    • Daniel Scheffler's avatar
      Bugfix within AC · 770aa9f0
      Daniel Scheffler authored
      algorithms.L1C_P.AtmCorr:
      - options: bugfix for not excluding specific bands from AC output
      - _join_results_to_inObjs():
          - bugfix for stacking data_ac bands in the wrong order
          - NaN values withon data_ac are now set to outFill value
      - updated __version__
      Former-commit-id: 577c72b9
      Former-commit-id: 1460d9b2
      770aa9f0
  27. 09 Feb, 2017 1 commit
  28. 08 Feb, 2017 1 commit
    • Daniel Scheffler's avatar
      Added support for atmospheric correction of Landsat-7 and Landsat-8. · 5f91570c
      Daniel Scheffler authored
      algorithms.gms_object.GMS_object:
      - arr.setter: catched an error during setting of arr.bandnames
      - revised ac_options
      algorithms.L1C_P:
      - L1C_object:
          - revised logger.deleter
          - added property 'options'
          - added _get_mask_clouds()
          - revised run_atmospheric_correction()
          - revised _join_results_to_inObjs()
      database:
      - removed folder ac_options -> AC options are now directly imported fom S2SCAPEM (thus version controlled)
      misc.path_generator:
      - revised get_path_ac_options() -> AC options are now directly imported fom S2SCAPEM (thus version controlled)
      - updated __version__
      Former-commit-id: a75ee7c3
      Former-commit-id: 0657813b
      5f91570c
  29. 27 Jan, 2017 1 commit
  30. 26 Jan, 2017 2 commits
    • Daniel Scheffler's avatar
      revised calculation of acquisition geometry arrays for Landsat · 8c7a5323
      Daniel Scheffler authored
      algorithms.GEOPROCESSING:
      - calc_VAA_using_fullSceneCornerLonLat():
          - implemented keyword 'orbit_params' as fallback; added docstring
          -added 90 degrees to VAA
      - calc_VZA_array(): added some notes
      - calc_RAA_array(): added docstring
      algorithms.L1A_P.L1A_object():
      - calc_mean_VAA(): updated calc_VAA_using_fullSceneCornerLonLat call
      algorithms.L1C_P.L1C_object():
      - VZA_arr.getter: added parameter 'meshwidth'
      - revised VAA_arr.getter
      algorithms.METADATA:
      - get_orbit_params(): added some notes
      - updated __version__
      Former-commit-id: b8a055bf
      Former-commit-id: d08a8a57
      8c7a5323
    • Daniel Scheffler's avatar
      Fixed unexpected value in cloud mask of Sentinel-2 · c8d34ab1
      Daniel Scheffler authored and Mathias Peters's avatar Mathias Peters committed
      algorithms.L1C_P.AtmCorr:
      - _join_results_to_inObjs(): fixed unexpected value in cloud mask of Sentinel-2
      - updated __version__
      Former-commit-id: 89a43619
      Former-commit-id: 93bc4d81
      c8d34ab1
  31. 25 Jan, 2017 1 commit
    • Daniel Scheffler's avatar
      Fixed pickling issues after failed AC. · 2636cfcb
      Daniel Scheffler authored and Mathias Peters's avatar Mathias Peters committed
      algorithms.L1C_P.AtmCorr:
      - run_atmospheric_correction(): AC input arrays are now deleted before object is passed to exception handler
      processing.pipeline:
      - L1C_map(): disabled auto-dumping of AC inputs in case of error
      processing-process_controller:
      - L1C_processing(): set CPUs to 5 for testing
      - updated __version__
      Former-commit-id: 2a997e23
      Former-commit-id: 064a13b1
      2636cfcb
  32. 24 Jan, 2017 3 commits
    • Daniel Scheffler's avatar
      Bugfixes · 45420946
      Daniel Scheffler authored and Mathias Peters's avatar Mathias Peters committed
      algorithms.gms_object.GMS_object:
      - set_pathes(): bugfix
      - __getstate__(): now deletes self._masks after L2C
      - revised log_for_fullArr_or_firstTile()
      algorithms.L1C_P.AtmCorr:
      - bugfix for not properly closing self.logger before logging into self.InObjs (fixes permission errors)
      misc.path_generator:
      - get_path_tempdir(): added hex-code for uniqueness
      - updated __version__
      Former-commit-id: d74b8bf8
      Former-commit-id: ffb3ccac
      45420946
    • Daniel Scheffler's avatar
      config update · 40c7cae3
      Daniel Scheffler authored
      Former-commit-id: 7b77c593
      Former-commit-id: 4462c756
      40c7cae3
    • Daniel Scheffler's avatar
      algorithms.gms_object.GMS_object: · cef2a152
      Daniel Scheffler authored
      - from_tiles(): bugfix for not updating array-dependent attributes
      - from_sensor_subsystems(): bugfix for not recreating loggers after subsystem merging
      - removed deprecated MetaObj2ODict()
      - get_subset_obj(): now operable from imBounds; fixed inconsistent attribute names
      - added to_tiles()
      algorithms.L2B_P.L2B_object:
      - added log message
      algorithms.METADATA:
      - get_LayerbandsAssignment: bugfix
      misc.helper_functions:
      - cut_GMS_obj_into_blocks(): now calls GMS_object.to_tiles()
      - revised cornerLonLat_to_postgreSQL_poly()
      misc.mgrs_tile:
      - EPSG: bugfix
      processing.pipeline:
      - L1A_map_1() and L2A_map(): changed call for object conversion into tiles
      - removed /database/cloud_classifier from version controlling -> now external resource
      - updated __version__
      Former-commit-id: c9f38cb9
      Former-commit-id: 9b805282
      cef2a152
  33. 21 Jan, 2017 1 commit
    • Daniel Scheffler's avatar
      first version fully operable for Sentinel-2 (including atmospheric correction and cloud masks) · 3282d365
      Daniel Scheffler authored
      algorithms.gms_object.GMS_object:
      - GMS_identifier: is only a getter now, not a singleton
      - added property 'MetaObj' / 'meta_odict' -> self-synchronizing
      - added LayerBandsAssignment.setter
      - revised arr.setter
      - revised mask_nodata.setter
      - mask_clouds.getter: bugfix
      - revised mask_clouds.setter
      - added property 'mask_clouds_confidence'
      - revised masks.setter
      - revised dem.setter
      - ac_options: number of CPUs are now passed (temporarily disabled)
      - revised ac_errors.setter
      - added LBA2bandnames
      - attributes2dict: bugfix for not addin meta_odict
      - revised from_tiles()
      - added from_sensor_subsystems(): a function for merging multiple subsystems of the same sensor (needed for Sentinel-2 and ASTER)
      - revised build_combined_masks_array()
      - get_subset_obj(): multiple bugfixes
      - to_GMS_file(): updated in the context of new property architecture
      - delete_previous_proc_level_results: added functionality to delete subsystem products after subsystems have been merged
      algorithms.gms_object.failed_GMS_object:
      - removed logger creation
      algorithms.L1A_P.L1A_object:
      - import_metadata(): updated METADATA call
      - calc_TOARadRefTemp: added warning when using Sentinel-2 Gains
      - calc_corner_positions(): added UTC timezone to datetime object
      algorithms.L1B_P.L1B_object:
      - _get_reference_image_params_pgSQL(): replaced temp_logger by self.logger
      - revised correct_spatial_shifts()
      algorithms.L1C_P:
      - L1C_object:
          - removed deprecated atm_corr()
      - AtmCorr:
          - __init__: added warning
          - revised data property
          - _meta_get_aux_data(): changed lonlat_arr to float16
          - run_atmospheric_correction(): added 'dump' keyword
          - revised _join_results_to_inObjs()
      algorithms.L2B_P.L2B_object:
      - spectral_homogenization(): bugfix
      algorithms.METADATA:
      - METADATA:
          - __init__(): is now initialized by GMS_identifier; does not directly run metadata reader
          - added read_meta()
          - Read_Sentinel2A_xmls(): updated setting of Gains
          - refactored to_meta_odict() to to_odict() and revised the function
          - added from_odict()
          - revised filter_layerdependent_metadata()
          - added 'map_odictKeys_objAttrnames'
          - get_LayerBandsAssignment():
              - added functionality to return full LBA for Sentinel-2 and ASTER
              - now properly handles bands removed after L1C and L2B
      io.Output_writer:
      - mask_to_ENVI_Classification(): bugfix
      misc.definition_dicts:
      - get_GMS_sensorcode(): added codes for Sentinel-2 full and ASTER full
      - get_outFillZeroSaturated(): added bool dtype
      - is_dataset_provided_as_fullScene(): added ASTER full and Sentinel-2 full
      misc.exception_handler:
      - log_uncaught_exceptions(): bugfix for 'disable_exception_handler'
      misc.helper_functions:
      - cut_GMS_obj_into_blocks(): bugfix
      misc.logging.GMS_logger:
      - added scene ID to formatter
      misc.mgrs_tile:
      - replaced deprecated reference
      misc.path_generator:
      - get_baseN(): bugfix
      - added get_path_ac_input_dump()
      processing.multiproc:
      - MAP: added functionality to disable multiprocessing
      processing.pipeline:
      - revised L2A_map(): new L2A calls
      processing.process_controller:
      - add_local_availability(): changed get_LayerBandsAssignment call
      - get_DB_objects(): bugfix for copied memory link during GMS object init
      - L1C_processing() raises NotImplementedError in tiled mode
      - L2A_processing(): added grouping of subsystems
      - updated __version__
      3282d365