Commit 6e4a2301 authored by Daniel Scheffler's avatar Daniel Scheffler
Browse files

Added URL checker and corresponding CI job. Removed travis related files....


Added URL checker and corresponding CI job. Removed travis related files. Replaced hard-coded links in documentation by cross-linking directives.
Signed-off-by: Daniel Scheffler's avatarDaniel Scheffler <danschef@gfz-potsdam.de>
parent 6c96453a
Pipeline #16760 failed with stages
in 62 minutes and 22 seconds
......@@ -45,6 +45,15 @@ test_styles:
when: always
test_urls:
stage: test
script:
- source /root/miniconda3/bin/activate ci_env
- pip install -U urlchecker
- make urlcheck
when: always
test_spechomo_install:
stage: test
script:
......
# Config file for automatic testing at travis-ci.org
language: python
python:
- 3.6
- 3.5
- 3.4
- 2.7
# Command to install dependencies, e.g. pip install -r requirements.txt --use-mirrors
install: pip install -U tox-travis
# Command to run tests, e.g. python setup.py test
script: tox
......@@ -58,6 +58,9 @@ lint: ## check style with flake8
pycodestyle spechomo --exclude="*.ipynb,*.ipynb*" --max-line-length=120 > ./tests/linting/pycodestyle.log
-pydocstyle spechomo > ./tests/linting/pydocstyle.log
urlcheck: ## check for dead URLs
urlchecker check . --file-types .py,.rst,.md,.json
test: ## run tests quickly with the default Python
python setup.py test
......
......@@ -21,17 +21,6 @@ SpecHomo - Spectral homogenization of multispectral satellite data
Status
------
.. .. image:: https://img.shields.io/travis/danschef/spechomo.svg
:target: https://travis-ci.org/danschef/spechomo
.. .. image:: https://readthedocs.org/projects/spechomo/badge/?version=latest
:target: https://spechomo.readthedocs.io/en/latest/?badge=latest
:alt: Documentation Status
.. .. image:: https://pyup.io/repos/github/danschef/spechomo/shield.svg
:target: https://pyup.io/repos/github/danschef/spechomo/
:alt: Updates
.. image:: https://gitext.gfz-potsdam.de/geomultisens/spechomo/badges/master/pipeline.svg
:target: https://gitext.gfz-potsdam.de/geomultisens/spechomo/commits/master
.. image:: https://gitext.gfz-potsdam.de/geomultisens/spechomo/badges/master/coverage.svg
......
......@@ -47,7 +47,8 @@ extensions = [
'sphinx.ext.todo',
'sphinxarg.ext',
'sphinx_autodoc_typehints',
'sphinx.ext.intersphinx'
'sphinx.ext.intersphinx',
'sphinx.ext.autosectionlabel'
]
# Add any paths that contain templates here, relative to this directory.
......@@ -136,10 +137,14 @@ def setup(app):
# Add mappings for intersphinx extension (allows to link to the API reference of other sphinx documentations)
intersphinx_mapping = {
'geoarray': ('http://danschef.gitext.gfz-potsdam.de/geoarray/doc/', None),
'python': ('http://docs.python.org/3', None),
'geoarray': ('https://danschef.gitext-pages.gfz-potsdam.de/geoarray/doc/', None),
'python': ('https://docs.python.org/3', None),
}
# Configure the autosectionlabel extension
autosectionlabel_prefix_document = True
autosectionlabel_maxdepth = 10
# -- Options for HTML output -------------------------------------------
......
......@@ -65,4 +65,3 @@ you through the process.
.. _pip: https://pip.pypa.io
.. _Python installation guide: http://docs.python-guide.org/en/latest/starting/installation/
.. _conda: https://conda.io/docs
.. _`dependencies of SpecHomo`: https://gitext.gfz-potsdam.de/danschef/arosics/-/blob/master/requirements.txt
......@@ -5,13 +5,12 @@ The classifiers included in the SpecHomo package
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The SpecHomo package provides a set of classifiers that can be used for spectral homogenization of various sensor
combinations (see `here <https://geomultisens.gitext-pages.gfz-potsdam.de/spechomo/doc/usage/
available_transformations.html#which-sensor-transformations-are-available>`__). These classifiers have been trained
based on simulated multispectral remote sensing images created by spectral convolution from hyperspectral airborne
datasets. These hyperspectral datasets were chosen with the intention to include as many different land-cover types
as possible in order to maximize spectral variability. This makes these classifiers suitable for many application
fields. Details on the underlying datasets and the used algorithms to create the classifiers can be found
in `Scheffler et al. 2020 <https://doi.org/10.1016/j.rse.2020.111723>`__.
combinations (see :ref:`here <usage/available_transformations:Which sensor transformations are available?>`). These
classifiers have been trained based on simulated multispectral remote sensing images created by spectral convolution
from hyperspectral airborne datasets. These hyperspectral datasets were chosen with the intention to include as many
different land-cover types as possible in order to maximize spectral variability. This makes these classifiers suitable
for many application fields. Details on the underlying datasets and the used algorithms to create the classifiers can
be found in `Scheffler et al. 2020 <https://doi.org/10.1016/j.rse.2020.111723>`__.
Which sensor transformations are available?
......
......@@ -7,11 +7,11 @@ class. Please see the linked content for a full documentation of this class.
For the sake of simplicity, the usage of this class is described below, at the
**example of Landsat-8 data, spectrally adapted to Sentinel-2A**. Transformations between various other sensors are
possible, see `here <https://geomultisens.gitext-pages.gfz-potsdam.de/spechomo/doc/usage/available_transformations.html
#which-sensor-transformations-are-available>`__.
possible, see :ref:`usage/available_transformations:Which sensor transformations are available?`.
First, load the Landsat-8 `surface reflectance`_ image that you want to transform to the spectral domain of Sentinel-2A
(we use the `geoarray`_ library for this - it is installed with SpecHomo):
First, load the Landsat-8 :ref:`surface-reflectance <usage/input_data_requirements:Surface Reflectance>` image that you
want to transform to the spectral domain of Sentinel-2A (we use the `geoarray`_ library for this - it is installed with
SpecHomo):
.. code-block:: python
......@@ -23,8 +23,7 @@ First, load the Landsat-8 `surface reflectance`_ image that you want to transfor
Please make sure, that the Landsat-8 input image contains the right bands in the correct order before you run the
homogenization! By running the :func:`list_available_transformations<spechomo.utils.list_available_transformations>`
function as described
`here <https://geomultisens.gitext-pages.gfz-potsdam.de/spechomo/doc/usage/available_transformations.html>`__, you can
function as described :doc:`here <./available_transformations>`, you can
find out, that the needed band list is ['1', '2', '3', '4', '5', '6', '7']. These band numbers refer to the
official provider band-names as described for Landsat at the
`USGS website <https://www.usgs.gov/faqs/what-are-band-designations-landsat-satellites>`__.
......@@ -66,9 +65,7 @@ follows:
* You can directly copy/paste possible input parameters for the
:meth:`predict_by_machine_learner<spechomo.prediction.SpectralHomogenizer.predict_by_machine_learner>` method
from the :func:`list_available_transformations<spechomo.utils.list_available_transformations>`
function as described
`here <https://geomultisens.gitext-pages.gfz-potsdam.de/spechomo/doc/usage/available_transformations.html
#which-sensor-transformations-are-available>`__.
function as described :ref:`here <usage/available_transformations:Which sensor transformations are available?>`.
* You may also save the homogenization results to other GDAL compatible image formats
(see :meth:`geoarray.GeoArray.save` for details).
* Further explanation on input parameters like `method`, `n_clusters`, `classif_alg` or `global_clf_threshold` is
......@@ -77,5 +74,4 @@ follows:
evaluation of the different homogenization algorithms available in the SpecHomo library.
.. _`surface reflectance`: https://geomultisens.gitext-pages.gfz-potsdam.de/spechomo/doc/usage/input_data_requirements.html#surface-reflectance
.. _`geoarray`: https://gitext.gfz-potsdam.de/danschef/geoarray
......@@ -59,7 +59,7 @@ req = [
req_setup = ['setuptools-git'] # needed for package_data version-controlled by GIT
req_test = ['coverage', 'nose', 'nose2', 'nose-htmloutput', 'rednose']
req_test = ['coverage', 'nose', 'nose2', 'nose-htmloutput', 'rednose', 'urlchecker']
req_doc = ['sphinx-argparse', 'sphinx_rtd_theme', 'sphinx-autodoc-typehints']
......
......@@ -251,7 +251,7 @@ class Cluster_Learner(object):
"""Predict target satellite spectral information using separate prediction coefficients for spectral clusters.
NOTE: This version of the prediction function uses the prediction coefficients of multiple spectral clusters
and computes the result as weighted average of them. Therefore, the classifcation map must assign
and computes the result as weighted average of them. Therefore, the classification map must assign
multiple spectral cluster to each input pixel.
# NOTE: At unclassified pixels (cmap_3D[y,x,z>0] == -1) the prediction result using global coefficients
......@@ -454,7 +454,6 @@ class ClassifierCollection(object):
except KeyError:
raise(KeyError("The classifier has no key '%s'. Available keys are: %s"
% (item, repr(self))))
# def save_to_json(self, filepath):
# a = 1
# pass
......
......@@ -278,7 +278,7 @@ class RefCube(object):
"%s != %s" % (LayerBandsAssignment, self.LayerBandsAssignment)
# reshape 2D spectra array to one image column (refcube is an image with spectral information in the 3rd dim.)
im_col = spectra.reshape(spectra.shape[0], 1, spectra.shape[1])
im_col = spectra.reshape((spectra.shape[0], 1, spectra.shape[1]))
meta = self.data.metadata # needs to be copied to the new GeoArray
......@@ -382,7 +382,7 @@ class RefCube(object):
def plot_sample_spectra(self, image_basename, cluster_label='all', include_mean_spectrum=True,
include_median_spectrum=True, ncols=5, **kw_fig):
# type: (Union[str, int, List], str, bool, bool, int, dict) -> plt.figure
# type: (Union[str, int, List], str, bool, bool, int, dict) -> 'plt.figure'
from matplotlib import pyplot as plt
if isinstance(cluster_label, int):
......
......@@ -35,6 +35,7 @@ dependencies:
- pydocstyle
- rednose
- sphinx-argparse
- urlchecker
# setup requirements
- setuptools-git
......
......@@ -35,6 +35,7 @@ Tests for spechomo.classifier
import os
import json
from unittest import TestCase
from tempfile import TemporaryDirectory
from spechomo.classifier import Cluster_Learner
from spechomo import __path__
......@@ -71,9 +72,9 @@ class Test_ClusterClassifier(TestCase):
outstr = json.dumps(jsonable_dict, sort_keys=True, indent=4)
self.assertIsInstance(outstr, str)
# def test_save_to_json(self):
# with TemporaryDirectory() as tmpDir:
# self.clf.save_to_json(os.path.join(tmpDir, 'clf.json'))
def test_save_to_json(self):
with TemporaryDirectory() as tmpDir:
self.clf.save_to_json(os.path.join(tmpDir, 'clf.json'))
# class Test_ClassifierCollection(TestCase):
......
......@@ -93,7 +93,8 @@ class Test_ReferenceCube_Generator(unittest.TestCase):
src_im = GeoArray(self.SHC.ims_ref[0])
unif_random_spectra = self.SHC.cluster_image_and_get_uniform_spectra(src_im)
tgt_rsr = RSR(satellite='Sentinel-2A', sensor='MSI')
# tgt_rsr = RSR(satellite='Sentinel-2A', sensor='MSI')
tgt_rsr = RSR(satellite='Terra', sensor='MODIS', sort_by_cwl=True)
unif_random_spectra_rsp = \
self.SHC.resample_spectra(unif_random_spectra,
src_cwl=np.array(src_im.meta.band_meta['wavelength'], dtype=np.float).flatten(),
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment