Commit 275cf369 authored by Niklas Bohn's avatar Niklas Bohn
Browse files

Merge branch 'enhancement/deploy_pypi' into 'master'

Enhancement/deploy pypi

See merge request !36
parents fa803de7 4dbb7e23
......@@ -88,3 +88,17 @@ pages: # this job must be called 'pages' to advise GitLab to upload content to
expire_in: 30 days
# only:
# - master
deploy_pypi:
stage: deploy
dependencies:
- test_sicor_enmap
script:
- source /root/miniconda3/bin/activate sicor_env
- pip install -U twine
- python setup.py sdist
- twine upload dist/*
only:
- /^v\d+\.\d+\.\d+([abc]\d*)?$/
except:
- dev
......@@ -10,6 +10,13 @@ New features:
* TBD
0.14.3 (2020-09-02)
-------------------
New features:
* The package is now available on the Python Package Index.
* Added 'deploy_pypi' CI job.
0.14.2 (2020-05-14)
-------------------
New features:
......
=====
SICOR
=====
.. figure:: docs/images/sicor_logo_lr.png
.. image:: http://enmap.gitext.gfz-potsdam.de/sicor/doc/_images/sicor_logo_lr.png
:width: 300px
:alt: SICOR Logo
Sensor Independent Atmospheric Correction of optical Earth Observation (EO) data from both multispectral and
hyperspectral instruments. Currently, SICOR can be applied to Sentinel-2 and EnMAP data but the implementation of
......@@ -16,43 +18,55 @@ Get It
------
Sicor depends on some open source packages which are usually installed without problems by the automatic install
SICOR depends on some open source packages which are usually installed without problems by the automatic install
routine. However, for some projects, we strongly recommend resolving the dependency before the automatic installer
is run. This approach avoids problems with conflicting versions of the same software.
Using conda_, the recommended approach is:
.. code-block:: console
.. code-block:: bash
# create virtual environment for sicor, this is optional
conda create -y -q --name sicor python=3
conda create -y -q -c conda-forge --name sicor python=3
source activate sicor
conda install -y -q -c conda-forge gdal pytables h5py llvmlite pyfftw scikit-learn
conda config --add channels conda-forge
conda install --yes -q gdal numpy scikit-image matplotlib pyproj rasterio shapely geopandas pyresample pytables h5py llvmlite pyfftw scikit-learn numba
Install Git Large File Storage (git-lfs_).
To install SICOR, clone the following repository and install sicor to your local python:
.. code-block:: bash
.. code-block:: console
git_lfs_v='2.1.1'
/bin/bash -i -c "wget https://github.com/git-lfs/git-lfs/releases/download/v${git_lfs_v}/git-lfs-linux-amd64-${git_lfs_v}.tar.gz; \
tar -zxvf git-lfs-linux-amd64-${git_lfs_v}.tar.gz; \
cd git-lfs-${git_lfs_v}; \
bash ./install.sh"
git clone https://gitext.gfz-potsdam.de/EnMAP/sicor.git
cd sicor
python setup.py install
Install Git Large File Storage (git-lfs_). You can install Git LFS on your MacOS/Linux computer using a package manager
such as Homebrew or MacPorts:
To install Sicor, clone the following repository and install sicor to your local python:
.. code-block:: bash
brew install git-lfs
git lfs install
.. code-block:: console
After installation, some needed LFS files have to be downloaded:
git clone https://gitext.gfz-potsdam.de/EnMAP/sicor.git
cd sicor; make install
.. code-block:: bash
git lfs pull
If you like to run SICOR in multispectral mode for Sentinel-2, several additional tables have to be downloaded.
In this case, you also need to install the ECMWF API client:
.. code-block:: bash
pip install ecmwf-api-client
make download-tables
SICOR repository operations can be started using make, available options are:
.. code-block:: console
$ make
$ make
make options: (run make [option] to perform action):
......@@ -158,14 +172,6 @@ From command line (currently, only applicable to multispectral case):
sicor_ecmwf.py --help
sicor_ac.py --help
Status
------
.. figure:: https://gitext.gfz-potsdam.de/EnMAP/sicor/badges/master/build.svg
.. figure:: https://gitext.gfz-potsdam.de/EnMAP/sicor/badges/master/coverage.svg
See also the latest coverage_ report or test_report_.
Features
--------
......@@ -179,8 +185,6 @@ Features
* CH4 retrieval from hyperspectral data: work in progress
.. include:: ../docs/authors.rst
.. _coverage: http://enmap.gitext.gfz-potsdam.de/sicor/coverage/
.. _test_report: http://enmap.gitext.gfz-potsdam.de/sicor/coverage/report.html
.. _documentation: http://enmap.gitext.gfz-potsdam.de/sicor/doc/
......
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# coding: utf-8
# SICOR is a freely available, platform-independent software designed to process hyperspectral remote sensing data,
# and particularly developed to handle data from the EnMAP sensor.
# This file contains the package setup tools.
# Copyright (C) 2018 Niklas Bohn (GFZ, <nbohn@gfz-potsdam.de>),
# German Research Centre for Geosciences (GFZ, <https://www.gfz-potsdam.de>)
# This software was developed within the context of the EnMAP project supported by the DLR Space Administration with
# funds of the German Federal Ministry of Economic Affairs and Energy (on the basis of a decision by the German
# Bundestag: 50 EE 1529) and contributions from DLR, GFZ and OHB System AG.
# This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public
# License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later
# version.
# This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied
# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
# You should have received a copy of the GNU General Public License along with this program.
# If not, see <https://www.gnu.org/licenses/>.
from setuptools import setup, find_packages
from importlib import util
import warnings
requirements_save_to_install_with_setuptools = [
'scikit-image', 'glymur', 'pyprind', 'geopandas',
'dicttoxml', 'tables', 'pandas', 'xlrd', 'psutil', 'sympy', 'pyproj',
'cerberus', 'scipy', 'tqdm', 'dill', 'shapely', 'geoarray', 'mpld3',
'jsmin', 'iso8601', 'pint', 'matplotlib', 'sphinx-argparse', 'numpy',
'pillow', 'pylint', 'mypy', 'pycodestyle', 'pydocstyle', 'flake8',
'sphinx', 'arosics', 'numba', 'netCDF4', 'pyrsr']
"scikit-image", "glymur", "pyprind", "geopandas",
"dicttoxml", "tables", "pandas", "xlrd", "psutil", "sympy", "pyproj",
"cerberus", "scipy", "tqdm", "dill", "shapely", "geoarray", "mpld3",
"jsmin", "iso8601", "pint", "matplotlib", "sphinx-argparse", "numpy",
"pillow", "pylint", "mypy", "pycodestyle", "pydocstyle", "flake8",
"sphinx", "arosics", "numba", "netCDF4", "pyrsr"]
other_requirements = { # dict of "[needed import]: [proposed command for install]
"gdal": "conda install -c conda-forge gdal",
......@@ -23,8 +45,17 @@ other_requirements = { # dict of "[needed import]: [proposed command for instal
"sklearn": "conda install -c conda-forge scikit-learn"
}
with open("README.rst") as readme_file:
readme = readme_file.read()
version = {}
with open("sicor/version.py", encoding="utf-8") as version_file:
exec(version_file.read(), version)
requirements = requirements_save_to_install_with_setuptools
setup_requirements = ["setuptools-git"]
test_requirements = requirements + ["coverage", "mock"]
# test for packages that do not install well with pip
......@@ -41,44 +72,48 @@ if len(not_installed) > 0:
))
setup(
name='sicor',
packages=find_packages(exclude=['tests*']),
version='0.14.2',
description="Sensor Independent Atmospheric Correction",
authors="Niklas Bohn, André Hollstein, René Preusker",
author_email='nbohn@gfz-potsdam.de',
url='https://gitext.gfz-potsdam.de/EnMAP/sicor',
package_data={"options": ["sicor/options/*"]},
scripts=[
"bin/sicor_ac.py",
"bin/sicor_ecmwf.py",
"bin/sicor_ac_EnMAP.py"
authors="Niklas Bohn, Daniel Scheffler, Maximilian Brell, André Hollstein, René Preusker",
author_email="nbohn@gfz-potsdam.de",
classifiers=[
"Development Status :: 4 - Beta",
"License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
"Natural Language :: English",
"Programming Language :: Python :: 3.6",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8"
],
description="Sensor Independent Atmospheric Correction",
data_files=[
("data", [
"sicor/sensors/S2MSI/GranuleInfo/data/S2_tile_data_lite.json",
"sicor/sensors/S2MSI/GranuleInfo/data/S2_tile_data_full.json",
"sicor/sensors/S2MSI/data/Sentinel-2 MSI Spectral Responses.xlsx",
"sicor/sensors/S2MSI/data/S2A_SNR_model.csv",
"sicor/sensors/S2MSI/data/S2A_SNR_model.xlsx",
"sicor/AC/data/k_liquid_water_ice.xlsx",
"sicor/AC/data/newkur_EnMAP.dat",
"sicor/AC/data/solar_irradiances_400_2500_1.dill"
])],
keywords=["SICOR", "EnMAP", "EnMAP-Box", "hyperspectral", "remote sensing", "satellite", "atmospheric correction"],
include_package_data=True,
install_requires=requirements,
license="GNU General Public License v3 (GPLv3)",
zip_safe=False,
classifiers=[
'Development Status :: Beta',
'Natural Language :: English',
'Programming Language :: Python :: 3.5+',
long_description=readme,
long_description_content_type="text/x-rst",
name="sicor",
package_dir={"sicor": "sicor"},
package_data={"sicor": ["AC/data/*"]},
packages=find_packages(exclude=["tests*", "examples"]),
scripts=[
"bin/sicor_ac.py",
"bin/sicor_ecmwf.py",
"bin/sicor_ac_EnMAP.py"
],
setup_requires=setup_requirements,
test_suite='tests',
tests_require=test_requirements
tests_require=test_requirements,
url="https://gitext.gfz-potsdam.de/EnMAP/sicor",
version=version["__version__"],
zip_safe=False
)
# check for pygrib
if not util.find_spec('pygrib'):
warnings.warn('You need to install pygrib manually (use pip install pygrib) if you are using Linux'
......
import sys
min_major, min_minor = 3, 5
major, minor, micro, releaselevel, serial = sys.version_info
if (major, minor) < (min_major, min_minor):
raise ValueError("This is python %i.%i, sicor needs %i.%i or higher." % (major, minor, min_major, min_minor))
#!/usr/bin/env python
# coding: utf-8
__version__ = '0.14.2'
# SICOR is a freely available, platform-independent software designed to process hyperspectral remote sensing data,
# and particularly developed to handle data from the EnMAP sensor.
# This file contains the package version info.
# Copyright (C) 2018 Niklas Bohn (GFZ, <nbohn@gfz-potsdam.de>),
# German Research Centre for Geosciences (GFZ, <https://www.gfz-potsdam.de>)
# This software was developed within the context of the EnMAP project supported by the DLR Space Administration with
# funds of the German Federal Ministry of Economic Affairs and Energy (on the basis of a decision by the German
# Bundestag: 50 EE 1529) and contributions from DLR, GFZ and OHB System AG.
# This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public
# License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later
# version.
# This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied
# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
# You should have received a copy of the GNU General Public License along with this program.
# If not, see <https://www.gnu.org/licenses/>.
__version__ = '0.14.3'
__versionalias__ = '20200514.01'
__author__ = 'Niklas Bohn'
......@@ -2,7 +2,7 @@
context_dir="./context"
dockerfile="sicor_ci.docker"
tag="sicor_ci:0.14.2"
tag="sicor_ci:0.14.3"
gitlab_runner_prefix="sicor_gitlab_CI_runner"
echo "Remove existing runners"
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment