Commit 308b824e authored by Daniel Scheffler's avatar Daniel Scheffler
Browse files

Updated requirements. Added CI job 'test_gms_preprocessing_install'. Updated...


Updated requirements. Added CI job 'test_gms_preprocessing_install'. Updated installation instructions.
Signed-off-by: Daniel Scheffler's avatarDaniel Scheffler <danschef@gfz-potsdam.de>
parent c172757e
Pipeline #13117 failed with stages
in 17 minutes and 16 seconds
......@@ -49,6 +49,45 @@ test_gms_preprocessing:
when: always
test_gms_preprocessing_install:
stage: test
script:
- source /root/miniconda3/bin/activate
- conda update conda
- conda create -y -q -c conda-forge --name gms_test python=3
- source activate gms_test
# update conda and python
- conda update -n base -c conda-forge conda
# - conda update -c conda-forge python
# avoid package incompatibilities
# - conda config --set channel_priority strict # otherwise gdal or libgdal may be installed from defaults channel
# install some dependencies that cause trouble when installed via pip
- conda install -y -c conda-forge numpy gdal scikit-image pyproj lxml geopandas ipython matplotlib basemap scikit-learn=0.23.2 shapely pyhdf python-fmask holoviews
# install not pip-installable deps of arosics
- conda install -y -c conda-forge pyfftw pykrige
# install not pip-installable deps of sicor
- conda install -y -q -c conda-forge glymur pygrib cachetools pyhdf h5py pytables llvmlite numba
- pip install ecmwf-api-client # FIXME remove as long as this is no more needed for the hyperspectral part of SICOR
# install gms_preprocessing
- pip install -e .
- cd ..
- pwd
- ls
# test importability
- python -c "import gms_preprocessing; print(gms_preprocessing)"
- python -c "from gms_preprocessing import ProcessController; print(ProcessController)"
only:
- master
- dev
test_styles:
stage: test
script:
......
......@@ -133,26 +133,24 @@ Using conda_, the recommended approach is:
.. code:: console
# create virtual environment for gms_preprocessing, this is optional
conda create -y -q --name gms_preprocessing python=3
conda create -c conda-forge --name gms_preprocessing python=3
source activate gms_preprocessing
conda install -y -q -c anaconda hdf5 pytables; \
conda install -y -q -c conda-forge gdal pygrib h5py; \
conda install -y -q -c numba llvmlite; \
conda install -y -q -c conda-forge psycopg2 python-fmask"
Then install sicor:
# install some dependencies that cause trouble when installed via pip
conda install -c conda-forge numpy gdal scikit-image pyproj lxml geopandas ipython matplotlib basemap scikit-learn=0.23.2 shapely pyhdf python-fmask holoviews
.. code-block:: console
git clone https://gitext.gfz-potsdam.de/EnMAP/sicor.git
cd sicor; make install
# install not pip-installable deps of arosics
conda install -c conda-forge pyfftw pykrige
Clone the following repository and install gms_preprocessing to your local python:
.. code-block:: console
# install not pip-installable deps of sicor
conda install -c conda-forge glymur pygrib cachetools pyhdf h5py pytables llvmlite numba
pip install ecmwf-api-client
# install gms_preprocessing
git clone https://gitext.gfz-potsdam.de/geomultisens/gms_preprocessing.git
cd gms_preprocessing; make install
cd gms_preprocessing
pip install .
To enable lock functionality (needed for CPU / memory / disk IO management), install redis-server_:
......@@ -161,13 +159,6 @@ To enable lock functionality (needed for CPU / memory / disk IO management), ins
sudo apt-get install redis-server
.. .. The pip installer does not work as long as the packages arosics, geoarray, py_tools_ds and sicor are not public.
.. .. * Use the pip-Installer:
.. .. pip install git+https://gitext.gfz-potsdam.de/geomultisens/gms_preprocessing.git # for HTTPS authentification
.. .. pip install git+git@gitext.gfz-potsdam.de:geomultisens/gms_preprocessing.git # for SSH-Key authentification
License
-------
......
py_tools_ds>=0.12.4
geoarray>=0.8.33
arosics>=0.8.5
git+https://gitext.gfz-potsdam.de/EnMAP/sicor.git
git+https://gitext.gfz-potsdam.de/geomultisens/spechomo.git
sicor
spechomo
pyrsr>=0.3.1
matplotlib
numpy
......
......@@ -44,7 +44,7 @@ requirements = [
'pandas', 'spectral>=0.16', 'geopandas', 'iso8601', 'pyinstrument', 'geoalchemy2', 'sqlalchemy', 'natsort',
'psycopg2', 'py_tools_ds>=0.12.4', 'geoarray>=0.8.33', 'arosics>=0.8.7', 'six', 'tqdm', 'jsmin', 'cerberus',
'nested_dict', 'openpyxl', 'timeout_decorator', 'redis', 'retools', 'redis-semaphore', 'psutil', 'pyrsr>=0.3.1',
'spechomo @ git+https://gitext.gfz-potsdam.de/geomultisens/spechomo#egg=spechomo'
'spechomo', 'sicor'
# spectral<0.16 has some problems with writing signed integer 8bit data
# fmask # conda install -c conda-forge python-fmask
# 'pyhdf', # conda install --yes -c conda-forge pyhdf
......
......@@ -6,7 +6,6 @@ channels:
dependencies:
- python>=3.7.*
- pip
- pyqt
- numpy
- gdal
- scikit-image
......@@ -21,7 +20,6 @@ dependencies:
- pyhdf
- python-fmask
- holoviews
- bokeh
# arosics
- pyfftw
......@@ -82,4 +80,4 @@ dependencies:
- geoarray>=0.8.33
- arosics>=0.8.7
- pyrsr>=0.3.1
- git+https://gitext.gfz-potsdam.de/geomultisens/spechomo.git#egg=spechomo
- spechomo
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment