doc_usage_ci.md 6.96 KB
Newer Older
1
<!--
Maximilian Dolling's avatar
bak2  
Maximilian Dolling committed
2
SPDX-FileCopyrightText: 2020 Helmholtz Centre Potsdam - GFZ German Research Centre for Geosciences, Germany (https://www.gfz-potsdam.de/)
3 4 5 6

SPDX-License-Identifier: CC0-1.0
-->

7
# Software Quality Assurance
Maximilian Dolling's avatar
Maximilian Dolling committed
8 9 10 11 12

---

## CI/CD usage

13
The following manual describes how to use the Software Quality Assurance with GitLab CI/CD.
Maximilian Dolling's avatar
Maximilian Dolling committed
14 15
It is highly advisable to know how to use git and GitLab when using this method.

16
> **Note:** It is possible to hook the Software Quality Assurance to GitHub Actions and other code hosting platforms. 
Maximilian Dolling's avatar
Maximilian Dolling committed
17 18 19 20 21

---

## Decide on way to retrieve the report

22
There are two different ways to get the generated report by the `Software Quality Assurance`.
Maximilian Dolling's avatar
Maximilian Dolling committed
23 24 25 26 27

**Artifact**

When you opt for this option, the report is available via the GitLab web-interface.
You can simply download.
Maximilian Dolling's avatar
Maximilian Dolling committed
28
More information available [here](https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html)
Maximilian Dolling's avatar
Maximilian Dolling committed
29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45

This option is easier to set up.

**Commit** (recommended)

> **Note:** this option is **required** by certain (future) services. 

---

### 1) Project member permissions
To apply certain changes described below, you need at least `Maintainer` [permissions](https://docs.gitlab.com/ee/user/permissions.html).

### 2) Host the project on any GitLab
How to do that, is described [here](https://docs.gitlab.com/ee/gitlab-basics/create-project.html)

### 3) Enable CI/CD

Maximilian Dolling's avatar
Maximilian Dolling committed
46
In the GitLab web-interface from the page of your repository navigate to `config > General > Visibility, project features, permissions`.
Maximilian Dolling's avatar
Maximilian Dolling committed
47 48 49 50
Enable `Pipelines` and set them to `Only Project Members`.

### 4) Add GitLab Runner

51
To have an available Runner is **required** for the Software Quality Assurance to function.
Maximilian Dolling's avatar
Maximilian Dolling committed
52 53 54 55
This will not be covered here.
Talk to your local admin get one up and running.

**Runner requirements**
56
* tag: `sqa`
Maximilian Dolling's avatar
Maximilian Dolling committed
57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76
* executor: docker

> **WARNING:** Be aware that the runtime for the services can go up to several hours, depending on the project size!

Further information are available [here](https://docs.gitlab.com/runner/install/).

---

### 5.1) Retrieve report by Artifact

#### 5.1.1) Set up .gitlab-ci.yml

This yaml block is set up for project without prior GitLab Ci/CD setup.
If you already have `.gitlab-ci.yml` in your project, there should be someone withing your project who can help you adapt.

Copy the content of the following code block to a file named `.gitlab-ci.yml` in the base directory of your project.
Commit and push your changes.

```yaml
stages:
77
  - sqa
Maximilian Dolling's avatar
Maximilian Dolling committed
78

79 80
sqa:
  stage: sqa
Maximilian Dolling's avatar
Maximilian Dolling committed
81
  image:
82
    name: "gitext.gfz-potsdam.de:5000/software/services/fair/software-quality-assurance/software-quality-assurance:latest"
Maximilian Dolling's avatar
Maximilian Dolling committed
83 84
    entrypoint: [""]
  tags:
85
    - sqa
Maximilian Dolling's avatar
Maximilian Dolling committed
86
  script:
Maximilian Dolling's avatar
Maximilian Dolling committed
87 88
    - cp -r "$CI_PROJECT_DIR" /repo
    - rm -rf /repo/meta
Maximilian Dolling's avatar
Maximilian Dolling committed
89
    - docker-entrypoint.sh --all
Maximilian Dolling's avatar
Maximilian Dolling committed
90
    - cp -rT /repo "$CI_PROJECT_DIR"
Maximilian Dolling's avatar
Maximilian Dolling committed
91 92 93 94 95
  only:
    refs:
      - master
  artifacts:
    paths:
96
      - meta/software-quality-assurance-report.md
Maximilian Dolling's avatar
Maximilian Dolling committed
97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114
    expire_in: 30 days
```

---

### 5.2) Retrieve report by commit

This manual focuses on [UNIX](https://en.wikipedia.org/wiki/Unix) systems like Linux or MacOS.

> **Note:** for Windows the commands can slightly differ, but is fully compatible. 

#### 5.2.1) Set up deploy key

First we need to generate a key pair via the terminal.
It will function as [deploy key](https://docs.gitlab.com/ee/ssh/#deploy-keys).
More information on generating a key pair can be found [here](https://docs.gitlab.com/ee/ssh/#generating-a-new-ssh-key-pair).

```console
115
ssh-keygen -t rsa -b 4096 -C "[PROJECT_NAME]_software-quality-assurance" -f [OUTPUTFILE] -q -N ""
Maximilian Dolling's avatar
Maximilian Dolling committed
116 117
```

Maximilian Dolling's avatar
Maximilian Dolling committed
118
In the GitLab web-interface from the page of your repository navigate to `config > CI/CD > Deploy Keys`.
Maximilian Dolling's avatar
Maximilian Dolling committed
119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141
Paste the content of `[OUTPUTFILE].pub` to the field `key` and give it any unique name (like *PROJECT_NAME*_deploy-key)
Check the 'Write access allowed' checkbox.

Save the changes.

> **Note:** it is recommended to either store `[OUTPUTFILE].pub` securely or delete it.

#### 5.2.2) Set up Variables

We need to tell our CI/CD the identity of the GitLab server we are using.
To do that, we request it from the server via a terminal command.
It returns two lines with the public identity of the GitLab server.

```console
ssh-keyscan -t rsa [GITLAB_DOMAIN]]

# example

ssh-keyscan -t rsa gitext.gfz-potsdam.de
# gitext.gfz-potsdam.de:22 SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ubuntu0.3
gitext.gfz-potsdam.de ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsiq5txhABcG9oePf47SCd+w/tfSs6NWKaMRd2BfBi9DGORQX4wwV4CYvSJbMfJvQO5SMGwhnEAXe0bWihC2V8X7lzcePrpjfP+uH2lmipcFbV9g3iQpM7Fusr96IV65v/qQ6HE4+KkHJLR1vEKj1AOaVgDww7CHhVAHCkvqcSwwtfOjBbqyfQ2Su7O6UNzs1ecQcIXnQNO8ebzimv3c8mKdo9j8i5eiWVqzRivmi/R3XIIc2T695mTLyxUplqfyvZpDL90F9VEjr/3HM9/3dASbonaTH8SoMRCxOy8Fpz0Va1MdpSor0uoemQjbu8aZzGB0+tsE5eWjuFaZib8QDF
```

Maximilian Dolling's avatar
Maximilian Dolling committed
142
Now open the GitLab web-interface and navigate to `config > CI/CD > Variables` from from the page of your repository.
Maximilian Dolling's avatar
Maximilian Dolling committed
143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164
We need to fill in two variables here.
Click on 'Add Variable'.

As key type in `GITLAB_DEPLOY_KEY`.
As value paste the content of `[OUTPUTFILE]` (was generated in 5.2.1).
Click on 'Add Variable'.

For the second one:
As key type in `SSH_KNOWN_HOSTS_GITLAB`.
As value paste both output lines of the `ssh-keyscan` command (see above).
Click on 'Add Variable'.

#### 5.2.3) Set up .gitlab-ci.yml

This yaml block is set up for project without prior GitLab Ci/CD setup.
If you already have `.gitlab-ci.yml` in your project, there should be someone withing your project who can help you adapt.

Copy the content of the following code block to a file named `.gitlab-ci.yml` in the base directory of your project.
Commit and push your changes.

```yaml
stages:
165
  - sqa
Maximilian Dolling's avatar
Maximilian Dolling committed
166

167 168
sqa:
  stage: sqa
Maximilian Dolling's avatar
Maximilian Dolling committed
169
  image:
170
    name: "gitext.gfz-potsdam.de:5000/software/services/fair/software-quality-assurance/software-quality-assurance:latest"
Maximilian Dolling's avatar
Maximilian Dolling committed
171 172
    entrypoint: [""]
  tags:
173
    - sqa
Maximilian Dolling's avatar
Maximilian Dolling committed
174 175 176 177 178 179 180 181 182
  before_script:
    - mkdir -p ~/.ssh
    - chmod 700 ~/.ssh
    - echo "$SSH_KNOWN_HOSTS_GITLAB" > ~/.ssh/known_hosts
    - chmod 644 ~/.ssh/known_hosts
  script:
    - eval $(ssh-agent -s)
    - echo "$GITLAB_DEPLOY_KEY" | tr -d '\r' | ssh-add -
    - git config user.name "GitLab CI:${CI_PROJECT_NAMESPACE}/${CI_PROJECT_NAME}"
183
    - git config user.email "[PROJECT_NAME]_software-quality-assurance"
Maximilian Dolling's avatar
Maximilian Dolling committed
184 185 186 187 188 189 190
    - cp -r "$CI_PROJECT_DIR" /repo
    - docker-entrypoint.sh --all
    - cp -rT /repo "$CI_PROJECT_DIR"
    - cd meta
    - find -mindepth 1 -maxdepth 1 -type d -exec rm -r {} \;
    - cd ..
    - git add -f .
191
    - git commit -m "[AUTOMATIC] generated meta/software-quality-assurance-report.md by gitlab ci"
Maximilian Dolling's avatar
Maximilian Dolling committed
192 193 194 195
    - git push "git@${CI_SERVER_HOST}:${CI_PROJECT_NAMESPACE}/${CI_PROJECT_NAME}.git" "HEAD:${CI_COMMIT_REF_NAME}"
  only:
    refs:
      - master
Maximilian Dolling's avatar
Maximilian Dolling committed
196 197
  except:
    changes:
198
      - "meta/software-quality-assurance-report.md"
Maximilian Dolling's avatar
Maximilian Dolling committed
199 200 201
```

**Important**
202
In the yaml file above fill in the line `- git config user.email "[PROJECT_NAME]_software-quality-assurance"`
Maximilian Dolling's avatar
Maximilian Dolling committed
203 204 205 206 207 208
The content of user.email must be identical with what was entered in 5.2.1)

> **Note:** it is recommended to either store `[OUTPUTFILE]` securely or delete it.

### 6) Examples

Maximilian Dolling's avatar
Maximilian Dolling committed
209
An example of both variants applied can be found [here](../.gitlab-ci.yml)