Skip to content
GitLab
Projects
Groups
Snippets
Help
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
S
Software Quality Assurance
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
10
Issues
10
List
Boards
Labels
Service Desk
Milestones
Merge Requests
0
Merge Requests
0
Operations
Operations
Metrics
Incidents
Packages & Registries
Packages & Registries
Package Registry
Container Registry
Analytics
Analytics
Repository
Value Stream
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
id2
S
Software
S
services
FAIR
Software Quality Assurance
Commits
919c9247
Commit
919c9247
authored
May 15, 2020
by
Maximilian Dolling
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
added authors list
parent
4c1f805f
Changes
14
Hide whitespace changes
Inline
Side-by-side
Showing
14 changed files
with
162 additions
and
16 deletions
+162
-16
.gitlab-ci.yml
.gitlab-ci.yml
+1
-1
README.md
README.md
+8
-5
build/dependencies/list_authors/dependencies_OS.txt
build/dependencies/list_authors/dependencies_OS.txt
+2
-0
build/dependencies/list_authors/requirements.txt
build/dependencies/list_authors/requirements.txt
+1
-0
build/docker/dep/Dockerfile
build/docker/dep/Dockerfile
+2
-0
build/docker/dev/Dockerfile
build/docker/dev/Dockerfile
+1
-1
build/docker/dev/docker-entrypoint.sh
build/docker/dev/docker-entrypoint.sh
+9
-0
build/docker/dev/entrypoint-manual.txt
build/docker/dev/entrypoint-manual.txt
+1
-0
doc/doc_service_description.md
doc/doc_service_description.md
+10
-1
meta/ci-services-report.md
meta/ci-services-report.md
+48
-8
src/services/list_authors/service_list_authors.py
src/services/list_authors/service_list_authors.py
+54
-0
src/services/list_authors/service_list_authors.sh
src/services/list_authors/service_list_authors.sh
+11
-0
src/services/report/service_report.py
src/services/report/service_report.py
+5
-0
test/test_report.md
test/test_report.md
+9
-0
No files found.
.gitlab-ci.yml
View file @
919c9247
...
...
@@ -5,7 +5,7 @@ stages:
-
ci-services
variables
:
VERSION
:
0.
2
.0
VERSION
:
0.
3
.0
DOCKER_DST_IMAGE_NAME
:
ci-services
build-deploy:dep:
...
...
README.md
View file @
919c9247
...
...
@@ -14,9 +14,10 @@ Either via the [GitLab CI/CD](https://docs.gitlab.com/ee/ci/) (recommended) or a
> **Example:** You can find a full report of all available services [here](meta/ci-services-report.md)
**Current available services:**
*
Programming
language d
etection
*
Programming
Language D
etection
*
License Check
*
Report Generation
*
List Authors
---
...
...
@@ -54,6 +55,12 @@ You can find a detailed description of all implemented services [here](doc/doc_s
## Further planned features
*
license check
*
R
*
Java
*
Fortran
*
JavaScript
*
C, C++
*
Matlab
*
license header check
*
license generation
*
comment check
...
...
@@ -65,7 +72,3 @@ You can find a detailed description of all implemented services [here](doc/doc_s
*
curate meta data
---
## Help
TODO =)
\ No newline at end of file
build/dependencies/list_authors/dependencies_OS.txt
0 → 100644
View file @
919c9247
git
bash
\ No newline at end of file
build/dependencies/list_authors/requirements.txt
0 → 100644
View file @
919c9247
PyYAML==5.3
build/docker/dep/Dockerfile
View file @
919c9247
...
...
@@ -13,6 +13,7 @@ COPY ./build/dependencies /dependencies
RUN
apk add
--update
--no-cache
$(
cat
/dependencies/base/dependencies_OS.txt
)
RUN
apk add
--update
--no-cache
$(
cat
/dependencies/language_detection/dependencies_OS.txt
)
RUN
apk add
--update
--no-cache
$(
cat
/dependencies/license_check/dependencies_OS.txt
)
RUN
apk add
--update
--no-cache
$(
cat
/dependencies/list_authors/dependencies_OS.txt
)
# install ruby dependencies
RUN
bundle
install
--gemfile
=
/dependencies/language_detection/Gemfile
...
...
@@ -21,6 +22,7 @@ RUN bundle install --gemfile=/dependencies/license_check/Gemfile
# install python dependencies
RUN
pip
install
-r
/dependencies/base/requirements.txt
RUN
pip
install
-r
/dependencies/license_check/requirements.txt
RUN
pip
install
-r
/dependencies/list_authors/requirements.txt
# cleanup
RUN
rm
-rf
/dependencies
...
...
build/docker/dev/Dockerfile
View file @
919c9247
...
...
@@ -9,7 +9,7 @@ RUN apk upgrade
# copy entrypoint script to executables
COPY
./build/docker/dev/docker-entrypoint.sh /usr/local/bin/
COPY
./build/docker/dev/entrypoint-manual.txt /usr/local/bin/
RUN
echo
'0.
2
.0'
>
/version.txt
RUN
echo
'0.
3
.0'
>
/version.txt
# copy service scripts to root dir
COPY
./src/services /services
...
...
build/docker/dev/docker-entrypoint.sh
View file @
919c9247
...
...
@@ -13,6 +13,7 @@ fi
if
[
"
$1
"
=
"--all"
]
||
[
"
$1
"
=
"-a"
]
then
/services/language_detection/service_language_detection.sh
/services/list_authors/service_list_authors.sh
/services/license_check/service_license_check.sh
/services/report/service_report.sh
chmod
-R
777 /repo/meta
...
...
@@ -32,6 +33,13 @@ then
exit
0
fi
if
[
"
$1
"
=
"--authors"
]
||
[
"
$1
"
=
"-al"
]
then
/services/list_authors/service_list_authors.sh
chmod
-R
777 /repo/meta
exit
0
fi
if
[
"
$1
"
=
"--license"
]
||
[
"
$1
"
=
"-li"
]
then
/services/license_check/service_license_check.sh
...
...
@@ -51,6 +59,7 @@ then
/services/language_detection/service_language_detection.sh
--test
/services/license_check/service_license_check.sh
--test
/services/report/service_report.sh
--test
/services/list_authors/service_list_authors.sh
--test
chmod
-R
777 /repo/meta
exit
0
fi
...
...
build/docker/dev/entrypoint-manual.txt
View file @
919c9247
...
...
@@ -7,6 +7,7 @@ Needs a directory to run services on bound to '/repo'
Available services:
-a , --all runs all services (reporting is last)
-al, --authors list all authors of the software
-h , --help shows this help page
-la, --language lists used programming languages in given repo
-li, --license lists all used libraries and their licenses
...
...
doc/doc_service_description.md
View file @
919c9247
...
...
@@ -6,6 +6,7 @@
*
language detection
*
license check
*
list authors
*
report
### General
...
...
@@ -29,8 +30,16 @@ The source code can be found [here](../src/services/language_detection).
---
### List Authors
The list authors service can be run with the
`-al`
or the
`--authors`
flag when running it either local or via CI/CD.
It uses the command
`git log --pretty="%ae" | sort | uniq`
to generate the list.
The source code can be found
[
here
](
../src/services/list_authors
)
.
---
### Report
The report service will read and merge all the sub-reports.
It will then generate a markdown version of the assembled report.
It will then generate a
cense_check
markdown version of the assembled report.
The source code can be found
[
here
](
../src/services/report
)
.
\ No newline at end of file
meta/ci-services-report.md
View file @
919c9247
...
...
@@ -7,18 +7,27 @@
**Software Name:**
ci-services
**Software Location:**
[
gitext
](
https://gitext.gfz-potsdam.de/software/services/fair/ci-services
)
**Software Location:**
local
**Last Commit:**
67169faf84c36524704ac9e420300a4ac99460e5
**Last Commit:**
0
**Report Time:**
14/05/2020 13:22:28
**Report Time:**
0
**Report Version:**
0.
2
.0
**Report Version:**
0.
3
.0
---
## Completed Services
### Authors List
> **Description: Lists all people that committed to this repository**
*
ci-services@gitext.gfz-potsdam.de
*
mdolling@gfz-potsdam.de
### License Check
> **Description: Generates a list of dependencies and their license**
...
...
@@ -32,13 +41,44 @@
|PSF-2.0|8|collections
<br>
datetime
<br>
json
<br>
os
<br>
pathlib
<br>
re
<br>
shutil
<br>
sys|
|mock|1|stdlib|
#### Unknown objects
**unknown language**
*
test/testfiles/unknown language/test.testlanguage
**unknown license**
*
Python / ./downloads/numpy-1.9.2-cp34-none-win32.whl / unknown location
*
Python / Mopidy-Dirble / unsupported version modifier (~=1.1)
*
Python / beautifulsoup4 / unknown version
*
Python / coverage / unsupported version modifier (!=3.5)
*
Python / docopt / unsupported version modifier (>=0.5.0)
*
Python / green / unknown version
*
Python / http://wxpython.org/Phoenix/snapshot-builds/wxPython_Phoenix-3.0.3.dev1820+49a8884-cp34-none-win_amd64.whl / unknown location
*
Python / keyring / unsupported version modifier (>=4.1.1)
*
Python / nose / unknown version
*
Python / nose-cov / unknown version
*
Python / rejected / unknown version
### Programming Language Detection
> **Description: Detects used programming languages. Output values are bytes of code.**
|language|bytes of code|
|---|---|
|Dockerfile|
1944
|
|Python|
22987
|
|Ruby|
1566
|
|Shell|
3193
|
|Dockerfile|
0
|
|Python|
0
|
|Ruby|
0
|
|Shell|
0
|
src/services/list_authors/service_list_authors.py
0 → 100644
View file @
919c9247
import
os
import
sys
from
datetime
import
datetime
from
pathlib
import
Path
import
yaml
# setting vars
repo_dir
=
sys
.
argv
[
1
]
working_dir
=
repo_dir
+
sys
.
argv
[
2
]
test
=
False
if
"--test"
in
sys
.
argv
:
test
=
True
with
open
(
'/repo/.ci-services.yml'
)
as
settings_yaml
:
settings
=
yaml
.
load
(
settings_yaml
,
Loader
=
yaml
.
FullLoader
)
settings_yaml
.
close
()
# collect report data
project_name
=
os
.
popen
(
'basename $(git remote get-url origin)'
).
read
()[:
-
5
]
last_commit_ref
=
os
.
popen
(
'git rev-parse HEAD'
).
read
()[
0
:
40
]
project_location
=
os
.
popen
(
'echo $CI_PROJECT_URL'
).
read
()[:
-
1
]
if
project_location
==
''
:
project_location
=
'local'
report_time
=
datetime
.
now
().
strftime
(
"%d/%m/%Y %H:%M:%S"
)
report_version
=
''
with
open
(
'/version.txt'
,
'r'
)
as
file
:
report_version
=
file
.
read
().
replace
(
'
\n
'
,
''
)
# assure that report directory exists
Path
(
working_dir
).
mkdir
(
parents
=
True
,
exist_ok
=
True
)
authors
=
[]
with
open
(
'{}/authors.txt'
.
format
(
working_dir
))
as
authors_txt
:
for
line
in
authors_txt
:
authors
.
append
(
line
[:
-
1
])
authors_txt
.
close
()
# generate report as dict
report
=
{
'name'
:
project_name
,
'report time'
:
report_time
,
'last commit hash'
:
last_commit_ref
,
'location'
:
project_location
,
'report version'
:
report_version
,
'Authors List'
:
{
'description'
:
'Lists all people that committed to this repository'
,
'result'
:
authors
}}
# create report
# write projects to file
with
open
(
working_dir
+
'/subreport_list authors.yml'
,
'w'
)
as
f
:
data
=
yaml
.
dump
(
report
,
f
)
src/services/list_authors/service_list_authors.sh
0 → 100755
View file @
919c9247
#!/usr/bin/env bash
REPO_DIR
=
"/repo"
REPORT_DIR
=
"/meta/list_authors"
cd
"
$REPO_DIR
"
||
exit
1
mkdir
-p
"
$REPO_DIR$REPORT_DIR
"
git log
--pretty
=
"%ae"
|
sort
|
uniq
>
"
$REPO_DIR$REPORT_DIR
/authors.txt"
python /services/list_authors/service_list_authors.py
"
$REPO_DIR
"
"
$REPORT_DIR
"
"
$1
"
src/services/report/service_report.py
View file @
919c9247
...
...
@@ -118,3 +118,8 @@ with open(working_dir + '/ci-services-report.md', "a+") as md:
for
lib
in
report_dict_ordered
[
subreport
][
'result'
][
'unknowns'
][
'libraries'
][
language
]:
reason
=
report_dict_ordered
[
subreport
][
'result'
][
'unknowns'
][
'libraries'
][
language
][
lib
]
md
.
write
(
'* {} / {} / {}
\n\n
'
.
format
(
language
,
lib
,
reason
))
# list authors
if
subreport
==
'Authors List'
:
for
subreport_result
in
report_dict_ordered
[
subreport
][
'result'
]:
md
.
write
(
"* {}
\n\n
"
.
format
(
subreport_result
))
test/test_report.md
View file @
919c9247
...
...
@@ -19,6 +19,15 @@
## Completed Services
### Authors List
> **Description: Lists all people that committed to this repository**
*
ci-services@gitext.gfz-potsdam.de
*
mdolling@gfz-potsdam.de
### License Check
> **Description: Generates a list of dependencies and their license**
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment