rwx.work/spcd
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
Shell to Python Continuous Deployment https://spcd.rwx.work
1430 commits 2 branches 0 tags 7.5 MiB
Shell 74.9%
Python 25.1%
Find a file
Marc Beninca f0aa902904
Some checks failed
/ arch (push) Successful in 1m39s
Details
/ alpine (push) Successful in 42s
Details
/ fedora (push) Successful in 11m26s
Details
/ opensuse (push) Successful in 2m19s
Details
/ alma (push) Successful in 2m31s
Details
/ rocky (push) Successful in 3m57s
Details
/ ubuntu (push) Successful in 1m26s
Details
/ debian (push) Failing after 1m3s
Details
doc/projects
2024-09-16 01:40:27 +02:00
.forgejo/workflows language → locale 2024-08-30 22:57:57 +02:00
.gitea/workflows workflows/jobs 2024-08-30 04:55:11 +02:00
.github/workflows workflows/jobs 2024-08-30 04:55:11 +02:00
spcd doc/projects 2024-09-16 01:40:27 +02:00
.editorconfig indent .sh with spaces 2024-08-19 10:35:27 +02:00
.gitignore ignore dist 2024-06-14 14:26:12 +02:00
.gitlab-ci.yml check-project/ruff 2024-08-07 23:32:56 +02:00
.shellcheckrc disable/3043 2024-08-31 00:51:56 +02:00
license.md license 2024-05-03 16:33:15 +02:00
pyproject.toml doc/quiet 2024-09-16 01:18:03 +02:00
readme.md tasks/link 2024-09-15 22:44:49 +02:00
spcd.sh indent 2024-09-09 00:46:16 +02:00

readme.md

Shell to Python Continuous Deployment

Get a common set of Python based CI / CD commands
from various contexts of CA, DNS, CI and OS / OCI.

Why

Project workflows can occur in:

  • an online or offline context, dealing with custom CA & DNS servers
  • various Continuous Integration platforms, with different workflow syntaxes
  • specific Operating Systems, having different commands and package names
  • specific containers for JavaScript based actions (ForgeJo, Gitea, GitHub)

What if there was a unified way to do so?

How

Picture it…

Group environment variables

  • numbered
    • Certificate Authority armored certificates
    • Domain Name System servers IP addresses
  • ref names for deployment contexts of feature, staging & release
  • alternate locations of
    • Operating Systems & Software packages
    • Git repositories of
      • this project
      • its parent framework
      • projects not available in some distributions
        • shunit2
  • SSH key & hosts for deployments
  • a standalone POSIX shell script payload

Workflow files

  • calling the POSIX shell script payload as their very first step
  • using a unified YAML syntax made of actions implemented in Python

Integration & Deployment

Shell

  • setting up the chosen Operating System container
    • Domain Name System resolving
    • package manager configuration & repositories
    • Certificate Authorities trusting
    • Operating System updating
  • installing Git to clone
    • this project
    • its parent framework
  • installing both Python
    • system environment
    • virtual environment
  • generating a Python module to switch context

Python

  • setting up an OpenSSH client
    • known hosts
    • private key
  • and finally parsing commands from
    • system
    • workflow
  • to run available actions
    • check project
    • build project
    • browse workspace
    • synchronize

What

Features

Handle project workflows in a unified way:

  • whether the network infrastructure is

    • online
    • offline with custom
      • Certificate Authorities
      • Domain Name System servers

  • whatever the Continuous Integration platform

    • ForgeJo
    • Gitea
    • GitHub
    • GitLab
    • SourceHut

  • whatever the Operating System container

    • Alma
      • 9
      • 8
    • Alpine
      • 3.20
      • 3.19
    • Arch
      • 20240818 (.0.255804)
      • 20240101 (.0.204074)
    • Debian
      • Bookworm (12)
      • Bullseye (11)
    • Fedora
      • 40
      • 39
    • OpenSUSE
      • 15.6
      • 15.5
    • Rocky
      • 9
      • 8
    • Ubuntu
      • Noble (24.04)
      • Jammy (22.04)

Environment variables

Variable Description Default
SPCD_CA_n Numbered CA certificates
SPCD_DNS_n Numbered name servers 9.9.9.9
SPCD_GIT_RWX RWX Git repository rwx
SPCD_GIT_SHUNIT ShUnit Git repository shunit2
SPCD_GIT_SPCD SPCD Git repository spcd
SPCD_REF_FEATURE Feature deployment ref f
SPCD_REF_RELEASE Release deployment ref main
SPCD_REF_STAGING Staging deployment ref dev
SPCD_SSH_HOSTS domain.tld ssh-type pub
SPCD_SSH_KEY SSH private key
SPCD_TXT_LOCALE Locale to use for text en_US
SPCD_URL_ALMA Alma repository URL https://repo.almalinux.org
SPCD_URL_ALPINE Alpine repository URL https://dl-cdn.alpinelinux.org
SPCD_URL_ARCH Arch repository URL https://geo.mirror.pkgbuild.com
SPCD_URL_DEBIAN Debian repository URL https://deb.debian.org
SPCD_URL_EPEL EPEL repository URL https://dl.fedoraproject.org
SPCD_URL_FEDORA Fedora repository URL https://rpmfind.net
SPCD_URL_OPENSUSE OpenSUSE repository URL https://download.opensuse.org
SPCD_URL_PYTHON Python repository URL https://pypi.org
SPCD_URL_ROCKY Rocky repository URL https://dl.rockylinux.org
SPCD_URL_UBUNTU Ubuntu repository URL https://ubuntu.mirrors.ovh.net

Operating Systems containers

Latest

os https up ca python graphviz plantuml shellcheck shunit shfmt gource ffmpeg
Arch 20240818 ☑ ☑ 3.12 12.0 1.2023.13 0.10 2.1.8 3.8 0.54 7.0.2
Alpine 3.20 ☑ ☐ 3.12 9.0 1.2024.4 0.10 2.1.8 3.8 0.54 6.1.1
Fedora 40 ☑ ☑ 3.12 → 3.13 9.0 1.2024.6 0.9 2.1.6 3.7 0.55 6.1.2
Debian Bookworm ☐ ☐ 3.11 2.42 1.2020.2 0.9 2.1.8 3.6 0.54 5.1.6
OpenSUSE 15.6 ☐ ☑ 3.6 → 3.12 2.48 1.2020.9 0.8 2.1.6 3.5 0.54 4.4.4
Ubuntu Noble ☐ ☐ 3.12 u2.42 u1.2020.2 u0.9 2.1.8 u3.8 0.54 6.1.1
Alma / Rocky 9 ☑ ☑ 3.9 → 3.12 2.44 e1.2024.6 e0.8 e5.1.4

Previous

os https up ca python graphviz plantuml shellcheck shunit shfmt gource ffmpeg
Alpine 3.19 ☑ ☐ 3.11 9.0 1.2023.12 0.9 2.1.8 3.7 0.54 6.1.1
Fedora 39 ☑ ☑ 3.12 → 3.13 8.1 1.2024.6 0.9 2.1.6 3.5 0.55 6.1.1
OpenSUSE 15.5 ☐ ☑ 3.6 → 3.11 2.48 1.2020.9 0.8 2.1.6 3.5 0.54 4.4.4
Alma / Rocky 8 ☑ ☑ 3.6 → 3.12 2.40 e1.2024.6 e0.6

Older Python

os https up ca python graphviz plantuml shellcheck shunit shfmt gource ffmpeg
Ubuntu Jammy ☐ ☐ 3.10 u2.42 u1.2020.2 u0.8 2.1.6 u3.4 0.51 4.4.2
Debian Bullseye ☐ ☐ 3.9 2.42 1.2020.2 0.7 2.1.6 0.51 4.3.7

Who

By

For

  • People feeling the need to aim for consistency in the CI / CD universe

Where

Chat

Forge

Deployment

When

Tasks for a 1st stable release

Documentation

  • build instead of readme
  • generate graphs
  • list behavior differences
    • Continuous Integration platforms
    • Operating Systems

Shell

  • comment
  • git/pypi fallback for rwx/spcd
  • handle errors
  • packages
    • configure
      • apk
      • pacman
      • zypper
    • repositories
      • codecs
        • epel
        • fedora
        • opensuse
      • opensuse
        • disable & enable https
  • locales
    • persist
  • test
  • workspace variable

Shell → Python

  • check file & variable sums
  • install system packages
    • epel
      • plantuml
      • shellcheck
    • graphviz
    • openssh
    • rsync
    • shfmt
    • shunit
  • python virtual environment
    • install system package
    • configure pip
    • venv
      • create
      • activate
  • query package
  • relay environment module name

Python

  • check
    • .py
    • .sh
      • maximum line length
  • comment
  • detect ssh private key type
  • fit banner to text
  • implement substeps
  • list sys.path
  • set workflow templates
    • actions
    • includes
  • try git repo url variable first for shunit
  • test

Task stack

  • automate versions fetching
  • gource, xvfb, xauth
  • handle openh264 repositories
  • link from workspace to actions root
  • rpm fusion
  • tex
  • translate to french
  • try to support nix