Shell to Python Continuous Deployment

Get a common set of Python CI & CD commands
from various contexts of CA, DNS, CI and OS / OCI.

Why

Project workflows can occur in:

• an online or offline context, dealing with custom CA & DNS servers
• various Continuous Integration platforms, with different workflow syntaxes
• specific Operating Systems, having different sets of commands and packages
• specific containers for JavaScript based actions (ForgeJo, Gitea, GitHub) What if there was a unified way to do so?

How

Picture it…

Group environment variables

• numbered
  • Certificate Authority armored certificates
  • Domain Name System servers IP addresses
• set group environment variables for
  • ref names for deployment contexts of feature, staging & release
  • alternate locations of
    • Git repositories of this project & its parent framework
    • Operating Systems & Software packages
  • SSH key & hosts for deployments
  • a standalone POSIX shell script payload

Workflow files

• calling the POSIX shell script payload as their very first step
• using a unified YAML syntax made of actions implemented in Python

Continous Integration & Deployment

• configuring the Operating System container
• installing Python system & virtual environments

What

Features

Handle project workflows in a unified way:

• whether the network infrastructure is
  • online
  • offline with custom
    • Certificate Authorities
    • Domain Name System servers
• whatever the Continuous Integration platform
  • ForgeJo
  • Gitea
  • GitHub
  • GitLab
  • SourceHut
• whatever the Operating System container
  • Alma
    • 9
    • 8
  • Alpine
    • 3.20
    • 3.19
  • Arch
    • 20240818 (.0.255804)
    • 20240101 (.0.204074)
  • Debian
    • Bookworm (12)
    • Bullseye (11)
  • Fedora
    • 40
    • 39
  • OpenSUSE
    • 15.6
    • 15.5
  • Rocky
    • 9
    • 8
  • Ubuntu
    • Noble (24.04)
    • Jammy (22.04)

Environment variables

Variable	Description	Default
SPCD_CA_n	Numbered CA certificates
SPCD_CMD_SUM	Command to check sums	sha512sum
SPCD_DNS_n	Numbered name servers	9.9.9.9
SPCD_GIT_MAIN	Main Git repository	spcd
SPCD_GIT_ROOT	Root Git repository	rwx
SPCD_GIT_SHUNIT	ShUnit Git repository	shunit2
SPCD_REF_FEATURE	Feature deployment ref	f
SPCD_REF_RELEASE	Release deployment ref	main
SPCD_REF_STAGING	Staging deployment ref	dev
SPCD_SSH_HOSTS	domain.tld ssh-type pub
SPCD_SSH_KEY	SSH private key
SPCD_URL_ALMA	Alma repository URL	https://repo.almalinux.org
SPCD_URL_ALPINE	Alpine repository URL	https://dl-cdn.alpinelinux.org
SPCD_URL_ARCH	Arch repository URL	https://geo.mirror.pkgbuild.com
SPCD_URL_DEBIAN	Debian repository URL	https://deb.debian.org
SPCD_URL_EPEL	EPEL repository URL	https://dl.fedoraproject.org
SPCD_URL_FEDORA	Fedora repository URL	https://rpmfind.net
SPCD_URL_OPENSUSE	OpenSUSE repository URL	https://download.opensuse.org
SPCD_URL_PYTHON	Python repository URL	https://pypi.org/simple
SPCD_URL_ROCKY	Rocky repository URL	https://dl.rockylinux.org
SPCD_URL_UBUNTU	Ubuntu repository URL	https://ubuntu.mirrors.ovh.net

Operating Systems containers

Latest

os	https	updt-ca	python	graphviz	plantuml	shellcheck	shunit	shfmt
Arch 20240818	☑	☑	3.12	12.0	1.2023.13	0.10	2.1.8	3.8
Alpine 3.20	☑	☐	3.12	9.0	1.2024.4	0.10	2.1.8	3.8
Fedora 40	☑	☑	3.12 → 3.13	9.0	1.2024.6	0.9	2.1.6	3.7
Debian Bookworm	☐	☐	3.11	2.42	1.2020.2	0.9	2.1.8	3.6
OpenSUSE 15.6	☐	☑	3.6 → 3.12	2.48	1.2020.9	0.8	2.1.6	3.5
Ubuntu Noble	☐	☐	3.12	u2.42	u1.2020.2	u0.9	2.1.8	u3.8
Alma / Rocky 9	☑	☑	3.9 → 3.12	2.44	e1.2024.6	e0.8	git

Previous

os	https	updt-ca	python	graphviz	plantuml	shellcheck	shunit	shfmt
Alpine 3.19	☑	☐	3.11	9.0	1.2023.12	0.9	2.1.8	3.7
Fedora 39	☑	☑	3.12 → 3.13	8.1	1.2024.6	0.9	2.1.6	3.5
OpenSUSE 15.5	☐	☑	3.6 → 3.11	2.48	1.2020.9	0.8	2.1.6	3.5
Alma / Rocky 8	☑	☑	3.6 → 3.12	2.40	e1.2024.6	e0.6	git

Older Python

os	https	updt-ca	python	graphviz	plantuml	shellcheck	shunit	shfmt
Ubuntu Jammy	☐	☐	3.10	u2.42	u1.2020.2	u0.8	2.1.6	u3.4
Debian Bullseye	☐	☐	3.9	2.42	1.2020.2	0.7	2.1.6

Who

By

Marc Beninca.

For

People feeling the need to aim for consistence in the CI & CD universe.

Where

• Discord
• IRC
• Web
  • Git
  • RSS
  • Workflows

When

First

• try git repo url variable first for shunit

Shell

• fit banner to text
• review repositories handling for systems
  • opensuse
    • codecs repository
    • disable & enable https
• write function to clone git repositories
• write unit tests

Shell → Python

• check file & variable sums
• install system packages
  • epel
    • plantuml
    • shellcheck
  • graphviz
  • openssh
  • rsync
  • shfmt
  • shunit
• python virtual environment
  • install system package
  • configure pip
  • venv
    • create
    • activate
• relay environment module name

Python

• check
  • .py
  • .sh
    • maximum line length
• detect ssh private key type
• fit banner to text
• implement substeps
• list sys.path
• set workflow templates
  • actions
  • includes
• write unit tests

Later

• turn readme into documentation