# Shell to Python Continuous Deployment Get a common set of Python based CI / CD commands\ from various contexts of CA, DNS, CI and OS / OCI. --- ## Why Project workflows can occur in: * an online or offline context, dealing with custom CA & DNS servers * various Continuous Integration platforms, with different workflow syntaxes * specific Operating Systems, having different commands and package names * specific containers for JavaScript based actions (ForgeJo, Gitea, GitHub) What if there was a unified way to do so? --- ## How Picture it… ### Group environment variables * numbered * Certificate Authority armored certificates * Domain Name System servers IP addresses * ref names for deployment contexts of feature, staging & release * alternate locations of * Operating Systems & Software packages * Git repositories of * this project * its parent framework * projects not available in some distributions * shunit2 * SSH key & hosts for deployments * a standalone POSIX shell script payload ### Workflow files * calling the POSIX shell script payload as their very first step * using a unified YAML syntax made of actions implemented in Python ### Integration & Deployment #### Shell * [X] setting up the chosen Operating System container * [X] Domain Name System resolving * [X] package manager configuration & repositories * [X] Certificate Authorities trusting * [X] Operating System updating * [X] installing Git to clone * [X] this project * [X] its parent framework * [X] installing Python * [X] system environment * [X] virtual environment * [ ] managed version * [X] generating a Python module to switch context #### Python * [X] setting up an OpenSSH client * [X] known hosts * [ ] private key * [ ] and finally parsing commands from * [X] system * [ ] workflow * [ ] to run available actions * [ ] check project * [ ] build project * [X] browse workspace * [ ] synchronize --- ## What ### Features Handle project workflows in a unified way: * whether the network infrastructure is * [X] online * [X] offline with custom * [X] Certificate Authorities * [X] Domain Name System servers * whatever the Continuous Integration platform * [X] ForgeJo * [X] Gitea * [X] GitHub * [X] GitLab * [ ] SourceHut * whatever the Operating System container | System | Latest | Previous | |:---------|:---------------------------|:---------------------------| | Alma | * [X] 9 | * [X] 8 | | Alpine | * [X] 3.20 | * [X] 3.19 | | Arch | * [X] 20240818 (.0.255804) | * [X] 20240101 (.0.204074) | | Debian | * [X] Bookworm (12) | * [ ] Bullseye (11) | | Fedora | * [X] 40 | * [X] 39 | | OpenSUSE | * [ ] 15.6 | * [ ] 15.5 | | Rocky | * [X] 9 | * [X] 8 | | Ubuntu | * [X] Noble (24.04) | * [ ] Jammy (22.04) | ### Environment variables | Variable | Description | Default | |:------------------|:-------------------------|:--------------------------------| | SPCD_CA_n | Numbered CA certificates | | | SPCD_DNS_n | Numbered name servers | 9.9.9.9 | | SPCD_GIT_RWX | RWX Git repository | rwx | | SPCD_GIT_SHUNIT | ShUnit Git repository | shunit2 | | SPCD_GIT_SPCD | SPCD Git repository | spcd | | SPCD_REF_FEATURE | Feature deployment ref | f | | SPCD_REF_RELEASE | Release deployment ref | main | | SPCD_REF_STAGING | Staging deployment ref | dev | | SPCD_SSH_HOSTS | domain.tld ssh-type pub | | | SPCD_SSH_KEY | SSH private key | | | SPCD_TXT_LOCALE | Locale to use for text | en_US | | SPCD_URL_ALMA | Alma repository URL | https://repo.almalinux.org | | SPCD_URL_ALPINE | Alpine repository URL | https://dl-cdn.alpinelinux.org | | SPCD_URL_ARCH | Arch repository URL | https://geo.mirror.pkgbuild.com | | SPCD_URL_DEBIAN | Debian repository URL | https://deb.debian.org | | SPCD_URL_EPEL | EPEL repository URL | https://dl.fedoraproject.org | | SPCD_URL_FEDORA | Fedora repository URL | https://rpmfind.net | | SPCD_URL_OPENSUSE | OpenSUSE repository URL | https://download.opensuse.org | | SPCD_URL_PYTHON | Python repository URL | https://pypi.org | | SPCD_URL_ROCKY | Rocky repository URL | https://dl.rockylinux.org | | SPCD_URL_UBUNTU | Ubuntu repository URL | https://ubuntu.mirrors.ovh.net | ### Operating Systems containers #### Latest | os | https | up ca | python | graphviz | plantuml | shellcheck | shunit | shfmt | gource | ffmpeg | |:----------------|---|---|------------:|------:|-----------:|------:|------:|-----:|-----:|-------:| | Arch 20240818 | ☑ | ☑ | 3.12 | 12.0 | 1.2023.13 | 0.10 | 2.1.8 | 3.8 | 0.54 | 7.0.2 | | Alpine 3.20 | ☑ | ☐ | 3.12 | 9.0 | 1.2024.4 | 0.10 | 2.1.8 | 3.8 | 0.54 | 6.1.1 | | Fedora 40 | ☑ | ☑ | 3.12 → 3.13 | 9.0 | 1.2024.6 | 0.9 | 2.1.6 | 3.7 | 0.55 | 6.1.2 | | Debian Bookworm | ☐ | ☐ | 3.11 | 2.42 | 1.2020.2 | 0.9 | 2.1.8 | 3.6 | 0.54 | 5.1.6 | | OpenSUSE 15.6 | ☐ | ☑ | 3.6 → 3.12 | 2.48 | 1.2020.9 | 0.8 | 2.1.6 | 3.5 | 0.54 | 4.4.4 | | Ubuntu Noble | ☐ | ☐ | 3.12 | u2.42 | u1.2020.2 | u0.9 | 2.1.8 | u3.8 | 0.54 | 6.1.1 | | Alma / Rocky 9 | ☑ | ☑ | 3.9 → 3.12 | 2.44 | e1.2024.6 | e0.8 | | | | e5.1.4 | #### Previous | os | https | up ca | python | graphviz | plantuml | shellcheck | shunit | shfmt | gource | ffmpeg | |:----------------|---|---|------------:|------:|-----------:|------:|------:|-----:|-----:|-------:| | Alpine 3.19 | ☑ | ☐ | 3.11 | 9.0 | 1.2023.12 | 0.9 | 2.1.8 | 3.7 | 0.54 | 6.1.1 | | Fedora 39 | ☑ | ☑ | 3.12 → 3.13 | 8.1 | 1.2024.6 | 0.9 | 2.1.6 | 3.5 | 0.55 | 6.1.1 | | OpenSUSE 15.5 | ☐ | ☑ | 3.6 → 3.11 | 2.48 | 1.2020.9 | 0.8 | 2.1.6 | 3.5 | 0.54 | 4.4.4 | | Alma / Rocky 8 | ☑ | ☑ | 3.6 → 3.12 | 2.40 | e1.2024.6 | e0.6 | | | | | #### Older Python | os | https | up ca | python | graphviz | plantuml | shellcheck | shunit | shfmt | gource | ffmpeg | |:----------------|---|---|------------:|------:|-----------:|------:|------:|-----:|-----:|-------:| | Ubuntu Jammy | ☐ | ☐ | 3.10 | u2.42 | u1.2020.2 | u0.8 | 2.1.6 | u3.4 | 0.51 | 4.4.2 | | Debian Bullseye | ☐ | ☐ | 3.9 | 2.42 | 1.2020.2 | 0.7 | 2.1.6 | | 0.51 | 4.3.7 | --- ## Who ### By * [Marc Beninca](https://marc.beninca.link) ### For * People feeling the need to aim for consistency in the CI / CD universe --- ## Where ### Chat * [Discord](https://discord.com/channels/983145051985154108/1255894420092485652) * [IRC](ircs://irc.libera.chat/#spcd) ### Forge * [Repository](https://forge.rwx.work/rwx.work/spcd) * [RSS](https://forge.rwx.work/rwx.work/spcd.rss) * [Workflows](https://forge.rwx.work/rwx.work/spcd/actions) ### Deployment * [Site](https://spcd.rwx.work) --- ## When ### Tasks for a 1st stable release #### Documentation * build instead of readme * generate graphs * list behavior differences * Continuous Integration platforms * Operating Systems #### Shell * comment * git/pypi fallback for rwx/spcd * handle errors * packages * configure * apk * pacman * zypper * repositories * codecs * epel * fedora * opensuse * opensuse * disable & enable https * locales * persist * test * workspace variable #### Shell → Python * check file & variable sums * install system packages * epel * plantuml * shellcheck * graphviz * openssh * rsync * shfmt * shunit * python virtual environment * install system package * configure pip * venv * create * activate * query package * relay environment module name #### Python * check * .py * .sh * maximum line length * comment * detect ssh private key type * fit banner to text * implement substeps * list sys.path * set workflow templates * actions * includes * try git repo url variable first for shunit * test ### Task stack * automate versions fetching * gource, xvfb, xauth * handle openh264 repositories * link from workspace to actions root * rpm fusion * tex * translate to french * try to support nix * uv