294 lines
8 KiB
Markdown
294 lines
8 KiB
Markdown
# Shell to Python Continuous Deployment
|
|
|
|
Get a common set of Python based CI / CD commands\
|
|
from various contexts of CA, DNS, CI and OS / OCI.
|
|
|
|
---
|
|
|
|
## Why
|
|
|
|
Project workflows can occur in:
|
|
* an online or offline context, dealing with custom CA & DNS servers
|
|
* various Continuous Integration platforms, with different workflow syntaxes
|
|
* specific Operating Systems, having different commands and package names
|
|
* specific containers for JavaScript based actions (ForgeJo, Gitea, GitHub)
|
|
|
|
What if there was a unified way to do so?
|
|
|
|
---
|
|
|
|
## How
|
|
|
|
Picture it…
|
|
|
|
### Group environment variables
|
|
|
|
* numbered
|
|
* Certificate Authority armored certificates
|
|
* Domain Name System servers IP addresses
|
|
* ref names for deployment contexts of feature, staging & release
|
|
* alternate locations of
|
|
* Operating Systems & Software packages
|
|
* Git repositories of
|
|
* this project
|
|
* its parent framework
|
|
* projects not available in some distributions
|
|
* shunit2
|
|
* SSH key & hosts for deployments
|
|
* a standalone POSIX shell script payload
|
|
|
|
### Workflow files
|
|
|
|
* calling the POSIX shell script payload as their very first step
|
|
* using a unified YAML syntax made of actions implemented in Python
|
|
|
|
### Integration & Deployment
|
|
|
|
#### Shell
|
|
|
|
* [X] setting up the chosen Operating System container
|
|
* [X] Domain Name System resolving
|
|
* [X] package manager configuration & repositories
|
|
* [X] Certificate Authorities trusting
|
|
* [X] Operating System updating
|
|
* [X] installing Git to clone
|
|
* [X] this project
|
|
* [X] its parent framework
|
|
* [X] installing both Python
|
|
* [X] system environment
|
|
* [X] virtual environment
|
|
* [X] generating a Python module to switch context
|
|
|
|
#### Python
|
|
|
|
* [X] setting up an OpenSSH client
|
|
* [X] known hosts
|
|
* [ ] private key
|
|
* [ ] and finally parsing commands from
|
|
* [X] system
|
|
* [ ] workflow
|
|
* [ ] to run available actions
|
|
* [ ] check project
|
|
* [ ] build project
|
|
* [X] browse workspace
|
|
* [ ] synchronize
|
|
|
|
---
|
|
|
|
## What
|
|
|
|
### Features
|
|
|
|
Handle project workflows in a unified way:
|
|
|
|
* whether the network infrastructure is
|
|
* [X] online
|
|
* [X] offline with custom
|
|
* [X] Certificate Authorities
|
|
* [X] Domain Name System servers
|
|
|
|
* whatever the Continuous Integration platform
|
|
* [X] ForgeJo
|
|
* [X] Gitea
|
|
* [X] GitHub
|
|
* [X] GitLab
|
|
* [ ] SourceHut
|
|
|
|
* whatever the Operating System container
|
|
* [X] Alma
|
|
* [X] 9
|
|
* [X] 8
|
|
* [X] Alpine
|
|
* [X] 3.20
|
|
* [X] 3.19
|
|
* [X] Arch
|
|
* [X] 20240818 (.0.255804)
|
|
* [X] 20240101 (.0.204074)
|
|
* [X] Debian
|
|
* [X] Bookworm (12)
|
|
* [ ] Bullseye (11)
|
|
* [X] Fedora
|
|
* [X] 40
|
|
* [X] 39
|
|
* [ ] OpenSUSE
|
|
* [ ] 15.6
|
|
* [ ] 15.5
|
|
* [X] Rocky
|
|
* [X] 9
|
|
* [X] 8
|
|
* [X] Ubuntu
|
|
* [X] Noble (24.04)
|
|
* [ ] Jammy (22.04)
|
|
|
|
### Environment variables
|
|
|
|
| Variable | Description | Default |
|
|
|:------------------|:-------------------------|:--------------------------------|
|
|
| SPCD_CA_n | Numbered CA certificates | |
|
|
| SPCD_DNS_n | Numbered name servers | 9.9.9.9 |
|
|
| SPCD_GIT_RWX | RWX Git repository | rwx |
|
|
| SPCD_GIT_SHUNIT | ShUnit Git repository | shunit2 |
|
|
| SPCD_GIT_SPCD | SPCD Git repository | spcd |
|
|
| SPCD_REF_FEATURE | Feature deployment ref | f |
|
|
| SPCD_REF_RELEASE | Release deployment ref | main |
|
|
| SPCD_REF_STAGING | Staging deployment ref | dev |
|
|
| SPCD_SSH_HOSTS | domain.tld ssh-type pub | |
|
|
| SPCD_SSH_KEY | SSH private key | |
|
|
| SPCD_TXT_LOCALE | Locale to use for text | en_US |
|
|
| SPCD_URL_ALMA | Alma repository URL | https://repo.almalinux.org |
|
|
| SPCD_URL_ALPINE | Alpine repository URL | https://dl-cdn.alpinelinux.org |
|
|
| SPCD_URL_ARCH | Arch repository URL | https://geo.mirror.pkgbuild.com |
|
|
| SPCD_URL_DEBIAN | Debian repository URL | https://deb.debian.org |
|
|
| SPCD_URL_EPEL | EPEL repository URL | https://dl.fedoraproject.org |
|
|
| SPCD_URL_FEDORA | Fedora repository URL | https://rpmfind.net |
|
|
| SPCD_URL_OPENSUSE | OpenSUSE repository URL | https://download.opensuse.org |
|
|
| SPCD_URL_PYTHON | Python repository URL | https://pypi.org |
|
|
| SPCD_URL_ROCKY | Rocky repository URL | https://dl.rockylinux.org |
|
|
| SPCD_URL_UBUNTU | Ubuntu repository URL | https://ubuntu.mirrors.ovh.net |
|
|
|
|
### Operating Systems containers
|
|
|
|
#### Latest
|
|
|
|
| os | https | updt-ca | python | graphviz | plantuml | shellcheck | shunit | shfmt |
|
|
|:----------------|---|---|------------:|------:|-----------:|------:|------:|-----:|
|
|
| Arch 20240818 | ☑ | ☑ | 3.12 | 12.0 | 1.2023.13 | 0.10 | 2.1.8 | 3.8 |
|
|
| Alpine 3.20 | ☑ | ☐ | 3.12 | 9.0 | 1.2024.4 | 0.10 | 2.1.8 | 3.8 |
|
|
| Fedora 40 | ☑ | ☑ | 3.12 → 3.13 | 9.0 | 1.2024.6 | 0.9 | 2.1.6 | 3.7 |
|
|
| Debian Bookworm | ☐ | ☐ | 3.11 | 2.42 | 1.2020.2 | 0.9 | 2.1.8 | 3.6 |
|
|
| OpenSUSE 15.6 | ☐ | ☑ | 3.6 → 3.12 | 2.48 | 1.2020.9 | 0.8 | 2.1.6 | 3.5 |
|
|
| Ubuntu Noble | ☐ | ☐ | 3.12 | u2.42 | u1.2020.2 | u0.9 | 2.1.8 | u3.8 |
|
|
| Alma / Rocky 9 | ☑ | ☑ | 3.9 → 3.12 | 2.44 | e1.2024.6 | e0.8 | | |
|
|
|
|
#### Previous
|
|
|
|
| os | https | updt-ca | python | graphviz | plantuml | shellcheck | shunit | shfmt |
|
|
|:----------------|---|---|------------:|------:|-----------:|------:|------:|-----:|
|
|
| Alpine 3.19 | ☑ | ☐ | 3.11 | 9.0 | 1.2023.12 | 0.9 | 2.1.8 | 3.7 |
|
|
| Fedora 39 | ☑ | ☑ | 3.12 → 3.13 | 8.1 | 1.2024.6 | 0.9 | 2.1.6 | 3.5 |
|
|
| OpenSUSE 15.5 | ☐ | ☑ | 3.6 → 3.11 | 2.48 | 1.2020.9 | 0.8 | 2.1.6 | 3.5 |
|
|
| Alma / Rocky 8 | ☑ | ☑ | 3.6 → 3.12 | 2.40 | e1.2024.6 | e0.6 | | |
|
|
|
|
#### Older Python
|
|
|
|
| os | https | updt-ca | python | graphviz | plantuml | shellcheck | shunit | shfmt |
|
|
|:----------------|---|---|------------:|------:|-----------:|------:|------:|-----:|
|
|
| Ubuntu Jammy | ☐ | ☐ | 3.10 | u2.42 | u1.2020.2 | u0.8 | 2.1.6 | u3.4 |
|
|
| Debian Bullseye | ☐ | ☐ | 3.9 | 2.42 | 1.2020.2 | 0.7 | 2.1.6 | |
|
|
|
|
---
|
|
|
|
## Who
|
|
|
|
### By
|
|
|
|
* [Marc Beninca](https://marc.beninca.link)
|
|
|
|
### For
|
|
|
|
* People feeling the need to aim for consistency in the CI / CD universe
|
|
|
|
---
|
|
|
|
## Where
|
|
|
|
### Chat
|
|
|
|
* [Discord](https://discord.com/channels/983145051985154108/1255894420092485652)
|
|
* [IRC](ircs://irc.libera.chat/#spcd)
|
|
|
|
### Forge
|
|
|
|
* [Repository](https://forge.rwx.work/rwx.work/spcd)
|
|
* [RSS](https://forge.rwx.work/rwx.work/spcd.rss)
|
|
* [Workflows](https://forge.rwx.work/rwx.work/spcd/actions)
|
|
|
|
### Deployment
|
|
|
|
* [Site](https://spcd.rwx.work)
|
|
|
|
---
|
|
|
|
## When
|
|
|
|
### Tasks for a 1st stable release
|
|
|
|
#### Documentation
|
|
|
|
* build instead of readme
|
|
* generate graphs
|
|
* list CI behavior differences
|
|
|
|
#### Shell
|
|
|
|
* comment
|
|
* display locales
|
|
* default
|
|
* available
|
|
* effective
|
|
* selected
|
|
* available
|
|
* effective
|
|
* factorize
|
|
* locales
|
|
* handle errors
|
|
* packages
|
|
* configure
|
|
* apk
|
|
* pacman
|
|
* zypper
|
|
* repositories
|
|
* codecs
|
|
* epel
|
|
* fedora
|
|
* opensuse
|
|
* opensuse
|
|
* disable & enable https
|
|
* persist locales
|
|
* test
|
|
|
|
#### Shell → Python
|
|
|
|
* check file & variable sums
|
|
* install system packages
|
|
* epel
|
|
* plantuml
|
|
* shellcheck
|
|
* graphviz
|
|
* openssh
|
|
* rsync
|
|
* shfmt
|
|
* shunit
|
|
* python virtual environment
|
|
* install system package
|
|
* configure pip
|
|
* venv
|
|
* create
|
|
* activate
|
|
* query package
|
|
* relay environment module name
|
|
|
|
#### Python
|
|
|
|
* check
|
|
* .py
|
|
* .sh
|
|
* maximum line length
|
|
* comment
|
|
* detect ssh private key type
|
|
* fit banner to text
|
|
* implement substeps
|
|
* list sys.path
|
|
* set workflow templates
|
|
* actions
|
|
* includes
|
|
* try git repo url variable first for shunit
|
|
* test
|
|
|
|
### Task stack
|
|
|
|
* automate versions fetching
|
|
* handle openh264 repositories
|
|
* tex
|
|
* translate to french
|
|
* try to support nix
|