spcd/readme.md

# Shell to Python Continuous Deployment

Get a common set of Python based CI / CD commands\
from various contexts of CA, DNS, CI and OS / OCI.

---

## Why

Project workflows can occur in:
* an online or offline context, dealing with custom CA & DNS servers
* various Continuous Integration platforms, with different workflow syntaxes
* specific Operating Systems, having different commands and package names
* specific containers for JavaScript based actions (ForgeJo, Gitea, GitHub)

What if there was a unified way to do so?

---

## How

Picture it…

### Group environment variables

* numbered
  * Certificate Authority armored certificates
  * Domain Name System servers IP addresses
* ref names for deployment contexts of feature, staging & release
* alternate locations of
  * Operating Systems & Software packages
  * Git repositories of
    * this project
    * its parent framework
    * projects not available in some distributions
      * shunit2
* SSH key & hosts for deployments
* a standalone POSIX shell script payload

### Workflow files

* calling the POSIX shell script payload as their very first step
* using a unified YAML syntax made of actions implemented in Python

### Integration & Deployment

#### Shell

* [X] setting up the chosen Operating System container
  * [X] Domain Name System resolving
  * [X] package manager configuration & repositories
  * [X] Certificate Authorities trusting
  * [X] Operating System updating
* [X] installing Git to clone
  * [X] this project
  * [X] its parent framework
* [X] installing both Python
  * [X] system environment
  * [X] virtual environment
* [X] generating a Python module to switch context

#### Python

* [X] setting up an OpenSSH client
  * [X] known hosts
  * [ ] private key
* [ ] and finally parsing commands from
  * [X] system
  * [ ] workflow
* [ ] to run available actions
  * [ ] check project
  * [ ] build project
  * [X] browse workspace
  * [ ] synchronize

---

## What

### Features

Handle project workflows in a unified way:

* whether the network infrastructure is
  * [X] online
  * [X] offline with custom
    * [X] Certificate Authorities
    * [X] Domain Name System servers

* whatever the Continuous Integration platform
  * [X] ForgeJo
  * [X] Gitea
  * [X] GitHub
  * [X] GitLab
  * [ ] SourceHut

* whatever the Operating System container
  * [X] Alma
    * [X] 9
    * [X] 8
  * [X] Alpine
    * [X] 3.20
    * [X] 3.19
  * [X] Arch
    * [X] 20240818 (.0.255804)
    * [X] 20240101 (.0.204074)
  * [X] Debian
    * [X] Bookworm (12)
    * [ ] Bullseye (11)
  * [X] Fedora
    * [X] 40
    * [X] 39
  * [ ] OpenSUSE
    * [ ] 15.6
    * [ ] 15.5
  * [X] Rocky
    * [X] 9
    * [X] 8
  * [X] Ubuntu
    * [X] Noble (24.04)
    * [ ] Jammy (22.04)

### Environment variables

| Variable          | Description              | Default                         |
|:------------------|:-------------------------|:--------------------------------|
| SPCD_CA_n         | Numbered CA certificates |                                 |
| SPCD_DNS_n        | Numbered name servers    | 9.9.9.9                         |
| SPCD_GIT_RWX      | RWX Git repository       | rwx                             |
| SPCD_GIT_SHUNIT   | ShUnit Git repository    | shunit2                         |
| SPCD_GIT_SPCD     | SPCD Git repository      | spcd                            |
| SPCD_REF_FEATURE  | Feature deployment ref   | f                               |
| SPCD_REF_RELEASE  | Release deployment ref   | main                            |
| SPCD_REF_STAGING  | Staging deployment ref   | dev                             |
| SPCD_SSH_HOSTS    | domain.tld ssh-type pub  |                                 |
| SPCD_SSH_KEY      | SSH private key          |                                 |
| SPCD_TXT_LOCALE   | Locale to use for text   | en_US                           |
| SPCD_URL_ALMA     | Alma repository URL      | https://repo.almalinux.org      |
| SPCD_URL_ALPINE   | Alpine repository URL    | https://dl-cdn.alpinelinux.org  |
| SPCD_URL_ARCH     | Arch repository URL      | https://geo.mirror.pkgbuild.com |
| SPCD_URL_DEBIAN   | Debian repository URL    | https://deb.debian.org          |
| SPCD_URL_EPEL     | EPEL repository URL      | https://dl.fedoraproject.org    |
| SPCD_URL_FEDORA   | Fedora repository URL    | https://rpmfind.net             |
| SPCD_URL_OPENSUSE | OpenSUSE repository URL  | https://download.opensuse.org   |
| SPCD_URL_PYTHON   | Python repository URL    | https://pypi.org                |
| SPCD_URL_ROCKY    | Rocky repository URL     | https://dl.rockylinux.org       |
| SPCD_URL_UBUNTU   | Ubuntu repository URL    | https://ubuntu.mirrors.ovh.net  |

### Operating Systems containers

#### Latest

| os | https | updt-ca | python | graphviz | plantuml | shellcheck | shunit | shfmt |
|:----------------|---|---|------------:|------:|-----------:|------:|------:|-----:|
| Arch 20240818   | ☑ | ☑ |        3.12 | 12.0  |  1.2023.13 |  0.10 | 2.1.8 |  3.8 |
| Alpine 3.20     | ☑ | ☐ |        3.12 |  9.0  |  1.2024.4  |  0.10 | 2.1.8 |  3.8 |
| Fedora 40       | ☑ | ☑ | 3.12 → 3.13 |  9.0  |  1.2024.6  |  0.9  | 2.1.6 |  3.7 |
| Debian Bookworm | ☐ | ☐ |        3.11 |  2.42 |  1.2020.2  |  0.9  | 2.1.8 |  3.6 |
| OpenSUSE 15.6   | ☐ | ☑ | 3.6  → 3.12 |  2.48 |  1.2020.9  |  0.8  | 2.1.6 |  3.5 |
| Ubuntu Noble    | ☐ | ☐ |        3.12 | u2.42 | u1.2020.2  | u0.9  | 2.1.8 | u3.8 |
| Alma / Rocky 9  | ☑ | ☑ | 3.9  → 3.12 |  2.44 | e1.2024.6  | e0.8  |       |      |

#### Previous

| os | https | updt-ca | python | graphviz | plantuml | shellcheck | shunit | shfmt |
|:----------------|---|---|------------:|------:|-----------:|------:|------:|-----:|
| Alpine 3.19     | ☑ | ☐ |        3.11 |  9.0  |  1.2023.12 |  0.9  | 2.1.8 |  3.7 |
| Fedora 39       | ☑ | ☑ | 3.12 → 3.13 |  8.1  |  1.2024.6  |  0.9  | 2.1.6 |  3.5 |
| OpenSUSE 15.5   | ☐ | ☑ | 3.6  → 3.11 |  2.48 |  1.2020.9  |  0.8  | 2.1.6 |  3.5 |
| Alma / Rocky 8  | ☑ | ☑ | 3.6  → 3.12 |  2.40 | e1.2024.6  | e0.6  |       |      |

#### Older Python

| os | https | updt-ca | python | graphviz | plantuml | shellcheck | shunit | shfmt |
|:----------------|---|---|------------:|------:|-----------:|------:|------:|-----:|
| Ubuntu Jammy    | ☐ | ☐ |        3.10 | u2.42 | u1.2020.2  | u0.8  | 2.1.6 | u3.4 |
| Debian Bullseye | ☐ | ☐ |        3.9  |  2.42 |  1.2020.2  |  0.7  | 2.1.6 |      |

---

## Who

### By

* [Marc Beninca](https://marc.beninca.link)

### For

* People feeling the need to aim for consistency in the CI / CD universe

---

## Where

### Chat

* [Discord](https://discord.com/channels/983145051985154108/1255894420092485652)
* [IRC](ircs://irc.libera.chat/#spcd)

### Forge

* [Repository](https://forge.rwx.work/rwx.work/spcd)
* [RSS](https://forge.rwx.work/rwx.work/spcd.rss)
* [Workflows](https://forge.rwx.work/rwx.work/spcd/actions)

### Deployment

* [Site](https://spcd.rwx.work)

---

## When

### Tasks for a 1st stable release

#### Documentation

* build instead of readme
* generate graphs
* list CI behavior differences

#### Shell

* comment
* display locales
  * default
    * effective
  * selected
    * effective
* handle errors
* packages
  * configure
    * apk
    * pacman
    * zypper
  * repositories
    * codecs
      * epel
      * fedora
      * opensuse
    * opensuse
      * disable & enable https
* persist locales
* test

#### Shell → Python

* check file & variable sums
* install system packages
  * epel
    * plantuml
    * shellcheck
  * graphviz
  * openssh
  * rsync
  * shfmt
  * shunit
* python virtual environment
  * install system package
  * configure pip
  * venv
    * create
    * activate
* query package
* relay environment module name

#### Python

* check
  * .py
  * .sh
    * maximum line length
* comment
* detect ssh private key type
* fit banner to text
* implement substeps
* list sys.path
* set workflow templates
  * actions
  * includes
* try git repo url variable first for shunit
* test

### Task stack

* automate versions fetching
* handle openh264 repositories
* tex
* translate to french
* try to support nix