TELCODOCS-2000 documenting the AGOF 2

content/learn/vp_agof.adoc

@@ -0,0 +1,313 @@
+---
+menu:
+  learn:
+    parent: Validated patterns frameworks
+title: Ansible GitOps Framework 
+weight: 23
+aliases: /ocp-framework/agof/
+---
+
+:toc:
+:imagesdir: /images
+:_content-type: ASSEMBLY
+include::modules/comm-attributes.adoc[]
+
+== About the Ansible GitOps framework (AGOF)for validated patternS
+
+The link:/patterns/agof/[Ansible GitOps Framework] provides an extensible framework to do GitOps with https://docs.ansible.com/platform.html[Ansible Automation Platform] (AAP), and as such provides useful facilities for developing patterns (community and validated) that function with Ansible Automation Platform as the GitOps engine.
+
+AGOF comes with code to install VMs in AWS, if desired, or else it can work with previously provisioned VMs, or a functional AAP Controller endpoint
+
+When there is no access to OpenShift Container Platform, AGOF provides a standalone solution for deploying and managing validated patterns. This framework leverages GitOps principles without relying on OpenShift.
+
+Ansible Automation Platform includes automation controller, a web-based UI interface allows users to define, operate, scale, and delegate automation across their enterprise.
+
+== Pattern directories tour
+
+Examining any of the existing patterns reveals the important organizational part of the validated patterns framework. Let's take a look at a couple of the existing validated patterns: Multicluster GitOps and Industrial Edge.
+
+===  Multicloud GitOps
+
+The Multicloud GitOps approach enables centralized management of multiple cloud deployments across both public and private clouds, including workloads and the secure handling of secrets across environments. This approach is built on patterns that consist of two key components: a "common" element, which serves as a foundational framework shared by nearly all patterns, and a pattern-specific element that builds on the common framework with tailored content. This section focuses on the standardized directory structure used in Multicloud GitOps repositories for validated patterns in multicloud setups. By following the well-structured directory layout illustrated here, teams can streamline deployment processes across clouds, reduce configuration drift, enhance automation, and maintain a single source of truth for both infrastructure and application code.
+
+[source,text]
+----
+~/g/multicloud-gitops on main ◦ tree -L 2
+.
+├── ansible
+├── ansible.cfg
+├── charts
+│   └── all
+│   └── region
+├── common
+│   ├── acm
+|   ├── ansible
+|   ├── Changes.md
+│   ├── clustergroup
+│   ├── common -> .
+│   ├── examples
+│   ├── golang-external-secrets
+│   ├── hashicorp-vault
+│   ├── letsencrypt
+|   ├── LICENSE
+|   ├── Makefile
+|   ├── operator-install
+|   ├── README.md
+│   ├── reference-output.yaml
+│   ├── scripts
+│   ├── tests
+│   └── values-global.yaml
+├── LICENSE
+├── Makefile
+├── overrides
+│   ├── values-AWS.yaml
+│   └── values-IBMCloud.yaml
+├── pattern.sh -> ./common/scripts/pattern-util.sh
+├── README.md
+├── tests
+├── values-global.yaml
+├── values-global-one.yaml
+├── values-hub.yaml
+├── values-secret-multicloud-gitops.yaml
+└── values-secret.yaml.template
+
+20 directories, 77 files
+----
+
+First we notice some subdirectories: charts and common, along with `values-` yaml files.
+
+===  Industrial edge
+
+[source,text]
+----
+~/g/industrial-edge on stable-2.0 ◦ tree -L 2
+.
+├── ansible
+├── ansible.cfg
+├── Changes.md
+├── charts
+│   ├── datacenter
+│   └─  factory
+|   └── secrets
+├── common
+│   ├── acm
+|   ├── ansible
+|   ├── Changes.md
+│   ├── clustergroup
+│   ├── common -> .
+│   ├── examples
+│   ├── golang-external-secrets
+│   ├── hashicorp-vault
+│   ├── letsencrypt
+|   ├── LICENSE
+|   ├── Makefile
+|   ├── operator-install
+│   ├── scripts
+│   ├── tests 
+│   └── values-global.yaml
+├── docs
+│   ├── images
+│   └── old-deployment-map.txt
+├── images
+│   ├── import-cluster.png
+│   ├── import-with-kubeconfig.png
+│   └── launch-acm-console.png
+├── LICENSE
+├── Makefile
+├── overrides
+|   ├── values-prod-imagedata.yaml
+│   └── values-test-imagedata.yaml
+├── README.md
+|── pattern.sh -> ./common/scripts/pattern-util.sh
+├── scripts
+│   ├── secret.sh
+│   └── sleep-seed.sh
+├── SUPPORT_AGREEMENT.md
+├── tests
+├── values-datacenter.yaml
+├── values-factory.yaml
+├── values-global.yaml
+├── values-hub.yaml -> values-datacenter.yaml
+└── values-secret.yaml.template
+
+25 directories, 98 files
+----
+
+We see the same or similar files in the both patterns directories. 
+
+== The `common` directory
+
+The core components that make the Validated Patterns framework are contained in the common repository. These include:
+
+* OpenShift GitOps configuration
+* Supports our clusterGroup and GitOps policies
+* Validated Pattern framework build scripts and Makefiles
+* Secrets Management with HashiCorp Vault
+* Operator CRDs and other assets
+* Various utility scripts
+
+The common repository contains all the shared manifests for the Validated Patterns Framework. These components are configured to work together within the GitOps framework. Instead of duplicating configurations across patterns, shared technologies are centralized in this common directory. Pattern-specific post-deployment configurations, if needed, should be added to the Helm charts in the charts directory. Typically, you won't need to modify the common directory unless working on the framework itself.
+
+=== Breakdown of common repository 
+
+This table details the key components of the common repository: 
+
+[cols="2,2", options="header"]
+|===
+| Component | Description
+
+| acm  | Contains the helm charts which contains policies and is used to configure the deployment of the Advance Cluster Manager.
+| ansible  | This directory contains the ansible roles and modules that support the secrets management for a pattern.
+| clustergroup | Contains the helm chart used to create namespace, subscriptions, projects, and applications described in the values files. This is the seed for all patterns.
+| golang-external-secrets  | Helm chart for External Secrets Operator that integrates external secret management systems like AWS Secrets Manager, HashiCorp Vault, Google Secrets Manager, Azure Key Vault, IBM Cloud Secrets Manager, Akeyless.
+| hashicorp-vault  | Contains the helm chart for HashiCorp Vault. 
+|operator-install  | Contains the helm chart used by the Validated Patterns Operator to create the openshift-gitops component and create the initial ArgoCD applications for the Validated Pattern.
+| scripts  | Directory which contains utility scripts used by the Validated Pattern Framework.
+|===
+
+== The `charts` directory
+
+This is where validated patterns keep the helm charts for a pattern. The helm charts are used to deploy and manage the various components of the applications deployed at a site. By convention, the charts are broken out by site location. You may see `datacenter`, `hub`, `factory`, or other site names in there.
+
+[NOTE]
+====
+The site naming convention is flexible, allowing users to modify it to suit their environment. 
+====
+
+Each site has sub-directories based on application or library component groupings.
+
+From https://helm.sh/docs/chart_template_guide/getting_started/[Helm documentation:]
+_Application charts_ are a collection of templates that can be packaged into versioned archives to be deployed.
+
+_Library charts_ provide useful utilities or functions for the chart developer. They're included as a dependency of application charts to inject those utilities and functions into the rendering
+pipeline. Library charts do not define any templates and therefore cannot be deployed.
+
+These groupings are used by OpenShift GitOps to deploy into the cluster. The configurations for each of the components inside an application are synced every three minutes by OpenShift GitOps to make sure that the site is up to date. The configuration can also be synced manually if you do not wish to wait up to three minutes.
+
+[source,text]
+----
+.
+├── datacenter
+│   ├── external-secrets
+│   ├── manuela-data-lake
+│   ├── manuela-tst
+│   ├── opendatahub
+│   └── pipelines
+├── factory
+│   └── manuela-stormshift
+└── secrets
+    └── pipeline-setup
+----
+The configuration YAML for each component of the application is stored in the templates subdirectory. 
+
+[source,text]
+----
+.
+├── external-secrets
+│   ├── Chart.yaml
+│   ├── templates
+│   └── values.yaml
+├── manuela-data-lake
+│   ├── Chart.yaml
+│   ├── templates
+│   └── values.yaml
+├── manuela-tst
+│   ├── Chart.yaml
+│   ├── templates
+│   └── values.yaml
+├── opendatahub
+│   ├── Chart.yaml
+│   ├── templates
+│   └── values.yaml
+└── pipelines
+    ├── Chart.yaml
+    ├── extra
+    ├── images
+    ├── README.md
+    ├── templates
+    └── values.yaml
+----
+== The `scripts` directory
+
+In some cases, an Operator or Helm chart may require additional configuration. When extra code is needed for deployment, it should be placed in the scripts directory. Typically, consumers of a validated pattern won't interact directly with these scripts, as they are executed by the existing automation (for example through the Makefile or OpenShift GitOps). If extra adjustments are required for your application, place the scripts here and run them through automation. The scripts directory should generally be treated as off-limits unless you're modifying the framework itself.
+
+== Applications and `values-` files
+
+Helm uses `values.yaml` files to pass variables into charts. Variables in the `values.yaml` file can be overridden in the following ways:
+
+. By a `values.yaml` file in the parent directory
+. By a values file passed into the `helm <install/upgrade>` command using `-f`
+. By specifying an override individual value in the the `helm` command with `--set`
+
+For more information on values files and their usage see the https://helm.sh/docs/chart_template_guide/values_files/[values files section] of the Helm documentation.
+
+This section is meant as an introduction to the `values-` files that the framework uses to override values in the chart templates. In the Getting Started pages there will be more specific usage details.
+
+=== There are three types of `value-` files.
+
+. *`values-global.yaml`*:
+This is used to set variables for helm charts across the pattern. It contains the name of the pattern and sets some other variables for artifacts like, image registry, Git repositories, GitOps syncPolicy etc.
+. *`values-<site>.yaml`*:
+Each specific site requires information regarding what applications and subscriptions are required for that site. This file contains a list of namespaces, applications, subscriptions, the operator versions etc. for that site.
+. *`values-secret.yaml.template`*:
+All patterns require some secrets for artifacts included in the pattern. For example credentials for GitHub, AWS, or Quay.io. The framework provides a safe way to load those secrets into a vault for consumption by the pattern. This template file can be copied to your home directory, the secret values applied, and the validated pattern will go look for `values-secrets.yaml` in your home directory. *Do not leave a `values-secrets.yaml` file in your cloned git directory or it may end up in your (often public) Git repository, like GitHub.*
++
+[NOTE]
+====
+This file has nothing to do with helm and can be copied either to your home directory or to the `~/.config/validatedpatterns/` folder. The naming should be `values-secret-<pattern-name>.yaml`. Ideally it should be encrypted with the ansible-vault.
+====
+
+=== Values files can have some overrides.
+
+. Version overrides can be used to set specific values for OCP versions. For example *`values-hub-4.16.yaml`* allows you to tweak a specific value for OCP 4.16 on the Hub cluster.
+. Version overrides can be used to set specific values for specific cloud providers. For example *`values-AWS.yaml`* would allow you to tweak a specific value for all cluster groups deployed on AWS.
+
+=== Other combination examples include:
+
+. *`values-hub-Azure.yaml`* only apply this Azure tweak on the hub cluster.
+. *`values-4.16.yaml`* apply these OCP 4.16 tweaks to all cluster groups in this pattern.
+
+Current supported cloud providers include *`AWS`*, *`Azure`*, *`GCP`*, and *Nutanix*.
+
+== Environment values and Helm
+
+The purpose of the values files is to leverage Helm's templating capabilities, allowing you to dynamically substitute values into your charts. This makes the pattern very portable.
+
+The following `messaging-route.yaml` example shows how the AMQ messaging service is using values set in the `values-global.yaml` file for Industrial Edge.
+
+[source,yaml]
+----
+apiVersion: route.openshift.io/v1
+kind: Route
+metadata:
+  labels:
+    app: messaging
+  name: messaging
+spec:
+  host: messaging-manuela-tst-all.apps.{{ .Values.global.datacenter.clustername }}.{{ .Values.global.datacenter.domain }}
+  port:
+    targetPort: 3000-tcp
+  to:
+    kind: Service
+    name: messaging
+    weight: 100
+  wildcardPolicy: None
+----
+
+The values in the `values-global.yaml` will be substituted when the YAML is applied to the cluster.
+
+[source,yaml]
+----
+global:
+  pattern: industrial-edge
+
+...
+
+  datacenter:
+    clustername: ipbabble-dc
+    domain: blueprints.rhecoeng.com
+
+  edge:
+    clustername: ipbabble-f1
+    domain: blueprints.rhecoeng.com
+----