Skip to content

Trade Study: GitOps

Conducted by: Tess and Jeff

Backlog Work Item: #21672

Decision Makers: Wallace, whole team

Overview

For Memory, we will be creating a cloud native application with infrastructure as code. We will use GitOps for Continuous Deployment through pull requests infrastructure changes to be reflected.

Overall, between our two options, one is more simple and targeted in a way that we believe would meet the requirements for this project. The other does the same, with additional features that may or may not be worth the extra configuration and setup.

Evaluation Criteria

  1. Repo style: mono versus multi
  2. Policy Enforcement
  3. Deployment Methods
  4. Deployment Monitoring
  5. Admission Control
  6. Azure Documentation availability
  7. Maintainability
  8. Maturity
  9. User Interface

Solutions

Flux

Flux is a tool created by Waveworks and is built on top of Kubernetes' API extension system, supports multi-tenancy, and integrates seamlessly with popular tools like Prometheus.

Flux Acceptance Criteria Evaluation

  1. Repo style: mono versus multi
  2. Flux supports both as of v2
  3. Policy Enforcement
  4. Azure Policy is in Preview
  5. Deployment Methods
  6. Define a Helm release using Helm Controllers
  7. Kustomization describes deployments
  8. Deployment Monitoring
  9. Flux works with Prometheus for deployment monitoring as well as Grafana dashboards
  10. Admission Control
  11. Flux uses RBAC from Kubernetes to lock down sync permissions.
  12. Uses the service account to access image pull secrets
  13. Azure Documentation availability
  14. Great, better when using Helm Operators
  15. Maintainability
  16. Manage via YAML files in git repo
  17. Maturity
  18. v2 is published under Apache license in GitHub, it works with Helm v3, and has PR commits from as recently as today
  19. 945 stars, 94 forks
  20. User Interface
  21. CLI, the simplest lightweight option

Other features to call out (see more on website)

  • Flux only supports Pull-based deployments which means it must be paired with an operator
  • Flux can send notifications and receive webhooks for syncing
  • Health assessments
  • Dependency management
  • Automatic deployment
  • Garbage collection
  • Deploy on commit

Variations

Controllers

Both Controller options are optional.

The Helm Controller additionally fetches helm artifacts to publish, see below diagram.

The Kustomize Controller manages state and continuous deployment.

We will not decide between the controller to use here, as that's a separate trade study, however we will note that Helm is more widely documented within Flux documentation.

Flux v1

Flux v1 is only in maintenance mode and should not be used anymore. So this section does not consider the v1 option a valid option.

GitOps Toolkit

Flux v2 is built on top of the GitOps Toolkit, however we do not evaluate using the GitOps Toolkit alone as that is for when you want to make your own CD system, which is not what we want.

ArgoCD with Helm Charts

ArgoCD is a declarative, GitOps-based Continuous Delivery (CD) tool for Kubernetes.

ArgoCD with Helm Acceptance Criteria Evaluation

  1. Repo style: mono versus multi
  2. ArgoCD supports both
  3. Policy Enforcement
  4. Azure Policy is in Preview
  5. Deployment Methods
  6. Deploy with Helm Chart
  7. Use Kustomize to apply some post-rendering to the Helm release templates
  8. Deployment Monitoring
  9. Argo CD expose two sets of Prometheus metrics (application metrics and API server metrics) for deployment monitoring.
  10. Admission Control
  11. ArgoCD use RBAC feature. RBAC requires SSO configuration or one or more local users setup. Once SSO or local users are configured, additional RBAC roles can be defined
  12. Argo CD does not have its own user management system and has only one built-in user admin. The admin user is a superuser, and it has unrestricted access to the system
  13. Authorization is handled via JWT tokens and checking group claims in them
  14. Azure Documentation availability
  15. Argo has documentation on Azure AD
  16. Maturity
  17. Has PR commits from as recently as today
  18. 5,000 stars, 1,100 forks
  19. Maintainability
  20. Can use GitOps to manage it
  21. User Interface
  22. ArgoCD has a GUI and can be used across clusters

Other features to call out (see more on website)

  • ArgoCD support both pull model and push model for continuous delivery
  • Argo can send notifications, but you need a separate tool for it
  • Argo can receive webhooks
  • Health assessments
  • Potentially much more useful multi-tenancy tools. Manages multiple projects, maps them to teams, etc.
  • SSO Integration
  • Garbage collection

Architecture

Results

This section should contain a table that has each solution rated against each of the evaluation criteria:

Solution Repo style Policy Enforcement Deployment Methods Deployment Monitoring Admission Control Azure Doc Maintainability Maturity UI
Flux mono, multi Azure Policy, preview Helm, Kustomize Prometheus, Grafana RBAC Yes on Azure YAML in git repo 945 stars, 94 forks, currently maintained CLI
ArgoCD mono, multi Azure Policy, preview Helm, Kustomize, KSonnet, ... Prometheus, Grafana RBAC Only in their own docs manifests in git repo 5,000 stars, 1,100 forks GUI, multiple clusters in same GUI

Decision

ArgoCD is more feature rich, will support more scenarios, and will be a better tool to put in our tool belts. So we have decided at this point to go with ArgoCD.

References

  1. GitOps
  2. Enforcement
  3. Monitoring
  4. Policies
  5. Deployment
  6. Push with ArgoCD in Azure DevOps