Top DevOps Tools — The Complete Ecosystem

🗺️ The DevOps Landscape

⚙️ CI/CD — Build & Deploy Pipelines

GitHub Actions — Full Docker + k8s Pipeline

# .github/workflows/ci-cd.yml
name: CI/CD Pipeline
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

env:
  REGISTRY: 123456789012.dkr.ecr.us-east-1.amazonaws.com
  IMAGE: 123456789012.dkr.ecr.us-east-1.amazonaws.com/myapp

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run tests
        run: docker compose run --rm app pytest

  build-push:
    needs: test
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'
    outputs:
      image-tag: ${{ steps.meta.outputs.tags }}
    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1

      - name: Login to ECR
        run: aws ecr get-login-password | docker login --username AWS --password-stdin ${{ env.REGISTRY }}

      - name: Build and push
        id: meta
        run: |
          TAG=${{ env.IMAGE }}:${{ github.sha }}
          docker build --cache-from ${{ env.IMAGE }}:latest -t $TAG -t ${{ env.IMAGE }}:latest .
          docker push $TAG
          docker push ${{ env.IMAGE }}:latest
          echo "tags=$TAG" >> $GITHUB_OUTPUT

  deploy:
    needs: build-push
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          repository: myorg/infra
          token: ${{ secrets.INFRA_TOKEN }}

      - name: Update image tag and push
        run: |
          sed -i "s|image: .*|image: ${{ needs.build-push.outputs.image-tag }}|" manifests/myapp/deployment.yaml
          git config user.email "ci@company.com" && git config user.name "CI"
          git commit -am "deploy: myapp ${{ github.sha }}" && git push

Jenkins — Declarative Pipeline

// Jenkinsfile
pipeline {
    agent any
    environment {
        REGISTRY = '123456789012.dkr.ecr.us-east-1.amazonaws.com'
        IMAGE    = "${REGISTRY}/myapp"
    }
    stages {
        stage('Test') {
            steps {
                sh 'docker compose run --rm app pytest'
            }
        }
        stage('Build') {
            steps {
                sh """
                    aws ecr get-login-password --region us-east-1 | \
                      docker login --username AWS --password-stdin ${REGISTRY}
                    docker build -t ${IMAGE}:${BUILD_NUMBER} -t ${IMAGE}:latest .
                    docker push ${IMAGE}:${BUILD_NUMBER}
                    docker push ${IMAGE}:latest
                """
            }
        }
        stage('Deploy') {
            when { branch 'main' }
            steps {
                sh """
                    sed -i "s|image: .*|image: ${IMAGE}:${BUILD_NUMBER}|" manifests/myapp/deployment.yaml
                    git commit -am "deploy: build ${BUILD_NUMBER}" && git push
                """
            }
        }
    }
    post {
        failure { slackSend color: 'danger', message: "Build failed: ${BUILD_URL}" }
        success { slackSend color: 'good',   message: "Deployed: ${IMAGE}:${BUILD_NUMBER}" }
    }
}

🏗️ Infrastructure as Code

Terraform — AWS ECS Service Example

# main.tf
terraform {
  required_providers {
    aws = { source = "hashicorp/aws", version = "~> 5.0" }
  }
  backend "s3" {
    bucket = "my-terraform-state"
    key    = "production/terraform.tfstate"
    region = "us-east-1"
  }
}

provider "aws" { region = "us-east-1" }

resource "aws_ecs_cluster" "main" {
  name = "production"
  setting {
    name  = "containerInsights"
    value = "enabled"
  }
}

resource "aws_ecs_service" "myapp" {
  name            = "myapp"
  cluster         = aws_ecs_cluster.main.id
  task_definition = aws_ecs_task_definition.myapp.arn
  desired_count   = 3
  launch_type     = "FARGATE"

  network_configuration {
    subnets          = var.private_subnet_ids
    security_groups  = [aws_security_group.app.id]
    assign_public_ip = false
  }

  load_balancer {
    target_group_arn = aws_lb_target_group.myapp.arn
    container_name   = "myapp"
    container_port   = 8080
  }
}

output "alb_dns" {
  value = aws_lb.main.dns_name
}

terraform init         # download providers
terraform plan         # preview changes
terraform apply        # apply changes
terraform destroy      # tear down everything
terraform output       # show output values
terraform state list   # list managed resources

⛵ Helm — Kubernetes Package Manager

Helm packages Kubernetes manifests into reusable, versioned "charts". Install complex apps with a single command.

# Add repositories
helm repo add stable        https://charts.helm.sh/stable
helm repo add bitnami       https://charts.bitnami.com/bitnami
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

# Search for charts
helm search repo postgres

# Install a chart
helm install my-postgres bitnami/postgresql \
  --namespace db --create-namespace \
  --set auth.postgresPassword=secret \
  --set primary.persistence.size=20Gi

# Install with values file
helm install monitoring prometheus-community/kube-prometheus-stack \
  --namespace monitoring --create-namespace \
  -f values.yaml

# Upgrade release
helm upgrade my-postgres bitnami/postgresql --set image.tag=16.0.0

# Rollback
helm rollback my-postgres 1

# List releases
helm list -A

# Uninstall
helm uninstall my-postgres -n db

# Create your own chart
helm create myapp
helm package myapp/
helm install myapp ./myapp-0.1.0.tgz

📊 Monitoring & Observability

AlertManager Config

# alertmanager.yml
global:
  resolve_timeout: 5m
  slack_api_url: 'https://hooks.slack.com/services/xxx/yyy/zzz'

route:
  group_by: ['alertname', 'namespace']
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 12h
  receiver: 'slack-critical'
  routes:
    - match: { severity: critical }
      receiver: pagerduty
    - match: { severity: warning }
      receiver: slack-critical

receivers:
  - name: 'slack-critical'
    slack_configs:
      - channel: '#alerts'
        send_resolved: true
        title: '{{ .GroupLabels.alertname }}'
        text: '{{ range .Alerts }}{{ .Annotations.description }}{{ end }}'

  - name: 'pagerduty'
    pagerduty_configs:
      - routing_key: 'your-pagerduty-key'

📋 Logging — Loki & ELK Stack

LogQL — Querying Loki

# Stream all logs from a namespace
{namespace="myapp"}

# Filter by container and search for error
{namespace="myapp", container="api"} |= "ERROR"

# Regex filter
{app="nginx"} |~ "POST .* 5[0-9][0-9]"

# Parse JSON logs and filter by field
{app="api"} | json | level="error" | response_time > 1000

# Count errors per minute
sum(rate({namespace="myapp"} |= "ERROR" [1m])) by (pod)

# Show last 100 lines
{namespace="myapp"} | limit 100

🔍 Distributed Tracing

🔒 Security Tools

Trivy — Scan Images in CI

# Install
curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sh

# Scan a Docker image
trivy image nginx:latest

# Scan and fail build on HIGH/CRITICAL CVEs
trivy image \
  --severity HIGH,CRITICAL \
  --exit-code 1 \
  myapp:latest

# Scan filesystem (Dockerfile, package.json, etc.)
trivy fs .

# Scan Kubernetes manifests
trivy k8s --report summary cluster

# Output JSON for SBOM
trivy image --format json -o sbom.json myapp:latest

Vault — Dynamic DB Credentials

# Start Vault (dev mode)
vault server -dev -dev-root-token-id="root"

# Enable database secrets engine
vault secrets enable database

# Configure PostgreSQL connection
vault write database/config/mydb \
  plugin_name=postgresql-database-plugin \
  connection_url="postgresql://{{username}}:{{password}}@db:5432/myapp" \
  allowed_roles="app-role" \
  username="vault" \
  password="vaultpass"

# Create role with TTL
vault write database/roles/app-role \
  db_name=mydb \
  creation_statements="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}'; GRANT SELECT,INSERT,UPDATE ON ALL TABLES IN SCHEMA public TO \"{{name}}\";" \
  default_ttl="1h" \
  max_ttl="24h"

# Get temp credentials (your app calls this)
vault read database/creds/app-role
# Returns: username=v-app-role-xyz  password=A1b2C3d4  lease_duration=1h

🕸️ Service Mesh

🔄 GitOps Tools

🗄️ Container Registries

✅ Tool Choices by Team Size