# Terraform: Eliminating phantom diffs using ignore_changes and replace_triggered_by

## The Problem: Write-Only Fields and Perpetual Diffs

A particularly common source of phantom diffs is Azure DevOps service endpoint resources. Passwords and API keys are write-only — the API accepts them on create/update but never returns them on read. Every `terraform plan` sees a gap in state and reports a diff.

```hcl
resource "azuredevops_serviceendpoint_nuget" "this" {
  project_id            = var.project_id
  service_endpoint_name = "nuget-feed"
  url                   = "https://artifacts.corp.example.com/repository/nuget-hosted/"
  username              = "deployer"
  password              = var.nuget_api_key
}
```

Every `terraform plan`, without fail:

```
# azuredevops_serviceendpoint_nuget.this will be updated in-place
~ resource "azuredevops_serviceendpoint_nuget" "this" {
      id                    = "..."
    ~ password              = (sensitive value)
      # (4 unchanged attributes hidden)
  }
```

## The Naive Fix (and Its Blind Spot)

The obvious answer is `ignore_changes`:

```hcl
resource "azuredevops_serviceendpoint_nuget" "this" {
  # ...
  password = var.nuget_api_key

  lifecycle {
    ignore_changes = [password]
  }
}
```

This kills the phantom diff, but creates a new problem: **when you actually rotate the API key**, Terraform silently ignores the change. You'd have to remember to manually run:

```bash
terraform apply -replace="azuredevops_serviceendpoint_nuget.this"
```

That's fragile, easy to forget, and doesn't scale across dozens of service connections.

## The Better Fix: `ignore_changes` + `replace_triggered_by`

Combine both lifecycle features to get the best of both worlds — no phantom diffs during normal operation, but automatic recreation when the secret actually changes.

```hcl
# A trigger resource that tracks the secret value.
# Only changes when the actual secret value changes.
resource "terraform_data" "nuget_api_key_version" {
  input = var.nuget_api_key
}

resource "azuredevops_serviceendpoint_nuget" "this" {
  project_id            = var.project_id
  service_endpoint_name = "nuget-feed"
  url                   = "https://artifacts.corp.example.com/repository/nuget-hosted/"
  username              = "deployer"
  password              = var.nuget_api_key

  lifecycle {
    ignore_changes = [password]
    replace_triggered_by = [
      terraform_data.nuget_api_key_version
    ]
  }
}
```

## How This Works

1. **`ignore_changes = [password]`** eliminates the phantom diff on every plan — Terraform stops comparing the write-only password field against state.

2. **`terraform_data.nuget_api_key_version`** stores the current value of the secret in state via its `input` attribute. It only reports a change when `var.nuget_api_key` *actually* changes.

3. **`replace_triggered_by`** watches the trigger resource. When the secret rotates and `terraform_data` detects a real change, it forces a full replacement of the service endpoint — ensuring the new credential is applied.

### Behavior Matrix

| Scenario | Behavior |
|---|---|
| Normal plan, no changes | Clean plan, zero diffs |
| Secret rotated in `var.nuget_api_key` | `terraform_data` changes → service endpoint is **replaced** with new password |
| Other attributes changed (URL, name) | Normal in-place update, no interference |
| `terraform plan` run repeatedly | Consistently clean — no phantom diff |

## Scaling the Pattern

This pattern works for any write-only field: Docker registry passwords, Maven deploy tokens, PyPI API keys, generic service connection secrets, etc.

```hcl
# --- Trigger resources (one per secret) ---

resource "terraform_data" "docker_password_version" {
  input = var.docker_password
}

resource "terraform_data" "maven_password_version" {
  input = var.maven_deploy_token
}

resource "terraform_data" "pypi_password_version" {
  input = var.pypi_api_token
}

resource "terraform_data" "npm_password_version" {
  input = var.npm_auth_token
}

# --- Service connections ---

resource "azuredevops_serviceendpoint_dockerregistry" "this" {
  project_id            = var.project_id
  service_endpoint_name = "docker-registry"
  docker_registry       = "https://registry.corp.example.com"
  docker_username       = "deployer"
  docker_password       = var.docker_password
  registry_type         = "Others"

  lifecycle {
    ignore_changes       = [docker_password]
    replace_triggered_by = [terraform_data.docker_password_version]
  }
}

resource "azuredevops_serviceendpoint_maven" "this" {
  project_id            = var.project_id
  service_endpoint_name = "maven-repo"
  url                   = "https://artifacts.corp.example.com/repository/maven-releases/"
  username              = "deployer"
  password              = var.maven_deploy_token

  lifecycle {
    ignore_changes       = [password]
    replace_triggered_by = [terraform_data.maven_password_version]
  }
}

resource "azuredevops_serviceendpoint_npm" "this" {
  project_id            = var.project_id
  service_endpoint_name = "npm-registry"
  url                   = "https://artifacts.corp.example.com/repository/npm-hosted/"
  access_token          = var.npm_auth_token

  lifecycle {
    ignore_changes       = [access_token]
    replace_triggered_by = [terraform_data.npm_password_version]
  }
}

resource "azuredevops_serviceendpoint_generic" "pypi" {
  project_id            = var.project_id
  service_endpoint_name = "pypi-repo"
  server_url            = "https://artifacts.corp.example.com/repository/pypi-hosted/"
  password              = var.pypi_api_token

  lifecycle {
    ignore_changes       = [password]
    replace_triggered_by = [terraform_data.pypi_password_version]
  }
}
```

## Modularizing the Pattern

If you manage many service connections through a module, you can encapsulate the pattern:

```hcl
# modules/service_connection_with_secret/main.tf

variable "project_id" {}
variable "endpoint_name" {}
variable "server_url" {}
variable "username" { default = "" }
variable "password" { sensitive = true }

resource "terraform_data" "password_version" {
  input = var.password
}

resource "azuredevops_serviceendpoint_generic" "this" {
  project_id            = var.project_id
  service_endpoint_name = var.endpoint_name
  server_url            = var.server_url
  username              = var.username
  password              = var.password

  lifecycle {
    ignore_changes       = [password]
    replace_triggered_by = [terraform_data.password_version]
  }
}

output "endpoint_id" {
  value = azuredevops_serviceendpoint_generic.this.id
}
```

Usage:

```hcl
module "pypi_connection" {
  source        = "./modules/service_connection_with_secret"
  project_id    = azuredevops_project.this.id
  endpoint_name = "pypi-repo"
  server_url    = "https://artifacts.corp.example.com/repository/pypi-hosted/"
  password      = var.pypi_api_token
}
```

## Important Caveats

### Replacement is Destructive

`replace_triggered_by` forces a **destroy + create**, not an in-place update. For service connections this is usually fine, but be aware that:

- The service endpoint ID changes on replacement
- Pipelines referencing the endpoint by ID (not name) may briefly fail during apply
- Any `azuredevops_resource_authorization` tied to the old endpoint ID needs to be recreated

If zero-downtime is critical, add `create_before_destroy`:

```hcl
  lifecycle {
    ignore_changes         = [password]
    replace_triggered_by   = [terraform_data.nuget_api_key_version]
    create_before_destroy  = true
  }
```

### Sensitive Values in `terraform_data`

The `terraform_data` resource stores its `input` in state. If your state backend isn't encrypted, the secret will be visible in the state file. Ensure you:

- Use an encrypted state backend (Azure Blob with encryption, S3 with SSE, etc.)
- Restrict state file access with proper IAM/RBAC policies
- Consider using `sensitive = true` on the variable to suppress console output

### When NOT to Use This Pattern

- **The provider correctly handles the field.** Some providers do read secrets back from the API — no phantom diff means no need for this pattern.
- **Terraform v1.11+ write-only attributes.** If your provider supports the new `write_only` attribute modifier, use that instead — it's the native solution to this problem.
- **You want in-place updates, not replacements.** If the resource is expensive to recreate (e.g., databases with data), the replacement semantics of `replace_triggered_by` may be too aggressive.

## Summary

| Approach | Phantom Diff? | Detects Secret Rotation? | Automation |
|---|---|---|---|
| No lifecycle rules | Yes (every plan) | Yes | Automatic |
| `ignore_changes` only | No | No | Manual `-replace` needed |
| `ignore_changes` + `replace_triggered_by` | No | Yes | Fully automatic |

The `ignore_changes` + `replace_triggered_by` pattern gives you clean plans *and* automatic secret rotation handling. It's the practical middle ground until write-only attributes become widespread across providers.