Grafana API Key Rotation

Metadata

ID	Created	Author	Status
0001	20/12/2023	Jacob Woffenden	🟢 Active

We are unable to programatically rotate and consume a Grafana API key in Terraform, therefore we must rotate this externally.

To achieve this, we have created an AWS Lambda function to automatically recreate the key and upload it AWS Secrets Manager every 14 days.

Should this process fail, the following instructions can be used to rotate it manually:

1. Log in to the Grafana instance using the AWS access portal.

2. Navigate to the “API Keys” section under “Administration”.

3. Delete the expired API Key.

4. Generate a new API Key by clicking on the “Add API Key” button, and using the following details:

   Key name: observability-platform-automation

   Key role: Admin

   Time to live: 1209600

5. Copy the new API Key to your clipboard.

6. Login to the respective AWS environment, either observability-platform-development or observability-platform-production.

7. Open AWS Secrets Manager

8. Update grafana/api-key by selecting “Retrieve secret value”

9. Click “Edit”

10. Update the text with the new API Key.

11. Click “Save”

Repeat these steps for both dev and alpha environments.

This page was last reviewed on 4 April 2024. It needs to be reviewed again on 4 July 2024 by the page owner #observability-platform .