[GSoC-273] Fixing the github action Unmanaged Service Account Keys

.github/workflows/README.md

@@ -556,4 +556,3 @@ PostCommit Jobs run in a schedule against master branch and generally do not get
 | [ Modify the GCP User Roles according to the infra/users.yml file ](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_UsersPermissions.yml) | N/A | [![.github/workflows/beam_Infrastructure_UsersPermissions.yml](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_UsersPermissions.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_UsersPermissions.yml?query=event%3Aschedule) |
 | [ Service Account Keys Management ](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_ServiceAccountKeys.yml) | N/A | [![.github/workflows/beam_Infrastructure_ServiceAccountKeys.yml](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_ServiceAccountKeys.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_ServiceAccountKeys.yml?query=event%3Aschedule) |
 | [ Upgrade GCP Libraries BOM ](https://github.com/apache/beam/actions/workflows/beam_Upgrade_GCP_BOM.yml) | N/A | [![.github/workflows/beam_Upgrade_GCP_BOM.yml](https://github.com/apache/beam/actions/workflows/beam_Upgrade_GCP_BOM.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_Upgrade_GCP_BOM.yml?query=event%3Aschedule) |
-| [ Unmanaged Service Accounts Keys Audit ](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml) | N/A | [![.github/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml?query=event%3Aschedule) |

.github/workflows/beam_Infrastructure_PolicyEnforcer.yml

@@ -35,6 +35,7 @@ concurrency:
 permissions:
   contents: read
   issues: write
+  id-token: write
 
 jobs:
   beam_Infrastructure_PolicyEnforcer:
@@ -45,21 +46,21 @@ jobs:
       - uses: actions/checkout@v7
         with:
           persist-credentials: false
-      
+
       - name: Setup Python
         uses: actions/setup-python@v7
         with:
           python-version: '3.13'
-          
+
       - name: Install Python dependencies
         working-directory: ./infra/enforcement
         run: |
           python -m pip install --upgrade pip
           pip install -r requirements.txt
-          
+
       - name: Setup gcloud
         uses: google-github-actions/setup-gcloud@aa5489c8933f4cc7a4f7d45035b3b1440c9c10db
-          
+
       - name: Run IAM Policy Enforcement
         working-directory: ./infra/enforcement
         env:
@@ -70,7 +71,7 @@ jobs:
           EMAIL_ADDRESS: ${{ secrets.ISSUE_REPORT_SENDER_EMAIL_ADDRESS }}
           EMAIL_PASSWORD: ${{ secrets.ISSUE_REPORT_SENDER_EMAIL_PASSWORD }}
           EMAIL_RECIPIENT: "dev@beam.apache.org"
-        run: python iam.py --action print
+        run: python iam.py --action announce
 
       - name: Run Account Keys Policy Enforcement
         working-directory: ./infra/enforcement
@@ -82,4 +83,4 @@ jobs:
           EMAIL_ADDRESS: ${{ secrets.ISSUE_REPORT_SENDER_EMAIL_ADDRESS }}
           EMAIL_PASSWORD: ${{ secrets.ISSUE_REPORT_SENDER_EMAIL_PASSWORD }}
           EMAIL_RECIPIENT: "dev@beam.apache.org"
-        run: python account_keys.py --action print
+        run: python account_keys.py --action announce

infra/enforcement/README.md

@@ -134,16 +134,18 @@ The enforcement tools are integrated with GitHub Actions to provide automated co
 
 ### Workflow Configuration
 
-The repository includes workflows for different security domains:
-- **IAM Policy Enforcer** (`.github/workflows/beam_Infrastructure_PolicyEnforcer.yml`): Runs weekly on Mondays at 9:00 AM UTC.
-- **Unmanaged Keys Audit** (`.github/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml`): Runs daily at 00:00 UTC. It manages the continuous execution of the `account_keys.py` script to swiftly detect rogue service account keys generated outside the official rotation system.
-- **Manual trigger**: Can be triggered manually via `workflow_dispatch`
-- **Actions**: Runs both IAM and Account Keys enforcement with the `announce` action
+The enforcement tools are consolidated into a single daily workflow (`.github/workflows/beam_Infrastructure_PolicyEnforcer.yml`) that runs automatically at 00:00 UTC.
+
+This unified workflow executes both security domains sequentially:
+- **IAM Policy Enforcement:** Validates user bindings against the defined policies.
+- **Unmanaged Keys Audit:** Detects rogue service account keys generated outside the official rotation system.
 
 **Note**:
-- The email service is configured to use gmail
-- The recipient email is set to `dev@beam.apache.org` for Apache Beam project notifications
-- The `GITHUB_TOKEN` is automatically provided by GitHub Actions and doesn't need to be configured manually
+- **Manual trigger**: The workflow can also be triggered manually via `workflow_dispatch`.
+- **Actions**: It executes the respective Python scripts using the `announce` action.
+- The email service is configured to use gmail.
+- The recipient email is set to `dev@beam.apache.org` for Apache Beam project notifications.
+- The `GITHUB_TOKEN` is automatically provided by GitHub Actions and doesn't need to be configured manually.
 
 ## Account Keys
 

infra/enforcement/account_keys.py

@@ -19,13 +19,15 @@
 import yaml
 import argparse
 import os
+from datetime import datetime, timezone
 from typing import List, Dict, TypedDict, Optional
 from google.cloud import secretmanager
 from google.cloud import iam_admin_v1
 from google.cloud.iam_admin_v1 import types
 from sending import SendingClient
 
 SECRET_MANAGER_LABEL = "beam-infra-secret-manager"
+IAC_DRIFT_SA_KEY = "IAC_DRIFT_SA_KEY"
 
 class AuthorizedUser(TypedDict):
     email: str
@@ -312,22 +314,24 @@ def check_compliance(self) -> List[str]:
 
         # Check that all service accounts that exist are declared
         for service_account in live_service_accounts:
+
             if self._denormalize_account_email(service_account) not in [account["account_id"] for account in file_service_accounts]:
                 msg = f"Service account '{service_account}' is not declared in the service account keys file."
                 compliance_issues.append(msg)
                 self.logger.warning(msg)
-            else:
-                iam_keys = self._get_user_managed_keys_from_iam(service_account)
-                if iam_keys:
-                    secret_name = f"{self._denormalize_account_email(service_account)}-key"
-                    legal_keys = []
-                    if secret_name in managed_secrets:
-                        legal_keys = self._get_verified_keys_from_secret_manager(secret_name)
-                    unmanaged_keys = set(iam_keys) - set(legal_keys)
-                    for unmanaged_key in unmanaged_keys:
-                        msg = f"SECURITY ALERT: Unmanaged key '{unmanaged_key}' detected on account '{service_account}'. This key was created outside of Beam's service account management system. "
-                        compliance_issues.append(msg)
-                        self.logger.warning(msg)
+
+            iam_keys = self._get_user_managed_keys_from_iam(service_account)
+
+            if iam_keys:
+                secret_name = f"{self._denormalize_account_email(service_account)}-key"
+                legal_keys = []
+                if secret_name in managed_secrets:
+                    legal_keys = self._get_verified_keys_from_secret_manager(secret_name)
+                unmanaged_keys = set(iam_keys) - set(legal_keys)
+                for unmanaged_key in unmanaged_keys:
+                    msg = f"IAC_DRIFT_SA_KEY: Unmanaged key '{unmanaged_key}' detected on account '{service_account}'. This key was created outside of Beam's service account management system. "
+                    compliance_issues.append(msg)
+                    self.logger.warning(msg)
 
         extracted_secrets = [f"{self._denormalize_account_email(account['account_id'])}-key" for account in file_service_accounts]
 
@@ -376,13 +380,14 @@ def create_announcement(self, recipient: str) -> None:
             self.logger.info("No compliance issues found, no announcement will be created.")
             return
 
-        unmanaged_keys_issues = [issue for issue in diff if "SECURITY ALERT" in issue]
-        general_issues = [issue for issue in diff if "SECURITY ALERT" not in issue]
+        unmanaged_keys_issues = [issue for issue in diff if IAC_DRIFT_SA_KEY in issue]
+        general_issues = [issue for issue in diff if IAC_DRIFT_SA_KEY not in issue]
 
         if general_issues:
             self.logger.info(f"Found {len(general_issues)} general compliance issues. Triggering announcement...")
-            title = f"Account Keys Compliance Issue Detected"
-            body = f"Account keys for project {self.project_id} are not compliant with the defined policies on {self.service_account_keys_file}\n\n"
+            title = f"[{IAC_DRIFT_SA_KEY}] Action Required: Unauthorized Service Accounts Detected"
+            body = f"Unauthorized Service Accounts Report\n\n"
+            body += f"Account keys for project {self.project_id} are not compliant with the defined policies on {self.service_account_keys_file}\n\n"
             for issue in general_issues:
                 body += f"- {issue}\n"
 
@@ -406,23 +411,44 @@ def print_announcement(self, recipient: str) -> None:
         """
         if not self.sending_client:
             raise ValueError("SendingClient is required for printing announcements")
-            
+
         diff = self.check_compliance()
 
         if not diff:
             self.logger.info("No compliance issues found, no announcement will be printed.")
             return
 
-        title = f"Account Keys Compliance Issue Detected"
-        body = f"Account keys for project {self.project_id} are not compliant with the defined policies on {self.service_account_keys_file}\n\n"
-        for issue in diff:
-            body += f"- {issue}\n"
+        unmanaged_keys_issues = [issue for issue in diff if IAC_DRIFT_SA_KEY in issue]
+        general_issues = [issue for issue in diff if IAC_DRIFT_SA_KEY not in issue]
 
-        announcement = f"Dear team,\n\nThis is an automated notification about compliance issues detected in the Account Keys policy for project {self.project_id}.\n\n"
-        announcement += f"We found {len(diff)} compliance issue(s) that need your attention.\n"
-        announcement += f"\nPlease check the GitHub issue for detailed information and take appropriate action to resolve these compliance violations."
+        if general_issues:
+            self.logger.info("Printing general compliance announcement...")
+            title = f"[IAC_DRIFT_SA_KEY] Action Required: Unauthorized Service Accounts Detected"
+            body = f"Unauthorized Service Accounts Report\n\n"
+            body += f"Account keys for project {self.project_id} are not compliant with the defined policies on {self.service_account_keys_file}\n\n"
+            for issue in general_issues:
+                body += f"- {issue}\n"
 
-        self.sending_client.print_announcement(title, body, recipient, announcement)
+            announcement = f"Dear team,\n\nThis is an automated notification about compliance issues detected in the Account Keys policy for project {self.project_id}.\n\n"
+            announcement += f"We found {len(general_issues)} compliance issue(s) that need your attention.\n"
+            announcement += f"\nPlease check the GitHub issue for detailed information and take appropriate action to resolve these compliance violations."
+
+            self.sending_client.print_announcement(title, body, recipient, announcement)
+
+        if unmanaged_keys_issues:
+            self.logger.info("Printing security dashboard update for unmanaged keys...")
+            timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S UTC")
+            print("\n" + "="*60)
+            print("SIMULATING GITHUB SECURITY ISSUE CREATION/UPDATE")
+            print("="*60)
+            print(f"Title: [{IAC_DRIFT_SA_KEY}] Action Required: Unmanaged Service Account Keys Detected\n")
+            print(f"Body:\n### Unmanaged Keys Audit Report ({timestamp})")
+            print(f"The following unauthorized or unmanaged keys were detected in `{self.project_id}`:\n")
+            for issue in unmanaged_keys_issues:
+                print(f"- {issue}")
+            print("\n*Please investigate and revoke these keys if they are not part of the official rotation system.*\n")
+            print("### History\n<details>\n<summary>Click to expand</summary>\n\n[... Previous reports would be collapsed here ...]\n</details>")
+            print("="*60 + "\n")
 
     def generate_compliance(self) -> None:
         """