Fixing Short CloudWatch Log Retention With Terraform

Dec 4, 2025 by Alex Johnson 53 views

SonarCloud Security Hotspot: Addressing Short CloudWatch Log Retention (terraform:S6413)

Introduction

In this article, we will address a critical security hotspot identified by SonarCloud concerning the retention duration of CloudWatch logs within a Terraform configuration. Specifically, the issue flagged is terraform:S6413, which highlights the security sensitivity of defining a short log retention duration. We'll delve into the details of this issue, explain why it matters, provide a step-by-step guide on how to fix it, and discuss the broader implications for your infrastructure's security and compliance. By understanding and rectifying this issue, you can significantly improve your ability to investigate security incidents, conduct forensic analysis, and meet regulatory requirements. The aim is to ensure that your logging practices provide sufficient data for security analysis while adhering to best practices for data retention.

Understanding the Security Hotspot

What is the Issue?

The core issue identified by SonarCloud is that the current CloudWatch log group configuration has a log retention period of only 7 days. According to security best practices, this duration is insufficient for thorough security incident investigation and digital forensic analysis. A shorter retention period means that valuable log data, which could be crucial in understanding security breaches or other operational issues, is lost after just one week. This can severely hinder your ability to trace malicious activity, establish timelines of events, and identify the root causes of security incidents.

Rule and Severity

The specific rule that SonarCloud is flagging is terraform:S6413, which is designed to detect instances where log retention durations are set too low. This rule is part of SonarSource's broader suite of security checks for Infrastructure as Code (IaC) projects. The severity of this issue is classified as a Major Security Hotspot, indicating that it represents a significant security risk that should be addressed promptly. Ignoring such hotspots can leave your infrastructure vulnerable to undetected security breaches and can complicate compliance efforts. A Major Security Hotspot means that the issue has the potential to directly impact the confidentiality, integrity, or availability of your systems and data.

Specifics of the Issue

The issue was found in the main.tf file, specifically on line 342. This location points to the resource definition for an aws_cloudwatch_log_group named ecs. The current code snippet shows that the retention_in_days attribute is set to 7, which is below the recommended minimum. This configuration means that any logs generated by the ECS service will only be stored for seven days before being automatically deleted. While this might seem like a minor configuration detail, the implications for security and compliance are substantial. Without adequate log retention, you may not have the data needed to respond effectively to security incidents or meet audit requirements.

Why Short Log Retention Matters

The duration for which you retain logs is a critical factor in maintaining a secure and compliant infrastructure. Short log retention periods can have several significant consequences, affecting your ability to respond to security incidents, perform forensic analysis, and meet compliance requirements.

Impaired Incident Response

When a security incident occurs, logs are often the first place security professionals turn to understand what happened. Logs provide a detailed record of system and application activities, including user actions, network traffic, and system events. With short log retention, the data needed to backtrace malicious actor actions during security incidents may no longer be available. This can severely impair your ability to understand the scope and impact of a breach, identify the attack vectors used, and take appropriate remediation steps. Without sufficient log data, you may be operating in the dark, making it difficult to contain the incident and prevent future occurrences.

Challenges in Digital Forensics

Digital forensics involves the systematic examination of digital evidence to identify, preserve, recover, analyze, and present facts and opinions about the information. Log data is a crucial component of digital forensic investigations, providing a timeline of events and helping to piece together the sequence of activities leading up to an incident. Short log retention periods can make it impossible to perform thorough forensic analyses, as the necessary data may have been purged. This can hinder your ability to understand the full extent of a security breach, identify all affected systems and data, and take steps to prevent similar incidents in the future.

Difficulty in Establishing Timelines

Establishing timelines of attacker activities is essential for understanding the progression of a security incident. Logs provide timestamps for each event, allowing you to reconstruct the sequence of actions taken by an attacker. With short log retention, critical events may be missing from the log data, making it difficult to create an accurate timeline. This can complicate incident response efforts, as it may not be possible to fully understand the attacker's tactics, techniques, and procedures (TTPs). Without a clear timeline, it can be challenging to identify all affected systems and data, and to implement effective countermeasures.

Compliance and Regulatory Requirements

Many compliance standards and regulations, such as GDPR, HIPAA, and PCI DSS, mandate specific log retention periods. These requirements are designed to ensure that organizations maintain sufficient records for auditing, security monitoring, and incident investigation purposes. Short log retention periods can put your organization in violation of these requirements, leading to fines, penalties, and reputational damage. Compliance is not just a matter of ticking boxes; it's about demonstrating that you have adequate controls in place to protect sensitive data and maintain the security of your systems. By adhering to recommended log retention periods, you can ensure that you meet your compliance obligations and reduce the risk of regulatory scrutiny.

Implementing the Recommended Fix

To address the security hotspot identified by SonarCloud, you need to increase the log retention period for the CloudWatch log group. The recommended fix is to set the retention period to at least 30 days, or longer based on your organization's compliance requirements.

Step-by-Step Guide

Locate the Resource Definition: Open your main.tf file and navigate to line 342, where the aws_cloudwatch_log_group resource named ecs is defined.
Modify the retention_in_days Attribute: Change the value of the retention_in_days attribute from 7 to 30 (or a higher value if required by your compliance policies). The updated code should look like this:
```
resource "aws_cloudwatch_log_group" "ecs" {
  name              = "/ecs/${local.project_name}"
  retention_in_days = 30  # Recommended: 30 days or more

  tags = local.tags
}
```
Consider Compliance Requirements: Before finalizing the change, review your organization's compliance requirements for log retention. Some regulations may mandate retention periods longer than 30 days. If necessary, adjust the retention_in_days value accordingly.
Run terraform plan: After making the changes, run terraform plan in your terminal. This command will show you a preview of the changes that Terraform will make to your infrastructure. Review the output to ensure that the only change being made is the update to the retention_in_days attribute.
Run terraform apply: If the plan looks correct, run terraform apply to apply the changes to your infrastructure. Terraform will update the CloudWatch log group to use the new retention period.
Run terraform fmt: Before committing your changes, run terraform fmt to format your code according to Terraform's style conventions. This will help ensure consistency and readability.
Commit and Push: Commit your changes to your version control system and push them to your remote repository.

Code Example

Here's the code snippet showing the recommended fix:

resource "aws_cloudwatch_log_group" "ecs" {
  name              = "/ecs/${local.project_name}"
  retention_in_days = 30  # Recommended: 30 days or more

  tags = local.tags
}

Best Practices for Log Retention

Setting the log retention period to at least 30 days is a good starting point, but there are several other best practices you should consider to ensure effective log management.

Organizational Compliance Requirements

Always align your log retention policies with your organization's compliance requirements. Different regulations may mandate specific retention periods for different types of logs. For example, GDPR requires organizations to retain personal data for only as long as necessary, while other regulations may specify minimum retention periods for security logs. Understand your compliance obligations and set your log retention periods accordingly.

Log Rotation and Archiving

Implement log rotation and archiving strategies to manage the volume of log data. Log rotation involves creating new log files periodically and archiving the old ones. This can help prevent log files from growing too large and becoming difficult to manage. Archiving involves moving old log data to a separate storage location, such as Amazon S3, for long-term retention. This allows you to retain log data for extended periods without impacting the performance of your logging systems. Make sure your archived logs are securely stored and easily accessible when needed.

Log Analysis and Monitoring

Regularly analyze your logs to identify security threats and operational issues. Use log analysis tools and techniques to detect anomalies, suspicious activity, and other indicators of compromise. Implement real-time monitoring to alert you to potential problems as they occur. Log analysis and monitoring are essential for proactive security management and can help you detect and respond to security incidents more quickly. Consider using Security Information and Event Management (SIEM) systems to centralize log data and automate analysis and monitoring.

Secure Log Storage

Ensure that your log data is stored securely to prevent unauthorized access and tampering. Use encryption to protect log data both in transit and at rest. Implement access controls to restrict who can view and modify log data. Regularly audit your log storage environment to ensure that it remains secure. Consider using immutable storage solutions to prevent log data from being altered or deleted, which can be crucial for forensic investigations.

Centralized Logging

Centralize your logging infrastructure to make it easier to collect, store, and analyze log data. Use a centralized logging service, such as Amazon CloudWatch Logs, to collect logs from multiple sources in your environment. Centralized logging simplifies log management and makes it easier to correlate events across different systems. It also provides a single point of access for log data, which can streamline security investigations and compliance efforts. Consider using log aggregation tools to collect logs from various sources and forward them to your centralized logging system.

Conclusion

Addressing the SonarCloud security hotspot related to short CloudWatch log retention duration is a critical step in ensuring the security and compliance of your infrastructure. By increasing the log retention period to at least 30 days and following best practices for log management, you can significantly improve your ability to investigate security incidents, conduct forensic analysis, and meet regulatory requirements. Remember to consider your organization's specific compliance needs and adjust the retention period accordingly. Regular log analysis and monitoring, along with secure log storage, are essential for maintaining a robust security posture. By prioritizing these aspects, you can protect your organization from potential threats and ensure the integrity of your systems and data.

For more information on CloudWatch Logs and best practices for log management, visit the AWS CloudWatch Logs Documentation.