Skip to main content

​
Index Patterns

Index patterns define which OpenSearch indices contain logs for detection. Each pattern represents a log source type (e.g., Windows logs, network logs).

​
Understanding Index Patterns

An index pattern tells CHAD:
  • Which indices contain the logs
  • What timestamp field to use
  • How to map Sigma fields to your schema
  • Auth token for log shipping authentication
  • Security settings (IP allowlist, rate limiting)
  • Health thresholds for monitoring

​
Creating Index Patterns

  1. Navigate to Settings > Index Patterns
  2. Click Create Index Pattern
  3. Fill in the configuration
  4. Click Save

​
Required Fields

FieldDescriptionExample
NameFriendly nameWindows Security Logs
Index PatternOpenSearch index patternwinlogbeat-*
Timestamp FieldTime field name@timestamp

​
Pattern Syntax

Use wildcards to match multiple indices:
PatternMatches
logs-*logs-2024.01, logs-2024.02, etc.
winlogbeat-*All winlogbeat indices
auditbeat-*,filebeat-*Multiple patterns (comma-separated)

​
Sigma Logsource Mapping

Link Sigma logsources to index patterns: 
# Sigma rule logsource
logsource:
  product: windows
  service: security
In CHAD, map this to your index pattern:
Sigma ProductSigma ServiceIndex Pattern
windowssecuritywinlogbeat-*
windowssysmonsysmon-*
linuxauthfilebeat-*
network-zeek-*
Rules automatically deploy to matching index patterns.

​
Log Shipping Authentication

Each index pattern has a unique auth token for authenticating log shipping requests.

​
Getting the Token

  1. Open the index pattern
  2. Click Settings tab
  3. Copy the Auth Token

​
Using the Token

Include in the Authorization header when sending logs: 
curl -X POST \
  -H "Authorization: Bearer YOUR_AUTH_TOKEN" \
  -H "Content-Type: application/json" \
  -d '[{"@timestamp": "...", "event": {...}}]' \
  https://chad.example.com/api/logs/winlogbeat
The auth token is sensitive. Store it securely in your log shipper configuration.

​
Security Settings

​
IP Allowlist

Restrict which IPs can send logs to this index pattern:
  1. Open the index pattern
  2. Click Security tab
  3. Add allowed IPs or CIDR ranges
Examples:
  • Single IP: 192.168.1.100
  • CIDR range: 10.0.0.0/8
Leave empty to allow all IPs.

​
Rate Limiting

Prevent log flooding with rate limits:
SettingDescriptionDefault
Requests/minuteMax API requests100
Events/minuteMax log events50,000
Enable rate limiting in the index pattern settings.

​
Field Mapping Configuration

Each index pattern has its own field mappings:
  1. Open the index pattern
  2. Click Field Mappings
  3. Map Sigma fields to your schema
Example mappings:
Sigma FieldYour Field
CommandLineprocess.command_line
Useruser.name
SourceIPsource.ip
See Field Mappings for detailed configuration.

​
Health Monitoring

Configure health thresholds per index pattern:
SettingDescriptionDefault
No Data AlertAlert if no logs for N minutes15 min
Error Rate ThresholdAlert if error rate exceeds %5%
Latency WarningWarn if queries exceed ms500 ms
Latency CriticalCritical if queries exceed ms2000 ms

​
Example: Strict Monitoring

For critical log sources: 
No Data Alert: 5 minutes
Error Rate Threshold: 1%
Latency Warning: 200 ms
Latency Critical: 500 ms

​
Example: Relaxed Monitoring

For batch or periodic logs: 
No Data Alert: 60 minutes
Error Rate Threshold: 10%
Latency Warning: 1000 ms
Latency Critical: 5000 ms

​
Threat Intelligence

Enable TI enrichment per index pattern:
  1. Open the index pattern
  2. Click Threat Intelligence
  3. Select which TI sources to use
  4. Map fields for IOC extraction
IOC Field Mapping:
IOC TypeYour Field
IP Addresssource.ip, destination.ip
Domaindns.question.name
URLurl.original
Hashfile.hash.sha256

​
Managing Index Patterns

​
Edit

  1. Click the index pattern
  2. Modify settings
  3. Save changes
Changes take effect immediately for new alerts.

​
Delete

  1. Click the index pattern
  2. Click Delete
  3. Confirm deletion
Deleting an index pattern removes associated percolators. Rules remain but become undeployed.

​
Disable

Temporarily disable without deleting:
  1. Click the index pattern
  2. Toggle Enabled off
  3. Detection pauses for this pattern

​
Index Pattern vs OpenSearch Index

ConceptDescription
OpenSearch IndexActual data storage in OpenSearch
Index PatternCHAD configuration pointing to indices
CHAD doesn’t create data indices—it uses existing ones created by your log shipper.

​
Common Patterns

​
Elastic Beats

winlogbeat-*      # Windows Event Logs
auditbeat-*       # Linux Audit
filebeat-*        # Various file-based logs
packetbeat-*      # Network traffic

​
Fluentd

logs-windows-*    # Windows logs via Fluentd
logs-linux-*      # Linux logs via Fluentd

​
Custom

security-events-* # Custom security events
network-traffic-* # Network flow data
cloud-audit-*     # Cloud provider audit logs

​
Troubleshooting

​
No indices found

  1. Verify the pattern matches actual indices
  2. Check OpenSearch permissions
  3. Test with GET _cat/indices/your-pattern-*

​
Rules not matching

  1. Verify logsource mapping
  2. Check field mappings
  3. Test with sample logs

​
Health alerts firing

  1. Check if logs are actually flowing
  2. Verify timestamp field is correct
  3. Adjust thresholds for your volume

​
Next Steps

Field Mappings

Configure field translation

Health Monitoring

Configure monitoring thresholds