Introducing Squadcast’s Key Based Deduplication

We are excited to share another feature update with all our valued customers! We have recently gone live with our Key Based Deduplication feature, enabling you to define dedup keys using customizable templates for configured alert sources. With this feature, you can automatically group similar incidents and effectively deduplicate alerts.

This blog  provides detailed information on accessing the feature, exploring its primary functionalities, and the best practices that will improve  your Incident Management process.

The Impact of Alert Noise on On-Call Efficiency

The issue of alert fatigue has become increasingly prevalent and troublesome in DevOps and Site Reliability Engineering (SRE) teams, due to the overwhelming volume of available data. The purpose of using monitoring tools to send alerts is to foster a proactive Incident Management culture. However, this objective is gradually eroded.

Identifying the problem becomes apparent when the quantity of low-priority/warning alerts significantly outweighs the number of actionable alerts. As a result, critical incidents of high severity are either detected much later than they should be or go unnoticed altogether.

Consequently, it is crucial to ensure that On-Call engineers, responsible for responding to these incidents, are not burdened with excessive alert noise. The challenge now revolves around finding a solution that captures all the relevant data while selectively notifying engineers only for the alerts that require immediate action.

Key Based Deduplication to the Rescue

Key Based Deduplication is an efficient way to avoid duplicate entries when processing incoming Events alongside existing Incidents. It works by generating a Dedup key using a user-defined template specific to events from an Alert Source. This key helps identify and group duplicates more effectively.

How does Key Based Deduplication work

1. To utilize the Key Based Deduplication feature, the user needs to opt-in to a specific Service.
2. The user has the option to define a template for generating  Dedup keys for each alert source within the Service. Additionally, the user specifies a duration (x) for the Deduplication Window.
3. When an incoming Event occurs, the Dedup key is calculated based on the defined template.
4. This key is then compared (using equality) to any previous Incidents within the Deduplication Window (which is the specified duration).
5. If an open incident with the same Dedup key is found, the current Event is deduplicated against that Incident.
6. However, if no matching incident is found, a new Incident is created.
7. Once the Deduplication Duration (x) elapses, the system recalculates the Dedup key using the defined template.
8. This process continues for ongoing deduplication.

How to configure Key Based Deduplication

Let's start with the Prerequisites 

The User Role associated with the user in the Team must have required permissions to manage Services and hence ability to manage Key Based Deduplication.

Here is how you add Dedup keys to configure Key Based Deduplication:

Step1: Go to Services in Squadcast, then navigate to Service Overview and select or search for your desired Service.

Step2: Expand the accordion on the extreme right. In the Automation section, click on View All.

Step3: In the Key Based Deduplication section, click on Add Dedup Key.

Step4: Choose an alert source to start creating Dedup keys for your incoming Events.

Step5: On the right side, you can see the payload of the latest alert for the selected Alert Source. (Please Note: You can configure only one Dedup key per Alert Source.)

Step6: Define a template that generates Dedup keys using variables from the referenced payload on the right, specifically for a particular alert source of a Service. For guidance on writing templates, please consult Go's standard library. (A Dedup key is calculated for the incoming event based on the user-defined template.)

Step7: Specify the Deduplication Time in minutes or hours. (Note: The maximum time limit is 48 hours.)

Step8: Click Save.

How to delete Key Based Deduplication Rules

Deleting a Key Based Deduplication Rule is easy peasy:

1. Click on the Key Based Dedup Rule for a selected Service.
2. On the right-hand side, click on More Options and then Delete.
3. Click Delete again to confirm.

And you are done!

Conclusion

Are you still reading this? Fantastic! At Squadcast, we strive to simplify our platform and services to provide you with a smooth user experience. We believe that our Key Based Deduplication feature brings us closer to this objective. We encourage you to give this feature a try and share your feedback or experience in the comments or with our support teams. Cheers!

Interested in learning more? Take a look at our Key Based Deduplication Documentation.