Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

correlation alerts #877

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@
import org.opensearch.rest.RestHandler;
import org.opensearch.script.ScriptService;
import org.opensearch.securityanalytics.action.*;
import org.opensearch.securityanalytics.correlation.alert.CorrelationAlertService;
import org.opensearch.securityanalytics.correlation.index.codec.CorrelationCodecService;
import org.opensearch.securityanalytics.correlation.index.mapper.CorrelationVectorFieldMapper;
import org.opensearch.securityanalytics.correlation.index.query.CorrelationQueryBuilder;
Expand All @@ -60,6 +61,7 @@
import org.opensearch.securityanalytics.logtype.LogTypeService;
import org.opensearch.securityanalytics.mapper.IndexTemplateManager;
import org.opensearch.securityanalytics.mapper.MapperService;
import org.opensearch.securityanalytics.model.CorrelationAlert;
import org.opensearch.securityanalytics.model.CustomLogType;
import org.opensearch.securityanalytics.model.ThreatIntelFeedData;
import org.opensearch.securityanalytics.resthandler.*;
Expand Down Expand Up @@ -171,7 +173,8 @@ public Collection<Object> createComponents(Client client,
return List.of(
detectorIndices, correlationIndices, correlationRuleIndices, ruleTopicIndices, customLogTypeIndices, ruleIndices,
mapperService, indexTemplateManager, builtinLogTypeLoader, builtInTIFMetadataLoader, threatIntelFeedDataService, detectorThreatIntelService,
tifJobUpdateService, tifJobParameterService, threatIntelLockService);
tifJobUpdateService, tifJobParameterService, threatIntelLockService,
new CorrelationAlertService(client, clusterService, xContentRegistry));
}

@Override
Expand Down Expand Up @@ -239,6 +242,7 @@ public ScheduledJobParser getJobParser() {
public List<NamedXContentRegistry.Entry> getNamedXContent() {
return List.of(
Detector.XCONTENT_REGISTRY,
CorrelationAlert.XCONTENT_REGISTRY,
DetectorInput.XCONTENT_REGISTRY,
Rule.XCONTENT_REGISTRY,
CustomLogType.XCONTENT_REGISTRY,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*/
package org.opensearch.securityanalytics.correlation.alert;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.opensearch.action.search.SearchRequest;
import org.opensearch.action.search.SearchResponse;
import org.opensearch.client.Client;
import org.opensearch.cluster.ClusterState;
import org.opensearch.cluster.service.ClusterService;
import org.opensearch.common.xcontent.LoggingDeprecationHandler;
import org.opensearch.common.xcontent.XContentType;
import org.opensearch.commons.alerting.model.Table;
import org.opensearch.core.action.ActionListener;
import org.opensearch.core.xcontent.NamedXContentRegistry;
import org.opensearch.core.xcontent.XContentParser;
import org.opensearch.index.query.BoolQueryBuilder;
import org.opensearch.index.query.QueryBuilders;
import org.opensearch.search.SearchHit;
import org.opensearch.search.builder.SearchSourceBuilder;
import org.opensearch.search.sort.FieldSortBuilder;
import org.opensearch.search.sort.SortBuilders;
import org.opensearch.search.sort.SortOrder;
import org.opensearch.securityanalytics.model.CorrelationAlert;

import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Objects;

public class CorrelationAlertService {
public static final String CORRELATION_ALERT_INDEX = ".opensearch-sap-correlations-alerts";
private static final Logger log = LogManager.getLogger(CorrelationAlertService.class);

Check warning on line 37 in src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java#L37

Added line #L37 was not covered by tests
private final Client client;
private final ClusterService clusterService;
private final NamedXContentRegistry xContentRegistry;

public CorrelationAlertService(Client client, ClusterService clusterService, NamedXContentRegistry xContentRegistry) {
this.client = client;
this.clusterService = clusterService;
this.xContentRegistry = xContentRegistry;
}

Check warning on line 46 in src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java#L42-L46

Added lines #L42 - L46 were not covered by tests

public void getCorrelationAlerts(ActionListener<CorrelationAlertsList> listener,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can we move the ActionListener as the last parameter for consistency?

Table table,
String severityLevel,
String alertState) {
try {
if (false == correlationAlertsIndexExists()) {
listener.onResponse(new CorrelationAlertsList(Collections.emptyList(), 0L));

Check warning on line 54 in src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java#L54

Added line #L54 was not covered by tests
} else {
FieldSortBuilder sortBuilder = SortBuilders
.fieldSort(table.getSortString())
.order(SortOrder.fromString(table.getSortOrder()));

Check warning on line 58 in src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java#L56-L58

Added lines #L56 - L58 were not covered by tests
if (null != table.getMissing() && false == table.getMissing().isEmpty()) {
sortBuilder.missing(table.getMissing());

Check warning on line 60 in src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java#L60

Added line #L60 was not covered by tests
}
BoolQueryBuilder queryBuilder = QueryBuilders.boolQuery();

Check warning on line 62 in src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java#L62

Added line #L62 was not covered by tests

if (false == Objects.equals(severityLevel, "ALL")) {
queryBuilder.filter(QueryBuilders.termQuery("severity", severityLevel));

Check warning on line 65 in src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java#L65

Added line #L65 was not covered by tests
}
if (false == Objects.equals(alertState, "ALL")) {
queryBuilder.filter(QueryBuilders.termQuery("state", alertState));

Check warning on line 68 in src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java#L68

Added line #L68 was not covered by tests
}
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder()
.version(true)
.seqNoAndPrimaryTerm(true)
.query(queryBuilder)
.sort(sortBuilder)
.size(table.getSize())
.from(table.getStartIndex());

Check warning on line 76 in src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java#L70-L76

Added lines #L70 - L76 were not covered by tests

SearchRequest searchRequest = new SearchRequest(CORRELATION_ALERT_INDEX).source(searchSourceBuilder);
client.search(searchRequest, ActionListener.wrap(

Check warning on line 79 in src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java#L78-L79

Added lines #L78 - L79 were not covered by tests
searchResponse -> {
if (0 == searchResponse.getHits().getHits().length) {
listener.onResponse(new CorrelationAlertsList(Collections.emptyList(), 0L));

Check warning on line 82 in src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java#L82

Added line #L82 was not covered by tests
} else {
listener.onResponse(

Check warning on line 84 in src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java#L84

Added line #L84 was not covered by tests
new CorrelationAlertsList(
parseCorrelationAlerts(searchResponse),
searchResponse.getHits().getTotalHits().value)

Check warning on line 87 in src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java#L86-L87

Added lines #L86 - L87 were not covered by tests
);
}
},

Check warning on line 90 in src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java#L90

Added line #L90 was not covered by tests
e -> {
log.error("Search request to fetch correlation alerts failed", e);
listener.onFailure(e);
}

Check warning on line 94 in src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java#L92-L94

Added lines #L92 - L94 were not covered by tests
));
}
} catch (Exception e) {
log.error("Unexpected error when fetch correlation alerts", e);
listener.onFailure(e);
}
}

Check warning on line 101 in src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java#L97-L101

Added lines #L97 - L101 were not covered by tests

public boolean correlationAlertsIndexExists() {
ClusterState clusterState = clusterService.state();
return clusterState.getRoutingTable().hasIndex(CORRELATION_ALERT_INDEX);

Check warning on line 105 in src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java#L104-L105

Added lines #L104 - L105 were not covered by tests
}

public List<CorrelationAlert> parseCorrelationAlerts(final SearchResponse response) throws IOException {
List<CorrelationAlert> alerts = new ArrayList<>();

Check warning on line 109 in src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java#L109

Added line #L109 was not covered by tests
for (SearchHit hit : response.getHits()) {
XContentParser xcp = XContentType.JSON.xContent().createParser(

Check warning on line 111 in src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java#L111

Added line #L111 was not covered by tests
xContentRegistry,
LoggingDeprecationHandler.INSTANCE, hit.getSourceAsString());
CorrelationAlert correlationAlert = CorrelationAlert.docParse(xcp, hit.getId(), hit.getVersion());
alerts.add(correlationAlert);
}
return alerts;

Check warning on line 117 in src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertService.java#L113-L117

Added lines #L113 - L117 were not covered by tests
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*/
package org.opensearch.securityanalytics.correlation.alert;

import org.opensearch.securityanalytics.model.CorrelationAlert;

import java.util.List;

/**
* Wrapper class that holds list of correlation alerts and total number of alerts available.
* Useful for pagination.
*/
public class CorrelationAlertsList {

private final List<CorrelationAlert> correlationAlertList;
private final Long totalAlerts;

public CorrelationAlertsList(List<CorrelationAlert> correlationAlertList, long totalAlerts) {
this.correlationAlertList = correlationAlertList;
this.totalAlerts = totalAlerts;
}

Check warning on line 23 in src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertsList.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertsList.java#L20-L23

Added lines #L20 - L23 were not covered by tests

public List<CorrelationAlert> getCorrelationAlertList() {
return correlationAlertList;

Check warning on line 26 in src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertsList.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertsList.java#L26

Added line #L26 was not covered by tests
}

public Long getTotalAlerts() {
return totalAlerts;

Check warning on line 30 in src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertsList.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/org/opensearch/securityanalytics/correlation/alert/CorrelationAlertsList.java#L30

Added line #L30 was not covered by tests
}

}
Loading
Loading