E2E tests and Documentation for Aggregated TAD

This PR introduces: - e2e tests for Aggregated TAD - Documentation for Aggregated TAD partially-solves: 168 Signed-off-by: Tushar Tathgur <[email protected]>
antrea-io · Jul 12, 2023 · bf9a302 · bf9a302
1 parent aa09d11
commit bf9a302
Show file tree

Hide file tree

Showing 4 changed files with 213 additions and 71 deletions.
diff --git a/ci/kind/test-e2e-kind.sh b/ci/kind/test-e2e-kind.sh
@@ -173,7 +173,7 @@ function run_test {
   rm -rf $TMP_DIR
   sleep 1
 
-  go test -v -timeout=30m antrea.io/theia/test/e2e -provider=kind --logs-export-dir=$ANTREA_LOG_DIR --skip=$skiplist
+  go test -v -timeout=45m antrea.io/theia/test/e2e -provider=kind --logs-export-dir=$ANTREA_LOG_DIR --skip=$skiplist
 }
 
 echo "======== Test encap mode =========="

diff --git a/docs/throughput-anomaly-detection.md b/docs/throughput-anomaly-detection.md
@@ -69,6 +69,34 @@ $ theia throughput-anomaly-detection run --algo "ARIMA"
 Successfully started Throughput Anomaly Detection job with name tad-1234abcd-1234-abcd-12ab-12345678abcd
 ```
 
+Throughput Anomaly Detection also provide support for aggregated throughput
+anomaly detection.
+There are three different types of aggregations that are included.
+
+- `external` : Aggregated flow for inbound traffic to external IP,
+  user could provide external-IP using `external-ip` argument for further
+  filtering.
+- `pod`: Aggregated flow for inbound/outbound pod traffic.
+- `svc`: Aggregated flow for traffic to service port, user could
+  provide a destination port name using `svc-name-port` argument for
+  further filtering.
+
+For aggregated flow `pod`, user can provide the following filter arguments.
+
+- `pod-label`: The argument aggregates inbound/outbound traffic using pod
+  labels.
+- `pod-name`: The argument aggregates inbound/outbound traffic using pod name.
+- `pod-namespace`: The argument aggregates inbound/outbound traffic using
+  podnamespace. However, this argument only works as a combination to any of
+  the above two arguments and can not be used alone.
+
+All the above aggregated flow are executed using the following command:
+
+```bash
+$ theia throughput-anomaly-detection run --algo "ARIMA" --agg-flow pod --pod-label \"test_key\":\"test_value\"
+Successfully started Throughput Anomaly Detection job with name tad-1234abcd-1234-abcd-12ab-12345678abcd
+```
+
 The name of the Throughput Anomaly Detection job contains a universally
 unique identifier ([UUID](
 https://en.wikipedia.org/wiki/Universally_unique_identifier)) that is
@@ -110,13 +138,24 @@ in table format, run:
 
 ```bash
 $ theia throughput-anomaly-detection retrieve tad-1234abcd-1234-abcd-12ab-12345678abcd
-id                                      sourceIP        sourceTransportPort     destinationIP   destinationTransportPort        flowStartSeconds        flowEndSeconds          throughput                     algoCalc            anomaly
-1234abcd-1234-abcd-12ab-12345678abcd    10.10.1.25      58076                   10.10.1.33      5201                            2022-08-11T06:26:54Z    2022-08-11 08:24:54     10004969097.000000000000000000  4.0063773860532994E9        true
-1234abcd-1234-abcd-12ab-12345678abcd    10.10.1.25      58076                   10.10.1.33      5201                            2022-08-11T06:26:54Z    2022-08-11 08:06:54     4005703059.000000000000000000   1.0001208294655691E10       true
-1234abcd-1234-abcd-12ab-12345678abcd    10.10.1.25      58076                   10.10.1.33      5201                            2022-08-11T06:26:54Z    2022-08-11 08:34:54     50007861276.000000000000000000  3.9735065921281104E9        true
+id                                      sourceIP        sourceTransportPort     destinationIP   destinationTransportPort        flowStartSeconds        flowEndSeconds          throughput              aggType         algoType        algoCalc                anomaly
+1234abcd-1234-abcd-12ab-12345678abcd    10.10.1.25      58076                   10.10.1.33      5201                            2022-08-11T06:26:54Z    2022-08-11T08:06:54Z    4.005703059e+09         e2e             ARIMA           1.0001208441920074e+10  true
+1234abcd-1234-abcd-12ab-12345678abcd    10.10.1.25      58076                   10.10.1.33      5201                            2022-08-11T06:26:54Z    2022-08-11T08:24:54Z    1.0004969097e+10        e2e             ARIMA           4.006432886406564e+09   true
+1234abcd-1234-abcd-12ab-12345678abcd    10.10.1.25      58076                   10.10.1.33      5201                            2022-08-11T06:26:54Z    2022-08-11T08:34:54Z    5.0007861276e+10        e2e             ARIMA           3.9735067954945493e+09  true
+```
+
+Aggregated Throughput Anomaly Detection has different columns based of the
+aggregation type.
+e.g.  agg-type pod output
+
+```bash
+$ theia throughput-anomaly-detection retrieve tad-5ca4413d-6730-463e-8f95-86032ba28a4f
+id                                   destinationServicePortName flowEndSeconds       throughput       aggType        algoType       algoCalc               anomaly
+5ca4413d-6730-463e-8f95-86032ba28a4f test_serviceportname       2022-08-11T08:24:54Z 5.0024845485e+10 svc            ARIMA          2.0863933021708477e+10 true
+5ca4413d-6730-463e-8f95-86032ba28a4f test_serviceportname       2022-08-11T08:34:54Z 2.5003930638e+11 svc            ARIMA          1.9138281301304165e+10 true
 ```
 
-User may also save the result in an output file in json format
+User may also save the result in an output file in json format.
 
 ### List all throughput anomaly detection jobs
 

diff --git a/test/e2e/framework.go b/test/e2e/framework.go
@@ -26,6 +26,7 @@ import (
 	"regexp"
 	"strconv"
 	"strings"
+	"sync"
 	"testing"
 	"time"
 
@@ -163,6 +164,7 @@ const (
 var (
 	errConnectionLost = fmt.Errorf("http2: client connection lost")
 	clickHousePodName = fmt.Sprintf("%s-0-0-0", clickHousePodNamePrefix)
+	mutex             sync.Mutex
 )
 
 type FlowVisibiltiySetUpConfig struct {