Add anomaly detector for AS hegemony cone (count of downstream networks)

[romain-fontugne]

Is your feature request related to a problem? Please describe.
Currently we cannot detect anomalies in the number of dependent networks because these value is computed just before storing it in the database.

Describe the solution you'd like

• We should compute the number of dependent networks from AS hegemony values in Kafka and push the results to Kafka.
• So we can also run an anomaly detector for it.
• And we'll need a new consumer to store these values to the postures database.

[rishabhknowss]

Hi Romain,

While reviewing the enhancement and going through the repository, I think that we need to add logic to compute the number of dependent networks in the psql/consumers/ASHegemony.py script.
The logic can be implemented as -

def compute_dependent_networks(as_hegemony_values):
    # Dictionary to store the count of dependent networks for each AS
    dependent_networks_count = {}

    # Iterate through AS hegemony values
    for as_value in as_hegemony_values:
        asn = as_value['asn']
        dependent_networks = as_value.get('dependent_networks', [])

        # Update the count for each dependent network
        for network in dependent_networks:
            if network not in dependent_networks_count:
                dependent_networks_count[network] = 1
            else:
                dependent_networks_count[network] += 1

    return dependent_networks_count

Additionally, creating a new consumer script to consume messages from the topic where the computed results are pushed, and adding a PostgreSQL schema for storing the count requests.
Well , I am new to Kafka as well as IHR and since I don't have hands-on experience with it, can you please guide me more towards implementing the solution

[romain-fontugne]

Hi,

A similar code is already in psql/consumers/ASHegemony.py:

kafka-toolbox/psql/consumers/ASHegemony.py

Line 193 in 89649fb

self.hegemonyCone[asn] += inc

We need to decouple the two functions that this script has, pushing ASHegemony values to Postgres and computing the number of dependent networks. So I think we should start with a new script that reads ASHegemony values from Kafka and store the number of dependent networks back into kafka.

[rishabhknowss]

Okay! On it

[ojninja16]

HI Romain, is this issue still open?..I would wanna work on this.