-
Notifications
You must be signed in to change notification settings - Fork 110
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bad values time_flow_start_ns, time_flow_end_ns when sending Newflow v9 and exporting them to Kafka #305
Comments
Hello, What is the original source of the capture that's being replayed? What's the SQL query used to plot those graphs? What are the data template fields used for timestamp? When outputting JSON, what are the actual values of the timestamps and how do they compare to what's shown in Wireshark? Could you configure Thank you |
Hello @lspgn and thank you for your rapid reply.
The captured traffic contains Netflow data spanning from Mar 13, 2024 16:52:19 (1710341539 -> first packet's reported timestamp in unix seconds at wireshark) to Mar 13, 2024 16:52:33 (1710341553 -> last packets reported timestamp in unix seconds at wireshark).
Those are production NetFlow v9 data sent to a collector.
I am investigating Druid and I have selected as __time the time_flow_start_ns that goflow2 is generating.
goflow2 -> time_flow_start_ns
I have selected a single flow to illustrate the behaviour in goflow2: {"type":"NETFLOW_V9","time_received_ns":1712219044800653208,"sequence_num":112310449,"sampling_rate":0,"sampler_address":"XXXXXXXX","time_flow_start_ns":1710281495000000000,"time_flow_end_ns":1710331885000000000 The time_received_ns seems fine as I am assuming it reports the date that this flow was received by goflow2. What seems not to be correct in my case is the time_flow_start_ns and the time_flow_end_ns. The difference between them is approximately 3 hours while they also report dates that are not correct. The NetFlow packet in Wireshark reports the following values: In pmacct, I did the same experiment and the following data are reported:
I could not pinpoint the flow output for that. |
I'll have a look at the code, it seems the (correct) calculation is
where GoFlow2 returns
(diff between the two: 59998 seconds) |
Could you have a look if #325 solves your issue? The buggy calculation was
The correct calculation was
|
Describe the bug
The goflow2 collector decodes badly the values of fields time_flow_start_ns and time_flow_end_ns in my case it expands data of 10 seconds among half day without even including the time range of the netflow data.
To Reproduce
Expected behavior
The results have been compared both to the wireshark output and also by another tools that collects NetFlow data (pmacct) and our data expand within the 10 seconds of our packet capture.
Plot from pmacct data:
Sampler device:
Data are replayed via tcpreplay using a packet capture of NetFlow data.
GoFlow2:
The text was updated successfully, but these errors were encountered: