forked from voipmonitor/sniffer
-
Notifications
You must be signed in to change notification settings - Fork 0
/
ChangeLog
1655 lines (1313 loc) · 100 KB
/
ChangeLog
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
10.1.26 30.01.2015
==================
- if RTP changes destination port but SSRC is the same create new RTP so the GUI can see that the change happened [VS-168]
10.1.25 30.01.2015
==================
- fix TCP manager racecondition when quick connections are made which leads to communication errors (and lags in the GUI when loading charts)
10.1.24 27.01.2015
==================
- fix G729 audio decode for 10ms variant
10.1.23 23.12.2014
==================
- do not use more memory then set in packetbuffer_total_maxheap in case the sniffer is in receiver mode and remote sniffer reconnect to the receiver and sends all traffic in burst (receiver will slow down)
10.1.22 11.12.2014
==================
- fix bad arithmetic when writing time of RTP packets (in wors case it was +- 1 sec) which can lead to show RTP packets before SIP packets in SIP history
10.1.21 11.12.2014
==================
- fix pcapscandir which ignored pcap files if they were added before some pcap file was processed (it read only first inotify event from the queue)
10.1.20 21.11.2014
==================
- fix crash in skinny when parsing multiple skinny messages in one packet (all versions)
10.1.19 21.11.2014
==================
- implement skinny_ignore_rtpip option for cisco skinny protocol in case call manager receives RTP always on the same port. More info in voipmonitor.conf
10.1.18 20.11.2014
==================
- close all file descriptors when upgrading / restarting sniffer to prevent leakage and other problems (manual restart after autoupgrade is recommended again until this version)
10.1.17 20.11.2014
==================
- clean buffers before issuing restart which might solve problem with allocating ringbuffer when upgrading from GUI
- fix potential crash in skinny packet handling
- fix updating destination number for ring group if connected call does reinvite
10.1.16 18.11.2014
==================
- fix crash in tcpreassembly
10.1.15 17.11.2014
==================
- move memory deallocation in threading mod 4 to different thread which offloads 10% for t0 thread
- add new option cdr_check_exists_callid: when storing CDR check if there is the same CDR with the same sip.Call-ID and replace it if the new one contains RTP and the old one not this option is intended for case where you need to mirror SIP signalling to RTP media nodes and every RTP voipmonitor node sends CDR based on the same SIP thus diplicating CDR and only one CDR have RTP associated. default is off
10.1.14 12.11.2014
==================
- fix crash on some linux versions when connecting to cloud (in cloud mode)
10.1.13 11.11.2014
==================
- fix billing partitions creation
10.1.12 10.11.2014
==================
- revert manager interface to use BLOCK socket which might fix recent issues with timeout
10.1.11 5.11.2014
=================
- fix crashing when SIP message is multipart/mixed (for example if contains SIPI). All versions >= 9.0
10.1.10 4.11.2014
=================
- fix issue with detect user agent for caller / called (VS-135)
- support charts for older librrd (<1.4)
- fix not storing CDR for some mysql versions / configurations if there is more sip reponse with the same text but case sensitive (200 OK | 200 ok)
- use /etc/localtime even if sniffer is statically linked with non default localtime location which fixes our 32bit sniffer builds wchih have bad timezone since 10.1.6. Print time on stdout when sniffer starts for control purpose.
- fix live audio monitor when there is duplicated RTP streams (choppy sound)
10.1.9 30.10.2014
=================
- Fix loading old files when scanpcapdir is used. (If files were moved/created in scanpcapdir and voipmonitor was not running, files were not loaded after voipmonitor started - only new files were processed.)
10.1.8 27.10.2014
=================
- 32 bit static compiled versions are now compatible with all > 2.6.18 (centos 5.10 etc.)
- add manager command cleanup_calls
10.1.7 16.10.2014
=================
- Implement new configuration option which will restrict spool cleaning based on time (for example cleanspool_enable_fromto = 1-5 will run cleaning only in night between 1-5 hours)
10.1.6 16.10.2014
=================
- Another attempt to fix stucked manager interface - clean threads with function pthread_detach() instead of pthread_attr_setdetachstate(&attr,PTHREAD_CREATE_DETACHED)
10.1.5 15.10.2014
=================
- fix parsing DTMF sip calls with DTMF RTP frames with payload 101 where payload 101 is not in SDP
10.1.4 13.10.2014
=================
- support older rrd library when generating graphs
- fix concurrent access to rrd graphs
- try to fix TCP manager problem with creating threads on 32bit (manager stuck)
10.1.3 13.10.2014
=================
- Stucked TCP manager was not properly set to non blocking socket thus the timeout never occured.
- fix pcap_dump_asyncbuffer was never applied thus always deafault to 100MB.
- add new manager commands - check_filesindex, reindexfiles_date %s - example: reindexfiles_date 2014-09-03, reindexfiles_datehour %s %i - example: reindexfiles_datehour 2014-09-03 15
10.1.2 10.10.2014
=================
- Try to fix stucked TCP manager on accept (use select before accept) and blocking all new connections until restart
10.1.1 7.10.2014
================
- add new library dependency into configure script
- remove unused rrd function to make it compatible with older librrd
10.1 6.10.2014
==============
- implement logging status information to RRD charts
10.0.42 6.10.2014
=================
- fix RTP stream for G729 10ms packetization - both reading and audio decoding which was completely ignored
10.0.41 26.9.2014
=================
- fix broken cleaning in version 10.0.40 if it was suspended due to increased I/O buffer above threshold. After upgrading you have to run "telnet localhost 5029 - reindexfiles"
10.0.40 26.9.2014
=================
- automatically suspend cleaning spooldir if async buffer > 50% to not kill all I/O and unsuspend if < 10%
- remotepartyid option is set to yes the SIP Remote-Party-ID is taken into account when storing caller/called number into the database. If callerid is anonymous and Remote-Party-ID is present the number is always taken from Remote-Party-ID if present regardless on the remotepartyid option. Default option is no.
- fix issues with memory allocation failed (if not enough memory)
10.0.39 26.9.2014
=================
- fix -v 0 parameter causing FULL BUFFER
- limit SIP REGISTER SQL queue to 1000000 then drop them to not cause buffering it until all ram is used.
- fix live sniffer for cloud mode
10.0.38 17.9.2014
=================
- fix reported packet loss (>65000) for calls with more than 2 RTP streams (bug introduced since 10.0.36)
10.0.37 12.9.2014
=================
- static builds are compatible with 2.6.18 kernels again
- fix active calls realtime audio monitor chopped sound - increase length of ringbuffer for chunkplayer
10.0.36 11.9.2014
================
- fix ignoring RTP streams in case there are more legs and MARK bit sets
- fix timezone issues for register failed when sniffer is in different timezone then mysql server
- do not print to syslog when audio fails to decode because of missing RTP
- fix duplicate errors for max_allowed_packet
- add column message.content_length
10.0.35 9.9.2014
================
- maintainance build (compiled with static libnss to prevent crashing on buggy libc)
10.0.34 9.9.2014
================
- fix G723 packetization - if not 30ms audio is out of sync
10.0.33 4.9.2014
================
- fix 10.0.28 - 10.0.32 versions where if RTP with the same SSRC changes destination IP it was treated incorrectly as new RTP stream causing low MOS score in some cases.
10.0.32 2.9.2014
================
- fix 10.0.30 and 10.0.31 which did not recognized called or caller RTP stream in some cases
10.0.31 2.9.2014
================
- fix create SQL table fraud_alert_info
- fix mysql loop connection has gone if error happens during mysql connect
10.0.30 1.9.2014
================
- fix case where call has multiple RTP streams (caller <--> rtp proxy A <--> rtp proxy B <--> called) - caller side was not decoded properly
- fix FAX calls (fax codec was not stored as payload number 1000)
- import configuration files from /etc/voipmonitor/conf.d/*
10.0.29 1.9.2014
================
- handle memory allocation failure when there is no free RAM (do not crash)
- fix support for multiple contacts for SIP REGISTER instead of replacing it with the latest REGISTER.
10.0.28 28.8.2014
=================
- fix problem with multiple SIP messages in one TCP message resulting in garbled SIP response codes
- it can happen that next SIP hop reuses the SDP unique session identifier but changing port and ip which results to replace it in the first hop in voipmonitor RTP tracker which resulted in non capturing some RTP packets. This version ensure that SDP unique RTP session is replaced only within the same SIP source IP
- fix content-type value in SIP MESSAGE db
- add --sipport command line option
- fix reading pcap with linux cooked header (when sniffing with -i any)
- modfy column type message in table message from TEXT to MEDIUMTEXT (only for table create) to support messages > 64kb
- fix issue with negated IP addresse in alerts when they are used in IP groups and not specified directly
- Set "487 Request Terminated CANCEL" ONLY if the last SIP message is CANCEL without further response messages
10.0.27 27.8.2014
=================
- fix all 10.X which can corrupt heap (if saving sip packets) when doing IP defragmentation
- fix 10.0.26 crashes and fix bogus packet message
- fix all versions bug where ARP packets and other IP protocols can cause undefined behaviour
- fix hide SIP message body in MESSAGE packet only if set opt_hide_message is set
10.0.26 21.8.2014
=================
- handle corrupted IP header packet and do not crash the sniffer
- fix header_ip address for large sip message method packets (VG-326)
- fix issue with alter table cache_number_location
10.0.25 21.8.2014
=================
- fix 10.0.24 dscp fix which caused immediate crash
10.0.24 20.8.2014
=================
- fix issue with dscp and multi rtp streams
- remove leading zeros when comparing number as international.
10.0.23 20.8.2014
=================
- fix low MOS score for all codecs with sample rate > 8khz (all wideband) and probably audio decoding
- fix no RTP stream when using SIP compat headers (content-lenght: = l:)
- trim long packet (reassembled from tcp) to max 65535 bytes
10.0.22 18.8.2014
=================
- when auto upgradeing sniffer close all connections properly so senders can reconnect
10.0.21 15.8.2014
=================
- compiler fix
10.0.20 15.8.2014
=================
- fix low MOS score when stream was properly paused > 20sec
10.0.19 14.8.2014
=================
- fix low MOS score for G729 silence suppresion / CNG
- in multiple sender one receiver (mirroring) prevent mixing RTP stream from multiple sniffers and prevent mixing DLT mixing in one pcap file.
10.0.18 12.8.2014
=================
- force RTP resync in case of reINVITE which fixes false packet loss detection and low MOS score.
10.0.17 12.8.2014
=================
- adjust jitterbuffer f1 simulator to not lower MOS score for latencies <50ms
10.0.16 7.8.2014
================
- fix cleaning of active REGISTER which did not expired
- fix clean_obsolete_dirs which cleaned more than it should (not enabled by default)
10.0.15 7.8.2014
================
- fix zombie calls when sniffer is loosing packets. Add new sipwithoutrtptimeout so rtptimeout can be keep at reasonable values (300). See new config/voipmonitor.conf comments.
- close call if there is no 18X or 2XX responses
- fix search country by phone prefix
- remove useless debug message "prevent recursion of connect to db"
- fix dtmf reorder when storing dtmf to database
- fix http issue with 100-continue before 200-OK with eq ack at end stream
- fix false high PDV in case there are DTMF frames
- fix IP capture rules which matches rules according to best fit based on higher network mask. for example 192.168.0.1/32 rule will match last if there is also rule 192.168.0.0/24.
10.0.14 5.8.2014
================
- do not report packet loss if RTP mark bit it set
10.0.13 5.8.2014
================
- fix: if RTP stream starts with CNG frame and seq number is > 0 it detected huge false packet loss and astronomic avg jitter.
10.0.12 5.8.2014
================
- fix MOS score in case there is large packet loss gap - the MOS did not reflected it at all
- fix last sip response code is always 0 if the call was canceled before the ring.
10.0.11 31.7.2014
=================
- fixes fixed jitterbuffer implementation - all versions. Certain packet loss patterns caused high MOS score.
10.0.10 30.7.2014
=================
- fix condition for ignore local ip in fraud alerts
- add sipports command to manager
- fix set local country for local number in fraud
- log to syslog when removing files in spooldir
- supress warning about ddos confusing message
10.0.9 28.7.2014
================
- automatically enable fraud modules
10.0.8 23.7.2014
================
- fix 10.0.6,10.0.7 release (crashes)
10.0.7 23.7.2014
================
- fix 10.0.6 release (crashes)
10.0.6 22.7.2014
================
- fix SQL queue which stuck when max_allow_packet in mysql is default 1MB (which is by default)
- do not add another IP/port but update the old one taken from SDP on RE-INVITE or other cases with SDP - take the unique session id from SDP header for matching which fixes problem with multiple RTP streams in calls which are not related to the call. Fixes all versions. On high throughput this would lead to less CPU and less I/O.
- fix offline reading pcap files - sql handler was terminated sooner
- fix: run pcapcommand after all packets are written to pcap file
- reset jitterbuffer simulators when receving DTMF to not affect MOS score negatively
- fix audio sync issue for g.711 CNG frames
- add siren7 decode (G.722.1) support
10.0.5 14.7.2014
================
- fix filter in live sniffer
- optimize query for select min id in database backup
- fix typo in configure script
- add plcdisable config option and --plc-disable cmd line for G711
- add information about sniffer upgrade file size to error log
- Fixed install command for libcurl4-openssl-dev
- add update_dstnum_onanswer option
- try to reconstruct both audio if 200 OK is missing (or 18X with SDP)
- support Yes or YES in voipmonitor.conf
10.0.4 30.6.2014
================
- fix high MOS scores for fixed50 and fixed200 MOS scores. Fixed50 acts like 250ms and fixed200 like 400ms. (9.3 regression - broken since 5.5.2014)
10.0.3 27.6.2014
================
- fix mirroring between 32bit and 64bit sniffers (9.3 regression)
10.0.2 27.6.2014
================
- fix mos score for g729 (if enabled in voipmonitor.conf which is not recommended anyway) which was always high even for huge packet / PDV loss
10.0.1 24.6.2014
================
- fix GRE protocol (10.0 regression)
- fix IP defragmentation
- fix crashing when decoding to WAV (10.0 regression)
- redesign timestamp reorder warning (10.0)
- fix SIP SMS hide message capture rules
10.0 19.6.2014
==============
after several months of releasing 9.4 and 10.0RC/BETA versions there is new stable version 10.0 with many enhancements and bug fixes. The most interesting part is transparent pcap file compression and saving packet optimizations which allows to store thousands of simultaneous calls which was not possible before. The sniffer is now able to write pcap files for very high number of calls per second (>6000 / sec). Read the full changelog below:
Optimizations
-------------
- reimplement saving pcap files and add native gzip pcap and graph files which is enabled by default. Zipping is done in threads which dynamically grows. Use open/write sys calls instead of libc fwrite/fopen/fclose to allow store files at >6000 calls/second rate (impossible with libc fwrite/fclose functions). Any writes to file are asynchronous by default using different queue which will not block processing packets. New parameters in config/voipmonitor.conf - pcap_dump_bufflength, pcap_dump_zip, pcap_dump_writethreads, pcap_dump_writethreads_max, pcap_dump_asyncwrite, pcap_dump_asyncbuffer. Those optimizations does not require cachedir for very high I/O loads now.
- properly dequeue remote sensors by the timestamp to prevent race conditions or if sniffing on interface = eth1,eth2
- Optimize SIP parser which is now much faster
- do not try to parse callid on UDP packet if it cannot be SIP (can save some CPU)
- save few kB for each jitterbuffer channel (3 per RTP stream) - removed unused structures
- release calls from memory sooner and do not wait on rtptimeout for some cases
- end calls sooner than the rtptimeout if there is no further communication after 401 SIP response
- be more tolerant in jitterbuffer simulater to burst RTP packets - add 200ms more so the MOS is not negatively affected
- if mysql procedure fails run repair table mysql.proc which is the most common source of the issue
- implement new mysql fine tunning options - mysqlstore_concat_limit and mysqlstore_concat* - see voipmonitor.conf for more details
- modify packetbuffer log drop statistics (heapoverrun and drop packets on interace)
- repair table mysql.proc at every sniffer start (it is broken after hard restart very often...)
- shorten register sip message timeout from 30 to 5 seconds (helps to manage RAM when there is REGISTER flood) and add option sip-register-timeout so it can be set to custom seconds
- implement autocleaning to prevent 100% disk space usage. in case the space is below 1% and below 5GB (which is default threshold) reindexfiles procedure will be executed and cleaning will be restarted. In case this will not help the new maxpoolsize will be set to size of current spool directory and will keep free space MIN(1% freespace, 5GB). Default is autoclean enabled, 1% free space or 5GB free space. New configuration options: autocleanspool = yes, autocleanspoolminpercent = 1, autocleanmingb = 5
- if filesindex directory is missing at all or there are no data past 12 hours reindexfiles will be initiated on sniffer start
- use old partition schema even if mysql is upgraded from 5.1 to 5.6 - it autodetects old partitions.
Bugfixes
--------
- SIP 200 packet was never saved if it was not BYE or INVITE - all 200 to CANCEL were missing. (probably all versions)
- if RTP src/dst port was the same as sipports the RTP was ignored (fixes all versions)
- do not end call prematurily on 481 transaction does not exist
- fix beeping in silence sequences for G711 ALAW/ULAW (all versions)
- fix g729 garbled audio for annex B / cng
- fix ignoring RTP if the first RTP is TELEVENT (flash/dtmf etc) leading to silence in audio recording
- fix if device wants to unregister (send REGISTER expires=0) and server replies 200OK expires>0 the sniffer did not deregister it from active calls
- fix reassembling TCP which would lead to crash
- fix bug causing deadlock on 32bit
- fix race condition leading to crash in get sniffer status via manager
- fix crash in case the call is one-way SIP and RTP frames are held in heap due to I/O
- do not end sniffer when sending mirroring traffic to receiver and the connection resets - properly reconnect once the receiver is up again.
- fix pcap files when sending traffic from different interfaces (TUN / ETH) to one receiver
- fix indication in CDR when packet loss in kernel / heap is detected so in the GUI CDR shows the loss might be not real
- ensure that the left channel of the was is always caller
- take size of files from OS sys calls and do not rely on pcap caplen which fixes bug in cleaning which thinkgs there is much more space consumed
- fix audio if caller/called changes codec from one sample rate to another
- fix low MOS score in case call changes packetization from 10 to 20 and back to 10 which is pretty rare but it can happen
- fix SIP TCP reassemble ignoring some TCP messages in SIP (all versions)
- fix SIP TCP reassemble if the first SIP packet does not contain Call-ID it was not tracked at all.
- fix SIP TCP reassemble duplicate TCP packet - last fragment was stored twice
- fix racecondition when storing multiple calls from file - some of calls might not be saved due to preliminary end of the program not waiting to finish all queues
- fix crash if SIP type MESSAGE has larger content-lenght than the whole message.
- fix skinny skip flag
- resolve mysql host only in main thread which fixes issues on some systems which crashes when resolv is in multiple threads. The static binary is now compatible with all centos versions again.
- fix parsing DSCP bit
- fix race condition leading to crash when saving to wav is enabled
- fix regression for all versions >9.1 when (and only for) G729 RTP streams were not scored by MOS at all and although there were loss or delays the MOS was always 4.5.
- fix typo in voipmonitor.conf - onowaytimeout -> onewaytimeout
- fix small memory leak in case there is T38 in SDP (all versions)
- fix redundant reconnect to db - flush previous query before try mysql_ping (problem since 9.1)
- fix skinny racecondition crash and add support for more version headers for some messages
- fix skinny called/calling swapped
New features
------------
- add support for capture rules by domain
- parse multiple SIP messages carried in one TCP packet
- add call per second and sip packets per second to the syslog info line
- if the call timeouts due to absolute timeout mark this in cdr.bye - 103 so the GUI can alert / see it
- support reading compressed pcap files (-r)
- add G722.1 and AMR codec support
- add option only_cdr_next for save cdr only to cdr_next table
- enable merging calls based on matching call-id. The idea is that the second leg with different call-id has in the first INVITE special header which has the call-id of the parent SIP call. The configuration option specifies name of the header.
- add syslog messages when upgrading sniffer
- add connect_duration to listcalls manager command
- add new configuraiton option absolute_timeout (default set to 4 * 3600s) which ends call longer than 4 hours by default.
- add new configuration option destroy_call_at_bye which ends call after 20 minutes (default 20*60) no matter how many RTP/SIP packets arrives after the BYE.
- parse custom headers also for SIP 2XX methods
- replace wget with libcurl when doing autoupgrades from the manager
- implement saving audio in ogg / stereo / various sample rate (new config saveaudio_stereo default is stereo now). Add new option ogg_quality so user can control quality of the ogg file
- allow RTP stream to be assigned to two independent calls with different call-id.
- store RTP mark bit into .graph file so the GUI will show where mark bits are set
- implement new cdr.a_last_rtp_from_end and cdr.b_last_rtp_from_end which stores number of seconds from last RTP packet to the end of call. This will be used to find calls where any of side ends RTP stream sooner. Such situation is not yet detectable without knowing those parameters.
- add support for other GRE encapsulation types (for IP protocol)
- when mirroring packets send id_sensor and DLT number on connect so configuration option mirror_bind_dlt is not needed anymore and allows to set id_sensor for each sender. It is not backward compatible with older versions thus the receiver must be >= 9.4 beta1
- add time shift information to syslog if detect bad packet order
- sip_send_ip = 127.0.0.1 (or 127.0.0.1:12345) sip_send_port = 12345 will send text SIP messages to TCP socket
- implement skip flag also for cisco skinny protocol
- sccp skinny: record RTP stream for some cases where signalization does not contain conference number = call reference which fixes ampty RTP
- use only first detect value for custom headers or last if set configuration value custom_headers_last_value
9.3 10.2.2014
=============
- fix crashing when decoding audio in certain cases and also when doing realtime listening
- do not connect to database if nocdr is set (-c)
- fix crash if non ascii is put into telnum filter
- fix freebsd compile
- implement storing rtp port to cdr_rtp database (cdr_rtpport option must be set to yes and it will upgrade database)
9.2 31.1.2014
=============
- do not put CNG frames into audio decoders which causes garbled g729 stream and causes low MOS score even thought there is no packet loss at all
- use separate MySQL connections for live packet sniffer and for storing REGISTER messages which adds parallelization and speedups INSERTS
- fix ipaccount for GRE protocol
- add --no-cache parameter to wget in download upgrade
- fix issue with double semicolon in sql query
- add support for DLT_IEEE802_11_RADIO reading pcap files and sniffing on raw wifi
9.1 29.1.2014
=============
- Fix MySQL deadlock in case the MySQL connection timeouts. This bug started since 8.4 - 9.0 verisons.
- Fix storing to MySQL if file is read from file (bug introduced in 8.4 and 9.0 versions) and do not fork when reading from file
- Optimize failed registrations which caused mysql overload in case of register flood
- fix configuration issue in case the datbase is partitioned and cdr_partition was not set
- fix MySQL indefinit reconnect in case the sniffer is stopping or starting which causes hang in linux bootup / shutdown process.
- fix undefined codec in case of CNG (comfort noise) frames.
- add new option saveaudio_reversestereo which reverts left and right channels
- add new manager command pcapstat which prints ringbuffer packet drop counter
- fix issue in interface option when list of interfaces contains spaces like: interface = eth0, eth1
- support mysql 5.0
9.0 17.1.2014
=============
Version 9.0 is formerly 8.4RC27 and this changelog sumarizes all changes since 8.3 (all features from 8.4)
Optimizations
-------------
- do not create RTP pcap file for every sip call but only if there is actually RTP which saves a lot of I/O
- take min MOS score from fixed2 jitterbuffer simulator by default which eliminates low mos score due to f1 and adapt sensitivness
- do not flood log with "to much INVITEs in this call" show the message only once increase MAX_IP_PER_CALL from 30 to 40
- Close SIP and SIP+RTP dump files ASAP to save some number of opened files and memory (Rob Gagnom)
- optimize thread0 packetbuffer size
- disable sendfile flood if not present in old kernel
- modify install script which will ask if the user wants to overwrite configuration file
- optimize SIP MESSAGE sql storing which accidentally access database for each SIP MESSAGE which leds in queueing sip message types
- do not create partitions every hour but every 12 hours
- add option disable_partition_operations which will not create partitions (in multisniffer environment all other sensors should have disabled partition creation which is redundant)
- enhance drop old partitions
- check if another instance of voipmonitor is running on background based on name of the configuration file (so more instances can still run on background with different configs)
- in case call is closed with 403 response shorten close window to 5 seconds so that call will not timeout on rtp_timeout (which is 5 min by default).
- change mtune and march to native in Makefile
- calls which end with 4XX or 5XX was not released from memory immediatelly and was waiting for rtptimeout if sip source port differs from the first INVITE.
Bugfixes
--------
- do not parse SIP data after content-lenght = X. This fix was already done before but only if verbose > 1
- save to register_failed all REGISTER which is not success and not only if the password is wrong.
- fix DSCP (reversed) and not working if pcapscandir is used.
- do not terminate call prematurely when SIP 501 "Not Implemented" is received
- fix spooldir cleaning (broken in 8.3 version) and reindexfiles which has problem if directory has more then 2GB
- fix potential crash in RTP processing
- fix crash on some 32 bit platforms
- fix audio decoding for comfort noise and dtmf - do not create new RTP instance if payload is one of 13,19,101
- fix callslimit which did not worked at all
- fix ODBC driver (tested for mssql) which was not maintained for a longer time
- fix traffic speed in verbose output
- If cdr proxy is enabled (which is by default) fix called IP in cdr and false proxy record in proxy column in case called make reintive. Fix proxy in case there is INVITE to fail-over - write the last IP of the last SIP and not the first (failed) one. Fix hangup indication (cdr.whohangedup) in case call travers through proxies (called was marked as hanged up always)
- following issue was fixed only when reading pcap files but not in realtime sniffing - prior verison 8.0.1 deduplicate (which is default off) was comparing only data without ip header and udp header so duplicate packets was matched also in case when IP addresses differs. This was good for some cases but it leads to completely ignore RTP streams in other cases. Now default option is to check duplicates based on not only data but ip headers too. To change this set deduplicate_ipheader = no. default = yes.deduplicate_ipheader = yes. fix recent deduplication fix which was wrong and it leads to remove packets which were not duplicated because it took less data than actual data (12 bytes less). This fixes T38 decoding in GUI for example.
- do not send SID frames to G729 decoder which fixes garbled sound for G729 RTP streams.
- escape database name in sql queries ('-' caused problems)
- fix detect partitions in cdr% tables when upgrading from older versions
- fix convert_dlt_sll2en10 parameter
- do not flood syslog when RTP files cannot be created
- fix IP header len when defragmenting packet which prevents to see some SIP packets in SIP history (and in wireshark).
- fix SDP: sometimes SDP could not be parsed if the IP was the last line in SDP in the packet thus ignorind the whole RTP
- fix parsing tags which can be terminated also by only new line instead of CRLF
- fix rare issue with new invite after last sip response 487
- do not report mysql error when recreating mysql stored functions. do not drop / create those functions if they are not different
- fix parsing custom headers for sip methods RES3XX, RES4XX, RES5XX, RES6XX
- fix incorrect 200 OK last response code when CANCEL is confirmed by 200 OK without request terminated.
- if voip SIP device (with source IP 1.1.1.1) sends in SDP that it wants RTP for example to 10.0.0.1 and port 10000 the call also sniff RTP from 1.1.1.1:10000 or to 1.1.1.1:10000 which does more problems than it solves. Now this behaviour is changed and if you need this option back you can enable it sdp_reverse_ipport=yes. This change fixes RTP packets assigned to wrong call and other weired problems.
- fix crash in case the sniffer run outs of file descriptor
- DTMF events are not always 101 - take it from the SDP which fixes audio sync and g729 garble issues and many graph files in case telephone events payload is not 101 (typically 100)
- do not try connect to mysql indefinitely when the sniffer is started. This was problem if mysql does not run and voipmonitor starts at boot which stucked the whole booting process.
- if the call is not connected it was supposed to set -1 to connect_duration but the column is unsigned thus it will end up with 0 - this fix will set NULL if call is not connected (as well as first_rtp_time and progress_time)
- automatically disable partitions for mysql <= 5.0
- fix mysqlcompress configuration option which has been always disabled.
- fix if sniffer is terminated by TERM signal (service voipmonitor stop) and there were some calls in sql queue and the mysql server has gone the sniffer tries indefinitely to connect.
- fix reloading capture rules which dissapeared due to mysql race condition
- fix mysql issues causing random corruption when doing reload
- fix long term issue (all versions) where the .graph file in CDR detail (in the GUI) suddenly jumps if sequence numbers changes either due to packet loss or due to VAD. Now graphs should not move up/down suddenly
- fix low MOS score for adaptive jitterbuffer in case a SIP session reinvites and RTP packets does not correctly increases sequence numbers or timestamps
- fix low MOS score for adaptive jitterbuffer even for 5 packet loss caused by normal shrink/grow internal jitterbuffer simulator.A
- fix inappropriate packet loss report if a SIP session reinvites and RTP stream with the same SSRC has hole which is normal (although the wireshark still report packet loss as well as any RFC RTP compliant RTP analyzer)
New features
------------
- implement new cdr columns - sipcallerport, sipcalledport - if configuration variable cdr_sipport is set to yes
- implement destination_number_mode which will take number from INVITE URI or To: SIP header. If destination_number_mode = 1 it will always save number from To: header. If destination_number_mode = 2 it will take number from INVITE URI. default: destination_number_mode = 1
- When scanpcapdir is being used, voipmonitor will by default read any new file that is created in that directory as soon as the file is closed. This is the normal setting if your packet capture software is tcpdump. If you are using a different packet capture software, you may need to change this setting to "rename" if that software writes to a temporary file, closes it, and then renames it to something else after the file is closed. Default setting is "newfile" (Rob Gagnon)
- packetbuffer mirroring - add support for multiple sender connections to one central receiver. Now the receiver will accept unlimite number of connections at the same time.
- support compile on FreeBSD (tested on FreeBSD 8.3)
- implement reading from list of interfaces which is better alternative to interface = any. Now it is possible to sniff on eth0 and eth2 within one sniffer - interface = eth0,eth2. Reading is done in separate threads which takes benefit over interface = any
- if SSRC changes during call and there is another SSRC change to previous one and the RTP packet has the same seq and timestamp does not reflect the gap - do not report huge packet loss
- implement GRE tunnel de-encapsulation
- parse caller and called number in function save_live_packet if call is NULL which allows to filter OPTION and SUBSCRIBE in GUI live sniffer
- Implement multipart/mixed SIP messages which voipmonitor ignored if there was SDP (for example SIP messages with SDP and ISUP)
- implement binary format for graph file which makes *.graph files 5 times smaller which greatly reduces disk space and I/O operations. This needs the latest GUI
- implement autocleaning in cachedir and automatically move files when sniffer was interrupted and restarted
- add pcap size limit - configuration option maxpcapsize (in MB)
- add upgrade, restart and sniffer stats via manager
- add option create_old_partitions and create_old_partitions_from to create old partitions
- add uptime to sniffer statistics
- for caller number use tag remote-party-id (if exists) and caller contain anonymous so the CDR still contains number of the caller
- SCCP: add support for CM7 header version (missing RTP)
- SCCP: add CM5CALL_INFO_MESSAGE 0x014A type so caller/called numbers are set
- implement ITU-T P.862 PESQ. pesq binary must be installed. mos_lqo in voipmonitor.conf must be enabled. in capture rules tables filter_ip or filter_telnum wav and mos_lqo must be anebled - cdr.a_mos_lqo_mult10 and cdr.b_mos_lqo_mult10 added.
- new configuration option onewaytimeout ends sip call after set seconds which does not receive any reply from the other side. Technivally it takes sip source ip address from first invite/register and if there is no other sip source ip different from the first source ip the call will be terminated after onowaytimeout seconds. If a call is terminated due to this timeout in cdr.bye column will be 101. Purpose of this timeout is to release memory as soon as possible in case there INVITE flood which does not have any reply. Another reason is when mirroring is set incorrectly and the sniffer sees only one side of a SIP signalization. If onowaytimeout is not set default is 15.
8.4RC27 16.1.2014
=================
- fix mysqlcompress configuration option which has been always disabled.
- fix FreeBSD endiannes again. Now RTP works
- calls which end with 4XX or 5XX was not released from memory immediatelly and was waiting for rtptimeout if sip source port differs from the first INVITE.
- fix if sniffer is terminated by TERM signal (service voipmonitor stop) and there were some calls in sql queue and the mysql server has gone the sniffer tries indefinitely to connect.
- new configuration option onewaytimeout ends sip call after set seconds which does not receive any reply from the other side. Technivally it takes sip source ip address from first invite/register and if there is no other sip source ip different from the first source ip the call will be terminated after onowaytimeout seconds. If a call is terminated due to this timeout in cdr.bye column will be 101. Purpose of this timeout is to release memory as soon as possible in case there INVITE flood which does not have any reply. Another reason is when mirroring is set incorrectly and the sniffer sees only one side of a SIP signalization. If onowaytimeout is not set default is 15.
- support sccp skinny header ver. 0x17
8.4RC26 10.1.2014
=================
- do not try connect to mysql indefinitely when the sniffer is started. This was problem if mysql does not run and voipmonitor starts at boot which stucked the whole booting process.
- fix clean partitions if cleandatabase is set
- if the call is not connected it was supposed to set -1 to connect_duration but the column is unsigned thus it will end up with 0 - this fix will set NULL if call is not connected (as well as first_rtp_time and progress_time)
- check if another instance of voipmonitor is running on background based on name of the configuration file (so more instances can still run on background with different configs)
- in case call is closed with 403 response shorten close window to 5 seconds so that call will not timeout on rtp_timeout (which is 5 min by default).
- fix freebsd endianess detection which resulted in ignoring RTP packets
- change mtune and march to native in Makefile
- automatically disable partitions for mysql <= 5.0
8.4RC25 8.1.2014
================
- fix clean partitions when opt_cleandatabase is set
- avoid race condition leading to crash when cleaning regfailed class (since 8.4RC24)
- print error when any of sniffing interface fails to initialize
- fix long startup of the sniffer optimizing mysql query checking cleaning
- document cdr_sipport option in voipmonitor.conf which was added recently
8.4RC24 3.1.2014
================
- fix memory leak in adaptive jitterbuffer if it is reset (leak since 8.4RC versions)
- DTMF events are not always 101 - take it from the SDP which fixes audio sync and g729 garble issues and many graph files in case telephone events payload is not 101 (typically 100)
8.4RC23 2.1.2014
================
- cache register_failed queries to not flood mysql database in case of register failed flood attack
8.4RC22 30.12.2013
==================
- do not parse SIP data after content-lenght = X. This fix was already done but only if verbose > 1
- implement new cdr columns - sipcallerport, sipcalledport - if configuration variable cdr_sipport is set to yes
- implement ITU-T P.862 PESQ. pesq binary must be installed. mos_lqo in voipmonitor.conf must be enabled. in capture rules tables filter_ip or filter_telnum wav and mos_lqo must be anebled - cdr.a_mos_lqo_mult10 and cdr.b_mos_lqo_mult10 added.
8.4RC21 19.12.2013
==================
- SCCP: add CM5CALL_INFO_MESSAGE 0x014A type so caller/called numbers are set
- fix sip register in caase REGISTER-OK (no 401 or 404) - broken since 8.4RC20
- fix crash in case the sniffer run outs of file descriptor
- fix DSCP RTP in CDR in case pcap is read by pcapscandir
- optimize SIP MESSAGE sql storing which accidentally access database for each SIP MESSAGE which leds in queueing sip message types
- modify install script which will ask if the user wants to overwrite configuration file
- add support for CM7 header version (missing RTP)
8.4RC20 16.12.2013
==================
- save to register_failed all REGISTER with only one REGISER meesage and also all REGISTER with one 401 or 403 reply (which were not stored to register failed)
- fix sip dscp which was reversed
8.4RC19 13.12.2013
==================
- fix wget issue - move parameter --no-check-certificate to first after command wget
- set FOREIGN_KEY_CHECKS=0 before drop table
- fix potential issue with small size of packetbuffer
- use id_sensor when doing reindexfiles which was deleteing records for other sniffers
8.4RC18 12.12.2013
==================
- for caller number use tag remote-party-id (if exists) and caller contain anonymous so the CDR still contains number of the caller
- do not reindex files repeteadly if the last cdr is old.
- fix freebsd compilation on 32bit
- do not terminate call prematurely when SIP 501 "Not Implemented" is received
- implement destination_number_mode which will take number from INVITE URI or To: SIP header. If destination_number_mode = 1 it will always save number from To: header. If destination_number_mode = 2 it will take number from INVITE URI. default: destination_number_mode = 1
- if SSRC changes during call and there is another SSRC change to previous one and the RTP packet has the same seq and timestamp does not reflect the gap - do not report huge packet loss
- disable sendfile flood if not present in old kernel
- suppress warning in query SET GLOBAL innodb_stats_on_metadata=0
- do not print "End of pcap file, exiting" to syslog
- optimize thread0 packetbuffer size
8.4RC17 5.12.2013
=================
- packetbuffer mirroring - add support for multiple sender connections to one central receiver. Now the receiver will accept unlimite number of connections at the same time.
- fix incorrect 200 OK last response code when CANCEL is confirmed by 200 OK without request terminated.
- if voip SIP device (with source IP 1.1.1.1) sends in SDP that it wants RTP for example to 10.0.0.1 and port 10000 the call also sniff RTP from 1.1.1.1:10000 or to 1.1.1.1:10000 which does more problems than it solves. Now this behaviour is changed and if you need this option back you can enable it sdp_reverse_ipport=yes. This change fixes RTP packets assigned to wrong call and other weired problems.
- When scanpcapdir is being used, voipmonitor will by default read any new file that is created in that directory as soon as the file is closed. This is the normal setting if your packet capture software is tcpdump. If you are using a different packet capture software, you may need to change this setting to "rename" if that software writes to a temporary file, closes it, and then renames it to something else after the file is closed. Default setting is "newfile" (Rob Gagnon)
8.4RC16 3.12.2013
=================
- support compile on FreeBSD (tested on 8.3)
- fix parsing custom headers for sip methods RES3XX, RES4XX, RES5XX, RES6XX
- do not report mysql error when recreating mysql stored functions. do not drop / create those functions if they are not different
- fix rare issue with new invite after last sip response 487
- Close SIP and SIP+RTP dump files ASAP to save some number of opened files and memory (Rob Gagnom)
8.4RC15 29.11.2013
==================
- fix excessive loss report for some cases in case of reinvite (regression since 8.4RC12)
- fix SDP: sometimes SDP could not be parsed if the IP was the last line in SDP in the packet thus ignorind the whole RTP
- fix parsing tags which can be terminated also by only new line instead of CRLF
- add uptime to sniffer statistics
- enhance drop old partitions
- add option create_old_partitions for create old partitions
8.4RC14 27.11.2013
==================
- do not flood log with "to much INVITEs in this call" show the message only once increase MAX_IP_PER_CALL from 30 to 40
- add upgrade, restart and sniffer stats via manager
- use global db handler sqlDbCleanspool for all cleanspool operations
8.4RC13 21.11.2013
==================
- fix IP header len when defragmenting packet which prevents to see all data in SIP history and in wireshark.
- fix parsing of useragent - broken since 8.4 RC4
8.4RC12 18.11.2013
==================
- fix reloading capture rules which dissapeared due to mysql race condition
- fix mysql issues causing random corruption when doing reload
- fix long term issue (all versions) where the .graph file in CDR detail (in the GUI) suddenly jumps if sequence numbers changes either due to packet loss or due to VAD. Now graphs should not move up/down suddenly
- fix low MOS score for adaptive jitterbuffer in case a SIP session reinvites and RTP packets does not correctly increases sequence numbers or timestamps
- fix low MOS score for adaptive jitterbuffer even for 5 packet loss caused by normal shrink/grow internal jitterbuffer simulator.A
- fix inappropriate packet loss report if a SIP session reinvites and RTP stream with the same SSRC has hole which is normal (although the wireshark still report packet loss as well as any RFC RTP compliant RTP analyzer)
- take min MOS score from fixed2 jitterbuffer simulator by default which eliminates low mos score due to f1 and adapt sensitivness
- disable cleanspool functions if not set parameters
8.4 RC11 13.11.2013
===================
- do not flood syslog when RTP files cannot be created
- fix cleaning spool directory - delete files which are not indexed and run reindex in case of inconsistency
- add pcap size limit - configuration option maxpcapsize (in MB)
- implement autocleaning in cachedir and automatically move files when sniffer was interrupted and restarted
- fix convert_dlt_sll2en10 parameter
8.4 RC10 6.11.2013
==================
- fix MySQL race condition causing crashes and undefinite cleaning of parition each second when mysql cleaning enabled.
- fix ebug in syslog
- fix memory leak caused by cleaning which should not run when in mirror sender mode
- implement GRE tunnel de-encapsulation
8.4 RC9 1.11.2013
=================
- fix reindexfiles which has problem if directory has more then 2GB
- clarify that the id_sensor is number between 1 - 65535 (16bit number)
- fix potential crash in RTP processing
- implement reading from list of interfaces which is better alternative to interface = any. Now it is possible to sniff on eth0 and eth2 within one sniffer - interface = eth0,eth2. Reading is done in separate threads which takes benefit over interface = any
- fix detect partitions in cdr% tables when upgrading from older versions
- do not create partitions every hour but every 12 hours
- add option disable_partition_operations which will not create partitions (in multisniffer environment all other sensors should have disabled partition creation which is redundant)
- escape database name in sql queries ('-' caused problems)
8.4 RC8 8.10.2013
=================
- fix implementation of multipart/mixed SIP messages in 8.4RC4 which caused memory corruption and crashes.
8.4 RC7 8.10.2013
=================
- fix broken logic in RC6 which ignored all RTP
8.4 RC6 8.10.2013
=================
- do not create RTP pcap file for every sip call but only if there is actually RTP which saves a lot of I/O
- fix crash on some 32 bit platforms
- fix recent deduplication fix which was wrong and it leads to remove packets which were not duplicated because it took less data than actual data (12 bytes less). This fixes T38 decoding in GUI for example.
8.4 RC5 2.10.2013
=================
- do not send SID frames to G729 decoder which fixes garbled sound for G729 RTP streams.
- implement binary format for graph file which makes *.graph files 5 time smaller which greatly reduces disk space and I/O operations. This needs the latest GUI
8.4 RC4 30.9.2013
=================
- Implement multipart/mixed SIP messages which voipmonitor ignored if there was SDP (for example SIP messages with SDP and ISUP)
- following issue was fixed only when reading pcap files but not in realtime sniffing - prior verison 8.0.1 deduplicate (which is default off) was comparing only data without ip header and udp header so duplicate packets was matched also in case when IP addresses differs. This was good for some cases but it leads to completely ignore RTP streams in other cases. Now default option is to check duplicates based on not only data but ip headers too. To change this set deduplicate_ipheader = no. default = yes.deduplicate_ipheader = yes
- Do not add SIP destination IP to proxy table if INVITE fails and proxy tries another destination
- fix hangup indication (cdr.whohangedup) in case call travers through proxies (called was marked as hanged up always)
- if cdr proxy feature is enabled (which is by default) and invite goes to fail-over write the last IP of the last SIP and not the first (failed) one.
- try to fix more files cleaning issues
8.4 RC3 27.9.2013
=================
- repack statically 64bit binary with the latest libc version to solve crashes on centos and other systems
8.4 RC2 25.9.2013
=================
- fix errors in syslog when moving files from cachedir / spooldir and fix potential clean procedure leaks
- If cdr proxy is enabled (which is by default) fix called IP in cdr and false proxy record in proxy column in case called make reintive
8.4 RC1 23.9.2013
=================
- fix ODBC driver (tested for mssql) which was not maintained for a longer time
- fix traffic speed in verbose output
- parse caller and called number in function save_live_packet if call is NULL which allows to filter OPTION and SUBSCRIBE in GUI live sniffer
- fix callslimit which did not worked at all
- reorganize addtofilesqueue which should solve some graph files left by cleaning process
- fix audio decoding for comfort noise and dtmf - do not create new RTP instance if payload is one of 13,19,101
8.3 11.9.2013
=============
- lock mysql.connect function which causes crash in statically linked binary
- implement filter_ip|telnum.script flag - this will run shell script for call matching the filter rule (new filtercommand option).
- add cachedir queue and throughput statistic to syslog. Example - cdq[10][1.1 MB/s] 10 files in queue waiting to be moved, 1.1MB/s is average transfer rate.
- Do not store body in SIP MESSAGE type after conte-lenght
- push REGISTER mysql queries into queue and release call structure immediately to not block memory. This fixes problem when sip register is enabled and mysql slows down for a while.
- index also zero lenght pcap and graph files so its deleted by cleaning procedure.
- fix audio sync issue in case silence detection is enabled and two consecutive mark bits are seen
8.2 9.9.2013
============
- add support for OPUS codec
- new option cdr_ua_enable (default yes) this option allows to skip storing cdr.a_ua and cdr.b_ua - this is workaround for those who has extreme cdr rate and number of user agents in database cdr_ua table is over 1000 and CPU is not powerfull enough to store cdr in real time. In future this option will be removed once we optimize this rutine. default = yes
- call mysql_library_init() which would potential mysql crash on init (since 8.1)
- lock listMAP structure which fixes random crash when analyzing sip resigters and looking at active calls at the same time (all versions)
- fix mysql queries in queue if importing pcap file
8.1.2 5.9.2013
==============
- implement reindexfiles manager command which is needed to recover cleaning when database deleted or index files were deleted
8.1.1 5.9.2013
==============
- fix 8.1 mysql crashing
- fix deadlock when heap is full
8.1 4.9.2013
============
Optimizations
-------------
- remove Call structure immediately and do not wait for CDR to be written. This allows buffering milions of SQL INSERTS with much less memory footprint.
- signifficantly speedup startup : set global innodb_stats_on_metadata=0; Slow query on information_schema.tables
Bug fixes
---------
- fix capture rules which did not matched if caller number matches partialy with some rule and there is called rules which should match but it will be never matched because internal b-tree cursor was not reset. this bug is in all versions < 8.1.
- packetbuffer fixes: fix indication heapoverrun, fix spam indication MEMORY IS FULL in syslog, add indication about processing block size in heap statA
- fix new cleaning implementation
- set mysql flag CLIENT_MULTI_RESULTS which fixes PROCEDURE voipmonitor.create_partition can't return a result set in the given context
- prior verison 8.0.1 deduplicate (which is default off) was comparing only data without ip header and udp header so duplicate packets was matched also in case when IP addresses differs. This was good for some cases but it leads to completely ignore RTP streams in other cases. Now default option is to check duplicates based on not only data but ip headers too. To change this set deduplicate_ipheader = no. default = yes.deduplicate_ipheader = yes
New features
------------
- new option allow-zerossrc - SSRC in RTP headers must not be equal zero according to RFC so voipmonitor is ignoring such RTP by default. If you still need to parse such packets enable it in voipmonitor.conf allow-zerossrc =yes
- convert from dlt ssl to en10 for pcap save (via configuration parameter convert_dlt_sll2en10) - this is needed only if you need to merge two pcap files with different encapsulation
- add res and virt memory counters into syslog stat
8.0.1 23.8.2013
===============
- fix cdr proxy column which contains duplicates in case of sip reinvites
- disable default "filter = udp or (..." in voipmonitor.conf template file which does not work if interface = any
- In case the RTP stream within the same SSRC changes codec, split the stream so in sip history it will show the change.
- fix RTP stream in sip history which shows number of received packets - 2
- fix skipdefault and implement skip in capture rules by tel number (forgot to implement)
- fix new spool cleaning in case cachedir is enabled
- fix new spool cleaning which did not count graph file sizes (disk space was higher then set because of this)
8.0 21.8.2013
==============
Optimizations
-------------
- redesign internal packet buffering - packets are read from kernel ringbuffer into dynamically allocated buffer:
* two memory copy is removed saving ~10% cpu
* static sized vmbuffer is replaced by dynamically allocated heap buffer which is allocating up to set size. Size of this buffer is now not limited (vmbuffer was limited to 4GB). Default limit in configuration is 2GB
* buffer can be compressed in memory (~50% ratio) default enabled
* if memory buffer runs out of space disk buffer will be used (if enabled) and compressed (if enabled)
* buffer can be sent over TCP to another voipmonitor instance and compressed (if enabled) which is now new recommended option for software mirroring. In case the transmission is interrupted packets are will buffer into memory or disk (compressed if enabled).
* new debug information (if at least -v 1) informing about heap size and overruns and three main threads CPU utilization
- reimplement cleaning spool directory. find command was replaced by indexing and now it is possible to set total size or maximum days for all files and sip / rtp / graph / audio separately
- if libpcap reports packet loss, save number 100 to cdr.bye which will indicate that snifer was loosing packets
- add partitions for register_state and register_failed and message tables
- Remove dependency on mysql scheduler and create table partitions within sniffer service thread
- create database if it does not exists
New features
------------
- implement skip flag in capture rules which allows to ignore whole call based on IP and Telnum filters
- add last sip response into listcall
- implement new option cdrproxy = yes (enabled by default) : in case SIP session travels accross several proxies (and Call-ID header does NOT change) and you would like to track all sip proxies and make them searchable in GUI / database. If disabled in cdr.sipcalledip will be destination IP from the first INVITE. If enabled in cdr.sipcalledip will be destination IP from the last INVITE and all IP from middle INVITE will be inserted in cdr_proxy table. In the GUI is new proxy column.
- implement skipdefault option in voipmonitor.conf if = yes it will ignore all calls unless skip flag is true in capture rules (sql.filter_*)
- implement database partition auto cleaning (voipmonitor.conf)
- implement dscp configuration option which will add dscp to cdr table. The dscp column holds dscp ip flag for sip and rtp caller/called. GUI will follow soon.
- add support for CLEARMODE codec
- add custom_headers for SIP message
Bug fixes
---------
- fix audio play (garbled sound) remove data from RTP packets beyond IP datalen which causes garbage in RTP packet for codec decoders (broken in all sniffer versions). This is in case the ethernet frame has checksum at the end of packet.
- modify mysql procedure for create partition - suppress create partition if table does not exists or contains no partitions
- fix critical bug (all sniffer versions) with call_id boundary check which can cause crash.
- fix custom headers - remove limit 128 chars and crop to max size 255 chars
- fix capture rules - if there is NULL in column ignore the flag completely. This fixes issue when the rule is created and user wants to override only one flag and leave others untouched (NULL)
- log error and reconnect to mysql in case of "query error in [drop procedure if exists __insert_10]: Commands out of sync; you can't run this command now"
- fix memcpy sizeof on various places
7.1.3
=====
- Enable mysql scheduler on mysql connect "SET GLOBAL event_scheduler = 1" which is needed to create partitions for cdr table.
7.1.2
=====
- fix savertp = header which should store only RTP headers and not full payload. Fix override in capture rules in filter_ip sql table (and capture rules in GUI) if RTP is turned off it will not store RTP even with savertp = header. If RTP is enabled it will store full RTP packets including audio even savertp = header.
7.1.1
=====
- fix writing first batch of CDR after reconnecting to database. -> MySQL server has gone away -> Commands out of sync; you can't run this command now
- when receiving INVITE after BYE extend call timeout about 1 minute after each new packet and not after rtptimeout
- fix not closing call in some corner cases - it can happen that the call is in memory for a long time
7.1
===
- do not parse SIP packets after content-lenght. Some implementations sends bigger packet with garbage at the end of packet which should not be parsed causing overwriting headers
- fix error in mysql insert for very rare cases
- output mysql queries into syslog instead of cout (if verbosity > 1)
- fix -s parameter which did not worked at all
- run readdump_libpcap in separate thread to be able to read real % for libpcap reading thread
- add qring status into syslog state
- when upgrading from older sniffer do not create new tables partitioned when the cdr table is not partitioned which will cause stop working writing CDR with RTP or DTMF next day
- fix T38 for case when the same RTP port is reused in reintive which was treated as no t38 in this case
- implement inotify events for pcapscandir which means that now the name of the pcap file in this directory can be any name which means more pcap files can be put there from various sources - events are filtered only when the file is closed for write which means you can transfer files directly to the directory
- store CDR sooner when the BYE messages is not properly closed (missing ACK etc)
7.0.1
=====
- fix capture rules direction logic. It was always only by destinatino instead of "both". Broken since version 7
6.5.4 -> 7.0 10.6.2013
======================
New features
------------