4.12 Stable

Major Changes

• Added detection of encrypted/obfuscated OpenVPN flows (#2547, #2560)
• Added detection of encrypted/obfuscated/proxied TLS flows (#2553)
• Implemented nDPI TCP fingerprint (6b6dad4)

For further details on these three topics, see https://www.ntop.org/ntop/a-deep-dive-into-traffic-fingerprints/

New Supported Protocols and Services

This is the list of the new supported protocols, grouped by category.
Information about these new protocols is available on https://github.com/ntop/nDPI/blob/dev/doc/protocols.rst

• NDPI_PROTOCOL_CATEGORY_IOT_SCADA:
  • NDPI_PROTOCOL_CNP_IP (#2521, #2531)
  • NDPI_PROTOCOL_ATG (#2527)
  • NDPI_PROTOCOL_TRDP (#2528)
  • NDPI_PROTOCOL_DICOM (4fd1227)
  • NDPI_PROTOCOL_CATEGORY_DATA_TRANSFER:
  • NDPI_PROTOCOL_LUSTRE (#2544)
• NDPI_PROTOCOL_CATEGORY_VPN:
• NDPI_PROTOCOL_NORDVPN (f350379)
• NDPI_PROTOCOL_SURFSHARK (5b0374c)
• NDPI_PROTOCOL_CACTUSVPN (c99646e)
• NDPI_PROTOCOL_WINDSCRIBE (2964c23)
• NDPI_PROTOCOL_CATEGORY_MUSIC:
• NDPI_PROTOCOL_SONOS (806f473)
• NDPI_PROTOCOL_CATEGORY_CHAT:
• NDPI_PROTOCOL_DINGTALK (#2581)
• NDPI_PROTOCOL_PALTALK (#2606)
• NDPI_PROTOCOL_CATEGORY_WEB:
  • NDPI_PROTOCOL_NAVER (#2610)
• NDPI_PROTOCOL_CATEGORY_SHOPPING:
  • NDPI_PROTOCOL_SHEIN (#2615)
  • NDPI_PROTOCOL_TEMU (#2615)
  • NDPI_PROTOCOL_TAOBAO (#2615)
• NDPI_PROTOCOL_CATEGORY_NETWORK:
  • NDPI_PROTOCOL_MIKROTIK (#2618)
• NDPI_PROTOCOL_CATEGORY_STREAMING:
  • NDPI_PROTOCOL_PARAMOUNTPLUS (#2628)
• NDPI_PROTOCOL_CATEGORY_VIRTUAL_ASSISTANT:
  • NDPI_PROTOCOL_YANDEX_ALICE (#2633)

New features

• Implemented JA4 raw fingerprint (42ded07)
• Add monitoring capability (for STUN flows) (#2588). See also: https://github.com/ntop/nDPI/blob/dev/doc/monitoring.md
• Implemented DNS host cache (55fa924)
• Add a configuration file to ndpiReader (#2629)

New algorithms

• Implemented algorithms for K-Nearest Neighbor Search (KNN) (#2554)
• Added ndpi_quick_encrypt() and ndpi_quick_decrypt() API calls (#2568)

New configuration knobs

Further information is available on https://github.com/ntop/nDPI/blob/dev/doc/configuration_parameters.md

• tls,subclassification, quic,subclassification, http,subclassification: enable/disable subclassification (#2533)
• openvpn,subclassification_by_ip, wiregurad,subclassification_by_ip: enable/disable sub-classification using server IP. Useful to detect the specific VPN application/app (85ebda4)
• openvpn,dpi.heuristics, openvpn,dpi.heuristics.num_messages: configure heuristics to detect OpenVPN flows (#2547)
• dpi.guess_ip_before_port: enable/disable guessing by IP first when guessing flow classifcation (#2562)
• tls,dpi.heuristics, tls,dpi.heuristics.max_packets_extra_dissection: configure heuristics to detect TLS flows (#2553)
• flow.use_client_ip_in_guess, flow.use_client_port_in_guess: configure guessing algorithm (#2569)
• $PROTO_NAME,monitoring: enable/disable monitoring state (#2588)
• metadata.tcp_fingerprint, tls,metadata.ja4r_fingerprint: enable/disable some fingerprints (6b6dad4, 42ded07)
• sip,metadata.attribute.XXX: enable/disable extraction of some SIP metadata (#2614)

Improvements

• Fixed probing attempt risk that was creating false positives (fc4fb4d)
• Fixes Viber false positive detection (5610145)
• ahocorasick: fix mem leaked AC_NODE_T object (#2258, #2522)
• Endian-independent implementation of IEEE 802.3 CRC32 (#2529)
• Improved Yahoo matching for Japanese traffic (#2539)
• HTTP, QUIC, TLS: allow to disable sub-classification (#2533)
• Bittorrent fixes (#2538)
• bins: fix ndpi_set_bin, ndpi_inc_bin and ndpi_get_bin_value (#2536)
• TLS: better state about handshake (#2534)
• OpenVPN: improve detection (c713c89)
• OpenVPN, Wireguard: improve sub-classification (85ebda4)
• oracle: fix dissector (#2548)
• RTMP: improve detection (#2549)
• RTP: fix identification over TCP (def86ba)
• QUIC: add a basic heuristic to detect mid-flows (#2550)
• Enhanced DHCP fingerprint (b77d3e3)
• dns: add a check before setting NDPI_MALFORMED_PACKET risk (#2558)
• Tls out of order (#2561)
• Added DHCP class identifier (7cc2432)
• Improved fingerprint serialization (40fefd5)
• Fixed handling of spurious TCP retransmissions (eeb1c28)
• TLS: improve handling of Change Cipher message (#2564)
• Added pki.goog domain name (26b1899)
• TTL Cache Fix (#2582)
• Added STUN fingerprint code (ab3e073)
• TLS: heuristics: fix memory allocations (#2577)
• TLS: detect abnormal padding usage (#2579)
• Enhanced DHCP fingerprint (4df60a8)
• STUN: fix monitoring of Whatsapp and Zoom flows (#2590
• Exports DNS A/AAAA responses (up to 4 addresses) (45323e3)
• Added new API calls for serializing/restoring the DNS cache (b9348e9)
• Fixed JA4 invalid computation due to code bug and uninitialized values (2b40611)
• Add configuration of TCP fingerprint computation (#2598)
• STUN: if the same metadata is found multiple times, keep the first value (#2591)
• STUN: minor fix for RTCP traffic (#2593)
• Added support for RDP over TLS (6dc4533)
• STUN: fix monitoring with RTCP flows (#2603)
• Fixes TCP fingerprint calculation when multiple EOL are specified (d5236c0)
• Added DHCP fingerprint (fecc378)
• DNS reponse addresses are now serialized in JSON (0d4c1e9)
• TikTok cleanup (a97a130)
• Added HTTP credentials extraction (412ca87)
• TLS: export heuristic fingerprint as metadata (#2609)
• SIP: rework detection and extract metadata (#2614)
• Zoom: fix heap-buffer-overflow (#2621)
• Small updates on domains list (#2623)
• RTP, STUN: improve detection of multimedia flow type (#2620)
• Update flow->flow_multimedia_types to a bitmask (#2625)
• Improved TCP probing attempt (9e67885)
• When triggering risk "Known Proto on Non Std Port", nDPi now reports the port that was supposed to be used as default (56e5244)
• SIP: export metadata via json (#2630)
• STUN: improve Whatsapp monitoring (#2635)
• Enhanced STUN stats](6b6b5c7)
• Added STUN custom support (ea1b8dc)
• signal: improve detection of chats and calls (#2637)
• STUN: fix monitoring (#2639)
• STUN/RTP: improve metadata extraction (#2641)
• Added minor Citrix improvement (727d08d)
• Telegram STUN improvement (4d17dc6)

Misc

• Fix verify_dist_tarball.sh after latest release (#2519)
• Removed unnecessary includes (#2525)
• Fixed initialization (e722554, 9b1736a)
• Fix url for downloading X/Twitter crawler IPs (#2526)
• Introduced ndpi_master_app_protocol typedef (53a6bae)
• Added ndpi_get_protocol_by_name* API call (f7ee92c)
• Changed NDPI_MALICIOUS_JA3 to NDPI_MALICIOUS_FINGERPRINT (bad0e60)
• Added ndpi_is_proto_* and ndpi_get_proto_by_* API call (9263d4d)
• Added ndpi_risk2code and ndpi_code2risk API call (5436ddd)
• Added print_ndpi_address_port in nDPi API (d769b23)
• Print risk code in ndpi_dump_risks_score (69fd4aa)
• Align serialized risk names to all others (first letter; uppercase letter) (#2541)
• wireshark: extcap: fix output data link type (#2543)
• wireshark: extcap: export flow risk info (23ae3d0)
• Added -E option for dumping flow fingerprint (fda3730)
• Reworked fingerprint export now in JSON (6de91c7)
• wireshark: extcap: rework trailer header (#2557)
• fuzz: try to be a little bit faster (#2559, #2570, #2578)
• domain lists are not loaded when -E is used (1d1edfc)
• Implemented ndpi_strrstr() (191694f, #2570)
• Allow IP guess before port in ndpi_detection_giveup (#2562)
• Replaced traces with debug messages (08a37dc)
• wireshark: lua: add script for QUIC fingerprints (#2566)
• Added new API calls ndpi_hex2bin and ndpi_bin2hex (42cfd29)
• Add enable/disable guessing using client IP/port (#2569)
• CI: add tests on macos-15 (#2571)
• Let the library returning the packet direction calculated internally (#2572)
• wireshark: extcap: allow configuration of OpenVPN/TLS heuristics via GUI (#2576)
• CI: remove macos-12 (#2592)
• Moved ndpi_lru in a separate file (7629b94)
• Added -N option for dumping/restoring the DNS cache (when enabled) (2e5edd2)
• Added JA4 stats (b53e4fc)
• Added support for printing JA4r when enabled (faaa5c5)
• Added TLS fingerprints (37a654e)
• Added ndpi_is_public_ipv4 (3e04321)
• Parser for ndpiReader JSON files (97ce729)
• Added -L for loading domain suffixes (afc4d9e)
• ndpiReader: add some statistics about monitoring (#2602)
• ndpiReader: explicitly remove non ipv4/6 packets (#2601)
• Fix ndpi_tot_allocated_memory calculation if ndpi_calloc() used (#2604)
• ndpiReader: fix command line options used by wireshark (#2605)
• ml tests for dga detection (#2607)
• Add new json serialization type ndpi_serialization_format_inner_json (8ad34b3)
• fuzz: improve coverage (#2612)
• Exported is_ndpi_proto definition (183175f)
• Crash fix when -f is specified with a non-existing pcap file (-i) (35ef56c)
• Unify ndpi debug logging to always use a u16 protocol id (#2613)
• Added ndpi_intoav6() (de8c326)
• Debian/Ubuntu packaging: use --enable-no-sign to build *.deb packages w/o signing those (#2616)
• ndpiReader: fix statistic about total number of flows (#2622)
• Update GitHub CI actions (#2627)
• Removed old USE_LEGACY_AHO_CORASICK code (170849f)
• Fix license typo (#2638)
• Update script to download Azure IP list ranges (#2640)
• Update all IPS lists (#2643)

New Contributors

• @wssxsxxsx made their first contribution in #2527
• @liwilson1 made their first contribution in #2539
• @YellowMan02 made their first contribution in #2607
• @Klavishnik made their first contribution in #2633
• @adipierro made their first contribution in #2638

Full Changelog: 4.10...4.12