ndpi web QOS set ndpi?

Discuss the technical details of Gargoyle and ongoing development

Moderator: Moderators

nulee
Posts: 16
Joined: Wed Oct 26, 2016 12:00 pm

ndpi web QOS set ndpi?

Post by nulee »

I have installed the nDPI I think oneself to modify WEB pages to add this option, like "luci - app - qos - guoguo" add

Code: Select all

if (tonumber(sys.exec("lsmod | cut -d ' ' -f 1 | grep -c 'xt_ndpi'"))) > 0 then
   ndpi = s:option(Value, "ndpi", translate("DPI protocol"))
   local pats = io.popen("iptables -m ndpi --help | grep -e '^--'")
   if pats then
      local l,s,e,_,prt_v,prt_d
      while true do
         l = pats:read("*l")
         if not l then break end
         s,e = l:find("%-%-[^%s]+")
         if s and e then
            prt_v=l:sub(s+2,e)
         end
         s,e = l:find("for [^%s]+ protocol")
            if s and e then
            prt_d=l:sub(s+3,e-9)
         end
         ndpi:value(prt_v,prt_d)
         end
         pats:close()
   end
end


What should I do, webgui directory structure most of these are not clear A friend to know?

Such as: js/qos. Js

Code: Select all

optionList = ["source", "srcport", "destination", "dstport", "max_pkt_size","min_pkt_size",  "proto", "connbytes_kb", "nDPI_protocol"];

Later I need to modify?

I am a Chinese, English level is limited Please forgive me
I am very fond of Gargoyle WEB, super love it :lol:

Code: Select all

iptables -m ndpi --help

iptables v1.4.21

Usage: iptables -[ACD] chain rule-specification [options]
       iptables -I chain [rulenum] rule-specification [options]
       iptables -R chain rulenum rule-specification [options]
       iptables -D chain rulenum [options]
       iptables -[LS] [chain [rulenum]] [options]
       iptables -[FZ] [chain] [options]
       iptables -[NX] chain
       iptables -E old-chain-name new-chain-name
       iptables -P chain target [options]
       iptables -h (print this help information)

Commands:
Either long or short options are allowed.
  --append  -A chain      Append to chain
  --check   -C chain      Check for the existence of a rule
  --delete  -D chain      Delete matching rule from chain
  --delete  -D chain rulenum
            Delete rule rulenum (1 = first) from chain
  --insert  -I chain [rulenum]
            Insert in chain as rulenum (default 1=first)
  --replace -R chain rulenum
            Replace rule rulenum (1 = first) in chain
  --list    -L [chain [rulenum]]
            List the rules in a chain or all chains
  --list-rules -S [chain [rulenum]]
            Print the rules in a chain or all chains
  --flush   -F [chain]      Delete all rules in  chain or all chains
  --zero    -Z [chain [rulenum]]
            Zero counters in chain or all chains
  --new     -N chain      Create a new user-defined chain
  --delete-chain
            -X [chain]      Delete a user-defined chain
  --policy  -P chain target
            Change policy on chain to target
  --rename-chain
            -E old-chain new-chain
            Change chain name, (moving any references)
Options:
    --ipv4   -4      Nothing (line is ignored by ip6tables-restore)
    --ipv6   -6      Error (line is ignored by iptables-restore)
[!] --protocol   -p proto   protocol: by number or name, eg. `tcp'
[!] --source   -s address[/mask][...]
            source specification
[!] --destination -d address[/mask][...]
            destination specification
[!] --in-interface -i input name[+]
            network interface name ([+] for wildcard)
 --jump   -j target
            target for rule (may load target extension)
  --goto      -g chain
                              jump to chain with no return
  --match   -m match
            extended match (may load extension)
  --numeric   -n      numeric output of addresses and ports
[!] --out-interface -o output name[+]
            network interface name ([+] for wildcard)
  --table   -t table   table to manipulate (default: `filter')
  --verbose   -v      verbose mode
  --wait   -w      wait for the xtables lock
  --line-numbers      print line numbers when listing
  --exact   -x      expand numbers (display exact values)
[!] --fragment   -f      match second or further fragments only
  --modprobe=<command>      try to insert modules using this command
  --set-counters PKTS BYTES   set the counter during insert/append
[!] --version   -V      print package version.

ndpi match options:
--ftp Match for FTP_CONTROL protocol packets.
--pop Match for MAIL_POP protocol packets.
--smtp Match for MAIL_SMTP protocol packets.
--imap Match for MAIL_IMAP protocol packets.
--dns Match for DNS protocol packets.
--ipp Match for IPP protocol packets.
--http Match for HTTP protocol packets.
--mdns Match for MDNS protocol packets.
--ntp Match for NTP protocol packets.
--netbios Match for NETBIOS protocol packets.
--nfs Match for NFS protocol packets.
--ssdp Match for SSDP protocol packets.
--bgp Match for BGP protocol packets.
--snmp Match for SNMP protocol packets.
--xdmcp Match for XDMCP protocol packets.
--smb Match for SMB protocol packets.
--syslog Match for SYSLOG protocol packets.
--dhcp Match for DHCP protocol packets.
--postgres Match for POSTGRES protocol packets.
--mysql Match for MYSQL protocol packets.
--hotmail Match for HOTMAIL protocol packets.
--directdownload Match for DIRECT_DOWNLOAD_LINK protocol packets.
--pops Match for MAIL_POPS protocol packets.
--applejuice Match for APPLEJUICE protocol packets.
--directconnect Match for DIRECTCONNECT protocol packets.
--socrates Match for SOCRATES protocol packets.
--coap Match for COAP protocol packets.
--vmware Match for VMWARE protocol packets.
--smtps Match for MAIL_SMTPS protocol packets.
--filetopia Match for FILETOPIA protocol packets.
--ubntac2 Match for UBNTAC2 protocol packets.
--kontiki Match for KONTIKI protocol packets.
--openft Match for OPENFT protocol packets.
--fasttrack Match for FASTTRACK protocol packets.
--gnutella Match for GNUTELLA protocol packets.
--edonkey Match for EDONKEY protocol packets.
--bittorrent Match for BITTORRENT protocol packets.
--epp Match for EPP protocol packets.
--avi Match for AVI protocol packets.
--flash Match for FLASH protocol packets.
--ogg Match for OGG protocol packets.
--mpeg Match for MPEG protocol packets.
--quicktime Match for QUICKTIME protocol packets.
--realmedia Match for REALMEDIA protocol packets.
--windowsmedia Match for WINDOWSMEDIA protocol packets.
--mms Match for MMS protocol packets.
--xbox Match for XBOX protocol packets.
--qq Match for QQ protocol packets.
--move Match for MOVE protocol packets.
--rtsp Match for RTSP protocol packets.
--imaps Match for MAIL_IMAPS protocol packets.
--icecast Match for ICECAST protocol packets.
--pplive Match for PPLIVE protocol packets.
--ppstream Match for PPSTREAM protocol packets.
--zattoo Match for ZATTOO protocol packets.
--shoutcast Match for SHOUTCAST protocol packets.
--sopcast Match for SOPCAST protocol packets.
--tvants Match for TVANTS protocol packets.
--tvuplayer Match for TVUPLAYER protocol packets.
--http_download Match for HTTP_DOWNLOAD protocol packets.
--qqlive Match for QQLIVE protocol packets.
--thunder Match for THUNDER protocol packets.
--soulseek Match for SOULSEEK protocol packets.
--ssl_no_cert Match for SSL_NO_CERT protocol packets.
--irc Match for IRC protocol packets.
--ayiya Match for AYIYA protocol packets.
--unencryped_jabber Match for UNENCRYPED_JABBER protocol packets.
--msn Match for MSN protocol packets.
--oscar Match for OSCAR protocol packets.
--yahoo Match for YAHOO protocol packets.
--battlefield Match for BATTLEFIELD protocol packets.
--quake Match for QUAKE protocol packets.
--vrrp Match for VRRP protocol packets.
--steam Match for STEAM protocol packets.
--halflife2 Match for HALFLIFE2 protocol packets.
--worldofwarcraft Match for WORLDOFWARCRAFT protocol packets.
--telnet Match for TELNET protocol packets.
--stun Match for STUN protocol packets.
--ipsec Match for IPSEC protocol packets.
--gre Match for GRE protocol packets.
--icmp Match for ICMP protocol packets.
--igmp Match for IGMP protocol packets.
--egp Match for EGP protocol packets.
--sctp Match for SCTP protocol packets.
--ospf Match for OSPF protocol packets.
--ipip Match for IP_IN_IP protocol packets.
--rtp Match for RTP protocol packets.
--rdp Match for RDP protocol packets.
--vnc Match for VNC protocol packets.
--pcanywhere Match for PCANYWHERE protocol packets.
--ssl Match for SSL protocol packets.
--ssh Match for SSH protocol packets.
--usenet Match for USENET protocol packets.
--mgcp Match for MGCP protocol packets.
--iax Match for IAX protocol packets.
--tftp Match for TFTP protocol packets.
--afp Match for AFP protocol packets.
--stealthnet Match for STEALTHNET protocol packets.
--aimini Match for AIMINI protocol packets.
--sip Match for SIP protocol packets.
--truphone Match for TRUPHONE protocol packets.
--icmpv6 Match for ICMPV6 protocol packets.
--dhcpv6 Match for DHCPV6 protocol packets.
--armagetron Match for ARMAGETRON protocol packets.
--crossfire Match for CROSSFIRE protocol packets.
--dofus Match for DOFUS protocol packets.
--fiesta Match for FIESTA protocol packets.
--florensia Match for FLORENSIA protocol packets.
--guildwars Match for GUILDWARS protocol packets.
--http_application_activesync Match for HTTP_APPLICATION_ACTIVESYNC protocol packets.
--kerberos Match for KERBEROS protocol packets.
--ldap Match for LDAP protocol packets.
--maplestory Match for MAPLESTORY protocol packets.
--mssql Match for MSSQL_TDS protocol packets.
--pptp Match for PPTP protocol packets.
--warcraft3 Match for WARCRAFT3 protocol packets.
--world_of_kung_fu Match for WORLD_OF_KUNG_FU protocol packets.
--slack Match for SLACK protocol packets.
--facebook Match for FACEBOOK protocol packets.
--twitter Match for TWITTER protocol packets.
--dropbox Match for DROPBOX protocol packets.
--gmail Match for GMAIL protocol packets.
--google_maps Match for GOOGLE_MAPS protocol packets.
--youtube Match for YOUTUBE protocol packets.
--skype Match for SKYPE protocol packets.
--google Match for GOOGLE protocol packets.
--dcerpc Match for DCERPC protocol packets.
--netflow Match for NETFLOW protocol packets.
--sflow Match for SFLOW protocol packets.
--http_connect Match for HTTP_CONNECT protocol packets.
--http_proxy Match for HTTP_PROXY protocol packets.
--citrix Match for CITRIX protocol packets.
--netflix Match for NETFLIX protocol packets.
--lastfm Match for LASTFM protocol packets.
--waze Match for WAZE protocol packets.
--skyfile_prepaid Match for SKYFILE_PREPAID protocol packets.
--skyfile_rudics Match for SKYFILE_RUDICS protocol packets.
--skyfile_postpaid Match for SKYFILE_POSTPAID protocol packets.
--citrix_online Match for CITRIX_ONLINE protocol packets.
--apple Match for APPLE protocol packets.
--webex Match for WEBEX protocol packets.
--whatsapp Match for WHATSAPP protocol packets.
--apple_icloud Match for APPLE_ICLOUD protocol packets.
--viber Match for VIBER protocol packets.
--apple_itunes Match for APPLE_ITUNES protocol packets.
--radius Match for RADIUS protocol packets.
--windows_update Match for WINDOWS_UPDATE protocol packets.
--teamviewer Match for TEAMVIEWER protocol packets.
--tuenti Match for TUENTI protocol packets.
--lotusnotes Match for LOTUS_NOTES protocol packets.
--sap Match for SAP protocol packets.
--gtp Match for GTP protocol packets.
--upnp Match for UPNP protocol packets.
--llmnr Match for LLMNR protocol packets.
--remotescan Match for REMOTE_SCAN protocol packets.
--spotify Match for SPOTIFY protocol packets.
--webm Match for WEBM protocol packets.
--h323 Match for H323 protocol packets.
--openvpn Match for OPENVPN protocol packets.
--noe Match for NOE protocol packets.
--ciscovpn Match for CISCOVPN protocol packets.
--teamspeak Match for TEAMSPEAK protocol packets.
--tor Match for TOR protocol packets.
--skinny Match for SKINNY protocol packets.
--rtcp Match for RTCP protocol packets.
--rsync Match for RSYNC protocol packets.
--oracle Match for ORACLE protocol packets.
--corba Match for CORBA protocol packets.
--ubuntuone Match for UBUNTUONE protocol packets.
--whois_das Match for WHOIS_DAS protocol packets.
--collectd Match for COLLECTD protocol packets.
--socks Match for SOCKS protocol packets.
--ms_lync Match for MS_LYNC protocol packets.
--rtmp Match for RTMP protocol packets.
--ftpdata Match for FTP_DATA protocol packets.
--wikipedia Match for WIKIPEDIA protocol packets.
--zmq Match for ZMQ protocol packets.
--amazon Match for AMAZON protocol packets.
--ebay Match for EBAY protocol packets.
--cnn Match for CNN protocol packets.
--megaco Match for MEGACO protocol packets.
--redis Match for REDIS protocol packets.
--pando Match for PANDO protocol packets.
--vhua Match for VHUA protocol packets.
--telegram Match for TELEGRAM protocol packets.
--vevo Match for VEVO protocol packets.
--pandora Match for PANDORA protocol packets.
--quic Match for QUIC protocol packets.
--whatsapp_voice Match for WHATSAPP_VOICE protocol packets.
--eaq Match for EAQ protocol packets.
--git Match for GIT protocol packets.
--drda Match for DRDA protocol packets.
--kakaotalk Match for KAKAOTALK protocol packets.
--kakaotalk_voice Match for KAKAOTALK_VOICE protocol packets.
--twitch Match for TWITCH protocol packets.
--quickplay Match for QUICKPLAY protocol packets.
--opendns Match for OPENDNS protocol packets.
--mpegts Match for MPEGTS protocol packets.
--snapchat Match for SNAPCHAT protocol packets.
--deezer Match for DEEZER protocol packets.
--instagram Match for INSTAGRAM protocol packets.
--microsoft Match for MICROSOFT protocol packets.
--hotspot_shield Match for HOTSPOT_SHIELD protocol packets.
--ocs Match for OCS protocol packets.
--office_365 Match for OFFICE_365 protocol packets.
--cloudflare Match for CLOUDFLARE protocol packets.
--ms_one_drive Match for MS_ONE_DRIVE protocol packets.
--mqtt Match for MQTT protocol packets.
--rx Match for RX protocol packets.
--sina Match for SINA protocol packets.
--starcraft Match for STARCRAFT protocol packets.
--teredo Match for TEREDO protocol packets.
--hep Match for HEP protocol packets.
--hangout Match for HANGOUT protocol packets.
--dpi_check Match for CHECK protocol packets.
Success

Lantis
Moderator
Posts: 5528
Joined: Mon Jan 05, 2015 5:33 am
Location: Australia

Re: ndpi web QOS set ndpi?

Post by Lantis »

Can you please share where you got nDPI from?
It is not simple to implement, however it is a feature I am willing to work on.
http://lantisproject.com/downloads/gargoyle_ispyisail.php for the latest releases
Please be respectful when posting. I do this in my free time on a volunteer basis.

nulee
Posts: 16
Joined: Wed Oct 26, 2016 12:00 pm

Re: ndpi web QOS set ndpi?

Post by nulee »

Lantis wrote:Can you please share where you got nDPI from?
It is not simple to implement, however it is a feature I am willing to work on.

:mrgreen:

Code: Select all

include $(TOPDIR)/rules.mk
include $(INCLUDE_DIR)/kernel.mk

PKG_NAME:=ndpi-netfilter
PKG_VERSION:=6bc9b60
PKG_RELEASE:=1
PKG_REV:=6bc9b60

PKG_SOURCE_PROTO:=git
PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION).tar.bz2
PKG_SOURCE_SUBDIR:=$(PKG_NAME)-$(PKG_VERSION)
PKG_SOURCE_URL:=https://github.com/betolj/ndpi-netfilter
PKG_SOURCE_VERSION:=$(PKG_REV)

PKG_BUILD_DIR:=$(KERNEL_BUILD_DIR)/$(PKG_NAME)-$(PKG_VERSION)

include $(INCLUDE_DIR)/package.mk

define Package/iptables-mod-ndpi
  SUBMENU:=Firewall
  SECTION:=net
  CATEGORY:=Network
  TITLE:=ndpi successor of OpenDPI
  URL:=http://www.ntop.org/products/ndpi/
  DEPENDS:=+iptables +iptables-mod-conntrack-extra +kmod-ipt-ndpi
  MAINTAINER:=Thomas Heil <heil@terminal-consulting.de>
endef

define Package/iptables-mod-ndpi/description
  nDPI is a ntop-maintained superset of the popular OpenDPI library
endef

define Build/Prepare
   $(call Build/Prepare/Default)
   tar -zxvf $(PKG_BUILD_DIR)/nDPI.tar.gz -C $(PKG_BUILD_DIR)
endef

MAKE_FLAGS += \
   KERNEL_DIR="$(LINUX_DIR)" \
   ARCH="$(LINUX_KARCH)" \
   NDPI_PATH=$(PKG_BUILD_DIR)/nDPI \
   CROSS_COMPILE="$(TARGET_CROSS)" \
   all

define Package/iptables-mod-ndpi/install
   $(INSTALL_DIR) $(1)/usr/lib/iptables
   $(INSTALL_BIN) $(PKG_BUILD_DIR)/ipt/libxt_ndpi.so $(1)/usr/lib/iptables
endef

define KernelPackage/ipt-ndpi
  SUBMENU:=Netfilter Extensions
  TITLE:= nDPI net netfilter module
  DEPENDS:=+kmod-nf-conntrack +kmod-nf-conntrack-netlink +kmod-ipt-compat-xtables
  FILES:= \
   $(PKG_BUILD_DIR)/src/xt_ndpi.ko
  AUTOLOAD:=$(call AutoProbe,xt_ndpi)
endef

$(eval $(call BuildPackage,iptables-mod-ndpi))
$(eval $(call KernelPackage,ipt-ndpi))

nulee
Posts: 16
Joined: Wed Oct 26, 2016 12:00 pm

Re: ndpi web QOS set ndpi?

Post by nulee »

Lantis wrote:Can you please share where you got nDPI from?
It is not simple to implement, however it is a feature I am willing to work on.

i set PKG_VERSION:=052718d
From:github.com/981213 & https://github.com/betolj/ndpi-netfilter

Lantis
Moderator
Posts: 5528
Joined: Mon Jan 05, 2015 5:33 am
Location: Australia

Re: ndpi web QOS set ndpi?

Post by Lantis »

Fantastic. Thanks.
I've been looking to port this code over for a while but have been too busy.

This is now my top priority.

Unfortunately that means I won't be able to give you a good answer to your question for quite some time.
http://lantisproject.com/downloads/gargoyle_ispyisail.php for the latest releases
Please be respectful when posting. I do this in my free time on a volunteer basis.

nulee
Posts: 16
Joined: Wed Oct 26, 2016 12:00 pm

Re: ndpi web QOS set ndpi?

Post by nulee »

Lantis wrote:Fantastic. Thanks.
I've been looking to port this code over for a while but have been too busy.

This is now my top priority.

Unfortunately that means I won't be able to give you a good answer to your question for quite some time.

Slowly any progress you can email me

ispyisail
Moderator
Posts: 4809
Joined: Mon Apr 06, 2009 3:15 am
Location: New Zealand

Re: ndpi web QOS set ndpi?

Post by ispyisail »


Lantis
Moderator
Posts: 5528
Joined: Mon Jan 05, 2015 5:33 am
Location: Australia

Re: ndpi web QOS set ndpi?

Post by Lantis »

It's going to be the replacement to layer7.
Layer 7 is currently broken because of SSL.
This gets around that, but... I'm worried about performance because of it.
http://lantisproject.com/downloads/gargoyle_ispyisail.php for the latest releases
Please be respectful when posting. I do this in my free time on a volunteer basis.

ispyisail
Moderator
Posts: 4809
Joined: Mon Apr 06, 2009 3:15 am
Location: New Zealand

Re: ndpi web QOS set ndpi?

Post by ispyisail »

I'm worried about performance because of it.


wondered about that.

nulee
Posts: 16
Joined: Wed Oct 26, 2016 12:00 pm

Re: ndpi web QOS set ndpi?

Post by nulee »

ispyisail wrote:
I'm worried about performance because of it.


wondered about that.

This is a performance comparison:
http://vbn.aau.dk/en/publications/comparison-of-deep-packet-inspection-dpi-tools-for-traffic-classification(513b6aea-77af-4cf4-a353-0919351d8ff3).html?goback=%2Egde_103762_member_251508791

Here I use MT7620A one vendor has made a DPI adapter I also have this But I prefer to open source, so...

Post Reply