Embed Size (px)
Citation preview
❑ Converts real-time network traffic to meta data
❑ Get complete access to your data
❑ Provides native support to any big data infrastructure
❑ Has highest performance in the industry
❑ Provides mobile and fixed net solution)
Why choose Cubro Probe?
How does the Cubro Probe work?
❑ The Probe receives network traffic from TAP networks or span ports
(passive)
❑ The Probe decodes the traffic up to layer 7 and extracts meta information
❑ The Probe correlates this decoded information and generates XDRs
(extended data records)
❑ The XDRs are sent to a database system where they are stored and
presented by an Application
❑ This Application is called typical Monitoring System
What makes Cubro Probe different?
❑ The Probes are based on a multi-core Cavium/ARM CPU
❑ The Probes consume much less power than Intel-based solution with
the same performance
❑ The Probe performance ranges from 10Gbps to 200Gbps
❑ No moving parts
❑ 1 U and 2 U form factor
❑ Agnostic to any 3rd party presentation systems
❑ UDP XDR output (dissector for Wireshark available)
❑ UDP to TCP converter available for long backhaul applications
What makes Cubro Probe Solution different?
✓ Cubro offers the full stack in monitoring from Layer 1 (TAP) to
aggregation (NPB) and load balancing to the Probe and middle ware
to data storage.
✓ Cubro is the only Vendor which supports all layers of monitoring
✓ Cubro’s integrated solutions for visibility in all layers and probing
offer customers value for money
Cubro Probe Solution
TAP (Layer 1)
NPB (Layer 2 - 6) aggregation and load balancing
Probing and meta data extraction (some of the meta data from the Probes are forwarded back to NPB for load balancing)
3rd’ party probing solutions are fed with traffic from the same source
Cubro Flow Monitoring Probe (FlowVista)*
Enterprise Network meta data extraction (Netflow V9)
Hardware based solution for Cavium multicore
DPI, Flow monitoring, Probing solution,
Application Filtering
* For more details ask for the FlowVista presentation
Cubro 2G/3G/4G/5G PS Domain Monitoring Probe
Network meta data extraction
Hardware based solution for Cavium/ARM multicore
DPI, Mobile Probing solution, mobile XDR
Different types of Cubro Probe
Performance 10 Gbps (1 CPU) Performance 20 Gbps (2 CPU)
Performance 60 Gbps (2 CPU)Performance 30 Gbps (1 CPU)
NG Probe > 100 Multicore ARM based solution
Performance 150 Gbps (2 CPUs)
2 x 100 Gbit interface
10 Gbps Cubro Probe
✓ 40 or 48 x 10Gbit ports ✓ 1 x Octeon III (32 cores each)✓ 1 x 64GB DDR Memory✓ Up to 30Gbps L7 filtering performance
(forwarding only)✓ Up to 20Gbps L7 correlation performance✓ Up to 10Gbps L7 meta data processing
20 Gbps Cubro Probe
✓ 40 or 48 x 10Gbit ports ✓ 2 x Octeon III (32 cores each)✓ 2 x 64 GB DDR Memory✓ Up to 60Gbps L7 filtering performance
(forwarding only)✓ Up to 40Gbps L7 correlation performance✓ Up to 20Gbps L7 meta data processing
60 Gbps Cubro Advanced Probing Platform
✓ 2 x 12 x 10Gbit 2 x 40Gbit ports ✓ 2 x Octeon III CN7890 CPU (48 cores each)✓ 2 x 64 GB DDR Memory✓ 2 x 240 billion instructions per sec✓ Up to 160 Gbps L7 filtering performance
(forwarding only)✓ Up to 120 Gbps L7 correlation performance✓ Up to 60 Gbps L7 meta data processing✓ External switches integration with EX32100 and
EX48600
NG Probe (only Mobile XDR)
Network traffic is growing more rapidly – we see see multiple 100Gbit networks on dailybasis and in some cases Terabit networks.
Probes with 30Gbit performance will not work. We need Probes with 100Gbit or moreperformance. Our approach is ARM Multicores.
We have the experience in Multicore design and Cubro NG Probe is the next stage oftransition. “We are in the Beta stage at the moment”, Sept 2017.
Cubro Intelligence-Probe-Layer
Aggregated Filtered Raw data
(Original packets)
Processed Meta data (XDR)
Cubro offers TAPs, Aggregators & Probes
Cubro Probe supported interfaces
We support all 2G/3G/4G/5G Interfaces
and approx. 1000 DPI protocols
for application detection
Current XDRs (extended data record)
Gn signaling record,
GTPv2 signaling record,
S3 signaling record,
DNS signaling record,
User service flow record,
MMS MO signaling record,
MMS MT signaling record,
WAP_CONNECT record,
WAP signaling record,
ONLINE_VIDEO record,
FTP record,
RTSP record,
EMAIL record,
VOIP record,
P2P record,
IM record,
S1 signaling record,
S1 EMM signaling record,
S1 ESM signaling record,
S1AP protocol switching record,
S1AP protocol RAB record,S1AP protocol management record,S6a record,S1 SMS record,S1 CS fallback record,SGS MM signaling record,SGS CS signaling record,X2 interface management record,X2 interface switching record,UU signaling record,UU switching record,UU-community measurement,UU-UE measurement,attach signaling,detach signaling,PDP Activation,PDP Deactivation,PDP Modification,RAU,BSSGPRANAP,Relocation,Service Request,2G Paging
User service flow record fields
User PortDestination PortTCP FIN timesUplink Dropped Packets NumberDownlink Dropped Packets NumberTotal Number Of Uplink Data PacketsTotal Number Of Downlink Data PacketsUplink TrafficDownlink TrafficWindow SizeMSS SizeRST DirectionBearing Layer ProtocolFragments flag SYN Number In TCP LinkingSuccessful Identification Of Three Shake HandsSYN ACK Number In TCP Linking
Time of Data TransmissionUser IP AddressState CodeNetwork CodeCell IDTracking CodeLocation CodeRouting Area ID2G/3G Network IDUser Location InformationIMSISubscriber numberIMEIAPNCharging IDSGSN User Plane Transmission IPGGSN User Plane Transmission IPDestination IPSGSN User Plane TEIDGGSN User Plane TEID
ACK Number InTCP LinkingUplink IP Fragment NumberDownlink IP Fragment NumberDisordered packet number of Uplink TCPDisordered packet number of Downlink TCPRetransmission packet number of Uplink TCPRetransmission packet number of Downlink TCPTCP RESET Number DirectionProtocol TypeResponse delay Of TCP LinkingConfirmation delay Of TCP LinkingDelay Between TCP Linking And The First RequestDelay Between he First Request And The First ACK
Currently supported DPI Application detection (and growing)
skype,http,ftp,sip ,secure_http ,irc ,secure_irc ,rtp ,rtsp ,facebook ,youtube ,myspace ,ftp_data ,hotline ,bittorrent ,direct_connect ,edonkey ,gnutella ,manolito ,windows_update ,rtcp ,mute ,soulseek ,h225h245 ,h245 ,flickr ,twitter ,mgcp ,h248 ,skinny ,joost ,sopcast ,pptv ,itunes ,unidata_ldm,gtalk_textchat ,msn_messenger ,yahoo_messenger ,nateon ,gnutella2 ,synoptics ,meta5 ,embl_ndt ,netcp ,netware_ip ,mptn ,kryptolan ,apple_update ,lotus_notes ,ssdp ,rtmp ,skype_phone ,tencent_qq ,icq ,nateon_voip ,live_audio ,gtalk_audio,yahoo_messenger_video_aud0 ,skype_audio ,yahoo_messenger_sip ,live_video ,baidu ,dealsea ,mynewplace ,reference ,qotd ,msp,mpm_flags ,mpm ,msmq ,zedo ,cnet ,goo ,alot ,softonic , zynga_poker ,mafiawars ,farmville ,tumblr ,bebo ,myyearbook ,livejournal ,asiantown ,blackplanet ,cafemom ,classmates ,flixster ,friendster ,fubar ,hi5 ,tagged ,xanga ,yuku ,camzap , ameba ,craigslist ,about_com ,ehow ,filefactory ,badongo ,2shared ,divshare ,limelinx ,mediafire ,sendspace ,speedyshare,uploading ,ziddu ,xunlei ,autoblog ,theboombox ,aol_answers, hortcuts ,games_com ,the_huffington_post ,moviefone ,stylelist ,mapquest ,patch ,techcrunch ,engadget ,dailyfinance ,aol ,ifeng ,hypp_tv ,svtplay ,weather ,babelgum ,cnn ,cnn_video ,cnn_ireport ,cnn_money ,cnn_si ,cnn_hln ,cnn_international ,cnn_arabic ,cnn_mexico,cnn_expansion ,cnn_espanol ,foxnews ,foxnews_video ,foxnews_business ,foxnews_ureport ,foxnews_radio , foxnews_latino ,foxnews_nation ,foxnews_insider ,go ,go_search ,nba ,nba_store ,nba_wnba ,nba_ihoops ,nba_usab ,nba_kids ,espn ,espn_shop ,espn_insider ,espn_radio ,espnplus ,espnstar , espn_australia ,espn_brazil ,espn_uk ,abc ,abc_news ,mlb ,mlb_shop ,mlb_video ,mlb_korea ,systat ,ctf ,dcp ,objcall ,statsrv ,uma ,sgmp ,cmip ,smux ,opalis_robot ,syslog ,rutube ,metacafe , vimeo ,ni_ftp ,ni_mail ,re_mail_ck ,qmtp ,mailq ,imsp ,csnet_ns ,gmail ,google_safe ,yahoo_mail ,yahoo_ad ,hotmail ,skydrive ,srssend ,odmr ,pop2 ,pop3 ,secure_pop3 ,imap ,secure_imap , smtp ,secure_smtp ,arns ,ssh ,nntp ,secure_nntp ,gopher ,finger ,telnet ,rmt ,decladebug ,rlogin ,ntp ,dns ,rsh ,snmp ,soap ,dhcp ,portmap ,kerberos ,nis ,socksv4 ,socksv5 ,ebay ,alibaba , walmart ,google ,google_analytics ,apple ,google_earth ,google_groups ,google_maps ,google_videos ,stumbleupon ,wikipedia ,msn_home ,live_groups ,bing ,bing_videos ,yahoo_groups ,yahoo_search ,yahoo_video ,yahoo_movies ,yahoo_home ,dailymotion ,doubleclick ,salesforce ,aim_login ,vnc ,xwindow ,smb ,rdp ,xdm ,tls_ssl ,tftp ,isakmp ,mysql ,xmlrpc ,ldap ,wmplayer ,timbuktu ,pcanywhere ,citrix ,gotomypc ,yahoo_games ,ezpeer ,crypto_logic_poker_games ,afreeca_video ,quicktime ,realplayer ,microsoft_mms ,jajah ,iax ,flash ,rpc2portmap ,gtalk_file_transfer ,buddy_buddy ,afreeca ,gom_player ,rakuten ,booking ,paypal ,mpeg_video ,linkedin ,linkedin_ad ,netrjs ,rje ,radius ,secure_ldap ,netbios_name ,netbios_session ,netbios_datagram ,llmnr ,mdns ,sipgate ,aim ,rfr ,nspi ,mapi ,exchange ,skype_video ,xmpp ,bftp ,pftp ,ftps ,ftps_data ,auditd ,swift_rvf ,acr_nema ,mcidas ,uucp_path ,cfdptkt ,qft ,antsp2p ,baibao_networking ,last_fm ,bbc_iplayer ,bbc_iplayer_download ,deluge,dijjer ,kuaibo ,avg_update ,norton_liveupdate ,feidian ,fileguri ,filetopia_file_transfer ,filetopia_networking ,subntbcst_tftp ,fxp ,soribada ,5min ,kugou ,su_mit_tg ,supdup ,rtelnet ,snagas,erpc ,packetix ,novastorbakcup ,pando ,zannet ,afpovertcp ,poco_networking ,songsari ,ppstream ,qq_apps ,qq_music ,qq_game ,qqlive ,rodi ,tvants ,vshare ,ares ,winmx ,tomatopang ,bittorrent_dna ,applejuice ,naver ,flashget ,nimbuzz ,truphone ,net2phone ,fring_voip ,zoho_chat ,zoho_mail ,zoho_docs ,zoho_wiki ,wow ,speedtest ,imesh ,picsearch ,morpheus ,zoho ,hotfile ,msdn ,google_translate ,kaspersky ,amazon ,zynga ,adobe ,avira_home ,avira_update ,mywebsearch ,ask ,914c_g ,anet ,vmpwscs ,softpc ,cailic ,mpp ,uarps ,cdc ,masqdialer ,direct ,sur_meas ,inbusiness ,link ,dsp3270 ,bhfhs ,set ,esro_gen ,openport ,nsiiops ,arcisdms ,hdap ,bgmp ,x_bone_ctl ,sst ,td_service ,td_replica ,manet ,gist ,ttp_mgmt ,uis ,asa ,ibm_app ,hp_alarm_mgr ,hp_managed_node ,hp_collector ,is99 ,etos ,nip ,hassle ,legent ,ulistproc ,clearcase ,codaauth ,qbik_gdp ,mortgageware ,dtk ,aurora_cmgr ,semantix ,scoi2odialog ,cloanto_net_1 ,ndsauth ,dtag_ste_sb ,csi_sgwp ,fatserv ,zserv ,asip_webadmin ,bhmds ,entrusttime ,k_block
,corerjd ,rescap ,cableport_ax ,personal_link ,asf ,babylon ,babylon_translate ,vuze ,battle_net ,ca_arcserve,cvs ,daap ,freecast ,gnucleus_lan ,icalendar ,rsvp_tunnel ,nfs ,openft ,peercast ,cups ,secondlife ,shoutcast ,slingbox ,live_web_messenger ,yahoo_web_messenger ,ipp ,e_policy_orchestrator ,icq_toolbar,megaupload ,google_docs ,rapid_share ,orkut ,picasa_web_album ,4shared ,adrive ,chargen,daytime ,time ,echo ,whois ,discard ,photobucket ,rexec ,rwhod ,bugzilla ,graboid_video ,qq_website ,web_qq ,qq_qzone ,qq_pet ,qq_international ,qq_mail ,qq_ftn ,lpd ,npp ,teamviewer ,mypeople_home ,mypeople_audio ,olive_phone ,yahoo_messenger_file_tran862,flash_youtube,tango_voip,tango_video_call ,netflix ,skype_file_transfer ,bgp ,z39_50 ,src ,comscm ,dna_cml ,mobilip_mn ,mobileip_agent ,dls ,dls_mon ,osu_nms ,kis ,remote_kis ,ocserver ,ocbinder ,genrad_mux ,vmnet ,xyplex_mux ,cl_1 ,s_net ,xns_courier ,nss_routing ,knet_cmp ,aed_512 ,jargon ,iso_ip ,cisco_sys ,cisco_tna ,cisco_fna ,gss_xlicen ,locus_con ,locus_map ,nxedit ,ansatrader ,smakynet ,ansanotify ,3com_tsmux ,gppitnp ,iso_tsap ,mit_dov ,mit_ml_dev ,covia ,acas ,xns_mail ,xns_auth ,xns_ch ,xns_time ,isi_gl ,dsp ,ncld ,nced ,decap ,dhcpv6 ,smsp ,bnet ,silverplatter ,onmux ,hyper_g ,ariel ,smpte ,ibm_opc ,rlp ,decvms_sysmgt ,snpp ,crs ,printer ,http_alt ,rrp ,corba_iiop ,corba_iiop_ssl ,rmi_activation ,rmi_registry ,sybase_sqlany ,netinfo_local ,wfremotertm ,icp ,epmap ,stun ,amazon_s3 ,hopster ,renren ,avast_remote ,vast ,spybot2_update ,worldfriends ,crossloop ,showmypc ,logmein ,cloudfront,
Big Data / Data enrichment
The illustration shows the silo architecture of thepast in which each system produces XDRs, counterand logs and other type of information which isstored in many different places, formats and DBs.
The information can only be correlated by speciallydesigned software and often the raw data is notaccessible to the customer.
Each request for a new report generates efforts andcost. As a result, a lot of useful information is notused and often there is a data overlap. Thisgenerates additional cost.
Big Data is not just a buzzword. This approachbrings real advantage to customers. Many datasources feed one big data storage system. Thissystem is a document storage system, which canhandle a very large amount of unstructured data.
• One data storage for all types of data
• One reporting tool
• One point of securing data
• One managed Infrastructure
http://bigdata-madesimple.com/11-interesting-big-data-case-studies-in-telecom/
Big Data Integration
Probe
Raw Packet Data
XDR as UDP stream real-time
ElasticSearch
HadoopCluster
XDR and KPI fetch (not real-time)
Kibana BI
HadoopClusterHadoopClusterHadoop Cluster
(Long time storage)
Mongo DB(Short term storage)
Elastic Search
Mid size provider data calculation is 4,5 million subscribers and 100Gbps data load
6 – 7 billion CDRs a day, 5 TB storage needed for the CDR per day
Revenue assurance, Fraud detection, Network planning, SLA…..
Real-time trouble shooting
Big Data
Generic Big Data Integration
Mongo DB
Probe
Raw Packet Data
XDR as UDP stream real-time
Mongo DB(Short term storage)
Real-time trouble shooting and raw packet capture
Big Data
Probe
Kafka Instances
Kafka Instances
Kafka Instances
Kafka Instances& Cubro Interface
Elastic search
HadoopIBM Q Radar
Many supported
outputs
Currently, the Apache Kafka (a distributed streaming platform) supports up to 50 different connectors to interconnect between several data sources like the Cubro Probe.
Some examples include: Vertica, Syslog, Hadoop, SQL, Hbase, InfluxDB, Hazelcast, S3, DynamoDB, Splunk …
Raw capture
Metadata handler
Probe
Probe
Original traffic
Original trafficMetadata handler
Metadata traffic
Metadata traffic
The meta data handler is the same hardware as the Probe, but with a different software. This software receives the original coder and converts the CDRs in other formats to support different monitoring system. For instance, converting to Kafka to Big Data analysis.
Big Data cluster
3rd party monitoring to reuse the existing data for other applications
The full solution for 1,5 TB This drawing shows a full 2G/3G/4G solution for 1,5 TB user plane traffic monitoring.
The first section is aggregation and load balancing.
The second section shows the probing layer.
To store such amount of data a clustered database, like Hadoop Elastic search is needed.
For this amount of data the cluster size is approx. 80 servers big.
Typically 7 – 8 billion CDRs are produced per day.
To store this amount of CDRs 3500 TB is needed per day.
Massive data aggregation is a must.
Big Data
Cubro BI iVision
▪ The Cubro BI works based on the Data in the Mongo DB
▪ The Cubro BI delivers
▪ KPIs
▪ Ladder diagrams
▪ Capture-Files
▪ Dashboards for live traffic
Some more KPIs
▪ Bearing Performance Analysis
▪ Session Management Performance Analysis
▪ Attachment Analysis
▪ Paging Analysis
▪ Service Request Analysis
▪ Tracking Area Updating Analysis
▪ PDN Link Analysis
▪ Relocation Analysis
▪ DNS Quality Analysis
▪ TCP Quality Analysis
▪ CSFB Message Quality Analysis
▪ CSFB Location Updating Quality Analysis
▪ Get/Post Process Performance Analysis
▪ PDP Process Analysis
▪ Download Speed and Time Analysis
▪ X2 Equipment Management Analysis
▪ X2 Switch Performance Analysis
▪ UU Switch Performance
▪ UU Signaling
▪ KPI Real-time Query
Application Analysis (Business TOP)
Real time subscriber viewSelect phone numbers or range Select phone typeSelect time frame
Drill down to application for this specific user
Another drill down for more details
Real time user location tracking
See the location, by drilldown you see also the application used Start query
Enter subscriber
CDR browser
Select the fields
Select the CDRSelect time frame
Select subscriber #
Ladder Diagram
Specific call/session full real capture
It is possible to configure parameters which triggers a full capture (all layers).
The files are stored in “.pcap” and can be downloaded via FTP. The deep storage is limited by the disc space. The files are meted out and truncated for easy handling.
Monitoring System offload loading
The smart Cubro Probe solution to use your existing monitoring system even when your data traffic is growing exponentially!
Monitoring offload
Reduce Monitoring cost by off-loading the existing monitoring system.
Reducing not only Capex but also Opex goes down because of less hardware resources.
Still the full meta data information on user plane traffic.
Monitoring offload
Monitoring System offload loading
The XDR output provides the full meta data of the user plane.
The IMSI filtered output provides the full packet stream of several 100 or a few thousand IMSIs for a full L7 analysis.
Full GTP-C output (Signaling only)
Monitoring offload
Monitoring System offload loading
SCENARIO
▪ Existing Monitoring System offload
▪ Separation of user plane and control plane
▪ IMSI filtering for user plane
▪ User plane XDR generation
▪ Full signaling forwarding
Monitoring offload
Signaling and user plane
First, we separate signaling traffic and user traffic.
The signaling traffic is forwarded to the existing monitoring system and is processed there.
The load on the signaling is low and even when the user load is growing the load on the signaling does not grow in the same way. It is more user amount related not bandwidth related.
Monitoring offload
Smart session user data filtering
In the second step the user traffic can be filtered.
To reduce the load, there are many options where we can filter - IMSI, IMEI, APN, Network Element Filtering, CELL ID and more.
It is possible to filter on a million IMSIs per unit. This can be used to monitor only gold customers or to fully monitor on capture specific -IMSI for special purposes, LI, DPI, ...
Monitoring offload
User Data XDR generation
We can produce meta data XDRs for“all” user data traffic. These XDRs hold a matrix of relevant information coming from the network.
These XDRs can be filtered to reduce the load.
The interval of the XDRs can be configured per session and per time to deliver the needed granularity of the information.
The XDR is an open format to support any database.
Monitoring offload
Monitoring System data enrichment
The smart Cubro Probe solution can enrich the data to use your existing monitoring system even when you miss some data!
Monitoring enrichment
Cubro Vision Application
This application runs on any server which supports DPDK. This feature is needed to process the XDRs very fast.
The XDRs are stored in a Mongo DB.
KPIs are calculated and also stored in the DB.
Mongo DB is non-SQL with very fast access and a very smart scaling approach to store multi TB of traffic.
MongoDB (from humongous) is a Free and open-source cross-platform document-oriented database program. Classified as a NoSQL database program, MongoDB avoids the traditional table-based relational database structure in favor of JSON-like documents with dynamic schemas (It calls the format BSON), making the integration of data in certain types of applications easier and faster. MongoDB is developed by MongoDB Inc. and is free and open-source, published under a combination of the GNU Affero General Public License and the Apache License.
Typical application DPI enrichment
DPI
The typical DPI solution can only handle pure IP traffic (GI), but in mobile networks some information is not available on this interface. For example, location information is not there or not in real-time.
Cubro solution can help to generate needed information in an XDR (example CELL ID + MSISDN + USERIP) to enrich the DPI data with the CELL ID.
Full raw packets
Full raw packets
Cubro Probe
Monitoring enrichment
Cubro Product Portfolio
ProbesMobile Probes and FlowVista
Network Packet BrokersPacketmasters and Sessionmasters
TAPsOptical, BIDI, Copper, Flex, Converter, Aggregation
BypassOptical and Copper
Misc.Breakout boxes and Media Converters
Thank you
EMEA
Cubro Network Visibility
Ghegastraße 1030 Vienna,
Austria
Tel.: +43 1 29826660
Fax: +43 1 2982666399
Email: [email protected]
Cubro US337 West Chocolate AveHershey, PA 17033
Tel.:717-576-9050Fax.: 866-735-9232
Email: [email protected]
Cubro Asia Pacific
8, Ubi Road 2 #04-12 ZervexSingapore 408538
Tel.: +65-97255386
Email: [email protected]
North America APAC Japan
Cubro Japan
8-11-10-3F, Nishi-Shinjuku, Shinjuku, Tokyo, 160-0023 Japan
Email: [email protected]
