Where do we go from here? Lessons and Landmarks from ?· Gateway with Kafka Spark Streaming Compute…

  • View
    212

  • Download
    0

Embed Size (px)

Transcript

  • Where do we go from here?Lessons and Landmarks from Real-World Cisco UCS Big Data Deployments

    Han Yang, Senior Product Manager, Cisco

    Raghunath Nambiar, CTO, Cisco

    BRKINI-2021

  • 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

    Cisco Spark

    Questions? Use Cisco Spark to communicate with the speaker after the session

    1. Find this session in the Cisco Live Mobile App

    2. Click Join the Discussion

    3. Install Spark or go directly to the space

    4. Enter messages/questions in the space

    How

    cs.co/ciscolivebot#BRKINI-2021

  • Extracting value from information is not primarily a matter of how much data you have . . ., its how much you use them to create new or better approaches to doing business.

  • Less than half of structured data is actively used in decision making

    Less than 1% of unstructured data is analyzed or used at all

    May, 2017

    How are you managing

    your data as part of

    your business strategy?

    https://hbr.org/2017/05/whats-your-data-strategy

  • What is your data

    acquisition strategy

    from new product and

    services?

    Are you consolidating

    your existing data sets

    into a data hub?

    Do you have AI center

    of excellence to pool

    talent?

    What is Your Corporate Strategy in the World of AI?

    Andrew Ng:

    Stanford Professor

    Led Google Brain

    Drove AI at Baidu

    EmTech, Dec, 2017: https://www.youtube.com/watch?v=NKpuX_yzdYs

  • 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential

    7

  • 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential

    15,000 206

    31 8,000

    connected athletes countries

    buildings rooms

    9 milliontickets sold & validated

    85,000soldiers &

    policemen

    70,000volunteers

    Olympic Village with

    fro

    m

    25,000media personnel

    42sports

    in

    Every person, venue,

    device was

    connected.

    It was the most

    connected event in

    the history

  • 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential

    The first truly connected Olympics

    5 billion

    1 trillion visitors to official Olympic web sites

    TV spectators from more than 200

    countries. Thats about 66% of world population

    hours of online video contentthats about equal to one person watching 20 years of video without a break

    hours of video watched via mobile

    170,000

    130 million

    54 million data feeds transported and processed

  • 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential

    Education: Connected

    classrooms over a stable,

    secure, reliable network.

    Crisis Management:

    Protect your business by

    seeing whats happening in

    real time.

    Health Care Management:

    Electronic Medical Records

    to spot trends in chronic

    diseases.

    the possibilities are endless.

    Be more effective. Make better decisions. Protect and secure.

    Operations: Massive scale of technology and products to

    plan, design, optimize your data center.

    BRKINI-2021 10

  • 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential

    Lessons being learned from smart cities

    finance, construction, transportations, media, utilities, health

  • 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential

    Cities

    1990: 10

    2015: 29

    2030: 41

    1990: 270

    2015: 488

    2030: 662

    Mega-Cities >1M Pop.

  • 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential

    Today, cities consume 75% of worlds resources and

    generates 80% of greenhouse gases

    2015

    3.9 Billion

    2030

    5.1 Billion

    Urban Population Growth

    4.3 Billion

    2020

    There are several areas of improvement

    Make cites cleaner, safer, efficient and sustainableSmart City

  • 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential

    Smart Cities: Dream vs. Reality

  • 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential

    Areas of

    Improvement

    Energy

    Transportation

    Environmental Monitoring

    Safety

    Emergency Preparedness

    Make the City Smart

  • BRKINI-2021https://hbr.org/2013/01/why-it-fumbles-analytics

  • 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

    Are You the Cause of Analytics Fumble?

    Traditional IT Project Big Data Analytics Project

    Typical Project

    Install ERP System

    Optimize supply chain performance

    Typical Project

    Understand customers needs and

    behavior

    Predict future growth markets

    Overarching Goals

    Improve efficiency

    Lower Cost

    Overarching Goals

    Challenge assumptions & bias for

    decision

    New insight into customer

    Project Structure

    Develop detailed plans to deploy,

    manage, and train

    Project Structure

    Discovery driven: Hypothesis -> Get

    Data -> Experiment

    BRKINI-2021 17

  • Types of Analytics

    1. Business Analytics

    2. Big Data Analytics

    3. IT Operation Analytics

    4. IoT Analytics

    5. Machine Learning

  • BRKINI-2021 19 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

    Areas of FocusCisco Validated Designs, Case Studies, Performance Benchmarks

    Business

    Analytics

    Big Data

    Analytics

    IT Operations

    Analytics

    IoT

    Analytics

    AI, ML/DL

    Analytics

    VSOM

  • Big Data Analytics

  • 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

    IOT Data

    Sources

    Web Data Sources

    Streaming Data Sources

    Gateway with Kafka Spark Streaming Compute Nodes

    HDFS

    No SQL

    Hbase,

    Cassandra

    .

    Apps

    Analytics DB

    Se

    rvin

    g L

    aye

    r

    Speed Layer Batch Layer

    Lambda Architecture on UCS

  • 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

    Cisco UCS at all layers, fully validated architectures

    with all major players

    Kafka

    Cisco UCS C240

    Real-Time Data Store

    UCS C220/C240Data Inject (CoAP/MQTT.XMPP Data Processing

    Batch

    Real-Time

    D

    a

    t

    a

    A

    n

    a

    l

    y

    t

    i

    c

    s

    F

    o

    g

    F

    o

    g

    F

    o

    g

    F

    o

    g

    F

    o

    g

    F

    o

    g

    Big Data Store

    UCS C240/C3160Serving Layer

    Speed Layer Batch Layer

    DATA

    Aggregator

    Cisco UCS C240C800/UCS Mini/UCS C240

    ISR 8x9 with 4G LTE and Dual

    802.11n a/g/n (WiFi) Radios

    Managed by Cisco FogDirector

    Cisco UCS Infrastructure for Big Data & Analytics

  • IT Operations Analytics

  • 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

    IT is accountable for:

    Becoming proactive and predictive

    Meeting/exceeding SLAs

    Reducing expenses

    Providing end-to-end operational visibility

    MORE

    DEVICES

    MORE

    DATA

    MORE

    APPSDelivered

    Continuously

    Multi-vendor, siloed IT

    infrastructures create management

    complexity

    MORE

    MOBILITY

    Network Compute OS ApplicationsStorage

    Need for ITOA

    IT silos create obstacles

    to consistent application,

    infrastructure performance

    Lack of visibility puts SLAs in jeopardy

    BRKINI-2021 24

  • 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

    Required: Real-time Insights, Comprehensive VisibilityIT operations analytics demands innovation in analytics software and infrastructure

    Analytics Software Analytics Infrastructure

    Single software platform integrates across

    infrastructure silos, enabling visibility to data anywhere

    Flexibility to identify, analyze new data sources

    Fast time to value

    Comprehensive IT management functionality

    to improve IT productivity

    +Monitoring Incident

    Mgmt

    Problem

    Mgmt

    Capacity

    Mgmt

    + +Low TCOConsistent, split-

    second

    response times

    Highly

    scalable

    + +

    IT needs an analytics platform that can meet their needs today and tomorrow

    BRKINI-2021 25

  • 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

    IT Operations Analytics on Cisco UCS with SplunkSplunk provides powerful analytics software platform for IT operations

    End-to-end operations visibility

    Enhances productivity

    Fast time-to-value

    Full continuum of IT management

    Smartphones and Devices

    Online Services

    PublicCloud

    Security

    Telecoms

    NetworksWeb

    Services

    Web Clickstreams

    RFID

    Call Detail Records

    On-Premises

    GPS Location

    Databases

    Packaged Applications

    Energy Meters

    Storage

    PrivateCloud

    Servers

    Messaging

    Desktops

    OnlineShopping Cart

    MonitoringProblem

    Management

    Capacity

    Management

    Incident

    Management

    Index Untapped Data:

    Any Source, Type, Volume

    + + +

    BRKINI-2021 26

  • IoT Analytics