Pathways to InsightData Platforms

2

Organisations with effective Data Platforms, can transform their data into insights to identify new needs, wants, products and services, with clarity and confidence.

The cloud has opened up infinite possibilities for data capture, analysis and manipulation. These exciting opportunities must be tempered with the risks of working with personal data and providing adequate security, within a highly regulated environment. At the same time, there is a need to be aware of cultural and public sensitivities towards the use of personal data.

What to do with data and how to leverage it best faces competing demands from across your organisation.

Data analysis and management

• Collect, collate and make sense of all the data collected

• Identify and understand which data is useful

• Manage costs of handling data

• Know what questions to ask to harness data effectively

• Scale insight generation quickly and effortlessly

• Use data effectively to form the foundation for AI

• Ensure technical competence matches data needs

Become customer centric

• Understand customers and changing expectations

• Put data collection at the heart of every customer interaction

• Improve personalisation of offerings

• Use insight to tailor products’ features

• Innovative ideas for better service

Governance and regulatory environment

• Be secure by design

• Curate and govern the data flowing in and out

• Assurance of service continuity

• Align with risk management objectives

• Manage third party risk

• Ability to manage data within parameters of compliance, security and changing regulation

3

They offer a compelling alternative, switching focus to solving business challenges by having the data to drive the insight and plot a course to the solution. Born-digital companies have demonstrated how to bypass legacy data systems and leverage cloud scale data processing and machine learning to drive insight into all facets of their business.

For an organisation that has relied on traditional data collection and storage, viewing its data stores as a burden, a change in mindset and culture is needed, to understand how data is the key to understanding and driving future business objectives. Data Platforms extend the reach of data collection by managing the whole range of functions – collecting, storing, analysing and integrating information and distributing the results as insight in real time, in one place.

Using Data Platforms

Data Platforms on the cloud offer tremendous promise to an organisation, through the possibility of immense storage, analysis of vast amounts of data, agility and responsiveness and by providing practically infinite scalability for machine learning workloads.

Data Platforms have features that can be used to provide intelligence that goes way beyond traditional reporting solutions, e.g. in addition to point of sale, they can identify SEO spend, website interactions, how long the customer was on the website, the platform they used, engagement with customer service and any feedback they provided.

Once these features become available and their impact is identified, data engineers can harness their power for additional insight applications resulting in fundamental changes to how you operate.

For this to happen, the data and insight strategy needs to be a part of a wider digital transformation programme.

4

Driving insight with Data Platforms

On the surface, tying together multiple systems looks quite easy. You simply pull data from one system into another and do data transformation. However, the reality is much more complex and nuanced.

We can help by designing and developing a cohesive Data Platform that makes the transition as smooth as possible. Our data engineers can weave a number of disparate products into a holistic Data Platform that is designed around your business needs with key benefits such as:

• Storage capacity for handling data in multiple formats

• Security and access requirements appropriate to the data

• Complex data manipulation capabilities for processing large data volumes cost effectively

• Fast streaming capabilities enabling real time business services

• Governance with tracking through reliable metadata and lineage

• Business intelligence through query and reporting of transformed data

• Machine and deep learning for the execution and training of models

• Analytics and visualisation supporting the use of AI or API endpoints

Pathways to Insight introducing our Data Platform service

Pathways to Insight is designed to align both your existing data and solutions that will meet your future needs, onto one Data Platform.

Our modular approach to Data Platforms lets you focus on key areas that will drive the greatest benefits. Our modules are designed to prioritise your business planning, smooth the transition and help you identify and optimise the key data assets that drive your success.

5

6

Developing insight for business performance

1. Creating a Data Platform strategy for your organisation

Our teams will develop a strategy with you to identify the key drivers and metrics that measure your success across your organisation.

• Assessment of your data landscape in order to identify and prioritise your data solutions portfolio concentrating on the three Cs: reducing costs, increasing competitiveness and improving compliance:

‒ Cost, clearly articulated cost reduction metrics that need to be achieved to deliver lower total costs. This ensures the Data Platform is built on value creation

‒ Competitiveness, the case for increasing speed of analysis, improved customer insight and data-driven action. This underpins the investment case for investing in Data Platforms

‒ Compliance, a regulatory and compliance case is compelling. With an increasing focus on regulation and stringent punishment for non-compliance along with reputational risk for security breach

• Define key objectives, target state and KPI’s informed by your business strategy

• Identify how the Data Platform capabilities will support the business strategy

• Produce a short term, medium term and long-term Data Platform roadmap with target dates, prioritised data subject areas.

2. Architecture and DesignCarve the Data Platform portfolio into a series of work packages and for each one:

• Define and design the Data Platform components

• Design the Data Platform to be implemented, including selection of the target cloud vendor

• Define success criteria including validation and verification

• Produce the work package plan.

3. Engineering and OperationsCustom build a multidisciplinary Data Platform team which consists of:

• User experience designers, enterprise architects, agile experts, software developers, cyber analysts, security architects, data engineers and scientists, DevOps and software engineers.

Deploy our proprietary methodologies and frameworks to accelerate work package delivery.

• This includes the 6point6's Software Factory, a proprietary framework that produces a consistent approach to fully automated Continuous Delivery at scale that adapts to multiple teams and a diverse range of technologies

• Leverage proven products and Platforms such as 6point6 Cloud Gateway to help data migration to the cloud at the pace and in the manner that suits your business

• Harness the power of our technology partners to find the right match for your data needs.

7

4. Assurance and GovernanceProvide assurance to all your stakeholders:

• Evidence that the Data Platform objectives are being met and the delivery phase conforms to the design

• Ensure data sources can be identified and verified

• Ensure Data Platform can be adapted and amended as required

• Establish governance and assure compliance across the supply chain.

5. Cyber Security for the Data Platform

Adopt secure by design principles:

• Help your existing cyber security practice to evolve as you transition to cloud

• Develop a proactive plan for cyber capability, based on your specific data, security needs and priorities, ensuring that agility and responsiveness are not compromised

• Create cloud-specific security e.g. perimeter security design, encryption or third-party providers, or adapt applications, depending on identified priorities and perceived risks.

6. Continuous HandoverSupport and mentor your in-house team as they gain ownership and eventual full control:

• Work collaboratively handing over knowledge and physical artefacts in incremental stages

• Help your technologists exploit Platform features, to adapt them to additional uses and build new digital products, ultimately making you self-sufficient

• Encouraging staff to embrace a digital and cloud culture and to value data for its potential.

8

Data Platform Scenarios: our experience distilled into repeatable patterns for common use cases

Over time we have developed and refined Data Platform scenarios from our in-depth work on a large number of Data Platforms. These scenarios enable us to accelerate the strategy, design and engineering process.

1. Scaling for availability

Organisations can be paying for capacity they don’t need. With the amount of data received, processed and stored increasing at ever faster rates, businesses need to be sure that their infrastructure can cope, and it can be tempting to overestimate data needs. We have the experience to match your Data Platform capacity to your business requirements, ensuring that you pay only for the capacity and support you need, creating value from your Data Platform.

2. Containerisedflexibility

Big Data technologies have typically required their own infrastructure to run, either systems such as Hadoop or cloud native offerings such as AWS’s EMR. Increasingly organisations are adopting container solutions, such as Docker and Kubernetes. This split in between own infrastructure and PaaS is costly from both an operational and process perspective. Containerised data processing enables data processing applications onto existing container Platforms, thereby aligning infrastructure and reducing costs, without reducing performance.

3. Decisions in the stream

We move monolithic, siloed and batch-processed data to a real-time, distributed, event-driven architecture, unlocking the benefits of your data faster and making decisions on the stream, generating new opportunities.

4. From puddles to data lakes

Data that is spread across multiple data silos, possibly with different access patterns and running on varied technologies, inflates costs. In addition, an inability to see the big picture, increases the risk of missed opportunities. We reduce the cost of data ownership and improve data insight, by appropriately consolidating data within feature stores, on modern Data Platforms.

5. Proof to production

Innovation labs and proof of concepts are increasingly common within businesses, allowing businesses to keep on top of new technologies and evaluate their fit and benefits. However, they struggle to get these solutions through into production. Proof to Production leverages our experience achieving the transition into a fully running Data Platform in production environments.

6. Integratingtheflow

Merging multiple systems together may seem straightforward: you simply pull data from one system into another. However, the reality is much more complex and there are pitfalls: schema differences, data duplication and join-up, different ID realms and reference data mapping all create challenges. Our data integration services allow businesses to be confident of successfully joining together not only their systems but also their data, processes, policies, governance, standards and tooling.

6point6 most common Data Platform scenarios:

9

7. Chaining the ledgers

Blockchain is gaining traction especially for distributed applications. We bring our experience of deploying and integrating blockchain into back office systems. We optimise the security benefits of encryption without compromising on agility and speed.

8. Features as a Feature

Machine and deep learning works loads are increasingly important to businesses. Especially when early pilot projects have shown promise. Feature Stores are an important part of a Data Platform, as we now provide the ability to trace data from ingress through to feature generation and into a comprehensive curated feature store that support all machine learning efforts.

Our 6point6 skills and knowledge base

We have teams of analysts, designers and engineers with hands-on experience of Data Platforms in organisations across every sector.

We deploy small teams with the right mix of skills and experience to solve your specific Data Platform project in its entirety.

Known as Pods, our teams give you ready-made, modular delivery capability focused on the real-world challenges that you will face when moving to Data Platforms.

Benefits of pods

1. Collaborative and supportiveWe aim to minimise impact on your team’s day-to-day role, handing over control in manageable increments and working collaboratively to ensure your staff are brought on the cultural journey to adopt the Data Platform and to gain the skills required to sustainably change the operating model,

2. Autonomy and value for moneyOur Pods are mission directed. By adopting our frameworks and methods, we gain consistency in ways of working that leverage the architecture and programme delivery teams.

3. ScalingOur outcome-focused Pods can be scaled: in the early life of the Data Platform programme our delivery managers focus on optimising the balance between number of workloads and number of team members per Pod. This continuous monitoring and balancing leads to better insight and certainty regarding delivery dates and predictable progress.

10

11

12

Where have we done Data Platforms before?

The Home Office

Our consultants, architects and engineers worked in partnership alongside Civil Servants and other Home Office suppliers to jointly achieve one of the largest cloud transformation projects ever across the UK Government and the largest AWS migration in Europe.

The Home Office is fundamental to the security and safety of the UK, acting as the lead government department for immigration and passports, drugs policy, crime, fire, counter-terrorism and the police. New technology is transforming the way immigration services are delivered; making it faster and easier for members of the public to apply for visas and for Home Office staff to make millions of decisions each year. These systems of critical national importance are dependent on a complex, cloud-based infrastructure. The Home Office identified that its existing hosting platform lacked the scalability and stability necessary to support its ambitious transformation agenda. Faced with the need to improve delivery efficiency, test capabilities and service performance, they selected the Amazon Web Services’ (AWS) hosting platform.

With the existing architecture consisting of more than 1,400 servers containing a vast amount of data, 27 data hosts and 36 external interfaces, this was not a simple lift and shift; it involved the migration of a very large infrastructure and vast amounts of data whilst maintaining continuity of business services.

6point6 played a key role in planning, enabling and executing the Cloud Migration to Amazon Web Services (AWS), by providing Digital Transformation, Cyber Security and Data Platform services to the Home Office in one of the largest cloud transformation programmes undertaken by government. The redesigned and simplified infrastructure has improved the stability and resilience of the Home Office’s cloud environment, enhanced reliability of deployment and improved the security of the production service. It has also increased the ability to scale and to optimise infrastructure costs. This has resulted in more efficient and better protected digital services, ready and available for operational business and live services.

13

Our thoughts

Why should I invest in a Data Platform?Data is the number one asset that you have as a business. Every interaction with customers, suppliers and employees leaves a trail of data that if collected, curated and processed can be used to further understand your business and where the opportunities lie.

Moving to a modern Data Platform is a key enabler and accelerator for Digital Transformation and business agility - data driven insight must go alongside innovation. You can’t go from looking in the rear-view mirror to predicting the future in a single bound.

Can’t I just do analytics, why do I need a Data Platform?The basis for great analytics is great data, and data in context takes analytics to a whole new level. The traditional query and reporting paradigm is good for understanding how many widgets were sold but does not tell you what drove that buying decision and how many widgets will be sold next week to the same customer segment.

This is where Data Platforms come into play by extending the reach of data collection. Data Platforms collect, integrate and distribute insight, going way beyond traditional reporting solutions and open up opportunities for more innovative applications, as consumer demands evolve.

14

15

Can I use a product to make Data Platforms simple?There is always more than just technology at play when delivering a Data Platform. While there are many data products available, these are no more than tools to aid in a much larger process and cannot be relied on as a ‘silver bullet’ to solve the task single handed. Data engineering is now a core capability that is able to weave a number of disparate products into a holistic Data Platform that can manage security, data ingestion, embedding machine and deep learning, as well as the dissemination of insight through compelling visualisations and decision-making APIs.

What is a Software Factory?DevOps has transformed how software applications are built, tested and released in recent years by building bridges between those who create digital products and services and those responsible for running them reliably using automation and in many cases cloud. But in solving one problem, it has created another. In many organisations DevOps had become largely an unregulated cottage industry subject to continuity risk and lacking in discipline and governance.

6point6’s Software Factory is a proprietary framework which produces a consistent approach to fully automated Continuous Integration and Continuous Development at scale. It is able to adapt to multiple teams and a diverse range of technologies. It is a pattern-based approach that empowers developers to create their own automation in a consistent way, evolving away from the DevOps ‘hero culture’. It is the foundation on which for Data Platforms and Digital Transformation rests.

Technical Debt. What is that?Financial debt is a good metaphor for what happens to an IT estate over time. Each shortcut, for example to save time or cost, is like taking out a small loan. Loans must be paid back, or they must at least be measured and quantified in order to inform the decision on whether or not to take out more loans. If not, then bankruptcy is inevitable at some point in the future and in IT terms, the risk-laden lead up to this is known as a ‘burning Platform’.

The reality is that all repayments that were missed over the years (all the missed upgrades, failures to patch, lack of code refactoring) must now be made in an accelerated time frame and more often than not, resulting in one very painful transaction. In our experience, the demands of regulatory change, disruption from competitors and cyber security breaches are common catalysts for a technical debt crisis.

Get in touchGARY RICHARDSON

Managing Director Emerging Technologygary.richardson@6point6.co.uk

About 6point6

Integrating digital technology into your business can result in fundamental changes to how you operate and deliver value to your customers. To go digital is to reinvent yourself to the core, opening yourself and your clients to a world of possibilities.

6point6 is a technology consultancy. We bring a wealth of hands-on experience to help businesses, including financial services providers, media houses and government, achieve more with digital. Using cutting edge technology and agile delivery methods, we help you reinvent, transform and secure a brighter digital future.

Visit us on www.6point6.co.uk Twitter: @6point6ltd LinkedIn: linkedin.com/company/6point6SIX.SIX.021