Sample Doc

Response to Airtel RFP Content Homogenization Platform

2012©TimesMobile Limited Classified Document

Usage by Authorization only

- 1 -

Airtel Content Homogenization Platform– Powered by timesmobile

Table of Contents

1.0: INTRODUCTION...................................................................................................................................................3

1.1 SCOPE OF WORK: REFERENCE AIRTEL RFP...................................................................................................3

2.0 REQUIREMENTS FULFILLMENT - IMPLEMENTATION PLAN..............................................................5

2.1 PROJECT EXECUTION PROCESS.......................................................................................................................52.2 TIMESMOBILE TO UNDERTAKE STEPS...............................................................................................................6

3.0: PROPOSED TECHNICAL SOLUTION............................................................................................................9

3.1 FRAMEWORK (LEVEL 1): OVERVIEW OF CORE BUSINESS VALUE DELIVERY...............................................103.2 FRAMEWORK (LEVEL 2): OVERVIEW OF CORE BUSINESS VALUE DELIVERY...............................................113.3 FRAMEWORK (LEVEL 3): SYSTEM DESIGN & COMPONENTS........................................................................143.4 PROPOSED DEPLOYMENT: ARCHITECTURE..................................................................................................153.5 PROPOSED DEPLOYMENT: HARDWARE & SOFTWARE SPECIFICATIONS.......................................................163.6 PROPOSED DEPLOYMENT: ESTIMATED COSTING FOR BASIC INFRASTRUCTURE..........................................17

3.6.1 Deployment at Airtel Data-centre – Main Operational Centre...........................................................173.6.2 Deployment at Airtel Data-centre – For Redundancy.......................................................................18

4.0: Project Plan............................................................................................................................................................19

Document Objective

This document intends to provide technical insight into the features and setup of Content Homogenization Platform platform, being offered to Airtel. Times Mobile proposes to evolve and deploy this metadata driven system in resolute architecture and setup environment as required by Airtel. This document also provides the insight into system’s architecture, deployment plan and operational metrics. This document serves as standard response to provide detailed proposal plan for agreement between times mobile and airtel; and is intended to be a reference handbook for all proposal metrics.


This document is part of standard product Usage by Authorization only- 2 –


1.0: Introduction

Summary

In response to the Request For Proposal (RFP) issued by Airtel for Content Homogenization Platform, timesmobile proposes the solution which will cater to all requirements as mentioned. This section summarizes the scope of work as required by Airtel for Content Homogenization platform. Here this section highlights the business values and offerings of this platform.In this proposal, timesmobile intends to provide the insight into modular and scalable design along with adherence to protocols and security standards of contemporary solution architecture.

The Content Homogenization Platform is intended to be employed as a central provider of content to diverse consumer domains viz. applications, URLS and interfaces. The platform is intended to be meta-data driven, for its role as integrator, moderator and refiner of diverse CMS sources. Therefore this document lists the salient deliverables and objectives of design & development tasks in this project. Constituent elements of Content Homogenization Platform which includes the high level and component architectural review has been illustrated in this document

1.1 Scope of Work: Reference Airtel RFP

Following is the scope of work stated by Airtel for its partner who will develop & deploy Content Homogenization Platform. Primarily the platform is required to generate metadata of all the audio/ video/imagery content and shall relate the same with the content ingested in the CMS(s).

Post creation of metadata, the homogenization system will do the following activities

1. Accept Metadata from CMS(s) (Onnet and Nearnet) in varied formats

2. De-Dup Data to remove multiple Metadata entries of the same content within a Product. Various automated tools can be used to quicken and ensure error free de duplication

3. Edit/update the metadata fields and ensure that all the fields contain the correct information in terms of factual info, spellings, credits etc

4. Enhance the metadata to contain further factual information, content type information ((Tempo, Bass, Pitch, sound genre etc) and lyrics related information about the content piece (Sub-Genre, Intent, event etc)

5. To achieve the above various manual and automated processes to be used like

Web crawling, direct CD label ingestion or any other similar method for factual information

Use of spectral analysis or similar tools for getting content information and a expert team to create information about the lyrics/mood of the song

6. System should also verify Copyright information related metadata across web data, Rights letters submitted to Airtel/CMS (s) Owners and ingested CP/label/Pay to names ingested at CMS(s)

7. The system should link all various content pieces together so as to create a interconnected database of content items with a exhaustive, correct and linked content metadata




8. Homogenization system will also provide guidelines to CMSs to minimize errors and standardize metadata entry on their systems. Ex- Sharukh khan spelling to be standardised

9. The system will expose the rich and complete metadata to downstream systems with below guidelines

System shall be dimensionally capable of supporting online and offline queries from Consuming Systems

A Web query URL shall be provided to business tears

The Metadata store shall provide connectors to the Consuming Systems/ CMS (s) for accepting data.

The Metadata store shall be capable of providing different types of access to the other platforms like read only, write only, read and write etc

10. The system will also provide a DataStream back to the core CMS with the updated and rectified metadata.

11. The system should use automated tools where ever possible to update ,rectify, dedupe etc

12. Homogenized metadata will be Bharti’s Property.

13. The system should be scalable as per Bharti need.




2.0 Requirements Fulfillment - Implementation Plan Summary

Based on the above mentioned requirements, timesmobile hereby submits the proposed technical and process solution to cater to all action items…

This section introduces the proposed system and its development metrics in adherence to specifications stated by Airtel.

2.1 Project Execution Process

Summary

This section highlights the stages in Inception – to – Release stage of SDLC of Content Homogenization Platform.

Main & Sub Stages in Development of Content Homogenization PlatformS. No.

Stage / Task Description

1 System Study Project initiation will occur with system study which will incorporate development and validation of functional requirement specifications.

The FRS will be guiding document for development and validation of each feature & functionalities.

The FRS will be accompanied by a detailed project plan and task metric sheets.

2 Project Plan Approval Approval of project plan by Airtel and management team of Times Mobile will be agreed.

Post approval of plan, project head and contact coordinates and protocols will be fixed.

3 Manpower Allocation Allocation of manpower with timeframe of association will be circulated and will be implemented as per the agreed project plan.

4 Scheduling Schedule of following will be agreed and shared with Airtel:

Timeframe of allocation of skilled resources and teams

Progress of project with tools such as project trackers

Schedule of unit releases, module releases, beta release and business release will be published.

5 Engagement with CMS Owners / Content Providers

During the begin phase (Logical Design Phase), communication between timesmobile development team solution architects, data integrators and CMS specialists will occur with external CMS owners / content providers.

These interactions will focus on attaining the knowledge of data signatures, content meta definitions etc being used by CMS owners.

Equipped with this knowledge, the solution architects will finalize the templates and metadata fields for evolution of Standardized Metadata in central CMS repository.

6 URL Error Elimination & Common Rules Definition

Evolution of standards for Metadata creation involving creation of Standard Template, Standard Naming Conventions, Names & Keywords Library and Labels Dictionary in the system.




The standardization process will enable the Content Homogenization Platform to eliminate the errors in URLs being used to access external CMS systems.

Standardization will also involve usage of uniform data exchange protocols from external CMS.

7 Enforcement Enforcement to adhere with new evolved data exchange mechanism will involve interaction of diverse teams from times mobile with CMS providers.

8 Phased Integration of CMS First release of Content Homogenization Platform will incorporate integration of one or few external CMS. On proving the flawless workflow, content metadata creation and linking with these CMS system, integration with subsequent providers will occur.

9 System Rollout Rollout of system for full fledged operation will be done after successful trail and operation of system with entire workflow with from CMS provider to consumer loop.

2.2 timesmobile to Undertake Steps

Summary

This section highlights the methodologies and tasking to incorporate the features, functionalities & capabilities in Content Homogenization Platform as required by Airtel.

Functional Requirements Fulfillment S. No.

Airtel Requirements How to Fulfill?

1 Accept Metadata From External CMS

The development phase will envisage for capability to create a repository of agreed templates of metadata. The templates will define metadata signature viz. essential fields, optional fields, and structure (hierarchy) of metadata values.

One or more data exchange templates (similar to XML DTD/ XML SCHEMA), can be used for receiving data from a specific CMS.

Varied templates can be used with varied formats of data such as video / audio / images etc.

The open standards data exchange mechanism will be applied to receive content metadata without dependency on Operating System/ Legacy Systems/ Devices and Formats.

Capability to render all relevant multimedia formats will be incorporated.

2 De-Dupe Operation The system will incorporate algorithms for Data Quality Enhancement; of which removal of duplicates (De-Duping) is of utmost significance.

De-Duping Map will execute periodically on content data collated from different CMS; with objective to single Metadata for unique content. Thereby the refined metadata will free from duplicates.

3 Edit/ Update Metadata for Correction

The system will comprise of library of correctly spelled words which will be continuously updated both with manual entries as well automatic data updates from standard web entities / portals.

All entries will be checked with library and errors will be listed; enabling administrator to choose Ignore/ Update Text / Update Library options.

Over a period of operation, the platform will evolve as highly optimized with correct nomenclature, keywords, and tags.

4 Metadata Enhancement by Adding Factual

The metadata template will have mandatory fields and attributes to totally define the factual information about contents.

The hierarchically nested elements will be used to define the




Information genres, album, composer, play-list etc data. Also separate elements in template will be used to define the

characteristics of content such as bass, tempo etc. The values to the elements in template will be set by

Homogenization System either automatically by reading the labels, creator’s records etc. The absent values will be substituted by prompting administrator / content moderator to pick appropriate value from library.

5 Mood / Emotion / Genre Detection for Content Metadata

Detection of mood is to be implemented by sensing the pitch and tone of audio / video file. The program will perform the spectrum analysis and sampling of all multimedia files (audio / video) to get their signature and thereby associating the mood label with each. The sensing of mood will be done with

DWCH (Daubechies Wavelet Coefficient Histogram) measurement by program will be done with respect to a complete ortho-normal set of functions for different frequency types.

Beat & tempo detection program will test all audios in ranges 45 to 150 beats / minute.

The acoustic feature of each audio will be processed to measure the “Spectral Flatness Measure” and “Spectral Crest Factor”. These techniques will be used to find the emotion of each content and then set it in its metadata.

Genre & emotion detection will be done by two methods : - 1) Artist & Genre information is stored in database and can be looked up. 2) Some methods work analogous to detecting emotion. Additionally the system will use Internet as a pool of information on artists to categorize them in certain genre.

To detect mood of audios, the system will use speech recognizing algorithm to convert voice to text and then another algorithm will interpret certain words to moods. The algorithm for detecting of mood is based on “Automatic Mood Detection Theory from Acoustic Music Data” proposed by Lui, Lu and Zang {LLZ2003}.

6 Web Crawler and CD Label Ingestion

The project will incorporate development of abstractions for managing data objects, data repositories, collections, and knowledge processes. Briefly, the data abstraction will specify an encoding format that enables the presentation application to correctly interpret the stream of bits defining the data object. The data repository abstraction will enable the association of a persistent (unique) identifier with each data object, independent of the local file name or where it is stored at any particular time instance. The collection abstraction will define the context (expressed as metadata) associated with the archived data, while the knowledge abstraction will provide the semantic relationships among attributes of an object and a description of the processing steps used to create and validate the data object. In line of these developments, the abstract data sourcing will be done through Web Crawler programs as well as incorporation of CD labels etc in metadata.

Scripted process will be incorporated for Ingestion of thumbnails and Ingestion of metadata as well.

7 Linked Data / Content

The user interface layer will enable enables the users to gain knowledge about relationships among the members of a collection, and to quickly identify and access data objects of interest. Knowledge describing relationships between information attributes of the collections, structural relationships described by the encoding format, and relationships between the different collections, will be managed by this layer.

8 Copyright Verification

Copyright verification of external CMS contents will be performed by APIs to work in resolute assistance by trust ensuring parties.

The copyright tools will be configurable to add new trust ensuring parties, set ownerships and infringement control metrics.

9 Enriched Library of Standardized Text

Partial name match / full name match and approximation techniques will be used to correct and set the erroneous names and values. The auto correct dictionary will be supported by name match pairs in standard library. As an example the system will be able to correct on of KATREEENA to Katrina.




10 MIS The MIS (Business Analytics) module incorporate Reporting Engine to process the core logic of generating reports based on user’s query.

The MIS module will provide Report Designer user interface for designing any of the following two types of reports:

Summary Reports Detailed Reports

The MIS module will also comprise of Report Rendering Engine, which will be able to automatically generate following types of reports and email them to recipients:

Daily reports Weekly reports Scheduled reports

Several types of reports can be generated to fulfill business objectives. Some of which are:

Updated Content Report Copyright Violation Reports Content Usage Summary Report Highly Loaded Content Provider Report Content Dispersion across Product Report Content Denial Report

11 Monitoring & Troubleshoot

The Content Homogenization Platform will also incorporate utilities for monitoring the system for all errors and malfunctions. The monitoring ecosystem will be supplemented with several monitoring reports which will provide insight into the system’s errors and events. Some of these are :

Daily/Weekly/monthly reports of system performance Daily/Weekly/monthly reports of turn around time across

all steps : New content search New meta data creation Errors Found & Corrections Made Content Delivery on Request

Daily/Weekly/monthly reports of errors The monitoring reports will be broadly classified in two categories: -

information report and alert. The information reports will be delivered to inboxes of recipients

addressed which are set in configuration. The alert reports will be delivered to recipients addressed which

are in escalation list library. In addition to the reports being delivered to recipient inboxes, they

could be viewed on web user interface. From the user interface, the reports can be downloaded in CSV, TEXT, EXCEL, PDF and XML formats.

12 Capacity Planning for Maximum 70% Usage

The free disk space on deployment servers and databases shall not dip down the 30% of overall storage space. The TPS calculation shall be aggregated periodically – 5 days interval to make a demand / load chart.

Based on the TPS and server capacity occupancy, the monitoring will raise an alert when capacity ceiling of 0.70 is reached.

13 Conditions To Scale Up the System

The deployment setup / servers’ usage pattern and load handling will be monitored daily and weekly aggregated report will be produced.

The condition to scale up the system will be when any or all of the following are discovered:

When peak processor utilization exceeds 20% benchmark

When free storage / disk spaces reduces from 30% of overall disk space.




3.0: Proposed Technical Solution

This section provides in its full, the technical solution offering by TimesMobile. It covers each and every requirement mentioned by Airtel in its RFP document. Apart from requirements mentioned in RFP document, Times Mobile comes up with various Value-add features that it can offer, over and above the much needed solution of Content Homogenization Platform

TimesMobile offers to build a robust platform to cater to all airtel’s requirements and generate information/output which is quite relevant, localized and time-driven.

Following sections describe the offered solution with level-1 and level-2 logical overview of the system with their details, to understand the system in a top-down approach. Further to this, other sections describe Airtel’s specific technical requirements and their responses.



3.1 Framework (Level 1): Overview of Core Business Value Delivery Meta-data Content Homogenization Platform : A Snapshot

The following diagram illustrates L1 positioning of Airtel Content Homogenization platform.

Single Source of High Quality Data / Content

Content Homogenization Platform’s embedded capability to homogenize content based on metadata integration & profiling, will ensure that consumer applications with get trustworthy content.

With its homogenization capability, the platform is intended to enhance quality of relevance / context and well organized content.

The platform will serve as single point of contact / interface for getting contents which have originated from diverse data sources.


Usage by Authorization only - 10 -


3.2 Framework (Level 2): Overview of Core Business Value Delivery Meta-data Content Homogenization Platform : Capabilities & Service Overview

The following diagram illustrates the functional entities and business modules of Airtel Content Homogenization platform.

Modular Design Content Homogenization Platform’s design comprises of logically encapsulated modules, each

of which consisting the unique set of functionalities. Each module will embed the algorithms for its specific role and functionalities as well as applying standards of security, encryption


This document is part of standard product Usage by Authorization only - 11 –


and data communication protocols. The platform is intended to apply XML based standards for open channel integration and data

compatibility with diverse CMS domains, devices and business entities. Also the platform will be optimized for faster delivery of accurate results according to

business needs. The modular design and open integration framework enabled through communications via

virtual IP will enable overall system to be scalable to become receptor from ever increasing count of content providers CMS.

Content Homogenization & Meta-data Repository Framework : Capabilities & Service Overview

Following is the brief description of salient features and contemporary design elements of Content Metadata Homogenization platform. Homogenization & Metadata Repository Framework The Framework is a composition of software programs & executables which contain the core

business logic of metadata based CMS system, operable by sourcing contents from diverse data providers and CMS databases.

Below are the highlights of the significant elements of processing core. External CMS Configuration Utilities Manage & allocate virtual IP for

Manage & Integrate diverse CMS databases and content provider domains with the Content Metadata Refining platform.

Incorporate drivers for integrating diverse provider’s databases Copyright & Trust Verification Register trust certificates for authorized content providers.

Manage authentications from digitally signed certificates, trust channels/ parties. Authenticate copyrights and register violations.

Web Crawler Use web crawler program to crawl and lookup contents on innumerable web pages for keyword based searches and related content

Use meta-crawler program that sends user requests to several other search engines and/or databases and aggregates the results into a single list or index which will be further processed for generating refined & customized meta-data.

Content Labeling Metadata creation involves appropriate labeling of content and its attributes. Content labeling module will redefine & set label for each unique content. The labels will be

used to produce linked contents as well. Metadata Corrector Module & Nomenclature Application

Program to correct metadata of collected content. Apply standardized nomenclature to produce new metadata for storage in central repository

database. De-duping & Data Domain Creation Map Contain algorithm for performing De-Duping Operation for removal of duplicates from content

collected from diverse CMS. This will ensure atomicity of content in central repository CMS. Contain algorithm for creating data domains and identification segments in metadata prior to

storage in central CMS. Each data domain will refer to an unique data-cube in storage. This ensures faster query processing by domain based data-mining & lookup techniques in central CMS.

Linked Data Creation Module The role of Content Homogenization Platform for creating linked contents will be fulfilled through this module. It will contain programs to generate links and reference pointers to similar contents available from other data sources. Also this will impart capability to embed linked classified and advertisements according to context of content. The link thus generate will be embedded in the metadata of content.




Refined & Final Metadata Generation Module This module is the assembly line for final metadata that will be stored in CMS repository database. This module will accept input from other modules of this platform. While assembling the metadata, this module ensures that metadata signature adheres to the metadata template and standards.

Metadata Pushback Engine This engine will comprise stored procedures and update-grams confirming to SQLXML standards, to insert metadata in central CMS database/ repository.

The pushback engine will also incorporate scheduler programs to periodically execute inserts and update content in central CMS database/repository.

On Demand Content Service Engine This engine will process all inbound queries to perform metadata based search, through a fixed query templates.

The output through metadata based searches will be used to provide accurate content in very quick response time.

Data Communication & Security Protocols This module will execute several protocols to apply secure communication channels from data sources and prevent loss & corruption of content.

Several security protocols will be used for validating unicast and multicast addresses. Access & Permission Control to CMS It will contain logic to control access and grant / revoke permissions to users / user channels /

application / networks for accessing CMS repository database and platform. External CMS & Business Domain Profiling Based on usability, and trustworthiness profiles and rank of contents from external CMS will

be created. Higher ranking CMS content will be given priority for serving to user channels. Promotions & Event Manager Exhaustive registry of promotions & events linked with content metadata.

Administration Administrator’s interface for CMS management & services upkeep. Administrator’s interface for incorporating relevant records

Business Analytics & MIS A dedicated module for generating content, content provider and consumption reports. Reports generated through this module will impart insight into operational & business

planning of CMS. The reports will also provide insight into Content dispersals, consumption, copyright violations

and other business metrics.

3.3 Framework (Level 3): System Design & Components Content Homogenization Platform: Design & Components Overview 2012©TimesMobile Limited Classified Document



3.4 Proposed Deployment: Architecture Deployment




3.5 Proposed Deployment: Hardware & Software Specifications With reference to the proposed deployment architecture, the hardware & software (including operating systems) required in deployment of Content Homogenization Platform are listed here: 2012©TimesMobile Limited Classified Document



Infrastructure for Main Operational LocationS.No.

Application Servers Machine H/W M/c Quantity

Operating System + AV OS License Database Server License

1 Application Servers

Xeon Based - IBM M3 Series - Size 2U (64GB RAM) 2

Windows 2008 Enterprise+AV 2

2 Database Servers (MS-SQL) Xeon Based - IBM M3 Series - Size 2U (64GB RAM)

2

Windows 2008Enterprise+AV

2

SQL Server 2008 or Later (Enterprise Edition) X 1

Total 4 4 2Deployment Overview

The above listed count of servers & associated software are intended for single location only. Times Mobile Ltd proposes to deploy Content Homogenization at two centers – one for operations & another for redundancy. The total count of

infrastructure will be multiplied with the count of locations.

Infrastructure for Centre to Impart Redundancy S.No.

Application Servers Machine H/W M/c Quantity

Operating System + AV OS License Database Server License

1 Application Servers

Xeon Based - IBM M3 Series - Size 2U (64GB RAM) 1

Windows 2008 Enterprise+AV 1


1

Windows 2008Enterprise+AV

1


Total 2 2 1

Redundancy

System Redundancy at Application Server: By the use of multiple application servers for hosting the application, 2012©TimesMobile Limited Classified Document



sufficient redundancy is enabled. This also enables load balancing, and therefore the entire node is scalable. Redundancy at Database Level: The application deployment has a very resilient and scalable database cluster, enabled

by it redundant setup.

Application Servers

Load balanced to share network load and direct user session to less occupied server. Ensures 99.99% availability, thereby providing negligible failure in application accessibility. Fail safe session management even in case of high traffic – a resilient system.

Database Server

Database server cluster is formed by two dedicated database servers working in tandem with application server cluster The database server cluster enables to provide “Hot Backup” to the system.

3.6 Proposed Deployment: Estimated Costing for Basic Infrastructure

3.6.1 Deployment at Airtel Data-centre – Main Operational Centre (For Single Location Only):

Costing for a single location

S.No.

Application Servers Machine H/W M/c Qty

Operating System + AV

OS License

DB Server License Estimated Cost In

Lacs1 Application Servers

(WAP/USSD/WEB)Xeon Based - IBM M3 Series - Size 2U (64GB RAM) 2

Windows 2008Enterprise+AV 2

7*2 = 14


2

Windows 2008 Enterprise+AV

2


(7 x 2) + 10 = 24.0

Total 4 4 2 34Estimated Total Cost of Infrastructure for Single Location = 34 Lac




3.6.2 Deployment at Airtel Data-centre – For Redundancy

Costing for a single location

S.No.

Application Servers Machine H/W M/c Qty

Operating System + AV

OS License

DB Server License Estimated Cost In

Lacs1 Application Servers

(WAP/USSD/WEB)Xeon Based - IBM M3 Series - Size 2U (64GB RAM) 1

Windows 2008Enterprise+AV 1

7*1 = 7


1

Windows 2008 Enterprise+AV

1


(7 x 1) + 10 = 17.0

Total 4 4 2 24

Estimated Total Cost of Infrastructure for Two Locations = 34 + 24 = 58 Lac

4.0: Project Plan 2012©TimesMobile Limited Classified Document



Summary

This section intends introduce the scope of staffing & development timeframe for the Content Metadata Homogenization platform.

Project Name: Airtel Content Metadata Homogenization Platform Resource Sheet - Cost of Development – Year : 2012

S.No. Resource Title

Resource

CountMan Days

Total Man Days

Cost /(Man Day) Total Cost

Man Hour Rate

1 Project Manager 1 84 84 9,0912 System Analyst 1 84 84 4,545

3Development Software Engineer - Lead 1 45 45 8,182

4Development Software Engineer 5 45 160 3,409

5 DBA 1 45 160 7,9556 QA Manager 1 52 52 4,5457 QA - Team 3 40 120 2,9558 Integration – Lead/Team 2 52 52 6,136

Total : 757



Documents

Sample Doc