Open Source Software and Open Data by Dr. Waqar Mahmood

25 novembre 2017 | 9 AM - 5 PM

Cap-Haitien, Haiti.

Open Source Software and Open DataThe way to move ahead

Dr. Waqar [email protected]

Al-Khawarizmi Institute of Computer Science UET, Lahore

mailto:[email protected]


“Share this code with your fellow users. Learn from it. Improve upon it. And when you're done, please give something back to the community”

- Richard Stallman Kidd, father of free software, GNU Initiator

• The open-source movement was started to spread the idea of open-source software. Programmers who support the open-sourcemovement philosophy contribute to the open-source communityvoluntarily.

• The goal is to promote the production of "high quality programs" aswell as "working cooperatively with other similarly minded people" toimprove technology.

Open Source Initiative


History of Open Source Software (OSS)• 1960's Bundled software with IBM • 1976 The original EMACS was a set of Editor MACroS for the TECO editor written by Richard Stallman • 1980’s Stallman founded the GNU project to write a complete open source operating system. • 1984 X originated at MIT. The X Org foundation leads. • 1987 Perl, the dynamic programming language was created and released by Larry Wall • 1980's – 1990's early development continues • 1990's late Acceptance of open source products • 1991-1992 final version Linux kernel; creation GNU/Linux

• 1995- Release of MySql• 1997 GNOME project leaders Miguel de lcaza and Federico Mena• 1998 Liberation of Netscape; Mozilla web browser officially launched• 1998 term “Open Source Software” replaces “Open Software”.• 2003 development of open source CMS such as WordPress

• 2008 Google released Chrome, an open source browser• 2013 Open source libraries for Deep Learning

• 2016 Microsoft joins Linux foundation


Open Source Adoption Trends


25 novembre 2017 | 9 AM - 5 PM

Cap-Haitien, Haiti.

Market Trends


Developing a Commercially Viable Product with OSS

Identify Business Need

Research & Customization

Commercialization

“Without open source, Facebook, Google, Amazon and nearly every other modern technology company would not exist” (Andreessen-

Horowitz – leading OSS firm, 2016)


Global Financial Benefits of OSS

• Savings of over $60 billion per year

• The Android OS, which runs on over 80 % of all smartphones in the world, is based on the open source operating system Linux.

• As of 2016, open source software served 75 % of the top 10,000 websites on the Internet.

• Open source has been embraced by for-profit businesses

– 78 % of companies use open source software substantially, only 3 % don’t use open source software


• IBM – Apache SystemML

– Robotic calculations and inference agent– Correlate documents– Detect change points in IoT sensor data– …

• Google– Chrome– TensorFlow– Google Maps– …

• Facebook• Infer (Identify bugs in code)• Redex (Byte code optimizer for android)

Open Source Empower Corporations ?


Microsoft Adapters for OS

https://opensource.microsoft.com/

• Open Source Growth has compelled Microsoft to develop adapters and repositories in the form of applications to connect Microsoft applications with Open Source Systems

• Microsoft would not survive if it did not take this step.


Benefits of Open Source

● FREE, Secure, Scalable

● Customizable (Software and OS)

● Simple and lightweight software

● More learning opportunity for students

● Opportunities for researchers, entrepreneurs and industry

Al-Khawarizmi Institute of Computer Science UET, LahoreAl-Khawarizmi Institute of Computer

Science UET, Lahore

A Brief Comparison

Open-Source

● Free

● Free source available

● Better performance

● Secure

● Difficult installation & configuration

● Simple & Rigid

● Configure from scratch

● Google for support

● Poor documentation or missing

Propriety

Licensed

Source not available

Relatively slow

Prone to threats

Relatively easy installation & configuration

Appealing & Attractive

Pre-configuration

Support by vendor

Properly documented



KICS Contributions Towards OSS

• Research Labs in house developments in OSS

• Releases in Open Source

• Short Trainings

• ICOSST

• Open Data Contributions

– CVML, CLE, AERIL


KICS Open Source Contributions

High Performance and Computer Lab

• Characterization, Evaluation, and Development of High Performance Network Services on Multi-Core Architectures.

• Multicore Processor Architecture and Communication (MPAC) library release to open source

• Urdu Search Engine (USE) development on open tools• Runtime Parallelization of Sequential Binary Code through JVM• Development of Type-II Hypervisor for MIPS64 Based Systems• MIPS64 - System Mode Emulation in QEMU• Analysis of Virtual Machine Scheduling


KICS Open Source ContributionsCenter for Language Engineering (CLE)

Localization • OSS Localization

– Windows Installer for Urdu NVu (Open Source Web Authoring System)

– Urdu Localization Terminology (For localization of OSS)

• Keyboards– CLE Urdu Mobile Keyboard

– CLE Urdu Keyboard for Android Based Devices

– CRULP Urdu Phonetic Keyboards

– CRULP BTK Phonetic Keyboards

Language Processing– Word Level Processing (word sense tagging, segmentation, spell checker ..)

– Phrase Level Processing (English to Urdu machine translator, Urdu Text to Speech system, Screen Reader etc..)


Open Source System : In-House Development

University Management System (UMS)• Using an open source framework Odoo, UMS Research Team

has developed a secure, robust, reliable, and professional University Management System, UMS

• This System covers OBE (Outcome Based Education) embraced by PEC (Pakistan Engineering Council) in 2014

• A first in the country in-house developed OBE based UMS on open tools

KICS ERP System• Odoo is also serving as KICS ERP system to manage our

resources like, accounting, purchases, stores, HR and salary.



Open Source Content Management System• UET & KICS Websites (+70 sub-domains using OSS)• UET & KICS Profile Management System• Conferences and Workshops websites (+30 Websites using OSS)

All Above portals are based on Open Source Systems using following tools;– OpenCMS– CodeIgniter– Wordpress– Moodle – Joomla– Laravel– Drupal – Yii etc



Library Management System using open sources Integrated Library System (ILS)

• UET Library Management System is developed in JavaEE using open-source tools KOHA and is deployed in complete open-source environment to provide a zero cost solution to public sector libraries

• Koha is a fully featured, scalable library management system. Development is sponsored by libraries of varying types and sizes, volunteers, and support companies worldwide.– An automated, secure and cost effective secure library management

system based on RFID (Radio Frequency Identification) technology.

• UET LMS integrates data for the books and journals in Urdu as well



Other Applications developed using OSS– IPAS: Invoice processing and Automation System for Power Industry

– Wi-CAM: Auto Configuration Server for Wi-Tribe internet service provider

– uHands: Emergency Management System

– mKisan is An Interactive Advisory System for Farmers

– OMR-Based Solutions

– RFID-Based Vehicle Access Control and Parking Solution

– 4G TRx developed on Open Source GNU Radio Plat form

– 4G TRx also developed for Virginia Tech Open Radio Platform


ICOSSTInternational Conference on Open Source Systems & Technologies for 11 years

Wi-CAM: Auto Configuration ServeruHands: Emergency Management SystemmKisan is An Interactive Advisory System

for Farmers


Trainees trained on OSS

367

315

417

310

586603

NUMBER OF TRAINEES TARINED ON OPEN SOURCE TOOLS & TECHNOLOGIES

2012 2013 2014 2015 2016 2017


1. Business Idea Competition

2. Xtreme Programming

3. Web Design Competition

4. Workshop – Bootstrap

5. Circuit Mania

21

BRAINIAC : IEEE Student Activities


Case Study

● KICS employees triggered a change from Proprietary to Open Source Systems in 2015.

● Problem Statement: Shift from

Windows(OS) —› Ubuntu

Licensed Softwares —› Open Source Applications


Office Applications

● Libre Office– Writer, Impress, Calc, Math, Draw, Base, CAD

● Tex Editor– TexMaker, lyx Document Processor

● PDF Reader– Acrobat Reader, Document Viewer(builtin)

● Project – Project Libre, Open Proj, Gantt Project

Al-Khawarizmi Institute of Computer Science UET, LahoreAl-Khawarizmi Institute of Computer

Science UET, Lahore

Windows Software Alternative Software

MS Office 2010 Libre Office

MS Project Project Libre, Open Project

MS Visio Libre Draw

MS Outlook Thunder Bird

Dreamweaver Aptana

Photoshop GimpShop

Corel Draw Inkscape, Xara

AutoCAD FreeCAD, Libre CAD

Macromedia Flash F4L (development phase)

Internet Explorer N/A

Safari N/A

Notepad++ Emac, gedit, geany

WinSCP FileZila

Active Directory LDAP & Samba server

Crystal Report iReport

MS SQL MySQL, MariaDB

Tortoise SVN Kde SVN

Drivers Most of drivers are Built-in

KICS Attendance Management System N/A


Open Data

“Numerous scientists have pointed out the irony that right at the historical moment when we have the technologies to permit worldwide availability and

distributed process of scientific data, broadening collaboration and accelerating the pace and depth of discovery ... we are busy locking up that data and preventing the use of correspondingly advanced technologies on

knowledge.”

- John Wilbanks, VP Science, Creative Common


Open Data• Open Data is:

– Freely available on the internet;

– Without financial, legal, or technical barriers

– Include both structured and unstructured formats


Open Data in Science

• Accelerates the process of research and discovery by reuse and validate previous results.

– Human Genome Project (HGM)• All human genomic sequence information should be freely available and in the

public domain in order to encourage research and development and to maximize its benefit to society

– Pubmed• Abstracts of Biomedical scientific documents used for research in text mining

– Youtube DatasetData set comprising of youtube video segments focusing on research for face recognition, human activities identification, visual scene interpretation.

– CodaLab Platform• Microsoft initiative. Run reproducible machine learning experiments for data driven

research. Include all video, textual and audio data.

– Telemetry Data, Weather Data, Demographic Data, Geographic Data, Agriculture Data, Economic Data, Space exploration Data, etc..


Open Data in Governance

• Facilitates government transparency, accountability and public participation

– Punjab School Monitoring• Real-time reporting with pictorial evidence, geo-tagging of sites

visited, and automatic SMS alerts.

– Punjab Child Vaccination• Free, Open and real time data about vaccination of Punjab.

– Judicial data

– Police Station crime and criminals related data

– Land Record data

– School Solarization Project with on-line monitoring and data availability


Our Contribution to Scientific Open Datasets

• Computer Vision and Machine Learning (CVML)

• Center for Language Engineering (CLE)

• Bioinformatics Research Lab

• Alternate Energy Research and Innovation Lab


Computer Vision and Machine Learning Lab (CVML)

• Data specific to the subcontinental conditions, i.e. Skin tone of people, clothing etc.

• Datasets for Face, Emotion and Gender classification.

• To find interaction between multiple/groups of human, Multi human activity dataset.

• Objects dataset which consists of objects from surveillance videos.


Facial Datasets | CVML KICS

Face recognition dataset (varying camera angles, with & without glasses, beard etc.):

– Subjects : 4000 males, 600 females

• Clothes dataset:

– Total subjects: 200 males, 40 females

– Types of clothes: shalwar qameez, pants and dress shirt, wasket

• Role of Safe City Project?


Center for Language Engineering (CLE)

Linguistic Resources

• Text Corpora (15+ Corpus )

• Image Corpora (45+ Corpus)

• Speech Corpora (10+ Corpus)

• Lexical Resources

– Urdu Wordnet

– Urdu Books N-Grams


Bioinformatics Research Lab

• Brain Data(EEG): For 30 subjects

Set of 30 emotions + actions such

– Move left, right, straight, backward.

– Eye Blink, Eye movement

– Hand movements

– Head based movements etc.

• Local Biological Databank

– Local data additions in ChemSpider and PubChem


Alternate Energy Research Lab

• Weather Station Data

– Parameters:

• Humidity, Air Temperature, Pressure, Wind Speed, Sun light (direct/indirect)

– Potential Benefits:

• Forecast energy generation

• Scheduled maintenance


Way to move ahead!

• Adaption of OSS by Government and Private Sector.

• Release of local data for community specific problems solving.

• Students involvement in international Open Source initiatives

• Industry academia linkage for data sharing

Data & Analytics

Open Source Software and Open Data by Dr. Waqar Mahmood