OSS Metrics for Market Readiness
The OW2 OSCAR Framework
Cédric Thomas, OW2
Paris Open Source Sumit
Paris 16-17 November, 2016
Nov 16, 2016 22016, Cedric Thomas
Agenda OSS Projects and the Value Chain
Market Readiness Observations
Evaluating Readiness and Maturity
Evaluating Open Source Maturity
OW2 OSCAR Approach
Nov 16, 2016 32016, Cedric Thomas
Project categoriesCode to productSupporting market readiness
OSS projects and the value chain
Nov 16, 2016 42016, Cedric Thomas
Community projects
Nov 16, 2016 52016, Cedric Thomas
Enterprise projects
Nov 16, 2016 62016, Cedric Thomas
Collaborative projects
Nov 16, 2016 72016, Cedric Thomas
Software is Code
Nov 16, 2016 82016, Cedric Thomas
What is a Software Product?
Developer Customer
* When you want to sell it or do business with it, then it becomes a product
Nov 16, 2016 92016, Cedric Thomas
What makes a Software Product?
Developer Customer
Documentation
Upgrades
Roadmap Training Etc.
Pricing Contracts Support Expertise
Packaging
* It's not just code anymore, it's the whole value proposition
102016, Cedric Thomas
Research & Development
Co
deP
OC
sU
se-c
ases
De
mon
stra
tors
Do
cum
enta
tion
Ro
adm
apU
pgr
ades
Bug
-fix
ing
Tra
inin
gS
uppo
rtP
acka
ging
Ca
se s
tudi
esC
olla
tera
lP
ricin
gC
ont
ract
sE
arly
ado
pter
sE
tc.
Pre
dict
abili
tyQ
ualit
yT
rust
Without the code, the rest does not exist,but it's the rest that gives market value to the code
DeliveryChallenge
What creates value?
Market Value
112016, Cedric Thomas
Research & Development
Co
deP
OC
sU
se-c
ases
De
mon
stra
tors
Do
cum
enta
tion
Ro
adm
apU
pgr
ades
Bug
-fix
ing
Tra
inin
gS
uppo
rtP
acka
ging
Ca
se s
tudi
esC
olla
tera
lP
ricin
gC
ont
ract
sE
arly
ado
pter
sE
tc.
Pre
dict
abili
tyQ
ualit
yT
rust
OW2 is an ecosystem platform that helps create value with open source projects
DeliveryChallenge
Market Value
Who creates value?The ecosystem
ContributorsDistrib. Vendors
Open Source Orgs.
Fiduciary Services Users
Systems Integrators
122016, Cedric Thomas
Research & Development
Co
deP
OC
sU
se-c
ases
De
mon
stra
tors
Do
cum
enta
tion
Ro
adm
apU
pgr
ades
Bug
-fix
ing
Tra
inin
gS
uppo
rtP
acka
ging
Ca
se s
tudi
esC
olla
tera
lP
ricin
gC
ont
ract
sE
arly
ado
pter
sE
tc.
Pre
dict
abili
tyQ
ualit
yT
rust
OW2 is an ecosystem platform that helps create value with open source projects
DeliveryChallengeCollaborative Development Technical Resources
Governance, Projects, Initiatives, Quality Program
Communication, Outreach, Marketplace
OSCAR
Market Value
Supporting market readiness and value creation
132016, Cedric Thomas
Code in the value chainEcosystems deliveryOpen source governanceIT industry support
Market Readiness Observations
142016, Cedric Thomas
Code is only a fraction of the software value chain
It's the whole value chain that creates market-ready offerings.
Users want a full business proposal, not just bare code.
Decision-makers expect market-ready offerings.
i.e. code complemented by: packaging, services, training, maintenance, support, etc.
https://commons.wikimedia.org/wiki/File:Ford_assembly_line_-_1913.jpg
152016, Cedric Thomas
Collaborative development does not deliver market-ready offerings.
Ecosystems are expected to deliver agreed-upon technologies, roadmaps, reference implementations, POCs and components.
Open source developers natural bias is to concentrate on core code functionalities.
Code is the soul of free and open source projects.
https://en.wikipedia.org/wiki/Eiffel_Tower
162016, Cedric Thomas
Successful collaborative projects implement flawless open source governance.
Open source governance best practices help build sustainable communities.
Code complementers more likely to contribute to trustworthy OSS projects.
Non-Profit open source organizations provide neutral support and sustainability.
https://pixabay.com/en/hammer-court-judge-justice-law-1707729/
172016, Cedric Thomas
Successful open source projects are supported by IT companies.
Corporate support ensures roadmap consistency and long-term sustainability.
Corporate support develops industry-grade distributions and market-ready offerings.
Corporate support helps grow market outreach, sign-up early adopters and provide use cases for mainstream market.
182016, Cedric Thomas
Technology Readiness LevelMarket readinessOpen source readiness
Evaluating Readiness and Maturity
192016, Cedric Thomas
Origin: Technology Readiness Level
A type of measurement system used to estimate the maturity level of a particular technology
In technology, there are usually nine readiness levels. TRL 1 is the lowest and TRL 9 is the highest.
A TRL number is obtained once the description has been achieved.
For example, successfully achieving TRL 4 does not move the technology to TRL 5.
Pioneered by NASA in the 80’s.
Adopted by the DOE and DOD for procurement and management of complex systems.
Idea(useless)
Operational(useful)
202016, Cedric Thomas
NASA/DOD TRL
http
://w
ww
.fra
nki
cham
aki
.co
m/w
p-c
ont
ent/
uplo
ad
s/2
014
/01/
nasa
-trl.
jpg
212016, Cedric Thomas http
://w
ww
.ndi
a.o
rg/D
ivis
ions
/Div
isio
ns/S
cie
nce
An
dE
ngin
ee
ring
Tech
nol
ogy
/Do
cum
en
ts/C
oyl
e%
20
ND
IA.p
df
222016, Cedric Thomas
http
s://s
teve
bla
nk.
files
.wo
rdp
ress
.co
m/2
01
3/11
/irl.j
pg
Investment Readiness Level
232016, Cedric Thomas
QualiPSoOW2 SQuATCII Badge Program
Evaluating Open Source Maturity
242016, Cedric Thomas
Measuring Open Source Data collection and dashboards
OpenHub on GitHub, RISCOSS Analyser on GitHub
Bitergia
License and IP analysis:
Black Duck Software, Palamida, DejaCode, TripleCheck
Analysis models
NASA Reuse Readiness Levels
Core Infrastructure Initiative Badge program
OSS Watch Software Sustainability Maturity Model
252016, Cedric Thomas
2007: QualiPSo European project
48 months (2007-2010)
22 organisations from 9 countries (3 continents)
It is all about TRUST
Trust cannot be claimed without being proved!!!
QualiPSo aimed at standardising the way OSS systems are built, offered and consumed.
262016, Cedric Thomas
272016, Cedric Thomas
2010: OW2 SQuATSoftware Quality Assurance and Trustworthiness
IP verification: FOSSology
Applied on all OW2 mature projects
Code verification: Antelink
Provides traceability of external libraries
Static analysis: Sonar
Set of OW2 Sonar rules
Code quality: Trustie
TSRR installation on OW2
Maturity analysis: Qualipso
OMM applied to OW2 projects
282016, Cedric Thomas
PDOC STD QTP LCS ENV DFCT MST CM PP REQM RDMP STK
0
1
2
3
4
3.22
3.75
3.43
43.83
3.71 3.754
3
4
3
3.89
OMM Basic level
Trustworthy elements assessment
Trustworthy elements
Assessed
value
PDOC STD QTP LCS ENV DFCT MST CM PP REQM RDMP STK
0
1
2
3
4
5
6
7
8
9
10
56%
75% 43%100%
83%86%
75%
100%
0%
100%33%
100%
22%
25%
57%
0%
17%0%
25%
0%
100% 0%33%
11%
0%
0%
0%
0%14%
0%
0%
0%
0%33%
0%11%
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
OMM Basic levelPractices assessment value
4 3 2 1
Trustworthy elements
Assessment
value of practices
OW2 Implementation of the QualiPSo OMM
292016, Cedric Thomas
2012: RISCOSS
CommercialProducts/Services
???
Antepedia
Business Users Integrators
Open source as a public resource freely accessible
But OSS come from very different backgrounds
Exploring and mapping the open source landscape
Need to identify, measure, evaluate existing software
Many tools and online services available
302016, Cedric Thomas
2015: CII Badge Program(Linux Foundation)
Core Infrastructure Initiative (CII)
Launched after the Heartbleed failure
Organized by The Linux Foundation
Supported by Amazon Web Services, Adobe, Bloomberg, Cisco, Dell, Facebook, Fujitsu, Google, Hitachi, HP, Huawei, IBM, Intel, Microsoft, NetApp, NEC, Qualcomm, RackSpace, salesforce.com, and VMware
https://www.coreinfrastructure.org/
David A. Wheeler at OW2con'16
312016, Cedric Thomas
CII BADGE PROGRAM Checklist:
Basics Project website
Project website content
FLOSS License
Documentation
Other
Change control Public version-controlled source repository
Version numbering
Release notes (ChangeLog)
Reporting Bug reporting process
Vulnerability reporting process
Quality Working build system
Automated test suite
New functionality testing
Warning flags
Security Secure development knowledge
Good cryptographic practices
Secured delivery mechanism
Publicly-known vulnerabilities fixed
Analysis Static code analysis
Dynamic analysis
322016, Cedric Thomas
More than just TRLMarket readinessPromotes best practices
OW2 OSCAR Approach
332016, Cedric Thomas
OSCAROpen Source Capability Assessment Radar
Based on SQuAT
Increase the Quality and Trustworthiness of OW2 projects
To facilitate decision making and adoption of OW2 projects
Through:
Quality assessment tools
A check-list of best practices to reach market maturity
OSCAR is SQuAT second generation
342016, Cedric Thomas
OSCAR An assessment method and a platform
Requirements
Metrics
Visual Reporting
Risk analysis
OM
M F
orm
Metrics / Scorecards
Documentation
Privacy / GDPR
Standards
Licenses and IP
Fossology
SonarQube
Static code analysis
Code / Commits / Bugs
Testing / CI / Release
Cloud Deployment
OM
M F
orm
Governance Engineering
More to come:- Accessibility- Deployability- Marketing- Funding
Nov 16, 2016 352016, Cedric Thomas
OMM Assessment Web Form
Nov 16, 2016 362016, Cedric Thomas
OMMAssessment
+ CII input
Nov 16, 2016 372016, Cedric Thomas
FOSSologyLicense analysis
Nov 16, 2016 382016, Cedric Thomas
SonarQubeStatic code analysis
Nov 16, 2016 392016, Cedric Thomas
Activeness Risk Drivers
Nov 16, 2016 402016, Cedric Thomas
Risk Models
Nov 16, 2016 412016, Cedric Thomas
OSCAR Market Readiness Scorecard
422016, Cedric Thomas
Summary
2016, Cedric Thomas
Summary Open source software come in different value chains
Software value chain and the open source delivery challenge
Readiness scaling helps make decision
Experience in open source market readiness analysis
OSCAR, the OW2 OSS market readiness assessment approach
2016, Cedric Thomas
Open source is a vehicle for collaborative innovation
Software value chain and the open source delivery challenge
Readiness scaling helps make decision
Experience in open source market readiness analysis
OSCAR, the OW2 open source market readiness approach
45
www.ow2.orgFor more details please contact Cedric Thomas, OW2 CEO, [email protected]
And now let's talkQ&ADisagreementsComplementsFeedbacketc.
Thank You