how microservices and fast data are driving mainstream adoption of

GOING REACTIVE 2016HOW MICROSERVICES AND FAST DATA ARE DRIVING MAINSTREAM ADOPTION OF REACTIVE SYSTEMS

2

INTRODUCTION ........................................................................................................................ 3

CHAPTER 1: The Survey and Respondents ........................................................................... 4

CHAPTER 2: Reactive Adoption Trends .............................................................................. 10

CHAPTER 3: Microservices and Reactive Systems ............................................................ 20

CHAPTER 4: Fast Data and Reactive Systems .................................................................... 29

CHAPTER 5: Conclusions and TL;DR .................................................................................... 36

BONUS: Tools & Technologies among all respondents ..................................................... 41

ADDITIONAL RESOURCES ..................................................................................................... 47

Table of Contents

3

INTRODUCTIONBy now, you have likely heard the famous quote from Marc Andreessen, “Software is eating the world.” Well, if you believe that is true, then the next logical question becomes, “How will your business react?”

Business leaders in nearly in every industry recognize the need for their business to react with the speed and agility of a software startup in order to survive. Blockbuster was bankrupted by Netflix. The entire hotel industry was disrupted by Airbnb. The phrase, “You’ve been Ubered,” will likely become part of our lexicon to describe industries that have been blindsided by the future.

To execute a competitive strategy that keeps businesses agile and adaptable to continuously evolving market conditions and competitors, developers and architects are helping their organizations by Going Reactive.

The Reactive Manifesto was penned in 2013 to provide business leaders and technologists with a cohesive approach and common vocabulary to accelerate the evolution toward Reactive systems. These systems are more flexible, loosely-coupled and scalable. This makes them easier to develop and amenable to change. They are significantly more tolerant of failure and when failure does occur they meet it with elegance rather than disaster.

In this survey, we set out to understand the journey of Going Reactive and uncovered some interesting adoption trends around microservices and Fast Data along the way. We enjoyed pulling this report together and hope you enjoy seeing how your peers are viewing the Reactive landscape.

http://www.reactivemanifesto.org/

CHAPTER 1: THE SURVEY AND RESPONDENTS

Meet the Developers, Architects and Managers from 28 industries

5

BEFORE JUMPING INTO THE FINDINGS, LET’S ADDRESS METHODOLOGYIn August 2015, we launched Reactive Revealed, an anonymous survey designed to understand experiences, opinions and technologies related to Going Reactive. To inspire respondents to share, we made a financial donation to support to Devoxx4Kids, an organization dedicated to getting the next generation of young geeks excited about programming. More than 3,000 people responded.

We ran the survey through an invitation to our contact list and a blind open call to O’Reilly Media newsletter recipients, readers of TheServerSide.com and Voxxed.com, so there is a certain amount of bias present in the report. Naturally, our customers and developers familiar with Typesafe and Reactive trends are more likely to reply to the call than anyone else. Also, it’s likely that people who find and take such surveys are less conservative in their technology choice than those who don’t.

That being said, the sample size, while not scientifically random, is double what’s considered statistically relevant for the total population of Java developers worldwide. With Oracle’s estimated 9 million developers working on the JVM, we calculated our margin of error using this handy tool.

http://www.devoxx4kids.org/

https://www.surveymonkey.com/mp/margin-of-error-calculator/

6

Our sample size of 3060 respondents has a 2% margin of error with a 95% confidence level and a 3% margin of error with a 99% confidence level. This means that these results can go plus or minus 2-3% in either direction, so we generally stayed away from drawing any conclusions using numbers that fell within that confidence interval. Consider this an interesting look at data culled from a very large JVM community, rather than a formal research study.

28 industries represented

62 tools & technologies included

3060 total respondents

2O questions

2% margin of error at a 95%confidence interval

7

Looking at the respondents, the majority are developers (59%). Just over one-third are architects (24%) or in management roles (10%), so we can appreciate a broader perspective in the responses. A small sliver of DevOps (3%) and Other (4%) have shared their experiences.

CURRENT JOB ROLE

MEET THE RESPONDENTS

24% Architect

10% Management/CIO/CTO

4% Other3% Dev Ops

59% Developer

8

Respondents come from a broad selection of 28 industries, with the top eight shown here. The Technology (21%) and Financial Services (13%) sectors take the first two places by a healthy margin. Online Services (8%) and Consulting (7%), Telecommunications (6%) and Business Services (6%) are all fairly close.

INDUSTRY OF OPERATION

4% Media

6% Telecommunications5% Government/Military

6% Business Services

7% Consulting8% Online Services

13% Financial Services

21% Technology

9

A slight majority (51%) of respondents work in teams of five people or less and just over one-third (36%) work in teams of six to 25 people. A combined 13% of respondents work with teams larger than 25 developers.

NUMBER OF DEVELOPERS ON PROJECT

51%

36%

7%

3%

3%

0-5

6-25

26-50

51-100

101 +

CHAPTER 2: REACTIVE ADOPTION TRENDS

Experiences and predictions about Going Reactive

11

REACTIVE ADOPTION TRENDSReactive systems embody four tenets: Message-driven, Resilient, Elastic, Responsive. We were curious to understand how quickly these systems were being adopted, the variables driving (or inhibiting) adoption, and whether or not a prescriptive approach to implementation had emerged.

James Governor Co-founder of RedMonk

The more I thought about Reactive the clearer it became that businesses, not just infrastructures, need to act in this way. Businesses need to be Reactive because you can’t predict the future and they will need new technical architectures to support the change, which looks a lot more like web computing. Agile, bursty, lean – that’s the future of business.

By means of an asynchronous, non-blocking message-driven approach, highly resilient and elastic systems can be formed, resulting in a consistently responsive user experience.

elastic resilient

message-driven

responsive

THE FOUR TENETS OF REACTIVE SYSTEMS

12

HOW CRITICAL IS GOING REACTIVE?

A significant majority of respondents (83%) feel that Reactive is a topic that requires attention, with nearly half of that group indicating Reactive is “something we needed yesterday”.

PERCEPTIONS OF THE GOING REACTIVE TREND

43%

Gaining momentum,should start looking

into it soon

12%

Enjoying a lot of buzz,but that’s probably it

40%

Totally important,something we needed

yesterday

5%

ZOMG,not Reactive again!

13

HOW QUICKLY ARE REACTIVE SYSTEMSBEING ADOPTED?We asked respondents to perform a comparative prediction of their organization compared to most successful enterprises when it comes to the adoption of Reactive systems. By 2018, respondents believe that 70% of their organizations and 80% of most successful enterprises and will have adopted Reactive systems.

Comparatively, the spread between their organizations and successful enterprises is fairly close in the “Already adopted” and “By 2016” segments. As time progresses, the comparative spread becomes more pronounced.While clearly identifying Reactive as a characteristic of successful enterprises, respondents perceive their own organizations as lagging behind. Let’s see what drivers and potential blockers they have identified.

PREDICTIONS FOR REACTIVE SYSTEMS ADOPTION

13%14%

Their organizationMost successful enterprises

Already adopted By 2016 By 2018 Later than 2018

26%

24%

41%

31%

20%

31%

14PRIMARY DRIVERS OF GOING REACTIVE

31%31%

ScalabilityScalability

22%22%

ResilienceResilience

17%17%ModernizationModernization

16%16%

VelocityVelocity

15%15%

EfficiencyEfficiency

WHAT IS DRIVING REACTIVE ADOPTION?Regarding the primary drivers of Going Reactive, no single benefit commands a majority. Scalability (31%) ranks #1, with Resilience (22%) in second place. The last three–Modernization (17%), Development Velocity (16%) and System Efficiency (15%) are all relatively close to each other. The good news is that Going Reactive in fact accomplishes all five.

15

WHAT IS BLOCKING YOU FROM GOING REACTIVE?Respondents were asked to identify all the potential blockers in Going Reactive. A happy one-fifth of respondents reported no barriers, and are actively going for it! (21%). Over half of respondents indicated that challenges on a human level are the main barriers: whether it is the need to retrain people in new technologies (35%) or the lack of interest by the team (23%).

POTENTIAL BLOCKERS OF GOING REACTIVE

About the same number of respondents reported that either the disruption of the change (27%) or lack of support from superiors at the top of the organization (26%) are perceived blockers. Respondents are less concerned about finding new staff (17%), or that Going Reactive will simply take too long to do (13%).

It will taketoo long to do

Finding new employees to workwith these systems

No barriers.We’re going for it!

Lack of interest by developersand operations staff

Not enough support at the topof the organization

Disruption to existing activitieswould be too high

Need to retrain existing staff to usenew technologies and methodologies

35%27%26%23%21%17%13%

16

WHO IS (OR SHOULD BE) RESPONSIBLE FOR TRYING NEW SYSTEM APPROACHES?Consistent with the market trend in which developers are the new “kingmakers”, a majority of respondents feel that Development teams (52%) are responsible for taking the first step in testing new technologies or architectural concepts. Architects (31%) are the second group that respondents identified as being responsible for disrupting “business as usual” and trying out new things. Respondents considered the CTO (11%) of the organization to be third, whereas the entire Executive Board (2%) is considered to be less responsible than the CTO role.

WHO SHOULD LEAD THE WAY TO GOING REACTIVE?

2ndArchitects

1stDevelopers

3rdCTO

31%

52%

11%

2ndArchitects

1stDevelopers

3rdCTO

31%

52%

11%

17

Just starting to lookinto the topic

Not applicable to me

Already learningor doing research

Building a prototype

Running a productionapplication

Developing aproduction application

4%

19%

29%14%

18%

16%

34%Power users

43%Active

starters

19%Entrylevel

WHERE ARE YOU TODAY IN THE GOING REACTIVE JOURNEY?

Of the 3060 overall respondents, 96% indicated involvement in Going Reactive at some level, with 4% saying that it’s not applicable to them. For purposes of easing trend analysis, we grouped respondents into three adoption profiles:

• The power users represent 34% of respondents, who are either developing a production application (18%) or running in production (16%).

• The active starters group is larger at 43%, comprised of respondents who are building a prototype (14%) or already learning or doing research (29%).

• The entry level group is less than a quarter of respondents, with 19% just starting to look into the topic.

GOING REACTIVE PHASE OF JOURNEY

18

WHICH APPROACH WOULD YOU TAKE TO GO REACTIVE TODAY?

Where implementing Reactive systems is concerned, multiple approaches can be considered. The most common approach by respondents is to refactor their legacy systems one module at a time (41%). From there, others would do a complete rewrite (24%), or shave down their existing stack and replace legacy bottlenecks or APIs (23%) that way. The smallest group would create a new front-end (12%) to integrate with a legacy back-end. Going module-by-module wins in popularity.

HOW TO APPROACH GOING REACTIVE

Create a greenfield front-end,use a legacy back-end

Shave the existing stackand replace legacy

bottlenecks or APIs

Completely rewrite a newproject to eventuallyreplace legacy system

Refactor legacy systemmodule by module

41%

24%

23%

12%

19

MAJOR FINDING: REACTIVE IS GOING MAINSTREAM

Based on the data, it’s fair to suggest that the concepts behind Reactive systems are entering the mainstream.

Our round-up of data and sentiment about Going Reactive indicates that more than 4 out of 5 (83%) respondents believe that Reactive systems is a topic that demands attention. When it comes to predictions for the future, respondents expect that 80% of successful enterprises will have adopted Reactive systems by 2018.

Drilling down, it’s interesting to note that 77% of respondents are Going Reactive to various degrees, with 43% already researching or prototyping and another 34% building or deploying production systems. As revealed in the next section, it’s this 34% of power users that are advancing the adoption of Microservices-based Architectures (MSA) when Going Reactive.

PREDICTIONS AND ACTIONS TAKEN TOWARDS GOING REACTIVE

Say that Reactive demands attention83%

Already researching and prototyping43%

Building and deploying production systems34%

Expect enterprise adoption by 201880%

CHAPTER 3: MICROSERVICES AND REACTIVE SYSTEMS

The embrace of Microservices and related tools in Reactive

21

MICROSERVICES AND REACTIVE SYSTEMSMicroservices has been gaining a lot of attention lately. Martin Fowler, the well-known co-author of the Agile Manifesto, has produced a popular explanation of microservices with James Lewis that has been getting a lot of traction with developers. Industry analyst Gartner is emphasizing the importance of microservices within its enterprise clientele, ranking microservice-based architecture (MSA) within its top 10 strategic technology trends for 2015.

With all of this buzz in the market, we were wondering what role microservices adoption was playing in the Reactive journey. For the purposes of this survey, we did not set out to define microservices. Rather, we simply asked respondents to self-select whether or not they considered their systems to be microservices-based. The data reveals a notable increase in MSA adoption as respondents progress along their Reactive journey from learning to deploying. Additionally, a preference for tools and technologies begins to emerge among adopters of MSA.

Gartner, Inc Top 10 Strategic Technology Trend: Microservices

Monolithic, linear application designs are giving way to a more loosely-coupled integrative approach. Microservice architecture is an emerging pattern for building distributed applications that support agile delivery and scalable deployment.

http://martinfowler.com/articles/microservices.html

http://www.gartner.com/newsroom/id/3143521

http://www.gartner.com/newsroom/id/3143521

22

Just Learning Building & DeployingResearching & Prototyping

16%

28%

50%

THE RISE OF MSA ADOPTION BY JOURNEY PHASE Out of 3060 overall users, 33% (or 1020 people) identified their systems as microservices-based. Overlaying this data with phase of journey reveals a sharp rise in MSA adoption. Among the respondents that are just beginning to learn about Reactive, 16% are using microservices, rising to 28% among those that are already doing research or building a prototype. Where MSA adoption truly becomes significant is among the power user group, where 50% are working with microservices.

MICROSERVICES ADOPTION BY PHASE OF JOURNEY

23

USING SCALA OR JAVA WITH MICROSERVICES?

Where MSA adopters are concerned, an increasing appetite for Scala (64%) and Java 8 (61%) is indicated, rising from approximately half of all respondents using these languages. Not surprisingly, usage of Java 7 or lower drops from 41% average to 30% among MSA adopters. These usage trends are consistent with the findings from our Java 8 survey, where nearly half of the respondents reported plans to jump directly to Java 8 and bypass Java 7 altogether.

USE OF JAVA AND SCALA WITH MICROSERVICES

41% Java 7 or lower

52%

Scala50%

Java 8 *

All respondents

30% Java 7 or lower

Java 8

64% Scala

61%

Microservices group:

34% Java 7 or lower

58% Java 8

76% Scala

Spark users:

27% Java7 or lower

60% Java 8

70% Scala

Power users:

41%

52%

50%

41%

52%

50%

- - - - ------ = average for all respondents

https://info.typesafe.com/COLL-2014-10-20-Java-8-II-Survey-Report-LP.html?lsd=COLL-2014-10-20-Java-8-II-Survey-Report&lst=GR2016

24

MAJOR FINDING: THE PREFERRED TOOLSET FOR MSAMicroservices-based architectures have their foundation in the core concepts of Reactive systems: asynchronous and message-driven, highly resilient against failures, elastically scalable on demand, and consistently responsive. In addition to Scala and Java 8, here are the tools and technologies that rise significantly in use among MSA adopters:

For all look at tools and technologies selected by all respondents, please go to the BONUS Chapter.

• Akka• Apache Cassandra • Apache Kafka • Apache Spark• Play Framework

• Amazon EC2• Docker• Apache Mesos

25

PREFERRED OPS/INFRA/DEVOPS TECHNOLOGIES WITH MICROSERVICES

29% Not Microservices-based

34% All respondents

47% Microservices-based architectures

Uses Docker


All respondents


43%

Uses Akka


18% All respondents


Uses Cassandra


17% All respondents


Uses Kafka


7% All respondents


Uses Mesos


22% All respondents


Uses Spark


40% All respondents


Uses Amazon EC2


30% All respondents


Uses Play

26% Spring: Microservices-based Architectures

40% Play: Microservices-based Architectures

30% Play: All respondents

29% Spring: All respondents

24% Play: Not Microservices-based

32% Spring: Not Microservices-based

Use of Spring and Play with Microservices

26


34% All respondents


Uses Docker


All respondents


43%

Uses Akka


18% All respondents


Uses Cassandra


17% All respondents


Uses Kafka


7% All respondents


Uses Mesos


22% All respondents


Uses Spark


40% All respondents


Uses Amazon EC2


30% All respondents


Uses Play








PREFERRED BIG [FAST] DATA TECHNOLOGIES WITH MICROSERVICES

27


34% All respondents


Uses Docker


All respondents


43%

Uses Akka


18% All respondents


Uses Cassandra


17% All respondents


Uses Kafka


7% All respondents


Uses Mesos


22% All respondents


Uses Spark


40% All respondents


Uses Amazon EC2


30% All respondents


Uses Play








PREFERRED DEVELOPER TECHNOLOGIES WITH MICROSERVICES

28


34% All respondents


Uses Docker


All respondents


43%

Uses Akka


18% All respondents


Uses Cassandra


17% All respondents


Uses Kafka


7% All respondents


Uses Mesos


22% All respondents


Uses Spark


40% All respondents


Uses Amazon EC2


30% All respondents


Uses Play







Use of Spring and Play with MicroservicesUSE OF SPRING AND PLAY WITH MICROSERVICES

As was mentioned earlier, a certain amount of bias is likely present in this report. Obviously, our customers and developers familiar with Typesafe are more likely to reply to the request for responses before others. Although usage is neck-and-neck when it comes to all respondents, in the MSA group a preference takes hold: Play Framework rises from 30% to 40%, whereas Spring Framework drops slightly from 29% to 26%.

CHAPTER 4: FAST DATA AND REACTIVE SYSTEMS

The embrace of data in motion and related tools in Reactive systems

30

FAST DATA AND REACTIVE SYSTEMSReactive systems for Mobile, Web and Internet of Things (IoT) increasingly operate on data in near real-time. As these systems embrace “data in motion”, traditional batch processing systems are being re-imagined as pure stream-based architectures. Specifications such as Akka Streams–among other technologies included in the Reactive Streams initiative–and stream processing technologies like Spark Streaming are emerging to provide the standards and plumbing necessary to implement such systems effectively.

In early 2015, we published the results of over 2000 respondents about Apache Spark adoption, and we wanted to check in to see what role, if any, Apache Spark adoption is playing in the Reactive journey. In this section we look at data that reveals the attitudes and appetites among Apache Spark users.

Wall Street Journal The Future of Internet is Flow

Forget the clunky Web. The future of the online world will be a river of information flowing through time.

https://info.typesafe.com/COLL-20XX-Spark-Survey-Report_LP.html?lst=WS&lsd=COLL-20XX-Spark-Survey-Trends-Adoption-Report

http://www.wsj.com/articles/the-future-of-the-internet-is-flow-1443796858

31

THE RISE OF SPARK ADOPTION IN PRODUCTION REACTIVE SYSTEMSOut of 3060 total respondents, 22% are working with Apache Spark. Overlaying this data with phase of journey reveals a sharp rise in Spark usage. A similar trend was revealed regarding Microservices-based architecture (MSA) adoption, increasing as the Reactive journey progresses from learning to deploying. For this segment of respondents, those that are just learning about Reactive represent 13%, increasing to 21% among those researching and prototyping, and climbing to 28% when building and deploying Reactive systems in production.

USAGE OF SPARK ALONG STAGE OF JOURNEY

Just Learning13%

Building & Deploying28%

Researching & Prototyping21%

32

MICROSERVICES AND SPARKNext, we were curious to see whether or not there was a correlation between Spark usage and MSA adoption. As indicated in the previous section, 30% of MSA adopters indicate using Spark. Inverting this data, however, paints a different picture: 46% of Spark users describe their system architecture as microservices-based. This correlation is interesting, and we look forward to learning more about this picture in the future. MSA using

Spark

30%Spark users

with MSA

46%

33

SCALA OR JAVA WITH SPARK?Although the average for Java 8 and Scala across all respondents is neck-and-neck, Scala rises from 50% to 76% usage among Spark users, and Java 8 rises slightly from 52% to 58%. Use of Java 7 and lower drops significantly from 41% to 30%.

With the needs of scalability paramount in Fast Data systems, it’s not surprising to see Scala as the preferred programming language for working with Spark. Not only was Spark written in Scala, but response data confirms the results of a recent report by Databricks that also show the majority of Spark developers prefer using the Scala API.

USE OF JAVA AND SCALA WITH APACHE SPARK

41% Java 7 or lower

52%

Scala50%

Java 8 *

All respondents

30% Java 7 or lower

Java 8

64% Scala

61%

Microservices group:

34% Java 7 or lower

58% Java 8

76% Scala

Spark users:

27% Java7 or lower

60% Java 8

70% Scala

Power users:

41%

52%

50%

41%

52%

50%

- - - - ------ = average for all respondents

https://www.quora.com/Why-is-Apache-Spark-implemented-in-Scala

http://cdn2.hubspot.net/hubfs/438089/Spark-Survey-2015-Infographic.png

34

43%

64%

Average Spark users

Akka Amazon EC2 Cassandra HadoopDocker

40%

55%

18%

44%

34%

52%

16%

46%

MesosKafka

17%

49%

7%

22%

MAJOR FINDING: THE SPECIFIC TOOLSET PREFERRED FOR FAST DATAAs traditional technologies that rely on offline batch processing are being replaced with technologies that support data in motion, respondents identify Apache Spark and related tools rising to the challenge. As the adoption of Fast Data architectures rises as respondents progress along the Reactive journey, the data reveals large increases in specific technologies among Spark users.

INCREASED TOOL USAGE AMONG APACHE SPARK USERSIn addition to Scala, we see a significant rise in the following tools for powering Fast Data systems using Spark:

• Akka

• Amazon EC2

• Apache Cassandra

• Docker

• Apache Hadoop

• Apache Kafka

• Apache Mesos

For all look at tools and technologies selected by all respondents, please go to the BONUS Chapter.

35

WHERE DOES HADOOP FIT INTO THE SPARK PICTURE?

We have been interested to see where Spark fits into the Hadoop ecosystem. In a recent article called Big Data Industry Predictions for 2016, the general consensus is that Spark is breathing new life into the well-established Hadoop ecosystem, with some comments even going so far as to predict that, “Spark will kill MapReduce, but save Hadoop”.

Overall Spark (22%) adoption is higher than Hadoop (16%) across all respondents. Spark is now firmly established within the big data ecosystem, and in use by 61% of Hadoop users. Pivoting, the data reveals that 46% of Spark users are also employing Hadoop; however, with over half of Spark users not using Hadoop, clearly other platform approaches are arising as alternatives.

Spark usersthat use Hadoop

46%Hadoop usersthat use Spark

61%

SPARK AND HADOOP TOGETHER

http://insidebigdata.com/2015/12/08/big-data-industry-predictions-2016/

http://insidebigdata.com/2015/12/08/big-data-industry-predictions-2016/

CHAPTER 5: CONCLUSIONS AND TL;DR

Three conclusions nicely wrappedup for busy readers

37

CONCLUSIONS (TL;DR)For those of you too busy to internalize everything in this report at once, we can narrow down the final points into three main conclusions:

Reactive system adoption is going mainstream

Reactive adoption is being driven by two key technology trends: Microservices and Fast Data

Microservices and Fast Data users are rallying around a preferred group of tools and technologies

1

2

3

38

Reactive system adoption is going mainstream

We acknowledge the responses in this report are likely biased. Nevertheless, based on the data from more than 3000 respondents, it’s fair to suggest that Reactive systems are entering the mainstream. An overwhelming majority of respondents (83%) report that Going Reactive is critical, with nearly half of the group indicating Reactive is “something we needed yesterday.” Respondents predict that 70% of their organizations and 80% of most successful enterprises will have adopted Reactive systems by 2018, citing improvements in scalability (31%) and resilience (22%) as the top drivers. What we found really amazing was the current rate of adoption being reported. A total of 77% of respondents indicate they are already Going Reactive to some degree with 43% researching and prototyping Reactive systems and 34% building and deploying systems to production.

39

Reactive adoption is being driven by two key technology trends: microservices and Fast Data

With all of the buzz in the market around microservices and Apache Spark, we were wondering what roles these technology trends were playing in the Reactive journey. For the purposes of this survey, we did not set out to define microservices. Rather, we simply asked respondents to self-select whether or not they considered their systems to be microservices-based. The data revealed a notable increase in microservices and Spark usage as respondents progressed along their Reactive journey from just beginning to learn about Reactive (16% microservices, 13% Spark), rising among those that are already doing research or building a prototype (28% microservices, 21% Spark), and climbing even higher within the Reactive power users that are building and deploying production systems (50% microservices, 28% Spark).

40

Microservices and Fast Data users are rallying around a preferred group of tools and technologies

Another key area we set out to understand was whether or not a particular set of tools and technologies were being embraced over others for building Reactive systems. We cast the lens through the data of microservices and Spark users and found a tremendous amount of overlap in their preferences. To these respondents, this is a prescription for Going Reactive:

43% 59% 64%

40% 52% 55%

18% 28% 44%

34% 47% 52%

17% 27% 49%

7% 13% 22%

30% 40% 38%

22% 30% -

SPARKUSERS

ALLRESPONDENTS

MICROSERVICES USERS

BONUS: TOOLS & TECHNOLOGIES AMONG ALL RESPONDENTS

Development, Operations and Big [Fast] Data tools in use

42

TOOLS & TECHNOLOGIESSeeing which tools are being used is always interesting. We included 62 tools and technologies to choose from, and though some may think this isn’t nearly enough, we wanted to get a reasonable group of tools used in Development, Infrastructure/DevOps, and Big [Fast] Data systems.

43

USAGE OF JAVA AND SCALA

IT’S NOW JAVA 8 AND SCALA, NOT OR...The combined forces of Java 8 and Scala, more than either language separately, represent a larger proportion of either language group. It’s great to see that Java and Scala users have found that both languages have their use for these types of systems.

With Java 8’s features borrowed from the world of functional programming, such as Lambdas, Futures and Streams, building Reactive systems has become more straightforward in the dominant JVM language.

With JDK 9 coming...soon, we also can expect the mainstream introduction of Reactive Streams for the JVM, an industry initiative working to standardize back-pressure mechanisms for streaming data and led by engineers from Netflix, Oracle, Pivotal, Red Hat, Typesafe and others. Rather than just Scala or Java alone, we hope to see the entirety of the JVM improve incrementally over time, taking the best approaches for the job regardless of semantics and nuances revered by the most outspoken fans of either language.

29%

50%

Uses Java 8 and Java 7 or lower together

Uses Scala

52% Uses Java 8

56% Uses Java 8 and Scala together

http://www.reactive-streams.org/

http://www.reactive-streams.org/

44

29% Spring Framework

24% Hibernate

18% Gradle

12% Scala IDE

24% Other

61% IntelliJ IDEA

52% Java 8

50% Scala

43% Akka

42% Maven

41% Java 7 or lower

39% sbt

34% Eclipse

30% Play Framework

8%

7%

Ant + Ivy

RxJava

5%

3%

2%

NetBeans IDE

None of the above

Vert.x

DEVELOPMENT TECHNOLOGIES IN USE

DEVELOPMENT TECHNOLOGIES IN USE

IntelliJ IDEA (61%) emerges as the most commonly used development tool among respondents, even overtaking programming languages in adoption. Eclipse IDE (34%) takes a distant second, with Scala IDE (12%) and NetBeans IDE (5%) rounding out the section.

Java 8 (52%) has slightly larger adoption than Scala (50%), and Java 7 or lower (41%) takes third place among programming languages. Incidentally, over 50% of both Java 8 and Scala users also employ the other language as well. Use of Java 7 or lower among Java 8 and Scala users drops below the average to less than 32% for either one.

Among build tools, Maven (42% overall, but 61% of Java 7 or lower users) and sbt (39% overall, but 73% of Scala users) are potentially competing for the first time in overall adoption, with Gradle (18%) in a respectable third position and Ant+Ivy (8%) maintaining a sliver of market share.

Akka (43%) is the 4th most-used tool among respondents. When looking at where Akka rises most, it’s across a pretty distributed group of respondents. Akka usage increases considerably to 73% for Scala users, 64% for Apache Spark users and 59% among Microservices-based architectures.

Between frameworks (whose choices were limited to just two for the sake of brevity), Play Framework (30%) and Spring Framework (29%) are nearly neck and neck.

45

INFRA/OPS/DEVLOPS TECHNOLOGIES IN USEAmong infrastructure and operations technologies, we can see Jenkins CI (49%) and Amazon EC2 (40%) maintaining strong footholds.

Docker (34%) has taken a strong third place in overall use, and Vagrant (19%) is in use by nearly one-fifth of respondents.

New Relic (13%) shows its dominance in the monitoring space compared to other players like Dynatrace (1%) and Takipi (1%).

Among provisioning & configuration tools, we see a close race between Puppet (14%), Ansible (13%) and Chef (11%). Heroku (8%), Apache Mesos (7%) and OpenStack (6%) have relatively small shares.

Jenkins CI

49%

Amazon W

eb Services

(i.e. E

C2)

40%Docker

34%

Vagrant

19%Puppet

14%New Relic

13%Ansib

le

13% Chef

11% Heroku

8%None of th

e Above

7%Apache M

esos

7%OpenStack

6% Consul

3%

Other

16%

46

32% MySQL

28% PostgreSQL

28% MongoDB

22% OracleDB

22% Apache Spark

19% Redis

18% Apache Cassandra

17% Apache Kafka

16% Apache Hadoop

12% MS SQL Server

10% Slick

7% Other

7% Hazelcast

5% YARN

26% Other

4%

4%

4%

Apache Storm

Cloudera CDH

IBM DB2

4%

4%

2%

None of the above

Heroku

Membase

2%

1%

1%

HortonWorks HDP

Apache Samza

Riak

DATA FOCUSED TECHNOLOGIES IN USE

BIG [FAST] DATA TECHNOLOGIES IN USE

When it come to Big [Fast] Data and Data Storage technologies, we see a line up of the usual suspects in the DB world: MySQL (32%), PostgreSQL (28%), MongoDB (28%) and Oracle DB (22%) comprise the top four places. With a prevalence of Java 8 and Scala users represented in these results, it’s not surprising to see MS SQL Server (12%) at a smaller uptake. Apache Spark (22%) is the 5th most used data technology, followed by Redis (19%), Apache Cassandra (18%), Apache Kafka (17%) and Apache Hadoop (16%).

47

ADDITIONAL RESOURCES

Typesafe (Twitter: @Typesafe) is dedicated to helping developers build Reactive applications on the JVM. Backed by Greylock Partners, Shasta Ventures, Bain Capital Ventures and Juniper Networks, Typesafe is headquartered in San Francisco with offices in Atlanta, Switzerland and Sweden. To start building Reactive applications today, learn about Reactive Platform.

© 2015 Typesafe

Reactive Streams, Akka Streams and Akka HTTP

DOWNLOAD

Microservices inProduction

DOWNLOAD

Fast Data: Big Data Evolved

DOWNLOAD

http://www.reactivemanifesto.org/

http://www.typesafe.com/products/typesafe-reactive-platform

https://info.typesafe.com/COLL-20XX-Enterprise-Architect-Akka-Streaming-Guide_LP.html?lst=WS&lsd=COLL-20XX-Enterprise-Architect-Akka-Streaming-Guide&_ga=1.34067814.615845212.1447436070

https://info.typesafe.com/COLL-20XX-Microservices-in-Production-WP_LP.html?lst=WS&lsd=COLL-20XX-Microservices-in-Production-WP&_ga=1.34067814.615845212.1447436070

https://info.typesafe.com/COLL-20XX-Fast-Data-Big-Data-Evolved-WP_LP.html?lst=WS&lsd=COLL-20XX-Fast-Data-Big-Data-Evolved-WP&_ga=1.34067814.615845212.1447436070

Documents

how microservices and fast data are driving mainstream adoption of