Building Eclipse projects on the new CBI build farm Generation... · More slaves Dedicated master...

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

Next Generation CI

Building Eclipse projects on the new CBI build farm

Frederic GurrEclipse Foundation

EclipseCon Europe - October 25, 2018

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

History & Status quo

2

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

History

3

2011-2012

More slaves

Dedicated master for performance tests

2013

Hudson instance per project

Part of the CBI initiative

Shared instance with slaves for Windows and MacOS

2009

Single Hudson instance

Maintained by the community

2010

1 Master, 2 Slaves

Maintained by Eclipse IT team.

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

History

4

2017

Mass migration to Jenkins

Migration tool https://github.com/eclipse-cbi/hipp2jipp

Custom scripts

2016

Hudson ecosystem not catching up

Started to evaluate Jenkins

2018

Jenkins instance per project

No more Hudson

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

Today’s infrastructure

5

• 220+ Jenkins instances

• Beefy machines(total ~250 cores, ~1.5TB RAM)

• A few dedicated slaves(e.g. for Windows and Mac)

https://ci.eclipse.org

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

Today’s infrastructure

6

• Managing 10s of Jenkins is “easy”, managing 100s of them is time consuming

• We want to spend our time on areas that add value: • Harden digital signature and packaging services• Ease publication/promotion (Maven central, Nexus, ...)• Better support of static analysis tools (SonarQube, …)• Better integration with GitHub

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

Today’s infrastructure

7

• We also want to offer better performance• Pool our hardware to use it more efficiently• QoS - better isolation and control over resources usage• Be able to add additional resources more easily

• Either cloud-based or on-premise

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

Next generation CI

8

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

A duet solution

9

( )

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

CloudBees Core

10

Operation Center

JIPPS

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

CloudBees Core

11

Operation Center

JIPPS

Kubernetes/OpenShift Cluster

Jobs run on temporary, dynamically provisioned agents

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

CloudBees Core

12

Operation Center

JIPPS

Kubernetes/OpenShift Cluster

Jobs run on temporary, dynamically provisioned agents

Dedicated agents

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

CloudBees Core

13

Operation Center

JIPPS

Kubernetes/OpenShift Cluster

Jobs run on temporary, dynamically provisioned agents

Dedicated agents

Shared agents

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

Next generation status quo

14

Already running on new infra:• Every new provisioned JIPP

• Early adaptors like Xtext

• ~40 Jakarta EE projects

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

CBC and cluster problems

15

• We had some stability issues in the beginning

• Responsive and excellent support by CloudBees

• Cluster is now much more stable

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

Migrating OSS Jenkins instances to CBC

16

• How?• No built-in solution for mass migration of jobs• => Custom solution

• What needs to be moved?• Job configurations• Views• Credentials• Maven settings

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

Roadmap

17

June2018

Setup

OpenShift/CJE running on Foundation infrastructure

August 2018

Early adopters

New JIPPs are created on CJE and a couple of volunteer projects are migrated

Soon

Migration

All JIPPs are migrated to CJE

October2018

JakartaEE

All JakartaEE projects run on CJE

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

18

FAQ & Best practices

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

FAQ

19

• Why does my build take so long and how can I make it faster?

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

FAQ

20

• Why does my build take so long and how can I make it faster?

• What are Jenkins pipelines and why/when should they be used?

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

Jenkins Pipelines

21

• Store your job config in your repo

• Config linked to code

• Config history

• Scripted Pipeline vs Declarative Pipeline

• Recommended for docker based builds

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

FAQ

22

• Why does my build take so long and how can I make it faster?

• What are Jenkins pipelines and why/when should they be used?

• Can I finally run docker based builds?

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

FAQ

23

• Why does my build take so long and how can I make it faster?

• What are Jenkins pipelines and why/when should they be used?

• Can I finally run docker based builds?• What do I need to do for the migration?

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

Best practices

24

• Modular builds and tests => Parallelization

• Play fair and don’t be a resource hog

• Build configuration in Code => Pipelines

• Docker

More: http://wiki.eclipse.org/CI_best_practiceshttps://wiki.eclipse.org/Jenkins#FAQ

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

Questions?

25

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

Takeaways

26

• Less administration => more tools and services

• New infra => better utilization and scaling• Stateless build agents will require

adaptations

Copyright (c) 2018, Eclipse Foundation, Inc. | Made available under the Eclipse Public License 2.0 (EPL-2.0)

27

Thank you!

Please evaluate the talks on eclipsecon.org!