Embed Size (px)
Citation preview
01
About me02
Andrey AdamovichJava/Groovy developer
DevOps guy, automation junkie
Coauthor of Groovy 2 Cookbook
Coorganizer of @latcraft and @devternity
Coach at @devchampions
Twitter: @codingandrey
••••••
03
Background04
BackgroundBig projects built by Ant, Maven, and eventually Gradle
Teams composed mostly of Java developers
Complex (sometimes, overengineered) architectures
Many environments (DEV, TEST, QA, SIT, UAT, PREPROD, PROD)
to support
••••
05
Problems IInfrastructure is influenced by (relatively) frequent architecture
changes (components, versions, layers)
We want our environments to be the same (or at least quite similar) to
avoid any side effects during development, testing and production
We don't want to spend hours/days/weeks on configuring each and
every new server and keeping them insync
•
•
•
06
Problems IIOperations guys are not always available (e.g. busy supporting
production systems or just not skilled enough)
Development infrastructure (Jenkins, Sonar, Version Control, Load
Testing etc.) also needs maintenance
We want to reuse experience available in our team and avoid throwing
in too many various trendy technologies that will fail our expectations
•
•
•
07
First Blood08
Ant + Gradleant.taskdef(
name: 'scp',
classname: 'o.a.t.a.t.o.ssh.Scp',
classpath: configurations.secureShell.asPath)
ant.taskdef(
name: 'sshexec',
classname: 'o.a.t.a.t.o.ssh.SSHExec',
classpath: configurations.secureShell.asPath)
01.
02.
03.
04.
05.06.
07.
08.
09.
09
Simple callant.sshexec(
host: host,
username: user,
password: password,
command: command,
trust: 'true',
failonerror: failOnError)
01.
02.
03.
04.
05.
06.
07.
10
Sshoogr
11
Sshoogr featuresGroovybased SSH DSL for:
Remote command execution
File uploading/downloading
Tunneling
•••
12
Sshoogr0.9.23!
13
Sshoogr usage (import)@Grab(
group='com.aestasit.infrastructure.sshoogr',
module='sshoogr',
version='0.9.23')
import static com.aestasit.ssh.DefaultSsh.*
01.
02.
03.
04.
05.
14
Sshoogr usage (defaults)defaultUser = 'root'
defaultKeyFile = new File('secret.pem')
execOptions {
verbose = true
showCommand = true
}
01.
02.
03.
04.
05.
06.
15
Sshoogr usage (connection)remoteSession {
url = 'user2:654321@localhost:2222'
exec 'rm ‐rf /tmp/*'
exec 'touch /var/lock/my.pid'
remoteFile('/var/my.conf').text = "enabled=true"
}
01.
02.
03.
04.
05.
06.
16
Sshoogr usage (multiline content)remoteFile('/etc/yum.repos.d/puppet.repo').text = '''
[puppet]
name=Puppet Labs Packages
baseurl=http://yum.puppetlabs.com/el/
enabled=0
gpgcheck=0
'''
01.
02.
03.
04.
05.
06.
07.
17
Sshoogr usage (appendable)remoteFile('/etc/motd') << 'Additional message'
new File('localFile') << remoteFile('/etc/motd')
localFile << remoteFile('/etc/motd') << 'msg'
01.
02.
03.
18
Sshoogr usage (file copying)remoteSession {
scp {
from { localDir "$buildDir/application" }
into { remoteDir '/var/bea/domain/application' }
}
}
01.
02.
03.
04.
05.
06.
19
Sshoogr usage (file copying)remoteSession {
scp {
from { remoteDir '/var/bea/domain/application' }
into { localDir "$buildDir/application" }
}
}
01.
02.
03.
04.
05.
06.
20
Sshoogr usage (file copying)remoteSession {
scp {
from {
remoteFile '/etc/init.d/service1'
remoteFile '/etc/init.d/service2'
}
into { localDir "$buildDir/application" }
}
}
01.
02.
03.
04.
05.
06.
07.
08.
09. 21
Sshoogr usage (command result)def result = exec(command: '/usr/bin/mycmd',
failOnError: false, showOutput: false)
if (result.exitStatus == 1) {
result.output.eachLine { line ‐>
if (line.contains('WARNING')) {
throw new RuntimeException("Warning!!!")
}
}
}
01.
02.
03.
04.
05.
06.
07.
08.
09. 22
Sshoogr usage (shortcuts)if (ok('/usr/bin/mycmd')) {
...
}
if (fail('/usr/bin/othercmd')) {
...
}
01.
02.
03.
04.
05.
06.
23
Sshoogr usage (tunnels)tunnel('1.2.3.4', 8080) { int localPort ‐>
def url = "http://localhost:${localPort}/flushCache"
def result = new URL(url).text
if (result == 'OK') {
println "Cache is flushed!"
} else {
throw new RuntimeException(result)
}
}
01.
02.
03.
04.
05.
06.
07.
08.
09. 24
Sshoogr usage (prefix/suffix)prefix('sudo ') {
exec 'rm ‐rf /var/log/abc.log'
exec 'service abc restart'
}
suffix(' >> output.log') {
exec 'yum ‐y install nginx'
exec 'yum ‐y install mc'
exec 'yum ‐y install links'
}
01.
02.
03.
04.
05.
06.
07.
08.
09. 25
Sshoogr: more featuresANSI coloured output
Download/upload progress
HTTP proxy
•••
26
Sshoogr: executablesshoogr ‐h 192.168.43.122
‐u ubuntu
‐l color
default.sshoogr
01.
02.
03.
04.
27
Sshoogr: *.sshoogrNo need to install Groovy
Can ommit @Grab
Can ommit connection details
Scripts are more portable
Ansible for Groovy!
•••••
28
Sshoogr: *.sshoogrremoteSession {
exec('uname ‐a')
exec('date')
...
}
01.
02.
03.
04.
05.
29
sdkman.io
30
sdkman.iocurl ‐s http://get.sdkman.io | bash
sdk install sshoogr 0.9.23
sdk default sshoogr 0.9.23
sdk list sshoogr
01.
02.
03.
04.
31
More goodies32
Intellij IDEADSL
33
Intellij IDEA DSL
34
Intellij IDEA DSL
35
SSHD Mock36
SSHD MockMockSshServer.with {
command('^ls.*$') { inp, out, err, callback, env ‐>
out << '''total 20
drwxr‐xr‐x 3 1100 1100 4096 Aug 7 16:52 .
drwxr‐xr‐x 8 1100 1100 4096 Aug 1 17:53 ..
drwxr‐xr‐x 3 1100 1100 4096 Aug 7 16:49 examples
callback.onExit(0)
}
...
01.
02.
03.
04.
05.
06.
07.
08.
09. 37
SSHD Mock ...
command('^whoami.*$') { inp, out, err, callback, env ‐>
out << "root\n"
callback.onExit(0)
}
command('^du.*$') { inp, out, err, callback, env ‐>
out << "100\n"
callback.onExit(0)
}
...
01.
02.
03.
04.
05.
06.
07.
08.
09.
10.38
SSHD Mock ...
// Create file expectations.
dir('.')
dir('/tmp')
...
01.
02.
03.
04.
05.
39
SSHD Mock ...
// Start server
startSshd(2233)
}
01.
02.
03.
04.
40
Gradleintegration41
Gradle integrationbuildscript {
repositories { mavenCentral() }
dependencies {
classpath 'com.a....sshoogr:sshoogr‐gradle:0.9.18'
}
}
apply plugin: 'secureShell'
01.
02.
03.
04.
05.
06.
07.
42
Gradle integrationtask remoteTask << {
remoteSession("user:password@localhost:22") {
exec 'rm ‐rf /tmp/cache/'
scp "$buildDir/cache.content",
'/tmp/cache/cache.content'
}
}
01.
02.
03.
04.
05.
06.
07.
43
Little brother44
Groowin@Grab('com.aestasit.infrastructure.groowin:groowin:0.1.8')
import static com.aestasit.winrm.DefaultWinRM.*
01.
02.
45
GroowinremoteManagement {
host = '127.0.0.1'
user = 'Administrator'
password = 'secret'
exec 'del', 'C:\\temp.txt'
remoteFile('C:\\my.conf').text = "enabled=true"
}
01.
02.
03.
04.
05.
06.
07.
46
PUnit47
PUnitSimple testing tool for verifying remote server state
Uses Sshoogr and JUnit
Reuse reporting features of JUnit
As simple as ...
••••
48
PUnit: example (derby)class DerbyInstallTest
extends BasePuppetIntegrationTest {
@Before
void installDerby() {
apply("include derby")
}
...
}
01.
02.
03.
04.
05.
06.
07.
08.
49
PUnit: example (derby)@Test
void ensureDerbyRunning() {
command('service derby status > derbystatus.log')
assertTrue fileText("/root/derbystatus.log")
.contains('Derby')
assertTrue fileText("/root/derbystatus.log")
.contains('is running.')
}
01.
02.
03.
04.
05.
06.
07.
08.
50
PUnit: example (derby)@Test
void ensureCanConnect() {
Thread.sleep(10000)
uploadScript()
command('/opt/derby/db‐derby‐10.9.1.0‐bin/bin/ij ' +
'testDataScript.sql > derbytest.log')
...
01.
02.
03.
04.
05.
06.
07.
51
PUnit: example (derby) ...
// Check if the log of the insert
// operation contains the word ERROR.
assertFalse(
"The script should return at least one error",
fileText("/root/derbytest.log")
.contains('ERROR')
)
...
01.
02.
03.
04.
05.
06.
07.
08.
09. 52
PUnit: example (derby) ...
// Check on data that was inserted into a table.
assertTrue(
"The log should contain a SELECT result",
fileText("/root/derbytest.log")
.contains('Grand Ave.')
)
}
01.
02.
03.
04.
05.
06.
07.
08.
53
PUnit: example (jenkins)session {
tunnel ('127.0.0.1', 8080) { int localPort ‐>
def driver = new HtmlUnitDriver(false)
driver.manage()
.timeouts()
.pageLoadTimeout(300, TimeUnit.SECONDS)
.implicitlyWait(30, TimeUnit.SECONDS)
driver.get("http://127.0.0.1:${localPort}/login")
...
01.
02.
03.
04.
05.
06.
07.
08.
09.
10.54
PUnit: example (jenkins) ...
def input = driver.findElement(By.name('j_username'))
input.sendKeys('john')
input = driver.findElement(By.name('j_password'))
input.sendKeys('123456')
input.submit()
...
01.
02.
03.
04.
05.
06.
07.
55
PUnit: example (jenkins) ...
def wait = new WebDriverWait(driver, 30)
wait.until ExpectedConditions.
presenceOfElementLocated (By.linkText('John Doe'))
...
}
}
01.
02.
03.
04.
05.
06.
07.
56
PUnit: example (svn)session {
tunnel ('127.0.0.1', 80) { int localPort ‐>
// Initilize repository connection data.
DAVRepositoryFactory.setup()
def url = SVNURL.create('http', null, '127.0.0.1',
localPort, 'repos/cafebabe', true)
def repository = SVNRepositoryFactory.create(url)
println "Verifying SVN repository at ${url}"
...
01.
02.
03.
04.
05.
06.
07.
08.
09. 57
PUnit: example (svn) ...
// Setup credentials.
def authManager = SVNWCUtil.
createDefaultAuthenticationManager('joe', '123456')
repository.setAuthenticationManager(authManager)
// Verify repository is at revision 0.
assertEquals 0, repository.getLatestRevision()
...
01.
02.
03.
04.
05.
06.
07.
08.
09. 58
PUnit: example (svn) ...
// Commit first revision.
ISVNEditor editor = repository.
getCommitEditor("Initial commit.", null)
editor.with {
openRoot(‐1)
addFile('dummy.txt', null, ‐1)
applyTextDelta('dummy.txt', null)
def deltaGenerator = new SVNDeltaGenerator()
01.
02.
03.
04.
05.
06.
07.
08.
09. 59
PUnit: example (svn) ...
def checksum = deltaGenerator.sendDelta('dummy.txt',
new ByteArrayInputStream("data".getBytes()),
editor, true)
closeFile('dummy.txt', checksum)
def commitInfo = closeEdit()
println commitInfo
}
...
01.
02.
03.
04.
05.
06.
07.
08.
09. 60
PUnit: example (svn) ...
// Verify repository is at revision 1 now.
assertEquals 1, repository.getLatestRevision()
}
}
01.
02.
03.
04.
05.
61
PUnit: continuous integration
62
PUnit: Jenkins build
63
Summary64
Sshoogr is...Battletested Groovy DSL for SSH connectivity
Executable and portable scripting tool
Easily integratable with any Java/Groovy library
•••
65
Sshoogr can be used for...Provisioning your servers and IoT devices
Executing remote orchestration commands
Testing and monitoring infrastructure state
•••
66
Next stepsResource definitions
Command rollbacks
Parallel execution
XSS utilities
Extend integration tests
Better documentation
••••••
67
Seekingcontributors!
68
Source codeSshoogr: https://github.com/aestasit/sshoogr
Sshoogr Gradle: https://github.com/aestasit/sshoogrgradle
Groowin: https://github.com/aestasit/groowin
Groowin Gradle: https://github.com/aestasit/groowingradle
••••
69
Thank you!70
Questions?71
