1

1

Lloyd C. Cha & Vishnu Mohan P.

Perforce European User ConferenceApril 2008

Improving Performance at Remote Sites

2

Requirements from all sites

Consistent view of the repository

Updates visible in real-time

Read and write access

Network considerations

Bandwidth

Latency

Packet loss

Distributed Development

2

3

Montalvo Systems

Fabless semiconductor startupcompany

Two geographically distant development sites

Perforce Server (main site)

Perforce Proxy (remote site)

4

Perforce at Montalvo Systems

Depot

Hardware design source files

Supporting scripts, testbenches, and tests

Mostly small files

Release

Qualified set of files and revisions

Known to pass basic regression test suite

Release filelist stored in client workspace

mtv wrapper script

Enforces consistent clientspecs

Handles release creation and retrieval

3

5

Perforce Proxy Configuration

6

Perforce Proxy

Metadata

Not cached on Perforce Proxy

Have list must be updated when client state changes

Communication between Proxy and Server

Status of cached files on Proxy

State of all changes on client

Efficiency of Proxy

Large files more efficient than small files

4

7

Tasks performed by p4 sync

Determine which files and versions to retreive

–Lookup information in db.label if filespec uses @

–Lookup information in db.have if filespec uses #have

–Translate filepath to depotspec

Retrieve actual data from proxy or server

–Update proxy cache from server if required

Write data to disk at Perforce client workspace

Update have list on Perforce Server

8

Functional Behavior of P4P

5

9

Experiments with Flush vs Sync

Testcase: sync to release x

Sync 14 min 33 sec

Flush 1 min 1 sec

sync -f #have 5 min 56 sec

TOTAL 6 min 57 sec

Separate flush/sync takes less

than half as much time

10

Turbo Mode

Goal:

Minimize Proxy/Server communication over WAN

Idea:

Update have list independently of client workspace update

Obtain release filelist from proxy

p4 flush updates have list

p4 sync -f <file>#have updates client workspace

p4 sync -f <file>#none for deleted files

Caveat:

Abnormal termination leaves client workspace out of sync with server have list state

6

11

Turbo Mode

12

Turbo Mode Implementation

7

13

Turbo Mode Results

N a t ive P ro xy Tu rb o M o d e

1 0 0 0 9 m in 1 7 s e c 6 m in 3 8 se c

5 0 0 5 m in 2 1 s e c 3 m in 5 9 s e c

10 0 2 m in 5 0 s e c 2 m in 4 s e c

6 3 5 s e c 3 1 se c

U p d a t e

S iz e

14

Turbo Mode Summary

Consistently faster performance, particularly on large updates

Resource utilization similar to native mode

Reliable results when not interrupted

8

15

Data locality

Source/Destination Physical location

Source of user request Client host (remote)

Proxy (remote) or Server (local)

Source for user data Proxy (remote) or Server (local)Server (local)

Destination of user data Client workspace (remote)

Source of release filelist

Destination of have list

16

Data flow

Data Source DestinationServer (local) Script memory (local)

Have list Script memory (local) Server (local)User data Proxy (remote) Client workspace (remote)

Release filelist

9

17

Remote Mode

Goal:

Minimize communication across WAN

Idea:

Execute transactions local to source/destination

Server site:

Obtain release filelist from server

p4 flush updates have list

Client site update of workspace:

p4 sync -f <file>#have (in parallel batches)

p4 sync -f <file>#none

Caveat:

Abnormal termination leaves client workspace out of sync with server have list state

18

Remote Mode

10

19

Remote Mode Implementation

20

Remote Mode Results

N a t ive P ro xy

1 0 00 9 m in 1 7 s e c 4 1 s e c

5 0 0 5 m in 2 1 se c 3 8 s e c

1 0 0 2 m in 5 0 se c 2 9 s e c

6 3 5 s e c 1 8 s e c

U p d a t e

S iz e

Re m o t e M o d e

( t yp ic a l)

11

21

Remote Mode Summary

Superior but inconsistent performance

Local analysis and flush command

Parallel client workspace update

Scales well with large numbers of files

Heavy use of bandwidth due to parallel

sync

Frequent hangs and aborted updates

22

Bypass Mode

Goals:

Rapid retrieval of data from Proxy cache

Minimize communication between Proxy and Server

Idea:

Bypass Proxy daemon and read cache directly

p4 flush updates server have list

mtv retrieves data directly from cache

• Text files reconstructed using RCS commands

• Binary files copied directly from cache

• Link targets read from RCS and created by script

• Unavailable files use native Perforce Proxy

Caveats:

Abnormal termination leaves client workspace out of sync with server

Proxy cache is world readable

12

23

Bypass Mode

24

Bypass Mode Implementation

13

25

Bypass Mode Results

N a t ive P ro xy B yp a s s M o d e

10 00 9 m in 17 se c 5 m in 3 5 se c

5 0 0 5 m in 2 1 s e c 3 m in 2 se c

1 00 2 m in 5 0 s e c 1 m in 3 4 se c

6 3 5 s e c 3 5 se c

U p d a t e

S iz e

26

Bypass Mode Summary

Consistently faster performance

Most efficient with large sets of files

Efficient use of bandwidth

Reliable transfer of information

Based on Turbo Mode

Lacks Remote Mode optimizations

14

27

Results summary

Update mode Normalized Performance RateNumber of Releases 1000 500 100 6

Native Proxy 1.00 1.00 1.00 1.00Turbo 1.40 1.34 1.37 1.13Remote 13.59 8.45 5.86 1.94Bypass 1.66 1.76 1.81 1.00

28

Future Work

Fix remote mode usability and reliability issues

Replace ssh method with daemon for remote commands

Debug random hangs

Port remote mode optimizations to bypass mode

Remote execution of p4 flush

Simultaneous flush and data retrieval

Multi-threaded data retreival

Improve retrieval of data from cache

Write daemon to replace direct RCS access

15

29

Future Work

Research details on internal Perforce communication protocols

Client – Proxy – Server

Rewrite mtv

Use Perforce supported API

Consider using labels vs. filelists

Filelists avoid lookups on Perforce Server

Labels reduce request traffic

30

Q & A