CLS: Control System

E. Matias, R. Berg, G. Wright, T. Wilson, T. Johnson, R. Tanner and H. ZhangCanadian Light Source

University of SaskatchewanInvited Talk – Particle Accelerator Conference 2005 Knoxville, Tennessee

CLS Control System Heritage

• Saskatchewan Accelerator Laboratory (SAL) operated from the late 1960s until 1999.– Control system evolved from

PDP-8 -> PDP-11 -> VAX -> NeXT and Sun workstations.

– IO was based on CAMAC with two CAMAC data highways.

– Some Micro84 PLCs.

• Control System was locally developed running on BSD UNIX.

• Was it open source? Yes, but only because so much

of it was locally developed.

• What to do with the grass field?

CLS Control System Heritage

• 1999 March 31 funding for CLS was approved. Nuclear physics program was discontinued.

• The existing Linac would need to be reconfigured and refurbished.

• Linac Controls:– CAMAC hardware would

need to be replaced.– Power supplies would need to

be upgraded.– RF control would need to be

redesigned.– The old computer hardware

would need to be replaced.• We need to make some

design choices....

CLS Control System Principles

• System design based on highly distributed control.• Extensive use of single board computers (originally used in

SAL).• Target lifetime of 15+ years.• Data communication over Ethernet when possible.• System must be user-friendly. • The accelerator and beamline systems must be maintainable by

a small team. • Reliability and availability of beam are critical to the success of

the facility.• Building an open source control system was not the initial goal, it

was the outcome.

• Accelerator complex must be complete by Dec. 2003 and the first phase of beamlines by Dec. 2004. The project must come in on budget.

• The options: (1) EPICS or (2) Isagraph/Virgo.

• EPICS was selected, since it had:– large built up accelerator and beamline user community;– availability of suitable drivers and utilities;– credibility with the CLS user community; and– good design.

• EPICS Extensions selected include:– EDM,– Accelerator Toolbox, – Gateway and– Data Archiver.

• EPICS extensions that were locally developed:– assortment of drivers,– IOC Auto-Save-Restore,– simple beamline scanning program, and– SQL Alarm Management Database.

Selection of a Distributed Control Platform

• The options: (1) Sun or (2) Linux.

• Linux was selected, since it had:– better hardware

availability, and– fairly equivalent reliability

levels.

• EPICS Extensions selected include:– EDM,– Array Display Tool,– StripTool, and– Knob Manager.

• We are now starting to deploy touch screens running Linux/EDM.

Selection of an Operator Workstation Platform

• PLC hardware extensively used– Telemecanique Momentum– Siemens S7/300– Siemens S7/400– Siemens S7/400 F (safety critical applications)

• Software Written using IEC 61131-3 Programming Language primarily Graphical Function Block Notation

PLC Control

Selection of a Hard Real-Time Operating System

• The Options: (1) RTEMS and (2) VxWorks.

• RTEMS was selected, since it had:– good experience from SAL,– additional flexibility with single board

computers, and– high level of reliability.

• IOCs are CLS/SIL embedded controllers (approx 150) based on the MC68360 25 MHz. Processor.

• Pros and Cons:– No dynamically loaded libraries;

must be linked prior to download.– Large number of IOCs (separation

of function but more points of failure).

• The options: (1) Matlab, (2) SciLab, or (3) root.

• Matlab was selected primarily because of the availability of the accelerator toolbox and staff experience.

• Matlab is commercial, the accelerator toolbox is open source.

• Software originates from ALS and SPEAR III.

• Augmented with other CLS specific utilities.

• Also being used as a commissioning tool for beamlines.

• Special care is required to maintain consistency with other parts of the control system.

Selection of an online scripting/analysis package

• Single board computers (EPICS/RTEMS) used for:– stepper motors,– power supply control,– vacuum equipment

monitoring,– radiation monitors, and– other RS-232 devices.

• PLC hardware/software used for machine protection.

• Industrial PCs with VME used for diagnostics.

• Linux servers used for high-level control, network services and EPICS/PLC interface.

Implementation

Linux and VME

• Using VME hardware connected to a Linux PC.

• SIS1100 PCI card <-> fiber optic link <-> SIS3100 VME module

• Maps VME backplane to IOC memory.

• Advantages:– PC can be physically separated from VME crate.– More than one VME crate per PC.– Multiple applications can access the same crate.– High throughput 25 to 80 Mbytes/sec block

transfer.

• Work ongoing on RTEMS support.

/dev/SIS1100_2(descriptor 2)

VME CRATE 1:Hardware

mapped memory

VME CRATE 2:Hardware

VME CRATE 3:Hardware

Fiber Optic link

FiberOpticLink

Fiber Optic Link

/dev/SIS1100_1(descriptor 1)

/dev/SIS1100_3(descriptor 3)

EPICSApplication

PV recordread

routines

PV recordwrite

routines

Linux IOCsis1100 PCI

card

sis1100 PCIcard

sis1100 PCIcard

• Provides fiber optic signal distribution of triggers.

• VXI based hardware• IOC running EPICS on

RTEMS.• Operator

Interface implemented using Glade.

• Glade wasselected forthe table andfile handlingcapabilities.

Timing System

• Beamline Controls are based on the same software and hardware as the accelerator systems.

• Each beamline is on a separate virtual network.

• The EPICS Gateway provides links between the different networks.

• Matlab is used for scripting.

Beamlines

Interface to Administrative Systems

• Administrative (financials and work-management) databases are based on MS-SQL.

• freeTDS was used to provide an interface into these systems.

• The control-system is a source of alarm-data that can be used to trigger preventative maintenance activities.

• This integration is still a work in progress.

VLANs for: each beamline, machine control, development, office, visitors

VME Crate

(Reflective Memory)

MicroStep

EROCIOC

RTEMS

FieldDev. RS-232

Devices

OPI

Linux

IOCStep Controller

RTEMS

Motors

MicroStep

OPI

Linux

OPI

Linux

Touch PanelOPI Linux

NetworkServer

(bootp, dhcp,auto restore)

Linux

DataArchiveServer

Linux

AlarmServer

MS-Win

MS-SQLServer

MS-Win

PowerEdgeIOC

Linux

PS BoardsIOC

RTEMS

PowerSupplies

EROCIOC

RTEMS

FieldDev.

EthernetDevices

PLC & GPIB

FieldDev.

MagnetsMotors

1GigBridge

IOC

Linux

FieldDev.

ProfibusPLC

System Architecture

Lessons Learned

• The use of open source did not compromise reliability or availability.

• Configuration management is critical to the success of the system.

• Layering of different software packages requires care to ensure the system is reliable.

• Additional flexibility in available in managing upgrades compared to commercial software.

• Dependence on suppliers continuing to support the product is not an issue.

• Our, building automation system is a case in point, where the vendor has discontinued the product and we are now replacing it with EPICS and PLC hardware.

Winter at the CLS