Accelerator control at iThemba LABS

Some background

• No formal reliability procedures • Cost considerations• SSC operational 24/7• Shutdown total of 2 months/year• Equipment often unavailable during shutdown• Want it working NOW• Inadequate testing time• Reliable delivery of beam

Some history

• Cyclotron control systems originally designed (late 70s) around a few mini-computers (HP 1000s running RTE)

• Control electronics and instrumentation interfaced via CAMAC

• Lab-built interactive devices (joysticks, set-point units, etc)

Some more history

• Control system migrated to distributed PC-based system in the early 90s

• Distributed memory-resident tables of control variables

• Originator node computers controlling interface electronics maintain their own control variables

• The console and other nodes requiring access to these variables link to this originator node

• Adequate diagnostic messages for debugging and more efficient fault-finding

• Communication over Ethernet LAN• Development of in-house interfaces

(SABUS)

Contributions to control system reliability

• Control system migrated to distributed PC-based system running OS/2

• If a particular node computer fails, other nodes are not affected

• Minimal preventative maintenance• All fans and filters checked, cleaned and/or

replaced during annual major shutdown• Daily archiving of control system database• Communication over Ethernet LAN• The original CSMA/CD Ethernet is a

“passive” broadcast bus which is resilient to most node failures

Contributions to control system reliability

• Development of in-house “simple” interfaces (SABUS) to control electronics

• Simple, robust, noise-immune 8-bit parallel differential bus connecting up to 15 SABUS crates per PC controller card

• Each crate contains up to 13 cards, each card controlling between 2 (e.g., power supplies) and 64 (e.g., relays) pieces of equipment

• An assortment of I/O cards developed using readily-available components

• Simple bus design allows for easier maintenance and development, easier migration to new/upgraded operating system and longevity of hardware

Contributions to control system reliability

• Design for graceful degradation• Highest level of control at the console nodes• Control can devolve down to the

instrumentation interface nodes with each having a graphical control screen for the variables originated on that node

• Enables convenient testing of variables and their associated interface electronics

• Many hardware interfaces can be switched into local mode and controlled from front panel

Improvements to control system infrastructure

• Network with 1Gb/s fibre-optic backbone to distributed managed switches

• VLAN with private IP addresses (restricted routing to campus VLAN)

• Nodes assembled from good quality components (motherboards, CPUs, RAM, disk drives, power supplies, etc)

• Adequate cooling (particularly of disk drives)• On-site stock of spares (PC components,

interface modules, etc)• Back-up clones of critical disk drives for fast

replacement following failures• Most nodes and I/O module crates powered

via individual UPSs

Current control system developments

• Migrate control system onto EPICS platform• Mature stable code• Active development in, and support from, a

number of similar international labs• Many useful utilities available in EPICS

(logging, archiving, alarming, etc.)• Run old and EPICS-based subsystems in

parallel• Retain hardware (SABUS) interfaces

EPICS migration roadmap

• Cyclotrons – all new developments on EPICS platform (for example, beam splitter control)

• Develop gateway between old table-based control variables and EPICS process variables

• Port old subsystems onto EPICS over time• Electrostatic accelerators’ controls at both lab

sites (Cape and Gauteng) have successfully migrated almost 100% to EPICS

EPICS-based VDG vacuum control

Improve resilience of cooling, power and network infrastructure

Eskom Transformer 1

Transformer 2

Diesel Generator

UPS

High-intensity beam project

• Increase 66MeV proton beam intensity up to 500µA

• New vertical beam target station to accommodate high intensities

• Tighter control required to reduce possibilities of equipment damage and other safety issues

• Flattop systems for SPC1 injector and SSC cyclotrons

• Non-destructive beam position monitors• Continuous beam position monitoring and

automatic alignment• Beam splitter to deliver beams simultaneously

to two radioisotope production targets

Control diagnostics for high-intensity beam

Target ruptures and other damage

Initial analysis of damage

Autoradiography of targets

Simulation of target charge distribution

Search for instabilities

Control diagnostics for high-intensity beam

• Urgent development of control diagnostics to increase protection

• Beam halo monitors to detect stray beam in high-energy beam line

• Real-time analysis of beam distribution on production target

• Monitoring of RF systems for instabilities