University of South Australia
Distributed Reconfiguration
Avishek Chakraborty, David Kearney, Mark Jasiunas
Outline
Motivation
Background
Our approach
Experiment
Conclusion
Motivation
• UAVs Limited power
complex signal / image processing applications
FPGAs
• Swarm of UAVs Constant surveillance – 52 hrs* Cooperative applications:
tracking, geo-referencing, bush fire monitoring
Can we do distributed computing with multiple FPGAs?
Migrating applications – dying UAV Larger application – distribute it in
the Swarm
Motivation
New AppUAV – 1 (FPGA)
UAV – 2 (FPGA)
• Mobility of Reconfigurable Applications– If one UAV fails
– Any other king of internal fault
– Can the state of a FPGA be migrated during runtime?
• New App – lack of computational resources– Distribute over a network of
FPGAs
Background
App
ReconfigME
• Hardware modules– Independent – Reusable – Third party
• Dynamic Reconfiguration– Slot based– ReconfigME
• Adaptive applications– Swapping of modules
during runtime – Detection / Tracking
Background
• ReconfigME– A preemptive framework
– A proxy based connection between SW and HW running on the FPGA board
– Built to support streaming applications but application developers had to do the check pointing tasks
• Currently there are quite a few number of frameworks that supports slot / tile based reconfiguration
ReconfigME
Application Development
• 1 – D and 2 – D reconfigurable framework exists– Runtime placement and routing – Not in the application development level
• Application development is difficult– Need to have core hardware knowledge– Even when third party modules are available; developers need to
worry about state saving– Inter module communication
• More complicated with multiple FPGAs– Need a higher level simple model
Kahn Process Network - KPN
• Process Networks– Communicates via infinite FIFOs– Writing channel is unblocking but
reading is blocking
• Benefits– Deterministic– Concurrency can be implemented
safely– Inherently distributed in nature– Distributed memory
• Suits with module based design– Applications can be easily
represented as KPN nodes
Kahn Process Network - KPN
• KPN buffers as registers– Application states are stored in
registers– If the buffers are saved then
application can be preempted restarted again from the same position
– Hence the whole application can be migrated to another FPGA
• A high-level framework is needed– Storing / restoring of buffers– Connection between the modules– To use the modules in a plug and play
fashion
b
a
d
c
Capturing State of a distributed application
Kahn Process Network - KPN
• Implementation KPN buffers cannot be infinite– Though some have proposed virtually infinite buffers– The above is directed acyclic graph that may experience deadlock (Thomas M.
Parks 1987)– Such constructs can be avoided– In KahnME buffering is done in software but this can be done in HW as well
Buffer Limits
• The KPN nodes as Agents– The nodes are autonomous– Migrates to the most suitable
platform– A solution for distributed
mapping
• Agent encapsulates the KPN node as well as the input buffer– Making them mobile with the
state– Migrate from one FPGA to
another
Agent-based KPN nodes
KPNnode
State-less State-full
Buf
fer
• Rule based Agents– Conditions for migration– The rule engine runs in a separate thread– Rules are written in a XML file
• Consists of HW and SW part– The SW part consists of the API to connect to the
FPGA and the Migration Control Thread– HW part is the module to be placed on the FPGA
Agent-based KPN nodes
Processing Thread
Migration Control Thread
States Modules
ReconfigME API
Rul
es
Events
Reactions
Agent
Platform
• An agent nodes can be classified as:– Source, Sink, and Process
– Source / Sink virtualizes the underlying hardware, like drivers
– Basic elements of any application
• Subtle issues: migration of source and sinks– Can be migrated! but not physically
– The agent will look for a similar source/sink in the swarm
– What if the platform along with source / sink node is dies ?
– Failure detections
Agent-based KPN nodes
Source
SinkP
roce
ss
Overall Model
KahnME
FPGA FPGA FPGA FPGA
ReconfigMEReconfigME ReconfigME ReconfigME
• Blue – B tracking experiment– Two Stationary UAV platforms and
another Ground-Station – Each UAV platforms having Vertex
1000E FPGA (I know very slow ) – The tracking application searches for
a Blue-B
• Blue-B Tracker– The application is launched in the
Ground-Station – it soon starts searching for
appropriate platform– Once the Blue-B is found the agent
will migrate to another platform if lost again; we did this by manually moving the camera
Tracking Experiment
Tracking ExperimentThe three platforms
• Migration Issues– The migrating agent’s size was 360kB.
– Size of the sates 1500 bytes
– Rest was of the Blue-B template !! (should have saved it locally but not
realistic)
– The migrating module consisted of 18.2 kB SW and 492 kB HW (EDIF)
– Hence any typical agent will need around 1.5 kB to 672 kB of data during the migration
• More Improvements– By compressing the data
– In practical situation migrations will be less– Newer board
Tracking Experiment
Results
• This work represents a first try to run application in a geographically distributed network of FPGAS – Distributed Reconfiguration
• We have argued that there is a need to standardized state saving techniques to built frameworks for DPR – reusable modules
• We have proposed that KPN buffers can be treated as registers holding states
• We have proposed to treat KPN nodes as autonomous agents to solve the problem of distributed reconfiguration
Conclusion