Team Introduction
http://acip.us Facilitator: Bob Urberger Computer Engineering majors Space Systems Research Lab
RASCAL Mission “Rascal is a two-spacecraft
mission to demonstrate key technologies for proximity operations…”
“After the on-orbit checkout, one 3U spacecraft is released and passively drifts away.... After a suitable distance, the released spacecraft will activate its propulsion system and return to within a few meters of the base. The second spacecraft will be released and the process repeated...”
RASCAL ACIP
Imaging Payload Awareness of Cubesat
EnvironmentComputer Vision
Low-Level ProcessingObject DetectionDistance Determination
High-Level DataNavigationThruster Control
RASCAL ACIP
Functional BreakdownUnique Face Identifier
Capture Images
Transfer Data
Process Images
Output/Store Control Data
RASCAL ACIP
Known Pattern
RawData
StructuredData
High-LevelData
Modules
RASCAL ACIP
LEDs
Camera
Computational Hardware
Unique Face Identification
Capture Images
Transfer Data
Process Images
Output/Store Control Data
LEDs Required to perform
classification Not enough detail
visible for other features
Three approaches Unique pattern of
LEDs for each face Unique combination
of colors for each face Both unique patterns
and colors
Unique pattern works regardless of camera spectrum Fails when face
partially visible Color combinations
only work with visible spectrum cameras Can classify cube
corners as well as faces with well chosen color patterns
RASCAL ACIP
Camera Potential Camera
Choices: FLIR Tau 640
640x480, 14-bit Visible spectrum
image sensor 2-5MP - 16 to 24 bit
color Parallel data output
from cameras
Component Requirements: FLIR
PCB integration Control signaling simple Low resolution,
monochromatic 16.1 MB/s input data rate @
30Hz Visible spectrum
Requires lens fixture Complex control signaling High resolution, wide color
range 2MP with 24 bit color:
○ 57.6 MB/s input data rate @ 30Hz
RASCAL ACIP
Processing Hardware Processing blocks in hardware Caching and system control managed in
software Timing and Gate Consumption Alternatives:
Pure software implementationPure hardware implementation
RASCAL ACIP
Imaging Functions Image Processing
Pre- Processing
DistanceDetection
ObjectDetection
ObjectClassification
RASCAL ACIP
ImageDataStructure
Distance Data
ImageEdges
Objects In Frame
Preprocessing Noise suppression
Color Conversion
Object enhancement
Image segmentation
Conversion and downsampling
RASCAL ACIP
Distance Detection Identify depth from a
single image Monocular Cues
Relative sizeComparison of
imaged objects to known shape scale at particular depth
Structured geometry identified should be easy to identify scale regular structureSquare or equilateral
triangle
RASCAL ACIP
Distance Detection in RASCAL
Square LED pattern on spacecraft faceCritical point identificationHomography estimationProjective transformPoint correspondence for scale
Hardware DomainParallel matrix multiplication
RASCAL ACIP
Object Detection Identifying Objects
in an Image
Region or Contour Based
Edge DetectionRelies heavily on Pre-
processing
RASCAL ACIP
(Columbia University)
Object Detection in RASCAL
Hardware DomainCanny/Deriche Sobel Operator
ConstraintsCubesat sizeEnvironmentalResolution
RASCAL ACIP
(Columbia University)
Objection Classification Post-object
detection / image segmentation
Support vector machine (metric space classification)
Assign a class based upon pre-programmed control data
RASCAL ACIP
Object Classification in RASCAL
RASCAL ACIP
Completed with bare-metal software
ARM Assembly / C Minimum distance principle
(efficient) Multi-tiered and/or multi-
dimensional space from attributes given
Determine a number of attributes with significant differences between faces
Testing: expect a very high level (>95%) of correct classifications
Constraints of Object Classification Must work with a
variety of backgrounds (Earth, Moon, Sun, Space, etc.)
Ideally real time (bounded) and low latency
Updated at >=10 Hz
Must function with different sizes (patterns can vary from a few pels to larger than the frame)
Definitive discrimination functions with high reliability
Alternative algorithm: neural nets, fuzzy logic
Output to Control System will output calculated information about
placement, attitude, distance, etc. In the future, a separate team will construct a
system to interpret data and convert to control signals/data
Since this is out of the scope of our project, the output format/setup is ultimately our choice
RASCAL ACIP
Functional Testing Output Unique Pattern
Capture ImagesStream ImagesVerify Control Signals
Transfer DataOscilloscopeFrame Buffer
RASCAL ACIP
Process ImageSoftware VerificationHardware Verification
Output/Store DataBuffer
System Testing Camera integration Hardware timing constraints Block connectivity verification
Blocks signal each other as intended Full pipeline simulation
Blocks interact as expected Physical synthesis testing
Data produced from each frame
RASCAL ACIP
Timeline
RASCAL ACIP
Data In and Out of System
Obtain Camera
Separate Processing into Blocks
Interface Camera with Hardware
Algorithm Verification in Software
Store Camera Data in Hardware
Preprocess Image
Achieve Block Functionalality
Merge Processing Blocks
Confirm Full Integration
Project Wrap-Up
10/28 11/17 12/7 12/27 1/16 2/5 2/25 3/17 4/6 4/26
Dates of Years 2013-2014
Project Task
Estimated Costs
Designed for very low budget and small amount of needed materials
Largely out of SSRL funding
Function Part Low Estimate High Estimate Notes
Obtain vision data Camera $4,000 $10,000 SSRL funding
Camera Specification $0 $10,000 SSRL funding
Display patterns LEDs $1 $10 SSRL funding
Algorithm Resources Books $0 $0 library & creative commons
Development Tools Zedboard $0 $0 donation
Xilinx Vivado $0 $0 donation
Oscilloscope $0 $0 provided
Desktop PC $0 $0 provided
TOTAL COST: $4,001 $20,010 RASCAL ACIP
Future Work for Integration
Thruster control system
Placement into spacecraft
Radiation, vibration, space-readiness
We will provide thorough documentation for future groups
RASCAL ACIP
Bibliography http://cubesat.slu.edu/AstroLab/SLU-03__Rascal.html Jan Erik Solem, Programming Computer Vision with
Python. Creative Commons. Dr. Ebel, Conversation Dr. Fritts, Conversation Dr. Mitchell, Conversation Milan Sonka, Vaclav Hlavac, Roger Boyle, Image
Processing, Analysis, and Machine Vision. Cengage Learning; 3rd edition.
http://www.cs.columbia.edu/~jebara/htmlpapers/UTHESIS/node14.html
RASCAL ACIP