Upload
others
View
19
Download
0
Embed Size (px)
Citation preview
CS-570Statistical Signal Processing
Spring Semester 2019
Grigorios Tsagkatakis
Lecture 1: Introduction to CS-570
Spring Semester 2017-2018CS-541 Wireless Sensor Networks
University of Crete, Computer Science Department1
Today’s Objectives
• CS-570 Overview
• Introduction to Statistical Signal Processing
CS-541 Wireless Sensor NetworksUniversity of Crete, Computer Science Department
2Spring Semester 2017-2018
About CS-570
• Lectures
Monday 12:00-14:00, H208
Wednesday 14:00-16:00, H204
• Office Hours: 1 Hour after each class
• Prerequisites: Digital Signal Processing (CS-370), Applied Mathematics for Engineers (CS-215), Probabilities (CS-217)
CS-541 Wireless Sensor NetworksUniversity of Crete, Computer Science Department
3Spring Semester 2017-2018
About CS-570
Practical Information
• 2 individual homeworks on the material taught (30% of your final grade)• Exercises on MATLAB/python• 1st assignment will be handed out at the beginning of March• 2 weeks time to complete each assignment (hard deadline).
• Standalone project (max for 2 students) (50% of your final grade)• Research topic• Experimental work / analysis on experimental data• Submission of a project report in a technical paper form (motivation, related work, problem
formulation, adopted methodology, results, conclusions & outlook)• Duration: mid of April - End of semester (~mid of June)
• Written Exam (20% of your final grade)
All above are compulsory for getting a grade at the end of the exam
CS-541 Wireless Sensor NetworksUniversity of Crete, Computer Science Department
4Spring Semester 2017-2018
Topics
CS-541 Wireless Sensor NetworksUniversity of Crete, Computer Science Department
5
Week 1: Introduction to Statistical Signal Processing & Review
Week 2: Introduction to optimization
Week 3: Signal sensing and reconstruction
Week 4: Computational imaging
Week 5: Deterministic signal processing
Week 6: High and low dimensional signal processing
Week 7: Statistical signal models
Week 8: Time-series modeling
Week 9: Distributed signal processing
Week 10: Machine learning for signal processing
Week 11: Applications in remote sensing
Week 12: Applications in Internet-of-Things
Week 13: Review and presentations
Spring Semester 2017-2018
6Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Signal Processing fundamentals
• Acquisition
• Processing
• Analysis
• Storage
• Transmission
7Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Temperature& Humidity
Image Sound
Pressure Vibration,Motion
Glucose (&biometrics)
Signal Processing fundamentals
• Acquisition
• Processing
• Analysis
• Storage
• Transmission
8Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Signal Processing fundamentals
• Acquisition
• Processing
• Analysis
• Storage
• Transmission
9Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Signal Processing fundamentals
• Acquisition
• Processing
• Analysis
• Storage
• Transmission
10Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Signal Processing fundamentals
• Acquisition
• Processing
• Analysis
• Storage
• Transmission
11Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Big Data
The 5VsVolume
12Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Big Data
The 5VsVolume
Velocity
13Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Big Data
The 5VsVolume
Velocity
14Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Big Data
The 5VsVolume
Velocity
15Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Big Data
The 5VsVolume
Velocity
Variety
16Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Big Data
The 5VsVolume
Velocity
Variety
Veracity
17Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Big Data
The 5VsVolume
Velocity
Variety
Veracity
Value
18Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Earth Observation
19Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Astrophysics
20Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Sky Survey Project Volume Velocity
Sloan Digital Sky Survey (SDSS)
50 TB 200 GB per day
Large Synoptic Survey Telescope (LSST )
~ 200 PB 10 TB per day
Square Kilometer Array (SKA )
~ 4.6 EB 150 TBper day
Internet-of-Things
21Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Biomedical signals
22Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Neuroscience
23Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Imaging technologies
24Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
25Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Fundamental Signal Processing
• Signal Sensing: Compressed Sensing
• Inverse problems: Signal Denoising, Enhancement
• Filtering
• Time-series modeling and prediction
• Modeling: dimensionality reduction
26Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
• -
27Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
28Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
power unit(battery based – limited lifetime!)
sensors(transducer, measuring a physical phenomenon e.g.
heat, light, motion, vibration, and sound)
Memory /storage
(data acquisition, and preprocessing, buffers
handling)
transceiver(connection to the outer-world, e.g. other sensor nodes, or data
collectors --sinks)
• Sensor Node
• Basic unit in sensor network
• Contains on-board sensors, processor, memory, transceiver, and power supply
CS-541 Wireless Sensor NetworksUniversity of Crete, Computer Science Department
29
Sensing node paradigm
microProcessor(communication with sensors &
transceivers , preprocessing, buffers handling, etc)
Spring Semester 2017-2018
Review of basic concepts
30Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Vectors
31Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
• A column vector where
• A row vector where
denotes the transpose operation
Vectors
• Vectors can represent an offset in 2D or 3D space
• Points are just vectors from the origin
• Data (pixels, gradients at an image keypoint, etc) can also be treated as a vector
• Such vectors don’t have a geometric interpretation, but calculations like “distance” can still have value
32Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Matrix
33Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
• A matrix is an array of numbers with size 𝑚 ↓ by 𝑛 →, i.e. m rows and n columns.
• If , we say that is square.
=
Basic Matrix Operations
34Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
• Addition
• Can only add a matrix with matching dimensions, or a scalar.
• Scaling
Matrix Operations
• Inner product (dot product) of vectors• Multiply corresponding entries of two vectors and add up the result
• x·y is also |x||y|cos( the angle between x and y )
• If B is a unit vector, then A·B gives the length of A which lies in the direction of B
35Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Matrix Operations
Matrix Multiplication
• Properties
• Powers• We can refer to the matrix product AA as A2, and AAA as A3, etc.
• Only square matrices can be multiplied that way
36Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Matrix Operations
• Transpose
• Determinant• returns a scalar
• Represents area of the parallelogram described by the vectors in the rows of the matrix
• For
• Properties:
37Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Matrix Operations
• Trace
• Invariant to a lot of transformations
• Properties:
38Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Special matrices
• Identity matrix I
• Diagonal matrix
• Symmetric matrix
• Skew-symmetric matrix
39Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Matrix Inverse
• Given a matrix A, its inverse A-1 is a matrix such that AA-1 = A-1A = I
• Inverse does not always exist. If A-1 exists, A is invertible or non-singular. Otherwise, it’s singular.
• For matrices that are invertible
40Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
System of linear equations
41Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Solution of system
• Inverse of a matrix
•Solution of systems of linear equations
• Provided A-1 exists
• If both x and y are solutions thenis also a solution for any real α
42Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
z = αx + (1 − α)y
Linear combinations
• Linear combination of some set of vectors {v(1), . . . , v(n)} is given by multiplying each vector v(i) by a corresponding scalar coefficient and adding the results:
•The span of a set of vectors is the set of all points obtainable by linear combination of the original vectors.
43Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Norms
44Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Norms
• L2 norm, also known as Euclidean norm
• L21 norm
• Infinite norm (or max norm)
• Frobenius norm (Matrix norm)
45Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Linear independence
46Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
• Suppose we have a set of vectors v1, …, vn
• If we can express v1 as a linear combination of the other vectors v2…vn, then v1 is linearly dependent on the other vectors. • The direction v1 can be expressed as a combination of the directions
v2…vn. (E.g. v1 = .7 v2 -.7 v4)
• If no vector is linearly dependent on the rest of the set, the set is linearly independent.• Common case: a set of vectors v1, …, vn is always linearly independent if
each vector is perpendicular to every other vector (and non-zero)
Linear independence
47Spring Semester 2019CS-570 Statistical Signal Processing
University of Crete, Computer Science Department
Not linearly independentLinearly independent set