CS 150 - Spring 2007 – Lec. #11: Course Project - 1

Videoconferencing Project

! Project Concept and Background

! Checkpoint Structure

! Bells and Whistles

CS 150 - Spring 2007 – Lec. #11: Course Project - 2

Objectives

! Broad “brush” overview of the project

! Details will be covered in the lab lectures, startingnext week

! NOTE: anything discussed in the lab lectures andproject checkpoint write-ups supercedes what Idescribe here!" Neil and Allen have a working implementation of the project

" They know the project better than I do! Listen to them!

CS 150 - Spring 2007 – Lec. #11: Course Project - 3

Course Project: VideoconferencingSystem

! Not quite this… but:" Video camera capture" CRT video display" Serial compressed video

2-way transmission betweentwo stations

" Wireless communications

" (no audio this semester)" Implemented in a

Xilinx FPGA on theCalinx boards in the lab

" Groups of two -- your Lab #4/#5 partner" Commit to a TA now for grading purposes

CS 150 - Spring 2007 – Lec. #11: Course Project - 4

Calinx EECS 150 Lab/Project Protoboard

Flash Card &

Micro-drive Port

Video Encoder &

Decoder

AC ’97 Codec &

Power Amp

Video & Audio Ports Four 100 Mb

Ethernet Ports

8 Meg x 32

SDRAM

Quad Ethernet

Transceiver

Xilinx

Virtex 2000ESeven Segment

LED Displays

Prototype

Area

CS 150 - Spring 2007 – Lec. #11: Course Project - 5

Complete Videoconferencing System

Display

Video Encoder

Video Encoder(Checkpoint #1)

Video Decoder

Camera

Videostream

Video

Decoder

Checkpoint #2

Checkpoint #4

SDRAM(Checkpoint #0)

Multiport SDRAM

Memory System

MultiportArbitration

Wireless Transceiver(Checkpoint #3)

CS 150 - Spring 2007 – Lec. #11: Course Project - 6

Checkpoint #0/#1/#2:SDRAM Interface

! Memory protocols" Bus arbitration" Address phase" Data phase

! DRAM is large, but few address lines and slow" Row & col address" Wait states

! Synchronous DRAM provides fast synchronous access currentblock" Little like a cache in the DRAM" Fast burst of data

! Arbitration for shared resource

CS 150 - Spring 2007 – Lec. #11: Course Project - 7

Checkpoint #1: Video Encoding! Pixel Array:

" Digital image represented bymatrix of values, where each is afunction of the informationsurrounding it in the image; singleelement in image matrix: pictureelement or pixel (includes info forall color components)

" Array size varies for differentapps and costs: some common sizesshown

! Frames:" Illusion of motion created

by successively flashing still

pictures called frames

High-Definition Television (HDTV), 2 Mpx

Workstation, 1 Mpx

PC/Mac, 1‡2 Mpx

Video, 300 Kpx

SIF,82 Kpx

High-Definition Television (HDTV), 1 Mpx

CS 150 - Spring 2007 – Lec. #11: Course Project - 8

Checkpoint #1: Video Encoding

! Video details fairly complex and involve many choices:" NTSC vs. PAL, HDTV, …

" Interleaved even-odd frames (TV) vs. progress scan (computer anddigital displays)

" Frame size, frame rate

" Pixel encodings: RGB, YUV/YCB (Luminance, Chrominance --brightness plus color difference signals)

" Subsampling to reduce data demands (compression trick)

" Inputs: ITU-R BT.601 Format (Digital Broadcast NTSC)

" Outputs: Component video, S-video to drive LCDs in lab

" Fortunately, Calinx board has a chip on-board that deals with muchof the grungy details …

CS 150 - Spring 2007 – Lec. #11: Course Project - 9

ITU-R BT.656 Details! Interfacing details for ITU-601

Pixels per line 858

Lines per frame 525

Frames/sec 29.97

Pixels/sec 13.5 M

Viewable pixels/line 720

Viewable lines/frame 487

! With 4:2:2 chroma sub-sampling,send 2 words/pixel (Cr/Y/Cb/Y)

! Words/sec = 27MEncoder runs off a 27MHz clock

! Control info (horizontal & verticalsynch) is multiplexed on data lines

! Encoder data stream show to right

! See video tutorial documents oncourse documentation web page!

718 719 720 721 0 1 2

359 360 0 1

359 360 0 1

736732( )

368366( )

368366( )

857863)(

Y 7

18

Y 7

19

C

360

B

Y 7

20

C

360

R

Y 7

21

C

359

B

C

359

R

Y 7

36(7

32)

C

368(3

66)

B

C 3

68(3

66)

R

Y 8

55( 8

61)

C

428(4

31)

B

Y 8

56( 8

62)

Y 8

57(8

63)

C

0B

Y 0

C

0R

Y 1

C

428(4

31)

R

C 0

B

Y 0

Y 1

C

0R

C

359

B

Y 7

18

Y 7

19

C

359

R

Last sampleof digital active line

Sample datafor O instant

First sampleof digital active lineH

Luminancedata, Y

Chrominancedata, CR

Chrominancedata, CB

Replaced bytiming reference

signal

Replaced bydigital blanking data

Replaced bytiming reference

signal

End ofactive video

Start ofactive video

Timing reference signals

Note 1 – Sample identification numbers in parentheses are for 625-line systems where these differ from those for 525-line systems. (See also Recommendation ITU-R BT.803.)

FIGURE 1

Composition of interface data stream

D01

CS 150 - Spring 2007 – Lec. #11: Course Project - 10

Checkpoint #1: Video Encoder

! Display driver processes pixels within frame buffer

! Drive ADV7194 video encoder device to output correct NTSC video

! Gain lots of experience reading data sheets

! Dictates the 27 MHz operation rate" Used throughout graphics subsystem

CS 150 - Spring 2007 – Lec. #11: Course Project - 11

Calinx On-Board Video Encoder! Analog Devices ADV7194: ITU 601/656 in, Composite Video Out

! Supports:" Multiple input formats and outputs

" Operational modes, slave/master

" Used in default mode: ITU-601 as slave

s-video output

! Digital input side connected to Virtex pins

! Analog output side wired to on boardconnectors or headers

! I2C interface for initialization:" Wired to Virtex

CS 150 - Spring 2007 – Lec. #11: Course Project - 12

SDRAM READ Burst Timing

CS 150 - Spring 2007 – Lec. #11: Course Project - 13

Checkpoint #2: Video Decode

! Pretty much the reverse of the encodingprocess of Checkpoint #1

! We will provide the base Verilog for video decode

! You will need to integrate video decode with yourSDRAM arbitrated write port

! Integrate with your Checkpoint #1

CS 150 - Spring 2007 – Lec. #11: Course Project - 14

Checkpoint #3: Wireless Transceiver

! This will involve interfacing tothe wireless transceiver chip onthe Calinx2 board

! Neil working on a cleardescription of how this works

CS 150 - Spring 2007 – Lec. #11: Course Project - 15

Checkpoint Build Up to Complete Project

! Week #7: Lab #6/Checkpoint #0 -- Basic SDRAM Subsystem

! Week #8: Checkpoint #1 -- SDRAM to Video Display (Encoder)

! Week #9: Checkpoint #2 -- Local Video System" Video Capture (Decoder) to SDRAM to Video Display (Encoder)

" Video Decoder Verilog will be provided to you

! Week #10/11 : Checkpoint #3 -- Wireless Transceiver" Midterm #2 scheduled for Week #10

" Spring break between Week #10 and #11

! Week #12/13: Checkpoint #4 -- Putting it altogether" Video Capture to SDRAM to Wireless Transceiver to SDRAM to

Video Display

! Week #14: Final Report

CS 150 - Spring 2007 – Lec. #11: Course Project - 16

Possible Bells and Whistles

! Still thinking about this but here are some ideas:" Performance tuning: larger remote display, higher refresh rate

" Sending more data per unit time via compression/decompressionthrough the wireless transceiver

" Your good idea here

" NOTE: We don’t necessary know how to implement these ourselves!(these haven’t been implemented in the TA solution, for example)

" NOTE: There will be a bonus for an early demo of the completeproject at the end of Week #12 (one week early)

" NOTE: Extra credit will be limited to 20% extra points and no extracredit unless the standard functionality works