Upload
chad-lawson
View
228
Download
2
Tags:
Embed Size (px)
Citation preview
LZRW3 Decompressordual semester project
Characterization Presentation
Students:Peleg RosenTal Czeizler
Advisors:Moshe PorianNetanel Yamin
6.4.2014
Presentation Content• Project Motivation and Goals• Project Requirements• Background – LZ77 algorithm• LZRW3 algorithm• Compressed file format• Proposed Solution• Decompression Example • Project Top Block Diagram• Planned GUI• Project Schedule and Gantt
Project MotivationWhy do we need an hardware data compression?
Why do we need an hardware data compression?-Reduce storage capacity
Project Motivation
Why do we need an hardware data compression?-Reduce storage capacity-Reduce amount of data to be handled
Project Motivation
Why do we need an hardware data compression?-Reduce storage capacity-Reduce amount of data to be handled-Reduce communication costs
Project Motivation
Why do we need an hardware data compression?-Reduce storage capacity-Reduce amount of data to be handled-Reduce communication costs-Speed
Project Motivation
Project Goals
Project Goals• Implementation of LZRW3 data decompression core.
• Implementation of LZRW3 data decompression core.
• Implementation of a verification environment.
Project Goals
Project Requirements
Project RequirementsPart A:
• Core Requirements:– Process data at the speed of 1 Gbps.– Support data blocks with output of 2KB – 32KB.– Relay only on the FPGA’s internal memory.– VHDL Implementation.
Part A:
• Core Requirements:– Process data at the speed of 1 Gbps.– Support data blocks with output of 2KB – 32KB.– Relay only on the FPGA’s internal memory.– VHDL Implementation.
• Full simulation environment (golden model and checkers).
Project Requirements
Part A:
• Core Requirements:– Process data at the speed of 1 Gbps.– Support data blocks with output of 2KB – 32KB.– Relay only on the FPGA’s internal memory.– VHDL Implementation.
• Full simulation environment (golden model and checkers).
Part B:
• Synthesis & implementation of FPGA device (Xilinx Virtex-5).
Project Requirements
Part A:
• Core Requirements:– Process data at the speed of 1 Gbps.– Support data blocks with output of 2KB – 32KB.– Relay only on the FPGA’s internal memory.– VHDL Implementation.
• Full simulation environment (golden model and checkers).
Part B:
• Synthesis & implementation of FPGA device (Xilinx Virtex-5).
• GUI implementation in VisualStudio.
Project Requirements
Background – LZ77 algorithm
Background – LZ77 algorithm Lossless data compression algorithm that was published by
Abraham Lempel and Jacob Ziv in 1977.
Output item : (offset , length ) + next literal In this case Output item = ( , ) Meaning :
BABDAC ABDBCAA
History buffer Look-ahead buffer
3 245 151 2 33B
Background – LZ77 algorithm Lossless data compression algorithm that was published by
Abraham Lempel and Jacob Ziv in 1977.
-> Take 3 literals -> Next literal will be BGo 5 steps back
Output item (Copy item): [slot address , length ] In this case Output item = [ , ]
BABDACABDBCAA21 30 3
LZRW3 compression algorithm
Hash Function
Hash Table
ABD
Slot address
1
Slot address
4 5 6
Slot address
BAB
0
ABD
Slot address
Send every 3 literals to the hash function
Put offset in the hash table
If the slot is occupied and the literals match - make copy item
6
Compressed file format
Structure
StructureFile header (8 byte)
Compressed file format
StructureFile header (8 byte)Groups:
Compressed file format
StructureFile header (8 byte)Groups: - control bytes (2 bytes)
Compressed file format
Compressed file format
StructureFile header (8 byte)Groups: - control bytes (2 bytes) - data bytes (16 - 32 bytes)* The last group might be smaller
Compressed file format
File headercontains information about the file size and whether it is compressed or not.
Compressed file format
Control bytesHold the information about the items type (copy or literal).
Compressed file formatLiteral itemsSingle bytes of data in their original form.
Compressed file format
Copy itemsTwo bytes representing hash table slot holding the offset and length of a literal sequence.
Proposed SolutionHeader decoding Determine file size and if it is compressed or not.
Proposed SolutionHeader decoding Determine file size and if it is compressed or not.
File decompressed!
File notcompressed
Proposed SolutionHeader decoding Determine file size and if it is compressed or not.
Control bytes decoding Determine number and location of copy items.
File decompressed!
Filecompressed
File notcompressed
Proposed SolutionHeader decoding Determine file size and if it is compressed or not.
Control bytes decoding Determine number and location of copy items.
File decompressed!
Filecompressed
File notcompressed Group decoding
Literal items: 1. Store in hash table2. Copy
Copy items:1. Decode2. Replace3. Copy
Proposed SolutionHeader decoding Determine file size and if it is compressed or not.
Control bytes decoding Determine number and location of copy items.
File decompressed!
Filecompressed
File notcompressed
Next group
Group decoding Literal items:
1. Store in hash table2. Copy
Copy items:1. Decode2. Replace3. Copy
Proposed SolutionHeader decoding Determine file size and if it is compressed or not.
Control bytes decoding Determine number and location of copy items.
Group decoding Literal items:
1. Store in hash table2. Copy
Copy items:1. Decode2. Replace3. Copy
EOFFile decompressed!
Filecompressed
File notcompressed
Next group
Decompression example
Decompression exampleHeader decodingThe header is composed of 8 bytes:
Meaningless zeroes
Decompression exampleHeader decodingThe header is composed of 8 bytes:
File size information: SIZE = (0 x 256) + 35 + 4 = 39
Byte 3 Byte 4
Decompression exampleHeader decodingThe header is composed of 8 bytes:
Compression indicator:0 = compressed1 = not compressed
Decompression example
Meaningless zeroes
Header decodingThe header is composed of 8 bytes:
Decompression exampleControl Bytes AnalysisThe control bytes are consisted of 2 bytes, each bit represent an item in the group:
Decompression exampleControl Bytes AnalysisThe control bytes are consisted of 2 bytes, each bit represent an item in the group:
0 – A literal item in the appropriate group position1 - A copy item in the appropriate group position
Decompression exampleControl Bytes AnalysisThe control bytes are consisted of 2 bytes, each bit represent an item in the group:
0 – A literal item in the appropriate group position1 - A copy item in the appropriate group position
Decompression exampleControl Bytes AnalysisThe control bytes are consisted of 2 bytes, each bit represent an item in the group:
0 – A literal item in the appropriate group position1 - A copy item in the appropriate group position
Decompression exampleData Bytes AnalysisBytes 11-22: literal item. Sent to hash function and copied as is to output file.
Decompression exampleData Bytes AnalysisBytes 11-22: literal item. Sent to hash function and copied as is to output file.
Decompression exampleData Bytes AnalysisBytes 11-22: literal item. Sent to hash function and copied as is to output file.
Compressed stream
Hashfunc.
Offset
Hash table
Decompression exampleHashing
The decompressor maintains a hash table identical to the one created by the compressor.
Decompression exampleData Bytes AnalysisBytes 23-24: copy item
Decompression exampleData Bytes AnalysisBytes 23-24: copy item
Decompression exampleData Bytes AnalysisBytes 23-24: copy item
Decompression exampleData Bytes AnalysisBytes 23-24: copy item
Length = 0 + 3
Decompression exampleData Bytes AnalysisBytes 23-24: copy item
The number of copied literals is 3.
The offset is found in slot 3645 in the hash table.
Length = 0 + 3
Decompression exampleData Bytes AnalysisBytes 23-24: copy item
The number of copied literals is 3.
The offset is found in slot 3645 in the hash table.
Length = 0 + 3
Project Top Block DiagramWM-
1
WS-3WM-
3
WS-2
WS-1
WM-2
LZRW3DECOMPRESSION
CORE
Project Top Block DiagramWM-
1
WS-3WM-
3
WS-2
WS-1
WM-2
LZRW3DECOMPRESSION
CORE
Serial compressed data
115200 bit/sec
Project Top Block DiagramWM-
1
WS-3WM-
3
WS-2
WS-1
WM-2
LZRW3DECOMPRESSION
CORE
Parallel data (bytes)
wishbone protocol125MHz
REUSE
Project Top Block DiagramWM-
1
WS-3WM-
3
WS-2
WS-1
WM-2
LZRW3DECOMPRESSION
CORECompresseddata bytes 125 MHz
REUSE REUSE
Project Top Block DiagramWM-
1
WS-3WM-
3
WS-2
WS-1
WM-2
LZRW3DECOMPRESSION
CORE
Decompresseddata bytes 125 MHz
REUSE REUSE
REUSE
Project Top Block DiagramWM-
1
WS-3WM-
3
WS-2
WS-1
WM-2
LZRW3DECOMPRESSION
CORE
Decompressed bytes
wishbone protocol 125 MHz
REUSEREUSE
REUSE
REUSE
Project Top Block DiagramWM-
1
WS-3WM-
3
WS-2
WS-1
WM-2
LZRW3DECOMPRESSION
CORE
Serial decompressed data
115200 bit/sec
REUSE REUSE
REUSE
REUSE
Load uncompressed file PROCESS & COMPARE
Input size
x
Output (golden model):Messages (progress): (sending packets to FPGA/ decompressing/
getting files back/ comparing results)
(browse)
RAW INPUT
COMPRESSED INPUT
STATUS
Planned GUI
Load expected output file (browse)
DECOMPRESSLoad compressed file (browse)
COMPARE
Manually insert data PROCESS & COMPARE
Date Goals
21/3/2014 – 5/4/2014 Project Characterization &Algorithm interpreting
6/4/2014 Characterization Presentation
7/4/2014 – 20/4/2014 Full Characterization of all blocks
21/4/2014 – 10/6/2014 •System blocks VHDL •Design
11/6/2014 Mid presentation
11/6/2014 – 25/6/2014 Building a simulation environment
26/6/2014 – 25/7/2014 Work on project paused for exams
Project Schedule 1/2
Date Goals
30/7/2014 – 3/9/2014 Simulation run & debug
4/9/2014 Part A - Final presentation
21/9/2014 – 5/10/2014 FPGA synthesis & implementation
6/10/2014 – 23/10/2014
GUI implementation
24/10/2014 – 8/11/2014
Tests & debug
9/11/2014 – 23/11/2014
Writing project portfolio
24/11/2014 Final project presentation
Project Schedule 2/2
Weeks: 0 - 5 6 – 12 13 – 19 20 - 26 27 - 32
Characterization & interpretation
Characterization presentation
Blocks characterization VHDL blocks
implementation Mid presentation Simulation
environment Exams Simulation – Cont. Part A - Final pres. FPGA synthesis GUI implementation Tests & debug Writing portfolio Final presentation
.........……
……………...………..
............….……………………………………………………….…….…
44
7
2
2
2
2
3
Project Gantt
2
.……………………………………….…….…