Topic:- Coding Scheme

(i).- Information theory

(ii).- Error coding scheme.

Submitted to:- Submitted By:-

Mr. Hemant Kumar Meena 1. Pawan Kumar Jangid

Assistant Professor 2. Sunil Kumar Yadav

Dept. of Electrical Engineering,MNIT, Jaipur 3. Manish Kumar Bagara

4. Chandan Kumar

What is information theory:-

Information theory is a branch of applied mathematics, electrical engineering, and computer science involving the quantification of information. Information theory was developed by Claude E. Shannon to find fundamental limits on signal processing operations such as compressing data and on reliably storing and communicating data.

Concept of Information Theory:-

A key measure of information is entropy, which is usually expressed by the average number of bits needed to store or communicate one symbol in a message. Entropy quantifies the uncertainty involved in predicting the value of a random variable. For example, specifying the outcome of a fair coin flip (two equally likely outcomes) provides less information (lower entropy) than specifying the outcome from a roll of a die (six equally likely outcomes).

Application of information theory:-

Lossless data compression (e.g. ZIP files)

Lossy data compression (e.g. Mp3s and jpegs)

Channel coding (e.g. For digital subscriber line (DSL)).

Important sub-fields of information theory are source coding, channel coding, algorithmic complexity theory, algorithmic information theory, information-theoretic security, and measures of information.

APPLICATIONS IN OTHER AREAS:-

Including statistical inference

Natural language processing,

Cryptography

Neurobiology

The evolution and function of molecular codes

Model selection in ecology thermal physics

Quantum computing

Linguistics, Plagiarism detection, Pattern Recognition, Anomaly Detection

Error coding scheme (Error detection and correction) :- Techniques that enable reliable delivery of digital data over unreliable communication

channels.

Error detection techniques allow detecting errors, while error correction enables reconstruction of the original data in many cases.

Definitions:-

The general definitions of the terms are as follows:-

1. Error detection is the detection of errors caused by noise or other impairments during transmission from the transmitter to the receiver.

2. Error correction is the detection of errors and reconstruction of the original, error-free data.

Error detection schemes:-

1 Repetition codes

2 Parity bits

3 Checksums

4 Cyclic redundancy checks (CRCs)

5 Cryptographic hash functions

6 Error-correcting codes

1. Repetition codes:-

Coding scheme that repeats the bits across a channel to achieve error-free communication.

The data are divided into blocks of bits. Each block is transmitted some predetermined number of times.

For example, to send the bit pattern "1011", the four-bit block can be repeated three times, thus producing "1011 1011 1011". However, if this twelve-bit pattern was received as "1010 1011 1011" – where the first block is unlike the other two – it can be determined that an error has occurred.

2. Parity bits:-

A parity bit is a bit that is added to a group of source bits to ensure that the number of set bits (i.e., bits with value 1) in the outcome is even or odd. It is a very simple scheme that can be used to detect single or any other odd number (i.e., three, five, etc.) of errors in the output. An even number of flipped bits will make the parity bit appear correct even though the data is erroneous.

Extensions and variations on the parity bit mechanism are horizontal redundancy checks, vertical redundancy checks, and "double," "dual," or "diagonal" parity (used in RAID-DP).

Figure - Even-parity checking scheme

3. Checksums:-

A checksum of a message is a modular arithmetic sum of message code words of a fixed word length. The sum may be negated by means of a ones'-complement operation prior to transmission to detect errors resulting in all-zero messages.

Checksum schemes include parity bits, check digits, and longitudinal redundancy checks. Some checksum schemes, such as the Damm algorithm, the Luhn algorithm, and the Verhoeff algorithm, are specifically designed to detect errors commonly introduced by humans in writing down or remembering identification numbers.

Figure- Calculating Checksum

4. Cyclic redundancy checks (CRCs)

A cyclic redundancy check (CRC) is a non-secure hash function designed to detect accidental changes to digital data in computer networks; as a result, it is not suitable for detecting maliciously introduced errors. It is characterized by specification of what is called a generator polynomial, which is used as the divisor in a polynomial long division over a finite field, taking the input data as the dividend, such that the remainder becomes the result.

CRCs are particularly easy to implement in hardware, and are therefore commonly used in digital networks and storage devices such as hard disk drives.

Figure - basic scheme for cyclic

redundancy checking

5. Cryptographic hash functions

• The output of a cryptographic hash function, also known as a message digest, can provide strong assurances about data integrity, whether changes of the data are accidental or maliciously introduced.

• Any modification to the data will likely be detected through a mismatching hash value.

• Furthermore, given some hash value, it is infeasible to find some input data that will yield the same hash value.

• If an attacker can change not only the message but also the hash value, then a keyed hash or message authentication code (MAC) can be used for additional security. Without knowing the key, it is infeasible for the attacker to calculate the correct keyed hash value for a modified message.

6. Error-correcting codes

Any error-correcting code can be used for error detection. A code with minimum Hamming distance, d, can detect up to d − 1 errors in a code word. Using minimum-distance-based error-correcting codes for error detection can be suitable if a strict limit on the minimum number of errors to be detected is desired.

Codes with minimum Hamming distance d = 2 are degenerate cases of error-correcting codes, and can be used to detect single errors. The parity bit is an example of a single-error-detecting code.

Error Correction :-The techniques that we have discussed

so far can detect errors, but do not correct them.

Error Correction can be handled in two ways:-

1 . Backward error correction:-

When an error is discovered, the receiver can have the sender retransmit the entire data unit.

2 . Forward error correction(Error-correcting code):-

An error-correcting code (ECC) or forward error correction (FEC) code is a system of adding redundant data, or parity data, to a message, such that it can be recovered by a receiver even when a number of errors (up to the capability of the code being used) were introduced, either during the process of transmission, or on storage.

• Since the receiver does not have to ask the sender for retransmission of the data, a backchannel is not required in forward error correction.

• Error-correcting codes are frequently used in lower-layer communication, as well as for reliable storage in media such as CDs, DVDs, hard disks, and RAM.

• In theory it is possible to correct any number of errors atomically.

• The number of bits required to correct multiple-bit or burst error is so high that in most of the cases it is inefficient to do so. For this reason, most error correction is limited to one, two or at the most three-bit errors.

Single-bit error correction:-

Concept of error-correction can be easily understood by examining the simplest case of single-bit errors.

• As we have already seen that a single-bit error can be detected by addition of a parity bit with the data, which needed to be send. A single additional bit can detect error, but it’s not sufficient enough to correct that error too. For correcting an error one has to know the exact position of error. For this one has to know the required number of redundant bits.

To calculate the numbers of redundant bits (r) required to correct d data bits .

2r ≥ d+r+1

Where d = no. of bits to be transmitted and

r = no. of redundant bits

Approach to a solution :-

1. Decide on the number of bits in the code word.

2. Determine the bit positions of the check bits.

3. Determine which parity bits check which positions

4. Calculate the values of the parity bits

The bit positions covered by each parity bit can be calculated by writing each bit position as a sum of the powers of 2:

1 = 1

2 = 2

3 = 1 + 2

4 = 4

5 = 1 + 4

6 = 2 + 4

7 = 1 + 2 + 4

8 = 8

9 = 1 + 8

10 = 2 + 8

11 = 3 + 8

12 = 4 + 8

a

Check bit 1 governs positions 1, 3, 5, 7, 9Check bit 2 governs positions 2, 3, 6, 7, 10Check bit 4 governs positions 4, 5, 6, 7, 12 Check bit 8 governs positions 8, 9, 10, 11, 12

man

Applications of Error Coding Scheme:-

Internet:-

In TCP/IP stack, error control is performed at multiple levels:

-In Ethernet carries a CRC checksum.

-IPv4 header use checksum to protect the contain of header.

- IPv6 header, use link layer technology.

-UDP provide checksum for error correction.

-TCP provides a checksum for protection of addressing information from the IP headers.

Deep-space telecommunications:-• The Voyager 1 and Voyager 2 missions, in 1977, uses

convolution codes and reed Muller codes to delivers color imaging amongst scientific information of Jupiter and Saturn.

• The Voyager 2 craft additionally Reed–Solomon code allowed for very powerful error correction.

• Space mission uses more powerful Turbo codes and LDPC code.

Satellite broadcasting :-

• The demand for satellite transponder bandwidth continues to grow.

• It uses forward error correction .

• QPSK coupled with Reed Solomon codes.

• PSK ,QAM increase transponder efficiency.

Data storage:-

• Error detection improve reliability of data storage.

• ORC use in group code recoding detect and correct the errors.

• Read Solomon code used compact discs to correct errors caused by scratches.

• Modern hard drives uses CRC codes to detect and correct the minor errors.

Error-correcting memory:-

DRAM memory provide protection against soft errors by relying on error correcting codes.

ECC used in deep space application due to increase radiation.

ECC generally use Hamming codes.

ECC supports check summing errors detected on PCI bus. .

Thank You!