Upload
msu-gml-videogroup
View
1.036
Download
2
Tags:
Embed Size (px)
DESCRIPTION
Citation preview
Восстановление сжатого видео
Моисейцев Алексей
Video GroupCS MSU Graphics & Media Lab
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
222
Содержание
Введение
Простые модели Adaptive Fuzzy Post-Filtering
DCT Re-application
Классификация Support Vector Regression
Modified Mean-Removed Classified Vector Quantization
Регуляризация
Заключение
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
3
Введение
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
4
Введение
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
666
Содержание
Введение
Простые модели Adaptive Fuzzy Post-Filtering
DCT Re-application
Классификация Support Vector Regression
Modified Mean-Removed Classified Vector Quantization
Регуляризация
Заключение
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
77
Adaptive Fuzzy Post-Filtering
Adaptive Fuzzy Post-Filtering for Highly Compressed Video. Hao-Song Kong, Yao Nie, Anthony Vetro, Huifang Sun, Kenneth E. Barner. ICIP 2004.
пикселязначениеновоеx
блокатипаотзависит
фильтрасила
eba
xx
xxxx
c
ba
j
N
j c
j
N
j cjc
~
,
),(
),(
),(~
/)(
2
2
1
1
22
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
88
Adaptive Fuzzy Post-Filtering
Adaptive Fuzzy Post-Filtering for Highly Compressed Video. Hao-Song Kong, Yao Nie, Anthony Vetro, Huifang Sun, Kenneth E. Barner. ICIP 2004.
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
99
Adaptive Fuzzy Post-Filtering
Adaptive Fuzzy Post-Filtering for Highly Compressed Video. Hao-Song Kong, Yao Nie, Anthony Vetro, Huifang Sun, Kenneth E. Barner. ICIP 2004.
PSNR 30.61 dB PSNR 30.93 dB
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
10
Adaptive Fuzzy Post-Filtering
Преимущества
Быстрая работа
Простота в реализации
Недостатки
Просто маскировка артефактов
Замыливание изображения
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
111111
Содержание
Введение
Простые модели Adaptive Fuzzy Post-Filtering
DCT Re-application
Классификация Support Vector Regression
Modified Mean-Removed Classified Vector Quantization
Регуляризация
Заключение
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
1212
DCT Re-application
1. Сдвинуть изображение по вертикали игоризонтали на (i,j)
2. Сжать изображение
3. Вернуть изображение обратно, т.е. сдвинуть на (-i,-j)
4. Повторить для всех сдвигов из [-3, 4] x [-3, 4]
5. Усреднить результат
Enhancement of JPEG-Compressed Images by Re-application of JPEG. Aria Nosratinia. Department of Electrical Engineering, University of Texas at Dallas, Richardson. 2002.
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
1313
DCT Re-application
Enhancement of JPEG-Compressed Images by Re-application of JPEG. Aria Nosratinia. Department of Electrical Engineering, University of Texas at Dallas, Richardson. 2002.
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
1414
DCT Re-application
Enhancement of JPEG-Compressed Images by Re-application of JPEG. Aria Nosratinia. Department of Electrical Engineering, University of Texas at Dallas, Richardson. 2002.
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
1515
ji ,
DCT Re-application
ji
jiji
ji xDQDy,
,),(
,
ниядекодирова и якодировани оператор -
сдвига оператор -
еизображени енноевосстановл -
еизображени сжатое -
Q
D
y
x
Enhancement of JPEG-Compressed Images by Re-application of JPEG. Aria Nosratinia. Department of Electrical Engineering, University of Texas at Dallas, Richardson. 2002.
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
1616
DCT Re-applicationПример работы
Enhancement of JPEG-Compressed Images by Re-application of JPEG. Aria Nosratinia. Department of Electrical Engineering, University of Texas at Dallas, Richardson. 2002.
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
1717
DCT Re-applicationПример работы
Enhancement of JPEG-Compressed Images by Re-application of JPEG. Aria Nosratinia. Department of Electrical Engineering, University of Texas at Dallas, Richardson. 2002.
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
1818
DCT Re-applicationПример работы
Enhancement of JPEG-Compressed Images by Re-application of JPEG. Aria Nosratinia. Department of Electrical Engineering, University of Texas at Dallas, Richardson. 2002.
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
19
DCT Re-application
Преимущества
Заметное уменьшение артефактов
Прост в реализации
Недостатки
Желательно знать параметры сжатия
Ориентирован на изображения
Невысокая скорость работы
(~58 умножений на пиксель)
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
202020
Содержание
Введение
Простые модели Adaptive Fuzzy Post-Filtering
DCT Re-application
Классификация Support Vector Regression
Modified Mean-Removed Classified Vector Quantization
Регуляризация
Заключение
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
2121
Support Vector Regression
Support Vector Machine
Compression Artifact Reduction Using Support Vector Regression. Sanjeev Kumar, Truong Nguyen, Mainak Biswas. ICIP 2006
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
2222
Support Vector Regression
l
i
iiCww1
*
21 ,
0,
)(,
)(,
*
*
ii
iii
iii
ybxw
bxwy
ii yx , - тренировочные данные
)(xfy - моделируемая функция
bxwy ),( - вид модели
Минимизация
при условии
Compression Artifact Reduction Using Support Vector Regression. Sanjeev Kumar, Truong Nguyen, Mainak Biswas. ICIP 2006
элемента каждого для отклонение,
элементоввсех для отклонение допустимое*
i
i
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
2323
Support Vector Regression
Compression Artifact Reduction Using Support Vector Regression. Sanjeev Kumar, Truong Nguyen, Mainak Biswas. ICIP 2006
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
2424
Support Vector Regression
Compression Artifact Reduction Using Support Vector Regression. Sanjeev Kumar, Truong Nguyen, Mainak Biswas. ICIP 2006
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
25
Support Vector Regression
Compression Artifact Reduction Using Support Vector Regression. Sanjeev Kumar, Truong Nguyen, Mainak Biswas. ICIP 2006
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
26
Support Vector Regression
Преимущества
Визуальное улучшение на некоторых типах изображений
Недостатки
Зависимость от обучающей выборки
Сложно прогнозируемое поведение
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
272727
Содержание
Введение
Простые модели Adaptive Fuzzy Post-Filtering
DCT Re-application
Классификация Support Vector Regression
Modified Mean-Removed Classified Vector Quantization
Регуляризация
Заключение
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
2828
MMRCVQ (Modified Mean-Removed Classified Vector Quantization)
Artifact reduction of compressed color images using modified mean-removed classified vector quantization. Jim Zone-Chang Lai, Yi-Ching Liaw, and Winston Lo. Journal of the Chinese Institute of Engineers, Vol. 27, No. 5, pp. 747-751 (2004)
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
2929
MMRCVQПостроение кодирующих последовательностей
Выбор блоков 4x4
Нормировка
Кластеризация
Алгоритм Linde-Buzo-Gray
данныеисходные,,1:
данныесжатые,,1:''
NixT
NixT
i
i
iiiiii xxzxxz ,'''
Artifact reduction of compressed color images using modified mean-removed classified vector quantization. Jim Zone-Chang Lai, Yi-Ching Liaw, and Winston Lo. Journal of the Chinese Institute of Engineers, Vol. 27, No. 5, pp. 747-751 (2004)
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
3030
MMRCVQLinde-Buzo-Gray
Artifact reduction of compressed color images using modified mean-removed classified vector quantization. Jim Zone-Chang Lai, Yi-Ching Liaw, and Winston Lo. Journal of the Chinese Institute of Engineers, Vol. 27, No. 5, pp. 747-751 (2004)
Выбор
начального
разбиения
K-meansОценка
дисперсии
Удвоение числа
кластеров
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
3131
MMRCVQLinde-Buzo-Gray
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
3232
MMRCVQLinde-Buzo-Gray
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
3333
MMRCVQ
Выбор кодирующих последовательностей
Разбиение несжатых данных
Выбор декодирующих последовательностей
NiMkliyxdyxdxR likiik ...1,...1:),'(),'(|
MiyC i ,...,1:''
i
Rx
j
iR
x
yij
Artifact reduction of compressed color images using modified mean-removed classified vector quantization. Jim Zone-Chang Lai, Yi-Ching Liaw, and Winston Lo. Journal of the Chinese Institute of Engineers, Vol. 27, No. 5, pp. 747-751 (2004)
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
3434
MMRCVQВосстановление
Artifact reduction of compressed color images using modified mean-removed classified vector quantization. Jim Zone-Chang Lai, Yi-Ching Liaw, and Winston Lo. Journal of the Chinese Institute of Engineers, Vol. 27, No. 5, pp. 747-751 (2004)
Классификация
блока
Однородный Текстура
Граница
Определение
направления
границы
Отсечение
плохих
результатов
Поиск Замена блока
+
-
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
3535
MMRCVQРезультаты
Artifact reduction of compressed color images using modified mean-removed classified vector quantization. Jim Zone-Chang Lai, Yi-Ching Liaw, and Winston Lo. Journal of the Chinese Institute of Engineers, Vol. 27, No. 5, pp. 747-751 (2004)
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
3636
MMRCVQРезультаты
Artifact reduction of compressed color images using modified mean-removed classified vector quantization. Jim Zone-Chang Lai, Yi-Ching Liaw, and Winston Lo. Journal of the Chinese Institute of Engineers, Vol. 27, No. 5, pp. 747-751 (2004)
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
37
MMRCVQ
Преимущества
Хорошее качество
Приемлемая скорость восстановления
Недостатки
Только устранение блочности
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
383838
Содержание
Введение
Простые модели Adaptive Fuzzy Post-Filtering
DCT Re-application
Классификация Support Vector Regression
Modified Mean-Removed Classified Vector Quantization
Регуляризация
Заключение
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
3939
Regularized Iterative Restoration
)()(2
fEgffJ
Compressed Video Enhancement using Regularized Iterative Restoration Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999
N
k
N
k
N
k
n
n
n
n
n
g
g
g
g
g
f
f
f
f
f
2
1
2
1
2
1
,,
nDfg
деградации ллинейнооператорD
шумn
кадр йнаблюдаемы g
кадрисходный
f
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
4040
Regularized Iterative RestorationПространственная регуляризация
2222)( lBVlBHlVBlHBs fQfQfQfQfE
22
22
otherwise0
1-width/81,2,...,lfor 8liif),1(),(),(
otherwise0
1-width/81,2,...,lfor 8liif),1(),(),(
fQfQQ
fQfQQ
jifjifjifQ
jifjifjifQ
BVBHB
VBHBB
BH
HB
Compressed Video Enhancement using Regularized Iterative Restoration Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
4141
Regularized Iterative RestorationВременная регуляризация
),( пикселя движения вектор -),(,
),(,),(),(
),(),(
1 1
2),(),(
jijinm
jinjmifjifffMFD
jiji
width
i
height
j
jiji
kllk
2
)(
),()(
MC
lllt
lk
lklt
fffE
ffMFDfE
Compressed Video Enhancement using Regularized Iterative Restoration Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
4242
Regularized Iterative RestorationРешение
Цель
Решение
k
i
MC
iitB
T
BSB
T
BS
k
i
k
i fffQQQQIgff 21
1
)()()(2
fEfEgffJ ttss
Compressed Video Enhancement using Regularized Iterative Restoration Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
4343
Regularized Iterative RestorationМногокадровое восстановление
)~
()~
()~
(
,...,,~
1
2
21
fEfEgffJ
ffff
ttss
L
l
ll
TT
L
TT
L
l
MC
llt
L
l
lBVlBHlVBlHBs
fffE
fQfQfQfQfE
1
2
1
22
2
22
1
)~
(
)~
(
Compressed Video Enhancement using Regularized Iterative Restoration Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
4444
Regularized Iterative RestorationМногокадровое восстановление
2222 ~~~~~~)
~(
21
MC
tBSBS fffQfQgffJ
kff
MC
t
k
B
T
BSB
T
BS
kk
f
fffQQQQIgff
~~
1~
~~~~~~
21
Compressed Video Enhancement using Regularized Iterative Restoration Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
4545
Regularized Iterative RestorationПримеры работы
# of
case
1 spatial only
2 16x16 ME
3 4x4 ME
4 dense 4x4 ME
5
bidirectional 4x4
ME
6
dense bidirectional
4x4 ME
Compressed Video Enhancement using Regularized Iterative Restoration Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
4646
Regularized Iterative RestorationПримеры работы
dense 4x4 ME
Compressed Video Enhancement using Regularized Iterative Restoration Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
4747
Regularized Iterative RestorationПримеры работы
dense bidirectional 4x4 ME
Compressed Video Enhancement using Regularized Iterative Restoration Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
4848
Regularized Iterative RestorationОптимизация
t
ttb
bbttb
2
'
inf 2
MC
tBs fffQgffJ 22
)( - цель минимизации
MC
t
ji
Bjis ffbfQbgfbfJji
,
2
,
2*
,),(
- новая цель минимизации
A Coding Artifacts Removal Algorithm Based On Spatial And Temporal Regularization. Susu Yao, Genan Feng, Xiao Lin, Keng Pang Lim and Weisi Lin. 2003.
MC
t
ji
Bs fffQgffJji
,
2
,)(
- дополнительная переменная
идея: добавить вспомогательную переменную, не изменяющую минимум J
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
4949
Regularized Iterative RestorationОптимизация
Алгоритм:
econvergencUntil
fixedbwithbfJf
ffQb
fixedfwithbfJb
repeat
gftionInitializa
nnn
nn
B
n
nnnn
}
,min
:,min
{
:
1*1
1
*1
0
Минимизация может быть выполнена методом градиентного спускаA Coding Artifacts Removal Algorithm Based On Spatial And Temporal Regularization. Susu Yao, Genan Feng, Xiao Lin, Keng Pang Lim and Weisi Lin. 2003.
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
5050
Regularized Iterative RestorationОптимизация
ji
MC
ji
n
jit
ji
n
ji
n
jiB
n
jis
ji
n
ji
n
ji
nn ffbfQbffbfJ,
2
,
1
,
,
1
,
21
,
1
,
,
21
,,
1* ),(
A Coding Artifacts Removal Algorithm Based On Spatial And Temporal Regularization. Susu Yao, Genan Feng, Xiao Lin, Keng Pang Lim and Weisi Lin. 2003.
mc
ji
t
n
jist
n
jitsmc
ji
t
n
jist
t
n
jist
n
ji
n
ji
t
n
jist
n
jisn
ji
t
n
ji
fb
bf
b
b
ffb
bxf
1,1
,
1
,
,1
,
1
,
1,,1
,
1
,
,
1
,
211211
1
2111
1
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
5151
Regularized Iterative RestorationПример работы
A Coding Artifacts Removal Algorithm Based On Spatial And Temporal Regularization. Susu Yao, Genan Feng, Xiao Lin, Keng Pang Lim and Weisi Lin. 2003.
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
52
Regularized Iterative RestorationПример работы
A Coding Artifacts Removal Algorithm Based On Spatial And Temporal Regularization. Susu Yao, Genan Feng, Xiao Lin, Keng Pang Lim and Weisi Lin. 2003.
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
5353
Regularized Iterative RestorationПример работы
A Coding Artifacts Removal Algorithm Based On Spatial And Temporal Regularization. Susu Yao, Genan Feng, Xiao Lin, Keng Pang Lim and Weisi Lin. 2003.
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
54
Regularized Iterative Restoration
Преимущества
Высокое качество восстановления
Недостатки
Очень медленная работа
Сложен в реализации
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
55
Заключение
Рассмотрены методы Adaptive Fuzzy Post-Filtering
DCT Re-application
Support Vector Regression
Modified Mean-Removed Classified Vector Quantization
Регуляризация
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
56
Литература
• A Coding Artifacts Removal Algorithm Based On Spatial And Temporal Regularization. Susu Yao, Genan Feng, Xiao Lin, Keng Pang Lim and Weisi Lin. 2003.
Artifact reduction of compressed color images using modified mean-removed classified vector quantization. Jim Zone-Chang Lai, Yi-Ching Liaw, and Winston Lo. Journal of the Chinese Institute of Engineers, Vol. 27, No. 5, pp. 747-751 (2004)
Compressed Video Enhancement using Regularized Iterative Restoration Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999
Compression Artifact Reduction Using Support Vector Regression. Sanjeev Kumar, Truong Nguyen, Mainak Biswas. ICIP 2006
Adaptive Fuzzy Post-Filtering for Highly Compressed Video. Hao-Song Kong, Yao Nie, Anthony Vetro, Huifang Sun, Kenneth E. Barner. ICIP 2004.
Enhancement of JPEG-Compressed Images by Re-application of JPEG. Aria Nosratinia. Department of Electrical Engineering, University of Texas at Dallas, Richardson. 2002.
A Tutorial on Support Vector Regression. Alex J. Smola† and Bernhard Sch¨olkop. 2003
Application Of The Motion Vector Constraint To The Regularized Enhancement Of Compressed Video. C. Andrew Segall And Aggelos K. Katsaggelos. 2001
CS MSU Graphics & Media Lab (Video Group)
Only for Maxus
5757
Вопросы
?
57