Upload
iminds
View
447
Download
1
Embed Size (px)
DESCRIPTION
Citation preview
Future MEDIA
2
Overview
What is this about?
Future media - evolutions and trends
Some key trends
Resulting research challenges / plans
Achievements so far
a lot of projects: please see IBBT website...
spin-off companies & patents
other
3
Future media – evolutions and trends
Approaching the ‘Zettabyte Era’
by 2012, annual global IP traffic will exceed half a
zettabyte, of which 90% will be video traffic
mobile data traffic will double each year
Super Hi-Vision or UDTV (Ultra High Definition Video)
7680 4320 pixels
Japan plans to deploy
UDTV broadcasting in 2015
requires ca. 600 Mbit/s
with current technologies
4
Future media – evolutions and trends
Emerging high-definition user-generated content
multiple versions of online content (e.g., YouTube)
soon to be consumed on high-resolution displays
need for on-the-fly adaptation
Managing huge content collections
automatic detection of concepts
interactive semantic search
multi-modal analysis and
retrieval algorithms
multimedia exploration systems
5
Future media – evolutions and trends
Recognition of many object classes,
scalable in the number of classes
(both for training and at run-time)
Limit the need for supervision
6
Future media – evolutions and trends
Exploit context information
(scene, other objects, geometry, …)
Couple textual descriptions to image content (nouns-
objects, adjectives-attributes, verbs-actions) and exploit text to provide contextual information
Future media – evolutions and trends
Distributed smart camera systems
multi-camera analysis, embedded and distributed
processing
applications connect to general purpose video analysis
services
applications concentrate on high-level aspects; camera
system handles low-level and access control
privacy issues
8
Future media – evolutions and trends
In the brain regions always connect bi-directionally,
in computer vision recognition all too often is
the endpoint of a one-way bottom-up pipeline.
Recognition has a pivotal role to play in terms of motion
analysis, 3D reconstruction, inpainting, ...
We need recognition to talk to all kinds of
complementary visual processes.
We even contend that a lot of progress
in these other areas could actually come from
recognition, rather than from further refinements along
the current lines of thinking
9
Future media – evolutions and trends
Applications based on recognition
automatic resizing
3D city reconstruction
traffic safety
content-based image retrieval
10
Future media – evolutions and trends
Applications based on recognition
video summarization
virtual editors
surveillance
…
11
Future media – evolutions and trends
HCI - interactive 3D environments
natural and intuitive interaction
techniques for emerging environments
‘serious games’
(education, therapy, ...)
Video surveillance
cameras are being installed virtually everywhere
video analysis as security sensor
intuitive control room applications
combining all event information
12
Future media – evolutions and trends
Mobile and context-sensitive interactive systems
13
Future media – evolutions and trends
Multi-modal interaction in 3D and virtual environments
14
Future media – evolutions and trends
Interactive workspaces
15
Future media – evolutions and trends
User-centered software engineering
16
Future media – evolutions and trends
Computer graphics and computer animation
modeling, rendering and animation
video-based CG and animation
stylized drawn animation
volume rendering and information visualisation
Image based modeling and rendering
for real world objects/subjects/scenes
Physically based illumination simulation for virtual
prototyping Real-time photo-realistic hair
and skin for games and movie special effects
17
In conclusion: some KEY TRENDS
Zillions of data
need for efficient indexing mechanisms
Different platforms
content reformatting
User-generated content, interactivity
semantic-level interaction with video content
multimodal data and multimodal interaction
Uncontrolled environments
increased robustness
better generalization
18
In conclusion: some KEY TRENDS
Towards cognitive image understanding
increase number of categories
reduce level of supervision
integrate cognitive information
in other image processing tasks
global reasoning: relations, context, ...
modular systems
"In 10 years we will have the first systems that can
interpret a never seen photo or video and that can be put into an unknown environment and react intelligently"
19
Resulting research challenges
Advanced video coding
coding techniques for very high resolutions
e.g., characteristics of 16x16 pixels
increasing coding efficiency without increasing
computational complexity (too much)
360p 720p 2360p
20
Resulting research challenges
New coding paradigm:
Distributed Video Coding
mobile video applications
multi-view video entertainment
visual sensor networks
wireless video cameras
wireless low-power surveillance
21
Resulting research challenges
Scalable graphics
MESHGRID
22
Resulting research challenges
Scalable graphics
wavelet-based approach
23
Resulting research challenges
Scalable graphics
move towards real-world applications
design, architecture
education, entertainment
cartography
nedicine, modeling
virtual reality
24
Resulting research challenges
Long-term goal
adaptation and delivery framework for personalized
and immersive interactive multimedia experiences
super HD / 3D
sensory effects
Immersive Multimedia Experience
(“rollercoaster experience”)
Annotation
Retrieval
Universal Multimedia
Experience
Adaptation
Delivery
Universal Multimedia
Access
Resulting research challenges
Content adaptation and transcoding
HDTV
originally acquired
high definition material
SDTV
legacy applications
common pan/scan
editing
(AVID)
Mobile (iPod)
semantically repurposed
original (4x) SR (4x)
Resulting research challenges
Transform low-resolution rubbish into high-resolution
content (first steps…)
27
Resulting research challenges
Multimedia data analysis and feature extraction
texture detection and classification
content-based multimedia
information retrieval
multi-camera video surveillance
object detection and tracking
immersive control rooms
Future media – evolutions and trends
Camera selection:
Making sense of the
information overload
Future media – evolutions and trends
Stitching: making sense of it
captured data
30
Resulting research challenges
Metadata technology and exploitation
semantic web technology
W3C activities on media
annotation
Open Linked Data
linking external
domain knowledge
enable reasoning
31
Resulting research challenges
Metadata technology for media production
metadata is much more than ‘data about media objects’
pursue model-driven product development
better communication: between staff and machines
during acquisition (associating audiovisual material)
for editing and repurposing (automate production tasks)
Computer Graphics
+ Scene Editing and Interaction
Add / remove / clone objects
Change shape / appearance / lighting
Animation
Interaction / Navigation
- Synthetic 3D models
realistic modelling is a painstaking task
Video/film
+ Ease of capture and visualisation
Film with camera
Project on screen
+ Film “language” well understood
- Scene editing difficult, no navigation/interaction as in
CG.
Resulting research challenges
Video-Based Computer Graphics bridges the gap (multi-view video as shape & appearance primitive in CG)
Resulting research challenges
Architectures supporting massive multi-user, multi-platform scalable
networked multimedia (virtual communities, gaming, networked virtual
environments, …)
NVE server(s) IP
34
Resulting research challenges
Game technology
efficient
production tools
(e.g., digital sculpting)
real-time interaction with environment (physics)
Resulting research challenges
Application domains: virtual communities, gaming,
networked virtual environments, …
Research topics:
scalability issues
scalability assessment
Resulting research challenges
Network intelligence
Error resilience
Error concealment
Resulting research challenges
Media-aware
security and authentication
watermarking
fingerprinting
content authentication
Applications
control of quality, legitimacy and authenticity of content
disclosure of digital archives
content distribution control
art authentication and dating
37
38
Achievements - a lot of projects...
HI-MASQUERADE
MUTABLE
IFIP
POKUMON
TELEON
GEISHA
PECMAN
COCOMEDIA
VIDEO Q-SAC
ISYSS
VICATS...
39
Achievements - spin-off companies
Androme
Voice and video over IP solutions
iDTV
computer graphics&animation
networked virtual environments
aQuartic
media streaming environment
metadata handling and exploitation
semantic web technologies (RDF, OWL)
content adaptation software platform (‘NinSuna’)
40
Achievements - spin-off companies
Eyetronics
systems & services for 3D scanning
high quality
flexible & efficient scanning
main applications: gaming & film production
GeoAutomation
3D acquisition of structural information in cities
based on mobile mapping (vehicle & cameras)
main application: digital surveying
41
Achievements - spin-off companies
QuESD
sports and video analysis systems
access control systems, including
embedded devices and video streaming solutions
eSaturnus
digital image acquisition & rendering
main applications: supporting surgery & endoscopy
42
Achievements - patents
Patents
active patent policy
while NOT excluding open-source approach!
some numbers:
5 granted
3 pending
43
Achievements - other
Strong participation in standardization
still image coding: JPEG
video coding: MPEG / VCEG / VQEG
semantic web: W3C
A lot of contract research
bilateral
various projects (EU, IWT, ...)
Strong collaboration with a lot of
recognized research partners
Contact info
IBBT Offices
Zuiderpoort Office Park
Gaston Crommenlaan 8 (bus 102)
B-9050 Gent-Ledeberg
Belgium
t: +32 9 331 48 00
f: +32 9 331 48 05
Or even better: just talk to one of us during iMinds
44