Upload
chris-bennett
View
302
Download
0
Tags:
Embed Size (px)
DESCRIPTION
This dissertation explores the recent technological advances in live television replays. Two companies – Replay Technology in America and national broadcaster NHK in Japan - are trialling systems that create slow motion replays in a style reminiscent of the Bullet Time sequences used in the 1999 movie, The Matrix. These virtual replays are set to enter the home television market in the near future and will offer the user a more unique viewing experience. Documenting the history of Bullet Time and sports instant replays and utilising interviews with industry professionals, alongside secondary research, shows the development of these systems, and allows this study to conclude that Bullet Time is still very much in use, but has evolved from its original form. This study also demonstrates that the technologies are still changing to meet the demands of a new generation of users, who grow ever more sophisticated in their home viewing expectations. Recommendations for a successful implementation to the home market have also been made.
Citation preview
University Of Portsmouth
School of Creative Technologies
Final year Project undertaken in partial fulfilment of the requirements for the BSc
(Honours) in Entertainment Technology
Virtual Replay in television and broadcasting: are broadcasters reusing old technology?
by
Chris Bennett
619311
Supervisor: Gary Bown
Project Unit: CT6CTPRO
March 2014
Project Type: Study
Abstract:
This dissertation explores the recent technological advances in live television replays. Two
companies – Replay Technology in America and national broadcaster NHK in Japan - are trialling
systems that create slow motion replays in a style reminiscent of the Bullet Time sequences
used in the 1999 movie, The Matrix. These virtual replays are set to enter the home television
market in the near future and will offer the user a more unique viewing experience.
Documenting the history of Bullet Time and sports instant replays and utilising interviews with
industry professionals, alongside secondary research, shows the development of these systems,
and allows this study to conclude that Bullet Time is still very much in use, but has evolved from
its original form. This study also demonstrates that the technologies are still changing to meet
the demands of a new generation of users, who grow ever more sophisticated in their home
viewing expectations. Recommendations for a successful implementation to the home market
have also been made.
1
I. Acknowledgements
I would like to thank
Farley Brian, Tim Kilgour, Diego Prilusky, Oren Haimovitch Yogev, Nick Trumble, Joe Leveson,
George Cook, Gary Bown, Kensuke Ikeya, Don Kennedy, Yuichi Iwadate and James Mackie
for their contributions to this study.
I would also like to thank my parents, David and Julie Bennett for their support.
2
II. Table of Contents Page Number
I. Acknowledgements 2
II. Table of Contents 3
1.0 Introduction 4
1.1 What is Bullet Time? 4
1.1.1 Classing the Visual Effect 7
1.2 Scope and Objectives 8
1.3 Hypothesis 8
2.0 Literature Review 9
2.1 History of Bullet Time 9
2.1.1 The First Uses of Time Slice as a Visual Effect 11
2.1.2 Similar Techniques to Time Slice 12
2.2 From Time Slice to Bullet Time: the 20th Century and a move to Virtual Cinematography
13
2.3 The Significance of Time Slice within the Slow Motion Field 14
2.3.1 Moving Forward: the next step into Live Television 18
2.3.2 The Importance of The Matrix 19
2.3.3 The Demise of Time Slice 20
2.4 Next Generation Time Slice: 3D Modelling 21
2.5 Marketing the Virtual Replay 23
2.5.1 Development for Commercial Enterprise 24
3.0 Methodology - Sources and Methods 27
3.1 Authenticating and Sourcing Information 27
3.2 Planning the Project 29
4.0 Findings 33
5.0 Conclusions 36
5.1 Recommendations 37
6.0 Evaluation 38
7.0 Reference List 40
7.1 List of Figures 48
8.0 Appendices 49
8.1 Email Communications 49
8.2 Interview Extracts 57
3
1.0 Introduction
The first live television bullet time replay was created in 2001; the 'EyeVision' system (Carnegie
Mellon goes to the Super Bowl, 2001, para 2) used more than thirty cameras to create a Bullet
Time replay of highlights within Superbowl XXXV. Over a decade later, companies in America
and Japan are releasing systems for live broadcast that are very similar to ‘EyeVision’. Will this
cause an increase in television manufacturing and on-demand service sales, similar to those
experienced with the introduction of High Definition television? Similar initiatives, such as 3D
stereoscopic television, have seen a fall in popularity (Anthony, 2013, para 2), and there is a
danger that the introduction of new slow motion Replay Technology will be nothing more than
a novelty, grabbing the attention of the user for a short period, before becoming an expected
standard to existing on-demand service packs.
The complex nature of these systems means there are few companies that manufacture live
television Bullet Time replay systems, and due to the commercial sensitivity of the work
involved, these companies do not tend to share information. There are also few official
publications or research papers available in this area. Therefore, research has been conducted
in the form of interviews, which have provided first hand insight into the industry that both
creates and utilises these systems.
1.1 What is Bullet Time?
This dissertation will look at the development of a video camera technique known as Bullet
Time, also commonly known as frozen time, flow motion or time slice (Argy, 2001, para 1). The
effect relies upon one simple technique - using multiple cameras to create a smooth, slow
motion replay of an object. Figure 1 shows a simple break down of a subject and the cameras.
As each camera takes a shot in sequential order (1 -7 in this example) the perspective of the
viewpoint changes, giving an alternating view of the subject. Cutting from one camera to the
next in a split second creates a dynamic movement of this viewpoint, and adjusting the interval
of time that each camera takes a shot, allows the filmmaker to change the speed of the
sequence. A smaller interval time creates the slow motion effect now widely known as Bullet
Time.
4
Figure 1. A simple example detailing the arrangement of cameras in a bullet time setup.
This differs from a typical camera setup where a camera will record movement of a subject
from one fixed viewpoint. Bullet Time rigs are more advanced and allow for direct manipulation
of both time and space within the same sequence and the end result is a fluid 'pan' around a
slow moving or stationary subject. A pan is a basic video move and can be described as the
"sweeping movement of a camera across a scene," (Rouse, 2007, para 1). The audience would
be treated to a full view around the object as opposed to the traditional fixed plane view. Pans
are limited by the speed at which a single camera can be moved, whereas Bullet Time has many
cameras already in the desired position which gives the director a way to slow down or 'stop'
5
time, creating a continuous fluid shot tracking around any given subject, and tying the footage
together into one seamless dynamic shot.
As the Bullet Time effect only features this slow motion panning view it is not always necessary
to use a bank of cameras synchronised together. There are other ways in which the Bullet Time
effect can be reproduced; Rober (Rober, 2013) re-created the concept in the manner shown in
figures 2 and 3. He moved one camera around the image at a high speed, as opposed to the
image moving between the multiple static cameras. He created a small setup that used a ceiling
fan motor to spin a single camera and its background around a stationary subject. When played
back, the background and camera appear to be fixed, and it is the subject that gives the illusion
of movement. This technique is inexpensive and replicates the effects of Bullet Time although
the effect is very rough and limited in its outcome, only able to capture footage of small
subjects that would fit between the background and camera.
Figure 2. Screenshots from Rober's video, showing the spinning motion of the setup.
6
Figure 3. A simple illustration detailing Rober's experiment in more detail.
1.1.1 Classing the Visual Effect
The Bullet Time style can be broken down and sub divided. There is the original format - an
analogue camera system that utilises a bank of cameras to shoot a subject, which is known as
Time Slice (Time Slice, n.d., para 1). The frames are then stitched together creating a short,
rough sequence of a subject. However, as no cameras are exactly alike, both digital sensors and
film cameras produce slightly differing colours and quality even if they are exactly the same
make, and these discrepancies are exaggerated the longer the sequence. Further problems are
caused by the gap between each camera when mounted on a rig, as due to their size, cameras
may be positioned a few centimetres from one another and are difficult to align perfectly.
When the frames are added together, there is a noticeable jump from one frame to the next;
this look can be utilised to create stylistic art films but is not regarded as suitable for large
budget productions due to its rough nature. In order to maintain continuity throughout this
dissertation, this analogue technique shall be referred to as 'Time Slice.'
The second format Bullet Time, uses the traditional camera setup of Time Slice but incorporates
computer graphics and modelling to create a smoother, more polished sequence.
Unfortunately, the added integration of computer graphics significantly increases both the
7
overall project cost and production time, limiting the technique to large budget companies.
This blend of Time Slice and computer graphics was first used in the Warner Brothers Matrix
films (Wachowski, Wachowski, 1999) and the term Bullet Time became an instant success.
From this point on, Bullet Time will only be used in reference to the Warner Brothers, Matrix
films technique, although many sources and quotes will refer to it more generally.
1.2 Scope and Objectives
This dissertation will focus on live television and the future of Bullet Time within the
entertainment industry. Two companies are currently making interesting breakthroughs with
instant replays incorporating Bullet Time techniques: American company Replay Technology
with their freeD system (Technology, n.d.), and Japan’s NHK (Nippon Hōsō Kyōkai, otherwise
known as the Japanese Broadcasting Company) who have released a multi camera system
(Press release, 2013, para 1). Both developments have come to fruition within the last twelve
months and are currently being trialled in live television sports scenarios. This dissertation will
explore and analyse the technologies that are in use today, showing how their development has
been heavily influenced by historic technologies and novel viewing experiences.
1.3 Hypothesis
'Manufacturers are looking for specific technologies to promote; older techniques such as
Bullet Time would be an appropriate technology for television and with further development
could see an integration into the next generation of television sets.'
8
2.0 Literature Review
2.1 History of Bullet Time
Although not known by this term, the first true experiment with Time Slice was instigated by
Eadweard Muybridge in 1878 with his study 'The Horse in Motion' (Horse in Motion, n.d.). As
the human eye cannot perceive fast action events and analyse the smallest of details, a popular
debate at the time was whether all four hooves left the ground when a horse was galloping.
Muybridge used twelve cameras and an elaborate series of electrical mechanisms to trigger the
shutters in sequence, with thousandths of a second intervals between each. An image taken
from Muybridge's sequence can be seen in Figure 4.
Figure 4. Muybridge's first Horse in Motion work.
Fortunately the set of cameras worked and the importance of the achievement was instantly
realised, reported the next day in an article that described Muybridge’s discovery as:
Second only, among the marvels of the age, to the wonderful discoveries of the telephone and phonograph (How a trotting horse, 1878, p. 1).
9
Muybridge's high speed photographs demonstrated a new actuality: photography could represent aspects of the world in motion that were beyond the capacity of human visible perception (Braun, 2012, p. 143.).
Muybridge changed perceptions of reality with these series of twelve photographs, capturing
time in exposures thousandths of a second long. Time could now be recorded and manipulated
to describe events, though at first only in the form of a simple moving image. In 1879 he
invented the Zoopraxiscope (Motion picture pioneer, 2011, para 5), a spinning disk with small
animated images which gave the illusion of movement. It was the first time people could see
animations come to 'life' and paved the way for greater developments that would eventually
lead to the start of basic motion pictures. In 1888, in America to give lectures on 'animals in
motion,' Muybridge met Thomas Edison and the two decided to work together. This event was
the inspiration for Edison to improve on the system developed by Muybridge, inventing what
has become the modern day movie in the form of a filmstrip. (Eadweard Muybridge, n.d., para
4) Muybridge's initial experiments were effectively the birthplace of the motion picture, and his
meeting with Edison, a significant milestone in its development. It was a natural progression to
develop the method and create the movie pictures of today.
In 1940, another form of camera technique was developed - super slow motion. Although it was
not a direct form of Time Slice, it did help to develop high speed photography and motion
cameras. Harold Edgerton (1903 - 1990), an esteemed professor and engineer, utilised many
different techniques to slow down time. He even created slow motion footage of bullets using
Xenon strobes to flash, capturing the bullet in flight. This technique worked well, as the flash of
a strobe exposes a piece of film creating a snapshot of the subject, as opposed to a blurry
exposure (Harold Edgerton, 1999, para 3).
This era marked an important step; the possibility to use Time Slice to slow down and analyse
what was really happening in the everyday world, rather than making inexact visual
observations. Both Muybridge's and Edgerton's work was important as they used new
techniques to gain an understanding of how things worked around them. Time Slice helped to
prove or disprove many theories of the time and Edgerton's slow motion work was initially used
to conduct scientific experiments on extraordinarily fast objects, such as bullets and explosions.
10
However, it also marked a change in the history of Time Slice and led to the start of visual
effects.
2.1.1 The first uses of Time Slice as a Visual Effect
In 1967, a Japanese anime cartoon called 'Speed Racer' (Yoshida, 1967) was released and the
opening credits featured a style that had previously been unseen in cartoons - two seconds of
the Time Slice effect (Figure 5).
Figure 5. Shots of the Speed Racer Bullet sequence in effect, less than one second in duration.
It was a very rough cut that only featured a few frames panning around a car and the lead
character but this was the first use of Time Slice in the entertainment industry (Rolli, n.d., para
5). This small clip helped to establish the technique in the Japanese comic industry and inspired
future development of the visual effect within the world of film. Traditional animation gave the
look from one fixed point of view, but this short sequence proved that it was possible for this
hypothetical camera to move too.
Another important step was realised in 1980, when Tim Macmillan (Time Slice, n.d.) at the
University of Bath created basic experiments that he would later come to call 'Time Slice.'
Macmillan constructed a 360 degree loop that featured hundreds of pinhole cameras. When
11
the film in each camera was exposed to the light from a flash strobe, it captured a slice of time
(BBC Tomorrows World 1993, 2010). This is the process technically known as 'Time Slice.'
This was the first time cameras would be used to create a 360 degree still shot capturing real
life subjects. At this stage the result was rough, and there was no stability within the shot;
colours and slight movement in each frame limited the initial Time Slice work to the art world,
with video producers unwilling to use an effect that would not suit their films. Although an
important moment for the technique, there was still some development needed, and this came
in the form of 'View Morphing' (Seitz & Dyer, n.d., para 3). Used heavily by Michel Gondry
(Michel Gondry, n.d., para 3), the method uses shots from two cameras distanced slightly apart.
Computer software then generates a predicted sequence of shots of the space between the
two cameras - these added shots are known as 'interpolated' frames. Gondry used this
interpolation method to create a bizarre new effect called 'View-Morphing. It was very popular
and Gondry became well known for his creative work in the entertainment and advertising
industry. Renowned visual effects supervisor, John Gaeta, referenced Gondry's work for his
inspiration on The Matrix film in 1999 (Wachowski, Wachowski, 1999):
For artistic inspiration for bullet time, I would credit... along with director Michel Gondry. His music videos experimented with a different type of technique called view-morphing and it was just part of the beginning of uncovering the creative approaches toward using still cameras for special effects. (200 Things That, 2006, p.136)
All of these experiments and techniques had a profound effect on what would become the
entertainment industry: each was a stepping stone to the next, and without one, the next in
line might not have been developed.
2.1.2 Similar Techniques to Time Slice
There are no theoretical limits to the Time Slice effect, where it is possible to orchestrate any
camera move imaginable in a full 360 degree field, traversing through both time and space.
However, in real world applications there are restrictions, primarily the cost of the system and
the creative vision of the director. The cost of a Time Slice rig increases exponentially with the
length of sequences and the complexity of moves envisioned by the director.
12
There are many cases where the Time Slice effect has been used in production, although
varying methods were used to create a related effect. Television manufacturer Phillips used a
similar technique in 2009 to create an advert called 'Carousel,' used to market a new television
(Berg, 2009). Using a continuous shot, one motion controlled camera on a track made its way
through a hospital complex. Instead of many cameras taking photographs at split second
intervals, trained stuntmen and actors simply held their poses, creating the illusion of paused
time. The two techniques are quite different but do create similar results. However it is still a
very expensive process, utilising cutting edge stabilised camera rigs. It would stand to reason
that only large budget film houses could afford the use of Time Slice, but in recent years there
has been an influx of independent film makers and individuals developing their own ways to
build a rig that mimics the Time Slice technique, thereby bringing costs down. Time Slice has
also featured in hundreds of films, adverts and games, all with differing results. In 2010 the surf
brand Rip Curl filmed a marketing campaign featuring the Time Slice technique, using a mix of
DSLR and small GoPro cameras (Masefield, 2010).
2.2 From Time Slice to Bullet Time: the 20th century and a move to Virtual
Cinematography
Many individuals had experimented with the Time Slice effect but before 1999 the technique
was not well known. It had made a few appearances in television shows and in the cinema such
as Kill and Kill (Hall, 1981) which featured a low budget slow motion sequence, but it was with
the release of the film, The Matrix (Wachowski, Wachowski, 1999) that the new form of Time
Slice - Bullet Time - was cast into the limelight, the effect being heavily used by the marketing
department to advertise the film. The visual effects supervisor, John Gaeta, originally coined the
expression 'Bullet Time,' as sequences featuring bullets held in slow motion were a highlight of
the film. Eight years later the term 'Bullet Time' was registered by Warner Brothers, U.S
Trademark Registration number: 3349252 (2007) and from then on the effect would usually be
known this way. The Warner Brothers method featured a bank of cameras synchronised
together, each camera taking a photo in series, generally with a fraction of a second gap
between each shot. The time gap between each camera in The Matrix film, which used over
13
100 cameras, generated sequences up to 12,000 frames per second (Godoski, 2011). There was
however, a slight problem with the camera positioning - even with the cameras packed as
closely as possible there was a noticeable gap between each frame within the sequence
producing a jumpy, stop motion effect. To overcome this, Michel Gondry's interpolation
method filled the gap between each camera with computer generated shots. These extra
interpolated frames helped to create super smooth, slow motion instead of a jittery rough cut
sequence (Wilkinson, 2010, para 1), essentially using Tim Macmillan's original technique in a
neater and more precise way. This was a new technique and would have been very time
consuming, and therefore only feasible for a large production house with a multimillion dollar
budget to spend on special effects. This was the start of 'Virtual Cinematography,' a move into a
virtual world, where a combination of real and virtual objects would be seamlessly merged. It
incorporated real life actors in front of green screens and the world of 3D modelling to create a
virtual world with real life attributes, people moving and interacting as they would in real life.
2.3 The Significance of Time Slice within the Slow Motion Field
Slow motion is the direct relative of Time Slice, with the two techniques arguably created within
the same 19th Century Muybridge experiment. Although very similar in nature, with both
providing a replayed sequence that manipulates time, there is one key difference; with Slow
Motion the movement of the camera is generally limited to a fixed plane of view, whilst Time
Slice sequences are free to move in any direction and orientation imaginable.
In 1878, it was common belief that an animal would always have "one hoof in contact with the
ground" (Padgett, n.d., para 1). With the multi camera setup Muybridge had derived, it was
obvious that this was not the case, as a still shot provided proof that at full gallop a horse would
be completely off the ground. As well as disproving a common theory, the technique was
important because it slowed down the modern world and allowed people to see the intricate
movements of a fast moving object. Muybridge's work "ushered in a new kind of photography,
one which captured the invisible and showed us what we cannot see with the naked eye"
(Edwards, 2010, para 1). His work altered the perception of reality, as he had essentially
created the first slow motion replay - a significant moment in history because, from this point
14
on, people would be able to see subjects moving far more slowly, allowing the human eye to
see the previously unseen. Scientific observations could now be made on the course of a
moving object. It is a fair assessment to conclude that in the late 19th century the Muybridge
experiment came out of necessity and a functional need to slow down the world led to a
breakthrough.
This work continued, with scientific experiments throughout the 1900s, with notable work in
the military. In 1950, the Magazine 'Popular Mechanics' featured an extract detailing the
American military's slow motion, explosive video testing. Their camera was able to create
photographs at a rate of "100 Million a second" (Windsor, 1950, p. 158). There was not much
more information retrievable from the extract but it does provide another insight into the
capabilities of slow motion and its ability to study the world around it. Cameras have continued
to progress in the last century and large digital sensors can record, in vast resolutions, the
minute details of fast action subjects. These developments have formed important
breakthroughs with life-saving applications; for example, the automotive industry uses slow
motion replays extensively, conducting crash tests which undoubtedly help to drive
improvements - "Video was shot in high definition super-slow motion, allowing us to see every
detail of a vehicle's passive safety mechanisms as they are employed" (High-Def, 2013, para 3).
Referring to the system devices such as air bags in cars, this source gives added credit to the
significance of Muybridge's work, as it would be very difficult to monitor these automotive
systems without his technique. Slow motion has become a very important technology and is
now regularly used, in every possible setting imaginable, from the entertainment industry to
scientific recording. Slow motion has improved our understanding of the modern world.
During the 1950s and throughout the development of slow motion cameras and practices,
there was a change in principles. Similar to the theories of modern Architecture, when ideology
changed with the idea that 'Form follows Function,' a term coined by Louis Sullivan (1896, p.
408), the entertainment industry started to use Slow Motion for visual effects. It appeared as a
stunning aesthetic form on camera and the key moments of a film could be elongated which
would help build suspense. There are many unsupported articles of information that try to
15
place the origins of Slow Motion visual effects, but according to Stephen Prince the short, slow
motion scene featured in the 1954 film, The Seven Samurai (Kurosawa, 1954), "Decisively
changed modern cinema" (1991, p.348); a few seconds of slow motion occur when a warrior is
murdered and his body falls slowly to the ground. Modern television and filmmaking has
adopted the slow motion visual effect and it is frequently shown on a variety of different media
sources, from the high tension boxing sequences in Raging Bull (Scorsese, 1980), to the
complex action scenes of 300 (Snyder, 2006). Live sports games frequently use slow motion to
replay highlights that are significant to the game. The slow motion draws out the length of the
sequence allowing the audience to see what is really happening in more detail. The internet
provides an example of how slow motion video is adored by its audience, with websites and
online blogs setup to demonstrate amazing forms displayed at a fraction of their real time
speed. It can be seen that Slow Motion is a powerful visual effect that will continue to be used
in the film and television industries:
From a transhumanist perspective, slow motion videography is a technology that allows us to extend our senses beyond their natural capacities. It allows the revelation of data hidden in the folds of time, just as a microscope allows us to appreciate the wonders of a fly's wing or a microbe's choreography. (Eagleman, 2009, para 5)
David Eagleman, a neuroscientist and writer with a background in the biological studies of
visual illusions and time perception (Time Perception, n.d.), provides a credible reason for the
admiration Slow Motion video receives, showing why it has maintained its popularity and
continued use. Analysing the features of the modern world is the key to the popularity of Slow
Motion; providing a dissection of the smallest details gives a hidden view that a normal eye
would never be able to see.
There is a great significance to slowing down time in motion pictures and it can be used to
achieve a variety of effects. As Jones (2005, p. 149) points out, “varying frame rate and shutter
speed in the camera can affect the apparent speed - and enhance the excitement - of an action
scene,” which is something that directors may want to ultimately control and manipulate.
Sanders (2014, para 5) agrees with this theory when he states that:
16
“Slow motion can also be used to raise the dramatic tension in a scene. If it’s played back at slow motion, you keep the audience at the edge of their seats longer, as they crave to know what happens next".
Directors rely on tension to keep the viewers interested - it helps to create an interesting film or
sequence.
The Time Slice effect used in today's industry is a very close relation of Slow Motion, with the
added dynamic movements of Time Slice giving it the edge in the entertainment industry. The
following extract suggests the importance of slowing down sequences to draw out tension in a
scene or to reveal something visually obscure:
Camera movement is relatively restricted, if not completely static, during conventional slowmotion sequences. In bullet time, on the other hand, it can be accelerated considerably. The description of the technique should perhaps be completed by adding that bullet time is frequently employed in order to visualize processes (e.g. a figure dodging bullets) that would be hardly visible, if at all, in real time. (Kindt & Muller, 2003, p.37).
This quote exemplifies the importance of Bullet Time, as it highlights the reason Time Slice is
significant within the slow motion criteria. It suggests that slow motion is necessary to properly
view these high speed scenes that would normally be missed. Kindt and Muller also provide a
notion that Time Slice techniques help to create the dynamic sequence flow that regular slow
motion would not be able to generate, when they refer to typical slow motion as 'restricted'
and 'static'. This additional movement of the camera is important because where a traditional
camera would be fixed in place, the audience is now provided a more detailed view. The pan of
the Time Slice camera reveals far more of the scene than other traditional techniques,
enhancing the story and providing an arguably more entertaining shot for the audience.
With the two technologies - Time Slice and Slow Motion - so closely linked, it is crucial to bear in
mind the fundamental differences. With both techniques developing together from essentially
the same source, each has its own specialised and applicable field in the modern era. Slow
motion is very functional, it has a wide range of uses but is not as stimulating as the more
visually appealing sequences generated by Time Slice.
17
2.3.1 Moving Forward: the next step into Live Television
As the visual effects industry moved forward so did the world of television. Live action instant
replays were developed in 1955 and helped to change the live television sports industry
(Saunders, 2013, para 16). Instant replay was devised to re-show a significant moment,
allowing the audience to go back in time and review the importance of something that had just
happened. The first trial use of this technique was December 7th, 1963:
Tony Verna, a hotshot 29-year-old director for CBS Sports, brought a 1,200-pound videotape machine to Philadelphia's Municipal Stadium. After several failed attempts early in the game, Verna was finally able to get the machine to work... However, the black-and-white replay was indistinguishable from live action, and there were no on-screen graphics informing fans what they were seeing. (Saunders, 2013, para 10).
Despite this faltering start, instant replays have become an everyday technique used heavily
within the sports broadcasting market. It was very significant and allowed the audience to re-
evaluate something that may have happened too quickly for the eye to properly review. It was
the first time the audience at home could start to make up their own minds and see what was
really happening.
It was not for another 38 years that the instant replay and Time Slice methods were brought
together - at the 2001 Superbowl XXXV (Carnegie Mellon goes to the Super Bowl, 2001, para 3).
Inventor Takeo Kanade devised a setup using more than thirty cameras, creating a rough but
functional system, coined by CBS (Columbia Broadcasting System) as 'EyeVision' (Carnegie
Mellon goes to the Super Bowl, 2001, para 2). A selection of shots shown in the broadcast can
be seen in figure 6.
18
Figure 6. A selection of shots demonstrating the Bullet Time sequence of 'EyeVision,' lasting less
than a second in duration.
The system had taken inspiration from The Matrix film (Wachowski, Wachowski, 1999) and a
research budget of approximately 2.5 million dollars (Martzke, 2001, para 9) was put aside to
trial a live version. When analysing live television from a theoretical perspective it would seem
that incorporating Time Slice instant replays would begin to create a logistical nightmare,
realistically requiring a large amount of computing power and co-ordination. The original large
multi camera rigs first developed by companies such as Warner Bros are not practical for live
broadcast replays. These rigs were huge, fixed in place and focused to one small subject area;
to equip a stadium with this kind of coverage would be a hugely expensive and very laborious
process. Although the 'EyeVision' system worked well it did not re-appear, as the cost and set
up time prohibited further expansion. It was another decade before Time Slice would reappear
on live television in any form.
2.3.2 The Importance of The Matrix
The first Matrix film (Wachowski, Wachowski, 1999) was important because it was the first time
special effects were used to further the narrative of the film as opposed to just being a great
effect in itself - something that was rarely achieved previously (Gills, 2005, p114). The films
19
premise is that the world around is not all that it seems and seeks to help the characters open
their eyes to the reality of the situation. According to Lisa Purse:
Bullet-time is, in its very dynamics, an explicit expression of the need to see everything, to see the whole. It is an expression both of the film’s mastery over the visual – its ability to show everything – but also the spectator’s mastery of the visual – his or her ability to see everything. (Geoff, K, 2005, P159)
Purse highlights the relationship between the narrative and effect. This is significant because
the narrative and effect would merge; the audience would not just see a stunning visual effect
but the flow of the narrative would be maintained. The effect had a serious impact on the
special effects industry and actually helped to sell the film and its subsequent sequels, not just
create an appealing on-screen effect. Farley Brian, a freelance special effects producer,
highlights how new effects can really influence the popularity of films:
In regards to The Matrix, the story of the techniques used actually sold the film. It is possible the film would have sold nowhere near as well without the seed that is Bullet Time. We had a project which didn't take well, only hitting a small audience, but it had new technology which the media picked up on and it marketed itself! (F.Brian, personal communication, November 21, 2013).
With any major new technology there is an associated hype. If the effect is very noticeable, the
audience may endeavour to find out more - how was it created etc, little details that enhance
their understanding and satisfy their curiosity. The associated press will typically help to market
a film, writing reviews and articles that explain more about what is really happening.
2.3.3 The Demise of Time Slice
Almost as suddenly as it sprang on the public scene, however, bullet time seemed to wear out its welcome... Put to scullery work as a sportcasting aid in the 2001 CBS Superbowl, parodied in Scary Movie (2000), Shrek (2001), and The Simpsons, bullet time became first a cliché, then a joke. (Rehak, 2007, p28).
The popularity of Time Slice took off between 1999 and 2003, featuring in practically every
media field possible. Cartoons, games, advertisements and television all rushed to feature it,
and with its overnight success came its quick decline. According to John G. Cawelti:
20
One can almost make out a life cycle characteristic of genres as they move from an initial period of articulation and discovery, through a phase of conscious self-awareness on the part of both creators and audiences, to a time when the generic patterns have become so well-known that people become tired of their predictability. (Rehak, 2007, p29)
When an effect becomes so widely used and predictable it ceases to be 'special.' With few
directors wanting to use it again, Time Slice quietly 'died', in a similar fashion to the overused
'Morph' effect used in the early 1990s (Rehak, 2007, P29):
Morphing — digitally transforming one image into another — was a flashy, obvious and very popular technique. But when it began to seem dated, no one wanted to feature the effect. In reality, morphing never went away; it just was used more subtly. “We now use morph as a way to smooth out elements, but no one’s aware of it. (Argy, 2001, para 5).
The creators of 'EyeVision' knew that it was important to not overdo the effect: "We want to
use it when it makes sense, not cram it in or overuse it." (Martzke, 2001, para 12). Even with
this limited playback it was clear that Time Slice had to succumb to its own success. Its overuse
and constant appearances in adverts and cartoons such as 'The Simpsons' saw a huge drop in
popularity, with no-one looking to employ a technique that had saturated in the broadcast and
entertainment field.
2.4 Next Generation Time Slice: 3D Modelling
Point cloud systems are an example of how Time Slice and 3D modelling can work together in
unison. Typically using a 3D scanner to analyse a large number of points on the surface of an
object, the relative distance of the points from the scanner allows for a precise map of the
object to be established. This technology is having a profound effect in the many different fields
in which it is employed, allowing designers to quickly map an object or place with high
precision. Time Slice can be used alongside this point cloud method, with multiple cameras
providing a 360 view of a subject. Compositing the images with point cloud software generates
a fully three dimensional model of the subject. Tim Kilgour believes Time Slice and Point Cloud
generation have become inextricably linked:
21
When used in film production, this system [Point Cloud technique] creates highly detailed composites of a subject which could be modelled quickly; it is less time consuming and costly than a visual artist building a 3D model from photographs.
When asked if the Time Slice method was no longer being utilised:
No I don't think so, we use it in different ways. Its evolved and mixed with other technology (T. Kilgour, personal communication, November 21, 2013).
The implications of this technology are vast, allowing for high precision and fast three
dimensional rendered models. This is an improvement on previous techniques as typical models
would require a long time to process and would be created by a developer from photographs
and measurements. Point clouds have generally been limited to infrared light or laser based
systems which are used to create points of data. An emitter sends out a beam of light which
bounces off an object, is received and subsequently analysed. The measured points are then
modelled giving a detailed three dimensional surface. Replay Technology in America has
developed a very similar method for their 'freeD' system but the company utilises ordinary
camera sensors. Technology officer Matteo Shapira spoke about the system:
We use synchronized feeds of high resolution video. The video then gets processed using our algorithms to create a 3D database of voxels. After the freeD database is created, we had built an interactive real-time rendering engine that allows viewing of the captured scene from any desired angle (Carroll, 2013, para 5).
Multiple cameras are used to map 'voxels' in space and create a full composite (Bringing 'The
Matrix' to the NFL, 2013). A voxel is a measurable unit that defines a point in space; similar to a
2 dimensional pixel, it holds an extra Z co-ordinate which is necessary to translate and map a
point in three dimensional space. (Rouse, 2007). The system is very quick, only taking up to a
minute to create a composite, allowing for use in live television broadcasts. The render can be
panned around or slowed down making for much more interesting replays. According to Oren
Haimovitch-Yogev, CEO of Replay Technology:
You don’t have to have a camera right in front of the scene. So we can have multiple angles and endless choices, and we can run the same scene in different ways. As a result, freeD also offers a unique way for fans to view controversial calls, creating a 3D effect and depth of field on a 2D plane. (The next big thing?, 2013, para 9)
22
This is important because it gives the broadcast operator much more control over the scene
which can be actively manipulated and reviewed. This fully interactive system is very similar to
the playback system of NHK but is more powerful. Having created a smooth slow motion replay
of the scene it harks back to the original Warner Bros (Wachowski, Wachowski, 1999) Bullet
Time rigs. Although the two systems are loosely related, it is obvious how much technology has
helped to improve both systems during the previous decade, with fewer cameras, quicker
processing times and more flexibility. Comparing Takeo Kanades 2001 ‘EyeVision’ (Carnegie
Mellon, 2001) technique against Replay Technology's (Technology, n.d.) work provides a good
way of charting the progress made in the field. ‘EyeVision’ was rough and jumped from cut to
cut, with many cameras slightly out of alignment. However this was new ground for live
television replays so was to be expected from a new technology. Although little has happened
in the last decade that built on the previous success of ‘EyeVision’, without this first trial system
the more promising freeD system may never have come to fruition.
2.5 Marketing the Virtual Replay
Technological trends come and go and those that can make money are heavily marketed. They
are typically trialled in the cinema and, if successful and suitable, are integrated with television
sets. It is worth analysing the recent push by manufacturers and service providers for the re-
introduction for 3D stereoscopic film. This saw the mass production of new stereoscopic movies
and television shows, even a new device for the home - the '3D television'. Marketed as the
future of TV, 3D sets were sold to the public and although at some initial cost, seemed popular.
However, according to Mark Harrison of BBC North, "3D is dead" (Svetlik, 2013, para1). Whilst
live replay technology is visually appealing, is it powerful enough to warrant investment from
television manufacturers and the services industry? These industries are constantly looking to
forward new technology that will encourage home users to replace their existing devices. The
developers at Replay Technology are keen to introduce a freeD system to the home user and it
would seem that freeD may be marketed in a similar fashion to 3D, something that Matteo
Shapira, the Chief Technology Officer of Replay Technology, believes will happen:
23
Our next step in our development pipeline is to allow the home viewers themselves to move around freely within the captured scene. What is now reserved for an event's producers and directors will become the domain of every sports fan, which is allowing full interactive viewing around the scene. (Carroll, 2013, para 8).
This would mark the start of another revolution in television and broadcasting, allowing the
consumer to take control and become fully interactive with their television. Although the re-
introduction of stereoscopic television may not have been as successful as manufacturers
would have hoped, it would seem that freeD has potential. Although Shapira may describe it as
a 'next step' it seems more likely that this is still a great distance away in terms of reality. Even
with the pause and rewind features of today's modern televisions, it seems a stretch that a full
replay system with interactive cameras is just around the corner. However, there are no
obvious drawbacks such as the need for stereoscopic polarising glasses and the headaches
often caused by long viewing times, commonly associated with stereoscopic television use. If
the freeD system could be formulated for market, it is quite likely that it would take off in
popularity.
2.5.1 Development for Commercial Enterprise
A purely revenue based approach suggests that Time Slice models are a good prospective
application for broadcasters and manufacturers. An uptake in this system could quite possibly
create a new era in live television and broadcasting with added interactivity for the user and a
more fulfilling experience all-round. The audience interacting with their content is a popular
21st century idea - merging people and the technology around them. With the invention of the
internet and the ever expanding world of online content, developers have come to realise that
the products they create have to meet a target audience. Known as Interaction Design
(Joshuajnoble, 2011, para 2), it is the process by which designers try to fit the needs of the
people that will be using the product or software. Constant development has seen the rise of
interactive games and applications as the developer attempts to create a complete immersion
of the user. With the recent invention of 'Smart TVs,' the audience can connect to the web and
do more than just watch standard television formats, giving the smart TV a unique selling point
24
(USP). According to Dan Brilot, Media Consulting Director for the market research agency
YouGov:
The ‘smart’ part of a Smart TV is not the main reason many people are buying them at the moment; it’s more about future-proofing their TV set in the same way that lots of people did when they bought HD TVs. (Brilot, 2012, page 2)
Brilot demonstrates a powerful point, that people are merely buying new televisions to future
proof. With consumers not necessarily worrying about the technological aspects, it would seem
that manufacturers simply need to create a series of television sets that would convince a
consumer it was worthwhile investing. These USPs help to sell products and generate added
revenue for manufacturers, who in turn advance the technology and improve the products,
generating new profit streams and revenue, ultimately following the standard product life
cycle developed by Raymond Vernon in 1966 (Vernon, 1966) see figure 7.
Figure 7. The product life cycle devised by Vernon (1966)
25
As markets start to become saturated with a certain product it falls into decline. In the case of
the black and white analogue television, the market had become saturated, there was no
reason for members of the public to replace their current television sets with out of date black
and white TVs and so manufacturers let the technology fall out of popularity. All products and
technologies follow this model, and although manufacturers are able draw out the maturity
phase with upgrades and other promotions, eventually the product will fall into decline and
another will need to be developed to take its place. As soon as a technology comes into
saturation and starts to decline, it is up to the producer to promote and advance another
system, either completely new or a spin off model. "This is an experimental field, a lot of the
experimental guys are out in the cold as no one cares about creativity, it's all done to generate
profit and hit a target audience" (F. Brian, personal communication, November 21, 2013). This is
why Time Slice-style interactive replays for television are a potential revenue stream for
broadcasters. Rarely seen before and able to add extra versatility and interactivity, it is a
sensible conclusion that the system will be adopted. Whether it becomes an add-on piece for
current standard sets or is manufactured as a new television set, remains to be seen.
26
3.0 Methodology - Sources and Methods
3.1 Authenticating and Sourcing Information
This study utilises information from a variety of sources, all of which can be broken down into
two sub classes - primary and secondary research. Both classes are very significant and it is
important to realise both the importance and limitations of each.
Primary research is a form of first hand data collection, from interviews, surveys and
experiments. It is more time consuming and difficult to retrieve, however the researcher
conducting the research is able to identify any problems with errors and is aware of any bias,
something that is not readily published in secondary research reports.
Secondary research is quick and easy to obtain, simply defined as the collection of data from
pre-existing research such as internet sources and books. Although vital to providing
background context, is it much harder to authenticate results and check the credibility of
sources.
Secondary research initially played a large part in this dissertation; readily available on the
internet and from press releases, it provided a good starting point to set the foundations of the
project. Creating a context and assessing each technology in detail provides a rich review that
aids insight into this industry. Historical information was documented in abundance; the
university library is a good source for books and research papers which were complimented by
use of the internet. However, it is important to be aware of inaccuracies and media bias that
exists on the internet; retrieving the original sources of information was necessary in order to
validate any information. Due to the specialised nature of the subject matter, there are only a
few people in the television and film industry who would know about the developments of
virtual replays in any significant detail, which compounded by the commercial sensitivity
surrounding new product developments, meant few publications or research papers exist on
the subject of Time Slice technologies.
The next step was to locate and contact industry professionals in order to obtain sources for
primary research. This was done via networking, contacting companies and through the use of
27
social media such as LinkedIn. Interviewing sources would provide a rich source of data from
which to extrapolate key points. Unfortunately, as the interviewees were located in the UK, US
and Japan, the project was limited to just two ‘face to face‘ interviews. After a number of
months networking, two participants were contacted who were willing to be interviewed. This
initial primary research would create establishing points on which to further explore the studies
hypothesis. The two industry professionals in question were; Farley Brian, a Senior Digital
Producer (F. Brian, personal communication, November 21, 2013) and Tim Kilgour, a Senior
Designer (T. Kilgour. personal communication, November 21, 2013). Both have dealt with bullet
time techniques and were aware of the industry field in great detail. Their previous experience
provided a rich source of primary data that lends credibility to the overall nature of the project.
Further research involved email correspondence with authors and developers from around the
world:
Bob Rehak, professor and chair at Swarthmore College, author of 'The Migration of
Forms: Bullet time as Microgenre'
Joe Leveson, Technical Director for Prime Focus Film
Kensuke Ikeya, developer of the NHK multi camera system
Oren Haimovitch Yogev, CEO of Replay Technology
Diego Prilusky, Creative director of Replay Technology
Don Kennedy, Media Producer DigInfo News.
The formulation of interviews was very important as they provided a further source of
qualitative research. There are two types of research:
qualitative, a way of attempting to interpret the things that people say, and
quantitative, which uses statistical data from questionnaires to quantify meaning.
The open ended questions associated with qualitative research can be used to allow the
interviewee to answer in their own words. "Although time consuming to conduct, qualitative
research tends to offer forth a wealth of varied information on a small case or set of cases over
a broad set of data" (Jasonhopper, 2011, para 4). This is important because the interviewee is
not limited in their response, allowing for a more honest opinion and answer, and occasionally
28
something is said that the interviewer did not think to ask. This is why qualitative research -
based interviews were the best method of obtaining results for this subject. Within a
specialised field it is necessary to ask the people that use the technology how it works and
transforms the world around it. A member of the general public would not usually know how
Bullet Time effects are created and would add little useful insight to the investigation. This
makes surveying the general public redundant with respect to this dissertation. To improve
reliability, time was taken to design each question asked in order to avoid imparting biased
questions and answers:
Respondents give inaccurate answers for a number of different reasons. They give inaccurate answers both consciously for reasons of their own, and also without any conscious realization that the information they are giving is inaccurate. (Brace, 2008, p 195.)
It was important to maintain the credibility of this investigation as, with such limited
information available, it was necessary to objectively assess all sources. A biased or leading
question could have affected the outcome of the investigation very easily. Avoiding sources
from possibly biased websites, such as Wikipedia, helped to eliminate possible unreliable
sources.
Due to the distances involved with certain contributors, some of the interviews were conducted
via email, a medium which does not typically provide a strong source of qualitative feedback.
Very often the responses received are ‘closed’ and only answer the question asked in a very
basic form. However, the email exchanges associated with this dissertation have been both
insightful and beneficial.
3.2 Planning the Project
In order to gain the information necessary to complete this project, it was important to start
planning from an early date. Initially starting in July 2013, after the first articles about NHK's
multi camera work (Press release, 2013, para 1) were released to the media in early June,
tracking down credible sources was key to the project. Derek Swetnam, the author of a popular
book covering dissertations, provided points to bear in mind during planning, suggesting that
29
work should be written up "as soon as the material is ready" (Swetnam, 2000, 3rd ed, p18). This
is important for a project of this type because any research that is left may be forgotten and
excluded. With so many technologies involved it is key to detailing all of their influences and
effects within the industry. Creating a basic structure for the project was undertaken with the
use of Gantt chart, a common way to plan and track the progress of projects: "One of the most
popular and useful ways of showing activities (tasks or events) displayed against time." (What is
a Gantt Chart?, 2012, para 1) Essentially a Gantt chart is a versatile and visual tool to effectively
communicate timescales and deadlines.
Working backwards from two known deadlines – submission and binding dates, the project was
broken into three logical phases:
Research – identifying sources, interviews and analysing feedback
Content – developing the basis of the dissertation, documenting the historic and future
developments
Finishing – completing the conclusion, abstract, references etc. This phase also included
the binding and submission of the paper.
For each defined phase, it was necessary to estimate how long it would take to complete each
sub-component. Although this produced a very rough time schedule, it also provided a visible
representation of the overall project and allowed a method to track project progress. It was
important to stick to deadlines and update the charts when necessary, otherwise it would have
been easy to lose track and fall behind.
30
The first chart (figure 8) was completed in July 2013 and shows an estimated 3 month period
for research, with much of the writing taking place over the subsequent months. However, this
is not an accurate time scale for what really occurred, as much of the research took far longer
than originally planned. Researching a topic of this type with little previous knowledge is very
time consuming; as one technology is revealed there is another one very closely related that
also needs to be researched and understood. This continued development took far longer than
anticipated and even in the last few months there was always new information being gathered.
For this reason it was necessary to create another chart.
Completed in January 2014, the second chart (figure 9) shows how long the research period had
taken even in the last stages of the project. With a due date in March, there was still a lot to
document and improve upon. Contacting various developers in order to gather more specific
details on their technologies took a lot longer than expected. The second chart is more
developed, with additional tasks such as binding and reference checking allocated in order to
more accurately forecast the project deadlines.
32
4. Findings
The face to face interviews with Brian and Kilgour provided strong positive correlation in
support of broadcasters building on existing technologies to develop new products and
services. FreeD and NHK's multi camera setup provides proof that technology is being reused
but as an evolution of old technology rather than the development of a new idea. During the
study's interview stage, when asked if Bullet Time was dying off, Kilgour answered: "No I don't
think so, we use it in different ways. Its evolved and mixed with other tech" (T.Kilgour, personal
communication, November 21, 2013). This was supported further by Brian: "Technologies
follow cycles and can be profoundly hit and miss, it's a bit shocking when you look back and find
these seeds," (F. Brian, personal communication, November 21, 2013). This quote supports the
question 'are broadcasters reusing old technology.' Brian points out that technologies tend to
reappear and may fall in and out of popularity, and goes on to refer to 'seeds' suggesting that
technologies such as Bullet Time will be reused in another form or variation as a launch pad for
something else. Many of the email interviewees went on to say that the technology is not just a
re-emergence but an evolution. Tim Kilgour believes:
The old rigid forms of Bullet Time have been developed; we now use these multi camera methods to create complex 3D models. Technology develops over time and the best pieces are cotton picked and used in new places. (T. Kilgour, personal communication, November 21, 2013).
Kilgour gives further credit to the theory that technology is being reincorporated but goes on to
say that it is developed further, describing it as an "evolution of sorts." The two interviews drew
attention to the evolution of these techniques as opposed to simply reusing the technique
developed from 2001. Both interviews produced credible evidence for a reuse of technology.
At first glance there did appear to be one piece of contradictory evidence; Joe Leveson, who
does not deal directly with live television, asked members of his team for thoughts on the
subject:
I did ask a couple of my supervisors about bullet time and according to them it's not used so much nowadays. They use what is called a Digi Double, which is a fully
33
rigged and textured model of the character. The reason for this is because the levels of texturing has moved forward quite a bit since bullet time became used in the late 90s. It's also cheaper and more freedom when it comes to planning shots. (J. Leveson, personal communication, October 27, 2013)
Although Leveson is right to suggest that stand alone Bullet Time techniques have become less
popular within the industry, there are examples of where it has become integrated with other
methods. Replay Technology and NHK systems are evidence of this but the reference to the
'Digi Double,' a 3D model of a subject, is an indication of the move into virtual cinematography.
Multi camera rigs and 3D modelling are used side by side to create one amalgamation of
coexisting technologies.
All of the evidence collected supports the hypothesis that companies are redeveloping and
refining existing technologies and combining them with other solutions and techniques to make
composite solutions – Point Clouds being a typical example.
This technology has helped to influence a new way of creating and viewing different dimensions with time based content. It’s been re-developed from film production to live capabilities as there is a lucrative market for televised sporting events, but as technology/software engineering seems to be progressing so quickly in this field, it will soon be superseded by a much simpler, cheaper, and more versatile solution for such purposes, usually the second wave of development makes the real money. (F. Brian, personal communication, March 6, 2014)
Brian predicts that despite the ongoing development of these replay systems that have only
recently been configured for live television purposes, they may soon be replaced by a next
generation system designed to maximise profit for companies. Brian's quote points out the
underlying fact driving this development - the profitability of the system. However according to
Diego Prilusky (personal communication, March 19, 2014):
I think that combination of tools, fast computing, digital image, low prices of cameras with very big sensor and a very talented and versatile team working together – a combination of all of those, made freeD to exist now and not a decade ago.
34
Prilusky is a developer of the freeD system and points out that development of the system was
driven through technological change as opposed to the company building a product specifically
for commercial enterprise.
It may seem that Brian's initial interview evidence contradicts this: "This is an experimental
field, a lot of the experimental guys are out in the cold as no one cares about creativity, it's all
done to generate profit and hit a target audience" (F. Brian, personal communication,
November 21, 2013). However freeD has only recently come into the public eye and it is
important to remember that being a new system, there is some way to go before the team at
Replay Technology will have a marketable product suitable for the home. It remains to be seen
if manufacturers will try to implement the virtual replay within their televisions. This is the
stage where the potential for commercial success will drive further development if the product
is found to be suitable.
35
5.0 Conclusions
It is important to take note that although the original Time Slice method has evolved, from the
original simple 19th century experiments of Muybridge and his multi camera film setup to the
3D renders of Replay Technology, it is possible to trace the process right back to its origins.
With the first use of the Time Slice effect to study horses in 1878, it would suggest that the
technology has progressed very slowly over the last two centuries. Up until 2012, Bullet Time
was still shot in exactly the same way as the original camera setup, with multiple cameras
taking one still, which was later combined to create the sequence. It is interesting that with the
original work of Muybridge leading to the creation of the first motion picture, that it would take
another 200 years for the method to be used again, and then only to add effects to said motion
pictures.
This dissertation answers the question 'Virtual Replay in television and broadcasting: are
broadcasters reusing old technology.' The evidence behind the development of Time Slice
replays provides support for the idea that technology in the industry is being reused but in an
evolved form. There is both primary and secondary evidence to support the idea that this
progression is not being driven because of breakthroughs in technology but as a natural
progression of ideas and techniques. This information can be seen from both the face to face
interviews and email communications.
A revolution in live television and broadcasting has seen 'interactivity' becoming a significant
key. Both designers and manufacturers will be looking to involve and immerse the audience as
much as possible, so it stands to reason that consumers will see either NHK's bullet time system
or Rapid Technology's freeD system in homes within the foreseeable future. The playback
offered by these two companies is ultimately controllable and enhances the standard viewing
that is currently available.
It is possible that these Bullet Time techniques are another trend following a cyclical pattern,
emerging for a few years before disappearing again but the promotional work done by these
companies will stand them in good stead. There have been many trends in the last half century,
relying on the fact that manufacturers need customers to want to replace their television sets
36
in order to drive turnover. Recent years have seen a number of advances in technology
generating a drive in demand for new televisions sets:
Quality of the image has substantially improved;
Size of the devices has reduced in depth, but increased in viewing size;
Reduction in cost;
More efficient television sets.
In addition, the home user is subject to a continued marketing campaign to subscribe to new
broadcasting services – movies, sports, comedy etc. In 2010 there was a push by manufacturers
to create three dimensional stereoscopic televisions in order to replace high definition
televisions (HDTVs), but the campaign did not go well, with many broadcasters dropping 3D
segments from their televised packages and overall sales of television sets falling. After the
recent dip in popularity of 3DTVs it would seem that virtual replays would make a worthy
replacement. As an additive package similar to a sky box or an internal system that would see a
whole new range of television sets, the freeD system seems to be a likely forerunner. This could
just be another technology that spends its time in the limelight, only to be dropped again when
a newer more potentially profitable system comes along, but for now it seems an appropriate
choice to improve the customers viewing experience.
5.1 Recommendations
It is in the best interests of the developers that have devised freeD and multi camera
technologies to prolong the life of Bullet Time style virtual replays. With the popularity drop in
2003, and the loss of similar technologies such as the morph, it should be advised that these
new systems are not overused. Heavy marketing and promotion would see a saturation of the
market and this in turn would likely lead to a drop in public popularity. Due to the great costs of
the system, broadcasters would no longer choose to feature unpopular technology and any
hope of integrating the system with television sets would be lost. It is therefore imperative to
retain strong patents on the system, and attempt to prevent competitors from releasing similar
technologies whilst refraining from over-marketing the system. Referring back to the product
life cycle (Raymond, 1966) this would lengthen the introduction and growth phase. Establishing
37
the product within the market is key, but preventing this saturation, whereby the public can
readily see it demonstrated in all aspects of everyday life, is of the utmost importance. If
consumers see a stunning effect every day it loses its appeal and becomes boring to watch.
Further studies are necessary to gauge public perception of the virtual replay, which if tracked
would help companies gather information as to whether marketing strategies are really helping
to sell their products.
Whilst the use of replay technology is visually stunning, it may be difficult to see how this
technology, in a domestic environment, would lead to wide scale development of new
television sets or specialised replay subscription offerings. Taking lessons from the failings of
'Eyevision' in 2001 and the significant drop in popularity of the Bullet Time effect in 2003, a
period where broadcasters would no longer use the technique, it is important to limit the use of
the freeD system in order to prevent a recurrence and subsequent loss of popularity. I t may be
the case that Bullet Time style replays will follow a similar course; a temporary growth phase,
followed by saturation and a quick decline. Whilst it is generally accepted that the consumer is
purchasing new televisions for the home, replay technology may not be seen as a must have
and as such may prevent a successful drive in sales. In addition, would a broadcaster like the
BBC or SKY readily invest in creating a specialist subscription channel, particularly considering
the recently perceived failure of 3DTV. This is an area that needs to be studied further, as it is
impossible to predict the potential success rate of a television system when it is at least a year
away from release. Following the recommendations suggested in this dissertation could see
that a domestic system is more than an overnight success but the overall public perception
remains to be seen.
6.0 Evaluation
This dissertation has thoroughly explored the background context and leading developments of
the 3D virtual replay. Due to the ever changing nature of the industry this project, in a similar
fashion, has also evolved. In its earliest stages, this dissertation analysed the likelihood of
independent filmmakers being able to readily produce Time Slice style sequences on a small
budget. However the freeD system and its potential for change within the industry was deemed
38
to be more significant. With this large change in direction the early initial findings did not have
such a strong relevance to the project. During certain interviews, such as those with Kensuke
Ikeya (Appendix 9), developer of NHK, and the initial meetings with Farley Brian (Appendix 16)
and Tim Kilgour (Appendix 15) questions were asked that were more relevant to the initial
direction. The interviews conducted later were more appropriate but did have an impact on the
time scale, with some research received as little as a week before the project deadline. In
hindsight, there was not much that could have been done to alter the course earlier in the
project but if this dissertation was conducted again it would be important to schedule early
interviews with more specific questions targeted at the 3D virtual replay.
39
7.0 Reference List
Agisoft Photoscan Professional Edition. (n.d.). Retrieved from Agisoft company website:
http://www.agisoft.ru/products/photoscan/professional/
Anthony, S. (2013, January 9). 3D tv is dead. Retrieved from:
http://www.extremetech.com/extreme/145168-3d-tv-is-dead
Argy, S. (2001). Frozen f/x still in action. Retrieved from:
http://variety.com/2001/film/news/frozen-f-x-still-in-action-1117792382/
BBC Tomorrows World 1993: Time Slice Films Feature [Webcast]. (2010, January 19). Retrieved
from: http://www.youtube.com/watch?v=fIpmUi8HI1k
Berg, A. (Director). (2009). Carousel [Advertisement]. Netherlands: Tribal DDB.
Box office / business for the Matrix (1999). (2013). Retrieved from:
http://www.imdb.com/title/tt0133093/business?ref_=tt_dt_bus
Brace, I. (2008). Questionnaire Design : How to Plan, Structure and Write Survey Material for
Effective Market Research (2nd ed.). London, GB: Kogan Page Ltd.
Braun, M. (2010, September 15). Eadweard Muybridge. London: Reaktion Books.
Braun, M. (2012). Eadweard Muybridge. London, GBR: Reaktion Books
Brilot, D. (2012, April 30). TV on demand. Retrieved from:
http://cdn.yougov.com/cumulus_uploads/document/j8hi9ko29n/Smart%20TV%20-
%20%20Consumer%20PR%20%28FINAL%29.pdf
Bringing ‘The Matrix’ to the NFL. (2013, September 9). Retrieved from:
http://www.tvtechnology.com/prntarticle.aspx?articleid=221089
Brown, S. (2013). From novelty to normal: 3DTV as a special effect. Critical Studies in Television;
Autumn 2013, 8(3). Retrieved from: http://ehis.ebscohost.com/eds/detail?sid=f4b18717-d70a-
43f3-b345-
40
782a4f1cc771%40sessionmgr110&vid=3&hid=109&bdata=JnNpdGU9ZWRzLWxpdmU
%3d#db=f3h&AN=92009808
Carnegie Mellon Goes to the Super bowl. (2001, January). Retrieved from:
http://www.ri.cmu.edu/events/sb35/tksuperbowl.html
Carreker, D. (2012). Game Developer's Dictionary : A Multidisciplinary Lexicon for Professionals
and Students. Boston: Cengage Learning.
Carroll, J. (2013, September 10). Innovator's take: 'Free' Dimensional video and the future of the
moving image medium. Retrieved from:
http://www.vision-systems.com/articles/2013/09/innovator-s-take-free-dimensional-video-
and-the-future-of-the-moving-image-medium.html
Charles, A. (2012). Interactivity: New media, Politics and Society. Retrieved from:
http://ehis.ebscohost.com/eds/detail?sid=170cc298-dde9-4954-87b0-
a23b5ab146d3@sessionmgr115&vid=3#db=nlebk&AN=542470
Costa, B. (2013, October 11).The next big thing? How Replay Technologies' freeD system is
taking sports tv by storm. Retrieved from: http://sportsvideo.org/main/blog/2013/10/the-next-
big-thing-how-replay-technologies-freed-system-is-taking-sports-tv-by-storm/
#sthash.GZHF78Ar.dpuf
Debevec, P. (1996). Modelling and Rendering Architecture from Photographs. Retrieved from:
http://people.ict.usc.edu/~debevec/Thesis/debevec-phdthesis-1996.pdf
Debevec, P. (2006, August). Virtual Cinematography: Relighting through Computation.
Retrieved from: http://graphics.stanford.edu/papers/lfphoto/comp-photo-articles/Data/
DEBEVEC.pdf
Dominques, E., Michalska, A. (n.d.). 15-463: Computational Photography - Final Project: View
Morphing. Retrieved from:
http://www.cs.cmu.edu/afs/andrew/scs/cs/15-463/f07/proj_final/www/amichals/
41
Eadweard Muybridge. (n.d.) Retrieved from:
http://www.wildfilmhistory.org/person/180/Eadweard+Muybridge.html
Eagleman, D. (2009, February 21). Why do we love slow motion video? Retrieved from:
http://www.sentientdevelopments.com/2009/02/why-do-we-love-slow-motion-video.html
Ebay demo [webcast]. (2013, September). Retrieved from: http://vimeo.com/73813340
Edwards, S. (2010, September 23). The photgraphic Reality of Eadweard Muybridge. Retrieved
from: http://www.currentintelligence.net/cameraobscura/2010/9/23/the-photographic-reality-
of-eadweard-muybridge.html
Fenlon, W. (2012). 48 FPS and Beyond: How High Frame Rate Films Affect Perception. Retrieved
from: http://www.tested.com/art/movies/452387-48-fps-and-beyond-how-high-frame-rates-
affect-perception/
Generating 3D content from multi- viewpoint images. (n.d.). Retrieved from:
http://www.nhk.or.jp/strl/english/aboutstrl1/r1-2-2.htm
Gills, S. (2005). The Matrix trilogy: Cyberpunk reloaded. 6a Middleton Place, Langham Street,
London W1W 7TE: Wallflower Press.
Godoski, A. (2011). Under the influence: The Matrix. Retrieved from:
http://www.screened.com/news/under-the-influence-the-matrix/2218/
Goldstein, M. (2008), DSLR Camera Prices Falling. Retrieved from:
http://www.photographyblog.com/news/dslr_camera_prices_falling/
GoPro. (2013). HERO3 Cameras. Retrieved from: http://gopro.com/hd-hero3-cameras
Hall, I. (1981, May). Kill and Kill again [Motion Picture]. South Africa/ United States: APC/ Again
Productions/Kavalier Films
Harold Edgerton. (1999, February). Retrieved from:
http://web.mit.edu/invent/iow/edgerton.html
42
High Def Slow motion crash testing shows art of safety. (2013, August 28). Retrieved from:
http://www.autonet.ca/en/2013/08/25/high-def-slow-motion-crash-testing-shows-art-of-
safety
High Speed Camera. (n.d.). Retrieved from:
http://edgerton-digital-collections.org/techniques/high-speed-photography
Hipstomp. (2011). Rip Curl & Time Slice Camera Array Collabaration Lets You Percieve Surfing as
Never Before. Retrieved from: http://www.core77.com/blog/technology/rip_curl_time-
slice_camera_array_collaboration_lets_you_perceive_surfing_as_never_before_20925.asp
Horse in Motion. (n.d.). Retrieved from:
http://www.tate.org.uk/whats-on/tate-britain/exhibition/eadweard-muybridge/exhibition-
guide/horse-motion
How a trotting horse appears when at a 2:20 Gait. (1878, June 18). Sacramento Daily Union, p.
1.
Hruska, J. (2011, July 5). Low 3-D uptake worries TV manufacturers. Retrieved from:
http://hothardware.com/News/Low-3D-Uptake-Worries-TV-Manufacturers/
Hsu, W., Zhang, Y., Ma, K. (n.d.). A Multi-Criteria Approach to Camera Motion Design for Volume
Data Animation. Retrieved from: http://vis.cs.ucdavis.edu/~ybzhang/papers/vis13hsu.pdf
Jasonhopper. (2011, February 16) Why do qualitative research?. Retrieved from:
http://methodlogical.wordpress.com/2011/02/16/why-do-qualitative-research/
Jones, G.E. (2005). 24P digital video: make your DV movies look like Hollywood. Boston:
Cengage Learning.
Joshuajnoble. (2011, June 21). What is interaction design? Retrieved from:
http://thefactoryfactory.com/wordpress/?p=823
Johnson, S. (2014, January 6). What is a point cloud? Retrieved from:
http://www.wisegeek.com/what-is-a-point-cloud.htm
43
Kindt, T., Muller, H.-H. (2003). Narratologia: What is Narratology?: Questions and Answers
Regarding the Status of a Theory. Berlin: Walter de Gruyter.
Kinect for Xbox 360. (n.d.). Retrieved from Microsoft website:
http://www.xbox.com/en-GB/Kinect
Kurosawa, A. (Director). (1954). Seven Samurai [Motion Picture]. Japan: Toho Studios.
Laird, J. (2013, October 12). Step aside 3D: 4K actually delivers in movies and games. Retrieved
from: http://www.techradar.com/news/computing/step-aside-3d-4k-actually-delivers-in-
movies-and-games-1181885
Lister, M., Dovey, J. & Giddings, S. (2008). New Media: A Critical Introduction. Florence, KY:
Routledge.
Martin, B., Hanington, B. (2012). Universal Methods of Design : 100 Ways to Research Complex
Problems, Develop Innovative Ideas, and Design Effective Solutions. Retrieved from:
http://ehis.ebscohost.com/eds/detail?sid=5bab1156-6c29-4c35-96c5-
2adfab4dc6b2@sessionmgr198&vid=1#db=nlebk&AN=576491
Martzke, R. (2001, January 23). Instant replay gets 'Matrix'-like view of key plays. Retrieved
from: http://usatoday30.usatoday.com/sports/2001-01-23-focus-matrix.htm
Masefield, R. (2010, October 20). Timeslice goes bullet-time with Rip Curl Mirage promo.
Retrieved from: http://www.masey.com.au/2010/10/timeslice-goes-bullet-time-with-rip-curl-
mirage-promo/
Michel Gondry. (n.d.) Retrieved from the Internet movie database website:
http://www.imdb.com/name/nm0327273/
Motion picture pioneer: Eadweard Muybridge and the zoopraxiscope. (2011, November 5).
Retrieved from: http://www.lomography.com/magazine/lifestyle/2011/11/05/100424
Padgett, K. (n.d.). Eadweard Muybridge - the bet that lead to moving pictures. Retrieved from:
http://agilewriter.com/Biography/Muybridge.htm
44
Patra, S., Bhowmick, B., Banerjee, S., Kalra, P. (n.d.). High resolution Point Cloud Generation
from Kinect and HD Cameras using graph cut. Retrieved from:
http://www.cse.iitd.ac.in/~mcs103515/VISAPP_2012_246_CR.pdf
Point Clouds. (n.d.) Retrieved from Pixar website: http://renderman.pixar.com/view/point-
clouds
Press release. (2013, May 9). Retrieved from:
http://www.nhk.or.jp/pr/english/press/pdf/20130509-001.pdf
Prince, S. (1991). The Warrior's Camera. Chichester, West Sussex. Princeton University Press.
Purse, L. (2005). The New Spatial Dynamics of the BulletTime Effect. In G. King (ed.) The
Spectacle of the Real: From Hollywood to Reality TV & Beyond, pp.151-160. Bristol: Intellect
Books.
Rehak, B. (2007). The Migration of Forms: Bullet Time as Microgenre. Film Criticism, 32 (1),
pp.26-48.
Rober, Mark. (2013, May 9). EPIC Matrix effect w/ a ceiling fan & 1 GoPro (ghetto bullet time)
[Webcast]. Retrieved from: http://www.youtube.com/watch?v=OEd5lTmeAH8#t=103
Rockwell, K. (2012). Nikon D1. Retrieved April 21st, 2013 from:
http://kenrockwell.com/nikon/d1.htm
Rolli, G., Sande, P. P., Rodriguez, F. D., Ewequiel, W. S. L. (n.d.). Bullet Time. Retrieved from:
http://sabia.tic.udc.es/gc/Contenidos%20adicionales/trabajos/Peliculas/FX/
Rouse, M. (2007, October). Definition pan (video). Retrieved from:
http://whatis.techtarget.com/definition/pan-video
Rouse, M. ( 2007, May). Definition - Voxel. Retrieved from:
http://whatis.techtarget.com/definition/voxel
45
Saunders, M., Lewis, P., Thornhill, A. (2012). Research methods for business students. Harlow:
Pearson Education Limited.
Saunders, P. (2013). How instant replay has revolutioned sports on tv. Retrieved from:
http://www.denverpost.com/sports/ci_24539959/how-instant-replay-has-revolutioned-sports-
tv
Sanders, S. (2014) Using Slow Motion in Action Movies. Retrieved from: http://www.steves-
digicams.com/knowledge-center/how-tos/filmmaking-tips/using-slow-motion-in-action-
movies.html
Scorsese, M. (Director). (1980). Raging Bull [Motion picture]. United States: United Artists.
Seitz, S., Dyer, C. (n.d.). View Morphing. Retrieved from:
http://www.cs.cmu.edu/~seitz/papers/sigg96.pdf
Smed, J., Niinisalo, H., Hakonen, H. (2005). Realizing the bullet time effect in multiplayer games
with local perception filters. Computer Networks, 49 (1), pp.27-37.
Snyder, Z. (Director). (2006). 300 [Motion Picture]. United States: Warner Bros.
Stroboscopic. (n.d.). Retrieved from:
http://edgerton-digital-collections.org/techniques/stroboscope
Sullivan, L. (1896). The tall office building artistically considered. Lippincott's magazine. 403-
409.
Svetlik, J. ( 2013, January 17). 3D TV is dead, replaced by 4K, says BBC controller. Retrieved
from: http://www.techradar.com/news/television/hdtv/3d-tv-is-dead-replaced-by-4k-says-bbc-
controller-1125721
Swetnam, Derek. (2000, April 1). Writing your dissertation: how to plan, prepare and present
successful work. Oxford: How to Books.
46
Technology. (n.d.). Retrieved from Replay Technologies company website: http://replay-
technologies.com/technology/
The Eye network. (2001, January 28). Retrieved from:
http://sportsillustrated.cnn.com/football/nfl/2001/playoffs/news/2001/01/28/
superbowl_tv_ap/
The Matrix. (1999). Retrieved from the Internet Movie Database website:
http://www.imdb.com/title/tt0133093/
Time Slice. (n.d.). Retrieved from: http://www.timeslicefilms.com/people-2/tim-macmillan
Vernon, R. (1966). The Product Life Cycle. Retrieved from:
http://db.lib.uidaho.edu/ereserve/courses/b/business/380_01/life.pdf
Wachowski, A., Wachowski, L. (Directors). (1999). The Matrix [Motion Picture]. United States,
Australia: Warner Brothers Pictures/ Village Roadshow Pictures/ Silver Pictures/ Groucho II Film
Partnership.
Walford, A. (2007). What is Photogrammetry? Retrieved from:
http://www.photogrammetry.com/index.htm
Warner Bros. Entertainment Inc. (2007). U.S Trademark Registration number: 3349252.
Washington, DC: U.S. Patent and Trademark Office.
Wilkinson, S. (2010, December 31). Is Frame Interpolation a Godsend or the Devil's Spawn?
Retrieved from: http://www.soundandvision.com/content/frame-interpolation-godsend-or-
devils-spawn
What is a Gantt chart?. (n.d.). Retrieved from: http://www.gantt.com/index.htm
Yoshida, T. (Producer). (1967). Speed Racer [Television Series]. Tokyo: Fuji TV.
200 Things That Rocked Our World: Bullet Time. (2006, February). Empire, 200, 136.
47
7.1 List of Figures
PageFigure 1 A simple example detailing the arrangement of cameras in a bullet time
setup [Primary source].
5
Figure 2 Screenshots from Rober's video, showing the spinning motion of the setup.
Rober, M. (2013, May 9). Epic Matrix effect w/ a ceiling fan & 1 GoPro (ghetto
bullet time). Retrieved from: https://www.youtube.com/watch?v=OEd5lTmeAH8
6
Figure 3 A simple illustration detailing Rober's experiment in more detail. [Primary
source].
7
Figure 4 Muybridge's first Horse in Motion work.
Muybridge, E. (1878). The Horse in motion. Retrieved from:
http://cosmicdiary.org/mshowalter/files/2013/02/Figure-1.jpg
9
Figure 5 Shots of the Speed racer Bullet sequence in effect, less than one second in
duration. Superherocartoonsite. (2009, January 13). Speed Racer 1967
Cartoon intro. Retrieved from: https://www.youtube.com/watch?
v=suCm1w_KTiY
11
Figure 6 A selection of shots demonstrating the Bullet Time sequence of 'EyeVision',
lasting less than a second in duration. Best of EyeVision (pregame show and
examples. (2001, January 28). Retrieved from:
http://www.ri.cmu.edu/events/sb35/eyevision_best_of.mpg
19
Figure 7 The product life cycle devised by Vernon (1966) [Primary Source]. 25
Figure 8 Gantt chart 1, created July 2013. [Primary source]. 31
Figure 9 Gantt chart 2, created January 2014. [Primary source]. 31
48
8.0 Appendices
8.1 Email Communications
Appendix 1 - Bob Rehak, author of 'The migration of Forms: Bullet Time as Microgenre - February 16, 2014
Question #2, about the resurgence in popularity of bullet time, I'd address by suggesting that enough time has now passed (about a decade since the release of the two Matrix sequels) to erode the parodic connotations of the effect; in essence, the joke stopped being funny and then was forgotten entirely, rendering BT again spreadable across media. I noticed a variation of of bullet time, for example, in Dredd (2012) played absolutely seriously.
Hope this is helpful, and again, sorry I'm not equipped to answer your more technical questions.
Appendix 2 - Bob Rehak, author of 'The migration of Forms: Bullet Time as Microgenre - March 3, 2014
Hi Chris. I think this and your previous question about the future of bullet time both go back to my initial response that BT constitutes a powerful new mode of visualization -- a new phrase in the cinematic/spatial lexicon, if you will -- that is now in resurgence following the expiration of the "punchline" effect associated with the Matrix and its knockoffs & parodies. In my article I argue that technologies underlying bullet time don't matter -- the visual effect is reproducible using any number of modalities -- and that it's only the signature visual that travels. As for ideology, for me this is linked to the way BT is interpreted in relation to this or that parent text; in the Matrix it meant one thing, but with the waning of that association, BT is free to signify differently. It's basically like a copyright that has expired, putting the visual effect in the public domain, where it will surely continue to mutate and evolve.
Appendix 3 - Farley Brian, Senior Digital Producer - January 30, 2014
So, with point clouds, i do remember the radiohead project as the first time the process went viral, but my guess is that the technology was developed for military/oil exploration etc (which may be why theres no definitive articles on such things as they are both really quite secretive)... also it uses the mathematical basis of Delaunay triangulation (1934) so i guess the theory was there and it became really useful when the technology evolved (computation ability combined with laser/IR development)
A good article here on the radiohead project and related links http://www.graphics.com/article-old/building-radiohead%E2%80%99s-house-cards
49
And this motions the lidar sensor (uses lasers instead of ir as it can scan much larger areas) and in their blurb it mentions ‘designed for obstacle detection and navigation of autonomous ground vehicles and marine vessels’
With the replay Tech, they really don’t give much away on their site. The only thing i can think of is: each camera knows its position in relation to the other cameras/forming a 3d space just in relation to each other, then (at a guess) all of the cameras are configured to point at/work with one individual focus point (which then becomes the user control point for operating the system as a whole). Mapping the 360 background to this focal point is the bit i can’t decide on, the golf clip they use really does seem to suggest that maybe an additional 360 degree camera is used as well for mapping/mixing into the focus of the footage into a consistent 360 degree 3D space, what’s a bit of a giveaway is when you look at the background spectators, they operate just like cardboard cut outs as the camera moves, there’s no change to any angles as it moves from left to right, so maybe its two systems combined but the 360 camera fills the background. In terms of how it comps together i have seen so many impressive software based image processing algorithms (especially when you look at augmented reality apps that don’t rely on anything but the live camera to map to) i would hazard a guess that it uses a software approach as opposed to point clouds, but with a fair amount of pre configuration with regard to every camera position/integration of singular 360 capture for the background.
Appendix 4 - Farley Brian, Senior Digital Producer - March 6, 2014
Overall, I think this technology has helped to influence a new way of creating and viewing different dimensions with time based content. It’s been re-developed from film production to live capabilities as there is a lucrative market for televised sporting events, but as technology/software engineering seems to be progressing so quickly in this field, it will soon be superseded by a much simpler, cheaper, and more versatile solution for such purposes, usually the second wave of development makes the real money.
I would say that its real value lies in its influence, as it's an early representation of the future possibilities of time-based media, but not just tv and film but for new emerging formats.
Project tango and similar developments online really point to where the potential expansion of such techniques lie, i.e : http://www.youtube.com/watch?v=Qe10ExwzCqk
Its still at early stages but eventually i think we will see a similar effect that’s even more versatile through this area/medium of development.
50
Appendix 5 - Farley Brian, Senior Digital Producer - March 20, 2014
Interesting to see that a fair amount of post production needed to go into the matrix to get the effect correct. And with FreeD, really reminds me of http://photosynth.net/about.aspx which has been around for some time, i do remember initially they sold it as a combination of geo tagged shots submitted by various users who took photos at the same location, eventually all angles would make up a 360 degree representation.
Overall, i do think we are in the middle of a huge transition that’s going to outmode what film/traditional moving media is at present, and as they say the increase in computational power/software capabilities is really what’s driving these new ways of doing things and how their system came about, and now such effects and movie making overall are now not just in the hands of huge studios, but are slowly going to form a set of tools anyone can use for whatever purpose, thus expanding what the medium can do. Their electricity analogy certainly does point at how one unintentional thing can lead to / influence another and adding to this, the invention of the cathode ray tube / tv came about in the Victorian era when they were actually looking for evidence of the spiritual world...
Appendix 6 - Joe Leveson, Technical Director Prime Focus- October 27, 2013
Sorry for the delayed reply. It's been a very busy week. I did ask a couple of my supervisors about bullet time and according to them it's not used so much now days. They use what is called a digi double, which is a fully rigged and textured model of the character. The reason for this is because the levels of texturing has moved forward quite a bit since bullet time became used in the late 90's. It's also cheaper and more freedom when it comes to planning shots. If I were you I'd look to go down this route. I hope this information is helpful.
Appendix 7 - Joe Leveson, Technical Director Prime Focus- October 31, 2013
A digi double is a 3d character that has been modelled built and textured to look exactly like a real life character. It is then rigged (meaning a skeleton is put in side ) so that it can be animated.Here is an article of the man of steel:http://www.fxguide.com/featured/man-of-steel-vfx-milestones/If you scroll down you'll see an article a section on digi doubles. To get something like this done you will can contact and number of visual effects companies in soho. Which one will depend on your budget. The more you spend to the better it will look though a smaller company will do a good job and would be suited to take it on. A company that would do commericals and music videos is your best bet.
51
Appendix 8 - Joe Leveson, Technical Director Prime Focus- October 31, 2013
no worries. This is a digi double head:
http://www.youtube.com/watch?v=jAP59_FzKIo
I recommend going down this route as its much easier to find a company that does this and more flexible than bullet time.
Appendix 9 - Kensuke Ikeya, developer of NHK multi camera system - October 30, 2013
Sorry for may late replay.I answer to your question.
>1. Can I ask where you got the idea for this project, was it from previous Bullet Time rigs used in film/ television or person developing a similar system?
I am inspired by "Eye vision" which was developed by prof Takeo Kanade. http://www.ri.cmu.edu/events/sb35/tksuperbowl.htmlSome improvements are added to my system to make my system easier to use at actual TV program production site.
>2. What do you think the future holds for the technology, in your opinion will it be used in mainstream television and if so how many months/years would you predict it being integrated into common footage?
Practical use in Japanese TV program has already begun.
I used my system for replay of gymnastics sports scene in June and will also use for replay of figure skating in November.
In the future ( sorry, I can't predict), I want to use it for a free view point video and a integral 3D TV.
>3. What do you think of the multi-camera companies that use computer animation to create 3D footage such as; http://replay-technologies.com/
I think that movies replay-technologies company creates are high quality and very cool !
In the future, I want to create such high quality movies on real time from multi-viewpoint videos captured my multiple robotic camera.
>4. How many cameras can be used in the setup?
Now, I use 9 robotic cameras in the set up.
52
It is possible to increase number of robotic cameras infinitely.
Appendix 10 - Don Kennedy -Media Producer, October 21, 2013
We wrote the article based on an interview we conducted with a representative of NHK STRL, as well as this (rather sparse) press release: http://www.nhk.or.jp/pr/english/press/pdf/20130509-001.pdf
Also, someone commented on the YouTube page that a similar system was used during a Super Bowl broadcast in 2001. You can find out more information about that 'EyeVision' system as well as sample videos here: http://www.ri.cmu.edu/events/sb35/tksuperbowl.html
If you have any other questions please don't hesitate to get in touch.
Appendix 11 - Don Kennedy -Media Producer, October 28, 2013
You could possibly get in touch with Takeo Kanade, the computer vision researcher at Carnegie Mellon who developed the eye vision system.
Also, this company: http://replay-technologies.com/ are doing similar stuff but generating 3D environments using the footage and metadata from the camera.
Appendix 12 - Oren Haimovitch Yogev, CEO of Replay Technology - January 28, 2013
See these clips to better understand the technology. We are working on passive elements without I/R depth emitters.
http://vimeo.com/82561034
http://vimeo.com/82561032
http://vimeo.com/80774790
We are coming to the household in Late 2014-start of 2015.
Appendix 13 - Oren Haimovitch Yogev, CEO of Replay Technology - October 19, 2013
Take a look at this one , interview with our CTO and another 2 pieces below which will give you some data. http://www.vision-systems.com/articles/2013/09/innovator-s-take-free-dimensional-video-and-the-future-of-the-moving-image-medium.html
53
http://www.tvtechnology.com/prntarticle.aspx?articleid=221089 http://sportsvideo.org/main/blog/2013/10/the-next-big-thing-how-replay-technologies-freed-system-is-taking-sports-tv-by-storm/
Appendix 14 - Diego Prilusky, Creative director for Replay Technology - March 19, 2014
Your questions are reasonable and they are really tackling the basic of the existence of our product and why after a decade of the matrix effect, the bullet time, we still find enough interest around the world for our technology.
The answer really is in within the questions, mainly because the questions are referring to the bullet effect as his own, or more to the end result. And looking at the end result, the possibility of moving around a frozen moment in time can be achieved in many ways. As you might already investigated in the bullet time effects was developed for few years before it was finally used in The Matrix movie in 1999. The effect permitted to film a subject and create a movie, a sequence of images played back in a timeline of a timeless moment. The effect is quite amazing by the actual possibility to cheat our timeline existence by capturing a timeless event and perceiving it in our timeline perception.
Capturing a moment in time, was already invented two centuries ago by exposing a mixture of paper and chemicals to the light for a given moment, thus storing this light reflection into a finite capsule which we can always look at and re-experience this reflected light – a picture.Eadweard Muybridge continued this experiments of encapsulating a fraction of light by doing so sequentially. He created a rig of multiple cameras exposing each at a continues ordered time steps sampling the reflected light for any given moment. Thus the video was created, you can always replay this sequence of pictures and re-experience the recorded moment.
Very similar to what and how Muybridge invented the moving picture in a way, the bullet time was created. Instead of timing each camera to a sequential time interval, the cameras are all together simultaneously expose to light, taking a picture at the same moment of time. Now, although all cameras perceived the same moment, each one of them is positioned in a different position capturing a different angle of the subject. When placing enough cameras one near the other, creating a sequential path, the images can be played sequentially, in the same order the cameras source is positioned. By doing so we get a single moment in time (freeze bullet time) while the point of view change, which creates a sort of camera motion.
54
This same technique was used by EyeVision in 2001 for sport. The reason the video was very shaky and non-smooth was due to the fact that this technique lacks of single focus point. The cameras are aimed at a defined region, by the time EyeVision used robotic arms to dynamically aim all the cameras at once to a specific subject. This works good enough to get the freeze moment with motion once the images captured are played sequentially. But the shakiness comes from having each camera with a slightly different focal lens, and the subject not always at the center of the image. So by playing this in a timeline the centered subject changes position from picture to picture, even if it’s just slightly it will be enough to get a “shaky” result. It is equivalent to moving the camera all the time while filming a subject.In The Matrix, special effects department had to front the same challenge. The reason that it looks so smooth in the final result was due to the amount of cameras used, having a high number of cameras placed one just next the other simulates moving a single camera in that path. So let’s say each camera is 20cm from each other, placing 25 cameras next to each other will take a sum of 500cm. When playing this images sequentially at a frame rate of 25 frames per second, this setup will be equivalent of a camera motion moving 5 meters per second. So by placing the cameras as close as possible to each other is possible to get a final result equivalent to a slower camera motion. Secondly the images captured in The Matrix where post processed one by one and realigned digitally so all the focus points where the same, making sure the main subject is always positioned in the center frame. This removed the shaky effect of moving subject.EyeVision had a low camera number without a constant focus point. Also this shaky result can be perceived in all similar camera rig setup done by hobbies posted in any global video source like YouTube.
An additional limitation the above technique has, is the camera motion. The final camera motion of the frozen moment will be defined by the way the capturing cameras all placed next to each other. In order to change the camera motion, there is need to move physically the cameras in the rig by the new order wanted.
freeD is different from this techniques. freeD is a video format that encodes the images captures into a volumetric database. The similarity from the above is that for each given moment in time a pictured is captured simultaneously from different angles, same as the techniques above. But the instead of playing the captured images sequentially in a timeline, the pictures are analyzed and encoded to a single volumetric image. By having the three dimensional location of each camera used, each image has a known source position. Then by triangulating the same pixel from all the pictures we can get the 3d position of that pixel in a volumetric space, defined by 3 axis. Once the volumetric database is achieved, we get a special image that can be rendered from any position with a virtual camera, what we call the freeD
55
camera. This technique needs thousands more computation cycle than the previous used a decade ago. But the advantages are huge: There is no need to have a very large amount of cameras one next to each other to get a slower and smooth camera motion, with a 10% of cameras is enough to build a volumetric database that can be seeing from any angle. This will be one of the main reasons we get the polished, 3D fluidity.
Instead of playing each image one after the other, we combine all the information from multiple cameras to a single volumetric image that can be seeing from any angle. Because of the large amount of information we get captures, in order to reduce computing time we rescan each time only moving objects, while static objects get a volumetric database once. So you can say it’s a type of photogrammetry and previously composed 3D models. It’s very similar in a way to how a standard video codecs work, for example MPG, it will update in the video only the changing pixels. freeD is doing the same, but generating a special video format. A video format with enough information to see it from any angle.
So what’s new after a decade? I think that the answer is in the increase of GigaHertz being able to be calculated per second. The computational power today open doors to new techniques that before were too slow or just non-reasonable to do in near real-time. Also the movement of video from film to digital has created a huge amount possibilities to manipulate information never possible before.Could freeD exist a decade ago? Maybe, but it’s the same on asking why the electricity was invented when it was invented? What did they had at that time, that wasn’t before, that enabled the invention of electricity? There is not a straight answer. I think that combination of tools, fast computing, digital image, low prices of cameras with very big sensor and a very talented and versatile team working together – a combination of all of those, made freeD to exist now and not a decade ago.
56
8.2 Interview Extracts
Appendix 15 - Interview Transcript Extracts - Tim Kilgour, November 21, 2013
"I've worked for Fieldtrip for a while [a visual effects company] and I've spent a lot of time in the animation and visual effects industry."
"The work that Replay Tech is producing is pretty cool, looking at this video, there is what looks to be point cloud generation."
"Point clouds were developed to study terrain maps and things like that." Kilgour goes on to demonstrate examples of Point Clouds.
"These videos show the Kinect, a Microsoft camera and guys at home can create an interactive environment of their own home."
"When used in film production, this system [Point Cloud technique] creates highly detailed composites of a subject which could be modelled quickly, it is less time consuming and costly than a visual artist building a 3D model from photographs."
When asked if the Time Slice method was no longer being utilised:
"No I don't think so, we use it in different ways. Its evolved and mixed with other tech."
When asked how this would relate to Bullet Time;
"Bullet Time has come a long way, we used this on a job recently and you can see here [demonstrating a model of a 3d head] that the image we get is actually pretty good. A couple of cameras arranged round the model take an instant snapshot, point clouds interpret the data and assemble the 3D model. Saves a lot of time building models from photographs!"
"The old rigid forms of Bullet Time have been developed; we now use these multi camera methods to create complex 3D models. Technology develops over time and the best pieces are cotton picked and used in new places."
When asked if Bullet Time was dying off;
"No I don't think so, we use it in different ways. Its evolved and mixed with other tech."
Appendix 16 - Interview Transcript Extracts - Farley Brian, November 21, 2013
"Well, my names Farley and I've worked in every possible vfx job you can imagine. I'm now working for Pioneer creating web videos for their DJ equipment."
57
"I'm pretty sure a guy called Tim Macmillan was the first to use Bullet Time and then that was picked up by Warner. I remember the old QuickTime VR software which we used to hack and replicate this 360degree bullet time effect. It's pretty interesting how much it's come on. "
"I've had a look at these companies [Replay Technology and NHK] and I'm pretty impressed. It does look like they've [Replay Technology] used point clouds to shoot the subject which is framed inside a 3D environment."
"It's definitely true that the industry is changing, it's all about being interactive and everything working together. The new Bob Dylan music video is an awesome example, you can change the channel within the video and it's all synched together!" Brian demonstrates the video.
"On the flip side, this is an experimental field, a lot of the experimental guys are out in the cold as no one cares about creativity, it's all done to generate profit and hit a target audience"
"Technologies follow cycles and can be profoundly hit and miss, it's a bit shocking when you look back and find these seeds"
"In regards to The Matrix, the story of the techniques used actually sold the film. It is possible the film would have sold nowhere near as well without the seed that is Bullet Time. We had a project which didn't take well, only hitting a small audience, but it had new technology which the media picked up on and it marketed itself!"
58