View
6
Download
0
Category
Preview:
Citation preview
FUJIFILM Software Co., Ltd.
Since FUJIFILM Software debuted IMAGE WORKS in 2006, the image file management and sharing service quickly became a global standard for enterprises sharing large volumes of content. In 2016, the Nippon Professional Baseball Organization (NPB) adopted the solution to centrally manage game photos. At the time, the organization was faced with a new challenge of efficiently tagging photos. To provide game photos to corporate users, the players in each image needed to be tagged, which took considerable effort. FUJIFILM Software developed a player name auto-tagging function that employs an original image classification model using Microsoft Azure Cognitive Services and Microsoft Cognitive Toolkit. The company succeeded in drastically reducing the tagging workload. In addition, Azure Durable Functions was used to shorten processing times.
Tagging 300 photos from 3,000 creates enormous workloadThe NPB oversees the Central and Pacific Japanese professional baseball leagues and
contributes to the development of baseball culture in a variety of roles. In 2016, it began
operating the NPB Contents and Images Center (NPB CIC) to streamline the work of lending
out photos of professional baseball teams. The service centrally manages the photo assets of
each team, lends out team photos to users, and manages the associated invoices. NPB IMAGE
WORKS from FUJIFILM Imaging Systems provides the platform for the service.
IMAGE WORKS is a cloud service for sharing and managing large volumes of content, including
images and video. The solution enables safe and efficient management throughout the content
lifecycle, from sharing data during planning and creation to centralized management of
deliverables, data distribution, and deletion from the archives. Since its debut in 2006, IMAGE
WORKS has been highly regarded for its numerous impressive results, including adoption in
2016 for the G7 Hiroshima Foreign Ministers’ Meeting and the G7 Ise-Shima Summit.
NPB was also impressed with these results and adopted the solution for the launch of NPB CIC.
In the past, the photo management performed by each team had been implemented on a
unified platform.
CustomerFUJIFILM Software Co., Ltd.
Products and Services・ Azure・ Azure Cognitive Services・ Azure Functions・ Microsoft Cognitive Toolkit
IndustryDiscrete Manufacturing
SizeLarge (over 10,000 employees)
CountryJapan
Published December 2018
FUJIFILM boosts NPB player recognition, reduces photo tagging costs with Microsoft Azure
FUJIFILM Software Co., Ltd.
“Even after NPB adopted IMAGE WORKS, we would conduct
interviews to discover the company’s user needs,” says Riki Satou,
Manager of the Image Works Team at FUJIFILM Software.
According to Satou, an enormous amount of work time was spent
on tagging photos. “In order to lend a photo to a corporate user, the
players shown in each photo need to be tagged so that the photo of
the player can be found quickly. However, there can be as many as
3,000 photos per game and around 300 of those need to be selected
and determined as suitable for commercial use. NPB required an
automated and efficient system which automatically determined the
names of the players in the photos. To make the work more efficient,
we developed a player name auto-tagging function using AI.”
In December 2016, FUJIFILM Software adopted Microsoft Azure
Cognitive Services and other Microsoft AI services.
Riki SatouTeam ManagerImage Works TeamAdvanced Solution Development GroupServices DivisionFUJIFILM Software Co., Ltd.
Daichi HayataMCSE Cloud Platform and InfrastructureImage Works TeamAdvanced Solution Development GroupServices DivisionFUJIFILM Software Co., Ltd.
<IMAGE WORKS Screen>
Samples where faces cannot be identified
Samples where positions can be inverted
Samples where numbers cannot be identified
Samples where handedness can be determined
※ This pictures is for sample image. Actual product may vary.
FUJIFILM Software Co., Ltd.
Eventually, the company matched the players shown in photos by
comparing them against NPB BIP information, the official professional
baseball records provided by NPB, to reduce the number of possible
selections.
NPB BIP offered robust information to narrow down player names,
including the results of pitching and batting during a game, positions,
and handedness. By using AI, the company developed several models
to classify the players in the photos. These models extrapolated the
names of the players by categorizing them as pitchers or batters, or
whether they were right- or left-handed, then matching them to the
time that the photo was taken and the NPB BIP data, which narrowed
the candidates to no more than five players. To create these multiple
image classification models, the company used Microsoft Cognitive
Toolkit.
“We ultimately decided to create our own model with Cognitive
Toolkit; we were able to develop the model efficiently as it was
possible to conduct transfer learning based on Microsoft Research
ResNet (Deep Residual Net) model. Although it was the first time that
I had dealt with deep learning, I was able to use it proficiently with just
some basic knowledge, such as the kind of training data that should
be prepared and the amount needed. By this point, Microsoft had
held numerous hackathons for us and we had also received advice
from Microsoft partners. This support helped us to understand that
combining multiple models with limited functions would be more
accurate than trying to do everything with a single model. This
approach increased the probability that the correct selection would
be included in the final list of player names to more than 90 percent,” explains Hayata.
Creating multiple models for high-accuracy tag determinationsThere were two reasons why FUJIFILM Software adopted Azure AI
services for the player name auto-tagging function. First, IMAGE
WORKS has run on Azure since 2016, when Azure was initially adopted
as the platform. In 2017, SCIM (System for Cross-domain Identity
Management) had been deployed to enable system-linked user ID
management, and Azure Active Directory was included in this process.
Since the company was already using several Azure platform as a
service (PaaS) functions, using Azure AI services was a natural next
step.
Second, Azure AI services include a wide array of services, notably
Azure Cognitive Services, that can be used by those who are not AI
experts. Also, according to Satou, the company was impressed with
the support Microsoft provided for modernizing IMAGE WORKS and
using SCIM.
However, using AI to automatically extrapolate the names of players
in photos was no easy task. Daichi Hayata, from the MCSE Cloud
Platform and Infrastructure Image Works Team, and the developer
lead on the player name auto-tagging function, explains the
challenges.
“Many game photos are taken at an angle or from the side. Face API [a
function of Azure Cognitive Services] is extremely accurate if a photo
of a face is taken from the front, but it is rather limited if the photo is
from an angle. In fact, when we used Face API to determine the faces
in photos from games, the recognition rate was only about 20 percent.
We also considered a method in which determinations were made
using the names and numbers on uniforms,” says Hayata. “But there
were few photos that clearly showed the name or number together
with the player’s face, so it did not provide any major improvements in
accuracy. We worried about this dilemma for several months.”
“ We ultimately decided to create our own model with Cognitive Toolkit; we were able to develop the model efficiently as it was possible to conduct transfer learning based on Microsoft Research ResNet (Deep Residual Net) model. Although it was the first time that I had dealt with deep learning, I was able to use it proficiently with just some basic knowledge.”
Daichi Hayata MCSE Cloud Platform and Infrastructure Image Works Team Advanced Solution Development Group Services Division
FUJIFILM Software Co., Ltd.
“ In order to lend a photo to a corporate user, the players shown in each image need to be tagged…. To make the work more efficient, we developed a player name auto-tagging function using AI.”
Riki Satou Team Manager Image Works Team Advanced Solution Development Group Services Division
FUJIFILM Software Co., Ltd.
FUJIFILM Software Co., Ltd.
Azure Durable Functions reduces processing time, enables same-day taggingIn January 2018, FUJIFILM Software decided to combine multiple
determination models. From there, model development progressed
rapidly and was completed in June 2018, barely six months later.
However, just before completion, the company faced another
challenge. Running multiple classification models increased the
processing load. It took 40 minutes to determine 100 photos, which
the NPB and the teams using the service felt was slow.
“We initially thought that we could perform batch processing
overnight and have someone make the final determination the
next day, but we found out that the photographers of some teams
were tagging the photos themselves immediately after uploading.
Therefore, we needed to improve real-time performance,” explains
Satou.
The solution was Azure Durable Functions, which is provided by
a serverless compute service, Azure Functions. The service is an
extension of Azure Functions and Azure
WebJobs, and enables stateful functions to
be described in serverless environments.
Using the service for asynchronous
distributed processing of multiple functions
can shorten processing times dramatically.
Using this service with the player name auto-
tagging function reduced the processing
time to one-twentieth of the original time
and enabled same-day tagging.
“The cloud services of the different providers
generally offer the same functions these
days, but only Azure has Durable Functions,
which made us very happy to be using Azure.
There are about 3,000 photos per game on
average, and processing is completed within
about five minutes even if we register all
3,000 in NPB CIC,” says Hayata.
The following figure illustrates processing flow of the player name
auto-tagging function. First, photos are imported into IMAGE
WORKS, and Durable Functions is run on each image, performing
preprocessing tasks such as image resizing and rotation. Extrapolation
is then performed by combining Face API processing with classification
through the multiple models built using Cognitive Toolkit, and the
results are saved in a database.
NPB began trial use of the tagging function in June 2018, and currently
five teams are testing it. Users select the correct player name from
the list of candidates; tagging work that once took up to 3 to 4 hours
can now be completed in 30 minutes. In 2019, all teams that have
introduced NPB CIC are scheduled to make full use of the function.
“Although we are using AI in a form specifically for professional
baseball in this instance, a similar mechanism can be applied to other
sports,” says Satou. “We are already working on initiatives to apply
it to other sports, as well as considering expanding the function to
enterprises engaged in advertising and PR, and even adding video
analysis.”
The processing flow for the player name auto-tagging function provided to NPB. Multiple classification models created with Cognitive Toolkit are combined with Face API processing to extrapolate player names. In addition, running Durable Functions on the number of images has greatly reduced processing time.
①PhotoImport
⑤ClassificationModel Analysis
⑥ExtrapolationProcessing
Client
②Search ③
④Face API
BIP DataImport and formatting of
official professional baseball records
Durable Functions (Image Units)
AnalysisResult DB
Acquisition of images to
be analyzed
Durable Functions is run on the number
of images
・Exif Data Acquisition・Resizing・Rotation
Storage Face List Player Data
Preprocessing of Images
Implementation InquiryThis case study can also be accessed via the internet.https://customers.microsoft.com/Please note that the information contained in this case study was current as of the time of writing (December 2018) but may no longer be up to date.This case study is provided for information purposes only. Microsoft makes no warranties whatsoever, express or implied, in relation to the content of this document.
For product inquiries, please use the following contacts:■Website: http://www.microsoft.com/ja-jp/* All other company names, product names, and logos are registered trademarks or trademarks of their respective companies.* Please note that product features and specifications are subject to change without notice.
Microsoft Corporation
SE
Recommended