1

Copyright © 2017 Samsung. All Rights Reserved.

Home!  Sweet  Home!  Tizen  for  Home  Appliances:  

Family  Hub  Refrigerator  

Myungsik  Kim,  Elmurod  Talipov  Digital  Appliances,  Samsung  Electronics

2

1.  Family  Hub  :  Tizen  on  Home  Appliances    §  Features  /  SpecificaEons  /  Brief  UX  Guide    

2.  Supported  APIs  on  Family  Hub  §  Tizen  3.0  Public  APIs  §  Custom  (In-­‐House)  APIs  :  

-­‐    Refrigerator  Control  /  Voice  Features    /  Profile    /  OCF    3.  Running  app  on  Family  Hub  4.  Summary  

Agenda

3

1.  Family  Hub

4

Which  device  runs  Tizen  OS?  

A)  Family  Hub  Refrigerator C)  Robot  Vacuum  Cleaners  B)  Ovens  &  Cooktops  

Answer:  All  of  them,  and  more  home  appliances  run  Tizen  OS.

5

Family  Hub  is  a  smart  refrigerator  that  provides:

What  does  Family  Hub  mean?

Family

Connection ����������� ������������������  

Memo

Family Photo Album

Food Reminder / View inside Recipes Food

Management Shopping List����������� ������������������  

Music / Internet Radio����������� ������������������  Kitchen

Entertainment TV Mirroring����������� ������������������  

Get Connected, Get Closer to Your Family

Sharing Schedule

6

Recipe App View Inside

§  Family  Hub  is  a  smart  refrigerator  that  connects  families,  organizes  groceries  and  home  tasks,  and  provides  entertainment.    

What  are  the  Family  Hub  Features?

TV Mirroring Family Schedule Family Memo

7

OperaEng  System Tizen  3.0  Mobile  Profile

Display 21.5”  Full  HD  LCD

Graphic  ResoluEon 1080  x  1920  Full  HD

Browser  ResoluEon 540  x  960  (x2  scale  raEo)

Web  User  Agent Mozilla/5.0  (Linux;  Tizen  3.0;  FamilyHub)    AppleWebKit/537.3  (KHTML,  like  Gecko)  Version/2.3  Mobile  Safari/537.3  

ConnecEvity Wi-­‐Fi  802.11  b/g/n  2.4Ghz/5Ghz,  BT4.0  

Audio Dual  Microphone,  Voice  RecogniEon,  Stereo  Speakers

Camera 3  built-­‐in  Full  HD  Camera

Sensor Proximity  Sensor

Family  Hub  SpecificaLons

8

Brief  UX  Guide:  Home  Screen  Content  Area:  984  x  1776  

InteracEve  widgets  

ApplicaEon  Icon

Hardware  (External)  Keys:  01  Microphone  

-­‐ Tap  to  turn  the  microphone  on  or  off.  02  Menu  

-­‐ Tap  to  display  the  menus  available.  03  Home  -­‐  Tap  to  open  the  Home  screen.  -­‐  Tap  and  hold  to  open  the  Task  Manager.    

04  Return  -­‐ Tap  to  return  to  the  previous  screen.  

05  NoLficaLon  -­‐ Tap  to  display  the  Quick  Panel.  

ApplicaEons  can  request  to  display  noEficaEons  in  quick-­‐panel.  

More  noEficaEons  displayed  here More  noEficaEons  

displayed  here

9

Brief  UX  Guide:  Widgets  

§  In  addiEon  to  applicaEons,  interacEve  widgets  can  be  implemented  on  Family  Hub.  

§  Developer  may  show  ApplicaEon  Icon  or  Widget  Icon  

on  "All  Apps’  page.  

§  Available  widget  sizes:  

1x1(144x126),    3x1(480x126)

3x3(480x426),    3x6(480x876)

6x3(984x426),    6x6(984x876)

10

Brief  UX  Guide:  Icons  §  Developer  may  show  ApplicaEon  Icon  or  Widget  Icon  on  “All  Apps”  page.  §  If  an  applicaEon  has  widgets,  widget  icon  also  must  be  included.  

ApplicaLon    without  widget

 How  applicaLon  and  widget  icons  look  in  

‘All  Apps’  (applicaLon  tray). ApplicaLon    

with  1x1  widget

11

2.  Supported  APIs

12

Supported  Tizen  3.0  APIs  on  Family  hub NaLve  API    Supported  

Account   All  

ApplicaEon  Framework   All  

Base   All  

Content   All  

Context   All  

LocaEon   Map  Service  

Messaging   Email,  Push  

MulEmedia   All  (excl.  Radio),  Camera  (limited)  

Security   All  

Network   All  (excl.  NFC,  Smart  Card,  Cellular)  

Social   All  

System   All  

Telephony   None  

UI   All  

UIX   All  

Web   All  

Web  API    Supported  

Base   All  

Account   All  

ApplicaEon  Framework   All  

Content   All  

Messaging   Push  

MulEmedia   All  (except  FM  Radio)  

Network   All  (excl.  Cellular,  Smart  Card)  

Social   All  

System   All  

13

Refrigerator  Control  APIs  (1/2) §  3  API  Groups  :  Set,  Get,  NoEfy.  

–  Get  /  Set    -­‐  obtain  or  change  refrigerator  features.  –  NoEfy  –  register  callbacks  to  get  noEficaEon  on  alarm  or  value  change.  

Group   Feature   Sub  Feature   A`ribute   DescripLon  

Set  

Temperature  Unit   -­‐   {C  ,  F}   Unit  used  to  display  temperature  (Celsius  or  Fahrenheit)  

Desired  Temperature  Fridge   (integer  value)   Set  desired  temperature  for  fridge  room  

Freezer   (integer  value)   Set  desired  temperature  for  freezer  room  

Mode  SelecLon   FlexZone   {Freezer,  Soo  Freeze,    Meat/Fish,  Cheese/Vegetable,  White  Wine}   Set  mode  of  FlexZone  (if  available)  

Power  Fridge   {Start,  Stop}   AcEvate  power  cooling  of  fridge  room  

Freezer   {Start,  Stop}   AcEvate  power  freezing  of  freezer  room  

Ice  Maker  Power   {On,  Off}   Turn  on  Ice-­‐Maker  

Ice  Type   {Cubic,  Crushed}   Set  ice  type  (cubic,  crushed)  

Dispenser  Lock   {On,  Off}   Lock/Unlock  Dispenser  

Light   {On,  Off}   Turn  dispenser  light  on/off  

Inner  Light  Fridge  

{On,  Off}  Turn  fridge  room  light  on/off  

Freezer   Turn  freezer  room  light  on/off  

14

Refrigerator  Control  APIs  (2/2) Group   Feature   Sub  Feature   A`ribute   DescripLon  

Get  

Temperature  Unit   -­‐   {C  ,  F}   Get  current  temperature  unit  

Current  Temperature  Fridge   (integer  value)   Get  actual  (current)  temperature  of  fridge  room  Freezer   (integer  value)   Get  actual  (current)  temperature  of  freezer  room  

Desired  Temperature  Fridge   (integer  value)   Get  desired  (user  set)  temperature  of  fridge  room  Freezer   (integer  value)   Get  desired  (user  set)  temperature  of  freezer  room  

Door  Fridge  

{Open,  Close}  Get  door  state  of  fridge  room  (open  or  closed)  

Freezer   Get  door  state  of  freezer  room  (open  or  closed)  Mode   FlexZone   {Mode}   Get  current  mode  of  FlexZone  

Power  Fridge   {Start,  Stop}   Get  current  state  of  power  cooling  (fridge  room)  Freezer   {Start,  Stop}   Get  current  state  of  power  freezing  (freezer  room)  

Ice  Maker  Power   {On,  Off}   Get  current  state  of  power  (on  or  off)  Status   {Making,  Not  Making}   Get  current  state  of  ice  making  (making  or  not  making)  Type   {Cubic,  Crushed}   Get  current  type  of  ice  (cubic  or  crushed)  

Inner  Light  Fridge  

{On,  Off}  Get  inner  light  state  of  fridge  room  

Freezer   Get  inner  light  state  of  freezer  room  Filter   CondiEon   {Good,  Fair,  Need  Change}   Get  current  condiEon  of  dispenser  filter  

NoEfy  

Desired  Temperature  Fridge   (integer  value)   NoEfy  upon  desired  temperature  change  of  fridge  room  

Freezer   (integer  value)   NoEfy  upon  desired  temperature  change  of  freezer  room  

Current  Temperature  Fridge   (integer  value)   NoEfy  upon  actual  temperature  change  of  fridge  room  

Freezer   (integer  value)   NoEfy  upon  actual  temperature  change  of  freezer  room  

Door   -­‐   (state:  Open  or  Closed)   NoEfy  upon  door  opened  or  closed  Ice  Maker   Filter   (State:  Need  Change)   NoEfy  upon  filter  change  required  (condiEon  changed)  Mode   FlexZone  (Pantry)   (Selected  mode)   NoEfy  upon  FlexZone  mode  change  

Power  Fridge   (State:  On/Off)   NoEfy  upon  acEvaEon  or  deacEvaEon  of  power  cooling  Freezer   (State:  On/Off)   NoEfy  upon  acEvaEon  or  deacEvaEon  of  power  freezing  

15

How  to  Create  Handler  &  Register  Callbacks §  We  use    refrigerator_create  ()  to  create  handle  (instance),  refrigerator_set_noti_cb() is  used  to  register  callbacks,  here  we  registered      

call_noti_cb()  funcEon  to  receive  all  types  of  callbacks  one  by  one.  §  NoEficaEon  callback  receives  both  type  and  data  of  noEficaEon    and  refrigerator  control  provides  19  types  of  callbacks  which  are  listed  below.    

static bool app_create(void *data) {   appdata_s *ad = data;   int ret = refrigerator_create(&(ad->handle));   if (ret != REFRIGERATOR_ERROR_NONE)   return false;   for (int i = 0; i < (sizeof(noti_tbl)/sizeof(int)); i++) {   ret = refrigerator_set_noti_cb(ad->handle, noti_tbl[i], call_noti_cb, ad);   if (ret != REFRIGERATOR_ERROR_NONE)   return false;   }   ...   return true;  } void call_noti_cb(refrigerator_h handle, refrigerator_noti_e noti_id, void *data, void *user_data)  {   appdata_s *ad = user_data;   int value = *((int*)data);   switch(noti_id) {   case REFRIGERATOR_NOTI_TEMPERATURE_UNIT:   // DO SOMETHING HERE   break;   } . . .  }  

static int noti_tbl[] = {   REFRIGERATOR_NOTI_TEMPERATURE_UNIT,   REFRIGERATOR_NOTI_FRIDGE_CURRENT_TEMPERATURE,   REFRIGERATOR_NOTI_FRIDGE_DESIRED_TEMPERATURE,   REFRIGERATOR_NOTI_FRIDGE_POWER_COOL_STATE,   REFRIGERATOR_NOTI_FRIDGE_DOOR_STATE,   REFRIGERATOR_NOTI_FRIDGE_INNER_LIGHT_STATE,   REFRIGERATOR_NOTI_FREEZER_CURRENT_TEMPERATURE,   REFRIGERATOR_NOTI_FREEZER_DESIRED_TEMPERATURE,   REFRIGERATOR_NOTI_FREEZER_POWER_FREEZE_STATE,   REFRIGERATOR_NOTI_FREEZER_DOOR_STATE,   REFRIGERATOR_NOTI_FREEZER_INNER_LIGHT_STATE,   REFRIGERATOR_NOTI_FLEXZONE_MODE_CHANGE,   REFRIGERATOR_NOTI_FLEXZONE_DOOR_STATE,   REFRIGERATOR_NOTI_FLEXZONE_INNER_LIGHT_STATE,   REFRIGERATOR_NOTI_DISPENSER_LOCK_STATE,   REFRIGERATOR_NOTI_DISPENSER_LIGHT_STATE,   REFRIGERATOR_NOTI_DISPENSER_FILTER_CONDITION,   REFRIGERATOR_NOTI_ICEMAKER_STATUS,   REFRIGERATOR_NOTI_ICEMAKER_ICE_TYPE,  };  

16

How  to  Read  Temperature  Unit

static void read_temperature_unit(appdata_s *ad) {   int ret, unit = 0;   ret = refrigerator_get_temperature_unit(ad->handle, (refrigerator_temp_unit_e *)&unit);   if (ret != REFRIGERATOR_ERROR_NONE)   dlog_print(DLOG_ERROR, LOG_TAG, "Failed to get temperature unit (ret = %d)", ret);     if (unit == 0) dlog_print(DLOG_DEBUG, LOG_TAG, "Temperature unit is Celsius (°C)");   else dlog_print(DLOG_DEBUG, LOG_TAG, "Temperature unit is Fahrenheit (°F)"); . . .  } static void app_terminate(void *data)  {   appdata_s *ad = data;     for (int i = 0; i < (sizeof(noti_tbl)/sizeof(int)); i++)   refrigerator_unset_noti_cb(ad->handle, noti_tbl[i]);     refrigerator_destroy(ad->handle);  }  

§  To  get  temperature  unit  we  call  refrigerator_get_temperature_unit() by  passing  pointer  for  unit.  §  When  app  is  terminated  we  need  to  unset  callbacks  with  refrigerator_unset_noti_cb()  funcEon  and  destroy  handle  

(instance)  with  refrigerator_destroy().    

17

How  do  the  Family  Hub  Voice  Features  work?

OpLmized  for  Family  Hub

•  In-­‐app  voice-­‐control  interacEon  can  be  developed  using  Tizen  3.0  Voice  APIs.  

•  Voice  service  provides  informaEon  to  an  applicaEon  about  user’s  proper  meaning  as  User  speech.  

•  ApplicaEon  developer  needs  to  implement  funcEons  that  processes  user  intent.  

18

Voice-­‐control  workflow  in  applicaLon

Prepare  steps

Wakeup&  RecogniLon    steps

19

Voice  APIs  (1/2)   §  The  Voice  Control  API  is  provided  by  Tizen  3.0.    

You  can  find  the  API  Reference  and  Sample  code  on  Tizen  Developer  Site  (hqps://developer.Ezen.org).    ◦  Voice  Control:  hqp://bit.ly/2pS6lTr  ◦  Text-­‐to-­‐Speech:  hqp://bit.ly/2qyOQUu  ◦  Speech-­‐to-­‐Text:  hqp://bit.ly/2p3sDO7  

§  Voice  Parser  -­‐    To  parse  the  JSON  data  use  provided  Parser  API  to  get  the  proper  data  easily.  

§  Voice  DB  -­‐  To  understand  user  intent  about  ongoing  speech,  the  current  state  of  applicaEons  is  monitored.    To  correctly  idenEfy  current  state  your  applicaEon  needs  to  update  its  state  with  following  APIs:

int  sdp_response_data_parse(const  char  *raw_json,  sdp_response_data_h  *response_data);  int  sdp_response_data_get_agent(sdp_response_data_h  response_data,  char  **agent); int  sdp_response_data_get_action_name(sdp_response_data_h  response_data,  char  **action_name);  int  sdp_response_data_get_action_data(sdp_response_data_h  response_data,  sdp_action_data_h  *action_data); ...  int  sdp_response_data_destroy(sdp_response_data_h  response_data);

int  sdd_manager_create(sdd_manager_h  *manager,  const  char  *agent_name)  int  sdd_manager_insert(sdd_manager_h  manager,  sdd_bundle_h  bundle)  int  sdd_manager_delete(sdd_manager_h  manager,  sdd_bundle_h  bundle)  int  sdd_manager_update(sdd_manager_h  manager,  sdd_bundle_h  bundle)  int  sdd_manager_foreach_table(sdd_manager_h  manager,  const  char  *agent_name,  sdd_database_cb  callback,  void  *user_data);  ...  int  sdd_manager_destroy(sdd_manager_h  manager)  

20

Voice  APIs  (2/2)  :  How  to  register  custom  command   §  The  ApplicaEons  want  to  use  special  domains  (Recipe,  Music)  of  Samsung  Voice  Server  need  perform  following:  

§  Register  the  agent  name  when  set  the  command  list  by  using  Voice  Control  API.    §  When  the  Voice  Control  state  is  changed  from  iniEalized(VC_STATE_INITIALIZED)  to  ready(VC_STATE_READY),  set  the  command  

like  below.    

vc_cmd_h  vc_command  =  NULL; int  ret  =  vc_cmd_create(&vc_command); if  (VC_ERROR_NONE  !=  ret)      _E("vc_cmd_create() failed: [%s]",  _translateErrorToString((vc_error_e)  ret));  /* Set Command */ _D("Set VC Command"); ret  =  vc_cmd_set_command(vc_command,  “#org.tizen.recipe”); if  (VC_ERROR_NONE  !=  ret)      _E("vc_cmd_set_command() failed: [%s]",  _translateErrorToString((vc_error_e)  ret));  /* Set Command Type */ _D("Set VC Command Type"); ret  =  vc_cmd_set_type(vc_command,  VC_COMMAND_TYPE_BACKGROUND); if  (VC_ERROR_NONE  !=  ret)      _E("vc_cmd_set_command() failed: [%s]",  _translateErrorToString((vc_error_e)  ret)); /* Set Command Format */ _D("Set VC Command Format"); ret  =  vc_cmd_set_format(vc_command,  VC_CMD_FORMAT_ACTION); if  (VC_ERROR_NONE  !=  ret)      _E("vc_cmd_set_format() failed: [%s]",  _translateErrorToString((vc_error_e)  ret));

register  the  agent  name

21

Family  Hub  User  (Profile)  &  Cloud  APIs §  Family  Hub  server  provides  User  (Profile)  APIs  to  manage  

family  members  and  share  contents  within  the  family  members.  

§  User  (Profile)    is  created  with  Samsung  Account,  and    has  2  types:  

•   Primary  User  –  One  master  user,  and  can  add  other  members.  

•   Secondary  User  -­‐    member  of  the  family.  §  Following  server-­‐based  RESTful  APIs  are  available  

•  Profile  API  /  Calendar  API  •  Recipe  API  /  Push  API  

§  Samsung  Cloud  can  be  connected  with  user  profile  •  it  is  possible  to  share  pictures,  music,  notes,  etc.  

Samsung    Account

Family  Hub  Server

Samsung    Cloud

User  (Profile)  API

22

OCF  Stack

Easy  Setup  

Cloud  Manager

Server  Builder

ApplicaEon  Programming  Interface  (API)

IoTivity  (CoAP/DTLS)

Request  Handler

ApplicaEons

Network  Sub-­‐System(IPv4/WIFI  802.11)

Device  Discovery

Device  Manager

Family  Hub  OCF  APIs  §  Family  hub  support  Open  IoT  Plauorm,  Open  ConnecEvity  FoundaEon    (OCF)  §  Family  Hub  device  can  serve  as  both  controller  and  controlee.  

-­‐  Controlee:  Family  Hub  can  be  controlled  from  mobile  devices  using  OCF  interface.  

 -­‐  Controller:  Family  Hub  can  discover  and  manage  other                OCF-­‐enabled  devices.  

23

3.  Running  Apps

24

§  ApplicaEons  can  be  installed  and  run  on  Family  Hub  using  Tizen  SDK  or  command  line  development  tools  (sdb).  §  In  the  ConnecLon  Explorer  view,    select  the  device.  §  In  Project  Explorer  view,  right-­‐click  the  project  and  select  Run  As  >  Tizen  NaLve  ApplicaLon.    

§  Remote  Targets  will  be  available  with  Remote  Test  Lab.  

Target  Environment    

sdb

25

§  Family  Hub  emulator  is  not  included  in  Tizen  SDK,  but  emulator  can  be  configured.  1.   Run  emulator  manager  in  Tool  menu.

2.  In  the  Emulator  Manager,  click  the  “Create”  bu`on.

3.  Select  mobile-­‐3.0-­‐i386  (basic)  as  default  plakorm

Family  Hub  Emulator  (1/2)  

26

4.  Click  on  “+”  bu`on  to  create  a  new  emulator  template  and  configure  it.

5.  Configure  templates  as  below  figure,  i.e.,  set  resoluLon  to    Full  HD  (1920x1080),  set  Size  to  21.5  inches,  remove  all  sensors  except  Proximity  sensor  since  they  are  not  available  on  Family  Hub.  

Family  Hub  Emulator  (2/2)  

27

Summary  &  Future  Plan  

§  Tizen  is  broadly  used  on  Home  Appliances.  §  Family  Hub  is  going  to  be  reference  home  appliance  device  for  smart  home  services.  §  Tizen  on  Family  Hub  provides  APIs  for  development  of  rich  service  applicaLons.  

§   Samsung  Cloud,  Profile  APIs  §   Device  &  Voice  Control  APIs  §   Open  ConnecLvity  FoundaLon  (OCF)  APIs  

§  Tizen  SDK  for  Family  Hub      §   Available  on  developer.samsung.com  (June/2017)  §   Remote  Test  Lab  (3Q/2017)  

§  App  Store  &  3rd  Party  ApplicaLons  (near  future)

Family  Hub

28 Thank you! Any Question? ms_77.kim@samsung.com e.talipov@samsung.com

Get  connected,    get  closer  to  your  family