STM32 iPod iPhone Accessories Library - Presentation v0.2

STM32 – iPod/iPhoneAccessories Library

General PresentationGeneral Presentation15th November Draft 0.2

iPod application - Examples

2

How does it work? – 1/2

� Licensing plan:� Customer must be Apple MFi licensee* to get:

� “Authentication Coprocessor” (ACP) mandatory� 30-pin male connectors� Testing/certification tools and softwares� A customer notice must be signed and returned to division � The customer must also be registered on the audio

SharePoint by local ST Sales and marketing people.� After completion of these last 2 steps, the STM32

iPod/iPhone Accessories software can be delivered to the customer. More details will be provided shortly by the division.

� STMicroelectronics provides Apple firmware protocol “iPod Accessory Protocol” (iAP)

3

*MFi: “Made For iPod” program – click here

How does it work? – 2/2

� iPod 30-pin connector can provide audio data in different formats:� Digital audio:

� USB host core� USB Audio class for PCM audio data� USB HID class for control and information data� USB HID class for control and information data

� Analog audio:� UART communication for control and information� Direct output of analog audio signals on the 30-pin connector

4

iPod application features

iPod Accessory Protocol provided by ST:

� Full compatibility * through iPod/iPhone/iPad families� IDPS** authentication 2.0B� Digital audio (USB 2.0 FS)� Digital audio (USB 2.0 FS)� Analog audio (UART)� Automatic switch from USB to UART communication to

support older iPod

5

**IDPS: “Identify Device Preferences and Settings” mandatory for any new iPod accessory

* : Fully compatibility : the API/library is based on Apple_Release_2010-9. Full Testing, characterization is on-going

256 Kbyte Flash

Firmware summary

Audio

Cortex™ -M3 CPU

71.88 MHz

64 Kbyte SRAM

DMA

iAP StackVolume/ Ch Mixer

5 bands - Equalizer

Loudness Control

Audio Class* AudioDAC

Amp

14.7456 MHz quartz

SPI

PLL block

XTAL oscillators32 kHz + 3-25 MHz

TouchScreenQVGALCD

Speakers / Headset

USB

I²S

XXX

XXX *

FW/API from ST

Licensed FW fromHCC embedded

5 bands - Equalizer

HMI Control& Display

UART

Authentication Coprocessor

I2C

HID Class*

Audio Class*

FS USB Host *

iPod/iPhone 30-pin connector

iPod demoboard overview

7

Connectivity STM32F105 and iPod

iPod/iPhone 30-pin connector

USB Host

I2C

USB Host Connector STM32F105

Connectivity Line

Am

plifi

er

8

Authentication Coprocessor

DAC

I2C

I2S

USART I2C

Audio stream Control data

Am

plifi

er

Suitable STM32F105 products:-STM32F105RCT6: 256K/64pins-STM32F105RBT6: 128K/64pins-STM32F105VCT6: 256K/100pins-STM32F105VBT6: 128K/100pins

iPod “docking” demoboard

1. Extension connector: straightforward connectionto STM3210C eval board via CN1 and CN2

Key Features:

4

3. iPod/IPhone connector

2. iPod/iPhone Authentication Coprocessor 2.0B class 6 (ACP)

6

9

4. Hardware jumpers setting •I2C address selection for ACP module•Accessory Identify : UART or USB•Power select: iPod/iPhone board power source from iPod or STM3210C eval board

6. Analog Audio

•Input analog: Microphone•Audio Pre-amplifier module•Jack connector (analog audio output from iPod)

1

23

5. Video extension connector

5

STM32 iPod/iPhone control Ref Design (1/2)*MP3 MixerMedia Player

WAV, MP3,*WMA

Media Browser

*MP3 Encoder

Audio UtilitiesLoudness5 bands Equalizer

*Karaoke Game

Info & iPod Control

0

20

40

60

80

100 Volume ControlUsing Software Mixer

Title : Chariots Of fireArtist : VangelisAlbum :STMicroelectronicsGenre : InstrumentalYear : 1981

Tags DisplayMP3

0:25 0:30ProgressiveProgress Bar

Play Stop Browser

R

Repeat

Home

MuteMute Control

Cursor (Multi-Input) Joystick & Touch

Audio Player

GameInfo & Credits

iPod Control Play StopFast

Forward/Backward

Browser RepeatPause

Quit

Run-time UtilitiesWhile playing

1KHz

5dB

2.5dB

0dB

2.5dB

5dB

Audio Utilities

3KHz 6KHz 12KHz

Min Gain

0

5 Bands Equalizer Loudness Control

Max Gain

5dB

2.5dB

0dB

2.5dB

5dB

5dB

2.5dB

0dB

2.5dB

5dB

5dB

2.5dB

0dB

2.5dB

5dB

5dB

0dB

5dB

2.5dB

2.5dB

18KHz

Volume ControlUsing Software Mixer

0

20

40

60

80

100

Title : Chariots Of fireArtist : VangelisAlbum :STMicroelectronicsGenre : Instrumental

Song Info


Play StopFast

Forward/Backward

R

Repeat

Home

MuteMute Control

Pause


iPod Player

Next/Previous Track Either Loudness Control Or 4 to 5 bands Equalizers

* Unavailable feature

0

20

40

60

80

100 Volume ControlUsing Software Mixer

Title : Chariots Of fireArtist : VangelisAlbum :STMicroelectronicsGenre : InstrumentalYear : 1981

Tags DisplayMP3


RHome

MuteMute Control


Audio Player

STM32 iPod/iPhone control Ref Design (2/2)

11

� iPod control is using digital audio (USB communication)� This demo is NOT using any OS, but based on interrupts priorities� MP3/WAV decoding, EQ, volume control, are all done by software

within the STM32

Play StopFast

Forward/Backward

Browser RepeatPause

iPod firmware mechanism - USB data

Mem to I2S transfer

USB high

Audio post processing

HCC USB audio class driver

SOF period

DMA transferDMA transfer DMA transferDMA transfer DMA transferDMA transfer DMA transferDMA transfer DMA transferDMA transfer

Priority Transfer complete

12

IOE

Main loop

priority tasks

Background tasks: (iAP stack and GUI)Asynchronous input events (Joystick & touch screen)

Volume regulation with equalizer or Loudness control when enabled.

N x SOF Period

iPod demo firmware memory footprintTotal Demonstration requirements:

Flash: 171.21 Kbytes.Read Only data: 62.82 Kbytes.Code size: 108.39 Kbytes.RAM: 63.68 Kbytes.Heap: 28 Kbytes.Stack: 4Kbytes.Read Write Data: 31.68 Kbytes.

HCC stack resources:Flash: 37.34 Kbytes.

Code: 37.16 Kbytes.RO Data: 0.18 Kbytes.

RAM : 28.9 Kbytes.

MP3 Decoder resources:Flash: 22Kbytes

Code: 15 Kbytes.RO Data: 7 Kbytes.

RAM (from Heap): 12.45 Kbytes.

Equalizer resources:Flash: 1.64 Kbytes


RAM : 0.29 Kbytes.

Loudness control resources:Flash: 3.16 Kbytes.


13

GUI:Flash: 19.90 Kbytes.


RAM : 5.15 Kbytes. (4 Kbytes heap, 1 Kbytes Stack).

RAM : 0.61 Kbytes. (scratch : 0.5 Kbytes, persistent 0.11 Kbytes)

Mixer resources:Flash: 0.57 Kbytes.

Code: 0.57 Kbytes.RO Data: 0 Kbytes.

RAM : 16 bytes.

IAR C-standard library:Flash: 1.55 Kbytes.

Code: 1.55 Kbytes.RO Data: 2 bytes.

RAM: 68 bytes.

FW HAL Drivers:Flash: 9.75 Kbytes.

Code: 9.71 Kbytes.RO Data: 38 bytes.

RAM: 60 bytes

Connectivity Demo:Flash: 73.208 Kbytes.


8 Pictures (48 x 48 16bits).1 picture (80 x 36 16bits).Zero filled buffer: 4 Kbytes.

RAM: 2.03 Kbytes.

iPod firmware only: Flash: 30Kbytes – RAM: 45Kbytes

Back-up Technical details

iPod player(1/8)

� iPod Player controls iPod internal player engine� Authenticate with the iPod.� Synchronize screen contents with iPod Player

(progress bar, total time, current time, song info).� Sends Play, Pause, Stop, Fast forward, Fast

Backward, Next track, Previous track and Repeat

15

Backward, Next track, Previous track and Repeat commands.

� Retrieve PCM data stream from iPod.� Regulate output stream (Volume, audio post

processing,...).

iPod player(2/8)

USB audio streaming

Stream synchronization

Monitor buffer indexes

Indexes too far/near

Yes

apply additional filters

Two methods were implemented to handle the synchronization of data transfer from iPod to on board DAC(refer to next slides)

apply additional filters (Equalizers or loudness control) and volume control

Audio data processing

Audio stream rendering

Noapply additional filters (Equalizers or loudness control) and volume control

100 x frame length Circular Buffer

Write data to circular buffer

Transfer PCM to external DAC via I2S peripheral

iPod player(3/8) : Stream synchronization

USB transfer

I2S transfer

Under run conditionI2S needs buffer 10 which is not ready

Over run conditionAccumulative error

…

� Consumer period < Producer period

1 2 3 4 5 6 7 8 9 10 11

1 2 3 104 5 6 7 8 9 11

� Two methods were implemented to handle the synchronization of data transfer from iPod to on board DAC.

USB transfer

I2S transfer

Over run conditionUSB prepared buffer 10 while I2S

is starting to transfer buffer 9

Accumulative error

…� Consumer period > Producer period

1 2 3 4 5 6 7 8 9 10 11

1 2 3 94 5 6 7 8


- Method 1 : adjust USB Start of frame (SOF) period to cancel errors and keep USB audio data streaming synchronized with I2S transfer. (default method)

- This method is enabled by default, to have High end Audio Quality 1:1 ratio ( lossless samples)


- Method 2 : a software method to synchronize I2S and isochronous transfer. When needed add one sample at the end of the buffer or remove one sample from the end of the buffer to tune I2S transfer period.

New sample (left

��Case of adding samplesCase of adding samples

++

/2

++

/2

New sample (left and right) to beadded at the end

of bufferBuffer with N x samples

Buffer with N+1 x samples

Left

Right


��Case of removing samplesCase of removing samples

N-2

N-1

N-4

N-3

N-6

N-5

N-8

N-7

One Sample (left and right) to beremoved at the end of buffer

Left

Right

/4

WW

N-5

/4

N-6

WW

iPod player(7/8)

Mem to I2S transfer

USB high

Audio post processing

HCC USB audio class driver

SOF period

DMA transferDMA transfer DMA transferDMA transfer DMA transferDMA transfer DMA transferDMA transfer DMA transferDMA transfer

Priority Transfer complete

21

IOE

Main loop

USB high priority tasks

Background tasks: (iAP stack and GUI)Asynchronous input events (Joystick & touch screen)

Volume regulation with equalizer or Loudness control when enabled.

N x SOF Period

Priority

USB high priority tasks

USB High priority tasks call HCC Audio task & HCC transfer task

- USB tasks scheduling avoid “mutex dead locks” and “concurrency on the USB bus”.

iPod player(8/8)

USB data out

priority tasks

Preemption time

Blocked time

Running time

HID report are sent only after higher priority tasks execution

Time

Known limitation(1/5)

� iPod tracks Tags: Only the Title, Artist, Album and Genre tags are extracted.� For Demo purposes.

� iPod control application supports only iPod Accessory Protocol small packets.Accessory Protocol small packets.� For Demo purposes.

� Audio quality may be not good with on board speaker, it’s due to the on board speaker quality. For a better experience please use a headphone.

23


� iPod control track information (stream length, Title, Artiste, Album and Genre) are retrieved when iPod sends new chapter or new track event notification.

� iPod control streaming : supported sampling rate � iPod control streaming : supported sampling rate are 48KHz, 44.1 KHz and 32KHz as in Apple specifications.

� Next/Previous track buttons allow navigation within the selected data base records.

24


� When MP3 player demo detect Bit rate variable (VBR) it will reset:� Time progress to 0:00� Stream Time length to 0:00� Then "VBR Detected." Message will be displayed.

� Media browser: If the mass storage device � Media browser: If the mass storage device contains a big number of files some of them may not be displayed.� This is related to the heap space allowed for GUI

items and files/folders list. items and list are created dynamically (dynamic memory allocation).

25


� Maximum FIFO size set to 512 bytes: a received packet larger than 512 bytes will be improperly handled.

� Supported lingoes and minimal versionsLingoes Minimal lingo version

26

Lingoes Minimal lingo version

General Lingo 1.02

Simple remote lingo 1.00

Display remote lingo 1.02

Digital audio lingo 1.02[1]

1. iPod is not switched to extended mode (please refers to iPod Accessory protocol specification for more info)


� Firmware design switch iPod to USB configuration 2 (composite USB device): � iPod mass storage feature is unavailable when

connected to the STM32 docking station.

� I2S and USB clock synchronization: � I2S and USB clock synchronization: � Synchronization method which adjust USB start of

frame (SOF) packet period produce better audio quality, however this regulation method produce time lag in SOF period from standard 1ms period > 0.05% due to our I2S drift at 44.1KHz

27

Documents

STM32 iPod iPhone Accessories Library - Presentation v0.2