Introduction to Embedded Systems

Spring 2014SILICON VALLEY UNIVERSITY

CONFIDENTIAL 1

Introduction to Embedded Systems

Dr. Jerry Shiao, Silicon Valley University

SILICON VALLEY UNIVERSITY CONFIDENTIAL 2Spring 2014

Section 11 Storage Devices Embedded System Solid-State Storage Devices

Uses Solid-State Storage Devices ( i.e. ROM, NOR / NAND flash chips ).

How to effectively use each type of device? Each device has own capabilities, particularities, and limitations.

Tools specific to the device type. Used as Block Device?

Block Device layer can emulate a disk file system on Flash Device, but sub-optimal.

Erasing blocks: Flash memory blocks have to be erased before they can be used. Significant overhead, should be done when device is idle.

Random access: Disk file systems optimized to avoid disk seeks. Flash memory has no seek latency.

Wear leveling: Flash memory device wear out when a single block is repeatedly

overwritten. Flash File Systems spreads out writes evenly.


Section 11 Storage Devices Embedded System Solid-State Storage Devices Memory Technology Device ( MTD ) Subsystem

Provide common capabilities among various technologies. Controller:

Perform wear leveling, bad block recovery, power loss recovery, garbage collection and error correction.

Used in removable flash memory cards and USB flash drives. Flash File System:

Perform wear leveling by spreading writes over the media. Deals with long erase times of NAND flash blocks. Update flash blocks by:

Write a new copy of changed data to a fresh flash block. Remap the file pointers. Erase the old flash block, when the Flash File System is idle.

Used in embedded flash memories ( no controller ).


Section 11 Storage Devices Embedded System Solid-State Storage Devices Memory Technology Device ( MTD ) Subsystem

Flash File System Registered as Linux device. Linux enables processes to communicate with the device driver

and hardware as file-like objects. C Library I/O Operations (i.e. open / read / write / close).

MTD User Modules Software modules in Kernel that access the low-level MTD chip drivers. Chip drivers: DiskOnChip, NAND, RAM, ROM, CFI-Compliant Flash. Provides recognizable interfaces and abstractions. MTD /dev entries corresponds to MTD User Modules.

mtdN: Each entry is a separate MTD device or partition. Each MTD

partition acts as a separate MTD device. mtdrN:

Each entry is the read-only equivalent of the matching /dev/mtdN. mtdblockN

Each entry is the block device equivalent of the matching /dev/mtdN.


Section 11 Storage Devices MTD Subsystem Architecture


Section 11 Storage Devices MTD Subsystem Architecture MTD User Modules

JFFS2: Journaling file system having all the properties for a flash file system.

YAFFS: Journaling file system designed specifically to work within the constraints of

and exploit features of NAND flash. Char Device:

“Raw” device character access offering basic read and write functionality. Ioctl access to the erase function and other flash functionality.

Block Device: “Raw” device block access offereing basic read and write functionality. Ioctl access to the erase function and other flash functionality.

Flash Translation Layer ( FTL ): Not File System Interface. Used for compatibility with existing devices in the field. Allows flash memory devices to be used as standard Block Device. Provides wear levelling, powerfail resilience, and mapping out bad blocks. Adapts a fully functional file system to the constraints and restrictions

imposed by flash memory devices.


Section 11 Storage Devices MTD Subsystem Architecture MTD Chip Drivers

DiskOnChip (DoC) Flash: Flash Translation Layer providing a disk interface. Internally, a DoC module

contained a controller that implemented ECC, bad block re-mapping and wear leveling functions that were used to implement a file system.

CFI-Compliant Flash: Common Flash Memory Interface ( CFI ) is an open industry standard.

Flash chip will have information identifying : memory size, byte/word configuration, block configuration, voltages and timings.

Common solution, using JFFS2. JEDEC-Compliant Flash:

Standardization body developing standards for semiconductor devices and has adapted open industry standard CFI.


Section 11 Storage Devices MTD Subsystem Architecture


Section 11 Storage Devices MTD Subsystem Configuration Kernel Configuration Options.

make menuconfig: Enter MTD Submenu. CONFIG_MTD:

Include core MTD Subsystem support. CONFIG_MTD_CONCAT:

Combine multiple MTD devices or partitions into a single logical device (i.e. combine space from two or more separate devices into a single filesystem).

CONFIG_MTD_PARTITIONS: Divide MTD devices into separate partitions.

CONFIG_MTD_CHAR: Enables char device MTD User Module.

CONFIG_MTD_BLOCK: Enables block device MTD User Module.

CONFIG_MTD_BLOCK_RO: Enables read-only block device MTD User Module.

CONFIG_FTL: Enables Flash Translation Layer User Module in the Kernel.

CONFIG_NFTL: Enables NAND Flash Translation Layer in the Kernel.

CONFIG_NFTL_RW: Enables write to NFTL-formatted devices.


Section 11 Storage Devices MTD Subsystem Configuration Kernel Configuration Options.

make menuconfig: RAM/ROM/Flash Chip Drivers Submenu. Detect flash chips by CFI probe.

make menuconfig: Mapping Drivers for Chip Access Submenu. The CFI flash device-mapping driver for the board.


Section 11 Storage Devices MTD Subsystem MTD Partitioning

CFI Flash cannot be partitioned with fdisk utility ( for block devices only ). Partition Information is hardcoded in the mapping driver or board’s device tree. Mapping driver initialization will register partition with MTD subsystem.

static struct mtd_partition tqm8xxl_partitions [ ] = {

{ name: “ppcboot”, /* PPCBoot Firmware */offset: 0x00000000,size: 0x00040000,

}, { name: “kernel”, /* Default Kernel Image. */

offset: 0x00040000,size: 0x000C0000,

}, { name: “user”

offset: 0x00100000,size: 0x00100000,

}, { name: “initrd”, /* Ramdisk. */

offset: 0x00200000,size: 0x00200000,

}};


Section 11 Storage Devices MTD Subsystem Linux Kernel

~ # ls /devX0R loop3 mtd6 mtdr3 ptyp4 rtc0 ttyAM0audio loop4 mtd7 mtdr4 ptyp5 sda ttyAM1console loop5 mtdblock0 mtdr5 ptyp6 sda1 ttyp0core loop6 mtdblock1 mtdr6 ptyp7 sda2 ttyp1cuam0 loop7 mtdblock2 mtdr7 ram sda3 ttyp2cuam1 mem mtdblock3 nfsd ram0 sdb ttyp3dsp mixer mtdblock4 null ram1 sdb1 ttyp4fd mtd0 mtdblock5 ptmx ram2 sdb2 ttyp5kmem mtd1 mtdblock6 pts ram3 sdb3 ttyp6log mtd2 mtdblock7 ptyp0 ramdisk stderr ttyp7loop0 mtd3 mtdr0 ptyp1 random stdin urandomloop1 mtd4 mtdr1 ptyp2 rawctl stdout zeroloop2 mtd5 mtdr2 ptyp3 rtc tty~ #


Section 11 Storage Devices MTD Subsystem Linux Kernel

~ # cat /proc/mtddev: size erasesize namemtd0: 00040000 00020000 "RedBoot"mtd1: 00200000 00020000 "ramdisk"mtd2: 00180000 00020000 "zImage"mtd3: 00001000 00020000 "RedBoot config"mtd4: 00020000 00020000 "FIS directory"~ #

Bootloader: RedbootRedBoot> fis listName FLASH addr Mem addr Length Entry pointRedBoot 0x60000000 0x60000000 0x00040000 0x00000000RedBoot config 0x603C0000 0x603C0000 0x00001000 0x00000000FIS directory 0x603E0000 0x603E0000 0x00020000 0x00000000ramdisk 0x60040000 0x00800000 0x00200000 0x00800000zImage 0x60240000 0x00080000 0x00180000 0x00080000RedBoot>

MTD partitioning is exposed to user space as separate MTD devices ( mtd0, mtd1, etc ).


Section 11 Storage Devices MTD Subsystem MTD Utilities: Used for all types of MTD devices (/dev/mtdN)

flash_info device: Provides information regarding device’s erase region. flash_erase device start_address number_of_blocks: Erase a certain number

of blocks from a device starting at a given address. flash_eraseall [options] device: Erase the entire device. flash_unlock device: Unlocks all the sectors of a device. flash_lock device offset number_of_blocks: Locks a certain number of blocks

on a device. flashcp [options] filename flash_device: Copies a file to a flash device. mtd_debug operation [operation_parameteres]: MTD Debugging options.

MTD Filesystem Creation: mkfs.jjfs2 [options] -r directory -o output_file: Builds a JFFS2 filesystem. jffs2dump [options] image: Dumps the contents of a binarty JFFS2 image.

NTFL Tools: nftl_format device [start_address[size]]: Formats a DisckOnChip device.

FTL Tools: ftl_format [options] device: Formats a NOR flash device with FTL.

NAND Tools: nandwrite device input_file_start_address: Writes content of a file to a NAND

chip.


Section 11 Storage Devices Root Filesystem Setup the Root Filesystem:

1) Select Filesystem type.2) Pack the Filesystem’s content (files and directories).3) Write the Filesystem to a storage device on the target.

Characterizing Filesystems Online Write Support while in-use.

Most filesystems support online updates: adding, deleting, and altering files.

Persistence. Contents of filesystem must be preserved in-between system reboot.

Compression. Auto compress some or all of the data and metadata before written to

storage device. Auto decompress when read from storage device. Compress/decompress at block level, not at file. Overhead of decompress occurs at the block, once cached in Linux buffer

cache. Filesystems Wear Leveling algorithm more reliable than FTL, NFTL.


Section 11 Storage Devices Root Filesystem Characterizing Filesystems

Power-down Reliability. Power failure does not corrupt Filesystem information and metadata. Journaling Filesystem:

Maintains log of the modifications done as part of each transaction in the Filesystem.

Information on the log is deleted, when all of transaction’s operation are completed.

Physical Journal: Contains copies of blocks that are to be written to the storage device. Two copies made, once to journal, again to the storage device location.

Logical Journal: Compact information about the changes. Double writes, but not to the extent of Physical Journaling.

Log-Structured Filesystem: All data and metadata kept in the Filesystem are stored in a log structure that spans the storage device.

Write are transactions to the log, and log entries can be invalidated.

Similar to Physical Journal, without double-write penalty.


Section 11 Storage Devices Root Filesystem Filesystem Types Ext2 ( Second Extended Filesystem )

Built for Block Devices. Fast, supports 16 or 32 Tbytes of storage. Used for:

RAMDisk Read/Write mode. CompactFlash Device in IDE mode. NAND or NOR Flash in Read-Only mode. MTD Subsystem Raw Block

emulation or FTL/NFTL Translation Layer. Limitations:

Does not have power-down reliability. mke2fs command

dd if=/dev/zero of=initrd.img bs=1k count=4096 /sbin/mke2fs -F -v -m0 initrd.img mkdir initrd.dir mount -o loop initrd.img initrd.dir (cd initrd; tar -cf - .) | (cd initrd.dir ; tar -xvf -)


Section 11 Storage Devices Root Filesystem Filesystem Types Ext3 ( Third Extended Filesystem )

Adds Reliability to Ext2 via Journaling. Journals only Filesystem Metadata (i.e. File Attributes or inodes).

File updates does not go through Journaling. Mount option “data=journal” to journal file data updates also.

Used with: CompactFlash devices in IDE mode. NAND or NOR flashes via FTL/NFTL Translation Layer.

mount -t ext3 /dev/hda1 /media/cf -o noatime,data=journal Options specify journaling data updates and to not update inode access time

for each access.


Section 11 Storage Devices Root Filesystem Filesystem Types Cramfs

Simplistic , Compressed, Read-Only Filesystem for Embedded Systems. Max size of a file is 16 Mbytes. Used for:

Read-Only on NOR and NAND flash via “raw” MTD Block Device Emulation. NAND or NOR via FTL or NFTL Translation Layers. RAMDisk Image.

Limitations: Max size of a file is 16 Mbytes. Kernel Page size must be 4096 Bytes. Group-ID field is 8 bits, or 255 max groups.

$ mkcramfs rootfs/ images/cramfs.img Compresses the Filesystem (typically 50%).

# dd if=rootfs/cramfs.img of=/dev/mtd4 bs=1024 Write to the Storage Devce.


Section 11 Storage Devices Root Filesystem Filesystem Types Squashfs

Compressed, Read-Only Filesystem for Embedded Systems. Data, inodes, and directories are compressed. Full UID/GID (32 bits). Files up-to 2^64 Bytes are supported. Filesystem up-to 2^64 Bytes are supportedl Block sizes up-to 64KBytes. Increases the compression ratios (better than

4KByte Block size. Used for:

Systems with constrained disk sizes and memory. PowerPC, i586, Sparc, and ARM Archtectures.

Limitations: Not part of the Linux Kernel tree. Installed as a patch to the Kernel. Has to be installed from Squash archives.

mksqushfs source1 source2 … destination [ options ] Source1 … Pathnames of files or directories to be added to the resulting

Filesystem . Destination … Path of the Squashfs Filesystem.


Section 11 Storage Devices Root Filesystem Filesystem Types JFFS2

Compressed, Journaling, and provides Wear Leveling. Used for:

NOR or NAND on-board flashes or Disk-on-Chip Devices. Better compression than Cramfs.

Keeps all the Filesystem information in log structure that spans entire flash storage space.

Log is nodes, each describing a single block of data or metadata. Updates to file involves creating new node with changed information. Erase Block Summary (EBS) at end of each erase block.

Updated with each write to the block. EBS read during mount, instead of scanning the entire block.

Keeps track of free space on Filesystem. Threshold reached: Background garbage-collection process of stale nodes.

mkfs.jffs2 -r rootfs/ -o images/rootfs-jffs2.img -e 128KiB -r: Location of directory containing the root Filesystem. -o: Name of output file where Filesystem is stored.


Section 11 Storage Devices Root Filesystem Filesystem Types YAFFS2 (Yet Another Flash Filing System Version 2)

Journaling, and provides Wear Leveling. Used for:

Optimized for NAND on-board flashes. Uses less RAM to hold its state, scales better than JJFS2. Better garbage-collection than JFFS2.

Limitations: Does not support compression. Not part of the Linux Kernel tree. Download and installed as a patch to the

Kernel. $ mkyaffs2image ${PROJROOT}/rootfs yaffs2.img

Creates Filesystem image (yaffs2.img) with the content from ${PRJROOT}/rootfs directory.


Section 11 Storage Devices Root Filesystem Filesystem Types Tmpfs

Read/Write Filesystem stored in Virtual Memory. Not persistent.

Uses Linux Page and Dentry Cache for storing file data and metadata. Linux Page Cache is the Kernel’s internal data structure. Linux Dentry Cache hold information about directory entries in mounted

Filesystems. Used for:

Mount directories that do not require permanent storage (i.e. /tmp directory). Kernel Configuration Submenu: Enable “Virtual Memory Filesystem Support”. # mount -t tmpfs none /tmp -o size=4m

Size=4m: Limit size of the Filesystem. If “Size” not configured, Filesystem will grow in content to half of the available RAM.


Section 11 Storage Devices Root Filesystem Choosing Filesystem Type and Layout

List type of data stored on Filesystem. Executable binary files and libraries. Configuration information.

Requirements/Characteristics: Writeable. Persistent. Power-down reliable. Compressed.

Find combination of Filesystem type and hardware storage medium. Partitioning storage device.

Different partitions on the same storage device can contain different Filesystem type.

Applications, Libraries, and Static Data Load software back into RAM after reboot, RAMDisk. Do not need to keep any changes made to the software or files in RAM. Read-Only: Cramfs or Squashfs. Read-Write: Ext2


Section 11 Storage Devices Root Filesystem Applications, Libraries, and Static Data

RAMDisk: Load software back into RAM after reboot, RAMDisk. Do not need to keep any changes made to the software or files in RAM. Read-Write: Ext2 Read-Only: Cramfs

Read-Only: NOR or NAND. “raw” MTD Block Device Emulation, FTL, NFTL. Cramfs, Squashfs.

Dynamic Configuration Files and Data: Persistent, writable, and power-down reliable. NOR or NAND.

YAFFS2, JFFS2 Compact Flash or NAND with NFTL (DoC)

Ext3

Documents

Introduction to Embedded Systems