YAFFS: Yet Another Flash File System

2008-01-18Speaker : Yunjung Yoo

Contents• Introduction• History• Design• Terminology• Layout – NAND / RAM• Log-structured File System (1 ~ 5)• Mount / Scan• Read / Write• Read Syscall Flowchart (1 ~ 2)• Garbage Collection• ECC• YAFFS2• OOB Data• Checkpointing• Conclusion

Introduction

• Charles Manning, Aleph One• The first file system designed for NAND• YAFFS1 : 512 byte pages + 16 byte spare areas• YAFFS2 : 2048 byte pages + 64 byte spare areas

• Background– SmartMedia + FAT : low memory footprint / not robust– NOR + JFFS : based on JFS / long boot-time, RAM overhead

History

• Decided to create ’YAFFS’ - Dec 2001• Working on NAND emulator - March 2002• Working on real NAND (Linux) - May 2002• WinCE version - Aug 2002• ucLinux use - Sept 2002• Linux rootfs - Nov 2002• pSOS version - Feb 2003• Shipping commercially - Early 2003• Linux 2.6 supported - Aug 2004• YAFFS2 - Dec 2004• Checkpointing - May 2006

Terminology

• Flash-defined– Page - 2k flash page (512 byte YAFFS1)– Block - Erasable set of pages (typically 64 on 2K NAND)

• YAFFS-defined– Chunk - YAFFS tracking unit.– Usually == page.

Design• 각 파일은 id 를 갖는다 . – inode, id 가 0 이면 유효하지 않은 파일임• 파일은 chunk(= page) 로 나뉘어 저장된다 . (2K / 512B)• Chunk 는 0 과 그 이상의 숫자를 갖는다 . 0 은 그 파일의 헤더이다 .• 객체를 나타낸 자료구조 :

– yaffs_ObjectHeader(NAND), yaffs_Object(RAM)• 플래시의 각 페이지는 file id : chunk # 로 구분한다 .• Chunk 에 대한 정보는 Tags 로 OOB 영역에 저장된다 .• 파일의 내용을 변경할 경우 , 관련된 chunk 는 새로운 데이터와 같은

태그로 새로운 위치에 쓰여진다 .• 이때 , 이전의 페이지와 새로운 페이지를 구분하기 위해 write 가

일어날 때마다 2-bit serial#(YAFFS1) / sequence(YAFFS2) 가 순차적으로 증가한다 . 이것은 crash-recovery 할 때 분별할 수 있게 한다 .

• “Discarded” page 가 꽉 찬 block 은 GC 대상이 된다 .• “Deleted” object 는 “ Unlinked dir” 로 옮겨져 삭제된다 .

Layout | NANDPage Spare

Chunk Tag

ObjectHeader ChunkId == 0

Data ChunkId > 0

<YAFFS1 : Page Layout>

Byte# Usage

0~511 Data. (File data or header)

512~515 Tags

516 Data Status

517 Block Status

518~519 Tags

520~522 ECC

523~524 Tags

525~527 ECC

<YAFFS1: Tags bit usage>

Bit# Usage

18 File id

2 2-bit serial number

20 Page-id(chunk id)

10 Counter (the # of bytes used in the page

12 ECC on tags

2 Unused

Layout | RAM

Header(/)

Header(file1)

Data(file1)

Data(file1)

Header(dir1)

Header(file2)

Data(file2)

<NAND Flash>

Object(file2)

Object(file1)

Object(dir1)

Tnode(file1)

Object(/)

Tnode(file2)

<Main Memory>

Child

Sibling

Parent

Log-structured FS(1)

• Supposed the flash with 4 pages per block• First, Create the file

• Then, Write a few chunks

Block Chunk ObjID ChunkID Delete?

Comment

0 0 100 0 Live Object header for this files

Block Chunk ObjID ChunkID Delete?

Comment

0 0 100 0 Live Object header for this files

0 1 100 1 Live First chunk of data

0 2 100 2 Live Second chunk of data

0 3 100 3 Live Third chunk of data

Log-structured FS(2)

• Close the file >> Write a new object header for the file

Block Chunk ObjID ChunkID Delete?

Comment

0 0 100 0 Del Obsoleted object header

0 1 100 1 Live First chunk of data

0 2 100 2 Live Second chunk of data

0 3 100 3 Live Third chunk of data

1 0 100 0 Live New object header

Log-structured FS(3)

• Open the file for read/write and overwrite part of the first chunk in the file and close the file. The replaced data and object header chunks become deleted.

Block Chunk ObjID ChunkID Delete?

Comment

0 0 100 0 Del Obsoleted object header

0 1 100 1 Del Obsoleted First chunk of data

0 2 100 2 Live Second chunk of data

0 3 100 3 Live Third chunk of data

1 0 100 0 Del Obsoleted object header

1 1 100 1 Live New first chunk of data

1 2 100 0 Live New object header

Log-structured FS(4)

• Now resize the file to zero by opening the file with O_TRUNC and closing the file. This writes a new object header with length 0 and marks the data chunks deleted.

Block Chunk ObjID ChunkID Delete?

Comment

0 0 100 0 Del Obsoleted object header

0 1 100 1 Del Obsoleted First chunk of data

0 2 100 2 Del Obsoleted 2nd chunk of data

0 3 100 3 Del Obsoleted 3rd chunk of data

1 0 100 0 Del Obsoleted object header

1 1 100 1 Del Obsoleted first chunk of data

1 2 100 0 Del Obsoleted object header

1 3 100 0 Live New object header

Log-structured FS(5)

• Block#1 is dirty block!!• Rename the file, then we need new object header

Block Chunk ObjID ChunkID Delete?

Comment

0 0 100 0 Del Obsoleted object header

0 1 100 1 Del Obsoleted First chunk of data

0 2 100 2 Del Obsoleted 2nd chunk of data

0 3 100 3 Del Obsoleted 3rd chunk of data

1 0 100 0 Del Obsoleted object header

1 1 100 1 Del Obsoleted first chunk of data

1 2 100 0 Del Obsoleted object header

1 3 100 0 Del Obsoleted object header

2 0 100 0 Live New object header

Mount / Scan

• Mount 시 Spare 만 읽는다 .• (chunkId == 0) 인 경우 , Object Header 가 저장된 Page

이므로 , 이를 읽어서 Object 정보를 주 메모리에 동적으로 생성 .

• YAFFS2 의 경우 저장된 checkpoint 를 읽어들인다 .

ObjectHeader

ChunkID=0

File data

ChunkID=1

Object Header

ChunkID=0

....

....

Page1 Page2 Page 32

Read / Write

• Write– Page 단위로 기록한다 .– Page 보다 작거나 파일의 마지막 단편 부분은 정책상 Cache

된다 .

• Read– Cache 에 읽고자 하는 데이터를 찾는다 .– 없는 경우는 실제 Flash 로부터 읽어온다 .

• Read/Write 성능 향상을 위해 ChunkCache 라는 간단한 자료구조 사용– Read/write 회수 감소 , Flash memory 수명 증가 .– Response Time 감소 .

Read Syscall Flowchart(1)

sys_read

generic_file_read

generic_file_directIO do_generic_file_read

file_read_actoryaffs_readpage

VFSLayer

File System SpecificLayer

no yes

no yes

캐시의 내용을 사용자 영역에 복사

페이지 캐시 해시테이블에요청된 페이지가 존재하는가 ?

페이지 캐시를 사용하는가 ?

System Call handlingLayer

Read Syscall Flowchart(2)

yaffs_readpageVFS InterfaceLayer

CoreLayer

MTD InterfaceLayer

yaffs_readpage_unlock

yaffs_readpage_nolock

yaffs_ReadDataFromFile

yaffs_ReadChunkDataFromObject

yaffs_ReadChunkWithTagsFromNAND

nandmtd2_ReadChunkWithTagsFromNAND

Garbage Collection

• 3 blocks reserved for GC (384K)• If no deleted blocks, GC dirtiest• Soft Background deletion:

– Delete/Resize large files can take up to 0.5s– Incorporated with GC– Spread over several writes

• GC is deterministic - does one block for each write (default)

ECC

• Needs Error Correction Codes for reliable use• ECC on Tags and data• 22bits per 256 bytes, 1-bit correction, 2-bit detection• Lots of options:

– Hardware or software– YAFFS or MTD– New MTD, old MTD or YAFFS/Smartmedia positioning

• Make sure bootloader, OS and FS generation all match!• Can be disabled - not recommended!

YAFFS2

• Designed for new hardware:– >=1k page size– No re-writing– Can support Toshiba/Sandisk MLC parts

• Main difference is ‘discarded’ status tracking• ECC done by driver (MTD in Linux case)• Extended Tags (Extra metadata to improve performance)

• RAM footprint 25-50% less• faster (write 1-3x, read 1-2x, delete 4-34x, GC 2-7x)

OOB Data

• YAFFS1:– Derived from Smartmedia, (e.g byte 5 is bad block marker)– 16 bytes: 7 tags, 2 status, 6 ECC– YAFFS/Smartmedia or JFFS2 format ECC

• YAFFS2:– 64 bytes available in 2k page– MTD-determined layout– MTD or hardware does ECC– Tags normally 28 bytes (16 data, 12ecc)

Checkpointing• RAM structures saved on flash at unmount (10 blocks)• Structures re-read, avoiding boot scan• Invalidated by any write• Lazy Loading also reduces mount time.

Conclusion

• Very simple • NAND-friendly • Relatively frugal with resources (especially RAM) • Boot quickly • Journaling (robustness)