ELF: An Efficient Log-Structured Flash File ELF: An Efficient Log-Structured Flash File System For Micro Sensor NodesSystem For Micro Sensor Nodes

Hui Dai Michael Neufeld Richard HanHui Dai Michael Neufeld Richard HanUniversity of Colorado at BoulderUniversity of Colorado at Boulder

Computer Science DepartmentComputer Science Department

ACM SenSysACM SenSys 20042004

Speaker: Jinyoung Choi Speaker: Jinyoung Choi

OutlineOutline

IntroductionIntroduction Related WorkRelated Work Design IssueDesign Issues s (Including (Including ELF File System ArchitectureELF File System Architecture))

Implementation IssueImplementation Issues s (Based on (Based on TinyOSTinyOS and and MICA2 sensor motesMICA2 sensor motes))

Performance EvaluationPerformance Evaluation ConclusionConclusion

IntroductionIntroduction WSN (Wireless Sensor WSN (Wireless Sensor

Network)Network)

– Sporadic transfer of data– Hardware failure or environmental

condition– With a very low duty cycle for

radio transmission

Need to Need to flexiblyflexibly and and efficientlyefficiently exploit exploit local storagelocal storage on the on the sensor nodessensor nodes

Flash as persistent storage on micro sensor nodesFlash as persistent storage on micro sensor nodes– Lager storage– Robustness ↑

Two challenges of the design & implementation of FS Two challenges of the design & implementation of FS for sensor nodesfor sensor nodes– Quite resource-constrained

• RAM usage• Power consumption

– The characteristics of flash memory

IntroductionIntroduction

GoalsGoals– Allow access to flash with simple file operations– Extend the operational lifetime of the flash with wear

leveling techniques– Achieve a small memory footprint– Optimize common sensor file operations– Avoid excessive energy consumption– Provide Optional best-effort data reliability

IntroductionIntroduction

Related WorkRelated WorkFile System TypeFile System Type CharacteristicsCharacteristics ExampleExample

Log StructuredLog Structured Represented as a log of metadataRepresented as a log of metadata

Data gathered in cache before writing to diskData gathered in cache before writing to disk

Sprite Sprite LFS, BSD-LFS, BSD-LFSLFS

FlashFlash Large flash memoryLarge flash memory

Small battery backed SRAM for write bufferingSmall battery backed SRAM for write buffering

Large bandwidth bus between RAM and SRAMLarge bandwidth bus between RAM and SRAM

Allows garbage collectionAllows garbage collection

eNVy,MFFeNVy,MFFS,JFFSS,JFFS

Micro Sensor StorageMicro Sensor Storage Work with sensor flash storage on each sensor Work with sensor flash storage on each sensor nodenode

Provide efficient, reliable complete file operationProvide efficient, reliable complete file operation

Matchbox, Matchbox, ELFELF

Design IssuesDesign Issues File Access Behavior on Sensor NodesFile Access Behavior on Sensor Nodes

Sensor Data• Major part of the stored data• Data written sequentially without modifying prior records• Erased periodically

Configuration Data• Changes infrequently • Requires high reliability

Binary Program Images• Dynamic reprogramming • Store the binary image in flash memory before rebooting the system• Requires highest reliability

Log-Structured File System Techniques inLog-Structured File System Techniques in ELFELF

Traditional log-structured file system• Create a new log entry for each write operation• Delaying update for maximizing the disk’s bandwidth

ELF• Write operation = Write-Append + Write-Modify• Create a new log entry only for write-modify operation (∵ memory consumption ↓)• Delaying update for reducing the number of write on flash

• Page lifespan ↔ File consistency (Tradeoff) • Keep each log entry on a separate flash page

Design IssuesDesign Issues

Design IssuesDesign Issues Persistent Storage Capabilities on Sensor NodesPersistent Storage Capabilities on Sensor Nodes

Read a page into cacheRead a page into cache Less than 250usLess than 250us

Page ErasePage Erase

Write LimitWrite Limit

8ms8ms

10,000 times10,000 times

Writing a pageWriting a page

Erasing and writing a pageErasing and writing a page

14 ms14 ms

20 ms20 ms

Typical Number of PagesTypical Number of Pages

Typical Page SizeTypical Page Size

20482048

256+8=264 Byte256+8=264 Byte

Power ConsumptionPower Consumption 4mA Read 4mA Read CurrentCurrent

2uA Standby2uA Standby

EEPROM WriteEEPROM Write 8848 cycles(8.5ms at 8848 cycles(8.5ms at 1MHz)1MHz)

EEPROM ReadEEPROM Read 1cycle(from CPU)1cycle(from CPU)

CPU halted 4 cycles CPU halted 4 cycles after readafter read

EEPROM ERASEEEPROM ERASE About 4msAbout 4ms

Write LimitWrite Limit 100,000times100,000times

Power Power ConsumptionConsumption

2-8mA when 2-8mA when ProgrammingProgramming

Flash Memory Attributes in Mote EEPROM attributes in atmega128

EEPROM in ELF file systemEEPROM in ELF file system

– Used as persistent storage

– Cache directory structure for fast file access– Allows fast startup or system reboot

– Store system snapshot to enable file consistency– Allows crash recovery

Design IssuesDesign Issues

ELF Architecture OverviewELF Architecture Overview Resource AbstractionResource Abstraction

– In Run-time memory– In-memory representation of open

files– ELF Configuration data

General File OperationsGeneral File Operations– Logical abstraction of file/directory

operation

ELF Maintenance TasksELF Maintenance Tasks– Abstraction of System

maintenance tasks (e.g.) Maintaining snapshot of dire

ctory structure and file metadata in EEPROM

Distribution of ELF Data StructuresDistribution of ELF Data Structures

On-Flash Data StructureOn-Flash Data Structure

Physical NodesPhysical Nodes

Metadata• 16-bit node identifier• 16-bit version number

indicating age• 32-bit field storing length of the

node and included data

Types• ELF_DIR - a directory entry in the FS• ELF_FILE - metadata to operate on file• ELF_COMMON - changes to the file

Per-Page Meta DataPer-Page Meta Data

struct page_info{ uint16_t crc; uint16_t nextPage:11; uint16_t flags:5; uint16_t writeCounter; uint16_t magicNumber;}

In-Memory Data StructuresIn-Memory Data Structures

File AbstractionFile Abstraction

Node AbstractionNode Abstraction

File DescriptorFile Descriptor File abstraction Current offset in that file Open mode

File Operations in ELFFile Operations in ELF

ELF-Open ELF-Append

File Operations in ELFFile Operations in ELF

ELF-Modify ELF-Read

Garbage CollectionGarbage Collection ProcedureProcedure

– The “cleaner” task - the number of free pages drops below a preset threshold

– Cleaner traces the link inside the file pages– Sets the corresponding bits in bitmap– If still not enough, the files de-fragmented

Any page with more than 9000 writesAny page with more than 9000 writes – marked as “unusable”

RAM consumptionRAM consumption for ELF cleaner – for ELF cleaner – fairly lowfairly low

Performance EvaluationPerformance Evaluation Sequential Read PerformanceSequential Read Performance

– First filled the entire memory 500K log file in Flash– Each run repeated 50 times– MatchBox Read:

• FlashCacheRAM– ELF Read：

• FlashRAM

Sequential Write PerformanceSequential Write Performance

Performance EvaluationPerformance Evaluation

M : maximum write throughput M : maximum write throughput /memory consumption of Mat /memory consumption of MatchBoxchBox

Experiments performed with vExperiments performed with varying RAM bufferarying RAM buffer E1 and E1’ correspond to 0 b

yte buffer, E2 and E2’ correspond to 32 byte buffer and so on

E1 has the whole bitmap storeE1 has the whole bitmap stored in RAM whereas E1’ has onld in RAM whereas E1’ has only first 64 bytes bitmap stored iy first 64 bytes bitmap stored in RAMn RAM

Performance EvaluationPerformance Evaluation Random Read PerformanceRandom Read Performance

– MatchBox not support random reads

Random Write PerformanceRandom Write Performance– Write-modify operation

Performance EvaluationPerformance Evaluation

Wear Leveling PerformanceWear Leveling Performance– The experiment continues running until the total number of

erasures exceeds 10240,000– Each of 2000 256 bytes in the 512Kbytes flash are expected to

be written exactly 5000 times if absolutely uniform wear leveling is achieved

Performance EvaluationPerformance Evaluation

ConclusionConclusion ELF is a reliable and efficient file system ELF is a reliable and efficient file system

designed and implemented for micro sensor designed and implemented for micro sensor nodes.nodes.

ELF is the first log-structured file system for ELF is the first log-structured file system for sensor networks that provides garbage sensor networks that provides garbage collection and a best collection and a best effort effort recovery mechanismrecovery mechanism

ReferencesReferences[1] Mendel Rosenblum, John K. Ousterhout, “The Design and ImplemeMendel Rosenblum, John K. Ousterhout, “The Design and Impleme

ntation of a Log-Structured File System,” ACM Transactions on Contation of a Log-Structured File System,” ACM Transactions on Computer Systems,1992mputer Systems,1992

[2] Adam Dunkels, Niclas Finne, Joakim Eriksson, and Thiemo Voigt, “Adam Dunkels, Niclas Finne, Joakim Eriksson, and Thiemo Voigt, “Run-Time Dynamic Linking for Reprogramming Wireless Sensor NRun-Time Dynamic Linking for Reprogramming Wireless Sensor Networksetworks,” In Proceedings of the Fourth ACM Conference on Embedded Net,” In Proceedings of the Fourth ACM Conference on Embedded Networked Sensor Systems,2006worked Sensor Systems,2006

[3] Silberschatz, Galvin, Gagne, Operating System Concepts, Wiley