Lesson 6 - Advanced File IO Techniques

7/23/2019 Lesson 6 - Advanced File IO Techniques

1/34


2/34

Topics

A. File Format Comparison

B. Binary Files

C. TDMS Files


3/34


3

At their lowest level, all files written to your

computers hard drive are a series of bits

Binary

TDMS

ASCII


4/34


ASCII TDMS Direct Binary

Numeric

Precision

Good Best Best

Share

data

Best (Any program

easily)

Better (NI

Programs easily)

Good (only with

detailed meta

data)

Efficiency Good Best Best

Ideal Use Share data with other

programs when file

space and numeric

precision are not

important

Share data with

programs when

storing simple

array data and

metadata

Store numeric

data compactly

with ability to

random access


5/34

B. Binary Files

5

Use Binary File functions to interact directly

with a binary file


6/34

B. Binary FilesBits/Bytes?

6

A bit is a single binary value

Each bit is either on or off and is represented by a 1 or a

0

A byte is a series of 8 bits0 bit

00000000 byte


7/34

B. Binary FilesStoringBoolean Values

7

LabVIEW represents Boolean values as 8-bit values

in a binary file

Eight zeroes represents False [00000000], and any

other value represents True [00000001],

[01000111], [11111111], and so on

Files are divided into byte-sized chunks, making

them much easier to read and process


8/34

B. Binary FilesStoringBoolean Values

8

File Contents

Method A

00000001 00000001

00000000 00000001

00000000 00000001

Method B00101011


9/34

B. Binary FilesStoring Integers

9

Binary Value U8 Value

00000000 0

00000001 100000010 2

11111111 255


10/34

B. Binary FilesStoring Integers

10

Multi-byte integers are broken into separate bytes and

are stored in files in either little-endian or big-endian

byte order

Using the Write to Binary File VI, you can choosewhether you store your data in little-endian or big-

endian format

U32 Value Little-endian Value Big-endian Value

1 00000001

00000000

00000000

00000000

00000000

00000000

00000000

00000001


11/34

B. Binary FilesStoringOther Data Types

11

Strings are stored as a series of unsigned 8-bit

integers, each of which has a value in the ASCII

Character Code Equivalents Table

This means that no difference exists between writing

strings with the Binary File Functions and writing them

with the Text File Functions

Clusters are best represented in binary files by using

Datalog Files


12/34

B. Binary FilesStoring Arrays

12

Arrays are represented as a sequential list of the

elements

Element representation depends upon the element type

A header contains a 32-bit integer representing the size ofeach dimension

Example

A 2D array with a

header contains:row integer,

column integer,

then array data


13/34

B. Binary FilesSequential/Random Access

13

Two methods of accessing data:

Sequential AccessRead each item in order, starting

at the beginning of a file

Random AccessAccess data at an arbitrary pointwithin the file


14/34

B. Binary FilesSequential Access

14

To sequentially access all of the data in a file, you can

call the Get File Size function and use the result to

calculate the number of items in the file, based upon

the size of each item and the layout of the file

You can then wire the number of items to the count

terminal of the Read Binary VI


15/34

B. Binary FilesRandom Access

15

Use the Set Position VI to set the read offset to the

point in the file you want to begin reading

The offset is in bytes; therefore, you must calculate

the offset based upon the layout of the file


16/34

B. Binary FilesDatalog

16

Datalog is a specific type of binary file, designed for

storing a list of records to a file

Each record is represented by a cluster and can

contain multiple pieces of data with any data type


17/34

B. Binary FilesDatalogRandom Access

17


18/34

Exercise 6-1: Bitmap File Writer VI

18

Use Binary File I/O to write a file with a specified

format.


19/34

C. TDMS Files

19

TDMS

Technical Data Management Streaming

Use TDMS files for the following purposes: To store test or measurement data

To create a structure for grouping your data

To store information about your data To read and write data at high speeds


20/34

C. TDMS Files

20

TDMS file format:

TDMS file

Binary file (.tdms) that contains data and stores properties about

the data

TDMS_Index file

Binary index file (*.tdms_index) that provides consolidated

information on all the attributes and pointers in the TDMS file

Speeds up access to the data while reading

Automatically regenerated if lost

TDMS file format internal structure is publicly documented


21/34

C. TDMS FilesData Hierarchy

21

Channel

Stores measurement signals or raw data in a TDMS file

Each channel can have properties describing the data

The data stored in the signal is stored as binary data ondisk to conserve disk space and efficiency

Channel Group

Segment of a TDMS file that contains properties and one

or more channelsUse channel groups to organize your data and to store

information that applies to multiple channels


22/34

C. TDMS Files

22

Create TDMS Files in the following ways:

Use the Write to Measurement File and Read fromMeasurement File Express VIs

Allow you to quickly save and retrieve data from TDMSformat

Very little control over your data grouping and properties

Use the TDM Streaming API

Set of functions for opening, writing to, reading from, and

closing TDMS files Allows you to organize your data into channel groups and

channels


23/34

C. TDMS FilesTDMStreaming API

23


24/34

C. TDMS FilesWrite Data

24

Streams data to the specified TDMS file

Data subset to write is determined by group name and

channel name(s) input


25/34

C. TDMS FilesRead Data

25

Reads the specified TDMS file and returns data

from the specified channel and/or channel group


26/34

C. TDMS FilesSet Properties

26

Sets the properties of the TDMS file, channel

group, or channel


27/34

C. TDMS FilesGet Properties

27

Returns the properties of the TDMS file, channel

group, or channel


28/34

C. TDMS FilesFile Viewer

28

Opens TDMS file and presents the file data in

the TDMS File Viewer dialog box


29/34

C. TDMS FilesGrouping Data

29

Carefully consider the best way to group your data,

because the data grouping can have a significant

impact on both the execution speed and

implementation complexity of writes and reads

Things you should consider when choosing a grouping

scheme include the original format of your data and

how you want to process or view the data


30/34

Exercise 6-2: TDMS Reader VI

30

Learn how to read data from a TDMS file.


31/34

SummaryQuiz

1. You need to store testresults and organize thedata into descriptivegroups. In the future, youneed to efficiently viewthe test results by group.Which file storage formatshould you use?

a) Tab-delimited ASCII

b) Custom binary formatc) TDMS

d) Datalog

2. You need to write aprogram which savesPortable NetworkGraphics (PNG) imagefiles. Which file storageVIs should you use?

a) Storage file VIs

b) Binary file VIs

c) ASCII file VIs

d) Datalog file VIs

31


32/34

SummaryQuiz

32

3. You need to store data that other engineers will later analyze

with Microsoft Excel. Which file storage format should you

use?

a) Tab-delimited ASCII

b) Custom binary format

c) TDMS

d) Datalog


33/34

SummaryQuiz

a) 00001010

00000000

00000000

00000000

b) 00000000

00000000

0000000000001010

c) 00001010

d)

0101000000000000

00000000

00000000

4. Which of the following is a little-endian representation ofan unsigned 32-bit integer (U32) with a value of 10?


34/34

SummaryQuiz

True or False?

5. You can use the Binary File VIs to read ASCII files.

6. TDMS Files store properties only at the channel

or channel group level.

Documents

Lesson 6 - Advanced File IO Techniques