LIO exNode options

Description

ExNodes contain the metadata (file size, location of data, data encoding, etc) associated with a given object. Similar to a Linux inode. An exNode contains multiple segment definitions defining the data layout. The exNode section itself is pretty minimal. It just contains a unique id arbitrary text name. The [view] section has a list of segments for use and also defines the default segment to use. Each of these points to a different section to load with the naming convention [segment-**id*]* with id the segment id whish is a random 63-bit integer.

[exnode]

id: Unique exnode ID. 63-bit integer.
name: Arbitrary text field to descibe the exNode.

[view]

default: ID of the default segment to use for accessing data.
segment: Other available views. More than one segment declaration can occur. All are stored in a table.

Data Blocks

Data blocks define allocations acquired via the resource service and is where object data is stored. Each block has it’s own unique section in the exNode using a similar naming convention as for segments – [block-**id*]* where the id is a unique identifier assigned to the block. Each block has the following minimal set of parameters.

type: Type of data block. Currently onlt “ibp” is supported.
rid_key: Unique resource ID. This corresponds to the rid_key field defining the resource. It is this value that provides linking the resource and dynamic translation of the capabilities in case the resource is relocated.
size: Number of bytes currently used in the data block.
max_size: Maximum allowed size of the allocation.
ref_count: Number of times this data block is used in the exNode.

read_cap

write_cap

manage_cap: Capabilities for accessing the allocation.

Segment options

The Segment Service consists of software drivers that control the logical layout of data on physical resources. They perform block-level I/O using the Data Service to perform sophisticated fault tolerance and caching schemes.

segment_file

Simplest segment driver. Maps the segment to a local file.

type = file: Defines that this is a File segment.
file: Local File containing the data

segment_cache

Provides a hook into the caching service for storing data and handle impedance differences between the application wanting random I/O with the need for the segment driver perform fixed size block I/O.

type = cache: Defines the segment as a cache segment.
used_size: Amount of data used in the underlying segment. This will always be less than the size of the child segment
segment: Child segment containing data.

segment_jerase

This segment driver implements the functionality contained in Jim Plank’s Jerasure library.

type = jerase: Defines the segment as a Jerasure segment.
n_data_devs: Number of data devices
n_parity_devs: Number of parity devices
w: w value. See the Jerasure documentation
max_parity: Max amount of data to use for parity for any given I/O operation. Requests greater than this get broken into multiple I/O operations.
chunk_size: Amoung of data to read on each device before moving to the next
method: Jerasure method to use: “reed_sol_van”, “reed_sol_r6_op”, “cauchy_orig”, “cauchy_good”, “blaum_roth”, “liberation”, “liber8tion”, and “raid4”.

segment_lun

Takes a collection of data blocks and turns them into a logical unit or LON. Similar to what a hardware based RAID applicance does.

query_default

Default query string for selecting resources. All available resource attributes can be used. The query is comprised of a collection of ”.” separated tuples. The tuples are individual operaions broken into the following ”:” separated format:

OP:key:KEY_OP:val:VAL_OP

The OP, KEY_OP, and VAL_OP are integers representing the operations. These are given in the table below. The key and val are text string arguments that KEY_OP and VAL_OP use. For some operations these values are not used. In that case, “any” is normally used to signify this.

Value OP meaning

1 This is key/val matching operation.

2 Boolean NOT operation

3 Boolean AND operation

4 Boolean OR operation

Value KEY_OP/VAL_OP meaning

1 Exact match is required of the argument.

2 The argument must be a prefix for the resource key or val being checked.

3 Any value matches

Value Mutually exclusive options to be &’ed with the Key/VAl options above

64 Make sure that each selection is unique if multiple resources are requested.

128 Pick one for the field and use it for all resources requested.

n_devices

Number of virtual devices to create.

max_block_size

Max size of a data block before creating a new row.

excess_block_size

Extra space to allocate when growing an allocation to help minimize the number of grow operations.

max_size

Maximum size of all allocated space.

used_size

Current amount of space used.

chunk_size

Amoung of data to read on each device before moving to the next

n_shift

Device shift between successive stripes.

row

Value	OP meaning
1	This is key/val matching operation.
2	Boolean NOT operation
3	Boolean AND operation
4	Boolean OR operation

Value	KEY_OP/VAL_OP meaning
1	Exact match is required of the argument.
2	The argument must be a prefix for the resource key or val being checked.
3	Any value matches

Value	Mutually exclusive options to be &’ed with the Key/VAl options above
64	Make sure that each selection is unique if multiple resources are requested.
128	Pick one for the field and use it for all resources requested.

Tuple definiing the data blocks and physical layout of the LUN row. Multiple row definitions are allowed for large files. The tuples have the format:

seg_offset:seg_end:row_len:block_1:block_1_offset:block_2:block_2_offset:...

The first three fields define where the row fits in the file and define the file beginning and end offset along with the length of the row. The remaining fields define which data blocks and starting offsets to use. There are n_devices entries. One for each device. The block_i represents the block ID.

segment_log

Implements a log-structured file. A log-structured file only supports append operaionts. Dta is never overwritten. Instead it is appended to file file along with a new extent defining what was appended. This allows one to replay the log and create different versions of the file at any point in the log. The log segment is comprised of three other segments.

base: Base segment to use. This provides the normal flat address space used as teh base for the file to apply changes.
log: Log segment to use. This just contains all the extents (offset and length pairs) defining the changes.
data: All the data corresponding the to log extents.