LIO exNode options

Description

ExNodes contain the metadata (file size, location of data, data encoding, etc) associated with a given object. Similar to a Linux inode. An exNode contains multiple segment definitions defining the data layout. The exNode section itself is pretty minimal. It just contains a unique id arbitrary text name. The [view] section has a list of segments for use and also defines the default segment to use. Each of these points to a different section to load with the naming convention [segment-**id*]* with id the segment id whish is a random 63-bit integer.

[exnode]

id
Unique exnode ID. 63-bit integer.
name
Arbitrary text field to descibe the exNode.

[view]

default
ID of the default segment to use for accessing data.
segment
Other available views. More than one segment declaration can occur. All are stored in a table.

Data Blocks

Data blocks define allocations acquired via the resource service and is where object data is stored. Each block has it’s own unique section in the exNode using a similar naming convention as for segments – [block-**id*]* where the id is a unique identifier assigned to the block. Each block has the following minimal set of parameters.

type
Type of data block. Currently onlt “ibp” is supported.
rid_key
Unique resource ID. This corresponds to the rid_key field defining the resource. It is this value that provides linking the resource and dynamic translation of the capabilities in case the resource is relocated.
size
Number of bytes currently used in the data block.
max_size
Maximum allowed size of the allocation.
ref_count
Number of times this data block is used in the exNode.

read_cap

write_cap

manage_cap
Capabilities for accessing the allocation.

Segment options

The Segment Service consists of software drivers that control the logical layout of data on physical resources. They perform block-level I/O using the Data Service to perform sophisticated fault tolerance and caching schemes.

segment_file

Simplest segment driver. Maps the segment to a local file.

type = file
Defines that this is a File segment.
file
Local File containing the data

segment_cache

Provides a hook into the caching service for storing data and handle impedance differences between the application wanting random I/O with the need for the segment driver perform fixed size block I/O.

type = cache
Defines the segment as a cache segment.
used_size
Amount of data used in the underlying segment. This will always be less than the size of the child segment
segment
Child segment containing data.

segment_jerase

This segment driver implements the functionality contained in Jim Plank’s Jerasure library.

type = jerase
Defines the segment as a Jerasure segment.
n_data_devs
Number of data devices
n_parity_devs
Number of parity devices
w
w value. See the Jerasure documentation
max_parity
Max amount of data to use for parity for any given I/O operation. Requests greater than this get broken into multiple I/O operations.
chunk_size
Amoung of data to read on each device before moving to the next
method
Jerasure method to use: “reed_sol_van”, “reed_sol_r6_op”, “cauchy_orig”, “cauchy_good”, “blaum_roth”, “liberation”, “liber8tion”, and “raid4”.

segment_lun

Takes a collection of data blocks and turns them into a logical unit or LON. Similar to what a hardware based RAID applicance does.

query_default

Default query string for selecting resources. All available resource attributes can be used. The query is comprised of a collection of ”.” separated tuples. The tuples are individual operaions broken into the following ”:” separated format:

OP:key:KEY_OP:val:VAL_OP

The OP, KEY_OP, and VAL_OP are integers representing the operations. These are given in the table below. The key and val are text string arguments that KEY_OP and VAL_OP use. For some operations these values are not used. In that case, “any” is normally used to signify this.

Value OP meaning
1 This is key/val matching operation.
2 Boolean NOT operation
3 Boolean AND operation
4 Boolean OR operation
Value KEY_OP/VAL_OP meaning
1 Exact match is required of the argument.
2 The argument must be a prefix for the resource key or val being checked.
3 Any value matches
Value Mutually exclusive options to be &’ed with the Key/VAl options above
64 Make sure that each selection is unique if multiple resources are requested.
128 Pick one for the field and use it for all resources requested.
n_devices
Number of virtual devices to create.
max_block_size
Max size of a data block before creating a new row.
excess_block_size
Extra space to allocate when growing an allocation to help minimize the number of grow operations.
max_size
Maximum size of all allocated space.
used_size
Current amount of space used.
chunk_size
Amoung of data to read on each device before moving to the next
n_shift
Device shift between successive stripes.
row

Tuple definiing the data blocks and physical layout of the LUN row. Multiple row definitions are allowed for large files. The tuples have the format:

seg_offset:seg_end:row_len:block_1:block_1_offset:block_2:block_2_offset:...

The first three fields define where the row fits in the file and define the file beginning and end offset along with the length of the row. The remaining fields define which data blocks and starting offsets to use. There are n_devices entries. One for each device. The block_i represents the block ID.

segment_log

Implements a log-structured file. A log-structured file only supports append operaionts. Dta is never overwritten. Instead it is appended to file file along with a new extent defining what was appended. This allows one to replay the log and create different versions of the file at any point in the log. The log segment is comprised of three other segments.

base
Base segment to use. This provides the normal flat address space used as teh base for the file to apply changes.
log
Log segment to use. This just contains all the extents (offset and length pairs) defining the changes.
data
All the data corresponding the to log extents.
L-Store Documentation