Module read

APIs to read Arrow’s IPC format.

The two important structs here are the FileReader, which provides arbitrary access to any of its messages, and the StreamReader, which only supports reading data in the order it was written in.

Structs

FileMetadata

Metadata of an Arrow IPC file, written in the footer of the file.

FileReader

An iterator of RecordBatchTs from an Arrow IPC file.

ProjectionInfo

StreamMetadata

Metadata of an Arrow IPC stream, written at the start of the stream

StreamReader

Arrow Stream reader.

Enums

OutOfSpecKind

The different types of errors that reading from IPC can cause

StreamState

Encodes the stream’s status after each read.

Traits

SendableIterator

Functions

deserialize_footer

deserialize_schema

Deserialize an flatbuffers-encoded Schema message into ArrowSchema and IpcSchema.

get_row_count

Read the row count by summing the length of the of the record batches

get_row_count_from_blocks

Read the row count by summing the length of the of the record batches in blocks

prepare_projection

read_batch

Reads the record batch at position index from the reader.

read_file_dictionaries

Reads all file’s dictionaries, if any This function is IO-bounded

read_file_metadata

Read the Arrow IPC file’s metadata

read_stream_metadata

Reads the metadata of the stream

Type Aliases

Dictionaries

how dictionaries are tracked in this crate