Module arrow2::io::parquet::write

APIs to write to Parquet format.

Re-exports

pub use parquet2::fallible_streaming_iterator;

Structs

ColumnDescriptor

A descriptor for leaf-level primitive columns. This encapsulates information such as definition and repetition levels and is used to re-assemble nested data.

CompressedDataPage

A CompressedDataPage is compressed, encoded representation of a Parquet data page. It holds actual data and thus cloning it is expensive.

Compressor

A FallibleStreamingIterator that consumes EncodedPage and yields CompressedPage holding a reusable buffer (Vec<u8>) for compression.

DynIter

DynIter is an implementation of a single-threaded, dynamically-typed iterator.

DynStreamingIterator

Dynamically-typed FallibleStreamingIterator.

FileSink

Sink that writes array chunks as a Parquet file.

FileWriter

An interface to write a parquet to a Write

RowGroupIterator

An iterator adapter that converts an iterator over Chunk into an iterator of row groups. Use it to create an iterator consumable by the parquet’s API.

SchemaDescriptor

A schema descriptor. This encapsulates the top-level schemas for all the columns, as well as all descriptors for all the primitive columns.

WriteOptions

Write options of different interfaces on this crate

Enums

CompressedPage

A CompressedPage is a compressed, encoded representation of a Parquet page. It holds actual data and thus cloning it is expensive.

Compression

EncodedPage

A EncodedPage is an uncompressed, encoded representation of a Parquet page. It may hold actual data and thus cloning it may be expensive.

Encoding

ParquetType

Representation of a Parquet type. Used to describe primitive leaf fields and structs, including top-level schema. Note that the top-level schema type is represented using GroupType whose repetition is None.

Version

The parquet version to use

Traits

FallibleStreamingIterator

A fallible, streaming iterator.

Functions

array_to_page

Converts an Array to a CompressedPage based on options, descriptor and encoding.

array_to_pages

Returns an iterator of EncodedPage.

can_encode

Checks whether the data_type can be encoded as encoding. Note that this is whether this implementation supports it, which is a subset of what the parquet spec allows.

compress

row_group_iter

Maps a Chunk and parquet-specific options to an RowGroupIter used to write to parquet

to_parquet_schema

Creates a parquet SchemaDescriptor from a Schema.

to_parquet_type

Creates a ParquetType from a Field.

Type Definitions

KeyValue

RowGroupIter