Struct polars::export::arrow::io::parquet::read::Decompressor

pub struct Decompressor<R> where
    R: Read,  { /* private fields */ }

This is supported on crate feature io_parquet only.

Decompressor that allows re-using the page buffer of PageIterator.

Implementation

The implementation depends on whether a page is compressed or not.

PageIterator(a), CompressedPage(b), Decompressor(c), DecompressedPage(d)

un-compressed pages:

page iter: a is swapped with b decompress iter: b is swapped with d, b is swapped with a therefore:

• PageIterator has its buffer back
• Decompressor’s buffer is un-used
• DecompressedPage has the same data as CompressedPage had

compressed pages:

page iter: a is swapped with b decompress iter:

• b is decompressed into c
• b is swapped with a
• c is moved to d
• (next iteration): d is moved to c therefore, while the page is available:

• PageIterator has its buffer back
• Decompressor’s buffer empty
• DecompressedPage has the decompressed buffer after the page is used:
• PageIterator has its buffer back
• Decompressor has its buffer back
• DecompressedPage has an empty buffer

Implementations

impl<R> Decompressor<R> where
    R: Read, 

pub fn new(iter: PageIterator<R>, buffer: Vec<u8, Global>) -> Decompressor<R>

Creates a new Decompressor.

pub fn into_buffers(self) -> (Vec<u8, Global>, Vec<u8, Global>)

Returns two buffers: the first buffer corresponds to the page buffer, the second to the decompression buffer.

Trait Implementations

impl<R> FallibleStreamingIterator for Decompressor<R> where
    R: Read, 

type Item = DataPage

The type being iterated over.

type Error = ParquetError

The error type of iteration.

fn advance(&mut self) -> Result<(), ParquetError>

Advances the iterator to the next position.

fn get(&self) -> Option<&<Decompressor<R> as FallibleStreamingIterator>::Item>

Returns the current element.

fn next(&mut self) -> Result<Option<&Self::Item>, Self::Error>

Advances the iterator, returning the next element.

fn size_hint(&self) -> (usize, Option<usize>)

Returns bounds on the number of remaining elements in the iterator.

fn all<F>(&mut self, f: F) -> Result<bool, Self::Error> where
    F: FnMut(&Self::Item) -> bool, 

Determines if all elements of the iterator satisfy a predicate.

fn any<F>(&mut self, f: F) -> Result<bool, Self::Error> where
    F: FnMut(&Self::Item) -> bool, 

Determines if any elements of the iterator satisfy a predicate.

fn by_ref(&mut self) -> &mut Self

Borrows an iterator, rather than consuming it.

fn count(self) -> Result<usize, Self::Error>

Returns the number of remaining elements in the iterator.

fn filter<F>(self, f: F) -> Filter<Self, F> where
    F: FnMut(&Self::Item) -> bool, 

Returns an iterator which filters elements by a predicate.

fn find<F>(&mut self, f: F) -> Result<Option<&Self::Item>, Self::Error> where
    F: FnMut(&Self::Item) -> bool, 

Returns the first element of the iterator which satisfies a predicate.

fn for_each<F>(self, f: F) -> Result<(), Self::Error> where
    F: FnMut(&Self::Item), 

Calls a closure on each element of an iterator.

fn fuse(self) -> Fuse<Self>

Returns an iterator which is well-behaved at the beginning and end of iteration.

fn map<F, B>(self, f: F) -> Map<Self, F, B> where
    F: FnMut(&Self::Item) -> B, 

Returns an iterator which applies a transform to elements.

fn map_ref<F, B>(self, f: F) -> MapRef<Self, F> where
    F: Fn(&Self::Item) -> &B,
    B: ?Sized, 

Returns an iterator which applies a transform to elements.

fn map_err<F, B>(self, f: F) -> MapErr<Self, F> where
    F: Fn(Self::Error) -> B, 

Returns an iterator that applies a transform to errors.

fn nth(&mut self, n: usize) -> Result<Option<&Self::Item>, Self::Error>

Returns the nth element of the iterator.

fn position<F>(&mut self, f: F) -> Result<Option<usize>, Self::Error> where
    F: FnMut(&Self::Item) -> bool, 

Returns the position of the first element matching a predicate.

fn skip(self, n: usize) -> Skip<Self>

Returns an iterator which skips the first n elements.

fn skip_while<F>(self, f: F) -> SkipWhile<Self, F> where
    F: FnMut(&Self::Item) -> bool, 

Returns an iterator which skips the first sequence of elements matching a predicate.

fn take(self, n: usize) -> Take<Self>

Returns an iterator which only returns the first n elements.

fn take_while<F>(self, f: F) -> TakeWhile<Self, F> where
    F: FnMut(&Self::Item) -> bool, 

Returns an iterator which only returns the first sequence of elements matching a predicate.