pyarrow.ipc.RecordBatchStreamReader¶
-
class
pyarrow.ipc.RecordBatchStreamReader(source)[source]¶ Bases:
pyarrow.lib._RecordBatchStreamReaderReader for the Arrow streaming binary format.
- Parameters
source (bytes/buffer-like, pyarrow.NativeFile, or file-like Python object) – Either an in-memory buffer, or a readable file object.
Methods
__init__(source)Initialize self.
from_batches(schema, batches)Create RecordBatchReader from an iterable of batches.
get_next_batch(self)read_all(self)Read all record batches as a pyarrow.Table.
read_next_batch(self)Read next RecordBatch from the stream.
read_pandas(self, **options)Read contents of stream to a pandas.DataFrame.
Attributes
Shared schema of the record batches in the stream.
Current IPC read statistics.
-
static
from_batches(schema, batches)¶ Create RecordBatchReader from an iterable of batches.
- Parameters
schema (Schema) – The shared schema of the record batches
batches (Iterable[RecordBatch]) – The batches that this reader will return.
- Returns
reader (RecordBatchReader)
-
get_next_batch(self)¶
-
read_all(self)¶ Read all record batches as a pyarrow.Table.
-
read_next_batch(self)¶ Read next RecordBatch from the stream.
- Raises
StopIteration: – At end of stream.
-
read_pandas(self, **options)¶ Read contents of stream to a pandas.DataFrame.
Read all record batches as a pyarrow.Table then convert it to a pandas.DataFrame using Table.to_pandas.
- Parameters
**options (arguments to forward to Table.to_pandas) –
- Returns
df (pandas.DataFrame)
-
schema¶ Shared schema of the record batches in the stream.
-
stats¶ Current IPC read statistics.