Multi-file datasets

Arrow Datasets allow you to query against data that has been split across multiple files. This sharding of data may indicate partitioning, which can accelerate queries that only touch some partitions (files).

Factory

The Dataset$create() factory method instantiates a Dataset and takes the following arguments:

• sources: a list of DataSource objects

• schema: a Schema

Methods

• $NewScan(): Returns a ScannerBuilder for building a query

• $schema: Active binding, returns the Schema of the Dataset

See also

open_dataset() for a simple way to create a Dataset that has a single DataSource.

Super class

arrow::Object -> Dataset

Methods

Public methods

• Dataset$NewScan()

• Dataset$clone()

Inherited methods

• arrow::Object$initialize()

• arrow::Object$pointer()

• arrow::Object$print()

• arrow::Object$set_pointer()

---

Method NewScan()

Start a new scan of the data

Usage

Dataset$NewScan()

Returns

A ScannerBuilder

---

Method clone()

The objects of this class are cloneable with this method.

Usage

Dataset$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.