org.bytedeco.arrow_dataset

Class ParquetFactoryOptions

java.lang.Object

org.bytedeco.javacpp.Pointer

org.bytedeco.arrow_dataset.ParquetFactoryOptions

All Implemented Interfaces:

AutoCloseable

@Namespace(value="arrow::dataset")
 @Properties(inherit=arrow_dataset.class)
public class ParquetFactoryOptions
extends Pointer

\brief Options for making a FileSystemDataset from a Parquet _metadata file.

Nested Class Summary

Nested classes/interfaces inherited from class org.bytedeco.javacpp.Pointer

Pointer.CustomDeallocator, Pointer.Deallocator, Pointer.NativeDeallocator, Pointer.ReferenceCounter

Field Summary

Fields inherited from class org.bytedeco.javacpp.Pointer

address, capacity, limit, position

Constructor Summary

Constructors

Constructor and Description
ParquetFactoryOptions() Default native constructor.
ParquetFactoryOptions(long size) Native array allocator.
ParquetFactoryOptions(Pointer p) Pointer cast constructor.

Method Summary

Modifier and Type	Method and Description
ParquetFactoryOptions	getPointer(long i)
String	partition_base_dir() For the purposes of applying the partitioning, paths will be stripped of the partition_base_dir.
ParquetFactoryOptions	partition_base_dir(String setter)
PartitioningOrFactory	partitioning() Either an explicit Partitioning or a PartitioningFactory to discover one.
ParquetFactoryOptions	partitioning(PartitioningOrFactory setter)
ParquetFactoryOptions	position(long position)
boolean	validate_column_chunk_paths() Assert that all ColumnChunk paths are consistent.
ParquetFactoryOptions	validate_column_chunk_paths(boolean setter)

Methods inherited from class org.bytedeco.javacpp.Pointer

address, asBuffer, asByteBuffer, availablePhysicalBytes, calloc, capacity, capacity, close, deallocate, deallocate, deallocateReferences, deallocator, deallocator, equals, fill, formatBytes, free, getDirectBufferAddress, getPointer, getPointer, getPointer, hashCode, interruptDeallocatorThread, isNull, isNull, limit, limit, malloc, maxBytes, maxPhysicalBytes, memchr, memcmp, memcpy, memmove, memset, offsetAddress, offsetof, offsetof, parseBytes, physicalBytes, physicalBytesInaccurate, position, put, realloc, referenceCount, releaseReference, retainReference, setNull, sizeof, sizeof, toString, totalBytes, totalCount, totalPhysicalBytes, withDeallocator, zero

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

ParquetFactoryOptions

public ParquetFactoryOptions()

Default native constructor.

ParquetFactoryOptions

public ParquetFactoryOptions(long size)

Native array allocator. Access with Pointer.position(long).

ParquetFactoryOptions

public ParquetFactoryOptions(Pointer p)

Pointer cast constructor. Invokes Pointer(Pointer).

Method Detail

position

public ParquetFactoryOptions position(long position)

Overrides:

position in class Pointer

getPointer

public ParquetFactoryOptions getPointer(long i)

Overrides:

getPointer in class Pointer

partitioning

@ByRef
public PartitioningOrFactory partitioning()

Either an explicit Partitioning or a PartitioningFactory to discover one. If a factory is provided, it will be used to infer a schema for partition fields based on file and directory paths then construct a Partitioning. The default is a Partitioning which will yield no partition information. The (explicit or discovered) partitioning will be applied to discovered files and the resulting partition information embedded in the Dataset.

partitioning

public ParquetFactoryOptions partitioning(PartitioningOrFactory setter)

partition_base_dir

@StdString
public String partition_base_dir()

For the purposes of applying the partitioning, paths will be stripped of the partition_base_dir. Files not matching the partition_base_dir prefix will be skipped for partition discovery. The ignored files will still be part of the Dataset, but will not have partition information. Example: partition_base_dir = "/dataset"; - "/dataset/US/sales.csv" -> "US/sales.csv" will be given to the partitioning - "/home/john/late_sales.csv" -> Will be ignored for partition discovery. This is useful for partitioning which parses directory when ordering is important, e.g. DirectoryPartitioning.

partition_base_dir

public ParquetFactoryOptions partition_base_dir(String setter)

validate_column_chunk_paths

@Cast(value="bool")
public boolean validate_column_chunk_paths()

Assert that all ColumnChunk paths are consistent. The parquet spec allows for ColumnChunk data to be stored in multiple files, but ParquetDatasetFactory supports only a single file with all ColumnChunk data. If this flag is set construction of a ParquetDatasetFactory will raise an error if ColumnChunk data is not resident in a single file.

validate_column_chunk_paths

public ParquetFactoryOptions validate_column_chunk_paths(boolean setter)