org.bytedeco.arrow_dataset

Class ParquetFileFormat

java.lang.Object

org.bytedeco.javacpp.Pointer

org.bytedeco.arrow_dataset.FileFormat

org.bytedeco.arrow_dataset.ParquetFileFormat

All Implemented Interfaces:

AutoCloseable

@Namespace(value="arrow::dataset")
 @Properties(inherit=arrow_dataset.class)
public class ParquetFileFormat
extends FileFormat

\brief A FileFormat implementation that reads from Parquet files

Nested Class Summary

Nested classes/interfaces inherited from class org.bytedeco.javacpp.Pointer

Pointer.CustomDeallocator, Pointer.Deallocator, Pointer.NativeDeallocator, Pointer.ReferenceCounter

Field Summary

Fields inherited from class org.bytedeco.javacpp.Pointer

address, capacity, limit, position

Constructor Summary

Constructors

Constructor and Description
ParquetFileFormat()
ParquetFileFormat(long size) Native array allocator.
ParquetFileFormat(Pointer p) Pointer cast constructor.
ParquetFileFormat(ReaderProperties reader_properties) Convenience constructor which copies properties from a parquet::ReaderProperties.

Method Summary

Modifier and Type	Method and Description
LongOptionalFuture	CountRows(FileFragment file, Expression predicate, ScanOptions options)
FileWriteOptions	DefaultWriteOptions() \brief Get default write options for this format.
boolean	Equals(FileFormat other)
ParquetFileFormat	getPointer(long i)
FileReaderResult	GetReader(FileSource source)
FileReaderResult	GetReader(FileSource source, ScanOptions arg1) \brief Return a FileReader on the given source.
SchemaResult	Inspect(FileSource source) \brief Return the schema of the file if possible.
BoolResult	IsSupported(FileSource source) \}
FileFragmentResult	MakeFragment(FileSource source, Expression partition_expression, Schema physical_schema) \brief Create a Fragment targeting all RowGroups.
ParquetFileFragmentResult	MakeFragment(FileSource source, Expression partition_expression, Schema physical_schema, int[] row_groups)
ParquetFileFragmentResult	MakeFragment(FileSource source, Expression partition_expression, Schema physical_schema, IntBuffer row_groups)
ParquetFileFragmentResult	MakeFragment(FileSource source, Expression partition_expression, Schema physical_schema, IntPointer row_groups) \brief Create a Fragment, restricted to the specified row groups.
FileWriterResult	MakeWriter(OutputStream destination, Schema schema, FileWriteOptions options, FileLocator destination_locator) \brief Create a writer for this format.
ParquetFileFormat	position(long position)
TimeUnit.type	reader_options_coerce_int96_timestamp_unit()
ParquetFileFormat	reader_options_coerce_int96_timestamp_unit(TimeUnit.type setter)
StringUnorderedSet	reader_options_dict_columns() \defgroup parquet-file-format-arrow-reader-properties properties which correspond to members of parquet::ArrowReaderProperties.
ParquetFileFormat	reader_options_dict_columns(StringUnorderedSet setter)
RecordBatchGeneratorResult	ScanBatchesAsync(ScanOptions options, FileFragment file)
ScanTaskIteratorResult	ScanFile(ScanOptions options, FileFragment file) \brief Open a file for scanning
String	type_name() \brief The name identifying the kind of file format

Methods inherited from class org.bytedeco.arrow_dataset.FileFormat

default_fragment_scan_options, default_fragment_scan_options, MakeFragment, MakeFragment, MakeFragment

Methods inherited from class org.bytedeco.javacpp.Pointer

address, asBuffer, asByteBuffer, availablePhysicalBytes, calloc, capacity, capacity, close, deallocate, deallocate, deallocateReferences, deallocator, deallocator, equals, fill, formatBytes, free, getDirectBufferAddress, getPointer, getPointer, getPointer, hashCode, interruptDeallocatorThread, isNull, isNull, limit, limit, malloc, maxBytes, maxPhysicalBytes, memchr, memcmp, memcpy, memmove, memset, offsetAddress, offsetof, offsetof, parseBytes, physicalBytes, physicalBytesInaccurate, position, put, realloc, referenceCount, releaseReference, retainReference, setNull, sizeof, sizeof, toString, totalBytes, totalCount, totalPhysicalBytes, withDeallocator, zero

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

ParquetFileFormat

public ParquetFileFormat(Pointer p)

Pointer cast constructor. Invokes Pointer(Pointer).

ParquetFileFormat

public ParquetFileFormat(long size)

Native array allocator. Access with Pointer.position(long).

ParquetFileFormat

public ParquetFileFormat()

ParquetFileFormat

public ParquetFileFormat(@Const @ByRef
                         ReaderProperties reader_properties)

Convenience constructor which copies properties from a parquet::ReaderProperties. memory_pool will be ignored.

Method Detail

position

public ParquetFileFormat position(long position)

Overrides:

position in class Pointer

getPointer

public ParquetFileFormat getPointer(long i)

Overrides:

getPointer in class Pointer

type_name

@StdString
public String type_name()

Description copied from class: FileFormat

\brief The name identifying the kind of file format

Overrides:

type_name in class FileFormat

Equals

@Cast(value="bool")
public boolean Equals(@Const @ByRef
                                           FileFormat other)

Overrides:

Equals in class FileFormat

reader_options_dict_columns

@Name(value="reader_options.dict_columns")
 @ByRef
public StringUnorderedSet reader_options_dict_columns()

\defgroup parquet-file-format-arrow-reader-properties properties which correspond to members of parquet::ArrowReaderProperties. We don't embed parquet::ReaderProperties directly because column names (rather than indices) are used to indicate dictionary columns, and other options are deferred to scan time. \{

reader_options_dict_columns

public ParquetFileFormat reader_options_dict_columns(StringUnorderedSet setter)

reader_options_coerce_int96_timestamp_unit

@Name(value="reader_options.coerce_int96_timestamp_unit")
public TimeUnit.type reader_options_coerce_int96_timestamp_unit()

reader_options_coerce_int96_timestamp_unit

public ParquetFileFormat reader_options_coerce_int96_timestamp_unit(TimeUnit.type setter)

IsSupported

@ByVal
public BoolResult IsSupported(@Const @ByRef
                                      FileSource source)

\}

Overrides:

IsSupported in class FileFormat

Inspect

@ByVal
public SchemaResult Inspect(@Const @ByRef
                                    FileSource source)

\brief Return the schema of the file if possible.

Overrides:

Inspect in class FileFormat

ScanFile

@ByVal
public ScanTaskIteratorResult ScanFile(@SharedPtr
                                               ScanOptions options,
                                               @Const @SharedPtr @ByRef
                                               FileFragment file)

\brief Open a file for scanning

Overrides:

ScanFile in class FileFormat

ScanBatchesAsync

@ByVal
public RecordBatchGeneratorResult ScanBatchesAsync(@SharedPtr
                                                           ScanOptions options,
                                                           @Const @SharedPtr @ByRef
                                                           FileFragment file)

Overrides:

ScanBatchesAsync in class FileFormat

CountRows

@ByVal
public LongOptionalFuture CountRows(@Const @SharedPtr @ByRef
                                            FileFragment file,
                                            @ByVal
                                            Expression predicate,
                                            @SharedPtr
                                            ScanOptions options)

Overrides:

CountRows in class FileFormat

MakeFragment

@ByVal
public FileFragmentResult MakeFragment(@ByVal
                                               FileSource source,
                                               @ByVal
                                               Expression partition_expression,
                                               @SharedPtr @ByVal
                                               Schema physical_schema)

\brief Create a Fragment targeting all RowGroups.

Overrides:

MakeFragment in class FileFormat

MakeFragment

@ByVal
public ParquetFileFragmentResult MakeFragment(@ByVal
                                                      FileSource source,
                                                      @ByVal
                                                      Expression partition_expression,
                                                      @SharedPtr @ByVal
                                                      Schema physical_schema,
                                                      @StdVector
                                                      IntPointer row_groups)

\brief Create a Fragment, restricted to the specified row groups.

MakeFragment

@ByVal
public ParquetFileFragmentResult MakeFragment(@ByVal
                                                      FileSource source,
                                                      @ByVal
                                                      Expression partition_expression,
                                                      @SharedPtr @ByVal
                                                      Schema physical_schema,
                                                      @StdVector
                                                      IntBuffer row_groups)

MakeFragment

@ByVal
public ParquetFileFragmentResult MakeFragment(@ByVal
                                                      FileSource source,
                                                      @ByVal
                                                      Expression partition_expression,
                                                      @SharedPtr @ByVal
                                                      Schema physical_schema,
                                                      @StdVector
                                                      int[] row_groups)

GetReader

@ByVal
public FileReaderResult GetReader(@Const @ByRef
                                          FileSource source,
                                          ScanOptions arg1)

\brief Return a FileReader on the given source.

GetReader

@ByVal
public FileReaderResult GetReader(@Const @ByRef
                                          FileSource source)

MakeWriter

@ByVal
public FileWriterResult MakeWriter(@SharedPtr
                                           OutputStream destination,
                                           @SharedPtr @ByVal
                                           Schema schema,
                                           @SharedPtr
                                           FileWriteOptions options,
                                           @ByVal
                                           FileLocator destination_locator)

Description copied from class: FileFormat

\brief Create a writer for this format.

Overrides:

MakeWriter in class FileFormat

DefaultWriteOptions

@SharedPtr
public FileWriteOptions DefaultWriteOptions()

Description copied from class: FileFormat

\brief Get default write options for this format.

Overrides:

DefaultWriteOptions in class FileFormat