org.bytedeco.arrow_dataset

Class FileSystemDatasetWriteOptions

java.lang.Object

org.bytedeco.javacpp.Pointer

org.bytedeco.arrow_dataset.FileSystemDatasetWriteOptions

All Implemented Interfaces:

AutoCloseable

@Namespace(value="arrow::dataset")
 @Properties(inherit=arrow_dataset.class)
public class FileSystemDatasetWriteOptions
extends Pointer

\brief Options for writing a dataset.

Nested Class Summary

Nested classes/interfaces inherited from class org.bytedeco.javacpp.Pointer

Pointer.CustomDeallocator, Pointer.Deallocator, Pointer.NativeDeallocator, Pointer.ReferenceCounter

Field Summary

Fields inherited from class org.bytedeco.javacpp.Pointer

address, capacity, limit, position

Constructor Summary

Constructors

Constructor and Description
FileSystemDatasetWriteOptions() Default native constructor.
FileSystemDatasetWriteOptions(long size) Native array allocator.
FileSystemDatasetWriteOptions(Pointer p) Pointer cast constructor.

Method Summary

Modifier and Type	Method and Description
String	base_dir() Root directory into which the dataset will be written.
FileSystemDatasetWriteOptions	base_dir(String setter)
String	basename_template() Template string used to generate fragment basenames.
FileSystemDatasetWriteOptions	basename_template(String setter)
arrow_dataset.ExistingDataBehavior	existing_data_behavior() Controls what happens if an output directory already exists.
FileSystemDatasetWriteOptions	existing_data_behavior(arrow_dataset.ExistingDataBehavior setter)
FileWriteOptions	file_write_options() Options for individual fragment writing.
FileSystemDatasetWriteOptions	file_write_options(FileWriteOptions setter)
FileSystem	filesystem() FileSystem into which a dataset will be written.
FileSystemDatasetWriteOptions	filesystem(FileSystem setter)
FileFormat	format()
FileSystemDatasetWriteOptions	getPointer(long i)
int	max_open_files() If greater than 0 then this will limit the maximum number of files that can be left open.
FileSystemDatasetWriteOptions	max_open_files(int setter)
int	max_partitions() Maximum number of partitions any batch may be written into, default is 1K.
FileSystemDatasetWriteOptions	max_partitions(int setter)
long	max_rows_per_file() If greater than 0 then this will limit how many rows are placed in any single file.
FileSystemDatasetWriteOptions	max_rows_per_file(long setter)
Partitioning	partitioning() Partitioning used to generate fragment paths.
FileSystemDatasetWriteOptions	partitioning(Partitioning setter)
FileSystemDatasetWriteOptions	position(long position)
arrow_dataset.FileWriterVisitor	writer_post_finish() Callback to be invoked against all FileWriters after they have called FileWriter::Finish().
FileSystemDatasetWriteOptions	writer_post_finish(arrow_dataset.FileWriterVisitor setter)
arrow_dataset.FileWriterVisitor	writer_pre_finish() Callback to be invoked against all FileWriters before they are finalized with FileWriter::Finish().
FileSystemDatasetWriteOptions	writer_pre_finish(arrow_dataset.FileWriterVisitor setter)

Methods inherited from class org.bytedeco.javacpp.Pointer

address, asBuffer, asByteBuffer, availablePhysicalBytes, calloc, capacity, capacity, close, deallocate, deallocate, deallocateReferences, deallocator, deallocator, equals, fill, formatBytes, free, getDirectBufferAddress, getPointer, getPointer, getPointer, hashCode, interruptDeallocatorThread, isNull, isNull, limit, limit, malloc, maxBytes, maxPhysicalBytes, memchr, memcmp, memcpy, memmove, memset, offsetAddress, offsetof, offsetof, parseBytes, physicalBytes, physicalBytesInaccurate, position, put, realloc, referenceCount, releaseReference, retainReference, setNull, sizeof, sizeof, toString, totalBytes, totalCount, totalPhysicalBytes, withDeallocator, zero

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

FileSystemDatasetWriteOptions

public FileSystemDatasetWriteOptions()

Default native constructor.

FileSystemDatasetWriteOptions

public FileSystemDatasetWriteOptions(long size)

Native array allocator. Access with Pointer.position(long).

FileSystemDatasetWriteOptions

public FileSystemDatasetWriteOptions(Pointer p)

Pointer cast constructor. Invokes Pointer(Pointer).

Method Detail

position

public FileSystemDatasetWriteOptions position(long position)

Overrides:

position in class Pointer

getPointer

public FileSystemDatasetWriteOptions getPointer(long i)

Overrides:

getPointer in class Pointer

file_write_options

@SharedPtr
public FileWriteOptions file_write_options()

Options for individual fragment writing.

file_write_options

public FileSystemDatasetWriteOptions file_write_options(FileWriteOptions setter)

filesystem

@SharedPtr
public FileSystem filesystem()

FileSystem into which a dataset will be written.

filesystem

public FileSystemDatasetWriteOptions filesystem(FileSystem setter)

base_dir

@StdString
public String base_dir()

Root directory into which the dataset will be written.

base_dir

public FileSystemDatasetWriteOptions base_dir(String setter)

partitioning

@SharedPtr
public Partitioning partitioning()

Partitioning used to generate fragment paths.

partitioning

public FileSystemDatasetWriteOptions partitioning(Partitioning setter)

max_partitions

public int max_partitions()

Maximum number of partitions any batch may be written into, default is 1K.

max_partitions

public FileSystemDatasetWriteOptions max_partitions(int setter)

basename_template

@StdString
public String basename_template()

Template string used to generate fragment basenames. {i} will be replaced by an auto incremented integer.

basename_template

public FileSystemDatasetWriteOptions basename_template(String setter)

max_open_files

@Cast(value="uint32_t")
public int max_open_files()

If greater than 0 then this will limit the maximum number of files that can be left open. If an attempt is made to open too many files then the least recently used file will be closed. If this setting is set too low you may end up fragmenting your data into many small files.

max_open_files

public FileSystemDatasetWriteOptions max_open_files(int setter)

max_rows_per_file

@Cast(value="uint64_t")
public long max_rows_per_file()

If greater than 0 then this will limit how many rows are placed in any single file. Otherwise there will be no limit and one file will be created in each output directory unless files need to be closed to respect max_open_files

max_rows_per_file

public FileSystemDatasetWriteOptions max_rows_per_file(long setter)

existing_data_behavior

public arrow_dataset.ExistingDataBehavior existing_data_behavior()

Controls what happens if an output directory already exists.

existing_data_behavior

public FileSystemDatasetWriteOptions existing_data_behavior(arrow_dataset.ExistingDataBehavior setter)

writer_pre_finish

@ByRef
public arrow_dataset.FileWriterVisitor writer_pre_finish()

Callback to be invoked against all FileWriters before they are finalized with FileWriter::Finish().

writer_pre_finish

public FileSystemDatasetWriteOptions writer_pre_finish(arrow_dataset.FileWriterVisitor setter)

writer_post_finish

@ByRef
public arrow_dataset.FileWriterVisitor writer_post_finish()

Callback to be invoked against all FileWriters after they have called FileWriter::Finish().

writer_post_finish

public FileSystemDatasetWriteOptions writer_post_finish(arrow_dataset.FileWriterVisitor setter)

format

@SharedPtr
public FileFormat format()