org.bytedeco.parquet

Class FileMetaData

java.lang.Object

org.bytedeco.javacpp.Pointer

org.bytedeco.parquet.FileMetaData

All Implemented Interfaces:

AutoCloseable

@Namespace(value="parquet")
 @NoOffset
 @Properties(inherit=parquet.class)
public class FileMetaData
extends Pointer

\brief FileMetaData is a proxy around format::FileMetaData.

Nested Class Summary

Nested classes/interfaces inherited from class org.bytedeco.javacpp.Pointer

Pointer.CustomDeallocator, Pointer.Deallocator, Pointer.NativeDeallocator, Pointer.ReferenceCounter

Field Summary

Fields inherited from class org.bytedeco.javacpp.Pointer

address, capacity, limit, position

Constructor Summary

Constructors

Constructor and Description
FileMetaData(Pointer p) Pointer cast constructor.

Method Summary

Modifier and Type	Method and Description
void	AppendRowGroups(FileMetaData other) \brief Merge row groups from another metadata file into this one.
boolean	can_decompress() \brief Indicate if all of the FileMetadata's RowGroups can be decompressed.
String	created_by() \brief Return the application's user-agent string of the writer.
EncryptionAlgorithm	encryption_algorithm()
boolean	Equals(FileMetaData other)
String	footer_signing_key_metadata()
boolean	is_encryption_algorithm_set()
KeyValueMetadata	key_value_metadata()
static FileMetaData	Make(Pointer serialized_metadata, int[] inout_metadata_len)
static FileMetaData	Make(Pointer serialized_metadata, int[] inout_metadata_len, InternalFileDecryptor file_decryptor)
static FileMetaData	Make(Pointer serialized_metadata, IntBuffer inout_metadata_len)
static FileMetaData	Make(Pointer serialized_metadata, IntBuffer inout_metadata_len, InternalFileDecryptor file_decryptor)
static FileMetaData	Make(Pointer serialized_metadata, IntPointer inout_metadata_len)
static FileMetaData	Make(Pointer serialized_metadata, IntPointer inout_metadata_len, InternalFileDecryptor file_decryptor) \brief Create a FileMetaData from a serialized thrift message.
int	num_columns() \brief The number of top-level columns in the schema.
int	num_row_groups() \brief The number of row groups in the file.
long	num_rows() \brief The total number of rows.
int	num_schema_elements() \brief The number of flattened schema elements.
RowGroupMetaData	RowGroup(int index) \brief Return the RowGroupMetaData of the corresponding row group ordinal.
SchemaDescriptor	schema()
String	SerializeToString() \brief Return Thrift-serialized representation of the metadata as a string
void	set_file_path(BytePointer path)
void	set_file_path(String path) \brief Set a path to all ColumnChunk for all RowGroups.
int	size() \brief Size of the original thrift encoded metadata footer.
FileMetaData	Subset(int[] row_groups)
FileMetaData	Subset(IntBuffer row_groups)
FileMetaData	Subset(IntPointer row_groups) \brief Return a FileMetaData containing a subset of the row groups in this FileMetaData.
boolean	VerifySignature(Pointer signature) \brief Verify signature of FileMetaData when file is encrypted but footer is not encrypted (plaintext footer).
ParquetVersion.type	version() \brief Return the "version" of the file WARNING: The value returned by this method is unreliable as 1) the Parquet file metadata stores the version as a single integer and 2) some producers are known to always write a hardcoded value.
ApplicationVersion	writer_version() \brief Return the application's version of the writer.
void	WriteTo(OutputStream dst)

Methods inherited from class org.bytedeco.javacpp.Pointer

address, asBuffer, asByteBuffer, availablePhysicalBytes, calloc, capacity, capacity, close, deallocate, deallocate, deallocateReferences, deallocator, deallocator, equals, fill, formatBytes, free, getDirectBufferAddress, getPointer, getPointer, getPointer, getPointer, hashCode, interruptDeallocatorThread, isNull, isNull, limit, limit, malloc, maxBytes, maxPhysicalBytes, memchr, memcmp, memcpy, memmove, memset, offsetAddress, offsetof, offsetof, parseBytes, physicalBytes, physicalBytesInaccurate, position, position, put, realloc, referenceCount, releaseReference, retainReference, setNull, sizeof, sizeof, toString, totalBytes, totalCount, totalPhysicalBytes, withDeallocator, zero

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

FileMetaData

public FileMetaData(Pointer p)

Pointer cast constructor. Invokes Pointer(Pointer).

Method Detail

Make

@SharedPtr
public static FileMetaData Make(@Const
                                            Pointer serialized_metadata,
                                            @Cast(value="uint32_t*")
                                            IntPointer inout_metadata_len,
                                            @SharedPtr
                                            InternalFileDecryptor file_decryptor)

\brief Create a FileMetaData from a serialized thrift message.

Make

@SharedPtr
public static FileMetaData Make(@Const
                                            Pointer serialized_metadata,
                                            @Cast(value="uint32_t*")
                                            IntPointer inout_metadata_len)

Make

@SharedPtr
public static FileMetaData Make(@Const
                                            Pointer serialized_metadata,
                                            @Cast(value="uint32_t*")
                                            IntBuffer inout_metadata_len,
                                            @SharedPtr
                                            InternalFileDecryptor file_decryptor)

Make

@SharedPtr
public static FileMetaData Make(@Const
                                            Pointer serialized_metadata,
                                            @Cast(value="uint32_t*")
                                            IntBuffer inout_metadata_len)

Make

@SharedPtr
public static FileMetaData Make(@Const
                                            Pointer serialized_metadata,
                                            @Cast(value="uint32_t*")
                                            int[] inout_metadata_len,
                                            @SharedPtr
                                            InternalFileDecryptor file_decryptor)

Make

@SharedPtr
public static FileMetaData Make(@Const
                                            Pointer serialized_metadata,
                                            @Cast(value="uint32_t*")
                                            int[] inout_metadata_len)

Equals

@Cast(value="bool")
public boolean Equals(@Const @ByRef
                                           FileMetaData other)

num_columns

public int num_columns()

\brief The number of top-level columns in the schema. Parquet thrift definition requires that nested schema elements are flattened. This method returns the number of columns in the un-flattened version.

num_schema_elements

public int num_schema_elements()

\brief The number of flattened schema elements. Parquet thrift definition requires that nested schema elements are flattened. This method returns the total number of elements in the flattened list.

num_rows

@Cast(value="int64_t")
public long num_rows()

\brief The total number of rows.

num_row_groups

public int num_row_groups()

\brief The number of row groups in the file.

RowGroup

@UniquePtr
public RowGroupMetaData RowGroup(int index)

\brief Return the RowGroupMetaData of the corresponding row group ordinal. WARNING, the returned object references memory location in it's parent (FileMetaData) object. Hence, the parent must outlive the returned object.

Parameters:

index - [in] of the RowGroup to retrieve.

Throws:

ParquetException - if the index is out of bound.

version

public ParquetVersion.type version()

\brief Return the "version" of the file WARNING: The value returned by this method is unreliable as 1) the Parquet file metadata stores the version as a single integer and 2) some producers are known to always write a hardcoded value. Therefore, you cannot use this value to know which features are used in the file.

created_by

@StdString
public String created_by()

\brief Return the application's user-agent string of the writer.

writer_version

@Const
 @ByRef
public ApplicationVersion writer_version()

\brief Return the application's version of the writer.

size

@Cast(value="uint32_t")
public int size()

\brief Size of the original thrift encoded metadata footer.

can_decompress

@Cast(value="bool")
public boolean can_decompress()

\brief Indicate if all of the FileMetadata's RowGroups can be decompressed. This will return false if any of the RowGroup's page is compressed with a compression format which is not compiled in the current parquet library.

is_encryption_algorithm_set

@Cast(value="bool")
public boolean is_encryption_algorithm_set()

encryption_algorithm

@ByVal
public EncryptionAlgorithm encryption_algorithm()

footer_signing_key_metadata

@StdString
public String footer_signing_key_metadata()

VerifySignature

@Cast(value="bool")
public boolean VerifySignature(@Const
                                                    Pointer signature)

\brief Verify signature of FileMetaData when file is encrypted but footer is not encrypted (plaintext footer).

WriteTo

public void WriteTo(OutputStream dst)

SerializeToString

@StdString
public String SerializeToString()

\brief Return Thrift-serialized representation of the metadata as a string

schema

@Const
public SchemaDescriptor schema()

key_value_metadata

@Const
 @SharedPtr
public KeyValueMetadata key_value_metadata()

set_file_path

public void set_file_path(@StdString
                          String path)

\brief Set a path to all ColumnChunk for all RowGroups. Commonly used by systems (Dask, Spark) who generates an metadata-only parquet file. The path is usually relative to said index file.

Parameters:

path - [in] to set.

set_file_path

public void set_file_path(@StdString
                          BytePointer path)

AppendRowGroups

public void AppendRowGroups(@Const @ByRef
                            FileMetaData other)

\brief Merge row groups from another metadata file into this one. The schema of the input FileMetaData must be equal to the schema of this object. This is used by systems who creates an aggregate metadata-only file by concatenating the row groups of multiple files. This newly created metadata file acts as an index of all available row groups.

Parameters:

other - [in] FileMetaData to merge the row groups from.

Throws:

ParquetException - if schemas are not equal.

Subset

@SharedPtr
public FileMetaData Subset(@StdVector
                                       IntPointer row_groups)

\brief Return a FileMetaData containing a subset of the row groups in this FileMetaData.

Subset

@SharedPtr
public FileMetaData Subset(@StdVector
                                       IntBuffer row_groups)

Subset

@SharedPtr
public FileMetaData Subset(@StdVector
                                       int[] row_groups)