File Metadata
The parser needs to read some metadata before parsing column data. The first one is the file metadata, located at the end of the file, right before the footer magic number.
Task
Implement the read_file_metadata function in src/file_metadata.rs. It takes the entire file data
as Bytes and returns a FileMetaData struct.
pub fn read_file_metadata(data: Bytes) -> Result<FileMetaData> {
todo!("step02: read file metadata.")
}To parse it, you should read the 4-byte file metadata length first, then the raw file metadata, and
use the read_thrift_metadata introduced in the Overview to convert it to
FileMetaData.
Test
The corresponding test is step02_file_metadata.
-// mod step02_file_metadata;
+mod step02_file_metadata;
Hints and Solution
Hint (how to get raw file metadata in bytes)
The 4-byte file metadata length can be parsed using
Bytes::get_u32_le.
Remember it is right before the footer magic number.
let metadata_size = data.slice(data.len() - 8..).get_u32_le();Then the raw file metadata in bytes can be extracted like this.
let metadata_bytes = data.slice(data.len() - 8 - metadata_size..data.len() - 8);Hint (how to parse file metadata)
The FileMetaData can be parsed using read_thrift_metadata.
let (metadata, _) = read_thrift_metadata::<FileMetaData>(metadata_bytes)?;Solution
pub fn read_file_metadata(data: Bytes) -> Result<FileMetaData> {
let metadata_size = data.slice(data.len() - 8..).get_u32_le() as usize;
let metadata_bytes = data.slice(data.len() - 8 - metadata_size..data.len() - 8);
let (metadata, _) = read_thrift_metadata::<FileMetaData>(metadata_bytes)?;
Ok(metadata)
}