Describe the bug
In native_iceberg_compat, the initialization had codes the following configuration flags -
conf.set("spark.sql.parquet.binaryAsString", "false");
conf.set("spark.sql.parquet.int96AsTimestamp", "true");
conf.set("spark.sql.caseSensitive", "false");
conf.set("spark.sql.parquet.inferTimestampNTZ.enabled", "true");
conf.set("spark.sql.legacy.parquet.nanosAsLong", "false");
These explicitly set the config to the default values for the configs.
We should apply the values specified by the end user instead and use defaults only if the value is not provided.
Steps to reproduce
No response
Expected behavior
No response
Additional context
No response
Describe the bug
In native_iceberg_compat, the initialization had codes the following configuration flags -
These explicitly set the config to the default values for the configs.
We should apply the values specified by the end user instead and use defaults only if the value is not provided.
Steps to reproduce
No response
Expected behavior
No response
Additional context
No response