Polybase old dates into ORC files fails
The following CETAS statement fails. Apparently there's an issue with old dates and ORC files? Please fix.
CREATE EXTERNAL FILE FORMAT ORC_Snappy
FORMAT_TYPE = ORC
, DATA_COMPRESSION = 'org.apache.hadoop.io.compress.SnappyCodec'
create EXTERNAL TABLE dbo.testexternaltable WITH (DATASOURCE = myds, LOCATION = '/testexternaltable', FILEFORMAT = ORC_Snappy) AS
select cast('1910-08-13' as date) as dt;
I am also getting a similar problem on SQL Server 2016. But with a future date value of 2040-10-15.
Gerhard Brueckl commented
this actually applies to all external file formats except DELIMITED_TEXT
I also tested it with PARQUET, RCFILE and ORC
Greg Galloway commented
I should have posted the error message I get:
Msg 107090, Level 16, State 1, Line 11
Query aborted-- the maximum reject threshold (0 rows) was reached while reading from an external source: 1 rows rejected out of total 1 rows processed.
Column ordinal: 0, Expected data type: DATE, Offending value: TemporalValue: 3876 (Column Conversion Error), Error: Error converting from RCFile Type DATE to Sql type Timestamp.