Types: Add support for BINARY columns

[amotl]

About

Improvements to be mainlined from https://github.com/pyveci/supertask/blob/19316e2/supertask/store/cratedb.py. The support for binary data types is being emulated by serializing them to STRING using base64. In that way, pickled data can be stored and retrieved without further ado.

[matriv]

Keep in mind that CrateDB also supports BitString type, but I don't think that's what we want here, right?

[amotl]

Keep in mind that CrateDB also supports BitString type, but I don't think that's what we want here, right?

Right. Here, it is about providing an emulation for a BLOB-type column. However, expanding type support to include the BIT type will be nice to have for the sake of completeness. So far, I didn't need it, but I am sure others will.

[kneth]

We need tests for BLOBs.

[bgunebakan]

Hi @florinutz @matriv,
Binary support finished, ready for another look. Thanks both for the earlier reviews. I kindly ask another look and your review again. I summary of what changed and why.

Fixed: payloads were capped at ~24 KB

The previous patch compiled binary columns to a plain STRING, which CrateDB indexes by default. Lucene's 32766-byte term limit therefore capped the payload at 24,573 bytes of binary, a 1 MB LargeBinary failed with Document contains at least one immense term. For a type called LargeBinary that's a defect, and there was no way around it: crate_columnstore=False was rejected by the isinstance(column.type, String) guard, because sa.LargeBinary isn't a String subclass.

Binary columns are now created as STRING INDEX OFF STORAGE WITH (columnstore = false). Verified round tripping 1 MiB and 5 MB byte-exact; WHERE =, ORDER BY and GROUP BY still work with both disabled.

Those clauses are emitted from CrateDDLCompiler.get_column_specification, not from visit_BLOB, visit_BLOB also serves CAST. There's a test pinning that.

Other changes

• BINARY and VARBINARY are covered too; previously they emitted unparseable DDL and crashed at bind time with module 'crate.client' has no attribute 'Binary'
• Renamed the impl class LargeBinary → BinaryBase64, so it stops shadowing sa.LargeBinary, and dropped it from the public exports.
• Removed the if dialect.dbapi is None branch, and replaced the copy-pasted SQLAlchemy docstring with one describing the actual emulation and its caveats (33% overhead, reflection yields sa.String, foreign writes must be valid base64)
• While measuring, I found binary wasn't the only gap: 10 of 44 core SQLAlchemy types emitted DDL CrateDB can't parse. This branch also fixes the other five (CLOB, NCHAR, NVARCHAR, DATETIME, DATE). I know it's out of the scope of this PR but I did a quick fix.

Tests

I added new test and validated the tests by reverting the fix.

Verified against CrateDB 6.4.3 and 6.5.0, on SQLAlchemy 1.3.24 and 2.0.50. (Unfortunately we don't have matrix to run tests for different CrateDB versions, I will tackle that on other PR.)

Best,