Database · native ODBC

Exasol into Apache Arrow with ADBC

Exasol 2025.1.14, read and written through adbcBridge — the ADBC driver that loads the database's ODBC driver — verified on Linux, macOS arm64 and Windows x64. Everything below is the compatibility matrix's record for this row.

Status

Linux✓ pass

PASS

macOS arm64✓ pass

PASS (EXASolution (via ODBC) 2025.01.0014, Exasol ODBC 26.2.8 universal, the unixODBC variant, against the Linux-hosted server over one LAN hop, so the numbers are network-bound: fetch 229–270k rows/s, ingest 6.6–7.3k rows/s, pyodbc within noise of the bridge on both; the entry ran verbatim with only EXAHOST changed)

Windows x64✓ pass

PASS (EXASolution (via ODBC) 2025.01.0014, Exasol ODBC 26.2.8 EXAODBC.dll registered as EXASolution Driver — installed machine-wide, because Windows' driver manager reads drivers from HKLM only and answers a DLL path in DRIVER= with IM002; exasol/docker-db under Docker Desktop, privileged, 6 GB; the entry verbatim, the three Exasol quirks apply; fetch 730–750k rows/s, ingest 40–42k rows/s with pyodbc at 105k on ingest, the same shape as Linux)

Driver and connection string

ODBC driver: Exasol ODBC 25.2.1 (libexaodbc.so, native wire on 8563)
Wire: native ODBC
adbcBridge: 0.1.1 (release)

The connection string the matrix used, as the connection strings reference records it (Databases with their own native protocol and ODBC driver); {drv} is the driver library or its name from odbcinst.ini, and the host, port and credentials are the test server's.

Driver={drv};EXAHOST=127.0.0.1:18563;EXAUID=sys;EXAPWD=exasol;SSLCertificate=SSL_VERIFY_NONE;

Python

import adbcbridge

# {drv}: the ODBC driver library (path), or its name from odbcinst.ini
conn = adbcbridge.connect(uri="Driver={drv};EXAHOST=127.0.0.1:18563;EXAUID=sys;EXAPWD=exasol;SSLCertificate=SSL_VERIFY_NONE;")
with conn.cursor() as cur:
    cur.execute("SELECT * FROM my_table")
    table = cur.fetch_arrow_table()      # pyarrow.Table

Polars and pandas

import polars as pl, pandas as pd

df = pl.read_database("SELECT * FROM my_table", connection=conn)   # Polars, Arrow-native
pdf = pd.read_sql("SELECT * FROM my_table", conn)                   # pandas 2.2+, ADBC connection

pip install adbcbridge brings the driver library; the Exasol ODBC driver is installed the way its vendor documents, then named in Driver=. Rust, Go, C# and Java use the same connection string through their ADBC driver managers — see the docs.

What the matrix recorded

in-memory columnar warehouse on its own first-party driver.

What this stack needed

Three driver quirks, all new and all keyed on SQL_DRIVER_NAME "exaodbc".
(1) Exasol has no binary column type at all — BLOB is 0A000 "Feature not supported: data type BLOB" and VARBINARY/BINARY/RAW are not words its parser knows — and the driver refuses the matching C type outright: SQLBindParameter with SQL_C_BINARY answers HY003 "Invalid application buffer type: SQL_C_BINARY" for any target column, so an Arrow binary column could not reach the server at all; it now goes as SQL_C_CHAR into a VARCHAR (binary_param_as_varchar), where the bytes store and read back byte for byte.
(2) A NULL parameter described as SQL_DECIMAL cannot be bound with SQL_C_DEFAULT: SQLExecute answers SI002 "C-Type not supported" and HY010 "Error creating prepared statement header", failing the whole statement.
That is not a corner case here — Exasol has no narrow integer type, INT/INTEGER/BIGINT are all aliases of DECIMAL, and SQLDescribeParam reports SQL_DECIMAL for every one of them, so a NULL in *any* numeric column hit it; SQL_C_CHAR with a NULL data pointer is accepted (null_decimal_param_as_char).
(3) SQL_GETDATA_EXTENSIONS is 0xf — `ANY_COLUMN