Apache Ignite into Apache Arrow with ADBC
Apache Ignite 2.17, read and written through adbcBridge — the ADBC driver that loads the database's ODBC driver — verified on Linux and Windows x64. Everything below is the compatibility matrix's record for this row.
Status
PASS
driver unavailable: platforms/cpp has only linux/ and win/ OS layers; the Darwin build stops at sys/sysinfo.h
PASS (Apache Ignite 2.17 container, ignite.odbc.dll 2.18.0 — an ANSI-only driver, no W entry points) — with the narrow-text route in this tree: the Windows driver manager maps every W call onto such a driver through the ANSI code page, so wide fetches came back héllo 🚀 and a statement literal 'héllo' matched nothing (pyodbc identical for both), while its narrow path is untouched UTF-8 in both directions (probed: SQL_C_CHAR reads byte-exact, a narrow SQLExecDirect literal matches). The fix keeps the existing wchar_as_utf8 fetch quirk alive on Windows for this driver and adds narrow_sql, which sends caller statement text through the narrow entry points; parameters were already narrow. Fetch 775,660 rows/s (316,230 through the wide path)
Driver and connection string
- ODBC driver
- Ignite ODBC (built from the sources in the image)
- Wire
- native ODBC
- adbcBridge
- 0.1.1 (release)
The connection string the matrix used, as the connection strings reference records it (Databases with their own native protocol and ODBC driver); {drv} is the driver library or its name from odbcinst.ini, and the host, port and credentials are the test server's.
Driver={drv};ADDRESS=127.0.0.1:11800;SCHEMA=PUBLIC;Python
import adbcbridge
# {drv}: the ODBC driver library (path), or its name from odbcinst.ini
conn = adbcbridge.connect(uri="Driver={drv};ADDRESS=127.0.0.1:11800;SCHEMA=PUBLIC;")
with conn.cursor() as cur:
cur.execute("SELECT * FROM my_table")
table = cur.fetch_arrow_table() # pyarrow.Table
Polars and pandas
import polars as pl, pandas as pd
df = pl.read_database("SELECT * FROM my_table", connection=conn) # Polars, Arrow-native
pdf = pd.read_sql("SELECT * FROM my_table", conn) # pandas 2.2+, ADBC connection
pip install adbcbridge brings the driver library; the Apache Ignite ODBC driver is installed the way its vendor documents, then named in Driver=. Rust, Go, C# and Java use the same connection string through their ADBC driver managers — see the docs.
What the matrix recorded
in-memory key-value grid with a SQL engine: no prebuilt Linux driver exists, so platforms/cpp is built root-free (-DWITH_ODBC=ON -DWITH_CORE=OFF, no JVM); driver quirks handled: no wide SQL type at all (SQL_WVARCHAR refused outright by SQLBindParameter, SQL_C_WCHAR sized in wchar_t) so the UTF-8 narrow path, as for Firebird, and column-wise parameter arrays that test the NULL indicator of row 0 for every row — a NULL below the first row is dropped rather than sent: a character column stores an empty string, and a BINARY column segfaults the client inside SQLExecute (WriteInt8Array is handed the -1 indicator as its length), so arrays are off (row-wise binding is refused outright).
What this stack needed
SQL_DRIVER_VERandSQL_DBMS_VERare both the hardcoded02.04.0000, neither the server's 2.17.0 nor the negotiated protocol 2.13.0.- Server side: every table is a cache and must declare a
PRIMARY KEY, which generated ingest DDL cannot, somode="create"is impossible (ingest_create=False) and the entry ingests by appending into a keyed table; identifiers fold to upper case and the driver reports no quote character.