Database · native ODBC

IBM Db2 into Apache Arrow with ADBC

IBM Db2 12.1, read and written through adbcBridge — the ADBC driver that loads the database's ODBC driver — verified on Linux, macOS arm64 and Windows x64. Everything below is the compatibility matrix's record for this row.

Status

Linux✓ pass

PASS

macOS arm64✓ pass

PASS (Db2 12.01.0500, IBM macarm64 clidriver; server amd64 emulated)

Windows x64✓ pass

PASS (Db2 12.01.0500 in the ibmcom/db2 container; IBM clidriver 12.1.4 from the free "ODBC and CLI" zip, registered by hand as the short alias IBM DB2 ODBC DRIVER → clidriver\bin\db2cli64.dll — the 78-character name db2cli install -setup writes gets IM002 from the Windows driver manager, and the db2clio.dll it points at is not in that package). Authentication=SERVER must be in the connection string on Windows: the default SERVER_ENCRYPT negotiation fails SQL1042C (sqlerrp SQLEXSLC, rc 205, the client-side security-plugin step) from every process but db2cli.exe, which carries its own gsk8/ICC manifests — python and PowerShell P/Invoke fail identically, whatever DB2CODEPAGE, cwd or environment. The whole workload passes with that one keyword; fetch 398,715 rows/s, array ingest 81,651

Driver and connection string

ODBC driver: Db2 clidriver (libdb2.so)
Wire: native ODBC
adbcBridge: 0.1.1 (release)

The connection string the matrix used, as the connection strings reference records it (IBM DRDA (Db2 and Informix)); {drv} is the driver library or its name from odbcinst.ini, and the host, port and credentials are the test server's.

Driver={drv};Database=adbc;Hostname=127.0.0.1;Port=50000;Protocol=TCPIP;Uid=db2inst1;Pwd=Adbc2026;

Python

import adbcbridge

# {drv}: the ODBC driver library (path), or its name from odbcinst.ini
conn = adbcbridge.connect(uri="Driver={drv};Database=adbc;Hostname=127.0.0.1;Port=50000;Protocol=TCPIP;Uid=db2inst1;Pwd=Adbc2026;")
with conn.cursor() as cur:
    cur.execute("SELECT * FROM my_table")
    table = cur.fetch_arrow_table()      # pyarrow.Table

Polars and pandas

import polars as pl, pandas as pd

df = pl.read_database("SELECT * FROM my_table", connection=conn)   # Polars, Arrow-native
pdf = pd.read_sql("SELECT * FROM my_table", conn)                   # pandas 2.2+, ADBC connection

pip install adbcbridge brings the driver library; the IBM Db2 ODBC driver is installed the way its vendor documents, then named in Driver=. Rust, Go, C# and Java use the same connection string through their ADBC driver managers — see the docs.

What the matrix recorded

32-bit SQLLEN (see adbc.odbc.sqllen_32bit); ingest DDL spells an Arrow string as the widest VARCHAR, not the LONG VARCHAR the driver's SQLGetTypeInfo(SQL_LONGVARCHAR) names (deprecated; will not sort, group or de-duplicate -- SQL0134N on ORDER BY, GROUP BY, DISTINCT and UNION -- and has no bulk-insert path: ~7k rows/s whatever the batch size against ~430k for VARCHAR(32672) in a 20,000-row test, 30-56x on a warm database and over 200x through adbc_ingest on a cold one; an earlier run recorded ~700x)