← All databases

Oracle into Apache Arrow with ADBC

Oracle 23ai Free, read and written through adbcBridge — the ADBC driver that loads the database's ODBC driver — verified on Linux, macOS arm64 and Windows x64. Everything below is the compatibility matrix's record for this row.

Status

Driver and connection string

ODBC driver

Instant Client ODBC 23

Wire

native ODBC

adbcBridge

0.1.1 (release)

The connection string the matrix used, as the connection strings reference records it (Databases with their own native protocol and ODBC driver); {drv} is the driver library or its name from odbcinst.ini, and the host, port and credentials are the test server's.

Driver={drv};DBQ=127.0.0.1:11521/FREEPDB1;UID=adbc;PWD=adbc;

Python

import adbcbridge

# {drv}: the ODBC driver library (path), or its name from odbcinst.ini
conn = adbcbridge.connect(uri="Driver={drv};DBQ=127.0.0.1:11521/FREEPDB1;UID=adbc;PWD=adbc;")
with conn.cursor() as cur:
    cur.execute("SELECT * FROM my_table")
    table = cur.fetch_arrow_table()      # pyarrow.Table

Polars and pandas

import polars as pl, pandas as pd

df = pl.read_database("SELECT * FROM my_table", connection=conn)   # Polars, Arrow-native
pdf = pd.read_sql("SELECT * FROM my_table", conn)                   # pandas 2.2+, ADBC connection

pip install adbcbridge brings the driver library; the Oracle ODBC driver is installed the way its vendor documents, then named in Driver=. Rust, Go, C# and Java use the same connection string through their ADBC driver managers — see the docs.

What the matrix recorded

set NLS_LANG=.AL32UTF8 for non-ASCII; no SQL_C_SBIGINT, so 64-bit ints are sent as numeric text.

What this stack needed

• SQORA's rowset is fixed for the life of a cursor: changing SQL_ATTR_ROW_ARRAY_SIZE on an open cursor is accepted (SQLSetStmtAttr succeeds, SQLGetStmtAttr reads it back) and then goes wrong in one of three unannounced ways.
• Raising it segfaults inside libsqora on a later SQLFetch (bcoReturnColData through bcoCacheFetch, a per-rowset slot sized at execute time) -- on a (NUMBER, CLOB) cursor and equally on cursors with no LOB at all; whether a given raise dies depends on how deep the cursor already is, so the same raise survives after one rowset and crashes after twelve.
• Where it does not crash it can rewind, re-delivering rows already returned and dropping others while still ending on the right total.
• Lowering it never crashes and is no better: the driver keeps stepping by the original size and returns only the first N rows of each block, so 100,000 rows read at 1,024 and dropped to 128 come back as 13,440-14,336 depending on where the change falls, SQL_NO_DATA and no diagnostic.
• Reproduced with plain SQLBindCol/SQLFetch, nothing of ours on the stack.
• The reader therefore settles the rowset before the first fetch and never moves it (fixed_rowset), and a value that outgrew its bound buffer cannot be repaired once the rowset holds more than one row -- SQLGetData answers HY109 there, SQLSetPos HY109 and SQLFetchScroll HY106 at any size, although SQL_GETDATA_EXTENSIONS advertises `SQL_GD_BLOCK

Read next

• Which older engine releases pass the matrix entry unchanged
• Oracle's SQORA driver takes a rowset change, then crashes or loses rows