← All databases

OpenLink Virtuoso into Apache Arrow with ADBC

OpenLink Virtuoso 7.2, read and written through adbcBridge — the ADBC driver that loads the database's ODBC driver — verified on Linux, macOS arm64 and Windows x64. Everything below is the compatibility matrix's record for this row.

Status

Driver and connection string

ODBC driver

Virtuoso ODBC (virtodbc.so, ANSI build)

Wire

native ODBC

adbcBridge

0.1.1 (release)

The connection string the matrix used, as the connection strings reference records it (Databases with their own native protocol and ODBC driver); {drv} is the driver library or its name from odbcinst.ini, and the host, port and credentials are the test server's.

Driver={drv};HOST=127.0.0.1:11111;UID=dba;PWD=adbc;

Python

import adbcbridge

# {drv}: the ODBC driver library (path), or its name from odbcinst.ini
conn = adbcbridge.connect(uri="Driver={drv};HOST=127.0.0.1:11111;UID=dba;PWD=adbc;")
with conn.cursor() as cur:
    cur.execute("SELECT * FROM my_table")
    table = cur.fetch_arrow_table()      # pyarrow.Table

Polars and pandas

import polars as pl, pandas as pd

df = pl.read_database("SELECT * FROM my_table", connection=conn)   # Polars, Arrow-native
pdf = pd.read_sql("SELECT * FROM my_table", conn)                   # pandas 2.2+, ADBC connection

pip install adbcbridge brings the driver library; the OpenLink Virtuoso ODBC driver is installed the way its vendor documents, then named in Driver=. Rust, Go, C# and Java use the same connection string through their ADBC driver managers — see the docs.

What the matrix recorded

ODBC-native server; driver quirks handled: SQL_C_WCHAR is a 4-byte wchar_t on both the parameter and the fetch side, not unixODBC's 2-byte SQLWCHAR (UTF-8 narrow path instead), SQL_C_SBIGINT is stored as 0 and read back as 0 without a diagnostic (64-bit ints sent as text declared SQL_NUMERIC, the one declaration that converts exactly), and datetime parameter arrays are strided by ColumnSize rather than by the C struct — a DATE32 bound with ColumnSize 0 repeats row 0 (so no parameter arrays); no BOOLEAN type.

What this stack needed

• Two more to know: an SQL_C_SLONG parameter is read as 8 bytes whatever BufferLength says, so two int32 parameters bound from adjacent slots fail with SR346: Integer out of range; and any Charset= value but the exact literal UTF-8 aborts the process inside SQLDriverConnect (GPF: Dkbox.c:638 Double free).
• The Unicode build virtodbcu.so is built to a 4-byte SQLWCHAR, so unixODBC's driver manager aborts reading its first error diagnostic — lurcher/unixODBC#239, see UPSTREAM.md; ingest 11.9k rows/s, fetch 1.04M rows/s

Read next

• Five things OpenLink Virtuoso's ODBC driver does silently and wrongly
• Vendor macOS drivers are built for iODBC's four-byte SQLWCHAR
• A NULL column that reads back as the previous row's value
• The process dies with SIGABRT on the first SQL error, and it is the manager