← All databases

SAP HANA Express into Apache Arrow with ADBC

SAP HANA Express 2.00.088 (HANA 2.0 SPS08), read and written through adbcBridge — the ADBC driver that loads the database's ODBC driver — verified on Linux, macOS arm64 and Windows x64. Everything below is the compatibility matrix's record for this row.

Status

Driver and connection string

ODBC driver

SAP HANA client ODBC 2.29.25 (libodbcHDB.so, native wire)

Wire

native ODBC

adbcBridge

0.1.1 (release)

The connection string the matrix used, as the connection strings reference records it (Databases with their own native protocol and ODBC driver); {drv} is the driver library or its name from odbcinst.ini, and the host, port and credentials are the test server's.

Driver={drv};SERVERNODE=127.0.0.1:39017;UID=SYSTEM;PWD=AdbcBridge2026;DATABASENAME=HXE;

Python

import adbcbridge

# {drv}: the ODBC driver library (path), or its name from odbcinst.ini
conn = adbcbridge.connect(uri="Driver={drv};SERVERNODE=127.0.0.1:39017;UID=SYSTEM;PWD=AdbcBridge2026;DATABASENAME=HXE;")
with conn.cursor() as cur:
    cur.execute("SELECT * FROM my_table")
    table = cur.fetch_arrow_table()      # pyarrow.Table

Polars and pandas

import polars as pl, pandas as pd

df = pl.read_database("SELECT * FROM my_table", connection=conn)   # Polars, Arrow-native
pdf = pd.read_sql("SELECT * FROM my_table", conn)                   # pandas 2.2+, ADBC connection

pip install adbcbridge brings the driver library; the SAP HANA Express ODBC driver is installed the way its vendor documents, then named in Driver=. Rust, Go, C# and Java use the same connection string through their ADBC driver managers — see the docs.

What the matrix recorded

SAP's in-memory column store, first-party server (saplabs/hanaexpress, which pulled anonymously) driven by SAP's own first-party driver from the free HANA client download — the EULA that gates it is passed with the cookie its checkbox sets, and the ODBC member of the tarball needs no installer, no LD_LIBRARY_PATH and no .ini of its own.

What this stack needed

• Every workload type is a native HANA type and round-trips exactly, emoji and microseconds included (i is int32; a bare CREATE TABLE is a column table, default_table_type = column, and a CREATE ROW TABLE passes the same workload).
• Two HANA spellings to know: TIMESTAMP takes no precision argument (TIMESTAMP(6) is 42000, 257 sql syntax error: incorrect syntax near "(") because it is always 7 fractional digits, and there is no multi-row VALUES at all (INSERT INTO t VALUES (1,'a'),(2,'b') is 42000, 257 ... incorrect syntax near ",", with literals as with parameters).
• Four driver quirks, all keyed on libodbchdb.
• The first is a correctness bug independent of this matrix: the driver decodes narrow statement text as Latin-1, not as the UTF-8 bytes unixODBC handed it, so the UTF-8 of héllo 🚀 (68 c3a9 6c6c6f 20 f09f9a80) is stored as the eleven Latin-1 characters those bytes spell (LENGTH(s) 11, not 8) — the same corruption the Windows driver manager causes for every driver, here caused by one driver on every platform.
• It hides because it is self-consistent within narrow statements, and shows the moment a literal must match a value sent as a bound parameter (which travels SQL_C_WCHAR and arrives correct).
• No connection property (CHAR_AS_UTF8, CHAR_SET, charset) and no locale (LC_ALL=C, en_US.UTF-8, C.UTF-8) changes it; SQLExecDirectW stores it exactly, so wide_sql (new) routes caller statement text through the W entry points on every platform — the mirror of the existing narrow_sql, reusing the conversion the Windows build already had.
• Second, SQLGetTypeInfo(SQL_TYPE_TIMESTAMP) names SECONDDATE first — HANA's whole-second timestamp (COLUMN_SIZE 19, MAXIMUM_SCALE 0, no CREATE_PARAMS) — with the 7-digit TIMESTAMP only in rows two and three, so generated ingest DDL silently dropped every microsecond; ddl_timestamp_type_name (new) gives the name outright, since SECONDDATE has no CREATE_PARAMS for fractional_time_type_format to ask for a scale and TIMESTAMP takes no precision argument either.
• Third, SQLGetTypeInfo(SQL_LONGVARCHAR) names CLOB, which HANA bars from ORDER BY (HY000, 264 invalid datatype: "V" LOB type in ORDER BY clause) and SELECT DISTINCT (HY000, 264 ... LOB type in distinct select clause) — the SQL Server TEXT trap — so ddl_string_as_max_varchar, the flag Db2 already sets, asks for VARCHAR(5000) instead, fully Unicode since HANA 2.0 merged VARCHAR into NVARCHAR.
• Fourth, prefer_param_arrays (the third driver to set it, after maodbc and Vertica, and here the server's doing rather than the driver's): with no multi-row VALUES, MultiRowSetup's probe is refused and ingest falls back to one execute per row, 7,675 rows/s against 1,142,092 with a bound array.
• Server side, startup is 158 s on a cold volume (system database, then the HXE tenant) and settles at ~5 GB under a 12 GB cap; 39013 (system DB) and 39017 (the HXE tenant) must be published unmapped, because the client resolves a tenant by asking for its port and reconnecting to exactly that number; --agree-to-sap-license and a world-writable mount holding passwords.json (the image's hxeadm is uid 12000) are required, and of SAP's four sysctls only kernel.shmmax is IPC-namespaced and settable per container — the entry was verified with the other three left at a stock Ubuntu's values.

Read next

• The SAP HANA ODBC client decodes narrow statement text as Latin-1
• Two ways a Go ODBC program dies before it prints anything