Internal vs. External Storage? What's the Limit of External Tables
External tables have survived 25 years from Oracle 9i to DuckLake. A look at when to store data internally vs. externally, with a hot/cold benchmark.
ODBC Extension
duckdb.orgThe odbc_scanner extension allows connecting to other databases (using their ODBC drivers) and run queries with the odbc_query or copy data from DuckDB with the odbc_copy functions. The extension is also available under the alias odbc. current_duckdb_version Installing and Loading On Linux and macOS the extension requires unixODBC driver manager to be installed. See below for installation instructions. The extension can be installed automatically, but needs to be loaded manually with: LOAD odbc; Usage Example -- load extension LOAD odbc; -- open ODBC connection to a remote DB SET VARIABLE conn = odbc_connect('Driver={Oracle driver};DBQ=//127.0.0.1:1521/XE', 'scott', 'tiger'); -- simple query FROM…
Internal vs. External Storage: What's the Limit of External Tables?
25 years of external tables, from Oracle 9i to DuckLake. When to use internal storage vs. external tables, why they keep getting re-added to every major data warehouse, and a benchmark showing the speed tax for cold data. | Reading time: 26 min read