Docs - dlthub:

• https://dlthub.com/docs/reference/installation
• https://dlthub.com/docs/tutorial/sql-database

sudo apt-get update && sudo apt-get update -y
 
mkdir dlthub_app
cd dlthub_app
 
python --version
python -m venv venv
 
source venv/bin/activate
 
pip install -U pip
 
pip install -U dlt
 
# databases
pip install "dlt[duckdb]"
pip install pymysql
 
# Configure Project
dlt init sql_database duckdb
 
# Install requirements not already met
pip install -r requirements.txt

Performance challenges:

• https://dlthub.com/docs/reference/performance#load