dbt-fal adapter is the ✨easiest✨ way to run your dbt Python models.
Starting with dbt v1.3, you can now build your dbt models in Python. However the developer experience with existing datawarehouse Python runtimes is not ideal.
dbt-fal provides the best environment to run your Python models that works with all other data warehouses. This includes Postgres, Redshift which do not have Python support, as well as Bigquery, Snowflake which are too hard to work with.
With dbt-fal, you can:
- Build and test your models locally
- Isolate each model to run in its own environment with its own dependencies
- Run your Python models in the ☁️ cloud ☁️ with elasticly scaling Python environments and pay for only what you use.
- Even add GPUs to your models for some heavy workloads such as training ML models. 🤖
1. Install dbt-fal
pip install dbt-fal[bigquery, snowflake] Add your current warehouse here
2. Update your
profiles.yml and add the fal adapter
Add another entry to
outputs in your desired profile (below we've added
target: dev_with_fal # target points at the new output
dev_with_fal: # Name of your new output
db_profile: dev_bigquery # This points to your main adapter
Don't forget to point to your main adapter with the
db_profile attribute. This is how the fal adapter knows how to connect to your data warehouse.
3. Run dbt
That is it! It is really that simple 😊