Using dbt-fal with fal cloud
fal cloud is our serverless compute solution that allows you to run Python models on a reliable and scalable infrastructure. Setting up dbt-fal with fal cloud is quick and straightforward.
0. Install fal
Skip this step, if you already have the latest version of fal installed,
pip install --upgrade fal[cloud]
1. Authenticate to fal cloud
fal cloud uses GitHub for authentication. Run the following command in your shell:
fal cloud login
Follow the link that's generated and login using GitHub. Come back to the shell, when ready.
2. Generate keys
Next, generate keys that will allow dbt to connect to fal cloud:
fal cloud generate-keys
This will print a message containing values for
KEY_SECRET. We will need these for setting up the dbt profile.
3. Update your dbt profiles.yml
In order to run your Python models in fal cloud, you should update the profiles.yml to include the newly generated credentials. Here's an example of how it should look like:
That's it. Doing a dbt run against this profile will execute your Python models in fal cloud.
4. (Optional) Define separate output for fal cloud
You can have fal outputs, e.g.:
In the example above, we have two fal outputs:
staging will execute your Python models only locally, whereas
prod will run them on fal cloud. So now you can control where your models are ran with a
This will run Python models locally:
And this will run Python models on fal cloud:
dbt run -t prod