fal-serverless is our serverless compute solution that allows you to run Python models on a reliable and scalable infrastructure. Setting up dbt-fal with fal-serverless is quick and straightforward.
0. Install fal-serverless
pip install fal-serverless
1. Authenticate to fal-serverless
fal-serverless uses GitHub for authentication. Run the following command in your shell:
fal-serverless auth login
Follow the link that's generated and login using GitHub. Come back to the shell, when ready.
2. Generate keys
Next, generate keys that will allow dbt to connect to fal cloud:
fal-serverless key generate
This will print a message containing values for
KEY_SECRET. We will need these for setting up the dbt profile.
3. Update your dbt profiles.yml
In order to run your Python models in fal-serverless, you should update the profiles.yml to include the newly generated credentials. Here's an example of how it should look like:
That's it. Doing a dbt run against this profile will execute your Python models in fal-serverless.
4. (Optional) Define separate output for fal-serverless
You can have fal outputs, e.g.:
host: cloud ## ask your account exec
In the example above, we have two fal outputs:
staging will execute your Python models only locally, whereas
prod will run them on fal-serverless. So now you can control where your models are ran with a
This will run Python models locally:
And this will run Python models on fal-serverless:
dbt run -t prod