Skip to content

DBT-environment-setup

In repo risk-s3-configuration, we have scripts which read sql scripts from repo majority_dbt and generate new event_processing_v2.json files

This document is about setting up environment to run the scripts locally, so after you change the sql scripts in majority_dbt repo you can preview the event_processing_v2.json.

Install Jdk

When I setting up the environment I have Java 17 installed. (Jansi has Java 19 installed) Both version works. Here is the link to the Jdk version 17 for windows in msi format. If the link does not work, just go to Oracle.com to search Jdk.

After installation, open a command prompt and execute this

java --version

Install Python 3.9.10

We have to use this version of python because later version of python does not work for Antlr4. We can always reverify this because both Python and Antlr release new version frequently.

Here is the link to Python 3.9.10 for windows 64 bit

After installation, open a command prompt and execute this

py --version

Multiple Python Versions
If you have another python version installed already and do not want to break anything. It is suggested to use Python virtual environment. Here is the official document about Python virtual environment. After you create python virtual environment, you need to remember to activate the venv before you run your python scripts. If you are using pycharm, you need to add your venv as a interpreter and select it for the project.

Install Python packages

Now go to risk-s3-configuration/scripts/dbt-model-to-config-translator and run following commands to install plugins

pip install -r requirements.txt

Run in Pycharm

  1. Install Pycharm Community Edition and open repo risk-s3-configuration as a project
  2. Go to scripts/dbt-model-to-config-translator
  3. Right click on main.py and select "Modify Run Configuration". In the popup

    1. Check if Python interpreter is the expected one (version 3.9.10)
    1. Update Parameters to --dir "{path-to-majority_dbt-repo}\models" --execute "all" Replace {path-to majority_dbt-repo} to the path on your computer. You can put absolute path here. If it is a relative path it is relative to the main.py file.
    1. Right click on main.py and select "Run".
    1. Go to s3 directory check the changes the script made.