Instructor Introduction

dbt Core vs. dbt Cloud

Course outline

Course Introduction

A Brief History of the Data Stack

Benefits of dbt - Inferring Dependencies

Benefits of dbt - Documentation & Testing

Benefits of dbt - Python-Like Functionality

How dbt Has Solved a Lot of Problems in the Data Stack

How dbt Fits in the Data Stack

Section Overview

(Windows) Installing Python 3.10

Downloading VSCode and Setting Up Shortcuts

Creating a GitHub account

Forking Vs. Cloning

Forking the Repository

(Optional) If You Have Issues Syncing Your Forked Repository

Installing the recommended VSCode Extensions

What's a Virtual Environment (venv)

Setting Up Our Virtual Environment and Installing Packages

Setting Up dbt for BigQuery

Note on Continual Course Updates

Trialling Our Model dbt Project

(Optional) Setting Up dbt Autocomplete

Run Through of How Our Final Project Will Look

Creating a Gmail Account

Setting up a BigQuery Project With Billing

(Optional) If You Have Issues With BigQuery Billing

The BigQuery UI

The Dataset You'll Be Using

(Mac) Installing Python 3.10

Creating Our First Model yml File

Adding Tests to Our First Model yml File

Setting Up Our Models to Materialise as Tables Instead of Views

Getting the Rest of Our Staging (stg) SQL Models Set Up

Using dbt clean to Get Table Materialisation Working

Getting the Rest of the Staging (stg) yml Files Set Up

Taking Stock of Our Staging (stg) Data Models

The Target Folder

Getting Our First Intermediate (int) SQL Model Set Up

Getting Our First Intermediate (int) yml File Set Up

The dbt init Command

Getting Our Mart SQL Model Set Up

Getting Our Mart yml File Set Up

Our Basic dbt Project Is Now Complete!

Version Control with GitHub

Setting up dbt Power User

How We'll Structure Our Project

Creating Our First Source (src) yml File

Creating Our First Staging (stg) SQL Model

Running Our First Staging (stg) SQL Model

Setting Default Test Severity

Setting Test Severity and Thresholds

The External dbt Packages We'll Be Using

dbt utils and dbt expectations

Custom Singular Tests

Custom Generic Tests

Applying Advanced Tests to Our Whole Project

Source Freshness Tests

Model Governance - Access & Groups

Model Governance - Contracts

Model Governance - Versions

The doc Function

Seed Files

dbt Snapshots

Materialisation Types

Materialisation Ephemeral Models

(Optional) Partitioning a Table in BigQuery

Model Governance Overview

Commands For a Clean dbt run

Using Different dbt Profiles

Selectors

Tags

Indirect Test Selection

dbt test With --warn-error

dbt build

dbt docs generate serve

Jinja Comments, Statements, and Expressions

The 3 Types of Macro Functions, Hooks, Operations

(Optional) dbt Jinja Function Reference

Macros Operations

Macros Functions (Building a Basic Macro)

Macros Hooks

Jinja Statements for Loops and Setting Variables

(Optional) Jinja Using the Target Function

Creating a dbt Cloud Account

Setting Up a Service Account

Connecting GitHub to dbt Cloud

The dbt Cloud IDE

Deploying Jobs on dbt Cloud