1 day EN / DE Max 16

Orchestrate BigQuery Workloads with Dataform

Dataform is a service for data analysts to develop, test, version control, and schedule complex SQL workflows for data transformation in BigQuery. In this course you will explore the components of Dataform core, learn how to define tables and dependencies in SQLX, document BigQuery tables and views, understand BigQuery security settings and how to manage these with Dataform, write assertions, execute SQL workflows, and explore additional advanced use cases.

€920,00 excl. VAT

Individual scheduling

The courses are held as dedicated group sessions. Once you've booked, we'll coordinate a date that works for your team and send invitations to all participants.

Prerequisites

Knowledge of SQL data analysis and BigQuery as discussed in BigQuery for Data
Analysis.

What you'll learn

  • Understand the components of Dataform core.
  • Create tables and views in BigQuery using Dataform.
  • Document BigQuery tables and views.
  • Understand BigQuery security settings using Dataform.
  • Use assertions to validate data in Dataform workflows.
  • Execute Dataform SQL workflows in an automated fashion.

Course modules
SQL workflow Repositories and workspaces Default files and folders Compiled graphs
Declare a data source. Create a table. Create an incremental table. Set partitioning and clustering options. Create an empty table. Create an external BigLake table. Create views and materialized views. Define dependencies.
Use column descriptions. Use globally defined JavaScript constants. Add labels.
IAM dataset and table/view access Column-level security Row-level security
Use built-in assertions. Create manual assertions. Use assertions to validate data in Dataform workflows.
Dataform code lifecycle. What happens during compilation. Customize and schedule compilation results. Execute workflows (UI, Cloud Scheduler, Cloud Composer). Logging and monitoring.
Create a BigLake table after file upload using Cloud Run functions. Build a Machine Learning pipeline with BigQuery ML. Work with Slowly Changing Dimensions Type 2.
Orchestrate BigQuery Workloads with Dataform