Skip to main content

Reliability UI 👀

Now let's investigate generated data in re_data UI. We first generate HTML/JSON from the data warehouse and then serve files.

re_data overview generate --start-date 2021-01-01 --interval days:1
re_data overview serve
info

We don't pass --end-date to re_data overview generate and this is causing re_data to use the default (today). We generate for this whole period to get also tests history (just ran) displayed

After running these commands you should be able to see views similar to those:

Alerts​

GraphExample

Alerts view lets you see if there are any problems currently detected in your data. re_data compares past & current metric computed using z_score to determinate if value is suspicious.

From the alerts view, you can go to learn more details about specific alerts in the graph view.

Schema Changes​

The alerts view also shows any schema changes detected. These schema changes include detecting when columns were added, removed or had its type changed. For tutorial purposes, we modify using the schema of the orders table using macros included in the default template.

# this operation adds a column to the orders table
dbt run-operation schema_change_add_orders_column
# run re_data models to detect the column added schema change
re_data run
# this operation drops the column added earlier
dbt run-operation schema_change_drop_orders_column
# re-run re_data models to detect the removed column
re_data run

Regenerating the user interface:

re_data overview generate --start-date 2021-01-01 --interval days:1
re_data overview serve

Will show:

AlertsWithSchemaChanges

Lineage​

GraphExample

Lineage view lets you investigate anomalies, metrics & schema changes on top of the dbt lineage graph.

Tests​

GraphExample

Tests view lets you see history and details (if you click on the test name) of the dbt tests runs.

Tables​

GraphExample

Tables view lets you investigate any table easily. Check out for anomalies, metrics, tests related to it.

In the next section, let's see how we can notify ourselves about problems in the data

info

Something didn't worked as expected?​

If you have more questions, got stuck anywhere, or something is not working as expected, please let us know on Slack! 😊, we will help you asap, and it will help us improve this quick start guide.