...
orders and order_payments are two base tables at ODS, partitioned by ds, here we create a DWD wide table dwd_orders.
| Code Block | ||
|---|---|---|
| ||
CREATE DYNAMIC TABLE dwd_orders |
...
PRIMARY KEY(ds, id) |
...
PARTITIONED BY (ds) |
...
FRESHNESS = INTERVAL '3' MINUTE |
...
AS SELECT |
...
o.ds |
...
o.id, |
...
o.order_number, |
...
o.user_id, |
...
... |
...
FROM |
...
orders as o |
...
LEFT JOIN products FOR SYSTEM_TIME AS OF proctime() AS prod |
...
ON o.product_id = prod.id |
...
LEFT JOIN order_pay AS pay |
...
ON o.id = pay.order_id and o.ds = pay.ds |
When a Create Dynamic Table statement is executed, it deduces the Schema of the table based on the query statement referenced, first creates a Dynamic Table in Catalog, and then automatically creates a data processing pipeline to refresh the table based on the specified Freshness value. Since FRESHNESS is set to 3 minutes here, a Flink Streaming Job is pulled up to refresh continuously.
...