You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
i used the extension for setting up the stream between firestore and bigquery.
After that i wanted to back-fill the data and noticed that the path_params are not back-filled.
To understand how my firestore looks like:
I have a collection called customer. Here I create and store the snapshots of each customer as I need it (document)
For each customer I have two further collections. 1. events 2. messages
Within the event collection I store each event ass a document.
Now when setting up the extension the extension is streaming path_params which contain the id of each document in the customer collection, so that i can connect both tables in bigquery, in other words i can check which customer made which events or which event was made by which customer.
When back-filling the data trough the cloud shell the path-params are not exported.
with JSON_VALUE function I can extract the value of the last column (path_params) of each changelog. However as mentioned before the path_params are not back_filled.
The extensions settings look like this:
I guess that the field "Wildcard column field with Parent Firestore Document IDS" set to true expoorts the path_params right ?
However when backfilling trough the google cloud shell, it doesn't ask for this Parent Firestore Document ids if these should be backfilled as well.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Hi guys,
i used the extension for setting up the stream between firestore and bigquery.
After that i wanted to back-fill the data and noticed that the path_params are not back-filled.
To understand how my firestore looks like:
I have a collection called customer. Here I create and store the snapshots of each customer as I need it (document)
For each customer I have two further collections. 1. events 2. messages
Within the event collection I store each event ass a document.
Now when setting up the extension the extension is streaming path_params which contain the id of each document in the customer collection, so that i can connect both tables in bigquery, in other words i can check which customer made which events or which event was made by which customer.
When back-filling the data trough the cloud shell the path-params are not exported.
with JSON_VALUE function I can extract the value of the last column (path_params) of each changelog. However as mentioned before the path_params are not back_filled.
The extensions settings look like this:
I guess that the field "Wildcard column field with Parent Firestore Document IDS" set to true expoorts the path_params right ?
However when backfilling trough the google cloud shell, it doesn't ask for this Parent Firestore Document ids if these should be backfilled as well.
Beta Was this translation helpful? Give feedback.
All reactions