Deleting _fivetran_deleted rows

[moisternw]

Hello,

I'm using Fivetran to send some data from a few data sources into a Snowflake warehouse instance. Occasionally, the data gets deleted in the sources, and we'd like to delete the data from the warehouse too. 

I know that the way Fivetran handles this is by adding a column `_fivetran_deleted`, and the value becomes `True` for any row that was deleted in the sources. I also think that there is no way around this given that no update has been given in this post. 

So, I want to write a script that deletes the rows in which `_fivetran_deleted = True`.

My question here is: Do all of the deleted rows get uploaded to Snowflake with every sync? Or do they only get uploaded once? That is, if I remove the row, will it get uploaded again later? Or will it no longer appear? The answer to this question changes the frequency with which my script has to run, which is why I'm asking. 

Any help would be very appreciated!

[mabroukgadri · Accepted solution]

Hello,

a deleted key from the source is only flagged once with '_fivetran_deleted = True' in the destination. It won't be inserted back in the destination if you delete it with your script unless it is inserted/deleted again in the source.

however, I would avoid deleting data in the destination and create a view on top of the destination table instead which filters out deleted rows

[mohibiz]

Hello guys, when data is deleted in source, fivetran sets `_FIVETRAN_DELETED= True` but '_FIVETRAN_SYNCED' timestamp is not updated. is that a noral behavior or it's configuration issue or it's a bug.?