02-05-2024 04:24 PM
Is it safe to delete historical records from the fivetran_api_call and fivetran_query tables? We have a test environment that is using a Salesforce connector, and the size of these tables overshadows all other data in the database, making database backups more difficult. Alternatively, if there is a way to limit the persistence of records in these tables, we'd prefer that approach to deleting data.
Solved! Go to Solution.
02-05-2024 10:30 PM - edited 02-05-2024 10:40 PM
Hey @pkch,
Thanks for reaching out.
Yes, you can delete the records from the Fivetran system tables mentioned below that we replicate for Salesforce as a source:
Manual deletion of records from these tables will not cause any failure. In the subsequent syncs we will replicate data again for those tables.
Additionally, if you'd like to test this out first, I would recommend creating a test Salesforce connector with a very small table selected for experimentation. If everything works well, you can broaden this approach for your other connectors and tables.
I have also raised a feature improvement request internally for you so we can smoothen this out in the near future.
Thank you!
02-05-2024 10:30 PM - edited 02-05-2024 10:40 PM
Hey @pkch,
Thanks for reaching out.
Yes, you can delete the records from the Fivetran system tables mentioned below that we replicate for Salesforce as a source:
Manual deletion of records from these tables will not cause any failure. In the subsequent syncs we will replicate data again for those tables.
Additionally, if you'd like to test this out first, I would recommend creating a test Salesforce connector with a very small table selected for experimentation. If everything works well, you can broaden this approach for your other connectors and tables.
I have also raised a feature improvement request internally for you so we can smoothen this out in the near future.
Thank you!
02-06-2024 08:47 AM
Thank you for confirming this!
02-06-2024 08:49 AM
Thank you for asking your question and for accepting @SVijay 's solution.
@SVijay @rikthefrog thank you both for your insights.
Cheers,
Toby
02-06-2024 01:00 AM
restrict the amount of data you cna run a small DBT job which calls a delete statement with a variable WHERE clause with something like sysdate() - days retention.