4
When running a full export of an entity to BYOD, the process will first delete the data from the BYOD db, will then re-insert the data. The statement is: DELETE T1 FROM ***Staging T1 WHERE T1.DATAAREAID = '***' AND T1.PARTITION = 'initial'.

In case of large tables, this delete is very expensive. Taking all Log IO from the Azure-DB. Also because no index exists on "PARTITION, DATAAREAID".

The request is to TRUNCATE instead of DELETE in cases where the system has only one DATAAREAID and PARTITION. If a system does have multiple dataareas, you should be able to provide a parameter to have control over truncate or delete.

Additional idea: When a system does have multiple PARTITIONS or DATAAREAS, why not put this in separate SCHEMA, separate TABLE, or separate PARTITION? I would personally prefer having each DATAAREA in a different schema.

Additional idea 2: Add a setup to D365, to add (NC) indexes to the external BYOD tables. These indexes would then be created during the process of publishing the entity. This would take away the need to maintain these type of index extensions externally.
Category: Data Management
STATUS DETAILS
Needs Votes

Comments

G

That idea would be great. As transaction database get larger, we start having this kind of issue - deleting records takes too much time.

Category: Data Management

G

Our example table has 30 Million rows.
Our Azure SQL P2 Premium machine with 250 DTU. 930 Dollar per month.
The DELETE action takes 2 hours. The BULK_INSERT action then takes another 1.5 hours.
If the DELETE would be replaced with a TRUNCATE, only the 1.5 hours would remain.

Category: Data Management