Allow multiple threshold/task-count bands per entity, or configure a target records-per-task value with a maximum task count. DMF should calculate the appropriate degree of parallelism for each execution rather than applying the same task count to every import above one global threshold.
Comments
The current configuration is global per entity and applies the same task count to every execution above a single threshold, regardless of the actual volume or legal entity.For example, with a threshold of 200 records and a task count of 8:An import of 20,000 products creates 8 tasks of roughly 2,500 records each, where parallel processing is likely beneficial.An import of 2,000 products also creates 8 tasks, now with only around 250 records each. This consumes the same batch/AOS capacity while potentially adding more scheduling and partitioning overhead than performance benefit.A more adaptive design could support either:Multiple volume bands per entity, such as 2 tasks above 400 records, 4 tasks above 1,500, and 8 tasks above 10,000; orA target records-per-task value combined with a maximum task count.This would allow DMF to calculate an appropriate degree of parallelism for each execution rather than treating every import above the threshold identically
Category: Data Management
