Idea description:

Retrieving large amounts of data from any source could be a time-consuming and resource-intensive process.

Microsoft documentation recommends using webhooks to retrieve data as it has multiple benefits - it reduces the number of calls from a client or integration, increases the throughput and reduces execution time. These benefits could result in major performance improvements for any integration trying to access Business Central data.

However, Microsoft Business Central currently supports creating only 200 webhook subscriptions for a single environment. This turns out to be a major blocker for anyone trying to gain from those performance improvements.

Business Central currently supports 22 webhook supported endpoints. Also, as of the latest release, Business Central recommends having a maximum of 300 companies in a particular environment. This brings the total number of webhook supported endpoints across all companies to ~6600 (22 times 300). As a single subscription can only contain a single endpoint/resource, this means we'll need to register ~6600 endpoints to receive updates from all of the endpoints.

Here is a major gap between the supported endpoints (200) and the total number of subscriptions required for a Business Central customer who has 300 companies.

Fivetran's use case:

Fivetran is a data pipeline product that enables our customers to consistently sync data from various sources to their data warehouse. As part of our Business Central connector, we need to identify the records which were synced previously but are deleted now so that we can mark those records as deleted at the customer's data warehouse. We were using webhooks for receiving delete events but we quickly hit the max subscription limit and are unable to proceed further. We created support ticket (#2303280010001261) but were unable to get a solution for our use case.

Ask:

Since Microsoft Business Central recommends using webhooks for various performance benefits, but at the same time limits our scalability of the solution that uses webhooks, we request the following:

1. Either increase the number of supported webhooks for a particular environment from 200 to 6600. This will match the number of subscriptions a Business Central customer might require if they have 300 companies in their environment.
2. Or provide the ability to add multiple resources in a single subscription. This way we'll add all endpoints from a single company into a single subscription (having the subscription limit increased to 300 to match the number of companies will also be a good-to-have feature).
3. Or provide us with an alternative feasible approach to fulfill our ability to fetch updates/deletes from all possible endpoints across all companies for an environment.