EnhancedOrgService – Load-balancing and auto API limits | Mage Series

This article is the second part of the series presenting the enterprise-grade library: EnhancedOrgService.

To catch up, please read the first part here.

Out of the library’s numerous features, this article will focus on two notable ones: load-balancing and auto API limit handling.

Load-balancing

NLPs (Network Load Balancers) are integral to any enterprise system. They carry the burden of two important metrics of system performance: availability and performance.

They commonly exist in hardware form; however, software load balancers can be used in less complex cases or when the budget doesn’t allow it.

You define the list of nodes for the library to connect to, and it offers the following algorithms for managing said nodes:

• Round-robin (default): keeps going down the list recursively in order each time a node is requested.
• Weighted round-robin: same as round-robin, but favours higher-rated nodes more.
• Static with fallback: uses a single node, but falls back on request/node failure.
• Least-loaded: uses the least loaded node — node with least pending requests.
• Least latency: uses the fastest nodes.

The library makes a periodic WhoAmIRequest to the nodes as a rudimentary health check to ensure communication with reliable nodes.

To create a basic two-node routing (load-balancing) service:

var service = await EnhancedServiceHelper
    .GetSelfBalancingService(
        new SelfBalancingParams(
            "<node-1-connection-string>",
            new PoolParams { IsAutoPoolSize = true }),
        new SelfBalancingParams(
            "<node-2-connection-string>",
            new PoolParams { IsAutoPoolSize = true }));

Let’s go through each part of this statement.

EnhancedServiceHelper: provides convenient methods to speed up initialisation.

GetSelfBalancingService:

1. Creates a node for each connection string given (even if duplicate, which is useful as I will show you).
2. Tests the nodes for their health and marks them as such.
3. Adds the nodes to an internal pool.
4. Embeds the pool in a service to automate request distribution as per the chosen algorithm.
5. Returns the service wrapped in the SDK’s IOrganizationServiceAsync2 form.

PoolParams: internal pool parameters. Most often, you won’t need more than to declare auto-sizing.

IsAutoPoolSize: automatically resizes the pool based on the x-ms-dop-hint sent by the server, which is exposed as the RecommendedDegreesOfParallelism service parameter.

Improved throughput

The following can be used to improve the throughput of the service greatly:

1. 1. Use more than one node, even if to the same server, with a self-balancing service. Don’t overdo it, though; max 2 or 3 duplicate nodes are enough, in my opinion.
   2. Use pool automatic resizing.
   3. Use the async/await pattern with the async interface (<...>Async service methods).
   4. Finally, add this statement anywhere before calling any of the library’s methods:

      EnhancedServiceHelper.AutoSetMaxPerformanceParams = true;

API-limits handling

Currently, Microsoft only exposes the x-ms-dop-hint header as a service property. In the future, when Microsoft exposes x-ms-ratelimit-time-remaining-xrm-requests and x-ms-ratelimit-burst-remaining-xrm-requests headers as properties, we can use them to further handle API limits.

For now, due to this limitation, all we can do is guess or react instead of being proactive.

The library automatically detects when it’s close to the ‘time-remaining’ limit and handles it transparently without the user noticing. For the exact details, please refer to the code logic.

If the measure above fails and one of the following errors is thrown, the library automatically keeps retrying until successful or an unrelated error is thrown:

• 0x80072322
• 0x80072321
• 0x80072326

Conclusion

We discussed a few important features of the library. We will meet again soon to design advanced scenarios and delve deeper into the library’s inner workings.

In the next part of the article, I will introduce more advanced features: advanced initialisation, auto-retry, stats, events, and caching.

Share

Tweet

Follow