all Technical posts

Logic Apps connector performance (1/3) - The bad

One of Logic Apps biggest selling points is the fact that the runtime is serverless, which results in a very scalable integration platform. I've always been amazed by the elasticity characteristics of the Logic App engine, however there's a huge limitation on the scalability of the out-of-the-box connectors. On many projects, we've suffered from this limited connector capacity. This blog post series will walk you through the limitations (The bad!), Microsoft's advised workarounds (The ugly!) and an approach that works great for some connectors (The good!).

This blog series consists of:

Load patterns

Integrations often result in bursty load. Some examples are:

  • Huge data export from a line of business application
  • Splitting of a big batch file into individual transactions
  • More and more event-driven integrations

The integration platform has two important responsibilities. On one hand, it has to process the messages as fast as possible, to meet the business requirements. On the other hand, it sometimes has to slow down (throttle) in case the receiving system can’t keep up with the pace. The latter responsibility can be easily configured via the concurrency control feature. However, on the performance side, the Logic Apps connectors have limitations.

Connector throttling

In the connector overview, each connector has documented throttling limits. The throttling happens per API connection. These are the current throttling limits of some important connectors:

SQL Server 100 calls / 10 seconds 10 calls / second
Blob Storage 600 calls / 60 seconds 10 calls / second
Event Grid 100 calls / 60 seconds 1,66 calls / second
File Storage 600 calls / 60 seconds 10 calls / second
SFTP 100 calls / 60 seconds 1,66 calls / second
Custom Connector 500 calls / 60 seconds 8,33 calls / second

 

Recently, we faced the following business requirement: receive and process 10K messages from an SFTP folder. The business gave us even 1 hour, which was very generous, in my opinion. No big deal we thought!  But…  for each message, we had at least 3 calls (GET file content, COPY file to archive, DELETE file from folder). This means 30K calls / hour, which equals 8,33 calls / second.  Far above the throttling limit of the SFTP connector!

Throttling behavior

Once a connector starts throttling, it returns an HTTP 429 “Too many requests”.  Based on this HTTP status code, Logic Apps will kick off its retry mechanism, which is by default 4 retries with a 20 seconds interval.  When there is sporadic throttling, this gracefully resolves the issue. However, when hitting the limits really hard, this results in a cascading effect that makes it only worse.

As a test setup, I’ve created a Logic App that gets fired through an HTTP request. The second action is writing a file to blob storage. By using artillery, I put the Logic App under high concurrent load. This causes throttling and cascading retries, as shown below.

Conclusion

In production-grade integration solutions, it’s not desired that the integration platform is the limiting factor in the end-to-end performance. Typically, the integration platform should be throttled, in order not to overwhelm the backend systems. The Logic Apps connectors should have much higher throttling limits, to fulfill their scalability promise. Several escalations to the product team resulted only in minor improvements.  Read in the next parts of this blog post series, how we can work around these limitations.

 

Subscribe to our RSS feed

Let's talk, what's your first name?

Press Enter

Press Enter

Hi , what's on your mind?

Press Enter

Great! Leave your email and we'll be in touch
Submit
Thanks, we’ll be in touch soon!