Minimizing API Throttling Risk During Cloud Office Migration

Out of several roadblocks that make it challenging for businesses to seamlessly migrate their cloud office data, API throttling is one. The larger the planned migration project, the higher the risk. However, there are strategic ways of getting around it to ensure the migration is not disrupted and business continuity is not impacted.

This post explores the details of API throttling, why it persists, what are its impacts, and strategies to lower its risk to a significant extent.

What is API Throttling?

Every cloud storage provider, such as Microsoft and Google, sets time-based API usage limits to avoid an overload of requests (API calls) that third-party applications can make. This approach helps maintain optimal performance of the clouds (e.g., OneDrive and Google Drive) and ensure a reliable service for customers. So, whenever an application exceeds the usage limit, the API throttles with the 420 error to limit the number of requests for a particular time window as part of offering equal bandwidth to every other application simultaneously.

From a migration perspective, migration tools such as CloudFuze X-Change need to use API calls to execute read and write processes to transfer data from the source to the destination cloud, e.g., from Box to OneDrive. But if more API calls are made than allowed for a particular time period, e.g., 25 seconds, then the API will throttle to temporarily limit the further number of requests.

In short, API throttling is the temporary hold on time-based API calls that cloud storage providers put when the allowed limit is exceeded.

API throttling can occur on either the source cloud (where data is read from) or the destination cloud (where data is written to). But in most cases, the risk of throttling is higher on the destination cloud.

The Impact of API Throttling

Any cloud service will introduce a 429 error of “too many requests” when its API starts throttling. As a result, the API calls are put on hold temporarily which, in turn, slows down the overall migration speed.

For businesses, downtime in the migration progress can have a significant impact on following the pre-planned migration timeline, especially when it comes to completing the project before the source cloud’s license renewal or during M&A completion.

Cloud storage providers can also entirely block the migration tool’s requests with the 503 error if it continues to exceed the usage limit within the predetermined time slot. In such cases, resolving the throttling issue becomes more complex and time-consuming.

Approaches to Resolve API Throttling

The industry-standard approach that cloud providers officially recommend is to wait until the hold period (throttling) is over and retry sending the requests. The retry requests should be executed only when the throttling is over to avoid the risk of 503 error.

CloudFuze uses a retry mechanism that automatically resends API requests in case a throttling occurs, after the wait time is over. This approach automatically re-migrates the data without any manual intervention.

Strategies to Lower API Throttling Risks

At CloudFuze, our primary strategy to lower the risk of API throttling is to implement codes that ensure we do not exceed the API usage limits in both the source and destination clouds. We regularly update our codes to adhere with the latest API usage limit changes that all cloud storage providers publish.
Another strategic approach that we implement is to perform migrations, especially large-scale projects, in a phased manner. We segregate the overall user account and data into several batches and migrate them one at a time. This approach helps keep the workload in check and ensure the tool regulates the flow of API requests to not exceed them.

Overcome Throttling and Other Migration Challenges with CloudFuze

With CloudFuze as a strategic migration partner, your team can steer clear of the myriad challenges that come with migrating cloud data and ensure complete success. Our migration team has proven expertise having helped renowned organizations such as Stryker, GSMA, Gatwick Airport, BuzzFeed, Michigan State University, and more, create migration success stories.