FetchApp: UrlFetchApp with Retries
Google Apps Script is often used to pull data from various services via HTTP requests. However, these requests sometimes fail due to network or service issues. The default behavior of UrlFetchApp is to throw an exception, which you have to catch. Otherwise, the script execution will be interrupted.
We often need more: send the request again instead of failing. There is no built-in way to do retries in Apps Script.
Solution
To solve this problem and not copy-and-paste code snippets from project to project, I created FetchApp (GitHub) – an open-source Google Apps Script library.
Its main features are:
- Optional Retries: Depending on the response code received.
- Delay Strategies: Choose between linear or exponential delay between retries.
- Custom Callbacks: Implement callbacks on failed attempts for tailored actions and logic.
- Enhanced Type Hints: Improved hints for UrlFetchApp’s
paramsargument. - Automatic Logging: Logs failed attempts automatically.
I use this library in many projects in production, especially where I have to communicate with unreliable third-party APIs, or there is a chain of related requests that I do not want to repeat if one of them randomly fails.
If you find this library useful, please give the repository a star and share the link with others.
Features, Use Cases, Examples
There are multiple ways and use cases to use FetchApp.
Drop-in Replacement for UrlFetchApp
FetchApp is designed to work as a drop-in replacement for UrlFetchApp. Caveat: FetchApp sets muteHttpExceptions: true in params unless explicitly specified otherwise.
1// `url` and `params` are defined elsewhere
2
3// regular UrlFetchApp
4const response1 = UrlFetchApp.fetch(url, params);
5
6// FetchApp without configuration is a pass-through to UrlFetchApp
7const response2 = FetchApp.fetch(url, params);
8
9// FetchApp with retries and delay enabled
10const config = {
11 maxRetries: 5,
12 successCodes: [200],
13 delay: 500,
14};
15const response3 = FetchApp.fetch(url, params, config);
16
17// If there are no `params`, pass an empty object
18const response4 = FetchApp.fetch(url, {}, config);
Configurable Client
If you need to use FetchApp multiple times, you can initiate a client to reuse the configuration:
1// FetchApp with retries and delay enabled
2const config = {
3 maxRetries: 5,
4 retryCodes: [500, 502, 503, 504],
5 delay: 500,
6};
7
8const client = FetchApp.getClient(config);
9
10// All client's fetch calls will use this config
11const response1 = client.fetch(url, params);
12
13// Partially modify the config for a specific request
14const response2 = client.fetch(url, params, { successCodes: [200] });
Success or Retry Response Codes
FetchApp retries requests depending on the response code received in two different modes:
successCodes: deem responses with these codes successful and return the response. If provided,retryCodesare ignored.retryCodes: requests with these codes are not successful; retry.
Examples:
1const response1 = FetchApp.fetch(
2 url,
3 params,
4 {
5 successCodes: [200], // Everything else leads to retries
6 maxRetries: 3,
7 },
8)
9
10const response2 = FetchApp.fetch(
11 url,
12 params,
13 {
14 retryCodes: [500, 502, 503], // Everything else is deemed successful
15 maxRetries: 3,
16 },
17)
18
19const response3 = FetchApp.fetch(
20 url,
21 params,
22 {
23 successCodes: [200], // Takes priority over retryCodes
24 retryCodes: [500, 502, 503], // Ignored
25 maxRetries: 3,
26 },
27)
Delay Between Requests
FetchApp supports constant or exponential delay between retries:
1const response1 = FetchApp.fetch(
2 url,
3 params,
4 {
5 successCodes: [200], // Everything else leads to retries
6 maxRetries: 3,
7 delay: 300, // Constant delay of 300ms after each request
8 },
9)
10
11const response2 = FetchApp.fetch(
12 url,
13 params,
14 {
15 successCodes: [200], // Everything else is deemed successful
16 maxRetries: 3,
17 // Exponential delay of 1, 2, 4, 8, etc. seconds
18 delay: 1000,
19 delayFactor: 2,
20 },
21)
22
23const response2 = FetchApp.fetch(
24 url,
25 params,
26 {
27 successCodes: [200], // Everything else is deemed successful
28 maxRetries: 10,
29 // Exponential delay of 1, 2, 4, 8, 10 seconds.
30 delay: 1000,
31 delayFactor: 2,
32 maxDelay: 10000, // Limit delay to maximum 10 seconds
33 },
34)
Throw Exceptions via Callbacks
If you need to throw an exception if all attempts fail, you can do it via a onAllRequestsFailure callback:
1// Throw an exception
2const throwException = ({ retries, url }) => {
3 throw new Error(`All ${retries + 1} requests to ${url} failed`);
4};
5
6const config = {
7 successCodes: [200],
8 maxRetries: 5,
9 delay: 500,
10 onAllRequestsFailure: throwException,
11};
12
13const response = FetchApp.fetch(url, params, config);
Send Notifications via Callbacks
One of the difficulties with Apps Script is that it functions as a black box unless you add logging and notifications to the script. With the onRequestFailure and onAllRequestsFailure callbacks, you can send notifications about failed requests.
1// Define the `sendNotification` function elsewhere
2
3// Send notification if access is denied, maybe because the credentials expired
4const accessDenied = ({ url, response }) => {
5 const responseCode = response.getResponseCode();
6 if ([401, 403].includes(responseCode)) {
7 sendNotification(`Received ${responseCode} when accessing ${url}`);
8 }
9};
10
11// Send a notification if all attempts failed
12const allAttemptsFailed = ({ retries, url }) => {
13 throw new Error(`All ${retries + 1} requests to ${url} failed`);
14};
15
16const config = {
17 successCodes: [200],
18 maxRetries: 5,
19 delay: 500,
20 onRequestFailure: accessDenied,
21 onAllRequestsFailure: allAttemptsFailed,
22};
23
24const response = FetchApp.fetch(url, params, config);
Autocomplete and Type Hints
FetchApp comes with JSDoc declarations that enable type hints and autocomplete in Google Apps Script IDE, both for UrlFetchApp’s params and for FetchApp’s config:
Unfortunately, due to the IDE limitations, rich autocomplete doesn’t work when you attach FetchApp as a library (as opposed to copying the code). Nevertheless, you still have the description of possible options in the type hints:
Contributions are welcome. Feel free to submit pull requests or issues on GitHub.