rxjs Common recipes Sending multiple sequential HTTP requests

Example

Making a sequence of HTTP requests has two primary reasons:

• Requests are depending on each other (the result from one requests is required for a consecutive request).

• We want to spread server load into multiple requests.

1. Making multiple dependent requests

This can be performed using the concatMap() operator to transform one response to parameters required for the consecutive request.

function mockHTTPRequest(url) {
    return Observable.of(`Response from ${url}`)
      .delay(1000);
}

function timestamp() {
  return (new Date()).getTime() - start;
}

var start = (new Date()).getTime();

Observable.of('url-1') 
  // first request
  .concatMap(url => {
    console.log(timestamp() + ': Sending request to ' + url);
    return mockHTTPRequest(url);
  })
  .do(response => console.log(timestamp() + ': ' + response))

  // second request
  .concatMap(response => {
    console.log(timestamp() + ': Sending request to ' + response);
    let newUrl = 'url-' + response.length; // create new requests url
    return mockHTTPRequest(newUrl);
  })
  .do(response => console.log(timestamp() + ': ' + response))

  // third request
  .concatMap(response => {
    console.log(timestamp() + ': Sending request to ' + response);
    let newUrl = 'url-' + response.length; // create another requests url
    return mockHTTPRequest(newUrl);
  })
  .subscribe(response => console.log(timestamp() + ': ' + response));

Operator concatMap() internally subscribes to the Observable returned the from its projection function and waits until it completes while re-emitting all its values.

This example timestamps each request and response:

3: Sending request to url-1
1010: Response from url-1
1011: Sending request to Response from url-1
2014: Response from url-19
2015: Sending request to Response from url-19
3017: Response from url-20

See live demo: https://jsbin.com/fewidiv/6/edit?js,console

Handling errors

If any of the HTTP requests fail we obviously don't want to continue because we're not able to construct the following request.

2. Making consecutive requests

In case we're not interested in the response from the previous HTTP request we can take just an array of URLs and execute them one after another.

function mockHTTPRequest(url) {
    return Observable.of(`Response from ${url}`)
      .delay(1000);
}

let urls = ['url-1', 'url-2', 'url-3', 'url-4'];
let start = (new Date()).getTime();

Observable.from(urls)
  .concatMap(url => mockHTTPRequest(url))
  .timestamp()
  .map(stamp => [stamp.timestamp - start, stamp.value])
  .subscribe(val => console.log(val));

This example prints timestamped responses:

[1006, "Response from url-1"]
[2012, "Response from url-2"]
[3014, "Response from url-3"]
[4016, "Response from url-4"]

See live demo: https://jsbin.com/kakede/3/edit?js,console

Delaying consecutive calls

We might also want to make a small delay between each request. In such case we need to append delay() after each mockHTTPRequest() call.

Observable.from(urls)
  .concatMap(url => {
    return mockHTTPRequest(url)
      .do(response => console.log(((new Date()).getTime() - start) + ': Sending request to ' + url))
      .delay(500);
  })
  .timestamp()
  .map(stamp => [stamp.timestamp - start, stamp.value])
  .subscribe(val => console.log(val));

This prints to console the following output:

2024: Sending request to url-1
[2833, "Response from url-1"]
4569: Sending request to url-2
[5897, "Response from url-2"]
7880: Sending request to url-3
[8674, "Response from url-3"]
9789: Sending request to url-4
[10796, "Response from url-4"]

See live demo: https://jsbin.com/kakede/4/edit?js,console

Handling errors

If we simply want to ignore when any of the HTTP requests fail we have to chain catch() ofter each mockHTTPRequest().

function mockHTTPRequest(url) {
  if (url == 'url-3') {
    return Observable.throw(new Error(`Request ${url} failed.`));
  } else {
    return Observable.of(`Response from ${url}`)
      .delay(1000);
  }
}

let urls = ['url-1', 'url-2', 'url-3', 'url-4'];
let start = (new Date()).getTime();

Observable.from(urls)
  .concatMap(url => mockHTTPRequest(url).catch(obs => Observable.empty()))
  .timestamp()
  .map(stamp => [stamp.timestamp - start, stamp.value])
  .subscribe(val => console.log(val));

This simply ignores the url-3 call:

[1004, "Response from url-1"]
[2012, "Response from url-2"]
[3016, "Response from url-4"]

See live demo: https://jsbin.com/jowiqo/2/edit?js,console

If we didn't use the catch() operator the url-3 would cause the chain to send an error notification and the last url-4 wouldn't be executed.

See live demo: https://jsbin.com/docapim/3/edit?js,console