An asynchronous client

The setup

A proxy server reproducing Wikipedia page data for programming languages has been installed on https://net7212.rfc1149.net. For example, by requesting https://net7212.rfc1149.net/wikipedia/Rust, you can get the raw HTML page corresponding to the Rust page on Wikipedia.

The issue with this server is its slowness: requesting a page takes at least 2 seconds (on purpose). Since in this lab we will make several requests to this server, we want them to be done in parallel (asynchronously) rather than sequentially.

Using reqwest

reqwest is a popular crate used to build HTTP clients. It supports asynchronous requests, and we will build on this. Its uses tokio as its runtime.

Exercise 0.a: Create a new binary Cargo project client with reqwest as a dependency, as well as tokio with the feature flag full. We will also add anyhow to handle errors, or color_eyre (don't forget to initialize it) as you prefer.

Make an asynchronous reqwest to get the Rust page

Exercise 0.b: In your main() async function, build a Client. With this object, do a get() request to get the "Rust" wikipedia page through the proxy server and retrieve the body text (see the reqwest documentation for how to do that). Display the number of bytes in the body.

⚠️ Don't forget to use the tokio::main attribute on your main() async function so that it gets executed in an asynchronous context:

#[tokio::main]
async fn main() -> anyhow::Result<()> {   // or color_eyre::Result<()>
  …
}

Time your main program

We want to see how long our main program takes to execute.

Exercise 0.c: Using the Instant type, measure and display the time taken by your main() program.

You may be interested by Instant::now() as well as Instant::elapsed(). Also, note that a Duration can be printed in Debug mode and print something like "29.8ms" or "2s". By using the floating point notation, you can even limit the number of digits after the decimal point with a format string like {:.3?}.

Make several asynchronous requests

Exercise 0.d: Reorganize your code to create this function which returns information about a programming language and use it from main():

async fn retrieve(client: &Client, language: &str) -> anyhow::Result<String>

(or eyre::Result<String> if you used eyre)

For example, calling retrieve(client, "Rust") should return the Rust page.

Exercise 0.d: In addition to information about the number of bytes in the "Rust" page body, print information about the "C" programming language as well.

You should notice that it takes twice the time, as requests are serialized. We will not make them parallel.

Exercise 0.e: Import the futures crate as a dependency. Using the join! macro, make both requests in parallel then print the results.

Note how the time is back to around 2s.

Take requests from the command line

Exercise 0.f: For every programming language name found on the command line, print the number of bytes in its page body.

For example, running cargo run --release -- C Rust Java should print something like

- C: 401467 bytes
- Rust: 461307 bytes
- Java: 362262 bytes
Elapsed time: 2.008s

You can either use clap or more simply std::env::args() to get the list of languages (in which case, do not forget to skip the first item). The join_all() function from the futures crate may be useful to do this.

You can use zip on an iterator to unite items of two iterators by pairs.