Quick Test Feature

(Re)introducing: The Quick Test

Just head out to the homepage and type in a website, with or without https://. Press Test Availability and you instantly get an overview of how your website is performing from several locations around the world.

We had this in the previous version of the Monitive website, where you would get a nice map and results as they come in. While this mainly served as a simple way to get a potential visitor to try out the Monitive check system, it also became useful when setting up a new site or domain that seems to misbehave from some locations.

And yes, servers can be misbehaving, and there’s an error message for that:

Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 127.0.0.1:53: server misbehaving

This happens when the DNS doesn’t correctly resolve lookup queries, and the weird part is that it might only happen from certain locations.

Another situation is when backbone providers have networking issues, making certain parts of the internet disconnected from others.

Saving the Quick Test results

Running a quick test is simple.

Our challenge was persisting the quick test results so that a user can send a link to a friend or colleague with the results page.

If we’d have a standard website, this would have been simple, just get the results parameter from the GET request and pull in the results. But not in our care, since the website is a static website assembled by Hugo at deploy, and doesn’t have any notion of databases and such.

That’s where Cloudflare Workers Sites come into play.

What we did, is whenever a request came in, Cloudflare would intercept it, and if it’s a Quick Test results page, it would do an API call to get the results, and inject it as JSON into the page that is then sent back to the browser.

It also replaces a generic variable such as $WEBSITE with the actual website being checked, so that the page title, meta description and everything in it is correctly pointing to the website that has been checked.

This is all completely transparent to the user or browser, as they only get the final rendered page with everything in it.

Cloudflare Workers can be thus augmented with simple Javascript code that gets the job done. It’s a type of server-side execution that has no actual server.

Therefore, since the static site is stored in the CDN (“on the edge” as they put it), and the Quick Test retrieval and embedding retrieval is done via server-less Javascript functions,  the overall system is still server-less.

Here is the Javascript code that does this for us:

/**
 * gatherResponse awaits and returns a response body as a string.
 * Use await gatherResponse(..) in an async function to get the response body
 * @param {Response} response
 */
async function gatherResponse(response) {
  const { headers } = response;
  const contentType = headers.get("content-type") || "";
  if (contentType.includes("application/json")) {
    return JSON.stringify(await response.json());
  } else if (contentType.includes("application/text")) {
    return await response.text();
  } else if (contentType.includes("text/html")) {
    return await response.text();
  } else {
    return await response.text();
  }
}

async function retrieveQuicktestResults(slug) {
  // @link https://developers.cloudflare.com/workers/examples/fetch-json
  const init = {
    headers: {
      "content-type": "application/json;charset=UTF-8",
    },
  };
  const url = `https://api1.monitive.com/quicktests?filter[slug]=${slug}&page[size]=1`;
  const response = await fetch(url, init);
  return await gatherResponse(response);
}

class AttributeRewriter {
  constructor(attributeName, results) {
    this.attributeName = attributeName;
    this.results = results;
  }
  element(element) {
    if (element.getAttribute("id") === "quickresults") {
      element.setAttribute(this.attributeName, this.results);
    }
  }
}

class MetaRewriter {
  constructor(attributeName, results) {
    this.attributeName = attributeName;
    this.results = results;
    const resultsObject = JSON.parse(this.results);
    this.website = resultsObject.data[0].attributes.service_json.url
      .replace("http://", "")
      .replace("https://", "");
    this.lastText = "";
  }
  text(text) {
    if (text.text.includes("$WEBSITE")) {
      text.replace(text.text.replace("$WEBSITE", this.website));
    }
  }
  element(element) {
    if (element.tagName !== "meta") {
      return false;
    }
    const attribute = element.getAttribute(this.attributeName);
    if (!attribute) {
      return false;
    }
    element.setAttribute(
      this.attributeName,
      attribute.replace("$WEBSITE", this.website)
    );
  }
}

async function handleEvent(event) {
  // . . . regular handling code . . .
  let rewriter = null;
  if (url.pathname.includes("/website-availability-test")) {
    const slug = url.pathname.match(/\/website-availability-test\/#?(.+)/);
    if (slug.length === 2) {
      const results = await retrieveQuicktestResults(slug[1]);
      rewriter = new HTMLRewriter()
        .on("div", new AttributeRewriter("data-test", results))
        .on("title", new MetaRewriter("", results))
        .on("script", new MetaRewriter("", results))
        .on("meta", new MetaRewriter("content", results));
      options.mapRequestToAsset = handleQuicktest();
    }
  }

  // . . . regular page retrieving code

  if (rewriter) {
      return rewriter.transform(response);
    }
    return response;
}

That’s it.

Instant, dynamic, API-powered, JSON-friendly, rewritten and rendered page. Boom! 💥

The API call slowdown

Timing a Quick Test result page looks something like this:

Response times are between 0.19 seconds and 1.13 seconds, depending on where it was called from.

If we didn’t have the API call that is retrieving the results, the timing would be more similar across the globe. But since the request is delayed until we get the API response, it’s proportional to the distance from the API server, which is currently hosted in Frankfurt, DE.

If we’d cache the response within Cloudflare, then we’d get more close to a global similar timing.

This, for example, is the timing for the homepage:

As you can see, from Germany the response comes in 0.04 seconds, Tokyo in 0.07 seconds, and Dallas in 0.06 seconds. Serving a static page from the edge really makes a difference.

But the caching story is for another day.