Hazard

Using a Proxy API for Web Development June 11, 2018

A couple years ago I worked on a website whose backend API was still under construction. To fill in the gaps, I created a proxy API that sat between the web client and API. For completed endpoints, it would forward requests to the real API, and for endpoints that didn’t exist yet, it would generate fake data. I’ve moved on to other projects since then, but I’ve continued to make a proxy API part of my development architecture. Here are a few ways I’ve used a Clojure Ring server in my daily workflow.

Generating data. I’ve written before about creating a fuzzing API using Clojure’s test.check library. This isn’t as necessary when working on a mature API where data is always available, but it can occasionally be useful to add to existing data. For instance, you can inject fake data into lists to test pagination.

Modifying responses. Even when you have data, it can be inconvenient to get the database into a state that exercises edge cases. With a proxy API, you can manipulate responses from the real API without bothering the underlying database, testing the effects of extreme values (such as zero) or scenarios that don’t make sense (like having completed 10 out of 5 items).

Delaying responses. HTTP calls to a local API are often much faster than the same calls in production. Network latency is better locally, dev machines may be faster than cloud instances, and testing databases are smaller. Consequently, developers may never see their application’s loading states. I’ve sometimes used a proxy API to artificially delay responses so I can simulate a more real-world app experience. Random delays can also uncover race conditions in the UI.

Preventing requests. You may have need to prevent certain requests from ever hitting your local API. For instance, the client may poll an API endpoint that’s very expensive or that needs special configuration to work locally. It’s easy to intercept these requests and never forward them to your local API.

Querying requests and responses. While API logs may document what URLs are being hit and what the HTTP response codes are, it can be difficult to get a closer look at the requests and responses without inserting ad hoc logging statements into the codebase. I’ve added some Ring middleware to my proxy API that saves the last 20 requests and their responses. When trying to figure out why the API is returning unexpected results, I can filter to the specific request and response to verify my assumptions about what’s being asked and answered, helping me quickly identify whether the problem is client-side or server-side. This technique tightens the debugging loop, as you don’t have to find the particular place or places in the code to add logging, nor do you have to wait for a server restart.

Taking action beyond the application. Since a proxy API is part of a development architecture, you can integrate it with your other development tools. For instance, I’ve configured mine to detect when the real API returns an Internal Server Error with a stack trace, in which case it parses the stack trace and opens my editor to the site of the exception. It’s a small change, but by never having to load the filename and line number into my working memory, my mind is free to focus on the actual problem.

Experience has taught me the usefulness of having a server that wraps the application’s true API. It’s like a Swiss Army knife, small but flexible. Clojure specifically is a big help here, as I can make changes to the proxy without losing accumulated data, such as the log of recent requests and responses.

If you have other development tooling you’d recommend, I’d love to hear about them on Twitter or by email .