Hazard

A Simple HTTP Proxy in Clojure July 1, 2018

In my last post, I described a few ways I’ve improved my development workflow by inserting a proxy API between a web client and back-end API. In this post, I’ll show you how to accomplish the same with Clojure using a simple Ring server.

The core request handler builds a URL from the incoming request’s host, path, and query string, then sends a request to another host using clj-http. When the response comes back, the handler adds the Connection: keep-alive header but otherwise returns the response unchanged.

(require '[clj-http.client :as http])

(defn build-url [host path query-string]
  (let [url (.toString (java.net.URL. (java.net.URL. host) path))]
    (if (not-empty query-string)
      (str url "?" query-string)
      url)))

(defn handler [req]
  (let [{:keys [host uri query-string request-method body headers]
         :or {host "http://localhost:8080"}} req]
    (->
      (http/request {:url (build-url host uri query-string)
                     :method request-method
                     :body body
                     :headers (dissoc headers "content-length")
                     :throw-exceptions false
                     :decompress-body false
                     :as :stream})
      (assoc-in [:headers "Connection"] "keep-alive"))))

This code supplies a default host to forward to. Configure it use the right host and port for your current back-end API.

I keep the running server in an atom, then start and stop it with a couple utility functions:

(require '[ring.adapter.jetty :as jetty])

(def server (atom nil))

(defn stop-server [server]
  (when @server
    (reset! server (.stop @server))))

(defn start-server [server app opts]
  (when @server (stop-server))
  (reset! server (jetty/run-jetty app opts)))

Start your proxy by evaluating the following code:

(def app #'handler)

(start-server server #'app {:port 8081 :join? false})

You can stop the server with:

(stop-server server)

Once your proxy is running, configure your HTTP client to send requests to it rather than your standard back-end. You’ve now injected a proxy API into your local stack.

But to get any good out of having a proxy in your development workflow, you’ll need some middleware. I keep a collection of useful middleware functions that I can mix and match as needed. Here’s a simple one that remembers the last request:

(def wrap-remember-previous [handler store]
  (fn [req]
    (reset! store req)
    (handler req)))

Add it to your Ring application by redefining app:

(def previous-request (atom nil))

(def app
  (-> #'handler
    (wrap-remember-previous previous-request)))

Because we’ve used Clojure vars when referring to handler and app (via the reader macro #'), we don’t need to stop and restart the Jetty server to see this change take effect. When we make an API call to the proxy, it will store the request in the atom previous-request. This is handy not only for inspecting requests, but also for replaying them, which is as easy as

(app @previous-request)

No client necessary.

Here’s a middleware that caches responses by the URL of their request:

(defn wrap-cache [handler cache]
  (fn [{:keys [uri] :as req}]
    (if-let [resp (get @cache uri)]
      resp
      (let [resp (handler req)]
        (swap! cache assoc uri resp)
        resp))))

Caching is useful when you’re testing some client code that makes expensive API calls. Here’s a middleware that also shortcuts requests to the back-end API but, instead of using a cache of requests, returns predefined responses:

(defn wrap-stubs [handler stubs]
  (fn [req]
    (or
      (some #(when (= (key %) (req :uri)) (val %)) stubs)
      (handler req))))

(def app
  (-> #'handler
    (wrap-stubs {"/api/route" {:status 200}})))

I’m returning an empty response, but you could also return fixture data or an error to see how it’s handled in the client.

If you want to modify the body of a response, make sure to update the Content-Length header. Here’s how:

(defn assoc-body [resp body]
  (-> resp
    (assoc :body body)
    (assoc-in [:headers "Content-Length"]
              (-> body (.getBytes "UTF-8") count str))))

You may also want a function like Clojure’s built-in update. This one detects JSON responses and uses Cheshire to handle it:

(require '[cheshire.core :as json])

(defn update-body [resp f & args]
  (let [f #(apply f % args)]
    (if (re-find #"application/json" (get-in resp [:headers "Content-Type"]))
      (assoc-body resp (-> resp :body (json/parse-string true) f json/generate-string))
      (assoc-body resp (-> resp :body f)))))

Now you can write middleware that alters response bodies, such as this middleware, which changes the JSON’s count to field to a different value:

(def app
  (-> #'handler
    ((fn [handler]
       (fn [req]
         (if (re-find #"list-of-things" (uri :req))
           (update-body (handler req) #(assoc % :count 400))
           (handler req)))))))

I’ve used middleware like this to change values, add and remove fields, lengthen lists, and return fake data (for which Clojure’s test.check library is incredibly valuable).

My favorite middleware, however, doesn’t just integrate with the front-end and back-end, it also integrates with my text editor. I’ll elide the specifics in the code below, as they’re out of scope for this post (and would likely be very different for you anyway), but the general shape of the function is this:

(defn wrap-open-to-error [handler]
  (fn [req]
    (let [{:keys [status] :as resp} (handler req)]
      (when (= 500 status)
        (-> resp :body
          (json/parse-string true)
          response->stacktrace
          open-stacktrace-in-editor))
      resp)))

This middleware assumes two key things: first, that your API’s debug mode returns stack traces when it throws an Internal Server Error, and second, that your text editor has a remote API. With those two features, you can create middleware that detects stack traces, parses the filenames and line numbers, and opens your text editor to the given files at the given locations. While it may not seem particularly burdensome to do this manually, it does lighten the mental load a little, quicken the process, and reduce the odds of opening the wrong file to the wrong location.

Some of the above middleware is a permanent part of my proxy server, but a lot of the proxy’s behavior is ad hoc depending on the specific debug problem. Clojure is invalable for this kind of tinkering. I can re-evaluate the Ring server’s logic without restarting the app or losing any saved state (such as my request/response caches).

If you’ve augmented your own development workflow with some custom tooling, I’d love to hear about it. You can contact me on Twitter or by email .