Hazard

Data over Code July 17, 2018

I used to make a lot of cURL requests to different APIs like GitHub, Pivotal Tracker, CircleCI, and Trello, but it’s difficult to remember the details of each API, so naturally I switched to using shell scripts. When I migrated my development tooling from shell scripts to Clojure, I ended up with several slightly different functions like this:

(defn fetch [root path params]
 (:body
  (clj-http.client/get (str root "/" path)
   {:headers {"Accept" "application/json"}
    :query-params (assoc params :token token)})))

Nothing too surprising. I’m using clj-http’s get function to make an GET request, root is a string with the API’s root URL, and token is a namespace-level var.

This is code over data. The HTTP method is supplied by the called function, and the full URL and parameters are passed as arguments.

This approach works, but isn’t flexible. To inspect the specifics of a request, I resorted to wrapping clj-http’s functions and using dynamic Clojure vars that could turn some logging on or off, which also worked, but was limited. What if I wanted to log a request without executing it? What if I wanted the request details as on object instead of a string? Add more dynamic vars, I guess. I needed a better approach.

Clj-http’s request function takes a map defining an HTTP request and executes it. All my helper functions really needed to do was return such a map:

{:url (str root "/" path)
 :method :get
 :headers {"Accept" "application/json"}
 :query-params (assoc params :token token)}

This is data over code, and Clojure is designed for data. For instance, here’s a function to add the elements that all requests have in common:

(defn with-common [req]
 (-> req
  (assoc-in [:headers "Accept"] "application/json")
  (assoc-in [:query-params :token] token)))

And I no longer need to pass root down, as the root URL can be added later.

(defn with-root [req root]
 (-> req
  (assoc :url (str root (req :path)))
  (dissoc :path)))

Now I can make a request like this:

(-> {:path "/some-path"
     :method :get}
 with-common
 (with-root root)
 http/request)

I used the threading macro -> here because it makes it easy to comment out the last step and see the request that’s being made:

(-> {:path "/some-path"
     :method :get}
 with-common
 (with-root root)
 #_http/request)

This request data can be manually inspected, or it can be passed to an alternative function, say, a hypothetical format-as-curl function that turns the request map into a cURL command so you can share it with your non-Clojure co-workers:

(-> {:path "/some-path"
     :method :get}
 with-common
 (with-root root)
 format-as-curl)

While code can be passed around as a first-class object, only data can be inspected, manipulated, and serialized.

I’ve applied this data mindset to other code, most notably SQL queries. Rather than tinker in an interactive session, I use Honey SQL to generate queries. Honey has several helper functions for constructing a map that represents a query, and I’ve added a few of my own to make building and testing queries as fluid as possible. That’s another great thing about data: anyone can create it.

If you have other pointers on exchanging code for data, I’m always looking for tips. You can reach me on Twitter or by email .