File download with ClojureScript

As I couldn’t find a recipe on how to provide some data from a ClojureScript application for download, here’s how. If you know how to do this in JavaScript already and if you’ve done any CLJS-JavaScript interop, there’s nothing new for you to learn here, as this is a pretty straight-forward translation of how to use the Blob API and clicking a temporary link in JavaScript,

(defn file-blob [datamap mimetype]
  (js/Blob. [(with-out-str (pp/pprint datamap))] {"type" mimetype}))

(defn link-for-blob [blob filename]
  (doto (.createElement js/document "a")
    (set! -download filename)
    (set! -href (.createObjectURL js/URL blob))))

(defn click-and-remove-link [link]
  (let [click-remove-callback
    (fn []
      (.dispatchEvent link (js/MouseEvent. "click"))
      (.removeChild (.-body js/document) link))]
    (.requestAnimationFrame js/window click-remove-callback)))

(defn add-link [link]
  (.appendChild (.-body js/document) link))

(defn download-data [data filename mimetype]
  (-> data
       (file-blob mimetype)
       (link-for-blob filename)

(defn export-data []
  (download-data (:data @some-state) "exported-data.txt" "text/plain"))

I’ve tried to break it down into pretty self-explaining pieces, but here is a bit of explanation: export-data would be used as an on-click handler on some UI element and would expect to gather the data to be exported in some way. Here, we’re just assuming the data is already stored in some state-atom and is a map. file-blob is pretty-printing the data somewhat, declaring the content to the given MIME type (e.g. text/plain) and then returning the newly created blob. Of course, you might want to change the pretty printing or the MIME type, depending on your data.

We’re not doing anything fancy with the created Blob object, we simply hand it over to URL.createObjectURL whose result we use as the href attribute of a newly created anchor element. Setting the download attribute will tell the browser not to navigate to the URL. After this we simply add this new link to the DOM and then execute the download by dispatching a MouseEvent on said link. This actually triggers the download, i.e. the browser will open up a file save dialog with the suggested name, so the only thing left to do is to clean up the link from the DOM.

Fresh strawberries -- web programming in Clojure

I recently started working on a web application in Clojure. This being my first contact with quite a lot of tools / libraries for web programming with Clojure, I ran into quite some stuff that was not obvious. First some background: It’s basically a very small application which will have only a very small number of users, so we don’t need a highly interactive, reactive UI. So, no need for a single-page application, a classical web application where the server serves up a small set of pages and has all the logic is just fine.

Starting out with ring, lein-ring and compojure was pretty straight-forward, going beyond the easy stuff not so much. One such problematic issue I ran into was feeding some configuration to Jetty (JNDI resources), as lein-ring does not seem to offer any way of reading an existing jetty-web.xml file. This implies that you’re basically down to using and configuring Jetty as described in the embedding Jetty examples. This snippet is what I’m using to declare my database connection (you’ll need to have depends for clj-dbcp, org.mortbay.jetty/jetty, org.mortbay.jetty/jetty-plus, org.mortbay.jetty/jetty-naming and javax.servlet in your project.clj):

  (ns myapp.jetty-config
     (:require [clj-dbcp.core :refer [make-datasource]])
     (:import ( Resource)
                 (java.util Hashtable)
                 (javax.naming InitialContext Context)))

  (defn make-my-datasource []
     {:adapter :mysql :host 'localhost :database 'mydb
      :username "myuser" :password "mypw"}))

  (defn setup-jetty-context []
     (let [ht (Hashtable.)]
       (.put ht Context/INITIAL\_CONTEXT\_FACTORY "org.mortbay.naming.InitialContextFactory")
       (.put ht Context/PROVIDER\_URL "org.mortbay.naming")
       (InitialContext. ht)
       (Resource. "java:comp/env/jdbc/etrans" (make-my-datasource))))

This being solved, during the course of the project, I aimed for running the application on Tomcat. Figuring out how to develop with Jetty and make it run on Tomcat also took quite a while. First of all, you don’t want to have the above Jetty configuration and dependencies dragged in when running lein ring war but instead rely on the “normal” persistence.xml mechanism to define persistence units. The key to the first issue is making use of leiningen’s profiles in combination with the :init keyword of the configuration options for lein ring. I.e., I removed all Jetty configuration and dependencies from the top-level ring configuration in my project.clj and added a :dev profile to it which has the required additional dependencies mentioned above plus a :ring section with an :init key pointing to a setup function which calls the required Jetty configuration functions:

    (defproject myproject "0.1.0-SNAPSHOT"
  :description "A small web application"
  :dependencies [[org.clojure/clojure "1.5.1"]
                 [org.clojure/tools.logging "0.2.6"]
                             ;; .. other dependencies, but nothing Jetty related
  :plugins [[lein-ring "0.8.8"]]
  :ring {:handler myapp.handler/app
  :web-xml "war-resources/web.xml"}
  {:dev {:dependencies [[javax.servlet/servlet-api "2.5"]
                      [org.mortbay.jetty/jetty "6.1.23"]
                      ;; etc.
  :ring {:handler myapp.handler/app
         :nrepl {:start? true}
     :init myapp.jetty-config/setup-jetty}}})

Making use of the persistence.xml file turned out to be pretty easy: You can place “normal” Tomcat configuration files in the directory specified by the :war-resources key. E.g. you can your persistence.xml in war-resources/META-INF/. You can also specify the location of your web.xml explicitly. I did an initial lein ring war run and have it generate an initial web.xml and modified it later on as needed. Another issue was that I wanted to run the application under Tomcat in parallel with other applications, i.e. with it’s own named web application context, which is simple enough. But running under Tomcat as some non-exclusive webapp (i.e. installing not as ROOT.war) forces you to use relative links. But how do you find out the right prefix for servlet global resources? Useful info gets added to your request in the :context, :path-info, :servlet-context and :servlet-context-path, according to ring.util.servlet. I used this wrapper for figure out what I needed:

  (defn wrap-show-request-context [handler]
     (fn [request]
        (when-let [context (:context request)]
           (logging/info (str "Request with context " context)))
        (when-let [pathinfo (:path-info request)]
           (logging/info (str "Request with path-info " pathinfo)))
        (when-let [servlet-context (:servlet-context request)]
           (logging/info (str "Request with servlet-context " servlet-context)))
        (when-let [servlet-context-path (:servlet-context-path request)]
           (logging/info (str "Request with servlet-context-path " servlet-pathinfo)))
        (-> request

Finally, I had a dependency on a local library which I resolve via the localrepo extension to leiningen. However, lein-ring knows nothing about local-repo extension, hence lein ring uberwar will not include the dependencies that only exist in your ~/.m2/repository/. The workaround is to make a WEB-INF/lib directory and place all required Jars in it, although figuring out all Jars that you need is a very tedious process.

Given the small scale of the web application, I decided very early on that hiccup should fit my small requirements, although I wanted to use bootstrap, too. The one problem I ran into is a typical functional programming issue: information becomes available at some point which is required exclusively in some other function way down the chain of callers. The concrete problem I ran into was actually using the right servlet-context in the main-page layout template to request the right resources. Handing down the parameter (or the request itself) to the view seems wrong, because then I basically make use of internal knowledge about the inner workings of my pages. I finally extended the wrapper above to bind a dynamic variable:

    (def ^:dynamic \*app-context\* nil)

    (defn wrap-context [handler]
       (fn [request]
         ;; ... logging code elided ...
        (binding [\*app-context\* (str (:context request) "/")]
           (-> request

I then refer/use this variable during the page setup. I decided against using an atom or some such, because I specifically don’t want to have other requests/threads to see/modify the current value from some other thread — this is actually intended to be a request-only accessible (global) variable. Not sure whether this is the best way to go, though, I guess there are more elegant solutions.

I had seen a talk on liberator at EuroClojure 2013, which promises a pretty declarative way of sorting out how to react to web requests, which looked nice enough to give it a try. Although things can get somewhat complex rather quickly, I still like it. One such issue is that the interaction between liberator’s context argument and compojure parameters isn’t always obvious. Basically all handlers need to take a context argument (possibly ignored), as is detailed somewhat more in the documentation of the execution model. Resources (i.e. their definitions) can have arguments as well, however, which are not related to the context — they need to be provided then when your handle calls the resource. I.e. check the following resource definition:

 (defresource someresource [someid]
       :available-media-types ["text/html" "application/json"]
       :method-allowed? (request-method-in :get)
       :exists? (fn [context]
                     {:something (find-something someid)})
           :handle-ok (fn [context]
                         (myview (get context :something))))

     (defroutes myapp
        (GET "/something/:someid" [someid] (someresource someid)))

The route definition uses compojures parameter extraction to hand someid over to the resource. The context argument to the decision function :exists? and to the handler :handle-ok doesn’t contain this, but you can access it via the parameter someid of the resource. Another thing which had me scratching my head was how to do a redirect on a GET request which I wanted as a result of logging out of the application. You have to combine liberator.representation/ring-response with ring.util.response/redirect. moved-permanently? and handle-moved-permanently or handle-see-other seem not to be intended for this, at least I was not able to use them for this purpose.

(defresource logout
  :available-media-types ["text/html"]
  :method-allowed? (request-method-in :get)
  :handle-ok (fn [context]
             (str (get-in context [:request :context]) "/")))))

I also have resources which handle GET and POST/PUT requests (the same resource). You can combine this and in principle it’s as simple as it sounds, but figuring out which handler gets called when is not: for example, is handle-ok called after put! created something? Liberator does come with quite some documentation and it’s tracing facility is really helpful, but it really takes some time getting into it. For instance, I don’t think it’s documented that the list of available media-types determines the default media-type generated. I.e. if your request doesn’t specify that you prefer to accept application/json and your available media-types has text/html as the first element, you’ll get text/html. I ran into this with my unit tests: you may need to use ring.mock.response/header to set the “accept” header (and don’t get fooled by ring.mock.request/content-type). I think, in the middle to long term my biggest concern is the question how stable this API is. It’s currently at 0.10, which leaves quite a lot of numbers before implying any notion of stability.

Then, I also used friend for the authentication, which is an interesting library whose current version of 0.2 again makes me wonder about the stability of the current API. I’m currently staying very close to the bare minimum of features and so far haven’t really run into any bigger technical issues. There is one thing that is quite apparent though: while it provides role-based authorization, friend is currently missing any idea of access rights, i.e. it’s lacking a connection of roles to rights. Hence you guard functionality with calls to authorize, not with a declaration of the required rights. Friend is also seriously lacking documentation, e.g. about *identity* or current-authentication which you might want to use to determine data about the currently logged-in user. I also had issues with testing my code after introducing friend, because it’s not at all obvious how to provide the needed authorization from test code. I ended up using midje’s mocking machinery (i.e. provided) to mock authorized? which is underlying friend’s authorize macro. Note that provided assumes that the one checkable directly above is actually calling the mocked code, you can’t have provided mocking for two or more checkables or in some let construct (background pre-requisites can help with this).

Finally, I used java-jdbc for accessing the database. Again, I’m wondering how stable is this API? There were large changes with 0.3, which broke compatibility with at least one of the SQL DSLs mentioned on the project page. This includes splitting out the old SQL DSL into it’s own module java-jdbc/dsl. And alas, documentation and examples are also somewhat lacking. At least the unit tests were useful to figure things out.

So, what have I seen so far? I guess a lot of interesting technology, notable lack of satisfying documentation and APIs in widely varying degrees of stability.

ObTitle: Franz Ferdinand, from “Right thoughts, right words, right action”

Page 1 of 1, totaling 2 entries