If somehow you happen to stumble upon this tutorial, please note that it not even close to being complete. Proceed at your own peril.
Though Enlive has been around for sometime now the Clojure community has been slow to embrace this useful library. I believe this is due simply to the lack of good introductory documentation. This is no slam against Christophe Grand. He’s a busy man and I’d rather he write up more excellent tutorials on how to optimize Clojure code to blazingly fast speeds than create introductory tutorials about Enlive. In anycase this is what open source communities are all about, right? Contribution!
But really my reasons for creating this tutorial are not entirely altruistic. I’ve been playing with Enlive off and on for sometime now and I realize that I keep coming back to it having half-forgotten what I learned the last time. So now I’m writing this tutorial – for myself and all the Lispers and Lispers To Be.
Not much. This tutorial is intended even for developers who have little or no exposure to Clojure. At the very least you’ll need to have the Java Virtual Machine (JVM) installed. Don’t expect a full blown discussion of Lisp but I’ll try to explain the less obvious things as I go along.
As the tutorials progress they will be more useful to you if you have some experience with a modern webframework that ships with a decent HTML templating library. If you do know Clojure the tutorials are fairly amenable to skimming. My only real assumption is that you have some experience at the command line.
Enlive presents a dramatically different approach to generating HTML then what’s out there.
- Code and markup are completely separate (the implications are enormous)
- You get to use CSS like syntax to manipulate HTML.
- Template inheritance isn’t some fancy trick, it just plain old function composition.
- You have access to the full power of Clojure to manipulate your templates (yes, macros!).
One good analogy might be “XSLT that not only doesn’t suck, it absolutely rulez”.
There are seven examples in total.
The first one covers grabbing the headlines and points from Hackers News. The next one shows how to make the code less redundant. The third scrapes the New York Times front page since that presents more challenges than Hacker News.
The fourth example shows how to use Compojure and Enlive together. The fifth example shows how things like looping are achieved without writing any code into the markup. The sixth example shows that Enlive can do all the fancy template inheritance magic you might be used to if you’re coming from Django or some other popular modern webframework.
The final tutorial is fairly advanced. It illustrates how Clojure, being a Lisp, can in fact template itself! I use this ability to illustrate how some downsides to Enlive can be overcome. In fact I think the seventh section begins to touch upon how Enlive could be the foundation for a truly incredible templating framework that would put most others to shame (and with a lot less code to boot).
The usual:
git clone git://github.com/swannodette/enlive-tutorial.git
In order to start playing around as fast possible you should use Leiningen. Leiningen is the easy_install (Python) and gems (Ruby) of the Clojure world. Phil Hagelberg and Co. have done a considerable amount of excellent work to make dependency management simple. I truly envy the new Clojurians who do not know the dark times before lien repl and lein swank :)
Once you have Leiningen installed, switch into this repository’s directory. From there run the following command:
lein deps
This will install all of the dependencies required for getting through the tutorial. This might take a minute and and will probably generate a lot of ouptut. Once the the dependencies are installed enter the following command at your terminal:
lein repl
This will launch a Clojure REPL (Read-Eval-Print-Loop) that has the classpath set properly. Be very thankful if you don’t know what the last sentence means. Managing the classpath is one of the few real annoyances when programming Clojure and it’s largely Java’s fault.
Enlive isn’t just good for websites, it’s also fantastic for scraping the content of webpages. It allows you to scrape content by using a syntax very similar to CSS selectors. In the REPL type the following lines (note that user=> is the REPL prompt, not something you type in):
user=> (ns tutorial.scrape1) nil tutorial.scrape1=> (load "scrape1") nil tutorial.scrape1=> *base-url* "http://news.ycombinator.com/"
The first line puts us into a namespace. Not that the REPL prompt reflects this. By switching into the tutorial namespace we use functions defined in the tutorial without having to qualify them. It’s much easier to type base-url then tutorial.scrape1/base-url*. In the next line we load the first tutorial into the REPL. Unlike many scripting languages, loading a file actually compiles it. Clojure is not interpreted.
Let’s see what’s in that file. Open up scrape1.clj with you favorite text editor (you can find it in repo/src/tutorial/). You’ll see it’s a fairly short program.
At the top of this file is the namespace declaration. This keeps your code from clashing with other people’s code when they try to use your library. The namespace declaration also includes another library, Enlive, via :require. In this case we are generating an alias so we don’t have to type the very long namespace for Enlive.
The function fetch-url grabs the contexts of a url synchronously. fetch-url uses html/html-resource (remember we aliased net.cgrand.enlive-html to html for convenience) another handy function defined in the Enlive library. It takes raw HTML and converts it into a nested data structure (think DOM minus tediousness).
Note that the function hn-headlines uses fetch-url. But it’s also surrounded by a lot of funny stuff. You might have noticed html/select; html/select takes parsed html content and selects the nodes specified by a Clojure vector that looks very similar to a CSS selector.
[:td.title a]
Now that looks kind of weird. But if you squint a little it might remind you of this:
td.title a
This is a CSS selector for matching all links inside of table elements that have the CSS class “title”. If you’re a Javascript hacker you should know this stuff by heart.
So let’s break this down. fetch-url grabs the contents of the url and parses it into a data structure. html/select takes it and extracts only those nodes that match the selector – it always returns a vector of nodes. We then use Clojure’s map function to interate over the vector’s elements applying a function to extract each nodes’ text-node, in this case html/text (map is actually lazy, but we’re not going to get into what that means in this Enlive tutorial).
Believe it or not, these 10 lines of code are enough to extract all of the headlines from the Hacker News front page. Let’s try it out at the REPL now.
tutorial.scrape1=> (hn-headlines)
("A 'lorem ipsum' for images." "Google Reader Can Now Track Changes to Any Website - Even Without a Feed" "jQuery 1.4.1 Released" ... "More")
Nice. After this the next function hn-points should make a lot more sense. It does the same thing but we grab the score from a different place in the markup. Try to run this function as well.
tutorial.scrape1=> (hn-points) ... output ..
The last function takes the output of the two different functions and prints out the headline and score for each item on Hackers News.
tutorial.scrape1=> (print-hn-headlines-and-points) ... output ...
print-hn-headlines-and-points looks like a doozy doesn’t it?
(defn print-headlines-and-points []
(doseq [line (map #(str %1 " (" %2 ")") (hn-headlines) (hn-points))]
(println line)))
Let’s break it down. Again we have map. We know that it maps a function over a vector to return a new vector of elemenets with that fuction applied.
#(str %1 " (" %2 ")") ; is just shorthand for
(fn [arg1 arg2] (str arg1 " (" arg2 ")")
This is an anonymous function. I’m not going to explain that here, they’re pretty popular these days. str is built in function for doing string concatenation.
Oddly this map is accepting not one list of things, but two! Check this out:
tutorial.scrape1=> (map + [1 2 3] [4 5 6]) (5 7 9)
Wow you can map two different vectors into one! Finally we have doseq. doseq is just a convenient way to work with lists when you’re dealing with side effects like printing to the REPL. I’m not going to get into that here. All it does is say take list of things, assign each thing one at time to a variable, and then execute the following expression (hopefully you’re actually doing something with that variable!
Not bad for 17 lines of code. One obvious problem here is that we make two separate requests for the Hacker News front page. Let’s fix this now.
Take a look at scrape2.clj. It’s also about 17 lines of code and it looks pretty much the same except that we not longer have one function to grab headlines and another for article points.
(defn hn-headlines-and-points []
(map html/text
(html/select (fetch-url *base-url*)
#{[:td.title :a] [:td.subtext first-child]})))
This select grabs what we’re interested at the same time.
#{[:td.title :a] [:td.subtext first-child]}
Is pretty much the same as:
td.title a, td.subtext:first-child
Let try out the functions. Start up the repl with lein repl if you’ve shut it down and run the following.
tutorial.scrape1=> (ns tutorial.scrape2) nil tutorial.scrape2=> (load "scrape2") nil tutorial.scrape2=> (hn-headlines-and-points) ... output ...
The results are interleaved, so we can use Clojure’s partition function to pair them up and output them just like we did in the previous scrape. The map looks a little bit different:
(defn print-headlines-and-points []
(doseq [line (map (fn [[h s]] (str h " (" s ")"))
(partition 2 (hn-headlines-and-points)))]
(println line)))
To get a sense of what partition does let’s use the REPL again:
tutorial.scrape2=> (partition 2 [1 2 3 4 5 6 7 8 9 0]) ((1 2) (3 4) (5 6) (7 8) (9 0))
Neat it lets us pair things together. Exactly what we need.
Man, but what’s up with the fn this time?
(fn [[h s]] (str h " (" s ")"))
Say hello to destructuring. A lot of popular languages allow you to destructure but probably not as ubiquitously as Clojure does. Here we know that we are going to receive a two element vector for each item in the vector we’re mapping over. So we’re just saying we to assign the first element of that pair to the local variable h and the other to v.
The rest of the function should be clear from the last tutorial.
By now you should feel like a pro. The third one, scrape3.clj tackles something considerably more difficult. Now to be clear this not that useful since the New York Times provides a fairly comprehensive list of RSS feeds. But the structure of the New Yorks Times page forces us to think about how to best leverage Enlive’s abilities.
Take a look at scrape3.clj. This is considerable longer.
This is where things begin to get really interesting. We’re going to use Compojure, and ultralight http framework. If you’re familiar with Rack or CherryPy you will feel right at home.
Let’s get started. If you aren’t running a repl be sure to start one up from the repo directory with lein repl.
Once you see the REPL prompt type the following:
tutorial.scrape3=> (ns tutorial.template1) nil tutorial.template1=> (load "template1") nil tutorial.template1 => (start-app)
You should see some output that lets you know that Compojure is starting up a webserver on port 8080. Point your browser at http://localhost:8080. You should see a very boring page. Point your browser at http://localhost:8080/change. You should see something slightly different.
First open template1.html and take a look at it. If you’re used to other templating solutions the most shocking thing should be that there absolutely no Clojure code in this file. And there never will be. Period.
Now let’s take a look at the code. By now the namespace part should be familiar so we’ll skip over that. After the ns declaration we’ll see our first template definition:
(html/deftemplate index "tutorial/template1.html" [ctxt] [:p#message] (html/content (:message ctxt)))
Every template has the argument list [name source args & forms]. An Enlive template is a macro that when compiled will create a function with the same name. This function will have the same signature as defined by args. forms consists of pairs of Enlive selectors and a function to execute for each node that matches the selector.
Here our template will find all p elements with the CSS id message. CSS ids should be unique so ideal this will only match a single element. Then we have the function which will receive this matching element.
(html/content (:message ctxt))
This means we’ll replace the content of any matching node with the value for the key :message in the ctxt hash-map. The important thing to grasp here is that html/content is a function which returns a function which whill receive the matched element.
For example what if we wanted to include a message if there is not value for :message in cxtxt? It would look something like this:
(html/deftemplate index "tutorial/template1.html"
[ctxt]
[:p#message] (fn [match]
(if-let [msg (:message ctxt)]
((html/content msg) match)
((html/content "Nothing to see here!") match))))
It should be clear here that html/content returns a function which will receive the matching element and modify it. This could be made slightly less verbose like so:
(html/deftemplate index "tutorial/template1.html"
[ctxt]
[:p#message] #(if-let [msg (or (:message ctxt)
"Nothing to see here!")]
((html/content msg) %)))
But even this is kinda meh. While Enlive does not have a great shortcut for expressing this pattern it’s easy to write macros (we’ll get to those later. But watch out it’s an advanced topic). I’ve included a handy macro called maybe-content which allows to write something like this:
html/deftemplate index "tutorial/template1.html"
[ctxt]
[:p#message] (maybe-content (:message ctxt)
"Nothing to set here!"))
Pretty slick eh? ;) This is just touching the tip of the iceberg.
The remainder of template1.clj is really specific to Compojure. We’re not going to get too deep into that here because these tutorials are about Enlive, not Compojure.
(defroutes example-routes
(GET "/"
(render (index {})))
(GET "/change"
(render (index {:message "We changed the message!"})))
(ANY "*"
[404 "Page Not Found"]))
This is simply the route defining syntax. It should relatively familiar to the working web developer especially if you have experience with any modern web framework. A couple things to note render is not a function of Enlive, it’s something I added via utils.clj in the repo. render isn’t magic it’s just a function that looks like this:
(defn render [t] (apply str t))
All this does is take a list of strings and concatenate them into one large string. This is because when an Enlive template function is called that’s what it returns, a list of strings.
Also note that our template function index must be called with at least one parameter. That’s because Enlive templates are just normal Clojure functions and there’s no such thing as optional parameters in Clojure. It’s quite simple to add that functionality via macros but it’s just not worth getting into here. The last bit of template1.clj is just Compojure boilerplate for starting and stopping the server.
Well that’s about it! You’ve seen your first Enlive template. While it may not seem like much yet, there was absolutely no mixing of code and HTML. If you bear with me till the third template tutorial, I think you’ll see some truly amazing repercussions from this fact.
One of the things you use all the time when generating web pages is looping over some piece of HTML because you need to present a list of things to the user. People just love lists. How can Enlive create lists of HTML when there’s no code in the template?! We’ll get into this in this tutorial.
If you don’t have a Clojure REPL running start a new one with lein repl at the commandline from the tutorial repo’s directory. Enter the following (if you’re continuing from the previous tutorial you should should stop the Compojure app for that tutorial first):
tutorial.template1=> (stop-app) nil tutorial.template1=> (ns tutorial.template2) nil tutorial.template2=> (load "template2") nil tutorial.template2=> (start-app) ... output ...
Open up the file template2.html in your text editor and give it a quick look over. Then open the file template2.html in your favorite web browser. It’s just page with a list of links, not that special. Point your browser at http://localhost:8080/. You should see pretty much the same thing except that we’ve dynamically inserted links.
How did we do that if we have no inline code to define the loop? Let’s get into the code. Open up template2.clj in your favorite text editor. At the top of the file you should see the by now familiar namespace declaration. After that we declare a variable for holding a dummy context which we’re going to pass to our template.
(def *dummy-context*
{:title "Enlive Template2 Tutorial"
:sections [{:title "Clojure"
:links [{:text "Macros"
:href "http://www.clojure.org/macros"}
{:text "Multimethods & Hierarchies"
:href "http://www.clojure.org/multimethods"}]}
{:title "Compojure"
:links [{:text "Requests"
:href "http://www.compojure.org/docs/requests"}
{:text "Middleware"
:href "http://www.compojure.org/docs/middleware"}]}
{:title"Clojars"
:links [{:text "Clutch"
:href "http://clojars.org/org.clojars.ato/clutch"}
{:text "JOGL2"
:href "http://clojars.org/jogl2"}]}
{:title "Enlive"
:links [{:text "Getting Started"
:href "http://wiki.github.com/cgrand/enlive/getting-started"}
{:text "Syntax"
:href "http://enlive.cgrand.net/syntax.html"}]}]})
This of course normally be something that would have a read out of a database. The take away here is Clojure makes it easy to define nested data structures. dummy-context is just a hash-map (aka dictionary, aka associative array) of two key-value pairs. The first pair is for the title of the page. The second pair is the list of sections. Each section also has a title as well as a list of link. Each link has some text and url. If you’re used of building up JSON data structures this should pretty familiar to you.
Using Enlive for templating usually involve steps. The first step is figuring out which part of the markup you want to make into a component. Each component will become a snippet. A snippet is reusable mini-template that you can use when constructing larger templates. Once you’ve figured which things should componentized out you have to determine the selector which will allow you to match exactly that part of the document.
Consider our situation. Our designer has handed us some nice markup and some CSS. To better communicate the final result they’ve included dummy content. With a traditional templating solution this is a big nono. With Enlive, working around it requires a minimal amount of effort. So the key here is to identify the “models”.
In our case we have two distinct models, the first is the pair of the section title and the links for that section. The second is the individual link. In templating laguage we would probably do something like this:
{% for section in sections %}
<h2 class="section-title">{{ section.title }}</h2>
<ul class="links">
{% for link in section.links %}
<a target="new" href="{{ link.href }}">{{ link.text }}</a>
{% end for %}
</ul>
{% endfor %}
On level, as you’ll see, there’s a lot less typing involved in this. However you all have something that’s a lot less reusable. The link loop and section loop are hopelessly interwinted. You have many pages on your site that use the same section pattern but not the internal link pattern. So Enlive makes you do more work upfront and very, very little work later on.
So let’s define our link component. We don’t want the dummy content so we really only want to match the very first link that satisfies our need, the selector looks something like this:
(def *link-sel* (html/selector [[:ul.links (nth-of-type 1)] :> html/first-child]))
This is very important. We only want to match only the first ul element that we find that has the link class and only the very child inside that. This is the selector that gets the job done. The snippet that will template this component of your page look like the following:
Now that we have our selector defsnippet (don’t for get that html is just an alias for net.cgrand.enlive-html):
(html/defsnippet link-model "tutorial/template2.html" *link-sel*
[{text :text href :href}]
[:a] (html/do->
(html/content text)
(html/set-attr :href href)))
Snippets are like templates with two main differences. One, snippets take a selector so that they can match only specific parts of an HTML document. The function produced by a defsnippet returns transformed content, not a list of string like deftemplate. This snippet destructures it’s first argument (a hash-map) to extract the value of the for the keys :text and :href. We’re also introduced to html/do→. This is a convenience, we often want to take the matched element and apply a series of transformations on it. In this case we want to set the content of the node as well as it’s href attribute.
Let’s try out or snippet to see that it worked:
Again we need to figure out the correct selector. Our HTML has some dummy content again. We only care about the first unique pair of h2 and p tags.
(def *section-sel* (selector [:body :> #{[:h2.section-title (nth-of-type 1)]
[:ul.links (nth-of-type 1)]}]))
Now that we have our selector we can define our section snippet like so. Pretty straightforward. Remember defsnippet just creates a function which can take whichever arguments you specify and returns the transformed markup. We’re simply creating links using link-model and putting those links inside of the ul in the section.
(defsnippet section-model "tutorial/template2.html" *section-sel*
[{:keys [title links]}]
[:h2] (content title)
[:ul] (content (map link-model links)))
Now let’s at the template to see how we put this all together:
(html/deftemplate index "tutorial/template2.html"
[{:keys [title sections]}]
[:#title] (html/content title)
[:body] (html/content (map section-model sections)))
As you can see it looks really similar to section-model. Again the main different is that templates don’t take selectors and the function they returns a list of string. So you had to do a little more typing but the beauty of this design is that you can mix and match this logic however you please.
For example say on a different page you only want the list of links.
(html/content (map link-model links))
Most templates are a giant mess of unreadable for loops. While it does seems like a little work to achieve the same effect with Enlive, consider the following. template2.html can actually show what the list will look like with dummy extra list items. These are removed because we only match the first child with our snippet. The designer doesn’t need to start webserver or install any web framework at all, she can work directly with HTML.
We now have a basic working idea of how templates work in Enlive. Template are simply functions. Following a well-known pattern it’s best to pass a single map to the template allowing the template to be easily changed over time as your requirementes grow. Now it’s still unclear if there any real advantages to the Enlive way. Hopefully in this tutorial we can prove it’s immense power.
Start a REPL if you don’t already have one running with lein repl. Type the following:
tutorial.template2=> (stop-app) nil tutorial.template2=> (ns tutorial.template3) nil tutorial.template3=> (start-app)
Point your favorite web brower to http://localhost:8080/base.html. You should see a fairly plain page. This is not a template. You can try opening up base.html as file with your browser and see that it’s identical. Now point your browser at http://localhost:8080/3col.html. You should see another page that has a 3 column layout. Now point your browser at http://localhost:8080/a/. The code required to do this follows:
(defn viewa [params session]
(base {:title "View A"
:main (three-col {})}))
If you look at the markup for base.html and 3col.html you will see that there is not one line of code! So how did we magically put these two things together with so little code! Once understand the what’s going, the whole concept of template inheritance is completely boring.
In Enlive, constructing pages is simply putting some functions together!
Take a look at http://localhost:8080/navs.html. You should see some truly ugly nav bars ;) Now point your browser at *http://locahost:8080/b/. You can see it’s easy for to define a site wide layout, a 3 column middle main layout, and customize the contents of each column. Again there’s absolute no code in the markup, only the following code is needed to construct this page:
(defn viewb [params session]
(let [nav1 (nav1 {:count (or (:count params) 0)})
nav2 (nav2)]
(base {:title "View B"
:main (three-col {:left nav1
:right nav2})})))
Pretty slick. Templating with Enlive is just write Clojure code. This very different from even the good HTML templating solutions out there- very few give you the full power of the language.
One last live example before we dive into the code. Point your browser at http://localhost:8080/c/. Huh, looks pretty much like b. Point your browser at http://localhost:8080/c/reverse. Notice something different?
We just flipped the two navs! How complicated is doing something like this?
(defn viewc [params session]
(let [navs [(nav1 {:count (or (:count params) 0)}) (nav2)]
navs (if (= (:action params) "reverse") (reverse navs) navs)
[nav1 nav2] navs]
(base {:title "View C"
:main (three-col {:left nav1
:right nav2})})))
We added two, yes, two new lines of code. If I was a web developer I’d be drooling right about now.
So how does this actually work? Open up template3.clj in your favorite text editor.
Now that you understand how to template with Enlive there’s something to consider. Enlive does not work by manipulating text, it actually manipuates structure. While this comes back in spades when you’re actually constructing pages this does unfortunately make the simple cases unweildy.
Let’s try something out before we dive into a lengthy explanation, type the following in the top level of the repo:
tutorial.template3=> (use 'tutorial.macroology)
nil
tutorial.macroology=> (templ-str foo "<span id='bar'></span>")
#'user/foo
tutorial.macroology=> (foo {:bar "Cool!"})
("<html>" "<body>" "<span id=\"bar\">" "Cool!" "</span>" "</body>" "</html>")
Wowzers! That was way simpler than what we’ve been doing this whole time. Why didn’t we just use something like this from the beginning?
Well that’s because this isn’t actually Enlive code. As I was putting together this tutorial I realized that the for the simple cases Enlive really gets whooped by the other templating engine out there. But Lisp has the answer for just this sort of problem.
Since we’ve been singing Enlive praises most of this time, let’s look at what there is to dislike about it.
One problem is that Enlive doesn’t really let you create templates from a string of markup. The following is invalid:
(deftemplate foo "<span id='foo'></span>" [ctxt] [:#foo] (content (:foo ctxt)))
In order to do that you need to hand an the string as an InputStream to deftemplate. You can do that like so:
(use 'net.cgrand.enlive-html) (import '[java.io ByteArrayInputStream]) (defn to-in-stream [str] (ByteArrayInputStream. (.getBytes str "UTF-8")))
Now you have a nice function to convert strings into InputStreams. You can now define the template like this:
(deftemplate foo (to-in-stream "<span id='foo'></span>") [ctxt] [:#foo] (content (:foo ctxt)))
But really that is kind of annoying. The simple case is just a lot worse then what other templating solutions provide. Can this be made any better?
Well this is Lisp after all!
One of the most powerful tools you have at your disposal are macros. Now these are not you should use just anywhere, but they can be quite helpful when you are finding some unbearably verbose.
In a ideal for the simplest case we would like something like the following:
(templ simple "<span id='foo'></span><span id='bar'></span>")
(simple {:foo "Hello ", :bar "world!"})
Can we have our cake and eat it to?
The following is somewhat advanced and most likely make the most if you have at least some passing familiarity with macros.
Macro allow to you create templates for your code.
When outputting numbers you need to convert them with str.
[:div.foobar] (content (str 1))
Since snippets take a selector sometimes you might not have set this value correctly. This is usually the case if you’re not seeing any output at all from a snippet. It’s really easy to test a snippet – remember they’re just functions.
Your template do not automatically reload. When you make edits to your html or you template I recommend running the following at the REPL:
(load "your-library-name")
Be careful, do not include the .clj extension. Also do not use -’s in your file name. If you want dashes you need to name with actual file using underscores.