Adding jQuery to Sandbar Forms

The previous post, which introduced sandbar.forms, covered the basics of creating server-side forms. This post will focus on the client and how one may easily integrate these forms with a client-side JavaScript library like jQuery.

The starting point will be a simple form for entering information about programmers. It will have text fields for the programmer's name, hire date and favorite programming language. The form will work as in the previous example; each field will be validated on the server after the form is submitted.

This form will be modified to have a datepicker, an autocomplete field for the programming language and will be validated while it is being filled in. To make it even more interesting, the autocomplete field will be populated with programming languages scraped from GitHub using enlive.

The starting point is shown below and is available in the Sandbar Examples project.

(ns sandbar.examples.fancy-form-before
  "A form which will be made fancy by the addition of jQuery."
  (:use [ring.adapter.jetty :only [run-jetty]]
        [ring.middleware.file :only [wrap-file]]
        [hiccup.core :only [html]]
        [hiccup.page-helpers :only [doctype link-to]]
        [compojure.core :only [defroutes GET]]
        [sandbar.core :only [icon stylesheet javascript]]
        [sandbar.stateful-session :only [wrap-stateful-session
                                         set-flash-value!
                                         get-flash-value]]
        [sandbar.validation :only [add-validation-error
                                   build-validator
                                   non-empty-string]])
  (:require [compojure.route :as route]
            [sandbar.forms :as forms]
            [sandbar.examples.database :as db]))

(defn layout
  ([content] (layout content []))
  ([content javascripts]
     (html
      (doctype :html5)
      [:html
       [:head
        (stylesheet "sandbar.css")
        (stylesheet "sandbar-forms.css")
        (icon "icon.png")]
       [:body
        (if-let [m (get-flash-value :user-message)] [:div {:class "message"} m])
        [:h2 "Sandbar Form Example"]
        content]])))

(defn home []
  (layout
   [:div
    (link-to "/developer/edit" "Add Developer")
    [:table
     [:tr
      [:th "Name"] [:th "Hire Date"] [:th "Favorite Language"] [:th ""]]
     (map #(let [{:keys [id name hire-date language]} %]
             [:tr
              [:td name] [:td hire-date] [:td language]
              [:td (link-to (str "/developer/edit/" id) "Edit")]])
          (db/all-users))]]))

(def properties {:name "Name"
                 :hire-date "Hire Date"
                 :language "Language"})

(forms/defform fancy-form "/developer/edit"
  :fields [(forms/hidden :id)
           (forms/textfield :name)
           (forms/textfield :hire-date {:size 10})
           (forms/textfield :language)]
  :load #(db/find-user %)
  :on-cancel "/"
  :on-success #(do
                 (db/store-user %)
                 (set-flash-value! :user-message "Developer has been saved.")
                 "/")
  :validator #(non-empty-string % :name :hire-date :language properties)
  :properties properties)

(defroutes routes
  (fancy-form (fn [request form]
                (layout form)))
  (GET "/" [] (home))
  (route/not-found "Not Found"))

(def application-routes (-> routes
                            wrap-stateful-session
                            (wrap-file "public")))

(defn run []
  (run-jetty (var application-routes) {:join? false :port 8080}))

The home page will display a table of programmers and will have links to add and edit them. The form is shown below.

If the form is submitted without filling in the data, the error messages shown below will be displayed.

With this, we have a pure server-side form. It will now be enhanced by the addition of JavaScript.

Add jQuery

The first step in improving this form will be to add jQuery. The requirements include jQuery itself, the UI library containing the datepicker and autocomplete, and the CSS for the theme of choice. In this example, the ui-lightness theme is used.

To bring these new resources into the project, the layout function is updated to include the additional style sheet and JavaScripts.

(defn layout
  ([content] (layout content []))
  ([content javascripts]
     (html
      (doctype :html5)
      [:html
       [:head
        (stylesheet "sandbar.css")
        (stylesheet "sandbar-forms.css")
        (stylesheet "ui-lightness/jquery-ui-1.8.4.custom.css")
        (icon "icon.png")]
       [:body
        (if-let [m (get-flash-value :user-message)] [:div {:class "message"} m])
        [:h2 "Sandbar Form Example"]
        content
        (map javascript
             (concat ["jquery-1.4.2.min.js" "jquery-ui-1.8.4.custom.min.js"]
                     javascripts))]])))

Notice that this function has been implemented to allow additional JavaScript files to be passed as arguments. The functions stylesheet and javascript are part of Sandbar and assume that files are located in public/css and public/js respectively.

This form will also require some custom JavaScript which will be put into a file named fancy-form.js. It should only be included on the form page so it will be passed as a parameter to the form layout as shown below.

(fancy-form (fn [request form]
                (layout form ["fancy-form.js"])))

Add a Datepicker

In order to work with the form elements from JavaScript, an id must be set for each of them. This may be done by passing a map of attributes to each field, as shown below.

:fields [(forms/hidden :id)
         (forms/textfield :name {:id :name})
         (forms/textfield :hire-date {:id :hire-date :size 10})
         (forms/textfield :language {:id :language})]

In this step, only the id for :hire-date needs to be added; but it is just as easy to add them all.

Adding a datepicker to a text field is simple with jQuery requiring only a single line of code other than the "document ready" function.


The resulting form with activated datepicker is shown below.

Add Autocomplete

The next improvement will be to make the language field an autocomplete field. The list of languages that will populate the autocomplete suggestions will be scraped from the Languages page on GitHub.

Enlive makes this simple, but first it will be included in the require section of the namespace form.

[net.cgrand.enlive-html :as enlive]

Create a function to scrape the language names, memoize it for performance and then create a function to get all names starting with a specific string.

(defn langs-from-github []
  (let [page (-> "http://github.com/languages/"
                 java.net.URL.
                 enlive/html-resource)
        hits (enlive/select page [:div#languages :li :a])]
    (mapcat #(:content %) hits)))

(def langs (memoize langs-from-github))

(defn langs-starting-with [s]
  (let [pattern (re-pattern (str "^" s ".*"))]
    (filter #(re-find pattern %) (langs))))

Each jQuery UI element has many features and options. This example will be simple, providing only the relative URI which will return a JSON formatted array of languages. jQuery will add a request parameter named "term" to the request which will need to be passed to langs-starting-with in order to filter the list. Since JSON is required, support for JSON will be added to the require section of the namespace form.

[clojure.contrib.json :as json]

A function is required that will get the "term" from the request and then return the JSON formatted array of filtered languages.

(defn langs-autocomplete [{params :params}]
  (let [term (get params "term")]
    (json/json-str (map #(hash-map :value %) (langs-starting-with term)))))

Up to this point, nothing has been done which requires support form sandbar.forms. In fact, no support is required to get this to work. All that is required is a route that will call the langs-autocomplete function and pass it the request. As a convenience, defform does provide some help here in the form of the :ajax option. This option is passed a sequence of function symbols for which it will create routes.

:ajax [langs-autocomplete]

This will actually create two routes: "/developer/edit/langs-autocomplete" and "/developer/langs-autocomplete" which will allow a single line addition to fancy-form.js to enable autocomplete on both the add and edit forms.


With these change in place, the language field will now provide suggestions as users type.

Add Live Validation

In the original code, a validator was added to the form which will ensure that values are entered in each field and display an error message if the values are missing. The validator is shown below.

#(non-empty-string % :name :hire-date :language properties)

It is always good to validate on the server, even if client-side validation is performed with JavaScript. It would not be good if the validation code had to be repeated on both the client and the server; especially when that validation code is complex. defform will allow the client to use the server-side validator via AJAX. To enable this, use the :ajax-validate-at option which is set to a name that will be used to create a route for accessing the validator.

:ajax-validation-at "validate"

This will create two routes following the same pattern as the :ajax option; allowing the client to use the relative path "validate" from both the add and edit form.

The desired behavior is that any time the user moves from one field to the next, the previous field will be validated. This may be implemented as shown below in the final version of fancy-form.js.


What does this do? The validateField function is called to validate a specific field, passing it the field's id. The value for the field is retrieved and sent via AJAX using the relative URI with parameters "validate?y=x" for some id y and value x. This URI was configured above with :ajax-validation-at. The response that comes back from this request will have two keys: status and errors. If the status is "fail" and there is an error for the field being validated, that error will be displayed. It could be displayed in any way, but in this example it is displayed in the previously hidden div for that field. The error div will have an id that starts with the field's id and ends with "-error". More than one field could be validated at a time by simply passing additional parameters in the validate request.

CSS Formatting

At this point there is one annoying behavior which occurs when the name field does not pass validation. The error message causes all the fields to move down after the datepicker has been displayed. For interactive validation it would be better if the error messages did not cause the fields to move. This can be done by adding the following CSS to this project.


With this change, the error messages will be placed next the field label and aligned to the right.

The goal has been reached. The form now provides a datepicker, autocomplete and will be validated as the user enters data.

Conclusion

sandbar.forms does not favor any particular JavaScript library. The only help that it provided here was in allowing easy access through AJAX to the validator function as well as any arbitrary function. I don't plan to take this library too far beyond that or to integrate more tightly with JavaScript. There are too many options for what can be done on the client with JavaScript. When a library starts to help you here, it usually only limits what you can do.

The complete code for this example is located in the Sandbar Examples project.

Taming HTML Forms with Clojure

HTML Forms may not be interesting but they are tedious. This post attempts to show you how to turn this tedious problem into an insignificant one.

The main thing that I hate about creating forms is that you usually have to do five things before you have a working form. I like it when I only have to do one thing to get something working; I like to see what I am working on and then iterate.

As of version 0.3.0, Sandbar provides support for forms, form layout and form validation. This post will show you how it works.

Imagine that we have the following code as a starting point. All of the namespaces and functions that we will use are listed here so you don't get confused later on.

(ns sandbar.examples.user-form
  (:use [ring.adapter.jetty :only [run-jetty]]
        [ring.middleware.file :only [wrap-file]]
        [compojure.core :only [defroutes GET]]
        [sandbar.core :only [get-param]]
        [sandbar.stateful-session :only [wrap-stateful-session
                                         set-flash-value!]]
        [sandbar.validation :only [build-validator
                                   non-empty-string
                                   add-validation-error]])
  (:require [compojure.route :as route]
            [sandbar.forms :as forms]
            [sandbar.examples.database :as db]
            [sandbar.examples.views :as views]))

(defroutes routes
  (GET "/" [] (views/home))
  (route/not-found "Not Found"))

(def application-routes (-> routes
                            wrap-stateful-session
                            (wrap-file "public")))

(defn run []
  (run-jetty (var application-routes) {:join? false :port 8080}))

The objective of this post is to create a form which may be used to manage users for an application. The main view of said application displays a table of users and has a link to "/user/edit" which should open a form for creating a new user. For each user in the table, a link to "/user/edit/:id" should open a form where one may edit an existing user with this id. For example, "/user/edit/7" would display a form to edit the user with id = 7.

Creating a simple form

Sandbar supports an iterative approach to building forms. For the first iteration a form is defined with a hidden :id field and a textfield for the :username. This definition is created using the defform macro. With each iteration, new options will be added to this definition.

(forms/defform user-form "/user/edit"
  :fields [(forms/hidden :id)
           (forms/textfield "Username" :username)])

defform creates a route-generating function. Below, this function is used to produce the routes that are needed for the user-form.

(defroutes routes
  (user-form (fn [request form] (views/layout form)))
  (GET "/" [] (views/home))
  (route/not-found "Not Found"))

The route-generating function that is created by defform takes a layout function as a parameter. Here we use our layout to layout the form page. After reloading the application, a click on the link to "/user/edit" will reveal the form shown below.

That's not bad for four lines of code. But this form doesn't really do anything. If we click on the buttons, nothing happens. To make it do something useful, defform will need a little more information.

:on-cancel "/"
:on-success #(do
               (db/store-user %)
               (set-flash-value! :user-message "User has been saved.")
               "/")

The current form has two buttons: Submit and Cancel. The :on-cancel option takes a URL where the user will be redirected when this button is pressed. The :on-success option takes a function of the form data which will return the URL to be redirected to when when this button is pressed.

In the function for :on-success, the data is saved and a message is stored in the Flash before returning this URL. This function will only be called after the form data has been validated. Notice that we could catch errors that may occur at a lower level and then decide where to redirect.

That takes care of form submission and cancellation, but what happens when we attempt to edit a user? The form behaves as if we adding a new user. There is one more option to add to make the form fully operational.

:load #(db/find-user %)

The :load option takes a function of the id. Remember that the id is passed in the link to the edit form ("/user/edit/:id") and can be anything which may be used to lookup the user to edit.

This simple form will now allow our users to add and edit information.

Adding more fields

The form works, but it doesn't collect all of the information that is required. It will be used to enter a password, first and last name and email address for each user as well as select the roles that each user is assigned. As a step forward, additional text fields will be added.

The new list of fields is shown below...

[(forms/hidden :id)
 (forms/textfield "Username" :username)
 (forms/password "Password" :password)
 (forms/textfield "First Name" :first-name)
 (forms/textfield "Last Name" :last-name)
 (forms/textfield "Email" :email)]

...resulting in this form.

It is easy to use strings as labels as has been done above. However, it is better to keep all of this information in one place so that it may be easily changed and perhaps internationalized in the future.

To extract all of the labels, create a map named 'labels'...

(def labels
     {:username "Username"
      :password "Password"
      :first-name "First Name"
      :last-name "Last Name"
      :email "Email Address"})

..and add it to the form definition using the :properties option.

:properties labels

The field list may now be written in a much more concise format.

[(forms/hidden :id)
 (forms/textfield :username)
 (forms/password :password)
 (forms/textfield :first-name :last-name :email)]

To select roles, a more complex field in required. A multi-checkbox is a set of checkboxes that allow the user to make multiple selections for a single form field. Creating these can be a lot of work, but with Sandbar it is one line of code. The following line is added to the end of the field list.

(forms/multi-checkbox :roles (db/all-roles) identity)

Here we are creating a form element named :roles using the sequence that we get back from calling (db/all-roles). The final argument is a function that will return the value that will be selected. (db/all-roles) will return a set of keywords #{:user :admin} so the identity function is used to simply return the keyword. If the data returned from (db/all-roles) were more complex, an arbitrary function could be used to generate a value from each element. The result of making the above change, and adding entries to the labels map for :roles, :user and :admin, is shown below.

This form doesn't just display the checkboxes, it actually works. The code required to marshal the data from the request parameters is automatically generated based on the field types and the data that is passed to each field's constructor. Sandbar also provides support for normal checkboxes, textareas, selects and multi-selects. Custom form elements may also be created.

Adding form validation

The form should have two required fields: username and password. Furthermore, the password should have at least 10 characters.

(defn password-strength [m]
  (if (< (count (:password m)) 10)
    (add-validation-error m :password "Passwords must have > 10 characters.")
    m))

(def user-form-validator
     (build-validator
      (non-empty-string :username :password labels)
      :ensure
      password-strength))

Two functions have been created: password-strength will look at the value of the password and make sure that it has more than 10 characters, user-form-validator is a compound validator. The build-validator macro creates validator functions from other validator functions. The :ensure keyword is used to indicate that we would like all of the previous validations to pass before attempting to do the third validation. The third validation is dependent on the other two. To learn more about form validation, see the Form Validation page in the Sandbar Wiki.

To use this validator, add it to the form definition as shown below.

:validator user-form-validator

After making the above changes, the form below is produced.

Notice that there is an asterisk indicating which fields are required. Trying to submit an empty form will generate the following error messages.

Entering a username and then a password with less than 10 characters will generate this error message.

Layout and Style

Having all of the fields in a single column is not ideal. It would be nice if the first and last name fields were side-by-side. To accomplish this, add a layout vector to the form definition.

:field-layout [1 1 2]

The meaning of this vector is that there should be one field in the first row, one in the second and two in the third. Any missing rows will have one element by default so they may be omitted here.

I also like my forms to have a particular style, with a header and footer. This style is built-in and named :over-under. It may be added like this:

:style :over-under

The form is coming along.

It would be better for the submit button title to read "Save" and for there to a "Save and New" button so that new users may be added quickly. It would also be nice to have a better title. To make these changes add the following two options to the form definition.

:title #(case % :add "Add New User" "Edit User")
:buttons [[:save] [:save-and-new "Save and New"] [:cancel]]

The final version of the form is shown below.

And here is the form definition with all of the changes that were made above.

(forms/defform user-form "/user/edit"
  :fields [(forms/hidden :id)
           (forms/textfield :username)
           (forms/password :password)
           (forms/textfield :first-name :last-name :email)
           (forms/multi-checkbox :roles (db/all-roles) identity)]
  :load #(db/find-user %)
  :on-cancel "/"
  :on-success #(do
                 (db/store-user %)
                 (set-flash-value! :user-message "User has been saved.")
                 "/")
  :properties labels
  :validator user-form-validator
  :field-layout [1 1 2]
  :style :over-under
  :title #(case % :add "Add New User" "Edit User")
  :buttons [[:save] [:save-and-new "Save and New"] [:cancel]])

Extending a form

In some circumstances a form may need its behavior to change at runtime. For example, showing an additional field when the current user is an administrator. Sandbar allows us to do this by extending a form. As a quick example, the form above may be extended so that an additional field is displayed and validated only when a user is being edited.

(forms/extend-form user-form :with edit-form
    :when (fn [action request form-data]
            (or (= action :edit)
                (get-param (-> request :params) :notes)))
    :fields [(forms/textarea :notes {:rows 5 :cols 70})]
    :validator #(non-empty-string % :notes labels))

The function under the :when key is a predicate which will determine when these changes should be applied to the form.

Other options

Arbitrary HTML attributes may be set on any of the form elements by passing a map of attributes as shown above when creating the :notes textarea.

There are two other options to defform that were not covered here: :default and :marshal. These two options allow for setting default values for the form and wrapping the marshal function in order to gain fine control of how data is collected from the request parameters.

For other and more detailed information, see the documentation in the Sandbar Wiki and also take a look at the example form code. The complete source code for this example is located in the Sandbar Examples project.

Conclusion

This feature of Sandbar may not be what you need all of the time but when it does work for you, it can be a big help. It can be a huge timesaver when creating prototypes and for building administrator interfaces. There is a lot that can be done to make this even better. For example, it would be nice if there were more form styles available and more options for field layouts. Given some time, it will have this and much more.

I think this shows the potential of Clojure as a great web programming language. It is an example of how macros may be used to encapsulate a pattern that would normally be repeated over and over.

Securing Clojure Web Applications with Sandbar - Part 2

In part 1 of this series, a simple authorization scheme was added to a small Clojure web application. Continuing with this example, this post will demonstrate how features of sandbar.auth may be used to add form-based authentication and channel security.

Form-based authentication and channel security

If you would like to follow along:

$ git clone git://github.com/brentonashworth/sandbar-examples.git
$ cd sandbar-examples/security
$ open src/sandbar/examples/part_two/start.clj
$ lein deps

This code is a bit different from what we ended with last time. A stylesheet has been added and the layout has been improved. The complete source for our starting point is shown below. Make sure you understand this code before moving on.

(ns sandbar.examples.part-two.start
  (:use (ring.adapter jetty)
        (ring.middleware file)
        (compojure core)
        (hiccup core page-helpers)
        (sandbar core stateful-session auth)))

(defn query [type]
  (ensure-any-role-if (= type :top-secret) #{:admin}
                      (= type :members-only) #{:member}
                      (str (name type) " data")))

(defn layout [content]
  (html
   (doctype :html4)
   [:html
    [:head
     (stylesheet "sandbar.css")
     (icon "icon.png")]
    [:body
     [:h2 "Sandbar Security Example"]
     content
     [:br]
     [:div (if-let [username (current-username)]
             [:div
              (str "You are logged in as " username ". ")
              (link-to "logout" "Logout")])]]]))

(defn data-view [title data & links]
  [:div
   [:h3 title]
   [:p data]
   (if (seq links) links [:div (link-to "home" "Home")])])

(defn home-view []
  (data-view "Home"
             (query :public)
             [:div (link-to "member" "Member Data")]
             [:div (link-to "admin" "Admin Data")]
             [:br]
             [:div (cond (any-role-granted? :admin)
                         "Hello administrator!"
                         (any-role-granted? :member)
                         "Hello member!"
                         :else "Click on one of the links above to log in.")]))

(defn member-view []
  (data-view "Member Page"
             (query :members-only)))

(defn admin-view []
  (data-view "Admin Page"
             (query :top-secret)))

(defn permission-denied-view []
 [:div
  [:h3 "Permission Denied"]
  [:div (link-to "home" "Home")]])

(defroutes my-routes
  (GET "/home*" [] (layout (home-view)))
  (GET "/member*" [] (layout (member-view)))
  (GET "/admin*" [] (layout (admin-view)))
  (GET "/logout*" [] (logout! {}))
  (GET "/permission-denied*" [] (layout (permission-denied-view)))
  (ANY "*" [] (layout (home-view))))

(defn authenticate [request]
  (let [uri (:uri request)]
    (cond (= uri "/member") {:name "joe" :roles #{:member}}
          (= uri "/admin") {:name "sue" :roles #{:admin}})))

(def app (-> my-routes
             (with-security authenticate)
             wrap-stateful-session
             (wrap-file "public")))

(defn run []
  (run-jetty (var app) {:join? false :port 8080}))

Encryption

Our application must ensure that passwords are not sent across the network in plain text. In a production environment, one would enable SSL support on the web server and purchase a legitimate SSL certificate from a valid authority for use with the site's domain. For development, it is good enough to create a self-signed certificate and turn on Jetty's SSL support.

Use Java's keytool to create a self-signed certificate.

$ keytool -genkey -alias sandbar -keyalg RSA -keystore my.keystore -keypass foobar
Enter keystore password:
Re-enter new password:
What is your first and last name?
[Unknown]: localhost
What is the name of your organizational unit?
[Unknown]: dev
What is the name of your organization?
[Unknown]: clojure
What is the name of your City or Locality?
[Unknown]: New York
What is the name of your State or Province?
[Unknown]: New York
What is the two-letter country code for this unit?
[Unknown]: US
Is CN=localhost, OU=dev, O=clojure, L=New York, ST=New York, C=US correct?
[no]: y

For this example, the password "foobar" was entered. keytool has created a keystore in the file named my.keystore. Make sure this file is located in the root directory of the security module (at the same level as the public directory).

To make use of this keystore, update the run function so that it matches the version shown below.

(defn run []
  (run-jetty (var app) {:join? false :ssl? true :port 8080 :ssl-port 8443
                        :keystore "my.keystore"
                        :key-password "foobar"}))

Setting :join? to false will cause the call to run-jetty to return so that the REPL may still be used. The :ssl-port defaults to 443; here we set it here to 8443. Everything else is straight forward. If you are following along, now would be a great time to start a REPL and test that everything is working as expected.

$ lein repl


user=> (use 'sandbar.examples.part-two.start)
user=> (run)

Navigating to https://localhost:8443/ and http://localhost:8080/ confirms that the application may be used over SSL or standard http and that everything works the same as it did before.

Note: You will get a warning message because the certificate that we are using is not legitimate. This is fine for development; do what you need to do to add an exception for this certificate.

Adding form-based authentication

In the last post, the with-security middleware was added and configured to use our authenticate function. In this section, the authenticate function will be replaced with an authentication function from sandbar.form-authentication and a pre-built login form will be added.

Start by adding the required namespaces sandbar.form-authentication and sandbar.validation.

Delete the authenticate function and replace authenticate with form-authentication in our with-security middleware. form-authentication will redirect a user to a login form when that user is not authenticated.

To implement the login form, add form-authentication-routes to the list of routes.

(form-authentication-routes (fn [_ c] (layout c))
                            (form-authentication-adapter))

The parameters to form-authentication-routes are: a layout function and something that satisfies the protocol FormAuthAdapter. The layout function must take two parameters: the request and the content to layout. The FormAuthAdapter protocol specifies two functions which allow us to adapt this component to our system. The functions are load-user and validate-password. load-user takes a username and password and returns a user map. A user map must at least have the keys :username and :roles. validate-password returns a function that can validate the user map created by load-user.

(defrecord DemoAdapter []
  FormAuthAdapter
  (load-user
   [this username password]
   (let [login {:username username :password password}]
     (cond (= username "member")
           (merge login {:roles #{:member}})
           (= username "admin")
           (merge login {:roles #{:admin}})
           :else login)))
  (validate-password
   [this]
   (fn [m]
     (if (= (:password m) (:username m))
       m
       (add-validation-error m "Username and password do not match!")))))

(defn form-authentication-adapter []
  (DemoAdapter.))

This implementation of load-user will simply look at the username to determine if the user is a member or admin. The validate-password implementation will ensure that the username and password are the same. add-validation-error is a function from sandbar.validation which is being used here to display an error message.

(For more information about validators, see the post Clojure Macros Make Me Happy. I will not go into any more detail here.)

After making these changes, saving them and reloading the namespace,

user=> (use :reload-all 'sandbar.examples.part-two.start)

we can return to our application where we should now have an operational login form. Try submitting the form while it is empty. Try entering a username and password that do not match. The form does the right thing in each situation.

Form Customization

At this point you may want to customize the field names on the form as well as the error messages that are displayed. While we are at it, let's make a custom logout landing page. Create a map with keys that correspond to the fields and errors that we would like to update as well as the key :logout-page that indicates where to go after we logout.

(def properties
     {:username "Username"
      :password "Password"
      :username-validation-error "Enter either admin or member"
      :password-validation-error "Enter a password!"
      :logout-page "/after-logout"})

Update the form-authentication-adapter constructor to merge these properties into our DemoAdapter. Because Clojure's records implement the persistent map interface, the form-authentication module uses the FormAuthAdapter as a map to look up field names and error messages for the login form.

(defn basic-auth-adapter []
  (merge (DemoAdapter.) properties))

After making this change, we will see our own field names and error messages displayed on the login form.

Next, create the logout landing page by replacing the empty map that was passed to logout! with properties and then creating a route

(GET "/after-logout" [] (layout (after-logout-view)))

and view.

(defn after-logout-view []
 [:div
  [:h3 "Logout"]
  [:p "You are no longer logged in!"]
  [:div (link-to "home" "Home")]])

In all the changes that have been made so far, a pattern is emerging; add components in the form of middleware or parametrized routes then adapt it to our project. This same pattern will be used in the next section to add channel security.

Channel security

One thing that you may have noticed is that even though SSL is enabled, there is no way to control when it is used and when not. The goal in this application is to secure passwords sent from the client to the server and, because of the additional delay, to use SSL on as few pages as possible. This may be done by adding the with-secure-channel middleware which takes four parameters: the routes to be wrapped, a security configuration, the port and the SSL port. After adding this middleware, the app var definition now looks like this:

(def app (-> my-routes
            (with-security basic-auth)
            wrap-stateful-session
            (wrap-file "public")
            (with-secure-channel security-config 8080 8443)))

The security configuration security-config is a vector of pairs. Each pair is a regular expression literal followed by a configuration. We use a vector here instead of a map because each entry is checked against the current URI in order with the first match being selected.

(def security-config
     [#"/login.*" :ssl
      #".*.css|.*.png" :any-channel
      #".*" :nossl])

Three keywords are used to represent the three kinds of channel security: :ssl, :nossl and :any-channel. The above configuration causes the login screen to always be accessed through SSL, images and stylesheets to go over any channel and everything else to go through standard http.

A future post will show how this same vector may be used to authorized access based on URI patterns.

If you did not follow along, you may want to take a look at the final version of the source for this exmaple.

Conclusion

Progress has been made, but this app is still not secure. Passwords should not be hard coded into an application like this. The application may be improved by having a way to easily assign different passwords to each user and store them securely. There will be at least two more posts in this series; one them will cover some of the new features of Sandbar which can help make this easy for the most common cases. The other will demonstrate how to authorize access to resources based on URI patterns.

Clojure Protocols and the Expression Problem

The official release of Clojure 1.2 is just around the corner. One of the most significant changes in this release is the addition of protocols and datatypes. Below, we will explore one way that these features may be used to improve your Clojure programs. Specifically, we will look at how protocols may be used to avoid the Expression Problem.

According to Wikipedia, the term "Expression Problem" was coined by Philip Wadler:

“The Expression Problem is a new name for an old problem. The goal is to define a datatype by cases, where one can add new cases to the datatype and new functions over the datatype, without recompiling existing code, and while retaining static type safety (e.g., no casts).”

We experience this problem when we want to add functionally to library code that is outside of our control. In the Java world, wrappers are commonly used to adapt a class to an interface. The problem with this approach is that the identity of the thing has changed and now our system has to deal with two types for the same thing. In the Ruby world, with its open classes, monkey patching may be used to add functions to a class. Let's face it, anything with the word monkey in it can’t be good. Monkey patching changes the class for everyone and can cause unforeseen problems. Both of these approaches add incidental complexity to our code which is the exact thing that we are trying to avoid by using Clojure.

In this post we will go over some working code that is intended to simulate a real life situation where we may use protocols to avoid the Expression Problem. The code is located in the protocol-examples project in the module named expresssion-problem. If you would like to follow along:

$ git clone git://github.com/brentonashworth/protocol-examples.git
$ cd protocol-examples/expression-problem
$ lein deps && lein javac && lein test com.corp.employee-test
$ open src/clj/com/corp/employee.clj


Simulating an integration problem

Imagine that we are creating a system to work with employees. We have created a library, shown below, which performs various payroll and benefit calculations on employee data. As is the custom of Clojure people, employees are represented as maps so that they gain all of the benefits of being "generically manipulable".

(ns com.corp.employee)

(defn- bonus [years performance]
  (* (+ (* years 500)
        1000)
     (/ performance 10)))

(defn- earned-vacation [years]
  (+ 10 (* years 2)))

(defn- vacation-value [rate years]
  (* rate (* 8 (earned-vacation years))))

(defn payroll [employee hours]
  (* hours
     (:rate employee)))

(defn total-payroll
  [coll hours-map]
  (reduce (fn [total next]
            (let [[{rate :rate} hours] (val next)]
              (+ total (* rate hours))))
          0
          (merge-with conj (group-by :name coll) hours-map)))

(defn employee-bonus [e]
  (apply bonus ((juxt :years :perf) e)))

(defn employee-vacation-value [e]
  (apply vacation-value ((juxt :rate :years) e)))

(defn total-emp-benefits [employee]
  (+ (employee-vacation-value employee)
     (employee-bonus employee)))

(defn total-benefits [coll]
  (reduce + (map total-emp-benefits coll)))

(defn make-employee
  [name years perf rate]
  {:name name :years years :perf perf :rate rate})

The problem

Now imagine that we have a new requirement. The existing payroll system will need to be able to use our new library to calculate benefits. They happen to be using Clojure as well but the payroll system is written in Java and cannot be changed. Their system has a Java class named com.company.Employee which contains the information that we need but does not conform to our interface. Specifically, they will need to be able to call total-emp-benefits and total-benefits passing instances of com.company.Employee.

We may start to solve this problem by creating some tests that demonstrate what we would like our system to do (if you are following along, these tests are already included in com.corp.integration-test).

(ns com.corp.integration-test
  (use (clojure test)
       (com.corp employee))
  (import com.company.Employee))

(def e1 (make-employee "jim" 2 8 20))
(def e2 {:name "sue" :years 15 :perf 6 :rate 60})
(def e3 (Employee. "james" 2 8 30M))
(def employees [e1 e2 e3])

(deftest test-employee
  (is (= (.calculatePayroll e3 40)
         1200M)))

(deftest test-total-emp-benefits
  (is (= (total-emp-benefits e3)
         4960M)))

(deftest test-total-benefits
  (is (= (total-benefits employees)
         33100M)))

In these tests we create two employee maps and one employee using the closed Employee class from the payroll system. Our first test is a sanity check; calling one of the methods that exists in Employee. The other two tests pass instances of Employee to functions in our system. The last two tests will not pass because our current implementation expects plain Clojure maps.

Whatever our solution is, we want to ensure that we do not change the caller's contract. There is a lot of code in production that is using our library and we don't want to force our callers to have to change their code.

What is our contract? We have six reporting functions that are designed to work with maps of employee data. We also have a constructor function that provides a more compact way to creating these maps. Our callers may or may not be using this constructor. The tests that we have in place are testing the caller's contract so if we can get all of the tests working without making any changes to them then we have been successful.

Clojure provides two solutions to this problem: multimethods and protocols. Multimethods dispatch on a function of their arguments, protocol functions dispatch on the type of their first argument. As long as we are happy with the form of dispatch, protocols provide the additional benefits of interface organization and faster dispatch. We will choose to use protocols for our solution.

Adding a protocol

We start by defining the Benefits protocol. We choose the functions from our current system that calculate benefits and take an employee as the first argument.

(defprotocol Benefits
 (employee-bonus [this])
 (employee-vacation-value [this]))

Because these functions are defined in our protocol, we can no longer use them as standalone functions. We must implement them for each datatype that we would like them to support. To start with, we want to ensure that Clojure maps continue to work. We use extend-protocol to implement the functions for maps using our existing implementations. If you are following along, add the following form and remove the functions employee-bonus and employee-vacation-value.

(extend-protocol Benefits
  clojure.lang.IPersistentMap
  (employee-bonus [this]
                  (apply bonus ((juxt :years :perf) this)))
  (employee-vacation-value [this]
                           (apply vacation-value ((juxt :rate :years) this))))

After making these changes our original tests should still work.

$ lein test com.corp.employee-test

Next, we solve the problem that we set out to solve by extending the type com.company.Employee with the Benefits protocol. The final extend-protocol form is shown below.

(extend-protocol Benefits
  
  clojure.lang.IPersistentMap
  (employee-bonus [this] (bonus (:years this)
                                (:perf this)))
  (employee-vacation-value [this] (vacation-value (:rate this)
                                                  (:years this)))

  com.company.Employee
  (employee-bonus [this] (bonus (.getYearsWithCompany this)
                                (.getCurrentPerformanceRating this)))
  (employee-vacation-value [this] (vacation-value (.getHourlyRate this)
                                                  (.getYearsWithCompany this))))

Now we may run all of our tests and... Congratulations! Everything works. From the callers perspective, we have added the functions employee-bonus and employee-vacation-value to the Employee class. Notice that we have not globally changed this class, these functions are only available within the context of the com.corp.employee namespace.

Improving performance

Because we have chosen to use protocols, we can make one final improvement to our library. We can create a record based on our protocol and then update the make-employee function to create instances of this record instead of creating maps. This will increase application performance. First we create the new record.

(defrecord StandardBenefits [name years perf rate]
  Benefits
  (employee-bonus [this] (bonus years perf))
  (employee-vacation-value [this] (vacation-value rate years)))

Next, update the make-employee function to create an instance of StandardBenefits instead of a map.

(defn make-employee [name years perf rate]
 (StandardBenefits. name years perf rate))

Run all the tests to confirm that everything works.

Notice that even the functions payroll and total-payroll continue to work when they are passed instances of StandardBenefits even thought they are not part of our protocol and are expecting plain Clojure maps. They work because defrecord provides a complete implementation of a persistent map which is the main advantage of using it over deftype.

If you were not following along then you may want to have a look at the finished version.

Conclusion

The most important thing about what we have done here is what we didn't do. We made no change to com.company.Employee, we didn't change our reporting functions or even the functions payroll and total-payroll which depend on the persistent map abstraction, and we made no change to the caller's contract.

For more information about Clojure's protocols see the datatypes and protocols pages on the Clojure web site. There is also a very informative presentation, "Clojure 1.2 Protocols", by Stuart Halloway.

Securing Clojure Web Applications with Sandbar - Part 1

Authorization in the model

This is the first in a series of posts that will show how to use Sandbar to secure web applications. Sandbar is a web application library which adds high level abstractions to Compojure and Ring. The sandbar.auth namespace provides code to make role-based security simple.

Security can be divided into two essential parts: authentication and authorization. In this post we will only explore authorization and are therefore getting only part of the security story. In part two we will add encryption, channel security and form based authentication. In part three we will authorize based on URI patterns. Finally, in the fourth installment, we will use Sandbar to create a user administration tool.

The complete code for this example is located in the sandbar-examples repository under security. If you would like to follow along then clone that repository, get the dependencies, start a REPL and open the sandbar.examples.part-one.start namespace in your favorite editor.

$ git clone git://github.com/brentonashworth/sandbar-examples.git
$ cd sandbar-examples/security
$ open src/sandbar/examples/part_one/start.clj
$ lein deps
$ lein repl


user=> (use ‘sandbar.examples.part-one.start)
user=> (run)

We will start with a small Compojure application and, throughout this series, add features to it. The complete source for our starting point is shown below.

(ns sandbar.examples.part-one.start
 (:use (ring.adapter jetty)
       (compojure core)
       (hiccup core page-helpers)))

(defn query [type]
 (str (name type) " data"))

(defn layout [content]
 (html
  [:html
    [:body
    [:h2 "Sandbar Authorization Example"]
    content]]))

(defn data-view [title data & links]
 [:div
  [:h3 title]
  [:p data]
  (if (seq links) links [:div (link-to "home" "Home")])])

(defn home-view []
 (data-view "Home"
            (query :public)
            [:div (link-to "member" "Member Data")]
            [:div (link-to "admin" "Admin Data")]))

(defn member-view []
 (data-view "Member Page"
            (query :members-only)))

(defn admin-view []
 (data-view "Admin Page"
            (query :top-secret)))

(defroutes my-routes
 (GET "/home*" [] (layout (home-view)))
 (GET "/member*" [] (layout (member-view)))
 (GET "/admin*" [] (layout (admin-view)))
 (ANY "*" [] (layout (home-view))))

(def app my-routes)

(defn run []
 (run-jetty (var app) {:join? false :port 8080}))

(The app var is not required here but having it allows us to load this code once and then make all of the changes below without having to restart our REPL.)

This is a complete application which will show three views: the home page will show public data, the member page will show members-only data and the admin page will show top-secret data. The query function retrieves the data that is displayed.

Add sandbar.auth, middleware and create an authentication function

We would like to have two roles: administrators and members. Only administrators may see top-secret data and only members may see members-only data. We would also like our application to have different behavior based on a user's role.

Before we may add any of Sandbar's functionality, we need to first add Sandbar to our namespace declaration.

(:use (ring.adapter jetty)
       (compojure core)
       (hiccup core page-helpers)
       (sandbar stateful-session auth))

Here we have added the sandbar.stateful-session and sandbar.auth namespaces.

Next, we add some middleware.

(def app
    (-> my-routes
        (with-security authenticate)
        wrap-stateful-session))

Our routes have been wrapped with the with-security and wrap-stateful-session middleware. with-security does all of the hard work. Here it has one argument, an authentication function, which we have not written. The wrap-stateful-session middleware is used by with-security to store user information.

authenticate is where you would do whatever you need to do to figure out who the current user is based on the information in the request. This function is very simple, it takes the request map as an argument and returns a single user map or nil. The user map has two keys: :name, which is a string, and :roles, which is a set of role keywords. For this example, we will use the function below.

(defn authenticate [request]
 (let [uri (:uri request)]
   (cond (= uri "/member") {:name "joe" :roles #{:member}}
         (= uri "/admin") {:name "sue" :roles #{:admin}})))

This function will not secure our application but it does demonstrate how simple it is to add your own authentication scheme. For our example, we simply authenticate the user based on the first URI that is accessed. This function will only be called if the current user is not authenticated.

After making these changes we should still have a working application. Reload the namespace in the REPL and try it out.

user=> (use :reload-all ‘sandbar.examples.part-one.start)

There is a lot of new code running, but the functionality has not changed. At least we know we haven't broken anything.

Protecting data in the model

All of our data is still out in the open. To protect it, we could use the ensure-any-role macro. It takes two arguments: a set of roles and a form that is being protected. The protected form will only be executed if the user is authenticated and has one of the roles in the set.

(defn query [type]
 (ensure-any-role #{:member :admin}
                  (str (name type) " data")))

This isn't really what we are looking for. A simple visit to the home page will result in a java.lang.StackOverflowError. Understanding why will help us understand what is going on behind the scenes. ensure-any-role will throw an exception when the current user is not authenticated. The with-security middleware will catch the exception and then call authenticate to authenticate this user. Our authentication function will not authenticate the user becuase it only authenticates visitors to the admin and member pages. Finally, with-security will redirect the user to the home page and the cycle continues.

This is probably a bug which needs fixing, but it does indicate that we have a problem. There is no way for an unauthenticated user to view the public data on the home page. We could start using conditionals but that will get ugly. Instead, there is a better macro to use in this situation named ensure-any-role-if.

(defn query [type]
 (ensure-any-role-if (= type :top-secret) #{:admin}
                     (= type :members-only) #{:member}
                     (str (name type) " data")))

This macro takes an odd number of arguments. The final argument is the form that you are protecting. The other arguments are pairs of predicates and sets of roles. If the predicate is true then the user must be a member of one of the roles in the corresponding set. If none of the predicates are true then the protected code will be run without authentication.

We can safely try it now. If you first click on the "Member Page" link you will see the members-only data. If you then try to click on the "Admin Page" link, nothing will happen. Our data has been protected.

Logout and permission denied

When ensure-any-role or ensure-any-role-if notice that the current user is authenticated but does not have the correct role, it will redirect the user to "/permission-denied". It would be nice if we could see a better permission denied page. Add the following route and implementation of the permission-denied-view function.

(GET "/permission-denied*" [] (layout (permission-denied-view)))

(defn permission-denied-view []
 [:div
  [:h3 "Permission Denied"]
  [:div (link-to "home" "Home")]])

We may also want to logout so that we can easily try out the different roles.

(GET "/logout*" [] (logout! {}))

The logout route will use a function named logout! from sandbar.auth to clear the current user from the session. The parameter to logout! is a map which may contain the key :logout-page. If this key is not found then the user will be redirected to “/” after logout.

The user will need some kind of link to allow them to logout easily. Add the following div to the end of the body section of the layout.

[:div (if-let [username (current-username)]
        [:div
          (str "You are logged in as " username ". ")
          (link-to "logout" "Logout")])]

The current-username function is another function from sandbar-auth. It will get the current user's username from the session or return nil if one does not exist. After reloading in the REPL, we now see the username at the bottom of the page, we may logout and we have a proper permissions denied page.

Checking the role

As a final flourish for this example we will add the following div to our layout underneath the content and above the logout code that we added above.

[:div (cond (any-role-granted? :admin)
            "Hello administrator!"
            (any-role-granted? :member)
            "Hello member!"
            :else "Click on one of the links above to log in.")]

This shows how to use the any-role-granted? predicate. Here we show a different message based on the role of the current user. any-role-granted? may be passed a single role or a set of roles. It will return true if the user is a member of one of those roles.

The complete source code for the new authorized version is located in the namespace sandbar.examples.part-one.complete.

Conclusion

We haven’t actually secured anything in this example. But hopefully you can see how that would be done by creating a different implementation of authenticate. In the next installment, we will look at one way to do this and add form based authentication to this example project as well as encryption and channel security.

For more information on Sandbar, see the wiki and check out the source code. The source for sandbar.auth is only 245 lines. Sandbar is still quite new and needs a lot of work. I hope this series will draw more attention to it and help to make it better.

Better Clojure Test Results on the Command Line

After letting folks know about Deview earlier this week, some people thought that a command line tool that improved test results would be a better fit for them. Deview has some features that you can't have in a command line tool, and more are planned for the future. However, I can see the value in having a simple tool that does one thing well. Even though I plan to use and continue to improve Deview, I took a day off and created lein-difftest.

lein-difftest is a completely new project that does not depend on Deview. It depends on many of the same libraries as Deview but is much simpler. It may be used as a Leiningen plugin or from the REPL.

To use it from Leiningen you simply add lein-difftest as a dev-dependency. See lein-difftest for more information. Once you have this new library in place, run lein test and you will see the new and improved results. The example below shows two test runs. One without lein-difftest and the other with.



To use it from the REPL there is a helper function named test-ns with takes care of reloading the target namespace and turning off color.

user> (use 'difftest.core)
user> (test-ns 'some.test.namespace)

Robert Hooke

This project is also a great example of using Robert Hooke to extend Leiningen. Before using Hooke, I had to copy the entire leiningen.test/test function into my plugin code so that I could modify one of its function calls. With Hooke, I simple intercept the call to said function and ensure that it has the correct arguments.

Better Clojure Test Results with Deview

Over the past couple of months I have been doing a lot of Clojure development. During this time there were two things that have stood out as workflow killers.

The first is caused by a great feature of Clojure. Because Clojure has a solid notion of equality, you may easily test arbitrarily large Clojure data structures for equality. The problem comes when they are not equal and you need to find out what is different. Like many of you, I use clojure.test. When a test fails, the actual and expected forms are printed. Finding the difference in two large forms can be a nightmare. To feel this pain, try to find the difference in the forms below.

actual: (not (= {:remote-addr "0:0:0:0:0:0:0:1%0", :scheme :http, :query-params {}, :session {:sandbar.stateful-session/session {:project {:host "localhost", :name "deview/deview-server", :port 9000, :_id 1, :_v 1, :_type :projects}}}, :form-params {}, :request-method :get, :query-string nil, :route-params {"*" "/deview/deview-server"}, :content-type nil, :cookies {"ring-session" {:value "0bcoa4e4-a852-4976-b2e9-a697a48f1ed6"}}, :server-name "localhost", :params {"*" "/deview/deview-server"}} {:remote-addr "0:0:0:0:0:0:0:1%0", :scheme :http, :query-params {}, :session {:sandbar.stateful-session/session {:project {:host "localhost", :name "deview/deview-server", :port 9000, :_id 1, :_v 1, :_type :projects}}}, :form-params {}, :request-method :get, :query-string nil, :route-params {"*" "/deview/deview-server"}, :content-type nil, :cookies {"ring-session" {:value "0bcca4e4-a852-4976-b2e9-a697a48f1ed6"}}, :uri "/stats/deview/deview-server", :server-name "localhost", :params {"*" "/deview/deview-server"}}))

On several occasions I spent so much time looking for differences that I forgot what I was doing before encountering the problem.

Another related issue is encountered when an exception is thrown during compile time or while running a test. Usually several screens of stacktrace are output; I usually only need to see the first line. It takes up a lot of time to switch focus to the terminal window and then scroll up to find that one line that I care about. If I have encountered exceptions on the last few test runs then I also have to be careful not to scroll too far and view the results from a previous test.

These two tasks take up much more than just time. It is frustrating when you are working to solve one problem and in the process need to stop and solve a different problem. Some call this yak shaving. They also have something in common; for both of them, I am trying to manually find something that the computer should be able to find for me.

I decided to solve this problem before moving forward on any of my current projects.

Building Blocks

Smart people have already been working on these problems.

At the June Bay Area Clojure Meetup, George Jahad presented a small library he had created to solve the first problem. The library is named difform. It has two useful functions difform and sort-form: when difform is passed two forms it will display the diff, sort-form will sort a Clojure form so that it can be diffed. This is good but I want to see that diff in my test results without having to copy and paste. When a test fails, I want to immediately see what the problem is without interrupting my train of thought.

Mark McGranaghan’s clj-stacktrace makes stacktraces much better. Using the stacktrace middleware provided with Ring is a big timesaver when working on Ring web applications. It would be great if I could use this on my test result stacktraces for all my projects.

Deview

Deview is what I came up with. It makes use of the above two libraries as well as some great features of clojure.test which make it easy to capture reporting information while tests are running.

Deview runs yours tests and reports test failures and successes. If an exception is thrown, clj-stacktrace is used to clean up the stacktrace. Deview will then filter it down to show relevant (according to me) trace elements. For example:

When a test fails, the difference between the expected and actual results are displayed. The diff below shows the changes in the two forms at the beginning of this post.

I have been using this for a couple of weeks and have found it to be a huge time saver.

For complete instructions on installing and using deview, see the project README.

Deview needs to be able to run your tests, so it has to run with your project’s classpath which contains all of your project’s dependencies. Current features of deview as well as some that are planned for the future, require deview to have quite a few dependencies of its own. In order to mitigate dependency conflicts, I decided on a client/server architecture where the server is small with few dependencies and the client can have any number of dependencies.

Deview also stores information about your projects. All of this information is stored with the client so as not to corrupt your project directory.

The server is the part that you add to your project. It only depends on Clojure and clj-stacktrace. The server will run in its own process with your project’s classpath. It will run your tests, read your project.clj file and read source files. It will not in any way modify anything in your project.

The client is a Compojure web application. It can be cloned from GitHub and run from the REPL.

Features

The current focus is on testing. Once you are up and running you will be able to see a list of all of your test namespaces. You may choose to run tests for an individual namespace or for all of them.

While tests are running, your view will be updated once per second showing the current running namespace and test.

If exceptions or failures occur, you will see a short stack trace or a diff as shown in a previous section.

If you choose to run all tests and experience failures in one namespace you may easily click on that namespace to re-run its tests.

To re-run the last set of tests you simply refresh the page. This allows you to easily re-run tests with a couple of keystrokes (Command-TAB, Command-R on a mac).

The Future

There are many ways that deview can be improved in the future. Some of them are listed here.

1. Set up a time interval so that tests are automatically run every now and again
   
2. Group tests
   
3. Run groups in parallel
   
4. Add some useful metrics other than lines of code
   
5. Browse and search sources
   
6. Add links to source in test error messages
   
7. Create test configurations where bindings can be applied, for example: set a flag to not run the tests that hit the database
   
8. When tests are running automatically; when a test fails in a namespace, only run the tests for that namespace until the issue is fixed then go back to running all tests.
   
9. Find all usages of a function in one or more projects
   
10. Interface with a good code coverage tool once one emerges
    

I have more ideas than I have time. If you find deview useful and would like to help make it better then I would love the help. For more information about deview see the project README.