Building a REST/JSON webservice using a pure functional programming language

May 31, 2016

Two years ago I joined a few friends which were working on a little hobby project. We were creating a website which allowed you to track your watch progression of your favorite television series. The project was called Teevy and is still online, but some parts of it might function a bit strange (no maintenance done to it lately).

A logged in user at teevy.co

A logged in user at teevy.co

The goal was to develop a Minimal Viable Product (MVP) first. We had no real business model for it, we were just passionate about what we do and this was just a hobby project for us to try new things.

In the next few minutes I want to tell you about my journey creating a REST/JSON backend for Teevy with a pure functional programming language called Haskell.

Haskell, a pure functional programming language

Haskell you say? Yes, Haskell. I didn’t know that much Haskell at that time. I read some books about it, and I just wanted to give it a go and learn it by applying it. So what are some nice properties of functional programming and Haskell?

Types

Types are a important recipe for Haskell and functional programming. They allow us to exactly capture what a function should accept as input and will give as output. They also describe how data structures look like.

Types can assure you that you are using the right shapes. This is in fact some sort of test. In fact some languages remove the need for a lot tests by using really expressive types (for example: vectors and matrices which have their dimensions included in the type). Also typing makes refactoring a lot easier, since the compiler will tell you when you are using the wrong shape.

Opposed to dynamic languages which will let you know that you’ve made an ‘oopsie’ at runtime. To cover that usually a lot of tests are written (which also need to be refactored after a refactor) to avoid such oopsies.

Side effects

In a pure functional programming all (side) effects are captured and are handled explicitly. Examples of effects are optional values (Option/Maybe), asynchronous values (IO/Future/Task), disjunction values (Either), concurrent values (STM), modification of values, etc.

The advantage is that you will handle all effects explicitly and therefore it reduces the risk of having a bug in your program. A well known problem with a C# and Java programs is having the null pointer exception at runtime in your program. In that case the program references to a unset reference in memory, causing the program to terminate. This might be okay if you have set up a exception handler, but mapping this error to a good human readable error requires you rethrow exceptions and write lots of boilerplate. And what if you have two or more potential null cases in your program? Yes, it will be a mess. Also these functions which do not handle these side effects explicitly do not compose.

What do I mean by this? Programs without explicit side effects can do things in between which might not give the same outcome every time. A pure expression like 1 + 1 will output 2 every time. In other words create programs which are a bunch of expressions instead of a bunch of statements. Expressions compose!

Type classes

A type class is a common structure, like a interface. But it is ad-hoc (that’s why we call it a type class instance) and can abstract over high-kinded types (Maybe a is a higher kinded type, it takes one type parameter to create a fully constructed type).

The most well known type classes in Haskell and functional programming are: Functor, Monoid, Monad, Applicative, etc.

For example we can have a Functor (which offers a map function) instance for a optional value and asynchronous values.

Composition, composition and composition

Every where you look in functional programming, you will see composition. Functions compose, functors compose, applicatives compose, lenses compose, natural transformations compose, contravariant functors compose, etc. When you dig deeper you’ll see it everywhere. In the end, when we are programming we decompose problems in small parts and compose them together to form a program.

The ecosystem

Almost all libraries written in Haskell can be found on hackage. Ranging from database drivers, web servers, parser libaries to numerical computing. Also the documentation of any library is very easy to comprehend. Some libraries I’ve used for Teevy are:

You can use cabal to install these packages, which I did back then. Nowadays you would use something like stack to manage your build (which uses cabal underneath).

Creating a webservice

So what do we need to build a simple webservice?

Let’s start with the data part.

Dealing with data

To create a webservice, you’ll need to define data types and create type class instances which can:

Our data

Let’s pretend we are dealing with a user, just to keep things simple. Here’s the definition of the user data record.

data User = User { name :: String, fileQuota :: Int }

JSON

I’ve used the aeson library to work with JSON. An example of a JSON decoder typeclass instance for a user looks like this (if you write it out by hand).

instance FromJSON User where
    parseJSON (Object v) = User <$>
                           v .: "name" <*>
                           v .: "fileQuota"
    -- A non-Object value is of the wrong type, so fail.
    parseJSON _          = empty

The symbols <$> and <*> are just functions. I won’t go much into detail, but these functions are related to Functor and Applicative constructs in functional programming. Aeson and Haskell can automatically derive the type class instance for the type User. Removing the need to write this boilerplate. Something like this:

instance FromJSON User

Finally we can use a decode function defined as decode :: FromJSON a => ByteString -> Maybe a. As you can see the generic parameter a requires you to have typeclass instance in scope of FromJSON. If that’s the case, it will take ByteString and converts it into a optional value Maybe a.

This a great example of typeclasses (decoder typeclass) and explicit effects (returning a optional value).

Database

We can also do the typeclass stuff for database access (using postgres-simple). Like this:

instance FromRow User where
    fromRow = User <$> field <*> field

The FromRow type class will extract columns from a result set and put them into a record. Selecting the right fields and the order of the fields in the query is important!

By supplying a instance of FromRow for the type User we can use it with query_ :: FromRow r => Connection -> Query -> IO [r]. As you can see it requires a FromRow type class instance again for r. If that’s settled, we can supply a Connection and Query. It will give us back a IO [r] which is a list of r values.

Using this we can query the database. Like so:

allUsers :: Connection -> IO [User]
allUsers conn = query_ conn "select * from user"

Scotty

Scotty is a Haskell web framework inspired by Ruby’s Sinatra, using WAI and Warp. A small example from their website:

{-# LANGUAGE OverloadedStrings #-}

import Web.Scotty

import Data.Monoid (mconcat)

main = scotty 3000 $ do
  get "/:word" $ do
    beam <- param "word"
    html $ mconcat ["<h1>Scotty, ", beam, " me up!</h1>"]

This will host a scotty server at port 3000. When you enter localhost:3000/mark it will print you "<h1>Scotty, mark me up!</h1>"

Quite simple huh? (except the monoid stuff if you haven’t seen it, but its just concat stuff really (with some laws)). In a few lines of code we’ve created a webservice, how cool is that? :)

To output JSON we can use json :: ToJSON a => a -> ActionM () instead of html. As you can see we need to have a ToJSON (encoder) type class instance in scope which is the opposite of FromJSON (decoder). I won’t go into detail, but you can make a instance by using instance ToJSON User. For example you could write json fetchedUsers where fetchedUsers :: [User] should return a array of users.

Config parsing

Every website nowadays has some configuration file which contains database connection strings, Facebook secrets, settings, etc. Haskell has a nice library to work with config files called: configurator

In the example below we load the properties using the configurator library:

data TvConfig = TvConfig {
        teevyConnectInfo :: ConnectInfo
    ,   teevyPort :: Int
    ,   teevySecret :: T.Text
    ,   tmdbApiKey :: TL.Text
    ,   facebookSecret :: B.ByteString
} deriving (Show, Eq)

processDatabaseInfo :: Config -> IO (Maybe ConnectInfo)
processDatabaseInfo cfg = do
  host <- lookup cfg "db.host"
  port <- lookup cfg "db.port"
  db <- lookup cfg "db.db"
  user <- lookup cfg "db.user"
  pass <- lookup cfg "db.pass"
  return (ConnectInfo <$> host <*> port <*> user <*> pass <*> db)

processConfig :: Config -> IO (Maybe TvConfig)
processConfig cfg = do
  db <- processDatabaseInfo cfg
  port <- lookup cfg "teevy.port"
  secret <- lookup cfg "teevy.secret"
  tmdbKey <- lookup cfg "tmdb.key"
  fbSecret <- lookup cfg "facebook.secret"
  return (TvConfig <$> db <*> port <*> secret <*> tmdbKey <*> fbSecret)

loadConfig :: String -> IO (Maybe TvConfig)
loadConfig path = do
  exists <- doesFileExist path
  if exists
  then withLoad [Required path] processConfig
  else return Nothing

The takeaways here once again: simple functions, declarative parsing of the different sections and taking failure into account.

Conclusion

It’s been two years ago since I’ve built Teevy. During these past two years I’ve learned a lot and some stuff can be greatly simplified or improved. Looking back at it, I am quite satisfied with what I have achieved. I learned a tremendous amount and I’ve created a nice MVP :) The source code can be found here

It also proves that you can write REST backends with Haskell. Many other people have done it and they also like the advantages of it. Some complementary work can be found at:

Learning Haskell

To learn Haskell, there are a few options. Like I have learned a lot from reading: Learn you a Haskell for a Great Good, but it seems Haskell programming gains more popularity lately.

Written by

Mark de Jong

Mark de Jong

Scala/Haskell Software Developer - Specialised in web-apps and high performance backend systems.