Month: August 2019

Function Monkey for F#

Over the last couple of weeks I’ve been working on adapting Function Monkey so that it feels natural to work with in F#. The driver for this is that I find myself writing more and more F# and want to develop the backend for a new app in it and run it on Azure Functions.

I’m not going to pretend its pretty under the covers but its starting to take shape and I’m beginning to use it in my new backend and so now seemed like a good time to write a little about it by walking through putting together a simple ToDo style API that saves data to CosmosDB.

Declaring a Function App

As ever you’ll need to begin by creating a new Azure Function app in the IDE / editor of your choice. Once you’ve got that empty starting point you’ll need to two NuGet package FunctionMonkey.FSharp to the project (either with Paket or Nuget):

FunctionMonkey.FSharp
FunctionMonkey.Compiler

This is currently in alpha and so you’ll need to enable pre-release packages and add the following NuGet repository:

https://www.myget.org/F/functionmonkey-beta/api/v3/index.json

Next by create a new module called EntryPoint that looks like this:

namespace FmFsharpDemo
open AccidentalFish.FSharp.Validation
open System.Security.Claims
open FunctionMonkey.FSharp.Configuration
open FunctionMonkey.FSharp.Models

module EntryPoint =
    exception InvalidTokenException
    
    let validateToken (bearerToken:string) =
        match bearerToken.Length with
        | 0 -> raise InvalidTokenException
        | _ -> new ClaimsPrincipal(new ClaimsIdentity([new Claim("userId", "2FF4D861-F9E3-4694-9553-C49A94D7E665")]))
    
    let isResultValid (result:ValidationState) =
        match result with
        | Ok -> true
        | _ -> false
                                    
    let app = functionApp {
        // authorization
        defaultAuthorizationMode Token
        tokenValidator validateToken
        claimsMappings [
            claimsMapper.shared ("userId", "userId")
        ]
        // validation
        isValid isResultValid
        // functions
        httpRoute "version" [
            azureFunction.http (Handler(getApiVersion), Get)
        ]
    }

Ok. So what’s going on here? We’ll break it down block by block. We’re going to demonstrate authorisation using a (pretend) bearer token and so we begin by creating a function that can validate a token:

exception InvalidTokenException

let validateToken (bearerToken:string) =
    match bearerToken.Length with
    | 0 -> raise InvalidTokenException
    | _ -> new ClaimsPrincipal(new ClaimsIdentity([new Claim("userId", "2FF4D861-F9E3-4694-9553-C49A94D7E665")]))

This is our F# equivalent of the ITokenValidator interface in the C# version. In this case we take valid to mean any string of length in the authorization header and if the token is valid then we return a ClaimsPrincipal. Again here we just return a made up principal. In the case of an invalid token we simply raise an exception – Function Monkey will translate this to a 401 HTTP status.

We’re going to validate the inputs to our functions using my recently released validation framework. Function Monkey for F# supports any validation framework but as such you need to tell it what constitutes a validation failure and so next we create a function that is able to do this:

let isResultValid result = match result with | Ok -> true | _ -> false

Finally we declare our Function App itself:

let app = functionApp {
    // authorization
    defaultAuthorizationMode Token
    tokenValidator validateToken
    claimsMappings [
        claimsMapper.shared ("userId", "userId")
    ]
    // validation
    isValid isResultValid
    // functions
    httpRoute "version" [
        azureFunction.http (Handler(fun () -> "1.0.0"), Get, authorizationMode=Anonymous)
    ]
}

We declare our settings (and optionally functions) inside a functionApp block that we have to assign to a public member on the module so that the Function Monkey compiler can find your declaration.

Within the block we start by setting up our authorisation to use token validation (line 3) and instruct it to use the token validator function we created earlier (line 4). In lines 5 to 7 we then set up a claims mapping which will set userId on any of our record types associated with functions to the value of the userId claim. You can also set mappings to specific command type property like in the C# version.

On line 9 we tell Function Monkey to use our isResultValid function to determine if a validation results constitutes success of failure.

Then finally on line 11 we declare a HTTP route and a function within it. If you’re familiar with the C# version you can see here that we no longer use commands and command handlers – instead we use functions and their input parameter determines the type of the model being passed into the Azure Function and their return value determines the output of the Azure Function. In this case the function has no parameters and returns a string – a simple API version. We set this specific function to not require authorisation.

Finally lets add a host.json file to remove the auto-prefixing of api to routes (this causes problems with things like Open API output):

{
  "version": "2.0",
  "extensions": {
    "http": {
      "routePrefix": ""
    }
  }
}

If we run this now then in PostMan we should be able go call the endpoint http://localhost:7071/version and receive the response “1.0.0”.

Building our ToDo API

If you’re familiar with Function Monkey for C# then at this point you might be wandering where the rest of the functions are. We could declare them all here like we would in C# but the F# version of Function Monkey allows functions to be declared in multiple modules so that the functions can be located close to the domain logic and to avoid a huge function declaration block.

To get started create a new module called ToDo and we’ll begin by creating a type to model our to do items – we’ll also use this type for updating out to do items:

type ToDoItem =
    {
        id: string
        title: string
        complete: bool
    }

Next we’ll declare a type for adding a to do item:

type AddToDoItemCommand =
    {
        userId: string
        title: string
        isComplete: bool
    }

And finally an type that represents querying to find an item:

type GetToDoItemQuery =
    {
        id: string
    }

Next we’ll declare our validations for these models:

let withIdValidations = [
   isNotEmpty
   hasLengthOf 36
]

let withTitleValidations = [
    isNotEmpty
    hasMinLengthOf 1
    hasMaxLengthOf 255
]

let validateGetToDoItemQuery = createValidatorFor<GetToDoItemQuery>() {
    validate (fun q -> q.id) withIdValidations
}
    
let validateAddToDoItemCommand = createValidatorFor<AddToDoItemCommand>() {
    validate (fun c -> c.userId) withIdValidations
    validate (fun c -> c.title) withTitleValidations
}

let validateToDoItem = createValidatorFor<ToDoItem>() {
    validate (fun c -> c.id) withIdValidations
    validate (fun c -> c.title) withTitleValidations
    validate (fun c -> c.owningUserId) withIdValidations
}

Ok. So now we need to create functions for adding an item to the database and another for getting one from it. We’ll use Azure CosmosDB as a data store and I’m going to assume you’ve set one up. Our add function needs to accept a record of type AddToDoItemCommand and return a new record of type ToDoItem assigning properties as appropriate:

let addToDoItem command =
    {
        id = Guid.NewGuid().ToString()
        owningUserId = command.userId
        title = command.title
        isComplete = command.isComplete
    }

The user ID on our command will have been populated by the claims binding. We don’t write the item to Cosmos here, instead we’re going to use an output binding shortly.

Next our function for reading a to do item from Cosmos:

let getToDoItem query =
    CosmosDb.reader<ToDoItem> <| query.id

CosmosDb.reader is a super simple helper function I created:

namespace FmFsharpDemo
open Microsoft.Azure.Cosmos
open System

module CosmosDb =
    let cosmosDatabase = "testdatabase"
    let cosmosCollection = "colToDoItems"
    let cosmosConnectionString = Environment.GetEnvironmentVariable("cosmosConnectionString")
    
    let reader<'t> id =
        async {
            use client = new CosmosClient(cosmosConnectionString)
            let container = client.GetContainer(cosmosDatabase, cosmosCollection)
            let! response = container.ReadItemAsync<'t>(id, new PartitionKey(id)) |> Async.AwaitTask
            return response.Resource
        }
    

If we inspect the signatures for our two functions we’ll find that addToDoItem has a signature of AddToDoItemCommand -> ToDoItem and getToDoItem has a signature of GetToDoItemQuery -> Async<ToDoItem>. One of them is asynchronous and the other is not – Function Monkey for F# supports both forms. We’re not going to create a function for updating an existing item to demonstrate handler-less functions (though as we’ll see we’ll duck a slight issue for the time being!).

There is one last step we’re going to take before we declare our functions and that’s to create a curried output binding function:

let todoDatabase =
    cosmosDb cosmosCollection cosmosDatabase

In the above cosmosDb is a function that is part of the Function Monkey output binding set and it takes three parameters – the collection / container name, the database name and finally the function that the output binding is being applied to. We’re going to use it multiple times so we create this curried function to make our code less repetitive and more readable.

With all that we can now declare our functions block:

let toDoFunctions = functions {
    httpRoute "api/v1/todo" [
        azureFunction.http (AsyncHandler(getToDoItem),
                            verb=Get, subRoute="/{id}",
                            validator=validateGetToDoItemQuery)
        azureFunction.http (Handler(addToDoItem),
                            verb=Post,
                            validator=validateAddToDoItemCommand,
                            returnResponseBodyWithOutputBinding=true)
            |> todoDatabase
        azureFunction.http (NoHandler, verb=Put, validator=validateToDoItem)
            |> todoDatabase
    ]
}

The functions block is a subset of the functionApp block we saw earlier and can only be used to define functions – shared configuration must go in the functionApp block.

Hopefully the first, GET verb, function is reasonably self-explanatory. The AsyncHandler case instructs Function Monkey that this is an async function and we assign a validator with the validator option.

The second function, for our POST verb, introduces a new concept – output bindings. We pipe the output of azureFunction.http to our curried output binding and this will result in a function being created that outputs to Cosmos DB. Because we’re using the Cosmos output binding we also need to add the Microsoft.Azure.WebJobs.Extensions.CosmosDB package to our functional project. We set the option returnResponseBodyWithOutputBinding to true so that as well as sending the output of our function to the output trigger we also return it as part of the HTTP response (this is optional as you can imagine in a more complex scenario that could leak data).

Finally for the third function our PUT verb also uses an output binding but this doesn’t have a handler at all, hence the NoHandler case. In this scenario the command that is passed in, once validated, is simply passed on as the output of the function. And so in this instance we can PUT a to do item to our endpoint and it will update the appropriate entry in Cosmos. (Note that for the moment I have not answered the question as to how to prevent one user from updating another users to do items – our authorisation approach is currently limited and I’ll come back to that in a future post).

Trying It Out

With all that done we can try this function app out in Postman. If we begin by attempting to add an invalid post to our POST endpoint, say with an empty title, we’ll get a 400 status code returned and a response as follows:

{
  "case": "Errors",
  "fields": [
    [
      {
        "message": "Must not be empty",
        "property": "title",
        "errorCode": "isNotEmpty"
      },
      {
        "message": "Must have a length no less than 1",
        "property": "title",
        "errorCode": "hasMinLengthOf"
      }
    ]
  ]
}

Now if we run it with a valid payload we will get:

{
  "id": "09482e8d-41aa-4c25-9552-b7b05bf0a787",
  "owningUserId": "2FF4D861-F9E3-4694-9553-C49A94D7E665",
  "title": "Buy underpants",
  "isComplete": false
}

Next Steps

These are with me really – I need to continue to flesh out the functionality which at this point essentially boils down to expanding out the computation expression and its helpers. I also need to spend some time refactoring aspects of Function Monkey. I’ve had to dig up and change quite a few things so that it can work in this more functional manner as well as continue to support the more typical C# patterns.

Then of course there is documentation!

F# Validation Framework

If you follow me on Twitter you’ll know I’ve been working on a first class F# interface for Function Monkey. I’d got to the point where it made sense to put together a nice little exemplar of how its used and I went looking for a simple, ideally declarative, validation framework. Something akin to Fluent Validation but for F#. As I’d been working with computation expressions to build the Function Monkey DSL they were foremost in my mind as an approach to a clean validation framework.

I had a hunt around but to my surprise couldn’t really find anything. Closest I came was a blog post that built out a validation framework and left things speculating if it would be possible to use a computation expression to clean things up (short version if you read on – yes you can).

Why not use Fluent Validation itself? While you can happily use NuGet packages written in C# within F# their API surface doesn’t feel very functional. Unsurprisingly if they’re written in C# they tend to focus on C# patterns and classes and so consuming them from a functional environment can feel a bit weird and look a bit odd.

As a result I started playing around with a few things and before I knew it I’d written the skeleton of a framework that used a declarative / DSL type approach to validations, it seemed promising and useful so I spent some more time fleshing out.

It’s available on GitHub, along with its documentation, and I’ll spend some time giving an overview of it here.

Simple Validation

First lets consider a very simple model to represent a customer of a store:

type Customer =
    {
        name: string
        dateOfBirth: DateTime
    }

Let’s begin by validating the name (we’ll come back to the date of birth later) to make sure its not empty and has a maximum length of an arbitrary 128 characters. We do this as shown below:

let customerValidator = createValidatorFor<Customer>() {
    validate (fun c -> c.name) [
        isNotEmpty
        hasMaxLengthOf 128
    ]
}

In this code we use the createValidatorFor function to return a validation builder for our type and within the curly braces we can declare our validations.

The validate command is one we’ll use a lot. It’s a function with two parameters – the first takes a lambda that returns the property we wish to validate and the second parameter is an array of our validators. In this case we’re saying the name property of a customer must not be empty and can have a max length of 128 characters.

If we look at customerValidator we’ll find that we’ve been returned a function with the signature Customer -> ValidateState and below is shown an example of calling this and triggering a validation failure:

let outputToConsole state =
    match state with
    | Ok -> printf "No errors\n\n"
    | Errors e -> printf "%O\n\n" e

{ name = "" ; dateOfBirth = DateTime.UtcNow.AddYears(-1) }
|> customerValidator
|> outputToConsole

If we run this we’ll see this output:

[{message = "Must not be empty";
 property = "name";
 errorCode = "isNotEmpty";}]

Complex Models

Lets extend our model to something more complex – an order that references a customer and a collection of ordered items:

type Customer =
    {
        name: string
        dateOfBirth: DateTime
    }

type Product =
    {
        name: string
        price: double
    }

type OrderItem =
    {
        product: Product
        quantity: int
    }

type Order =
    {
        customer: Customer
        items: OrderItem list
    }

We’ll now build a validator that validates:

  • That the customer is valid
  • That the order has at least 1 item
  • That each order item has a quantity of at least 1
  • That each product has a name and that name has maximum length of 128
  • And finally that each product has a price of greater than 0

This is going to introduce the concept of validating collections and we’ll also use deep property paths to validate the product:

let orderValidator = createValidatorFor<Order>() {
    validate (fun o -> o.customer) [
        withValidator customerValidator
    ]
    validate (fun o -> o.items) [
        isNotEmpty
        eachItemWith (createValidatorFor<OrderItem>() {
            validate (fun i -> i.quantity) [
                isGreaterThan 0
            ]
            validate (fun i -> i.product.price) [
                isGreaterThan 0.
            ]
            validate (fun i -> i.product.name) [
                isNotEmpty
                hasMaxLengthOf 128
            ]
        })
    ]
}

To create this validator we’ve introduced three new concepts:

  • Firstly we are referencing the existing customer validator using the withValidator command. This simply takes a validator function.
  • Next we are validating the items in the collection with the eachItemWith command. This also takes a validator function which, in this case, we are creating in line but we could take the same approach as with the customer – declare it separately and simply pass it in here.
  • Finally we are validating the product properties using multi-part property paths e.g. i.product.price. These functions literally are getter functions for our properties – the expressions are evaluated when the property runs and the return type informs what validators can be used.

First lets run this on a valid order:

{
    customer = { name = "Alice" ; dateOfBirth = DateTime.UtcNow.AddYears(-1) }
    items = [
        {
            quantity = 1
            product = { name = "Baked Beans" ; price = 3.50 }
        }
    ]
}
|> orderValidator
|> outputToConsole

This produces the output:

No errors

However lets now give the customer an empty name and set the price of a product to 0:

{
    customer = { name = "" ; dateOfBirth = DateTime.UtcNow.AddYears(-1) }
    items = [
        {
            quantity = 1
            product = { name = "Baked Beans" ; price = 0.0 }
        }
    ]
}
|> orderValidator
|> outputToConsole

When we run this we get the following (I’ve tidied up the standard output slightly for clarity):

[
    {
        message = "Must be greater than 0";
        property = "items.[0].product.price";
        errorCode = "isGreaterThan";
    };
    {
        message = "Must not be empty";
        property = "customer.name";
        errorCode = "isNotEmpty";
    }
]

We can see in the results that the expected validations have failed and that the paths to those properties identify the source of the error including the item index in the collection.

Custom Validators

Lets extend our customer validator to ensure the customer placing the order is over 18. We’ve stored their date of birth and so will need to calculate their age as part of the validation. We can do that by adding a custom validator.

Validators have a signature of string -> ‘propertyType -> ValidationState. The first parameter is the path to the property being validated, the second is the value being validated and the validator must return a ValidationState which is a discriminated union with cases of Ok and Errors.

That being the case our 18 or over validator will look like this:

let is18OrOlder propertyPath (value:DateTime) =
    let today = DateTime.Today
    let age = today.Year - value.Year - (match value.Date > today.Date with | true -> 1 | false -> 0)  
    match age >= 18 with
    | true -> Ok
    | false -> Errors([{ errorCode = "is18OrOlder" ; message="Not 18" ; property = propertyPath }])

And we can use it like any other validator:

let customerValidator = createValidatorFor<Customer>() {
    validate (fun c -> c.name) [
        isNotEmpty
        hasMaxLengthOf 128
    ]
    validate (fun c -> c.dateOfBirth) [
        is18OrOlder
    ]
}

You’ll notice that some validators take parameters – how do we do this? To accomplish that we write a function that accepts our validation parameters and returns the validator function. Lets change our validator to check if someone is over a parameterised age:

let isOverAge age =
    let validator propertyPath (value:DateTime) =
        let today = DateTime.Today
        let age = today.Year - value.Year - (match value.Date > today.Date with | true -> 1 | false -> 0)  
        match age >= age with
        | true -> Ok
        | false -> Errors([{ errorCode = "is18OrOlder" ; message="Not 18" ; property = propertyPath }])
    validator
        
let customerValidator = createValidatorFor<Customer>() {
    validate (fun c -> c.name) [
        isNotEmpty
        hasMaxLengthOf 128
    ]
    validate (fun c -> c.dateOfBirth) [
        isOverAge 18
    ]
}

Conditional Validation

Let’s now expand on our age related validation and rather than say all customers must be 18 or over let’s change the validation to be customers 18 or over cannot buy products with a name of Alcohol. We can use the validateWhen command to introduce this additional logic:

let calculateAge (dateOfBirth:DateTime) =
    let today = DateTime.Today
    let age = today.Year - dateOfBirth.Year - (match dateOfBirth.Date > today.Date with | true -> 1 | false -> 0)
    age
let orderValidator = createValidatorFor<Order>() {
    validate (fun o -> o.customer) [
        withValidator (createValidatorFor<Customer>() {
            validate (fun c -> c.name) [
                isNotEmpty
                hasMaxLengthOf 128
            ]
        })
    ]
    validate (fun o -> o.items) [
        isNotEmpty
        eachItemWith (createValidatorFor<OrderItem>() {
            validate (fun i -> i.quantity) [
                isGreaterThan 0
            ]
            validate (fun i -> i.product.price) [
                isGreaterThan 0.
            ]
            validate (fun i -> i.product.name) [
                isNotEmpty
                hasMaxLengthOf 128
            ]
        })
    ]
    validateWhen (fun o -> (o.customer.dateOfBirth |> calculateAge) < 18) (fun o -> o.items) [
        eachItemWith (createValidatorFor<OrderItem>() {                
            validate (fun i -> i.product.name) [
                isNotEqualTo "Alcohol"
            ]
        })
    ]
}

In this example our existing validations for the collection items will all run but if the customer is under 18 then we will run an additional block of validations. We can see this by running an order through the validator that includes a quantity of an item set to 0 and a product named Alcohol (for a customer younger than 18!):

{
    customer = { name = "Alice" ; dateOfBirth = DateTime.UtcNow.AddYears(-1) }
    items = [
        {
            quantity = 1
            product = { name = "Baked Beans" ; price = 3.50 }
        }
        {
            quantity = 0
            product = { name = "Alcohol" ; price = 10.50 }
        }
    ]
}
|> orderValidator
|> outputToConsole

This produces the following output:

[
    {
        message = "Must not be equal to Alcohol";
        property = "items.[1].product.name";
        errorCode = "isNotEqualTo";
    };
    {
        message = "Must be greater than 0";
        property = "items.[1].quantity";
        errorCode = "isGreaterThan";
    }
]

Correctly running the conditional and non-conditional validations for our younger customer.

Other Features and Finishing Off

The framework includes other features such as support for discriminated unions, lambda validators and additional conditional support which is all documented on the frameworks homepage.

This really was just something that “sprang out” and I’d love to hear your feedback over on Twitter.

Posted in F#

F# from a C# Developers Perspective – Part 4

Below is a random collection of things that have tickled me and/or I’ve greatly appreciated as I’ve started to write more code in F#.

Exceptions

I love how concise it is to declare and use an F# exception:

exception OoopsButterFingers

let catchTheBall () =
    raise OoopsButterFingers

Or to attach some data:

exception GetOffMyLawn of string

let strayOntoLawn () =
    raise (GetOffMyLawn "Oi!")

Single Case Discriminated Unions

Leaving the question of mutability aside I’m sure we’ve all got examples of mis-assigning IDs in models:

class User
{
    public string UserId { get; set; }
}

class Order
{
    public string OrderedByUserId { get; set; }
    public string OrderId { get; set; }
}

User user = getUser();
Order newOrder = new Order {
    OrderedByUserId = "newid",
    OrderId = user.UserId // oops, we've mis-assigned an ID 
};

It’s easily done right? And we could certainly improve that in C# but I love how simple this is in F#:

type UserId = | UserId of string

type OrderId = | OrderId of string

type User =
    {
        userId: UserId
    }
    
type Order =
    {
        orderedByUserId: UserId
        orderId: OrderId
    }
    
let user = { userId = UserId("abc") }

// This won't compile - both assignments will fail with type mismatches 
let newOrder = {
    orderedByUserId = "newId"
    orderId = user.userId
}

Currying is not named after a vindaloo

While explaining the concept of currying to a friend he asked the obvious question “why the heck is it called currying?”. I had no idea and briefly entertained the notion that it had been named in jest after, well, a curry. Which gave me a good chuckle.

However it turns out its named after an American mathematician and logician, one Haskell Curry. And so at the same time I also learned why the language Haskell is called Haskell. One of those things you feel somewhat of an ignorant philistine over not knowing.

And with that I’ll leave you with a photo of a delicious curry.

Photo via Unsplash – unsplash-logoSteven Bennett

Posted in F#

F# from a C# Developers Perspective – Part 3

I’ve been writing a lot of (production) F# since my last post on this topic and while I don’t consider myself an expert (by any means!) I’ve certainly become a lot more familiar with the language and have found myself falling into what Isaac Abraham refers to as “the pit of success”.

It was hard to understand, or at least appreciate, what he meant by this on my first pass through his book (which I recommend by the way – that really helped get me on my way) and I think that’s because F# doesn’t lead you towards success (i.e. bug free code) through any one language feature – but rather how all its features interlock and go together.

It really came together for me when I undertook a significant refactoring of some code and when I had finished and pressed build and it compiled without errors and warnings I realised that I knew what I’d done worked. I didn’t have to test it or try it. I knew it had to work (and there’s no twist in this tale – it did).

Amongst other things I had no mutable state, I’d used single case case discriminated unions to protect against mis-assignments, I’d used the type system to protect against bad data, I’d used the Seq functions almost exclusively to work on collections, I’d favoured matching over if/then/else constructs and had no missing matches, I’d used modules and namespaces to organise my code, and I had no nulls!

In fact as this application comes together, its a mobile app that I am building with the Fabulous framework – and the Model->View->Update architecture is perfect for a functional language, the only thing I really find myself worrying about is does the UI layout correctly.

It’s been a fascinating, and very positive, experience for someone who has been working with C# and JavaScript for so many years. I’d sum it up as this:

F# makes me work a little harder to express concepts in code but once I have I am more confident the result is correct and far less brittle and as a result I am more productive.

It’s not that C# “forces” you to write brittle code – but it makes it far easier to and so in some ways, putting the paradigms to one side and focusing on the un-brittleness of code (and as someone who has used all three languages), I am starting to think of F# being to C# as C# is to C++.

Posted in F#

Contact

  • If you're looking for help with C#, .NET, Azure, Architecture, or would simply value an independent opinion then please get in touch here or over on Twitter.

Recent Posts

Recent Tweets

Recent Comments

Archives

Categories

Meta

GiottoPress by Enrique Chavez