F# Validation Framework

If you follow me on Twitter you’ll know I’ve been working on a first class F# interface for Function Monkey. I’d got to the point where it made sense to put together a nice little exemplar of how its used and I went looking for a simple, ideally declarative, validation framework. Something akin to Fluent Validation but for F#. As I’d been working with computation expressions to build the Function Monkey DSL they were foremost in my mind as an approach to a clean validation framework.

I had a hunt around but to my surprise couldn’t really find anything. Closest I came was a blog post that built out a validation framework and left things speculating if it would be possible to use a computation expression to clean things up (short version if you read on – yes you can).

Why not use Fluent Validation itself? While you can happily use NuGet packages written in C# within F# their API surface doesn’t feel very functional. Unsurprisingly if they’re written in C# they tend to focus on C# patterns and classes and so consuming them from a functional environment can feel a bit weird and look a bit odd.

As a result I started playing around with a few things and before I knew it I’d written the skeleton of a framework that used a declarative / DSL type approach to validations, it seemed promising and useful so I spent some more time fleshing out.

It’s available on GitHub, along with its documentation, and I’ll spend some time giving an overview of it here.

Simple Validation

First lets consider a very simple model to represent a customer of a store:

type Customer =
    {
        name: string
        dateOfBirth: DateTime
    }

Let’s begin by validating the name (we’ll come back to the date of birth later) to make sure its not empty and has a maximum length of an arbitrary 128 characters. We do this as shown below:

let customerValidator = createValidatorFor<Customer>() {
    validate (fun c -> c.name) [
        isNotEmpty
        hasMaxLengthOf 128
    ]
}

In this code we use the createValidatorFor function to return a validation builder for our type and within the curly braces we can declare our validations.

The validate command is one we’ll use a lot. It’s a function with two parameters – the first takes a lambda that returns the property we wish to validate and the second parameter is an array of our validators. In this case we’re saying the name property of a customer must not be empty and can have a max length of 128 characters.

If we look at customerValidator we’ll find that we’ve been returned a function with the signature Customer -> ValidateState and below is shown an example of calling this and triggering a validation failure:

let outputToConsole state =
    match state with
    | Ok -> printf "No errors\n\n"
    | Errors e -> printf "%O\n\n" e

{ name = "" ; dateOfBirth = DateTime.UtcNow.AddYears(-1) }
|> customerValidator
|> outputToConsole

If we run this we’ll see this output:

[{message = "Must not be empty";
 property = "name";
 errorCode = "isNotEmpty";}]

Complex Models

Lets extend our model to something more complex – an order that references a customer and a collection of ordered items:

type Customer =
    {
        name: string
        dateOfBirth: DateTime
    }

type Product =
    {
        name: string
        price: double
    }

type OrderItem =
    {
        product: Product
        quantity: int
    }

type Order =
    {
        customer: Customer
        items: OrderItem list
    }

We’ll now build a validator that validates:

  • That the customer is valid
  • That the order has at least 1 item
  • That each order item has a quantity of at least 1
  • That each product has a name and that name has maximum length of 128
  • And finally that each product has a price of greater than 0

This is going to introduce the concept of validating collections and we’ll also use deep property paths to validate the product:

let orderValidator = createValidatorFor<Order>() {
    validate (fun o -> o.customer) [
        withValidator customerValidator
    ]
    validate (fun o -> o.items) [
        isNotEmpty
        eachItemWith (createValidatorFor<OrderItem>() {
            validate (fun i -> i.quantity) [
                isGreaterThan 0
            ]
            validate (fun i -> i.product.price) [
                isGreaterThan 0.
            ]
            validate (fun i -> i.product.name) [
                isNotEmpty
                hasMaxLengthOf 128
            ]
        })
    ]
}

To create this validator we’ve introduced three new concepts:

  • Firstly we are referencing the existing customer validator using the withValidator command. This simply takes a validator function.
  • Next we are validating the items in the collection with the eachItemWith command. This also takes a validator function which, in this case, we are creating in line but we could take the same approach as with the customer – declare it separately and simply pass it in here.
  • Finally we are validating the product properties using multi-part property paths e.g. i.product.price. These functions literally are getter functions for our properties – the expressions are evaluated when the property runs and the return type informs what validators can be used.

First lets run this on a valid order:

{
    customer = { name = "Alice" ; dateOfBirth = DateTime.UtcNow.AddYears(-1) }
    items = [
        {
            quantity = 1
            product = { name = "Baked Beans" ; price = 3.50 }
        }
    ]
}
|> orderValidator
|> outputToConsole

This produces the output:

No errors

However lets now give the customer an empty name and set the price of a product to 0:

{
    customer = { name = "" ; dateOfBirth = DateTime.UtcNow.AddYears(-1) }
    items = [
        {
            quantity = 1
            product = { name = "Baked Beans" ; price = 0.0 }
        }
    ]
}
|> orderValidator
|> outputToConsole

When we run this we get the following (I’ve tidied up the standard output slightly for clarity):

[
    {
        message = "Must be greater than 0";
        property = "items.[0].product.price";
        errorCode = "isGreaterThan";
    };
    {
        message = "Must not be empty";
        property = "customer.name";
        errorCode = "isNotEmpty";
    }
]

We can see in the results that the expected validations have failed and that the paths to those properties identify the source of the error including the item index in the collection.

Custom Validators

Lets extend our customer validator to ensure the customer placing the order is over 18. We’ve stored their date of birth and so will need to calculate their age as part of the validation. We can do that by adding a custom validator.

Validators have a signature of string -> ‘propertyType -> ValidationState. The first parameter is the path to the property being validated, the second is the value being validated and the validator must return a ValidationState which is a discriminated union with cases of Ok and Errors.

That being the case our 18 or over validator will look like this:

let is18OrOlder propertyPath (value:DateTime) =
    let today = DateTime.Today
    let age = today.Year - value.Year - (match value.Date > today.Date with | true -> 1 | false -> 0)  
    match age >= 18 with
    | true -> Ok
    | false -> Errors([{ errorCode = "is18OrOlder" ; message="Not 18" ; property = propertyPath }])

And we can use it like any other validator:

let customerValidator = createValidatorFor<Customer>() {
    validate (fun c -> c.name) [
        isNotEmpty
        hasMaxLengthOf 128
    ]
    validate (fun c -> c.dateOfBirth) [
        is18OrOlder
    ]
}

You’ll notice that some validators take parameters – how do we do this? To accomplish that we write a function that accepts our validation parameters and returns the validator function. Lets change our validator to check if someone is over a parameterised age:

let isOverAge age =
    let validator propertyPath (value:DateTime) =
        let today = DateTime.Today
        let age = today.Year - value.Year - (match value.Date > today.Date with | true -> 1 | false -> 0)  
        match age >= age with
        | true -> Ok
        | false -> Errors([{ errorCode = "is18OrOlder" ; message="Not 18" ; property = propertyPath }])
    validator
        
let customerValidator = createValidatorFor<Customer>() {
    validate (fun c -> c.name) [
        isNotEmpty
        hasMaxLengthOf 128
    ]
    validate (fun c -> c.dateOfBirth) [
        isOverAge 18
    ]
}

Conditional Validation

Let’s now expand on our age related validation and rather than say all customers must be 18 or over let’s change the validation to be customers 18 or over cannot buy products with a name of Alcohol. We can use the validateWhen command to introduce this additional logic:

let calculateAge (dateOfBirth:DateTime) =
    let today = DateTime.Today
    let age = today.Year - dateOfBirth.Year - (match dateOfBirth.Date > today.Date with | true -> 1 | false -> 0)
    age
let orderValidator = createValidatorFor<Order>() {
    validate (fun o -> o.customer) [
        withValidator (createValidatorFor<Customer>() {
            validate (fun c -> c.name) [
                isNotEmpty
                hasMaxLengthOf 128
            ]
        })
    ]
    validate (fun o -> o.items) [
        isNotEmpty
        eachItemWith (createValidatorFor<OrderItem>() {
            validate (fun i -> i.quantity) [
                isGreaterThan 0
            ]
            validate (fun i -> i.product.price) [
                isGreaterThan 0.
            ]
            validate (fun i -> i.product.name) [
                isNotEmpty
                hasMaxLengthOf 128
            ]
        })
    ]
    validateWhen (fun o -> (o.customer.dateOfBirth |> calculateAge) < 18) (fun o -> o.items) [
        eachItemWith (createValidatorFor<OrderItem>() {                
            validate (fun i -> i.product.name) [
                isNotEqualTo "Alcohol"
            ]
        })
    ]
}

In this example our existing validations for the collection items will all run but if the customer is under 18 then we will run an additional block of validations. We can see this by running an order through the validator that includes a quantity of an item set to 0 and a product named Alcohol (for a customer younger than 18!):

{
    customer = { name = "Alice" ; dateOfBirth = DateTime.UtcNow.AddYears(-1) }
    items = [
        {
            quantity = 1
            product = { name = "Baked Beans" ; price = 3.50 }
        }
        {
            quantity = 0
            product = { name = "Alcohol" ; price = 10.50 }
        }
    ]
}
|> orderValidator
|> outputToConsole

This produces the following output:

[
    {
        message = "Must not be equal to Alcohol";
        property = "items.[1].product.name";
        errorCode = "isNotEqualTo";
    };
    {
        message = "Must be greater than 0";
        property = "items.[1].quantity";
        errorCode = "isGreaterThan";
    }
]

Correctly running the conditional and non-conditional validations for our younger customer.

Other Features and Finishing Off

The framework includes other features such as support for discriminated unions, lambda validators and additional conditional support which is all documented on the frameworks homepage.

This really was just something that “sprang out” and I’d love to hear your feedback over on Twitter.

Posted in F#

F# from a C# Developers Perspective – Part 4

Below is a random collection of things that have tickled me and/or I’ve greatly appreciated as I’ve started to write more code in F#.

Exceptions

I love how concise it is to declare and use an F# exception:

exception OoopsButterFingers

let catchTheBall () =
    raise OoopsButterFingers

Or to attach some data:

exception GetOffMyLawn of string

let strayOntoLawn () =
    raise (GetOffMyLawn "Oi!")

Single Case Discriminated Unions

Leaving the question of mutability aside I’m sure we’ve all got examples of mis-assigning IDs in models:

class User
{
    public string UserId { get; set; }
}

class Order
{
    public string OrderedByUserId { get; set; }
    public string OrderId { get; set; }
}

User user = getUser();
Order newOrder = new Order {
    OrderedByUserId = "newid",
    OrderId = user.UserId // oops, we've mis-assigned an ID 
};

It’s easily done right? And we could certainly improve that in C# but I love how simple this is in F#:

type UserId = | UserId of string

type OrderId = | OrderId of string

type User =
    {
        userId: UserId
    }
    
type Order =
    {
        orderedByUserId: UserId
        orderId: OrderId
    }
    
let user = { userId = UserId("abc") }

// This won't compile - both assignments will fail with type mismatches 
let newOrder = {
    orderedByUserId = "newId"
    orderId = user.userId
}

Currying is not named after a vindaloo

While explaining the concept of currying to a friend he asked the obvious question “why the heck is it called currying?”. I had no idea and briefly entertained the notion that it had been named in jest after, well, a curry. Which gave me a good chuckle.

However it turns out its named after an American mathematician and logician, one Haskell Curry. And so at the same time I also learned why the language Haskell is called Haskell. One of those things you feel somewhat of an ignorant philistine over not knowing.

And with that I’ll leave you with a photo of a delicious curry.

Photo via Unsplash – unsplash-logoSteven Bennett

Posted in F#

F# from a C# Developers Perspective – Part 3

I’ve been writing a lot of (production) F# since my last post on this topic and while I don’t consider myself an expert (by any means!) I’ve certainly become a lot more familiar with the language and have found myself falling into what Isaac Abraham refers to as “the pit of success”.

It was hard to understand, or at least appreciate, what he meant by this on my first pass through his book (which I recommend by the way – that really helped get me on my way) and I think that’s because F# doesn’t lead you towards success (i.e. bug free code) through any one language feature – but rather how all its features interlock and go together.

It really came together for me when I undertook a significant refactoring of some code and when I had finished and pressed build and it compiled without errors and warnings I realised that I knew what I’d done worked. I didn’t have to test it or try it. I knew it had to work (and there’s no twist in this tale – it did).

Amongst other things I had no mutable state, I’d used single case case discriminated unions to protect against mis-assignments, I’d used the type system to protect against bad data, I’d used the Seq functions almost exclusively to work on collections, I’d favoured matching over if/then/else constructs and had no missing matches, I’d used modules and namespaces to organise my code, and I had no nulls!

In fact as this application comes together, its a mobile app that I am building with the Fabulous framework – and the Model->View->Update architecture is perfect for a functional language, the only thing I really find myself worrying about is does the UI layout correctly.

It’s been a fascinating, and very positive, experience for someone who has been working with C# and JavaScript for so many years. I’d sum it up as this:

F# makes me work a little harder to express concepts in code but once I have I am more confident the result is correct and far less brittle and as a result I am more productive.

It’s not that C# “forces” you to write brittle code – but it makes it far easier to and so in some ways, putting the paradigms to one side and focusing on the un-brittleness of code (and as someone who has used all three languages), I am starting to think of F# being to C# as C# is to C++.

Posted in F#

Writing and Testing Azure Functions with Function Monkey – Part 4

Part 4 of my series on writing Azure Functions with Function Monkey is now available on YouTube:

This part focuses on addressing cross cutting concerns in a DRY manner by implementing a custom command dispatcher.

I’ve also switched over to Rider as my main IDE now and in this video I’m making use of its Presentation Mode. I think it works really well but let me know.

F# from a C# Developers Perspective – Part 2

My customary warning on these posts: I am absolutely not a F# expert – these posts are about things that caught me out or surprised me as I begin to learn F# as an experienced C# developer. You can find part 1 here.

This is something that’s caught me out a lot while picking up F#. A typical function in F# might look like this:

let multiply valueOne =
	let userInput = int(System.Console.ReadLine())
	valueOne * userInput

Now how about a parameterless function, is it this:

let multiply =
	let valueOne = int(System.Console.ReadLine())
	let valueTwo = int(System.Console.ReadLine())
	valueOne * valueTwo

In short: no. This is not a function – the code will execute once and the result be assigned to the immutable constant multiply. To make this a function you need to add parentheses:

let multiply () =
	let valueOne = int(System.Console.ReadLine())
	let valueTwo = int(System.Console.ReadLine())
	valueOne * valueTwo

As both are perfectly valid syntax the compiler will process them without warning, it has no way of divining your intent. In this case you’ll probably notice the issue because of the need for user input but I had a really puzzling situation with the setting of the consoles foreground colour which never seemed to quite do what I expected. Needless to say – I’d made this mistake!

Posted in F#

F# from a C# Developers Perspective – Part 1

Note: for those waiting for me to post more videos about Function Monkey they are on the way, I’m on a working / cycling holiday in Mallorca and don’t really have the setup to record decent audio.

While I’m still a complete beginner with F# and very much in the heavy learning phase I thought it might be interesting to write about some of the things I’ve learned along the way.

I can’t stress enough that I’m a beginner with this stuff – so hopefully while its interesting please don’t treat the examples shown as great usage of F#, rather observations made on my way to gaining a better understanding.

A good place to start might be with writing the F# equivalent of this simple and fairly typical C# construct:

internal class Program
{
    private static int ProcessInput(string input, int state)
    {
        int newState = state;
        switch (input)
        {
            case "A": newState = state + 1;
                break;
            case "M": newState = state * 2;
                break;
        }

        Console.WriteLine(newState);
        return newState;
    }
    
    public static void Main(string[] args)
    {
        bool shouldQuit = false;
        int state = 0;
        
        while (!shouldQuit)
        {
            string input = Console.ReadLine().ToUpper();
            shouldQuit = input == "Q";
            if (!shouldQuit)
            {
                state = ProcessInput(input, state);
            }
        }
    }
}

Essentially keep looping and processing user input until the user signals they’re done. My first attempt at this looked very much like the C# equivalent:

open System

[<EntryPoint>]
let main argv =
    let processInputs value input =
        let newValue = match input with
            | "A" -> value + 1
            | "M" -> value * 2
            | _ -> value
        printf "%d\n" newValue
        newValue
        
    let mutable shouldQuit = false
    let mutable state = 0
    while not shouldQuit do
        let input = System.Console.ReadLine().ToUpper()
        shouldQuit <- input = "Q"
        state <- match shouldQuit with
            | true -> state
            | false -> processInputs state input
            
    0 // return an integer exit code

As you might know, in F# data is immutable by default and use of the mutable keyword is a smell – it may sometimes be necessary but its a good sign you’re going about something in a none F# way.

One way to solve this would be to replace the while loop with a recursive function however in this case the recursion is essentially unbounded as it depends on user input. To a C# mindset that is something likely to blow the stack with a good old StackOverflowException. Here’s an example of a recursive approach written in C# and the stack trace that results in the debugger after a few key presses:

class Program
{
    private static int ProcessInput(string input, int state)
    {
        int newState = state;
        switch (input)
        {
            case "A": newState = state + 1;
                break;
            case "M": newState = state * 2;
                break;
        }

        Console.WriteLine(newState);
        return newState;
    }

    private static int RecursiveLoop(int state)
    {
        string input = Console.ReadLine()?.ToUpper();
        if (input == "Q")
        {
            return state;
        }

        return RecursiveLoop(ProcessInput(input, state));
    }
    
    static int Main(string[] args)
    {
        return RecursiveLoop(0);
    }
}

Here’s the same code written in F#, with the mutable keyword eradicated, and the stack trace after a few key presses:

open System

[<EntryPoint>]
let main _ =
    let processInput value input =
        let newValue = match input with
            | "A" -> value + 1
            | "M" -> value * 2
            | _ -> value
        printf "%d\n" newValue
        newValue
        
    let rec inputTailLoop state =
        let input = System.Console.ReadLine().ToUpper()
        match input with
            | "Q" -> state
            | _ -> inputTailLoop (processInput state input)
            
    let startingState = 0
    let finishingState = inputTailLoop startingState
    finishingState

The stack has not grown! So what’s going on?

The recursive functions, in both C# and F#, are examples of something called tail recursion. Tail recursion occurs when the very last operation a function performs is to call itself and when this occurs their is no need to preserve any data on the stack.

The F# compiler, unlike the C# compiler, recognises this and essentially is able to treat the recursive method as an iteration and avoid the stack growth and by doing so it enables us to solve a whole new class of problems without requiring mutable data.

Having posted about this on Twitter this prompted some discussion amongst the F# wizards about how tail recursion is also generally considered an anti-pattern and Isaac Abraham came through with an example as to how this could be solved using F# sequences as shown below (my butchered version of his original!):

open System

[<EntryPoint>]
let main _ =
    let processInput value input =
        let newValue = match input with
            | "A" -> value + 1
            | "M" -> value * 2
            | _ -> value
        printf "%d\n" newValue
        newValue
    
    let inputs = seq {
        while true do
            yield System.Console.ReadLine()
    }
    let startingState = 0
    let finishingState =
        inputs
        |> Seq.map (fun input -> input.ToUpper())
        |> Seq.takeWhile (fun input -> not(input = "Q"))
        |> Seq.fold processInput startingState
    
    finishingState

Essentially what we’ve done here is be explicit about the iteration that was implicit in the tail recursion. I’d used the Seq namespace quite heavily but I’m not sure I’d have made the leap to using the seq { } construct to solve this problem without Isaac’s help (thank you!).

For anyone interested I’m writing a simple version of the classic Star Trek game to teach myself F#. The code is, I’m sure, woeful and it gets refactored massively almost daily at the moment – but you’re welcome to take a look, it’s on GitHub.

Finally I’d like to stress again – I’m an absolute beginner at F# so please, please, don’t use the code above as exemplars but rather curios!

Posted in F#

App Center and React Native Upgrades

I hit a very strange problem recently with Microsoft App Centre which I’ve been happily using to build and distribute a React Native app that was sat at version 0.55.

React Native 0.55 -> 0.56 was quite a big change as it adopted the new Xcode build system and bumped the minimum node version.

I needed to update the app to be compatible with Apple’s requirements and so spent some time moving it along to React Native 0.59. All seemed to be going fine and I was able to run a build through App Centre from my development / feature branch.

I merged this into master, did a diff to ensure my feature and master branches were identical, and pushed it to App Centre. And the build assigned to this branch failed – for some reason it wasn’t selecting the correct node version and I saw this error:

error react-native@0.59.3: The engine "node" is incompatible with this module. Expected version ">=8.3". Got "6.17.0"

The build definitions were identical and the source code was identical, I checked the App Centre agent version and it was the same too. I spent some time with a support team member who was helpful but ultimately as confused as me and attempted to force the node version selection with a post clone script. That didn’t work either but gave me a different error. An error that suggested that the build was now using node 8.

I scratched my head for a while and realised what I’d done. I’d opened the build definition and pressed save after adding the post clone script. You need to do this to get App Centre to see new and updated custom build scripts.

Of course it then dawned on me – App Centre isn’t figuring out the node version to use after cloning – its doing it at the same time it looks for custom build scripts. When you press save on a build definition. And I’d almost certainly inspected the build definition (looking for things I might need to change) on the first branch I tried and hit save.

I removed the post clone script and everything worked as expected.

Well perhaps not as expected – this really isn’t helpful behaviour from App Centre, confusing to both users and support staff, that hopefully they will resolve. You really expect your project to be built based on its assets at the time of build – not part from this and part from what is effectively the result of inspecting an earlier snapshot.

Function Monkey 2.1.0

I’ve just pushed out a new version of Function Monkey with one fairly minor but potentially important change – the ability to create functions without routes.

You can now use the .HttpRoute() method without specifying an actual route. If you then also specify no path on the .HttpFunction<TCommand>() method that will result in an Azure Function with no route specified – it will then be named in the usual way based on the function name, which in the case of Function Monkey is the command name.

I’m not entirely comfortable with the approach I’ve taken to this at an API level but didn’t want to break anything – next time I plan a set of breaking changes I’ll probably look to clean this up a bit.

The reason for this is to support Logic Apps. Logic Apps only support routes with an accompanying Swagger / OpenAPI doc and you don’t necessarily want the latter for your functions.

While I was using proxies HTTP functions had no route and so they could be called from Logic Apps using the underlying function (while the outside world would use the shaped endpoint exposed through the proxy).

Having moved to a proxy-less world I’d managed to break a production Logic App of my own because the Logic App couldn’t find the function (404 error). Redeployment then generated a more meaningful error – that routed functions aren’t supported. Jeff Hollan gives some background on why here.

I had planned a bunch of improvements for 2.1.0 (which I’ve started) which will now move to 2.2.0.

Writing and Testing Azure Functions with Function Monkey – Part 3

Part 3 of my series on writing Azure Functions with Function Monkey focuses on writing tests using the newly released testing package – while this is by no means required it does make writing high value acceptance tests that use your applications full runtime easy and quick.

Lessons Learned

It really is amazing how quickly time passes when you’re talking and coding – I really hadn’t realised I’d recorded over an hours footage until I came to edit the video. I thought about splitting it in two but the contents really belonged together so I’ve left it as is.

Contact

  • If you're looking for help with C#, .NET, Azure, Architecture, or would simply value an independent opinion then please get in touch here or over on Twitter.

Recent Posts

Recent Tweets

Recent Comments

Archives

Categories

Meta

GiottoPress by Enrique Chavez