Category: .NET Core

Creating FableTrek – Part 1

If you want to follow along with this series of posts the source code and be found here, the published game here and the source for this specific post here.

If you read my blog or follow me on Twitter you’ll know I’ve really fallen in love with F# this last year and am using the SAFE stack to build out my cycling performance application. A big part of that toolkit is Fable – the F# to JavaScript transpiler that also comes with some great bindings and tools for building single page applications.

A few people have asked me how to get started with Fable, why I think F# is so well suited for this kind of work, and how to make the transition from C# and so I thought I’d build out a simple game using Fable and talk about it in a series of posts / videos.

Now before anyone gets too excited this isn’t going to be the next Halo! Instead I’m going to rework my iOS version of the classic Star Trek game (PaddTrek) into Fable – now called FableTrek!

PaddTrek – an early iPad game

Its not going to be flashy but hopefully strikes the right balance between complex enough to show real problems and solutions and simple enough to understand. I don’t have a grand up-front plan for building out this game and I’m creating each part as I go – lets just hope it doesn’t turn out like the new Star Wars trilogy!

I’ll also add that I still class myself as a relative beginner with F# but I do manage to get stuff done.

With all that out of the way – lets take a look at the first code drop.

Exploring the code

To get started I used this bare bones starting point for a simple Fable app with webpack and debug support for Visual Studio Code (hat tip to Isaac Abraham who created this for me to help me out a little while back).

I’ve then added a few things to it:

One of the first things to wrap your head round with Fable is that it lives in both .NET and JavaScript worlds. And so in the root folder you’ll find a package.json and a webpack.config.js and in the src folder a F# project file – App.fsproj.

When adding dependencies you’ll often find yourself adding both a NuGet package and an npm package. I’m using the dotnet command line tools to manage the former and yarn for the latter.

The Fable compiler itself is generally invoked through webpack using the fable-loader plugin. I’ve got a fairly simple webpack file – I’ve added support for Stylus and a little work to output a production build to a deploy folder.

App.fs and Index.html – entry point

Starting up a React app in F# is not much different to doing the same with JavaScript or TypeScript. First you need a HTML file:

<!doctype html>
<html>
<head>
  <title>Fable</title>
  <meta http-equiv='Content-Type' content='text/html; charset=utf-8'>
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <link rel="shortcut icon" href="fable.ico" />
</head>
<body>
    <div id="trek-app"></div>
</body>
</html>

We’re going to create the React app inside the trek-app div block. You might notice their is no <script> tag – so how does any code get invoked? I’ve set up webpack to inject the <script> tag during the build process using the HtmlWebpackPlugin plugin – this way we can handle unique script filenames that include a hash component.

The code entry point for our app is App.fs in the source folder. At the top of the file first we import some dependencies:

open Elmish
open Fable.React
open App.Types
open App.State
open Fable.Core.JsInterop

importSideEffects "./game.styl"

The interesting line here is the final one where we import game.styl. This is our Stylus CSS file. Importing it like this will ensure webpack will compile it to CSS and include it in our final bundle – to do this I’ve used the stylus-loader package for webpack.

If we skip to the end of the file we can see a block of code that starts up a Fable Elmish based React app:

Program.mkProgram init update root
|> Program.toNavigable (parseHash Router.pageParser) urlUpdate
#if DEBUG
|> Program.withConsoleTrace
|> Program.withDebugger
#endif
|> Program.withReactBatched "trek-app"
|> Program.run

We’re passing in a whole load of stuff to this startup block – init, update, root, pageParser and urlUpdate are the basic building blocks of our Elmish app. And so I guess its time I answered the question: what is Elmish?

Elmish…

… is a set of abstractions that allow us to implement the Model View Update pattern in F# based on Elm. Essentially its a pattern where a user interface (a view) is created from immutable state and from which messages are passed back to our program to update the state (via a transformation – its immutable) and regenerate the view. To make this work their are a number of concepts which I’ve started to flesh out in our game.

Thomas Bandt has written a blog post around a simple counter example that introduces the same concepts that is worth reading before continuing. I’m going to go through the same concepts but using the game we’re creating which is a little more complex as we have more data to model and in our game I’m subdividing the system into smaller sub-applications.

Model

The model contains our application state and the best way to express this in F# is using a record. You can see our applications top level model in our App.Types.fs file:

type Model =
  {
    CurrentPage: Router.Page
    StartScreen: Interface.StartScreen.Types.Model option
  }
  static member Empty =
    { CurrentPage = StartScreenPage
      StartScreen = Some Interface.StartScreen.Types.Model.Empty
    }

Our root model keeps track of the current page we’re looking at (based on a URL – we’ll look at the router later) and the state for our sub-applications of which we currently only have one – the start screen. These are optional pieces of state that we create as required as the URL changes. I’ve added a static member that returns an empty model that we can use when we need to initiate a default model.

Being records the state is immutable and we’ll look at how we manage updates to this a little later.

The StartScreen model is currently just a placeholder which we’ll flesh out in later parts.

Messages

Messages are what we use to glue our system together. They’re typically sent by the user interacting with our views through a dispatcher and are handled by update functions that we’ll look at in a moment. Each of our sub-applications will have its own set of messages only visible within its own scope – this makes for a great way to isolate the moving parts of a system and is particularly useful in large systems like my cycling analytic website. It means I can change one sub-application without worrying about breaking others.

In our game you can find the messages for the start screen sub-application in Interface/StartScreen/Types.fs:

type StartScreenMsg =
  | NewGame

When we flesh out our start screen this message will be dispatched when the user clicks a new game button.

Discriminated unions are almost tailor made for messages. If you come from a C# background you might think they look like an enumerated type but they are much more! For one thing they can be associated with payloads and we already have an example of this – we use a top level set of messages that themselves encapsulate our sub-application messages to enable our program to route messages to the correct sub-application. You can see an example of this in App.Types.fs:

type Msg =
  | GameScreenDispatcherMsg
  | StartScreenDispatcherMsg of Interface.StartScreen.Types.StartScreenMsg

Updates

Update functions are how we respond to messages and update our state however they never modify state. Instead given the current state and a message they return new state and a command (often another message).

We don’t yet have a simple example of this but we do have an example of an update function coordinating between our sub-applications which you can find in App.State.fs:

let update msg model =
  match (msg, model) with
  | (StartScreenDispatcherMsg subMsg, { StartScreen = Some extractedModel }) ->
    let (subModel, subCmd) = Interface.StartScreen.State.update subMsg extractedModel
    { model with StartScreen = Some subModel}, Cmd.map StartScreenDispatcherMsg subCmd
  | _ ->
    console.error("Missing match in App.State")
    model, Cmd.none

If you’re not familiar with F# you might currently be asking what is this crazy wizardry. I’ll try and break it down! F# has great support for pattern matching – we begin a match set using the match with construct. In our case we’re saying match the msg and model together (what we’re actually doing is creating a tuple out of the message and the model).

Each pattern for consideration, they are evaluated in order, is started with the pipe | operator and in our case we have two possible matches.

Our first pattern match we look for a message of type StartScreenDispatcherMsg and a model that has the StartScreen attribute set to Some. While matching we can also extract values – in our case we’re extracting the sub message for the start screen and the model we have assigned to StartScreen. If we find these things then we pass them on to to our sub-applications updater – it also returns a model and a command. Finally we return a model with the new submodel returned by our child updater and whatever command it returned.

Our second match (the one that reads | _) is our fall through match. This will match anything and simply return the model and the command but pop an error into our browsers console. I’ve got no implementation at all yet for the GameScreenDispatcherMsg.

Routing

I briefly mentioned routing earlier when we looked at models – routing being the process of displaying the correct content within a SPA for a given URL. Elmish comes with some support for routing which I’ve used to build the router here which you can find in Router.fs. Firstly we use a disciminated union to define our routes:

type Page =
  | StartScreenPage
  | GameScreenPage

If we had sub-routes we could use the payload of a discriminated union to nest them but we don’t (at least not yet) so for now that’s it. Next we need a way to turn out union members into URL paths:

let private toHash page =
  match page with
  | StartScreenPage -> "#/"
  | GameScreenPage -> "#/game"

Here I’m using hash based paths but you can also use full paths. Finally we need a parser – a function that can extract values from routes and return a correctly structured Page discriminated union. We don’t have any parameters for the routes yet so this is quite simple:

let pageParser =
  oneOf [
    map GameScreenPage (s "game")
    map StartScreenPage top
  ]

Finally when a URL changes we need a way of updating the current page and state in our model. In App.State.fs we have a urlUpdate function:

let urlUpdate (result: Option<Router.Page>) model =
  match result with
  | None ->
    console.error("Error parsing url: " + window.location.href)
    model, Router.modifyUrl model.CurrentPage
  | Some page ->
    let model = { model with CurrentPage = page}
    match page with
    | StartScreenPage ->
      let (subModel, subCmd) = Interface.StartScreen.State.init () 
      { model with StartScreen = Some subModel }, Cmd.map StartScreenDispatcherMsg subCmd
    | _ -> model, Cmd.none

This is very similar in form to a regular update function except rather than take a message in takes our page discriminated union. Like the regular update it returns an updated model and command.

Views

Ok. With all that plumbing in place we need a way of transforming our state into user interface and we do that with views. Our root view can be found in App.fs:

let root model dispatch =
  div [Class "container"] [
    match model with
    | { CurrentPage = Router.Page.StartScreenPage ; StartScreen = Some extractedModel } ->
      Interface.StartScreen.View.root model dispatch
    | _ -> div [] [str "not found"]
  ]

Like much of our code so far this is somewhat infrastructural – it looks at our model, determines the current sub-application based on the route, and delegates its handling down to that (which you can find in Interface/StartScreen/View.fs):

let root model dispatch =
  div [Class "startScreen"] [
    div [Class "title"] [str "< work in progress >"]
  ]

If you come from a typical React background you’re probably used to using JSX – a DSL that is extremely similar to HTML. In F# we use a DSL that is, well, F#. Its still very similar in use but being F# is typesafe and because its F# you don’t have those slightly awkward breaks between JSX and JavaScript. We’ll see later on how easy it is to use with FP techniques to create complex interfaces cleanly.

Styling with Stylus

I’m building the CSS by hand for this project and to help me do that I’m using a preprocessor called Stylus. I like it because it removes a lot of the noise and clutter from a CSS file. You can find the, fairly limited, amounts of CSS in our game.styl file.

For the time being I’m referring to class names by strings but I may do something about this as we progress 🙂

Putting it all together

If we wind back to where we started and look at the block of code at the bottom of App.fs:

Program.mkProgram init update root
|> Program.toNavigable (parseHash Router.pageParser) urlUpdate
#if DEBUG
|> Program.withConsoleTrace
|> Program.withDebugger
#endif
|> Program.withReactBatched "trek-app"
|> Program.run

You can see how all these bits we’ve looked at are brought together in the first two lines of this to create our Elmish program that can now parse URLs, update models, and translate models into views.

Deploying to Azure Static Web Hosting with GitHub Actions

Right back at the start of this piece I shared a link to the game running live on the web – you can find it at https://www.fabletrek.com. Its not all that exciting at the moment:

But it is running – and its deploying automatically and directly of the GitHub repository using GitHub Actions and Azure Static Web Apps. This solution doesn’t come with automatic support for Fable but its pretty straightforward to add.

Start by creating the Azure Static Web App itself in Azure following these instructions. For the app artifact directory enter deploy but otherwise follow those instructions.

This will create a GitHub Action for you in the repository but if you run it you’ll get an error stating that the job they provide doesn’t know how to build out code. Their is support for using a custom build command (and we build our solution with the command yarn webpack) but I found using that gives an error with locating and running dotnet from within the Fable compiler webpack loader.

Instead what we need to do is move the build outside of their job so all we need to then do is pick up the contents of our deploy folder. To do that we add these steps:

      - name: Setup .NET Core
        uses: actions/setup-dotnet@v1
        with:
          dotnet-version: 3.1.402
      - name: Install .NET dependencies
        working-directory: "src"
        run: dotnet restore
      - name: Install JS dependencies
        run: yarn install
      - name: Build
        run: yarn webpack

This makes sure the correct version of .NET Core is installed, restores our NuGet and NPM packages, and then finally builds the solution.

We then need to modify the provided Azure Static Web App job to prevent it from trying to build the solution again. Their is no way to disable the build process but what you can do is supply it with a harmless build command – rather than build anything I simply run yarn –version:

         with:
          azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN_UNIQUE_FOR_YOU }}
          repo_token: ${{ secrets.GITHUB_TOKEN }} # Used for Github integrations (i.e. PR comments)
          action: "upload"
          ###### Repository/Build Configurations - These values can be configured to match you app requirements. ######
          # For more information regarding Static Web App workflow configurations, please visit: https://aka.ms/swaworkflowconfig
          app_location: "/" # App source code path
          api_location: "api" # Api source code path - optional
          app_artifact_location: "deploy" # Built app content directory - optional
          # we need a separate build step as the fable-compiler fails when run under the context of the static site builder
          # and their is no disable build property that I am aware of so we run a harmless command
          app_build_command: "yarn --version"
          ###### End of Repository/Build Configurations ######

With those changes made every time you commit code to your branch it will be built and deployed. You can find the complete action here. I then added a custom domain which gives you a free SSL cert too.

Next Steps

That’s the setup part done really – if you’re new to Fable, F# and Elmish then this I think is the more complex bit. In the next stage we’ll build out a start screen and get to the point where we can start a new game.

Azure SQL Database deployment with Farmer, DbUp and GitHub Actions

Farmer is a DSL for generating and executing ARM templates and one of the great things about it is that its based on .NET Core. That means that you can use it in combination with other components from the .NET ecosystem to create end to end typesafe deployment solutions.

As an aside – I recently posted a critique of Microsofts new DSL Bicep. One of the things I didn’t mention in that but did in a series of tweets was the shortcomings of inventing a new language that lives in its own ecosystem.

Ultimately Bicep will need to support “extension points” or you’ll have to wrap them in script and communicate information across boundaries (of course their can be benefits to that approach too). Not to mention they need to write all the tooling from scratch and developers / administrators need to learn another language.

By taking the approach Farmer has handling boundaries is a lot cleaner – as we’ll see – and we can take advantage of some neat language features.

In this example I’m going to provision an Azure SQL Database into Azure and then upgrade its schema using DbUp and we’ll run all this through GitHub Actions giving us an automated end to end deployment / upgrade system for our SQL database. You could do this with less F# code (almost none) but I also want to try and illustrate how this approach can form a nice framework for more complicated deployment scenarios so we’re also going to look at error handling across a deployment pipeline.

As all the components themselves are well documented I’m not going to go end to end on all the detail of each component here – instead I’m going to focus on the big picture and the glue. You can find the code for the finished demonstration on GitHub here.

Starting with a F# console app, adding the Farmer NuGet package, and the boilerplate Program.fs file first we need to declare our Azure resources – in this case a SQL Server and a database and then bring them together in an ARM template:

let demoDatabase = sqlServer {
    name serverName
    admin_username "demoAdmin"
    enable_azure_firewall
    
    add_databases [
        sqlDb { name databaseName ; sku DbSku.Basic }
    ]
}

let template = arm {
    location Location.UKWest
    add_resource demoDatabase
    output "connection-string" (demoDatabase.ConnectionString databaseName)
}

Pretty straightforward but a couple of things worth noting:

  1. Both serverName and databaseName are simple constants (e.g. let databaseName = “myDatabaseName”) that I’ve created as I’m going to use them a couple of times.
  2. Opening up the database to azure services (enable_azure_firewall) will allow the GitHub Actions Runner to access the database.
  3. On the final line of our arm block we output the connection string for the database so we can use it later.

That’s our Azure resources but how do we apply our SQL scripts to generate our schema? First we’ll need to add the dbup-sqlserver NuGet package and with that in place we’ll first add a Scripts folder to our solution and in my example four scripts:

DbUp keeps track of the last script it ran and applies subsequent scripts – essentially its a forward only ladder of migrations. If you’re adding scripts of your own make sure you mark them as Embedded Resource otherwise DbUp won’t find them. To apply the scripts we simply need some fairly standard DbUp code like that shown below, I’ve placed this in a F# module called DbUpgrade so, as we’ll see in a minute, we can pipe to it quite elegantly:

let tryExecute =
  Result.bind (fun (outputs:Map<string,string>) ->
    try
      let connectionString = outputs.["connection-string"]
      let result =
        DeployChanges
          .To
          .SqlDatabase(connectionString)
          .WithScriptsEmbeddedInAssembly(Assembly.GetExecutingAssembly())
          .LogToConsole()
          .Build()
          .PerformUpgrade()
      match result.Successful with
      | true -> Ok outputs
      | false -> Error (sprintf "%s: %s" (result.Error.GetType().Name.ToUpper()) result.Error.Message)
    with _ -> Error "Unexpected error occurred upgrading database"
  )

If you’re not familiar with F# you might wonder what this Result.bind function is. F# has a wrapper type for handling success and error states called options and a bunch of helper functions for their use. One of the neat things about it is it lets you chain lots of functions together with an elegant pattern for handling failure – this is often referred to as Railway Oriented Programming.

We’ve now declared our Azure resources and we’ve got a process for deploying our upgrade scripts and we need to bring it all together and actually execute all this. First lets create our deployment pipeline that first provisions the resources and then upgrades the database:

let deploymentPipeline =
  Deploy.tryExecute "demoResourceGroup" [ adminPasswordParameter ]
  >> DbUpgrade.tryExecute 

If we had additional tasks to run in our pipeline we’d join them together with the >> operator as I’ve done here.

To run the deployment we need to provide an admin passford for SQL server which you can see in this code snippet as sqlServerPasswordParameter and we need to do this securely – so it can’t sit in the source code. Instead as I’m going to be running this from GitHub Actions an obvious place is the Secrets area of GitHub and an easy way to make that available to our deployment console app is through an environment variable in the appropriate action (which we’ll look at later). We can then access this and format it for use with Farmber by adding this line:

let adminPasswordParameter =
  Environment.GetEnvironmentVariable("ADMIN_PASSWORD") |> createSqlServerPasswordParameter serverName

Farmer uses a convention approach to a parameter name – I’ve built a little helper function createSqlServerPassword to form that up.

(We could take a number of different approaches to this – ARM parameters for example – I’ve just picked a simple mechanism for this demo)

Finally to invoke all this we add this line at the bottom of our file:

template |> deploymentPipeline |> asGitHubAction

asGitHubAction is another little helper I’ve created that simply returns a 0 on success or prints a message to the console and returns a 1 in the event of an error. This will cause the GitHub Action to fail as we want.

That’s the code side of things done. Our finished Program.cs looks like this:

open System
open Farmer
open Farmer.Builders
open Sql
open Constants
open Helpers

[<EntryPoint>]
let main _ =
  let adminPasswordParameter =
    Environment.GetEnvironmentVariable("ADMIN_PASSWORD") |> createSqlServerPasswordParameter serverName

  let demoDatabase = sqlServer {
    name serverName
    admin_username "demoAdmin"
    enable_azure_firewall
      
    add_databases [
      sqlDb { name databaseName ; sku DbSku.Basic }
    ]
  }

  let template = arm {
    location Location.UKWest
    add_resource demoDatabase
    output "connection-string" (demoDatabase.ConnectionString databaseName)
  }

  let deploymentPipeline =
    Deploy.tryExecute "demoResourceGroup" [ adminPasswordParameter ]
    >> DbUpgrade.tryExecute 
  
  template |> deploymentPipeline |> asGitHubAction

All we need to do now is wrap it up in a GitHub Action. I’ve based this action on the stock .NET Core build one – lets take a look at it:

name: Deploy SQL database

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
    - name: Setup .NET Core
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: 3.1.301
    - name: Install dependencies
      run: dotnet restore
    - name: Build
      run: dotnet build --configuration Release --no-restore
    - name: Login via Az module
      uses: azure/login@v1.1
      with:
        creds: ${{secrets.AZURE_CREDENTIALS}}
        enable-AzPSSession: true
    - name: Run
      env:
        ADMIN_PASSWORD: ${{ secrets.ADMIN_PASSWORD}}
      run: dotnet DeployDb/bin/Release/netcoreapp3.1/DeployDb.dll

If you’re familiar with GitHub Actions most of this should be fairly self explanatory – there’s nothing special about our deployment code, its a standard .NET Core console app so we begin by building it as we would any other (again this is one of the things I like about Farmer – its just .NET, and if you’re using .NET there’s nothing else required). However after building it we do a couple of things:

  1. To do the deployment Farmer will use the Azure CLI and so we need to login to Azure via that. We do that in the Login via Az module step which is pretty stock and documented on GitHub here. I’ve stored the secret for the service principal in the secrets area of GitHub.
  2. In the final step we run our deployment – again its just a standard console app. You an see in this step the use of the env section – we take a secret we’ve called ADMIN_PASSWORD and set it as an environment variable making it available to our console app.

And that’s it! At this point you’ve got an automated solution that will make sure your Azure SQL database infrastructure and its schema are managed get up to date. Change the configuration of your SQL database and/or add a SQL script and this will kick off and apply the changes for you. If / when you run it for the first time you should see output like this from the build section of the Action:

I think its a simple, but powerful, example of infrastructure as code and the benefits of using an existing language and ecosystem for creating DSLs – you get so much for free by doing so. And if the rest of your codebase is in .NET then with Farmer you can share code, whether that be simple constants and names or implementation, easily across your deployment and runtime environments. Thats a big win. I’m slowly adding it into my Performance for Cyclists project and this approach here is largely lifted from their.

Finally I think its worth emphasising – you don’t need to really know F# to use Farmer and you certainly don’t need to be using it elsewhere in your solution. Its a pretty simple DSL build on top of F# and a fantastic example of how good F# is as a basis for DSLs. I’ve dug a little deeper into the language here to integrate another .NET tool but if all you want to do is generate ARM templates then, as you can see from the Farmer examples on its website, you really don’t need to get into the F# side (though I do encourage you to!).

An Azure Reference Architecture

There are an awful lot of services available on Azure but I’ve noticed a pattern emerging in a lot of my work around web apps. At their core they often have a similar architecture, deployment in Azure, and process for build and release.

For context a lot my hands on work over the last 3 years has been as a freelancer developing custom systems for people or on my own side projects (most recently https://www.forcyclistsbycyclists.com). In these situations I’ve found productivity to be super important in a few key ways:

  1. There’s a lot to get done, one or two people, and not much time – so being able to crank out a lot of work quickly and to a good level of quality is key.
  2. Adaptability – if its an externally focused green field system there’s a reasonable chance that there’s a degree of uncertainty over what the right feature set is. I generally expect to have to iterate a few times.
  3. I can’t be wasting time repeating myself or undertaking lengthy manual tasks.

Due to this I generally avoid over complicating my early stage deployment with too much separation – but I *do* make sure I understand where my boundaries and apply principles that support the later distribution of a system in the code.

With that out the way… here’s an architecture I’ve used as a good starting point several times now. And while it continues to evolve and I will vary specific decisions based on need its served me well and so I thought I’d share it here.

I realise there are some elements on here that are not “the latest and greatest” however its rarely productive to be on the bleeding edge. It seems likely, for example, that I’ll adopt the Azure SPA support at some point – but there’s not much in it for me doing that now. Similarly I can imagine giving GitHub Actions ago at some point – but what do I really gain by throwing what I know away today. From the experiments I’ve run I gain no productivity. Judging this stuff is something of a fine line but at the risk of banging this drum too hard: far too many people adopt technology because they see it being pushed and talked about on Twitter or dev.to (etc.) by the vendor, by their DevRel folk and by their community (e.g. MVPs) and by those who have jumped early and are quite possibly (likely!) suffering from a bizarre mix of Stockholm Syndrome and sunk cost fallacy “honestly the grass is so much greener over here… I’m so happy I crawled through the barbed wire”.

Rant over. If you’ve got any questions, want to tell me I’m crazy or question my parentage: catch me over on Twitter.

Architecture

Build & Release

I’ve long been a fan of automating at least all the high value parts of build & release. If you’re able to get it up and running quickly it rapidly pays for itself over the lifetime of a project. And one of the benefits of not CV chasing the latest tech is that most of this stuff is movable from project to project. Once you’ve set up a pipeline for a given set of assets and components its pretty easy to use on your next project. Introduce lots of new components… yeah you’ll have lots of figuring out to do. Consistency is underrated in our industry.

So what do I use and why?

  1. Git repository – I was actually an early adopter of Git. Mostly because I was taking my personal laptop into a disconnected environment on a regular basis when it first started to emege and I’m a frequent committer.

    In this architecture it holds all the assets required to build & deploy my system other than secrets.
  2. Azure DevOps – I use the pipelines to co-ordinate build & release activities both directly using built in tasks, third party tasks and scripts. Why? At the time I started it was free and “good enough”. I’ve slowly moved over to the YAML pipelines. Slowly.
  3. My builds will output four main assets: an ARM template, Docker container, a built single page application, and SQL migration scripts. These get deployed into a an Azure resource group, Azure container registry, blob storage, and a SQL database respectively.

    My migration scripts are applied against a SQL database using DbUp and my ARM templates are generated using Farmer and then used to provision a resource group. I’m fairly new to Farmer but so far its been fantastic – previously I was using Terraform but Farmer just fit a little nicer with my workflow and I like to support the F# community.

Runtime Environment

So what do I actually use to run and host my code?

  1. App Service – I’ve nearly always got an API to host and though I will sometimes use Azure Functions for this I more often use the Web App for Containers support.

    Originally I deployed directly into a “plain” App Service but grew really tired with the ongoing “now this is really how you deploy without locked files” fiasco and the final straw was the bungled .NET Core release.

    Its just easier and more reliable to deploy a container.
  2. Azure DNS – what it says on the tin! Unless there is a good reason to run it elsewhere I prefer to keep things together, keeps things simple.
  3. Azure CDN – gets you a free SSL cert for your single page app, is fairly inexpensive, and helps with load times.
  4. SQL Database – still, I think, the most flexible general purpose and productive data solution. Sure at scale others might be better. Sure sometimes less structured data is better suited to less structured data sources. But when you’re trying to get stuff done there’s a lot to be said for having an atomic, transactional data store. And if I had a tenner for every distributed / none transactional design I’ve seen that dealt only with the happy path I would be a very very wealthy man.

    Oh and “schema-less”. In most cases the question is is the schema explicit or implicit. If its implicit… again a lot of what I’ve seen doesn’t account for much beyodn the happy path.

    SQL might not be cool, and maybe I’m boring (but I’ll take boring and gets shit done), but it goes a long way in a simple to reason about manner.
  5. Storage accounts – in many systems you come across small bits of data that are handy to dump into, say, a blob store (poor mans NoSQL right there!) or table store. I generally find myself using it at some point.
  6. Service Bus – the unsung hero of Azure in my opinion. Its reliable. Does what it says on the tin and is easy to work with. Most applications have some background activity, chatter or async events to deal with and service bus is a great way of handling this. I sometimes pair this (and Azure Functions below) with SignalR.
  7. Azure Functions – great for processing the Service Bus, running code on a schedule and generally providing glue for your system. Again I often find myself with at least a handful of these. I often also use Service Bus queues with Functions to provide a “poor mans admin console”. Basically allow me to kick off administrative events by dropping a message on a queue.
  8. Application Insights – easy way of gathering together logs, metrics, telemetry etc. If something does go wrong or your system is doing something strange the query console is a good way of exploring what the root cause might be.

Code

I’m not going to spend too long talking about how I write the system itself (plenty of that on this blog already). In generally I try and keep things loosely coupled and normally start with a modular monolith – easy to reason about, well supported by tooling, minimal complexity but can grow into something more complex when and if that’s needed.

My current tools of choice is end to end F# with Fable and Saturn / Giraffe on top of ASP.Net Core and Fable Remoting on top of all that. I hopped onto functional programming as:

  1. It seemed a better fit for building web applications and APIs.
  2. I’d grown tired with all the C# ceremony.
  3. Collectively we seem to have decided that traditional OO is not that useful – yet we’re working in languages built for that way of working. And I felt I was holding myself back / being held back.

But if you’re looking to be productive – use what you know.

Why (not) .NET?

A frequent topic of conversation on Twitter, particularly if you follow a number of Microsoft employees, is why isn’t .NET used more or talked about more. The latest round of this I saw was in connection with microserviecs – “where’s all the chat about .NET Core microservices”. There’s often a degree of surprise, frustration and curiosity expressed that people outside of the existing .NET audience aren’t looking at .NET Core – or at least seemingly not in large numbers (I’m sure someone can pull out a statistic suggseting otherwise, and someone else pull out a statistic suggesting it is the case).

One indicator of usage is the TIOBE index which is focused on langauge. VB and C# currently rank 5 and 6 in terms of popularity with their history shown below:

Based on this .NET is certainly a successful runtime and you could argue that the release of .NET Core in 2016 (announced in 2014 with a lengthy preview period) halted what looked to be a significant declince in C# usage – I would assume due to the lack of cross-platform and growing popularity of the cloud.

But since then C# has pretty much flatlined – maybe trending up slightly (the peaks and troughs seem to be going up over the last year). VB… I know far less about the VB world an what could have caused that spike. I’m assuming it has little to do with .NET Core given the limitations of running on the platform.

I find myself in an interesting position with regard to topics like this – I’ve been using .NET for a long time (since the very first beta), its the ecosystem I’m most familiar with, and in general I’m fairly happy with it. On the other hand I now work as a CTO managing teams with a variety of tech stacks (.NET, Node / React, and PHP for the most part) and when looking at new build projects wrangle with what to build it in.

And you know what? There is very little about .NET (Core or otherwise) that stands out.

That it is now cross-platform is great and opens it up to use cases and developers for which it would otherwise be a no-go (myself included these days) but that’s a catch-up play by Microsoft not a unique feature. And the majority of developers who are not on Windows have already spent significant time getting productive in other runtimes and ecosystems – to trade that away and incur a productivity debt there has to be something to offset that debt.

Now being able to write end to end browser through to backend C# (and I’ll swing back to C# later) with Blazor looks compelling but you can do this with just about any language and runtime now through the wizardry of transpilers. Sure Blazor targets WASM and that has some advantages – but also has a boat load of disadvantages: it can be a big download and is an isolated ecosystem. And it can suffer from some of the same quirks transpilers do – in my, brief, time with it I ran into over eager IL stripping issues that were obtuse to say the least. The advantage of the approaches taken by things like Fable (F#), Reason (OCaml), and even Bridge (C#) is that it lets you continue to use the ecosystem you know. For example I was easily able to move to using F# on the front end with Fable and take my React knowledge with me. I can’t do that with Blazor. Why not use Bridge? Well then you’re back to the kind of issue sometimes levelled at Fable – reliance on a small band of OSS contributors on a fringe project. To be fair to Blazor – it seems fairly clearly targeted at the .NET hardcore and those who have not yet (and there can be many good reasons not to) stepped out of the pure Microsoft world.

Microsoft’s great tooling for .NET is often mentioned and I agree – its excellent. If you’re on Windows. JetBrains do have Rider as an excellent cross platform option but there are third party solutions for many languages and runtime. So again, from a Microsoft point of view, we’re back to the existing .NET audience.

.NET Core is fast right? So does that make it compelling? Its certainly a bonus and could be a swing factor if you’re pushing large volumes of transactions or crunching lots of data but for many applications and systems performance blocks are IO and architecture related. There’s a lot of room in that space before hitting runtime issues. But yes – I think .NET Core does offer some advantage in this space though its not the fastest thing out there if you care about benchmarks. Its worth noting it also doesn’t play well in serverless environments – its a poor performer on both AWS and Azure when it comes to coldstarts.

C# is a language I’ve used a lot over the last (nearly) 20 years and I’ve enjoyed my time with it. Its evolved over that period which has been mostly great but its starting to feel like a complicated and strange beast. Many of the changes have been fantastic: generics, expressions and lambdas to name but a few but it increasingly feels like its trying to be all things to all people as it shoehorns in further functional programming features and resulting in something with no (to me at least) obvious path of least resistance. It also tends to come with a significant amount of typing and ceremony – this doesn’t have to be the case and there have been some attempts to demonstrate this but again the path of least resistance is fairly confused.

Finally how about the “Microsoft problem”? What do I mean by this – a generation or two of developers, particularly outside of the enterprise space, have been (to put it politely) reluctant to use Microsoft technologies due to Microsofts behaviour in the 90s and early 2000s – the abuse of their Windows monopoly, attitude to the web, embrace extend extinguish, and hostility to open source all contributed to this. However there is plenty of evidence to suggest this is at least diminishing as a factor – look at the popularity of Visual Studio Code and TypeScript. Both widely adopted and in the kind of scenarios where Microsoft would have been anathema just a few years ago. The difference, to me, is that they don’t require you to throw anything away and offer signficant clear advantages: you can add TypeScript to an existing JS solution and quite quickly begin to leverage better tooling, fewer required unit tests, more readable code and you don’t need a team of developers to upend their world. Good luck doing the same with Blazor!

The choice of a language, runtime and architecture is rarely simple and if you’ve previous experience or existing teams is going to be as much about the past as it is the future – what do people know, what do they like, what are they familiar with. Change has to have a demonstratable benefit and .NETs problem is its just not bringing anything compelling to the table for those not already in its sphere and its not providing an easy route in.

My expectation is it continues to bubble along with its current share and its current audience. I think .NET Core was important to arrest a fall – the lack of cross-platform was seriously hurting it .NET – but its not going to drive any significant growth or change. My advice to any folk working on .NET at Microsoft: if you’re serious about pushing out of the bubble then look at your successes elsewhere (TypeScript and Code) and give people a way to bridge worlds. You’ll get nowhere with all or nothing and reinventing everything yourselves. Oh – and sort out the branding. 4.x, Core, now 5 – its confusing as all heck even for those of us who follow it and when the rebuttal of this is links to articles trying to explain it thats a sign you’ve got a problem not a solution. If I was looking at it from the outside I’d probably bounce off all this in confusion.

When I look ahead to where I think I’ll be heading personally: ever more down the functional programming route. It offers significant advantages (that I’ve spoken about before) and, at least with F#, a clear path of least resistance. As for runtimes I’ll probably stick in this hybrid world of React and .NET Core for some time – I can’t see any compelling reason to drop either of those at the moment.

As for my day job… the jury is out. Its going to come down to people, productivity and project more than anything else. I see no inherent advantage in .NET Core as a piece of technology.

.NET Open Source – A Mugs Game

I recognise my own motivations have at least as large a bearing on my views as the behaviour of vendors and so I should note that this is a purely personal perspective and from someone who currently largely operates in the .NET and Microsoft space. Your mileage may differ.

If you follow me on Twitter you might have noticed a growing disillusionment on my part with open source software – not its usefulness but rather the culture, business and economics.

OSS used to be a threat to big tech but, particularly with the growth of the cloud as a business model, its become a source of ideas, free products to package and host, and free labour for big tech. They figured out how to monteise it. The old Microsoft mantra of embrace, extend, extinguish is back but now applied to open source (the recent debacles around AppGet and, the excellent, Farmer are recent examples of this).

During a discussion on Twitter around this @HowardvRooijen made this comment:

Part of the issue is, 20 years in, everyone is still focusing on low level components… They are fun to hobby code (and then increasingly wearisome), if you move up the value stack MS are less likely to compete.

https://twitter.com/HowardvRooijen/status/1274412962439192578?s=20

He’s right but for me, emotionally, OSS was a sandpit where you could still almost scratch that 8-bit low level itch in a useful way. In someways a holdout of the “good old days” of the bedroom coder and there was a certain spirit to it that, for me, has been lost.

If you move up the value chain you’re talking about building products or things that more obviously can be turned into products. If these things are the things that are “safer” from big tech and Microsoft then absolutely we should not be creating them for free. Thats our income, our livelihood.

I have switched (more later) to move up the value chain with my cycling performance website but I currently have no intention of making this OSS. If it starts to get traction and feels worth it (i.e. it provides enough value to warrant a subscription) I will absolutely look to charge for it. OSS doesn’t help me build it and it doesn’t help me sell it – so it doesn’t make sense to.

What of the .NET Foundation? Is it a beacon of hope? Christ no. From its maturity model to silence over recent events (and seeming ignorance of the collateral damage of things like the AppGet issue) its very slanted towards a) Microsoft interests and b) the consumption of OSS. I’ve seen little support for small contributors. I was really concerned by the approach to the Maturity Model as that was entirely about trying to get contributors to “professionalise” to ease the adoption by “risk averse businesses”. Translation: get contributors to do more work so that profitable business can benefit.

I joined the Foundation with the idea that I could speak from the inside. In reality I’ve struggled to find the will to engage and I’ll almost certainly let my membership lapse at the end of its current term. The only thing I can see keeping me in is their support for user groups.

So where does this leave me? Basically I’m out. I’m done with it. At this point I think .NET OSS is a mugs game. I’m conciously upping my sponsorships of OSS projects I use so I continue to give something back but I’m not participating in this any more. There’s no good outcome here for contributors.

I’ll try and organise a handover of Function Monkey but I feel very dispirited by things at the moment and, frankly, am struggling to look at it.

I’ll continue to work on community contributions but in the form of writing and videos. But heck I’m even thinking that has to lead somewhere deliberate.

Where does that leave others? If you’re planning on starting a new OSS project I’d first encourage you to think long and hard about why and at the very least reconcile yourself with the likely end game (obscurity, hijacked in some way by Microsoft, hired perhaps) and understand that you’ve given all the value away. Are you really ok with that? Maybe you can build a consultancy business round it but history shows how hard this is and I think its getting harder.

I think there are easier and more effective ways to achieve most (if not all) of the outcomes OSS contribution might lead to.

Finally I’ll finish by saying: I think there are plenty of well meaning people at Microsoft. I think most of those promoting OSS from within Microsoft do so meaning well but ultimately as an organisation they have a very specific business model around open source software (absolutely its been discussed at senior levels of the business) and its one that is at odds with how I at least, perhaps naively, saw the “spirit” of open source software.

Using Function Monkey with MediatR

There are a lot of improvements coming in v4 of Function Monkey and the beta is currently available on NuGet. As the full release approaches I thought it would make sense to introduce some of these new capabilities here.

In order to simplyify Azure Functions development Function Monkey makes heavy use of commanding via a mediator and ships with my own mediation library. However there’s a lot of existing code out their that makes use of the popular MediatR library which, if Function Monkey supported, could fairly easily be moved into a serverless execution environment.

Happily Function Monkey now supports just this! You can use my existing bundled mediator, bring your own mediator, or add the shiny new FunctionMonkey.MediatR NuGet package. Here we’re going to take a look at using the latter.

First begin by creating a new, empty, Azure Functions project and add three NuGet packages:

FunctionMonkey
FunctionMonkey.Compiler
FunctionMonkey.MediatR

At the time of writing be sure to use the prerelease packages version 4.0.39-beta.4 or later.

Next create a folder called Models. Add a class called ToDoItem:

public class ToDoItem
{
    public Guid Id { get; set; }
    
    public string Title { get; set; }
    
    public bool IsComplete { get; set; }
}

Now add a folder called Services and add an interface called IRepository:

internal interface IRepository
{
    Task<ToDoItem> Create(string title);
}

And a memory based implementation of this called Repository:

internal class Repository : IRepository
{
    private readonly List<ToDoItem> _items = new List<ToDoItem>();
    
    public Task<ToDoItem> Create(string title)
    {
        ToDoItem newItem = new ToDoItem()
        {
            Title = title,
            Id = Guid.NewGuid(),
            IsComplete = false
        };
        _items.Add(newItem);
        return Task.FromResult(newItem);
    }
}

Now create a folder called Commands and in here create a class called CreateToDoItemCommand:

public class CreateToDoItemCommand : IRequest<ToDoItem>
{
    public string Title { get; set; }
}

If you’re familiar with Function Monkey you’ll notice the difference here – we’d normally implement the ICommand<> interface but here we’re implementing MediatR’s IRequest<> interface instead.

Next create a folder called Handlers and in here create a class called CreateToDoItemCommandHandler as shown below:

internal class CreateToDoItemCommandHandler : IRequestHandler<CreateToDoItemCommand, ToDoItem>
{
    private readonly IRepository _repository;

    public CreateToDoItemCommandHandler(IRepository repository)
    {
        _repository = repository;
    }
    
    public Task<ToDoItem> Handle(CreateToDoItemCommand request, CancellationToken cancellationToken)
    {
        return _repository.Create(request.Title);
    }
}

Again the only real difference here is that rather than implement the ICommandHandler interface we implement the IRequestHandler interface from MediatR.

Finally we need to add our FunctionAppConfiguration class to the root of the project to wire everything up:

public class FunctionAppConfiguration : IFunctionAppConfiguration
{
    public void Build(IFunctionHostBuilder builder)
    {
        builder
            .Setup(sc => sc
                .AddMediatR(typeof(FunctionAppConfiguration).Assembly)
                .AddSingleton<IRepository, Repository>()
            )
            .UseMediatR()
            .Functions(functions => functions
                .HttpRoute("todo", route => route
                    .HttpFunction<CreateToDoItemCommand>(HttpMethod.Post)
                )
            );
    }
}

Again this should look familiar however their are two key differences. Firstly in the Setup block we use MediatR’s IServiceCollection extension method AddMediatR – this will wire up the request handlers in the dependency injector. Secondly the .UseMediatR() option instructs Function Monkey to use MediatR for its command mediation.

And really that’s all their is to it! You can use both requests and notifications and you can find a more fleshed out example of this on GitHub.

As always feedback is welcome on Twitter or over on the GitHub issues page for Function Monkey.

Writing and Testing Azure Functions with Function Monkey – Part 3

Part 3 of my series on writing Azure Functions with Function Monkey focuses on writing tests using the newly released testing package – while this is by no means required it does make writing high value acceptance tests that use your applications full runtime easy and quick.

Lessons Learned

It really is amazing how quickly time passes when you’re talking and coding – I really hadn’t realised I’d recorded over an hours footage until I came to edit the video. I thought about splitting it in two but the contents really belonged together so I’ve left it as is.

Writing and Testing Azure Functions with Function Monkey – Part 2

Part 2 in my series of writing and testing Azure Functions with Function Monkey looks at adding validation and returning appropriate status codes from HTTP requests. Part 3 will look at acceptance testing this.

(Don’t worry – we’re going to look at some additional trigger types soon!)

Lessons Learned

I made a bunch of changes following some awesome feedback on Twitter and hopefully that results in an improved viewing experience for people.

I still haven’t got the font size right – I used a mix of Windows 10 scaling and zoomed text in Visual Studio but its not quite there. I also need to start thinking of actually using zoom when I’m pointing at something.

Hopefully part 3 I can address more of these things.

Contact

  • If you're looking for help with C#, .NET, Azure, Architecture, or would simply value an independent opinion then please get in touch here or over on Twitter.

Recent Posts

Recent Tweets

Recent Comments

Archives

Categories

Meta

GiottoPress by Enrique Chavez