Category: .NET

An Azure Reference Architecture

There are an awful lot of services available on Azure but I’ve noticed a pattern emerging in a lot of my work around web apps. At their core they often have a similar architecture, deployment in Azure, and process for build and release.

For context a lot my hands on work over the last 3 years has been as a freelancer developing custom systems for people or on my own side projects (most recently https://www.forcyclistsbycyclists.com). In these situations I’ve found productivity to be super important in a few key ways:

  1. There’s a lot to get done, one or two people, and not much time – so being able to crank out a lot of work quickly and to a good level of quality is key.
  2. Adaptability – if its an externally focused green field system there’s a reasonable chance that there’s a degree of uncertainty over what the right feature set is. I generally expect to have to iterate a few times.
  3. I can’t be wasting time repeating myself or undertaking lengthy manual tasks.

Due to this I generally avoid over complicating my early stage deployment with too much separation – but I *do* make sure I understand where my boundaries and apply principles that support the later distribution of a system in the code.

With that out the way… here’s an architecture I’ve used as a good starting point several times now. And while it continues to evolve and I will vary specific decisions based on need its served me well and so I thought I’d share it here.

I realise there are some elements on here that are not “the latest and greatest” however its rarely productive to be on the bleeding edge. It seems likely, for example, that I’ll adopt the Azure SPA support at some point – but there’s not much in it for me doing that now. Similarly I can imagine giving GitHub Actions ago at some point – but what do I really gain by throwing what I know away today. From the experiments I’ve run I gain no productivity. Judging this stuff is something of a fine line but at the risk of banging this drum too hard: far too many people adopt technology because they see it being pushed and talked about on Twitter or dev.to (etc.) by the vendor, by their DevRel folk and by their community (e.g. MVPs) and by those who have jumped early and are quite possibly (likely!) suffering from a bizarre mix of Stockholm Syndrome and sunk cost fallacy “honestly the grass is so much greener over here… I’m so happy I crawled through the barbed wire”.

Rant over. If you’ve got any questions, want to tell me I’m crazy or question my parentage: catch me over on Twitter.

Architecture

Build & Release

I’ve long been a fan of automating at least all the high value parts of build & release. If you’re able to get it up and running quickly it rapidly pays for itself over the lifetime of a project. And one of the benefits of not CV chasing the latest tech is that most of this stuff is movable from project to project. Once you’ve set up a pipeline for a given set of assets and components its pretty easy to use on your next project. Introduce lots of new components… yeah you’ll have lots of figuring out to do. Consistency is underrated in our industry.

So what do I use and why?

  1. Git repository – I was actually an early adopter of Git. Mostly because I was taking my personal laptop into a disconnected environment on a regular basis when it first started to emege and I’m a frequent committer.

    In this architecture it holds all the assets required to build & deploy my system other than secrets.
  2. Azure DevOps – I use the pipelines to co-ordinate build & release activities both directly using built in tasks, third party tasks and scripts. Why? At the time I started it was free and “good enough”. I’ve slowly moved over to the YAML pipelines. Slowly.
  3. My builds will output four main assets: an ARM template, Docker container, a built single page application, and SQL migration scripts. These get deployed into a an Azure resource group, Azure container registry, blob storage, and a SQL database respectively.

    My migration scripts are applied against a SQL database using DbUp and my ARM templates are generated using Farmer and then used to provision a resource group. I’m fairly new to Farmer but so far its been fantastic – previously I was using Terraform but Farmer just fit a little nicer with my workflow and I like to support the F# community.

Runtime Environment

So what do I actually use to run and host my code?

  1. App Service – I’ve nearly always got an API to host and though I will sometimes use Azure Functions for this I more often use the Web App for Containers support.

    Originally I deployed directly into a “plain” App Service but grew really tired with the ongoing “now this is really how you deploy without locked files” fiasco and the final straw was the bungled .NET Core release.

    Its just easier and more reliable to deploy a container.
  2. Azure DNS – what it says on the tin! Unless there is a good reason to run it elsewhere I prefer to keep things together, keeps things simple.
  3. Azure CDN – gets you a free SSL cert for your single page app, is fairly inexpensive, and helps with load times.
  4. SQL Database – still, I think, the most flexible general purpose and productive data solution. Sure at scale others might be better. Sure sometimes less structured data is better suited to less structured data sources. But when you’re trying to get stuff done there’s a lot to be said for having an atomic, transactional data store. And if I had a tenner for every distributed / none transactional design I’ve seen that dealt only with the happy path I would be a very very wealthy man.

    Oh and “schema-less”. In most cases the question is is the schema explicit or implicit. If its implicit… again a lot of what I’ve seen doesn’t account for much beyodn the happy path.

    SQL might not be cool, and maybe I’m boring (but I’ll take boring and gets shit done), but it goes a long way in a simple to reason about manner.
  5. Storage accounts – in many systems you come across small bits of data that are handy to dump into, say, a blob store (poor mans NoSQL right there!) or table store. I generally find myself using it at some point.
  6. Service Bus – the unsung hero of Azure in my opinion. Its reliable. Does what it says on the tin and is easy to work with. Most applications have some background activity, chatter or async events to deal with and service bus is a great way of handling this. I sometimes pair this (and Azure Functions below) with SignalR.
  7. Azure Functions – great for processing the Service Bus, running code on a schedule and generally providing glue for your system. Again I often find myself with at least a handful of these. I often also use Service Bus queues with Functions to provide a “poor mans admin console”. Basically allow me to kick off administrative events by dropping a message on a queue.
  8. Application Insights – easy way of gathering together logs, metrics, telemetry etc. If something does go wrong or your system is doing something strange the query console is a good way of exploring what the root cause might be.

Code

I’m not going to spend too long talking about how I write the system itself (plenty of that on this blog already). In generally I try and keep things loosely coupled and normally start with a modular monolith – easy to reason about, well supported by tooling, minimal complexity but can grow into something more complex when and if that’s needed.

My current tools of choice is end to end F# with Fable and Saturn / Giraffe on top of ASP.Net Core and Fable Remoting on top of all that. I hopped onto functional programming as:

  1. It seemed a better fit for building web applications and APIs.
  2. I’d grown tired with all the C# ceremony.
  3. Collectively we seem to have decided that traditional OO is not that useful – yet we’re working in languages built for that way of working. And I felt I was holding myself back / being held back.

But if you’re looking to be productive – use what you know.

Why (not) .NET?

A frequent topic of conversation on Twitter, particularly if you follow a number of Microsoft employees, is why isn’t .NET used more or talked about more. The latest round of this I saw was in connection with microserviecs – “where’s all the chat about .NET Core microservices”. There’s often a degree of surprise, frustration and curiosity expressed that people outside of the existing .NET audience aren’t looking at .NET Core – or at least seemingly not in large numbers (I’m sure someone can pull out a statistic suggseting otherwise, and someone else pull out a statistic suggesting it is the case).

One indicator of usage is the TIOBE index which is focused on langauge. VB and C# currently rank 5 and 6 in terms of popularity with their history shown below:

Based on this .NET is certainly a successful runtime and you could argue that the release of .NET Core in 2016 (announced in 2014 with a lengthy preview period) halted what looked to be a significant declince in C# usage – I would assume due to the lack of cross-platform and growing popularity of the cloud.

But since then C# has pretty much flatlined – maybe trending up slightly (the peaks and troughs seem to be going up over the last year). VB… I know far less about the VB world an what could have caused that spike. I’m assuming it has little to do with .NET Core given the limitations of running on the platform.

I find myself in an interesting position with regard to topics like this – I’ve been using .NET for a long time (since the very first beta), its the ecosystem I’m most familiar with, and in general I’m fairly happy with it. On the other hand I now work as a CTO managing teams with a variety of tech stacks (.NET, Node / React, and PHP for the most part) and when looking at new build projects wrangle with what to build it in.

And you know what? There is very little about .NET (Core or otherwise) that stands out.

That it is now cross-platform is great and opens it up to use cases and developers for which it would otherwise be a no-go (myself included these days) but that’s a catch-up play by Microsoft not a unique feature. And the majority of developers who are not on Windows have already spent significant time getting productive in other runtimes and ecosystems – to trade that away and incur a productivity debt there has to be something to offset that debt.

Now being able to write end to end browser through to backend C# (and I’ll swing back to C# later) with Blazor looks compelling but you can do this with just about any language and runtime now through the wizardry of transpilers. Sure Blazor targets WASM and that has some advantages – but also has a boat load of disadvantages: it can be a big download and is an isolated ecosystem. And it can suffer from some of the same quirks transpilers do – in my, brief, time with it I ran into over eager IL stripping issues that were obtuse to say the least. The advantage of the approaches taken by things like Fable (F#), Reason (OCaml), and even Bridge (C#) is that it lets you continue to use the ecosystem you know. For example I was easily able to move to using F# on the front end with Fable and take my React knowledge with me. I can’t do that with Blazor. Why not use Bridge? Well then you’re back to the kind of issue sometimes levelled at Fable – reliance on a small band of OSS contributors on a fringe project. To be fair to Blazor – it seems fairly clearly targeted at the .NET hardcore and those who have not yet (and there can be many good reasons not to) stepped out of the pure Microsoft world.

Microsoft’s great tooling for .NET is often mentioned and I agree – its excellent. If you’re on Windows. JetBrains do have Rider as an excellent cross platform option but there are third party solutions for many languages and runtime. So again, from a Microsoft point of view, we’re back to the existing .NET audience.

.NET Core is fast right? So does that make it compelling? Its certainly a bonus and could be a swing factor if you’re pushing large volumes of transactions or crunching lots of data but for many applications and systems performance blocks are IO and architecture related. There’s a lot of room in that space before hitting runtime issues. But yes – I think .NET Core does offer some advantage in this space though its not the fastest thing out there if you care about benchmarks. Its worth noting it also doesn’t play well in serverless environments – its a poor performer on both AWS and Azure when it comes to coldstarts.

C# is a language I’ve used a lot over the last (nearly) 20 years and I’ve enjoyed my time with it. Its evolved over that period which has been mostly great but its starting to feel like a complicated and strange beast. Many of the changes have been fantastic: generics, expressions and lambdas to name but a few but it increasingly feels like its trying to be all things to all people as it shoehorns in further functional programming features and resulting in something with no (to me at least) obvious path of least resistance. It also tends to come with a significant amount of typing and ceremony – this doesn’t have to be the case and there have been some attempts to demonstrate this but again the path of least resistance is fairly confused.

Finally how about the “Microsoft problem”? What do I mean by this – a generation or two of developers, particularly outside of the enterprise space, have been (to put it politely) reluctant to use Microsoft technologies due to Microsofts behaviour in the 90s and early 2000s – the abuse of their Windows monopoly, attitude to the web, embrace extend extinguish, and hostility to open source all contributed to this. However there is plenty of evidence to suggest this is at least diminishing as a factor – look at the popularity of Visual Studio Code and TypeScript. Both widely adopted and in the kind of scenarios where Microsoft would have been anathema just a few years ago. The difference, to me, is that they don’t require you to throw anything away and offer signficant clear advantages: you can add TypeScript to an existing JS solution and quite quickly begin to leverage better tooling, fewer required unit tests, more readable code and you don’t need a team of developers to upend their world. Good luck doing the same with Blazor!

The choice of a language, runtime and architecture is rarely simple and if you’ve previous experience or existing teams is going to be as much about the past as it is the future – what do people know, what do they like, what are they familiar with. Change has to have a demonstratable benefit and .NETs problem is its just not bringing anything compelling to the table for those not already in its sphere and its not providing an easy route in.

My expectation is it continues to bubble along with its current share and its current audience. I think .NET Core was important to arrest a fall – the lack of cross-platform was seriously hurting it .NET – but its not going to drive any significant growth or change. My advice to any folk working on .NET at Microsoft: if you’re serious about pushing out of the bubble then look at your successes elsewhere (TypeScript and Code) and give people a way to bridge worlds. You’ll get nowhere with all or nothing and reinventing everything yourselves. Oh – and sort out the branding. 4.x, Core, now 5 – its confusing as all heck even for those of us who follow it and when the rebuttal of this is links to articles trying to explain it thats a sign you’ve got a problem not a solution. If I was looking at it from the outside I’d probably bounce off all this in confusion.

When I look ahead to where I think I’ll be heading personally: ever more down the functional programming route. It offers significant advantages (that I’ve spoken about before) and, at least with F#, a clear path of least resistance. As for runtimes I’ll probably stick in this hybrid world of React and .NET Core for some time – I can’t see any compelling reason to drop either of those at the moment.

As for my day job… the jury is out. Its going to come down to people, productivity and project more than anything else. I see no inherent advantage in .NET Core as a piece of technology.

.NET Open Source – A Mugs Game

I recognise my own motivations have at least as large a bearing on my views as the behaviour of vendors and so I should note that this is a purely personal perspective and from someone who currently largely operates in the .NET and Microsoft space. Your mileage may differ.

If you follow me on Twitter you might have noticed a growing disillusionment on my part with open source software – not its usefulness but rather the culture, business and economics.

OSS used to be a threat to big tech but, particularly with the growth of the cloud as a business model, its become a source of ideas, free products to package and host, and free labour for big tech. They figured out how to monteise it. The old Microsoft mantra of embrace, extend, extinguish is back but now applied to open source (the recent debacles around AppGet and, the excellent, Farmer are recent examples of this).

During a discussion on Twitter around this @HowardvRooijen made this comment:

Part of the issue is, 20 years in, everyone is still focusing on low level components… They are fun to hobby code (and then increasingly wearisome), if you move up the value stack MS are less likely to compete.

https://twitter.com/HowardvRooijen/status/1274412962439192578?s=20

He’s right but for me, emotionally, OSS was a sandpit where you could still almost scratch that 8-bit low level itch in a useful way. In someways a holdout of the “good old days” of the bedroom coder and there was a certain spirit to it that, for me, has been lost.

If you move up the value chain you’re talking about building products or things that more obviously can be turned into products. If these things are the things that are “safer” from big tech and Microsoft then absolutely we should not be creating them for free. Thats our income, our livelihood.

I have switched (more later) to move up the value chain with my cycling performance website but I currently have no intention of making this OSS. If it starts to get traction and feels worth it (i.e. it provides enough value to warrant a subscription) I will absolutely look to charge for it. OSS doesn’t help me build it and it doesn’t help me sell it – so it doesn’t make sense to.

What of the .NET Foundation? Is it a beacon of hope? Christ no. From its maturity model to silence over recent events (and seeming ignorance of the collateral damage of things like the AppGet issue) its very slanted towards a) Microsoft interests and b) the consumption of OSS. I’ve seen little support for small contributors. I was really concerned by the approach to the Maturity Model as that was entirely about trying to get contributors to “professionalise” to ease the adoption by “risk averse businesses”. Translation: get contributors to do more work so that profitable business can benefit.

I joined the Foundation with the idea that I could speak from the inside. In reality I’ve struggled to find the will to engage and I’ll almost certainly let my membership lapse at the end of its current term. The only thing I can see keeping me in is their support for user groups.

So where does this leave me? Basically I’m out. I’m done with it. At this point I think .NET OSS is a mugs game. I’m conciously upping my sponsorships of OSS projects I use so I continue to give something back but I’m not participating in this any more. There’s no good outcome here for contributors.

I’ll try and organise a handover of Function Monkey but I feel very dispirited by things at the moment and, frankly, am struggling to look at it.

I’ll continue to work on community contributions but in the form of writing and videos. But heck I’m even thinking that has to lead somewhere deliberate.

Where does that leave others? If you’re planning on starting a new OSS project I’d first encourage you to think long and hard about why and at the very least reconcile yourself with the likely end game (obscurity, hijacked in some way by Microsoft, hired perhaps) and understand that you’ve given all the value away. Are you really ok with that? Maybe you can build a consultancy business round it but history shows how hard this is and I think its getting harder.

I think there are easier and more effective ways to achieve most (if not all) of the outcomes OSS contribution might lead to.

Finally I’ll finish by saying: I think there are plenty of well meaning people at Microsoft. I think most of those promoting OSS from within Microsoft do so meaning well but ultimately as an organisation they have a very specific business model around open source software (absolutely its been discussed at senior levels of the business) and its one that is at odds with how I at least, perhaps naively, saw the “spirit” of open source software.

Using Function Monkey with MediatR

There are a lot of improvements coming in v4 of Function Monkey and the beta is currently available on NuGet. As the full release approaches I thought it would make sense to introduce some of these new capabilities here.

In order to simplyify Azure Functions development Function Monkey makes heavy use of commanding via a mediator and ships with my own mediation library. However there’s a lot of existing code out their that makes use of the popular MediatR library which, if Function Monkey supported, could fairly easily be moved into a serverless execution environment.

Happily Function Monkey now supports just this! You can use my existing bundled mediator, bring your own mediator, or add the shiny new FunctionMonkey.MediatR NuGet package. Here we’re going to take a look at using the latter.

First begin by creating a new, empty, Azure Functions project and add three NuGet packages:

FunctionMonkey
FunctionMonkey.Compiler
FunctionMonkey.MediatR

At the time of writing be sure to use the prerelease packages version 4.0.39-beta.4 or later.

Next create a folder called Models. Add a class called ToDoItem:

public class ToDoItem
{
    public Guid Id { get; set; }
    
    public string Title { get; set; }
    
    public bool IsComplete { get; set; }
}

Now add a folder called Services and add an interface called IRepository:

internal interface IRepository
{
    Task<ToDoItem> Create(string title);
}

And a memory based implementation of this called Repository:

internal class Repository : IRepository
{
    private readonly List<ToDoItem> _items = new List<ToDoItem>();
    
    public Task<ToDoItem> Create(string title)
    {
        ToDoItem newItem = new ToDoItem()
        {
            Title = title,
            Id = Guid.NewGuid(),
            IsComplete = false
        };
        _items.Add(newItem);
        return Task.FromResult(newItem);
    }
}

Now create a folder called Commands and in here create a class called CreateToDoItemCommand:

public class CreateToDoItemCommand : IRequest<ToDoItem>
{
    public string Title { get; set; }
}

If you’re familiar with Function Monkey you’ll notice the difference here – we’d normally implement the ICommand<> interface but here we’re implementing MediatR’s IRequest<> interface instead.

Next create a folder called Handlers and in here create a class called CreateToDoItemCommandHandler as shown below:

internal class CreateToDoItemCommandHandler : IRequestHandler<CreateToDoItemCommand, ToDoItem>
{
    private readonly IRepository _repository;

    public CreateToDoItemCommandHandler(IRepository repository)
    {
        _repository = repository;
    }
    
    public Task<ToDoItem> Handle(CreateToDoItemCommand request, CancellationToken cancellationToken)
    {
        return _repository.Create(request.Title);
    }
}

Again the only real difference here is that rather than implement the ICommandHandler interface we implement the IRequestHandler interface from MediatR.

Finally we need to add our FunctionAppConfiguration class to the root of the project to wire everything up:

public class FunctionAppConfiguration : IFunctionAppConfiguration
{
    public void Build(IFunctionHostBuilder builder)
    {
        builder
            .Setup(sc => sc
                .AddMediatR(typeof(FunctionAppConfiguration).Assembly)
                .AddSingleton<IRepository, Repository>()
            )
            .UseMediatR()
            .Functions(functions => functions
                .HttpRoute("todo", route => route
                    .HttpFunction<CreateToDoItemCommand>(HttpMethod.Post)
                )
            );
    }
}

Again this should look familiar however their are two key differences. Firstly in the Setup block we use MediatR’s IServiceCollection extension method AddMediatR – this will wire up the request handlers in the dependency injector. Secondly the .UseMediatR() option instructs Function Monkey to use MediatR for its command mediation.

And really that’s all their is to it! You can use both requests and notifications and you can find a more fleshed out example of this on GitHub.

As always feedback is welcome on Twitter or over on the GitHub issues page for Function Monkey.

Avoiding Gremlin Injection Attacks with Azure Cosmos DB

I’ve written previously about some of the issues with using Cosmos DB as a graph database from .NET. One of the more serious issues, I think, is that the documentation doesn’t really demonstrate how to avoid an injection attack when using Gremlin as it presents examples using hard coded strings which are then just picked up and run through the Gremlin.NET library:

// Gremlin queries that will be executed.
private static Dictionary<string, string> gremlinQueries = new Dictionary<string, string>
{
    { "Cleanup",        "g.V().drop()" },
    { "AddVertex 1",    "g.addV('person').property('id', 'thomas').property('firstName', 'Thomas').property('age', 44)" },
    { "AddVertex 2",    "g.addV('person').property('id', 'mary').property('firstName', 'Mary').property('lastName', 'Andersen').property('age', 39)" },
    { "AddVertex 3",    "g.addV('person').property('id', 'ben').property('firstName', 'Ben').property('lastName', 'Miller')" },
    { "AddVertex 4",    "g.addV('person').property('id', 'robin').property('firstName', 'Robin').property('lastName', 'Wakefield')" },
    { "AddEdge 1",      "g.V('thomas').addE('knows').to(g.V('mary'))" },
    { "AddEdge 2",      "g.V('thomas').addE('knows').to(g.V('ben'))" },
    { "AddEdge 3",      "g.V('ben').addE('knows').to(g.V('robin'))" },
    { "UpdateVertex",   "g.V('thomas').property('age', 44)" },
    { "CountVertices",  "g.V().count()" },
    { "Filter Range",   "g.V().hasLabel('person').has('age', gt(40))" },
    { "Project",        "g.V().hasLabel('person').values('firstName')" },
    { "Sort",           "g.V().hasLabel('person').order().by('firstName', decr)" },
    { "Traverse",       "g.V('thomas').out('knows').hasLabel('person')" },
    { "Traverse 2x",    "g.V('thomas').out('knows').hasLabel('person').out('knows').hasLabel('person')" },
    { "Loop",           "g.V('thomas').repeat(out()).until(has('id', 'robin')).path()" },
    { "DropEdge",       "g.V('thomas').outE('knows').where(inV().has('id', 'mary')).drop()" },
    { "CountEdges",     "g.E().count()" },
    { "DropVertex",     "g.V('thomas').drop()" },
};

Focusing in on one of the add vertex examples and how it might be executed with the Gremlin.NET library:

private async Task BadExample()
{
    using (GremlinClient client = new GremlinClient(_gremlinServer, new GraphSON2Reader(),
        new GraphSON2Writer(), GremlinClient.GraphSON2MimeType))
    {
        await client.SubmitAsync(
            "g.addV('person').property('id', 'thomas').property('firstName', 'Thomas').property('age', 44)");
    }
}

We know from years of SQL that examples like this quickly become widespread injection prone pieces of code like the below, particularly if people are new to working with a new database (and in the case of graph databases and Gremlin – that’s most people):

private async Task BadExample(string firstName, int age)
{
    using (GremlinClient client = new GremlinClient(_gremlinServer, new GraphSON2Reader(),
        new GraphSON2Writer(), GremlinClient.GraphSON2MimeType))
    {
        await client.SubmitAsync(
            $"g.addV('person').property('id', '{firstName.ToLower()}').property('firstName', '{firstName}').property('age', {age})");
    }
}

The issue, if you’re not familiar with injection attacks, is that as a user I can enter a ‘ character in the input and break out to add my own code through a user interface that executes on the server – for example I could supply a firstName of:

James').property('myinjectedproperty','hahaha got ya

And I’ve managed to attach some data of my own choosing. Obviously I could do more nefarious things too.

Fortunately Gremlin does support parameterised queries and we can rewrite the code above more safely to look like this and leave the libraries and database to take care of this:

private async Task BetterExample(string firstName, int age)
{
    using (GremlinClient client = new GremlinClient(_gremlinServer, new GraphSON2Reader(),
        new GraphSON2Writer(), GremlinClient.GraphSON2MimeType))
    {
        Dictionary<string, object> arguments = new Dictionary<string, object>
        {
            { "firstNameAsId", firstName.ToLower() },
            { "firstName", firstName },
            { "age", age }
        };
        await client.SubmitAsync(
            $"g.addV('person').property('id', firstNameAsId).property('firstName', firstName).property('age', age)", arguments);
    }
}

With the uptake of Cosmos and Graph databases being new to most people I really wish the Cosmos team would update these docs with a security first mindset and its something I’ve fed back to them previously. Leaving the documentation as it stands is almost certainly going to lead to more insecure code being written than would otherwise be the case.

I’ll probably drop out a few short Cosmos posts over the next few days – I’m doing a lot of (quite interesting!) work with it at the moment.

Cosmos, Oh Cosmos – Graph API as a .NET Developer

I’ve done a lot of work with Cosmos over the last year as a document database and generally found it to be a rock solid experience, it does what it says on the tin, and so when I found myself with a project that was a great fit for a graph database my first port of call was Cosmos DB.

I’d done a little work with it as a Graph API but as this was a new project I visited the Azure website to refresh myself on using it as a Graph from .NET and found that Microsoft are now recommending that the Tinkerpop Gremlin .NET library be used from .NET. There are some pieces on the old Microsoft.Azure.Graphs package but it never made it out of preview and the direction of travel looks to be elsewhere.

While Gremlin .NET is easy to get started with if you try and use this in a realistic sense you quickly run into a couple of serious limitations due to it’s current design around error and response handling. It seems to be designed to support console applications rather than real world services:

  1. Vendor specific attributes of the responses such as RU costs, communicated as header values, are hidden from you.
  2. Errors from the server are presented only as text messages. Rather than expose status codes and interpretable values the Gremlin .NET library first converts these into messages designed for consumption by people. To interpret the server error you need to parse these strings.

The above two issues combine into a very unfortunate situation for a real world Cosmos Graph API application: when Cosmos rate limits you it returns a 500 error to the caller with the 429 error being communicated in a x-ms-status-code header. This doesn’t play well with resilience libraries such as Polly as you end up having to fish around in the response text for keywords.

I initially raised this as a documentation issue on GitHub and Microsoft have confirmed they are moving to open source libraries and working to improve them but best I can tell, today, for the moment you’ve got two choices:

  1. Continue to use the Microsoft.Azure.Graphs package – I’ve not used this in anger but I understand it has issues of its own related to client side performance and is a bit of a dead end.
  2. Use Gremlin.NET and work around the issues.

For the time being I’ve opted to go with option (2) as I don’t want to unpick an already obsolete package from new code. To support this I’ve forked the Gremlin.NET library and introduced a couple of changes that allow attributes and response codes to be inspected for regular requests and for exceptions. I’ve done this in a none breaking way – you should be able to replace the official Gremlin.NET package with this replacement and your code should continue to work just fine but you can more easily implement resilience patterns. You can find it on GitHub here.

If I was designing the API from fresh to work in a real world situation I would probably expose a different API surface – at the moment I really have followed the path of least none-breaking resistance in terms of getting these things visible. That makes me uncertain as to whether or not this is an appropriate Pull Request to submit – I probably will, if nothing else hopefully that will start a conversation.

Fixing a Common IoC Container Anti-pattern – the every class is public problem

An anti-pattern I’ve seen a lot over the last few years involves the registration of dependencies in an IoC container at the root of a project (or in a dedicated “IoC” project) – an approach enabled by making every single class in every assembly in the codebase public. It’s amazing how common it is and you see it in codebases that are poor in general and codebases that are otherwise well constructed. As such I find myself talking about it frequently and so it seemed a ripe topic for a blog post.

There are numerous issues with the “every class is public” approach:

  1. As someone reading or using the code I can no longer differentiate between the public API of a subsystem and the interfaces and classes designed for internal consumption.
  2. The registering project (for example an ASP.Net application) is making decisions about the lifecycle of components in another assembly and sub-system – and therefore about the internal implementation of that sub-system. This often leads to things getting out of sync and the issues arising from this kind of lifecycle registration / implementation mismatch can be subtle.
  3. The registering project has to be aware of every single thing in the system and reference every subsystem. One of the effective techniques to police code architecture is by looking at the dependency map and this is heavily polluted if you’re doing this.
  4. The scope of a code change is often larger than it should be and spans sub-systems when it doesn’t need to – if a project takes the root registration approach then adding a class and interface for internal use means I also have to visit the root project.
  5. If sub-systems are run within multiple hosts (for example a Web API and a queue processor) then registration is either duplicated in both root projects or an “IoC configuration” project is introduced: we’ve got ourselves in such a pickle that we now need a whole project dedicated to understanding both internal and external dependencies of sub-systems.

Encapsulation is a good thing – it shouldn’t be thrown away when moving from the class to the assembly level. It’s just as important there – perhaps even more so in modern codebases which are formed of many small classes with few methods rather than large classes with many methods.

I’ve provided a simple example of this common issue in the project you can find here:

https://github.com/JamesRandall/IoCAntipatternFix/tree/master/TheProblem

Conceptually in this project we’ve got three assemblies:

  1. A console app (ConsoleApp) that depends on (2)
  2. An assembly (Calendar) providing calendar functionality to the console app that depends on (3)
  3. An assembly (Notifications) providing notification functionality to the calendar assembly

From a required dependency point of view it looks like this:

But because of the every class is public issue it is actually implemented like this:

You can see the anti-pattern manifest itself in code in the RegisterDependencies method of Program.cs in the console app:

static IServiceProvider RegisterDependencies()
{
    IServiceCollection services = new ServiceCollection();
    services.AddTransient<Calendar.DataAccess.ICalendarRepository, Calendar.DataAccess.CalendarRepository>();
    services.AddTransient<Calendar.ICalendarManager, Calendar.CalendarManager>();
    services.AddSingleton<Notifications.INotifier, Notifications.Notifier>();
    services.AddTransient<Notifications.Channel.IEmail, Notifications.Channel.Email>();

    return services.BuildServiceProvider();
}

Does the console app have any business knowing that the ICalendarRepository is implemented by the CalendarRepository class? Should it even know about the email channel? Can it safely register the INotifier implementation as a singleton? The answer to all of those questions is no. Absolutely not.

The fix for this is pretty simple and it was great to see Microsoft adopt a version of it in ASP.Net Core as part of their formalisation of dependency inversion in that framework. All you need do is encapsulate the registration logic inside your sub systems – and if you need to conditionally configure the registration then pass through an options block (an example of this can be seen in my commanding framework).

I’m going to show two versions of the fix – one based on using the containers registration interface, which has the byproduct of your assemblies becoming tied to an IoC container, and another that doesn’t require this.

Solution with a container interface

The approach adopted by Microsoft in the ASP.Net Core assemblies and the related packages is to use extension methods on the container interface (in the Microsoft case that’s IServiceCollection). If we take this approach the registration in our console app now looks like this:

static IServiceProvider RegisterDependencies()
{
    IServiceCollection services = new ServiceCollection();
    services.AddCalendar();

    return services.BuildServiceProvider();
}

Additionally our console app no longer has a reference to the notification sub-system as this is now dealt with by the calendar’s AddCalendar registration method:

public static class ServiceCollectionExtensions
{
    public static IServiceCollection AddCalendar(this IServiceCollection serviceCollection)
    {
        serviceCollection.AddTransient<ICalendarRepository, CalendarRepository>();
        serviceCollection.AddTransient<ICalendarManager, CalendarManager>();

        serviceCollection.AddNotifications();

        return serviceCollection;
    }
}

Inside the calendar project only the interfaces intended for external consumption are marked as public with the rest moving to internal. It’s no longer possible to access the assemblies private implementation from the outside and we’ve moved the lifecycle and registration logic closer to the code that is written in line with those expectations.

And finally the notification assembly takes the same approach:

public static class ServiceCollectionExtensions
{
    public static IServiceCollection AddNotifications(this IServiceCollection serviceCollection)
    {
        serviceCollection.AddSingleton<INotifier, Notifier>();
        serviceCollection.AddTransient<IEmail, Email>();
        return serviceCollection;
    }
}

With this approach we’ve addressed all three of the concerns I raised at the start of this piece and have moved back to a place where encapsulation is used to help us both read the code and use it safely.

It could well be argued that having your sub-systems reference and be aware of the specific IoC container in use is itself another anti-pattern. I’d tend towards agreeing but it can be a pragmatic choice for an internal codebase – though it’s flawed if you are creating packages for others to use: you’ve built in a hard dependency on a specific IoC container. You can solve this by defining your own interface for proxying over a container and having people implement it or use a functional approach which we’ll look at next.

The code for the above approach can be found here:

https://github.com/JamesRandall/IoCAntipatternFix/tree/master/ContainerInterfaceSolution

Solution with functions

An alternative to the interface approach is to use a functional style passing down lambda expressions. If we take this approach our console application’s registration method now looks like this:

static IServiceProvider RegisterDependencies()
{
    IServiceCollection services = new ServiceCollection();
    Dependencies.AddCalendar(
        (iface, impl) => services.AddTransient(iface, impl),
        (iface, impl) => services.AddSingleton(iface, impl));

    return services.BuildServiceProvider();
}

We simply wrap the relevant lifecycle registration methods on IServiceCollection inside lambda expressions and pass them down to a registration method in our calendar sub system:

public static class Dependencies
{
    public static void AddCalendar(
        Action<Type, Type> addTransient,
        Action<Type, Type> addSingleton)
    {
        addTransient(typeof(ICalendarRepository), typeof(CalendarRepository));
        addTransient(typeof(ICalendarManager), typeof(CalendarManager));

        Notifications.Dependencies.AddNotifications(addTransient, addSingleton);
    }
}

This registers our types using the lambda expressions and passes them on to the notification dependency:

public static class Dependencies
{
    public static void AddNotifications(
        Action<Type, Type> addTransient,
        Action<Type, Type> addSingleton)
    {
        addSingleton(typeof(INotifier), typeof(Notifier));
        addTransient(typeof(IEmail), typeof(Email));
    }
}

Again this approach addresses the concerns that arise when implementation classes are made public and registration is centralised but with the added advantage that the sub-systems are independent of any specific IoC container. In my experience this also discourages people from misusing many of the “advanced” capabilities that can be found on IoC containers – but that’s a topic for another post.

The code for this approach can be found here:

https://github.com/JamesRandall/IoCAntipatternFix/tree/master/FunctionalSolution

Wrap up

Hopefully in the above I’ve highlighted a common pitfall and demonstrated two solutions to it. There are of course many other variants you can apply depending on your specific project. If you disagree or have any questions please feel free to reach out on Twitter.

Contact

  • If you're looking for help with C#, .NET, Azure, Architecture, or would simply value an independent opinion then please get in touch here or over on Twitter.

Recent Posts

Recent Tweets

Recent Comments

Archives

Categories

Meta

GiottoPress by Enrique Chavez