Category Archives: Microservice Analytics

Capturing and Tracing All HTTP Requests in C# and .Net

Modern applications are complex and often rely on a large number of external resources increasingly accessed using HTTP – for example most Azure services are interacted with using the HTTP protocol.

That being the case it can be useful to get a view of the requests your application is making and while this can be done with a tool like Fiddler that’s not always convenient in a production environment.

If you’re using the HttpClient class another option is to pass a custom message handler to it’s constructor but this relies on you being in direct control of all the code making HTTP requests and that’s unlikely.

A simple way of capturing this information without getting into all the unpleasantness of writing a TCP listener or HTTP proxy is to use the System.Diagnostics namespace. From .net 4.5 onwards the framework has been writing HTTP events to the System.Diagnostics.Eventing.FrameworkEventSource source. This isn’t well documented and I found the easiest way to figure out what events are available, and their Event IDs, is to read the source.

Once you’ve found the HTTP events it’s quite straightforward to write an event listener that listens to this source. The below class will do this and output the details to the trace writer (so you can view it in the Visual Studio Debug Output window) but you can easily output it to a file, table storage, or any other output format of your choosing.

To set it running all you need to do is instantiate the class.

If you’d like to see this kind of data and much more, collected, correlated and analysed then you might want to check out my project Hub Analytics that is currently running a free beta.

Microservice Analytics – Update

For those who’ve got in touch asking for a beta code – thank you for your interest, it’s very much appreciated and I’m nearly ready to start issuing them.

Before releasing them I wanted to make sure I was capturing user and session data properly and that that was getting integrated deep across the other measures and data I capture. Happily this is all working and I’ve just deployed it to my live site.

In this initial release, unless you configure data capture otherwise, then magic numbers will be generated for users and sessions (you can override this to provide your own IDs if you so choose) and these are correlated with everything else that is happening so that it is possible to look at the data from both perspectives:

  • What are your users doing and what system activity is that generating in your applications?
  • Given an system activity (say a SQL command or Web API call) which user initiated it and/or under which session?

There are some specific views to help with the above but in addition you can tag any user or session and then the whole user interface is filtered by that allowing you to explore quite organically this subset of data.

I’ve attached a couple of screenshots below (taken immediately after deploying the new code to live – so not many users and sessions captured yet!) and as I mentioned earlier I’m nearly ready to share those invite codes, thanks for your patience.

sessions

pinned

Microservice Analytics – New Analytic / Diagnostic Solution

I spend a lot of time working with highly distributed systems that use an SOA / microservice type architecture and increasingly are accessed through clients such as native apps, Cordova app, or JavaScript SPAs and it can be challenging to understand what is going on across multiple servers and devices when faults occur or you need to understand something.

I wanted a tool that was fairly non-invasive, had a highly interactive user interface, and would capture a wide breadth of data and was reasonably priced. There were solutions that did bits of what I wanted (particularly at the high end in terms of price) but nothing that really hit the sweet spot for me particularly in terms of being able to drill through and across data to form a coherent picture.

This sort of thing really is my cup of tea as a developer and so I went off and built it coming at this from two directions:

  1. When errors occur in the system I want to know everything that was happening around them across devices and servers and to be able to drill into it all and find related items.
  2. To understand how users are behaving and experiencing the system and to be able to jump off and drill into areas of interest.

I’ve been dogfooding on it for a while now and the first part is pretty much ready to roll and the second part has most of the plumbing in place and is getting expanded out over the next few weeks. It’s tentatively called Microservice Analytics and there are some screenshots below to whet your appetite.

It’s one of those projects that is almost endless in terms of where you might take it (I’ve got a list of enhancements as long as my arm just from my own usage) but at some point I need some feedback and to figure out it its useful to anyone but me and so I’m just starting to issue beta invite codes.

If you’re interested get in touch with a comment below.

overview

related_summary

 

related_timeline

http