Category: PowerShell

Publishing to GitHub Pages from Visual Studio Team Services

During the last major release of my commanding framework I wrote a lot of new documentation. I wrote this in Markdown and converted it to a documentation website using DocFX resulting in this website.

DocFX has a VSTS extension which make it easy to integrate into your build and release pipeline but I couldn’t find any simple way to publish it to GitHub Pages which is what I am using as a simple, and free, host. My initial solution was to use a short PowerShell script which took the following steps as part of a Release Plan:

  1. Clone the gh-pages branch from my repository into a folder
  2. Copy the output from the DocFX build artefact to the folder
  3. Commit the changes
  4. Push the branch to GitHub (which publishes the changes)

This worked nicely but it irked me not being able to do it in a more off the shelf manner and so I’ve tidied this up and released a VSTS Build and Release task to the marketplace. It come complete with documentation and hopefully it’s useful to people. The source code is available on GitHub.

Writing a simple VSTS build and release task is well documented and there are plenty of blog posts on the topic and so I won’t repeat that here but I did think it might be helpful to quickly cover just a couple of things that were often not covered.

Accessing Variables

I needed to find the default working folder within my task as I needed scratch space for cloning the gh-pages branch. VSTS exposes this as a built-in variable called System.DefaultWorkingDirectory – it’s a commonly used variable and you’ve probably seen it expressed as $(System.DefaultWorkingDirectory) when using tasks for releases.

You can access variables from within a script by using the Get-VstsTaskVariable cmdlet. For example:

$defaultWorkingDirectory = Get-VstsTaskVariable -Name 'System.DefaultWorkingDirectory'

Reporting Errors

You can write to the VSTS log using the Write-Host cmdlet but I wanted a nice red error message when things went wrong. You can do this by prefixing your message as shown below:

Write-Host "##vso[task.logissue type=error;]Your error message"

To exit the script and have it fail you need to exit with a non-zero return code and using a specific Exit method:

[Environment]::Exit(1)

 

And that’s it. As always I hope this is useful and I can be reached for discussion and questions on Twitter.

 

 

PowerShell, Binding Redirects, and Visual Studio Team Services

I’ve blogged previously about setting up binding redirects for Powershell with Newtonsoft.Json being a particularly troublesome package – it’s such a common dependency for NuGet packages that if you deal with a complex project you’ll almost certainly need a redirect in your app/web.config’s to get things to play ball and if you use the Azure cmdlets with others (such as your own) you’re likely to face this problem in Powershell.

I’ve recently moved my projects into Visual Studio Team Services using the new (vastly improved!) scriptable build system where I often make use of the PowerShell script task to perform custom actions. If you hit a dependency issue that requires a binding redirect to resolve then my previous approach of creating a Powershell.exe.config file for PowerShell won’t work in VSTS as unless you build a custom build agent you don’t have access to the machine at this level.

After a bit of head scratching I came up with an alternative solution that in many ways is neater and more generally portable as it doesn’t require any special machine setup. My revised approach is to hook the AssemblyResolve event and return a preloaded target assembly as shown in the example below:

Note that you can’t use the more common Register-ObjectEvent method of subscribing to events as this will balk at the need for a return value.

You can of course use this technique to deal with other assemblies that might be giving you issues.

Azure Resource Manager and Powershell 1.0

Microsoft recently updated Azure Powershell to 1.0 in the process introducing a large number of breaking changes to the CmdLet’s. Essentially they’ve removed Switch-AzureMode and instead of that the Azure Resource Manager cmdlets have all had an Rm introduced to them so for example:

New-AzureResourceGroupDeployment

becomes

New-AzureRmResourceGroupDeployment

For the most part the errors on upgrading a Powershell script are obvious and a rename will get you going however somewhat confusingly there are now pairs of cmdlet’s that do the same thing but only effect the different cmdlet sets.

An example of one that caught me out is Select-AzureSubscription. I’ve got numerous Azure subscriptions and scripts that deploy into different subscriptions depending on what I’m doing. Previously I used Select-AzureSubscription along with Switch-AzureMode and this worked. The problem is that Select-AzureSubscription still works – but it has no effect on the cmdlets that use the Rm prefix.

This led, confusingly, to a resource group being created in a different subscription to the one I intended and resources with the same name as in my intended resource group (resources which already existed) being created, or attempted to be created, in this new resource group. They failed as they already existed with the same names elsewhere.

The fix was to use Select-AzureRmSubscription.

 

How To: Publish an AngularJS Website with Azure Resource Manager Templates

I’ve recently been working on the deployment of some fairly complex micro-service projects using the new Azure Resource Manager and in the Microsoft space everything went fine, it’s all pretty self-explanatory, there’s some useful tooling in Visual Studio and some great quick-start templates.

Then I hit my UI layer. This is an AngularJS app which I prefer to work on outside of Visual Studio – I like to use the best tools for the job and while I definitely find Visual Studio is just that for C# and other Microsoft native technologies I find it often lags behind in the web space, though it’s got and continues to get a lot better.

Basically I have a folder in my project folder space (not in the .sln file) containing my AngularJS app and a bunch of typical web tools to run the build and packages: npm, bower and grunt. For various reasons although the project source is in GitHub I couldn’t (without a fair bit of hassle) deploy directly from their using the source code deployment model.

And so how do you publish a folder of files to an Azure website using Azure Resource Manager?

Below I’ll present one way using the Yeoman generated Angular template as a sample which, using grunt, builds to a folder called dist. As a prerequisite for this you’ll need Azure Powershell -I’m using 0.8 at the time of writing (1.0 contains some changes to the Azure Resource Manager cmdlets but as I write this it’s in preview, the changes are nothing earth shattering).

You can find source code for the below in GitHub.

Firstly you need to create an Azure Resource Manager template – the one we’re using is here. I’ve not going to cover every bit of this as Microsoft have plenty of documentation and samples however it’s worth looking at this bit towards the end:

"resources": [
  {
    "name": "MSDeploy",
    "type": "extensions",
    "location": "[parameters('location')]",
    "apiVersion": "2014-06-01",
    "dependsOn": [
      "[concat('Microsoft.Web/sites/', parameters('siteName'))]"
    ],
    "tags": {
      "displayName": "ContentDeploy"
    },
    "properties": {
      "packageUri": "[concat(parameters('packageUrl'), parameters('sasToken'))]",
      "dbType": "None",
      "connectionString": "",
      "setParameters": {
          "IIS Web Application Name": "[parameters('siteName')]"
      }
    }
  }
]

This invokes the deployment of a Web Deployment package at URL specified by packageUrl. I’m using an Azure Storage Account for this and so to access the package do so we need a Shared Access Signature which we’ll see how to generate later.

As I want to automate everything I’ve also got a template for creating a storage account into which the package can be uploaded and you can see this template here.

I’m going to glue these two templates together with a Powershell script using the Azure Powershell cmdlets. To get started with this launch Azure Powershell and before running any scripts add your Azure subscription:

Add-AzureAccount

You’ll be prompted to log into your account.

The Powershell script I’m using to glue all this is called deploy.ps1 and you can find it here. I’m going to break this down section by section. Firstly we just deploy a bunch of fairly (if you know Azure) self explanatory settings – the hosting plan, resource group, website and storage account to create / use and the location of those resources:

# Deployment settings
$hostingPlanName = "ArmAngularSampleHostingPlan"
$resourceGroupName = "ArmAngularSample"
$storageAccountName = "armangularsamplesa"
$location = "West Europe"
$siteName = "ArmAngularSampleWebsite"

Then we optionally select an Azure subscription (if you only have one subscription you won’t need to do this) and switch Powershell into Azure Resource Manager mode.

# Configure Azure and the Powershell shell
# Select-AzureSubscription -SubscriptionId {{your-subscription-id}}
Switch-AzureMode AzureResourceManager

Now we create the resource group within which our templates are going to run:

# Create Azure Resoure Group
New-AzureResourceGroup -Name $resourceGroupName -Location $location -Force

Next we deploy our storage resources using Azure Resource Manager and then we get hold of the access key – we’ll need that in a minute. In such a small deployment this split of storage and service might seem a little naive but in more complex scenarios it’s fairly common to do a bunch of work on storage before deploying updated services (and vice versa):

# Deploy storage
New-AzureResourceGroupDeployment -ResourceGroupName $resourceGroupName `
                                 -Name AzmSampleStorageDeployment `
                                 -TemplateFile "storagedeploy.json" `
                                 -appStorageAccountName $storageAccountName 				 
$storageAccountKey = (Get-AzureStorageAccountKey -ResourceGroupName $resourceGroupName -Name $storageAccountName).Key1

Having done that we get grunt to build a distribution version of our Angular site and then we invoke MSBuild to create a deployment package from the folder. To do this MSBuild needs a project file and nearly every example you see will use a .csproj file – which is fine if you’re using Visual Studio but what if we’re not? Well you can also use a .publishproj file which you can find here. This file is absolute boilerplate – it will work for any folder and so you can copy and use mine for your own projects. We need to move this into the dist folder that grunt has created (you can use it elsewhere but as dist is cleaned each time this is easiest).

# Build deployment package
grunt build
cp .\website.publishproj .\dist\website.publishproj
C:\"Program Files (x86)"\MSBuild\14.0\bin\msbuild.exe .\dist\website.publishproj /T:Package /P:PackageLocation="." /P:_PackageTempDir="packagetmp"
$websitePackage = ".\dist\website.zip"

The setting of /P:PackageLocation=”.” causes the package to be dropped in the dist folder and the /P:_PackageTempDir=”packagetmp” causes MSBuild to use a packagetmp subfolder of dist to be used for temporary files – this can be useful as it’s easy to run into the “classic” Windows file length issues otherwise (it will use a temp location in AppData by default).

Having generated a package (called website.zip and located in the dist folder) we need to upload it to somewhere Azure can access. I’m using the storage account we created earlier. So that the Azure Resoure Manager incarnation of Web Deploy can access the package we generate a Shared Access Signature with read privileges:

# Upload packages
$storageCtx = New-AzureStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $storageAccountKey
if (-Not (Get-AzureStorageContainer -Context $storageCtx | Where-Object { $_.Name -eq "packages" }) ) {
    New-AzureStorageContainer -Name "packages" -Context $storageCtx -Permission Off
}
Set-AzureStorageBlobContent -File $websitePackage -Container "packages" -Blob website.zip -Context $storageCtx -Force
$uploadedPackage = "https://$storageAccountName.blob.core.windows.net/packages/website.zip"
$sasToken = New-AzureStorageContainerSASToken -Container "packages" -Context $storageCtx -Permission r
$sasToken = ConvertTo-SecureString $sasToken -AsPlainText -Force

And finally we deploy our services via the Azure Resource Manager template we saw earlier:

New-AzureResourceGroupDeployment -ResourceGroupName $resourceGroupName `
                                 -Name AnalyticsLiveServiceDeployment `
                                 -TemplateFile "azuredeploy.json" `
                                 -hostingPlanName $hostingPlanName `
                                 -location $location `
                                 -siteName $siteName `
                                 -packageUrl $uploadedPackage `
                                 -sasToken $sasToken

And that’s it. If you run .\deploy.ps1 (you’ll need to set your own unique storage account and website name) then as long as you have an Azure subscription this will deploy and you’ll be able to see the Yeoman template running in Azure.

Hopefully that’s helpful and will save you some of the pain that always seems to come with the territory of deployment – particularly where MSBuild is involved!

Powershell and External IP Addresses

While doing work with Powershell and Azure Resource Manager templates today I wanted to configure the Azure SQL Database firewall to allow my build machine access to configure schema and stored procedures. To reliably know what your Internet facing IP one fairly reliable method is to bounce it back from a remote server and so I stood up a super simple website on Azure with a REST API to return to you your IP. You can find it’s swagger endpoint here. All that’s really behind the get action is a simple return of a request header:

public string Get()
{
    return HttpContext.Current.Request.UserHostAddress;
}

Getting hold of it from Powershell is pretty simple, you can use the Invoke-WebRequest cmdlet as follows (and I’m also removing the double quotes surrounding the returned json string):

$ipAddress = (Invoke-WebRequest https://afwhatsmyip.azurewebsites.net/ip).Content
$ipAddress = $ipAddress.Substring(1,$ipAddress.Length-2)

Feel free to use the endpoint yourself, I’m not planning on taking it down.

Azure, PowerShell and DLL hell

It’s great that Azure is supported so extensively within Windows PowerShell – it really helps with automating builds and adopting a continual delivery / DevOps approach however eventually, in my experience, you’ll find yourself wanting to supplement what’s available from Microsoft with some Cmdlets of your own. Sure you can access .Net from PowerShell but sometimes C# is just more expressive and maintainable.

Of course this is simple enough to do and an example of implementing Cmdlets that use the Storage and Service Bus SDKs to create and configure resources can be found in the Application Support framework (here’s a direct link to the relevant project).

However I guarantee you that given the pace of change in the Azure space then at some point you’ll hit a DLL binding issue. Your Cmdlet will reference a different version of the Storage DLL or you’ll be using NewtonSoft.Json v7 while the Storage SDK is using v6 or vice versa and you’ll get one of those delightful can’t find DLL issues. And here I was thinking that at one point .Net was supposed to solve DLL hell!

Fortunately there’s a fairly simple solution to this: the PowerShell executable is a .Net application like any other and you can add binding redirects to a powershell.exe.config file. On a clean Windows machine this file doesn’t exist but you can create it easily enough using notepad or your text editor of choice. If you’re using a 32-bit operating system then PowerShell can be found in the folder:

C:\Windows\System32\WindowsPowerShell\v1.0

Or on a 64-bit operating system in:

C:\Windows\SysWOW64\WindowsPowerShell\v1.0

In that folder either create a new file or edit the existing one – you’ll probably need to run your text editor with Administrator privileges. As an example my current powershell.exe.config file looks like this:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <runtime>
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <dependentAssembly>
        <assemblyIdentity name="Microsoft.Data.OData" publicKeyToken="31bf3856ad364e35" />
        <bindingRedirect oldVersion="0.0.0.0-5.6.4.0" newVersion="5.6.4.0" />
      </dependentAssembly>
      <dependentAssembly>
        <assemblyIdentity name="Microsoft.Data.Edm" publicKeyToken="31bf3856ad364e35" />
        <bindingRedirect oldVersion="0.0.0.0-5.6.4.0" newVersion="5.6.4.0" />
      </dependentAssembly>
      <dependentAssembly>
        <assemblyIdentity name="Microsoft.Data.Services.Client" publicKeyToken="31bf3856ad364e35" />
        <bindingRedirect oldVersion="0.0.0.0-5.6.4.0" newVersion="5.6.4.0" />
      </dependentAssembly>
      <dependentAssembly>
        <assemblyIdentity name="System.Spatial" publicKeyToken="31bf3856ad364e35" />
        <bindingRedirect oldVersion="0.0.0.0-5.6.4.0" newVersion="5.6.4.0" />
      </dependentAssembly>
      <dependentAssembly>
        <assemblyIdentity name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed" />
        <bindingRedirect oldVersion="0.0.0.0-7.0.0.0" newVersion="7.0.0.0" />
      </dependentAssembly>
    </assemblyBinding>
  </runtime>
</configuration>

Hope that helps with any PowerShell binding issues.

Contact

  • If you're looking for help with C#, .NET, Azure, Architecture, or would simply value an independent opinion then please get in touch here or over on Twitter.

Recent Posts

Recent Tweets

Invalid or expired token.

Recent Comments

Archives

Categories

Meta

GiottoPress by Enrique Chavez