Tuesday, July 30, 2024

Azure DevOps Unable to Edit Wiki

I recently discovered project-wide repository policies in Azure DevOps so naturally I took advantage of them. The next time I tried to update the wiki in that project I was met with an error that read "Pushes to this branch is not permitted. Please continue with 'Edit in Repos' and use a pull request to update this branch. Click on 'Learn More' to find out how to edit in Repos."

A quick Google search lead me to a Microsoft "answer" that at least explained what was happening, but didn't give me any insight on how to fix it. Fortunately, the next search result did. Since that's just someone's blog and I have no guarantee they'll keep it up forever, I'm just going to reproduce their information here.

Instead, you can apply a security exception.  To do this you go to the Wiki home page and edit the "Wiki security", accessed through the menu for the wiki.

Here you have the same permissions dialogue you'd find against a repository.  Find an appropriate group (Contributors makes sense, but you might want to allow other users too), and then grant "Bypass policies when pushing" flag for that group.

Thursday, August 31, 2023

Beta Versioning Angular Libraries

Several years ago I worked for a company that was developing a distributed front-end application using Angular. We had issues getting everything to work together and I worked out some sort of solution to that problem using library versioning that was not semver. I'm currently working on a very similar problem (a distributed UI application built with Angular) and decided to try to figure out an actual good way of doing what I was trying to do back then. And I think I got it!

For starters, here's what I wrote up about that failed attempt all those years ago:

I was a contributor on a large application that used Angular for the front end. We decided that each piece would be developed separately by individual teams and then brought together as one massive monolithic application. One of the problems we encountered with this process was how we tested those individual packages prior to making them available for consumption in the application. We had more than one case where everything worked locally, but upon publication the whole application broke because of a small defect in one package. I designed a solution using VSTS (our build tool) to publish beta packages to our internal npm feed, then trigger a custom build of the application that consumed the newly published beta package. Using this new solution developers were able to test their published packages on a deployed test environment without changing the production ready packages and potentially crashing the application. Although this seems like something Semver could have handled (and was intended to handle), Semver was unfortunately not an option in our environment so we had to find a different solution.

Today's problem is pretty similar. We have multiple libraries that fully contain dedicated functionality and then a single presentation application that brings those libraries together. (I owe myself and you a separate blog post on creating a library in Angular that just works and is easy to change and validate before publishing, but this isn't that post.) For the sake of this article we're going to call the libraries tundra-ui-core, tundra-ui-payment, and tundra-ui-presentation.

tundra-ui-core

This library contains anything that is shared across two or more other libraries and/or tundra-ui-presentation.

tundra-ui-payment

This library contains all of the components, services, etc. responsible for accepting payments.

tundra-ui-presentation

This is the actual application, which will consume tundra-ui-core and tundra-ui-payment.

The Problem

As we work on tundra-ui-payment we're probably going to publish multiple versions of it to our internal feed to test it out within tundra-ui-presentation, but we don't want to waste a bunch of real version numbers doing that. We've decided we'll publish alpha versions for our developer testing and beta versions for our QA process. These will coincide with the branching strategy we already have in place for tundra-ui-presentation so that when a dev version of that project is published it automatically installs alpha versions of all libraries and when a QA version is published it automatically installs beta versions of all libraries.

Although this works, it created a major pain point because we had to manually increment the version of the library so that it included an alpha version, then change it to a beta version when we merged into the QA environment (I'll try to remember to document the branching strategy another time). It caused lots of extra commits with comments like "forgot to increment version" or "forgot to remove beta".

The Solution

It turns out that it's now possible to update the version of the library in the package.json file during the build process, then commit that change back into git without triggering another CI build. It took some doing to get things just right, but early testing is very promising and I couldn't wait to document it here.

I got started by following this guide, but I had to make some pretty significant changes so I'm going to document my whole process here. First off, we use Azure DevOps for our repositories, build and release pipelines, and to host our internal npm feed. We do not host our own instance of Azure DevOps so we do have access to the latest features.

Permissions

We're going to need two permissions setup in our Azure DevOps instance for each project that's going to host a repository that uses this method. If our Azure DevOps project is called Tundra then we'll need to make sure the associated user account for the Tundra project has these permissions. We'll also need to configure one of these permissions for each repository in each project. I know that's a pain, but the good news you only have to do it once per repository.

Open your project in Azure DevOps and navigate to Project Settings:


In the Project Settings pane, scroll down and select Repositories:


Choose the repository you want to add permissions for and navigate to the Security tab:


Scroll down to the Users section (expand it if necessary) and select the <Project> Build Service <Organization> user. For example, if our Tundra project was in the ArcticSoftware Organization we'd be looking for "Tundra Build Service (ArcticSoftware). Set the permissions below to "Allow" for this user.

  • Contribute

If you have branch policies active on any of the branches you'll be committing back to during the build process, you'll also need to enable these permissions:

  • Force push (rewrite history, delete branches and tags)
  • Bypass policies when pushing

The next step is granting permissions to the Artifacts feed to that same user account. In the left pane, choose Artifacts:

Click the settings cog near the upper right corner of the screen:

Navigate to the Permissions tab:

If you do not see the same user as above (Tundra Build Service (ArcticSoftware) in our example), click the "Add users/groups" button, select "Contributor", search for and select the user, click the "Save" button.

We now have the necessary permissions configured for this to work. We just have to create the build pipeline. I did this with yaml (I'm learning to love it) so I'm going to go through each step and explain what's happening and why I did it the way I did.

Build Pipeline

variables:
  isMain: $[eq(variables['Build.SourceBranch'], 'refs/heads/main')]
  isRelease: $[contains(variables['Build.SourceBranch'], 'release/')]
  isHotfix: $[contains(variables['Build.SourceBranch'], 'hotfix/')]
  isReadyForDeployment: $[eq(variables['Build.SourceBranch'], 'refs/heads/ready-for-deployment')]
  packageName: 'tundra/ui-payment'

We're setting up some variables we'll use in conditions later. The really basic explanation of our branching strategy is that we have main, ready-for-deployment (which is a holding branch for the period between the end of our sprint and the time the code gets deployed to production), release/* and hotfix/* branches for things that are in QA, and then everything else (story and task branches basically).

- checkout: self
  persistCredentials: true
  clean: true
  fetchDepth: 0
  lfs: true
  submodules: recursive

We have to checkout the repository. This is a key step that I initially overlooked and it caused me some problems later. You can find out more about the options specified here on your own. This is what I needed.

- task: Npm@1
  displayName: 'install dependencies'
  inputs:
    command: install
    verbose: false

We have an .npmrc file setup for our internal feed that's in the same director as this .yaml file so we don't need to specify any credentials or anything here to install dependencies (remember that we setup permissions on our internal feed on an earlier step).

- task: npmAuthenticate@0
  displayName: 'authenticate with internal npm feed'
  inputs:
    workingFile: '.npmrc'

This was another step that caused a headache for me. The native npm install command will authenticate internally, but in order to access the npm feed through Powershell (which we'll do in the next step) we have to authenticate explicitly, which modifies the .npmrc file.

- powershell: |
   $sourceBranch = "$(Build.SourceBranch)".replace("refs/heads/", "")
   $packageJson = Get-Content "./projects/tundra/**/package.json" | ConvertFrom-Json
   $baseVersion = $packageJson.version.split("-")[0];
   $allVersions = npm view $packageJson.name versions --json --silent
   $alphaVersions = ($allVersions | where{$_ -like "*" + $baseVersion + "-alpha*"})
   $newSubversion = 1
   if ($alphaVersions -ne $null) {
    $alphaVersions = $alphaVersions.trim()
    $lastAlphaVersion = 0
    if ($alphaVersions[1].length -eq 1) {
      $lastAlphaVersion = $alphaVersions.split(".")[3].replace("""", "").replace(",", "")
    } else {
      $lastAlphaVersion = $alphaVersions[$alphaVersions.length - 1].split(".")[3].replace("""", "")
    }
    $newSubversion = [int]$lastAlphaVersion + 1
   }
   $newVersion = $baseVersion + "-alpha." + $newSubversion
   if ($newVersion -ne $packageJson.version) {
    $packageJson.version = $newVersion
    git checkout $sourceBranch --quiet
    git config --global user.email "BuildPipeline@arcticsoftware.com"
    git config --global user.name "Build Pipeline"
    ConvertTo-Json $packageJson -Depth 2 | Out-File "./projects/tundra/**/package.json"
    git restore .npmrc
    git commit -am "[skip ci] Pipeline Modification: Package version: $newVersion"
   } else {
    echo "The new version would have been the same as the current version, so nothing was changed."
    exit 0
   }
  displayName: 'Append -alpha.<alpha version> to version number'
  condition: and(ne(variables.isMain, true), ne(variables.isRelease, true), ne(variables.isHotfix, true), ne(variables.isReadyForDeployment, true))

This is where the heavy lifting starts. This is a pretty big Powershell script (obviously) that's doing a lot of stuff. I'm going to break this one down pretty much line-by-line so it makes more sense. 

$sourceBranch = "$(Build.SourceBranch)".replace("refs/heads/", "")

The built-in variables Azure DevOps gives us are great, but they weren't quite what we needed for a future step. We want to checkout the branch that triggered this build, but if that branch is in a folder we need to specify the folder and the branch name. If we use $(Build.SourceBranchName) we don't get the folder and if we use $(Build.SourceBranch) we get the folder and branch name, but also "refs/heads/". All we're doing here is getting the folder and branch name and saving it for later.

$packageJson = Get-Content "./projects/connect/**/package.json" | ConvertFrom-Json

We're using a couple of cmdlets available to Powershell to store the contents of the package.json file in a variable so we can find the version number and manipulate it.

$baseVersion = $packageJson.version.split("-")[0];

We want to isolate the base package version and this line does that. We use Semver so by "base package version" I mean if the value in the version field is "1.0.1-alpha.1" we just want the "1.0.1" part saved for later. By splitting the entire value on the "-" and taking the first part, we get just that.

$allVersions = npm view $packageJson.name versions --json --silent

This is possibly the coolest part of the whole thing. We're going to our internal feed and retrieving every version that's ever been published of this specific package (tundra-ui-payment). The --json switch saves the value as JSON and the --silent switch just prevents the pipeline from displaying error messages if the package doesn't exist at all.

The view versions command appears to return the packages in publication order, which is exactly what we want. However, I can't find any confirmation that will always be the case so this is a potential spot for an issue to arise.

$alphaVersions = ($allVersions | where{$_ -like "*" + $baseVersion + "-alpha*"})

Once we have all of the versions, we need to filter that list to get only the alpha versions of this base version. If our base version is 1.0.1 then we want to get all of the versions that are "1.0.1-alpha*" where the * is a wildcard standing in for any number of characters. This will return "1.0.1-alpha.1", "1.0.1-alpha.sigma" or anything else that matches the pattern (though we expect it to always be in the format "1.0.1-alpha.<number>").

$newSubversion = 1
   if ($alphaVersions -ne $null) {
    $alphaVersions = $alphaVersions.trim()
    $lastAlphaVersion = 0
    if ($alphaVersions[1].length -eq 1) {
      $lastAlphaVersion = $alphaVersions.split(".")[3].replace("""", "").replace(",", "")
    } else {
      $lastAlphaVersion = $alphaVersions[$alphaVersions.length - 1].split(".")[3].replace("""", "")
    }
    $newSubversion = [int]$lastAlphaVersion + 1
   }

The next several steps all go together and are pretty dumb to have to do, honestly. First we're creating a new variable ($newSubversion) and initializing it to 1. Then we're going to try to figure out what the last alpha version was that was published to our feed. Unfortunately, the where cmdlet of Powershell sometimes returns an array and sometimes returns a single value. If there are multiple matches we'll get back an array containing all matches, but if there's only one match then we get back just that value as a string (which is also an array of characters).

If we try to split on the "." character and there was only one result we'll end up with either an error or useless data. I don't remember which, but it wasn't right. That's where the second if statement comes into play. (The first if statement just checks whether we got any matches; that is: whether there are currently any alpha versions that match this semver version.) We're checking the length of the second index in the array. If the "array" is an array of characters then the length of the second index will be one (because a character has a length of one), but if the "array" is actually an array of versions then the length of the second index will be longer. Either way we want to get the very last part of the version so we know what the next subversion should be. If the last version published was 1.0.1-alpha.3 then we want to isolate the 3 so we can increment it to 4 and make the next version 1.0.1-alpha.4.

$newVersion = $baseVersion + "-alpha." + $newSubversion

We're combining the base version with "-alpha." and the new subversion number to get the full new alpha version number.

if ($newVersion -ne $packageJson.version) {

We actually have to check whether the version has changed because there's a possibility it hasn't. For example, if the previous build didn't publish the package to the feed successfully then the version won't change this time around. If we just proceed as though the version has changed we actually end up with an error and our build fails.

$packageJson.version = $newVersion

This is changing the value in the JSON representation of the package.json that we still have saved in memory (and which we'll write back to disk in a future step).

git checkout $sourceBranch --quiet

Remember that branch variable from earlier? This is where we use it. Even though this pipeline was triggered by a specific branch, we have to checkout the branch so we can commit the modified package.json file back to it.

git config --global user.email "BuildPipeline@arcticsoftware.com"
git config --global user.name "Build Pipeline"

git needs to know a little about "us" in order to allow us to commit our changes. You can use whatever values you want right here, but you have to do this.

ConvertTo-Json $packageJson -Depth 2 | Out-File "./projects/connect/**/package.json"

Just like we used cmdlets to get the contents of the package.json file into memory, we're dumping the updated JSON back into the file. I don't know what the -Depth option does. I just know that 2 works for us.

git restore .npmrc

We want to revert the changes to the .npmrc file because it was modified earlier to include a key that we don't want to commit to git. This is a weird way to do this, but it's the only way that worked. I originally tried to just stage package.json, but that command failed when it ran in Azure DevOps. It was weird. This works, though. The only files that should have changed are .npmrc and package.json so just restoring .npmrc means we can commit everything else.

git commit -am "[skip ci] Pipeline Modification: Package version: $newVersion"

This is where that happens. We're staging and committing all of the remaining modified files in one command. The key part of this comment is "[skip ci]" which tells Azure DevOps not to trigger a build pipeline for this commit. If we don't include that, our changes will just keep triggering new builds forever (or until Azure DevOps gets tired of our shenanigans and quits).

} else {
    echo "The new version would have been the same as the current version, so nothing was changed."
    exit 0
   }

Finally, we have to exit the Powershell script with a success code (0) if the version hasn't changed. If we don't do this, the whole build will fail. 

condition: and(ne(variables.isMain, true), ne(variables.isRelease, true), ne(variables.isHotfix, true), ne(variables.isReadyForDeployment, true))

This condition just specifies that this Powershell script should only run when the source (triggering) branch is not main, ready-for-deployment, release/*, or hotfix/*. In other words, only do this step for story or task branches.

Man, that was a lot. It took me a while to get all the nuances figured out. There's a nearly duplicate script that runs for the beta builds. I'm not going to put it here because it really is nearly identical to this version. Just picture all the references to "alpha" changed to "beta" and the condition requiring the branch to be release/* or hotfix/*.

If the source branch is ready-for-deployment, however, we want to strip off the "alpha" and "beta" parts of the package, which does require a different Powershell script.

- powershell: |
   $sourceBranch = "$(Build.SourceBranch)".replace("refs/heads/", "")
   $packageJson = Get-Content "./projects/tundra/**/package.json" | ConvertFrom-Json
   git config --global user.email "BuildPipeline@arcticsoftware.com"
   git config --global user.name "Build Pipeline"
   git checkout $sourceBranch --quiet
   $packageJson.version = $packageJson.version.split("-")[0];
   ConvertTo-Json $packageJson -Depth 2 | Out-File "./projects/tundra/**/package.json"
   git restore .npmrc
   git commit -am "[skip ci] Pipeline Modification: Package version: $newVersion"
  displayName: 'Remove -alpha.<alpha version> and -beta.<beta version> from version number'
  condition: eq(variables.isReadyForDeployment, true)

A lot of this is the same so it doesn't warrant a line-by-line explanation. The big difference here is that instead of building a new version/subversion number, we're just writing the base version as the version. If the merge had "1.0.1-alpha.19" as the version, this will replace that with just "1.0.1".

Before we publish the package to the internal feed (I've omitted the build step, but you'll see it in the full file I post at the end) we want to push our changes back to the source branch of the triggering repository. That's one more Powershell script.

- powershell: |
   $sourceBranch = "$(Build.SourceBranch)".replace("refs/heads/", "")
   git checkout $sourceBranch --quiet
   git push https://$(System.AccessToken)@dev.azure.com/arcticsoftware/Tundra/_git/$(Build.Repository.Name) -q -f
  displayName: 'Push the merged source branch back into the repository without triggering another run of the pipeline'
  condition: and(succeeded(), ne(variables.isMain, true))

We have to get the source branch again and check it out again. That's just to make sure we're on the branch in case the earlier step didn't end up changing the version. The key piece is the git push line, which uses another built-in variable ($(System.AccessToken)) to authenticate to the repository. This is why we had to add the Contribute permission earlier. I've specified the -f (force) flag here because we have branch policies on some of these branches that we need to override.

- task: Npm@1
  condition: and(succeeded(), ne(variables.isMain, true), ne(variables.isRelease, true), ne(variables.isHotfix, true), ne(variables.isReadyForDeployment, true))
  displayName: 'publish the library to the internal npm feed with the alpha tag'
  inputs:
    command: custom
    workingDir: '$(Build.Repository.LocalPath)/dist/$(packageName)'
    verbose: false
    customCommand: 'publish --tag alpha'
    customRegistry: useFeed
    customFeed: 'this will be your custom feed ID'

This is really the last relevant step. There are three of these tasks, each with different conditions and tags. We're publishing our previously built package to our internal feed. Once again we have a condition to execute this step for all branches that are not main, ready-for-deployment, release/*, or hotfix/*. You can see on the customCommand line that we're using the --tag switch of the npm publish command to tag this package with "alpha". This enables us to install this version of the package in tundra-ui-presentation without overwriting the production-ready tundra-ui-payment package. The other two blocks replace "alpha" with "beta" and "latest", which is a keyword used by npm to identify the latest live version of the package.

The other half of this is that our tundra-ui-presentation build pipeline installs packages tagged as alpha whenever a task or story branch is built, installs packages tagged as beta whenever a release/* or hotfix/* branch is built. Otherwise, it installs the latest packages. As long as we're using semver correctly and allowing npm to get updated minor or patch versions, we shouldn't have to modify tundra-ui-presentation just to get an updated version of a library.

Here's the whole file. I did all the work so you (or future me) don't have to!

pool:
  name: Azure Pipelines
  vmImage: 'ubuntu-latest'

variables:
  isMain: $[eq(variables['Build.SourceBranch'], 'refs/heads/main')]
  isRelease: $[contains(variables['Build.SourceBranch'], 'release/')]
  isHotfix: $[contains(variables['Build.SourceBranch'], 'hotfix/')]
  isReadyForDeployment: $[eq(variables['Build.SourceBranch'], 'refs/heads/ready-for-deployment')]
  packageName: 'tundra/ui-payment'

# Trigger the pipeline for every branch
trigger:
  - main
  - release/*
  - hotfix/*
  - ready-for-deployment

steps:
- checkout: self
  persistCredentials: true
  clean: true
  fetchDepth: 0
  lfs: true
  submodules: recursive

- task: NodeTool@0
  displayName: 'Use Node 14.x'
  inputs:
    versionSpec: 14.x

- task: Npm@1
  displayName: 'install dependencies'
  inputs:
    command: install
    verbose: false

- task: npmAuthenticate@0
  displayName: 'authenticate with internal npm feed'
  inputs:
    workingFile: '.npmrc'

# 1. Get the current contents of package.json of the library
# 2. Find the latest alpha package on the feed with this version number (e.g. 1.1.0-alpha.*)
# 3. If this package is not on the feed or there are no packages on the feed with this version number and "alpha", use 1 as the alpha version
# 4. If there is a package with this version number and "alpha", get the highest alpha version and use the next number as this alpha version
# 5. If the new alpha version will be the same as the old version, do nothing and exit
# 6. If the new alpha version will be different from the old version, save the new version and write the updated JSON back to the package.json file
# 7. Commit the changes back to the branch that triggered the pipeline
# Note: This step applies to all branches that are not: main, ready-for-deployment, release/*, or hotfix/*
- powershell: |
   $sourceBranch = "$(Build.SourceBranch)".replace("refs/heads/", "")
   $packageJson = Get-Content "./projects/tundra/**/package.json" | ConvertFrom-Json
   $baseVersion = $packageJson.version.split("-")[0];
   $allVersions = npm view $packageJson.name versions --json --silent
   $alphaVersions = ($allVersions | where{$_ -like "*" + $baseVersion + "-alpha*"})
   $newSubversion = 1
   if ($alphaVersions -ne $null) {
    $alphaVersions = $alphaVersions.trim()
    $lastAlphaVersion = 0
    if ($alphaVersions[1].length -eq 1) {
      $lastAlphaVersion = $alphaVersions.split(".")[3].replace("""", "").replace(",", "")
    } else {
      $lastAlphaVersion = $alphaVersions[$alphaVersions.length - 1].split(".")[3].replace("""", "")
    }
    $newSubversion = [int]$lastAlphaVersion + 1
   }
   $newVersion = $baseVersion + "-alpha." + $newSubversion
   if ($newVersion -ne $packageJson.version) {
    $packageJson.version = $newVersion
    git checkout $sourceBranch --quiet
    git config --global user.email "BuildPipeline@arcticsoftware.com"
    git config --global user.name "Build Pipeline"
    ConvertTo-Json $packageJson -Depth 2 | Out-File "./projects/tundra/**/package.json"
    git restore .npmrc
    git commit -am "[skip ci] Pipeline Modification: Package version: $newVersion"
   } else {
    echo "The new version would have been the same as the current version, so nothing was changed."
    exit 0
   }
  displayName: 'Append -alpha.<alpha version> to version number'
  condition: and(ne(variables.isMain, true), ne(variables.isRelease, true), ne(variables.isHotfix, true), ne(variables.isReadyForDeployment, true))

# 1. Get the current contents of package.json of the library
# 2. Find the latest beta package on the feed with this version number (e.g. 1.1.0-beta.*)
# 3. If this package is not on the feed or there are no packages on the feed with this version number and "beta", use 1 as the beta version
# 4. If there is a package with this version number and "beta", get the highest beta version and use the next number as this beta version
# 5. If the new beta version will be the same as the old version, do nothing and exit
# 6. If the new beta version will be different from the old version, save the new version and write the updated JSON back to the package.json file
# 7. Commit the changes back to the branch that triggered the pipeline
# Note: This step applies to all release/* and hotfix/* branches
- powershell: |
   $sourceBranch = "$(Build.SourceBranch)".replace("refs/heads/", "")
   $packageJson = Get-Content "./projects/tundra/**/package.json" | ConvertFrom-Json
   $baseVersion = $packageJson.version.split("-")[0];
   $allVersions = npm view $packageJson.name versions --json
   $betaVersions = ($allVersions | where{$_ -like "*" + $baseVersion + "-beta*"})
   $newSubversion = 1
   if ($betaVersions -ne $null) {
    $betaVersions = $betaVersions.trim()
    $lastBetaVersion = 0
    if ($betaVersions[1].length -eq 1) {
      $lastBetaVersion = $betaVersions.split(".")[3].replace("""", "").replace(",", "")
    } else {
      $lastBetaVersion = $betaVersions[$betaVersions.length - 1].split(".")[3].replace("""", "")
    }
    $newSubversion = [int]$lastBetaVersion + 1
   }
   $newVersion = $baseVersion + "-beta." + $newSubversion
   if ($newVersion -ne $packageJson.version) {
    $packageJson.version = $newVersion
    git checkout $sourceBranch --quiet
    git config --global user.email "BuildPipeline@arcticsoftware.com"
    git config --global user.name "Build Pipeline"
    ConvertTo-Json $packageJson -Depth 2 | Out-File "./projects/tundra/**/package.json"
    git restore .npmrc
    git commit -am "[skip ci] Pipeline Modification: Package version: $newVersion"
   } else {
    echo "The new version would have been the same as the current version, so nothing was changed."
    exit 0
   }
  displayName: 'Append -beta.<beta version> to version number'
  condition: or(eq(variables.isRelease, true), eq(variables.isHotfix, true))

# 1. Get the current contents of package.json of the library
# 2. Create a new local branch using the build number
# 3. Remove "-alpha.<subversion>" and "-beta.<subversion>" from the version number
# 4. Reset the subversion to 1
# 5. Write the updated JSON back to the package.json file
# 6. Commit the changes back to the local branch created in step 2
# Note: This step only applies to the ready-for-deployment branch
- powershell: |
   $sourceBranch = "$(Build.SourceBranch)".replace("refs/heads/", "")
   $packageJson = Get-Content "./projects/tundra/**/package.json" | ConvertFrom-Json
   git config --global user.email "BuildPipeline@arcticsoftware.com"
   git config --global user.name "Build Pipeline"
   git checkout $sourceBranch --quiet
   $packageJson.version = $packageJson.version.split("-")[0];
   ConvertTo-Json $packageJson -Depth 2 | Out-File "./projects/tundra/**/package.json"
   git restore .npmrc
   git commit -am "[skip ci] Pipeline Modification: Package version: $newVersion"
  displayName: 'Remove -alpha.<alpha version> and -beta.<beta version> from version number'
  condition: eq(variables.isReadyForDeployment, true)

- task: Npm@1
  displayName: 'build library with production configuration'
  inputs:
    command: custom
    verbose: false
    customCommand: 'run lib:build -- --configuration=production'

# 1. Switch to the source branch of this build
# 2. Push the updated source branch back into the repository without triggering another pipeline ([skip ci])
# Note: This step applies to all branches except main
- powershell: |
   $sourceBranch = "$(Build.SourceBranch)".replace("refs/heads/", "")
   git checkout $sourceBranch --quiet
   git push https://$(System.AccessToken)@dev.azure.com/arcticsoftware/Tundra/_git/$(Build.Repository.Name) -q -f
  displayName: 'Push the merged source branch back into the repository without triggering another run of the pipeline'
  condition: and(succeeded(), ne(variables.isMain, true))

# Publish the alpha library to the internal npm feed
- task: Npm@1
  condition: and(succeeded(), ne(variables.isMain, true), ne(variables.isRelease, true), ne(variables.isHotfix, true), ne(variables.isReadyForDeployment, true))
  displayName: 'publish the library to the internal npm feed with the alpha tag'
  inputs:
    command: custom
    workingDir: '$(Build.Repository.LocalPath)/dist/$(packageName)'
    verbose: false
    customCommand: 'publish --tag alpha'
    customRegistry: useFeed
    customFeed: 'this will be your custom feed ID'

# Publish the beta library to the internal npm feed
- task: Npm@1
  condition: and(succeeded(), or(eq(variables.isRelease, true), eq(variables.isHotfix, true)))
  displayName: 'publish the library to the internal npm feed with the beta tag'
  inputs:
    command: custom
    workingDir: '$(Build.Repository.LocalPath)/dist/$(packageName)'
    verbose: false
    customCommand: 'publish --tag beta'
    customRegistry: useFeed
    customFeed: 'this will be your custom feed ID'

  # Publish the latest library to the internal npm feed
- task: Npm@1
  condition: and(succeeded(), eq(variables.isMain, true))
  displayName: 'publish the library to the internal npm feed with the latest tag'
  inputs:
    command: custom
    workingDir: '$(Build.Repository.LocalPath)/dist/$(packageName)'
    verbose: false
    customCommand: 'publish --tag latest'
    customRegistry: useFeed
    customFeed: 'this will be your custom feed ID'








Friday, July 14, 2023

Test-Driven Development Tutorial

 Many years ago when I first learned how to do test-driven development I used a guide I found on C# Corner and I really liked it. Over the years I've referred people to that same guide, but when I checked it out again recently I realized it has never been updated so it still refers to Visual Studio features that are no longer available, and uses NUnit instead of XUnit. I figured I'd give it a makeover and post it here, but you should definitely credit Dom Millar for this one. Everything after this point is lifted straight from that post, with the only changes being those for Visual Studio versions, NUnit -> XUnit, and maybe some syntax.


Introduction - Defining the Battlefield

This tutorial is a short introduction to using Test Driven Development (TDD) in Visual Studio 2022 (VS2022) with C#. Like most of my examples it's based on a game.

By completing this tutorial you will:

  • Get a taste of TDD through a series of small iterations
  • Learn how VS2022 provides TDD support through a number of features
  • Learn a number of C# features

CannonAttack is a simple text based game in which a player enters an angle and velocity of a cannonball to hit a target at a given distance. The game uses a basic formula for calculating trajectory of the cannonball and the player keeps taking turns shooting at the target until it has been hit. I won't go into TDD theory in any great detail now, but you should check out the number of great references to TDD including:

http://en.wikipedia.org/wiki/Test_driven_development

http://www.codeproject.com/KB/dotnet/tdd_in_dotnet.aspx

C# .NET 2.0 Test Driven Development by Matthew Cochran

http://msdn.microsoft.com/en-us/library/dd998313(VS.100).aspx

The following are the fundamental steps of a TDD iteration:

  • RED - take a piece of functionality and build a test for it and make it fail the test by writing a minimum amount of code (basically just get it to compile and run the test)
  • GREEN - write minimal code for the test to make it succeed
  • REFACTOR - clean up and reorganize the code and ensure it passes the test and any previous tests
     

In this tutorial we will be progressing through a number of iterations of the TDD cycle to produce a fully functional simple application. Each iteration will pick up one or more of the requirements/specs from our list (see The CannonAttack Requirements/Specs below). We won't test for absolutely everything and some of the tests are fairly basic and simplistic, but I am just trying to keep things reasonably simple at this stage

VS2022 and C# 4.0:

This tutorial covers the use of VS2022 and targets a number of features of C# 4.0 [editor's note: the current version of C# is 11.0, but this guide goes out of its way to introduce features of C# 4.0 so I left all references to C# 4.0 intact],

  • Generating stubs for TDD in VS2022
  • Test Impact View in VS2022 [editor's note: this feature disappeared sometime between VS2010 and VS2022 and the closest feature I've been able to find is Live Unit Testing, which is only available in Enterprise Edition]
  • Tuples
  • String.IsNullOrWhiteSpace method

There are many more features of C# 4.0 and we will be covering them in future tutorials.

What you need:

  • This is a C# tutorial so a background in C# would be very useful
  • VS2022

The CannonAttack Requirements/Specs:

The following is a combination of Requirements and Specifications that will give us some guide in terms of the application we are trying to build: 

  • Windows Console Application
  • Player identified by an id, default is set to a constant "Human"
  • Single player only, no multi-play yet
  • Allow player to set Angle and Speed of the Cannon Ball to Shoot at a Target
  • Target Distance is simply the distance of the Cannon to Target, and is created randomly by default but can be overridden
  • Angle and Speed needs to be validated (specifically not greater than 90 degrees and Speed not greater than speed of light)
  • Max distance for target is 20000 meters
  • Base the algorithm for the calculation of the cannon's trajectory upon the following C# code (distance and height is meters and velocity is meters per second):
    • distance = velocity * Math.Cos(angleInRadians) * time;
    • height = (velocity * Math.Sin(angleInRadians) * time) - (GRAVITY * Math.Pow(time, 2)) / 2;
  • A hit occurs if the cannon is within 50m of the target
  • Game text will be similar to the following:
    Welcome to Cannon Attack
    Target Distance: 12621m
    Please Enter Angle: 40
    Please Enter Speed: 350
    Missed cannonball landed at 12333m
    Please Enter Angle: 45
    Please Enter Speed: 350
    Hit - 2 Shots
    Would you like to play again (Y/N)
    Y
    Target Distance: 2078m
    Please Enter Angle: 45
    Please Enter Speed: 100
    Missed cannonball landed at 1060m
    Please Enter Angle: 45
    Please Enter Speed: 170
    Missed cannonball landed at 3005m
    Please Enter Angle: 45
    Please Enter Speed: 140
    Hit - 3 shots
    Would you like to play again (Y/N)
    N
    Thank you for playing CannonAttack

OK so now we are ready to code, let's go...

Iteration 1 - Creating the Cannon

Steps

  • Start Visual Studio
  • Click Create a new project
  • From the list of available project types, select Console App (be sure to select the C# version and not the VB version) and click Next
  • Call the application CannonAttack and click Next
  • Select .NET 7.0 (Standard Term Support) from the Framework dropdown and click Create
  • When the solution has loaded right click on the solution in Solution Explorer and select ADD -> NEW PROJECT
    • If you can't see the Solution Explorer expand the View menu and choose Solution Explorer
  • Select xUnit Test Project and click Next
  • Call the test project CannonAttackTest and click Next
  • Select .NET 7.0 (Standard Term Support) from the Framework dropdown and click Create
  • Rename UnitTest1.cs to CannonAttackTest.cs
  • If you see the following select YES
  • You should see the Solution Explorer as:
  • Open the code window for the CannonAttackTest.cs file and you should see the default test method
    • The [Fact] attribute above a method in this file indicates VS2022 will add this method as a unit test
  • Rename the default test method to CannonShouldHaveAValidId
  • Add the code var cannon = new Cannon(); to the body of the unit test method
  • Your updated unit test should look like this:
[Fact]
public void CannonShouldHaveAValidId()
{
    var cannon = new Cannon();
}
  • You'll notice an error because the type Cannon does not exist yet
  • Place your insertion point on Cannon and use the CTRL+. keyboard shortcut to get the Intellisense menu to display
  • Choose Generate new type... from the Intellisense menu
  • Select CannonAttack from the Project dropdown and choose the Create new file radio button
    • Leave the default new file name as Cannon.cs
  • At the top of the test file, make sure to add a using statement for the CannonAttack namespace
  • Update the test method to this:
[Fact]
public void CannonShouldHaveAValidId()
{
    var cannon = new Cannon();
 
    Assert.NotNull(cannon.Id);
}
  • Once again the solution will not compile because the Cannon type does not have an Id property
  • Use the CTRL+. keyboard shortcut to get the Intellisense menu to display
  • This time choose Generate property 'Id' from the Intellisene menu
    • This creates an auto property which will meet our needs for now
  • The solution should compile at this point
  • Save all projects and run all tests by using the Test Explorer
    • If you can't see the Test Explorer expand the Test menu and choose Test Explorer
    • To run all tests, click the button with a solid green arrow on top of a green arrow outline
  • The test failed, which is correct and expected as the first phase of TDD is RED - failed!!!
  • Now that we have a red test, let's get this test to pass
  • Select Cannon.cs in the CannonAttack project and make the following change to the Id property
public string Id
{
    get
    {
        return "Human";
    }
}
  • Run all tests again by using the Test Explorer
    • You can also repeat the last test by using the keyboard shortcut CTRL+R, l
  • All tests should now be passing and your Test Explorer should look similar to this:
  • We have just completed the second stage of a TDD cycle, GREEN - Pass!!!
[editor's note: I've added the next few steps so that unit tests are written to verify the default value is used unless a value is provided]
  • Add two more unit tests called CannonShouldUseDefaultValueForIdWhenNoValueIsProvided and CannonShouldUseProvidedValueForIdWhenAValueIsProvided that looks like this:
[Fact]
public void CannonShouldUseDefaultValueForIdWhenNoValueIsProvided()
{
    var cannon = new Cannon();
 
    Assert.Equal("Human", cannon.Id);
}
[Fact]
public void CannonShouldUseProvidedValueForIdWhenAValueIsProvided()
{
    var cannon = new Cannon
    {
        Id = "Test Value"
    };
 
    Assert.Equal("Test Value", cannon.Id);
}
  • The second test won't compile right now, and the first test should pass
  • Modify the Cannon class to look like this so the code compiles and the third test fails:
namespace CannonAttack
{
    public class Cannon
    {
        public Cannon()
        {
        }
 
        public string Id
        {
            get
            {
                return "Human";
            }
            set { }
        }
    }
}
  • We now have three tests, two that pass and one that fails; it is time to refactor our code
  • Make the following changes to the Cannon class so that all three tests pass:
namespace CannonAttack
{
    public sealed class Cannon
    {
        private readonly string CANNONID = "Human";
        private string CannonId;
 
        public string Id
        {
            get
            {
                return string.IsNullOrWhiteSpace(CannonId) ? CANNONID : CannonId;
            }
            set
            {
                CannonId = value;
            }
        }
    }
}

We have made this class sealed so that it is not inherited by anything. Also, we have added a readonly string to store a default ID if not set by the user. I am going to use runtime constants (readonly) because they are more flexible than compile time constants (const) and if you are interested check out Bill Wagner's book (effective C# - 50 Ways to Improve your C#) for further details.

Let's run the tests again. Again they should compile and pass because although we have made some changes to the code, we should not have impacted the tests and this is an important part of the Refactor phase. We should make the changes we need to make the code more efficient and reusable, but it is critical that the same tests that we made pass in the Green phase still pass in the Refactor phase.

The refactoring is complete. Now for ITERATION 2 of the CannonAttack project.

Iteration 2 - One Cannon, and only one Cannon - Using the Singleton Pattern.

Like the previous iteration we will pick an element of functionality and work through the same sequence again RED->GREEN->REFACTOR. Our next requirement is to allow only a single player. Given that we can allow 1 player we really should only use one instance, let's create a test for only one instance of the cannon object. We can compare two objects to ensure they are pointing at the same instance like (obj == obj2).

  • Add a new test beneath the three we already have in CannonAttackTest.cs that looks like this:
[Fact]
public void CannonCannotBeCreatedMoreThanOnce()
{
    var cannon = new Cannon();
    var cannon2 = new Cannon();
 
    Assert.Equal(cannon, cannon2);
}
  • Run all tests using Test Explorer and you'll see the new test fail, so we are at stage 1 of TDD again (RED)
  • The reason that this failed is we have created two different instances. What we need is the singleton pattern to solve our problem. I have a great book on patterns called HEAD FIRST DESIGN PATTERNS if you want to know more about design patterns it's a great start - sure its Java but the code is so close to C# you should not have any real problems.
  • We are going to use the singleton pattern to meet the requirement of a single player. Really we don't want multiple instances of cannons hanging around - 1 and only 1 instance is needed. Insert the following Singleton code below the property for the ID.
namespace CannonAttack
{
    public sealed class Cannon
    {
        private readonly string CANNONID = "Human";
        private string CannonId;
        private static Cannon cannonInstance;
 
        private Cannon()
        {
        }
 
        public static  Cannon GetInstance()
        {
            if (cannonInstance == null)
            {
                cannonInstance = new ();
            }
 
            return cannonInstance;
        }
        public string Id
        {
            get
            {
                return string.IsNullOrWhiteSpace(CannonId) ? CANNONID : CannonId;
            }
            set
            {
                CannonId = value;
            }
        }
    }
}
  • If we try to run the tests we won't compile because the cannon object can't be created with Cannon cannon = new Cannon(); So make sure that we use Cannon.GetInstance() instead of new Cannon(). The four test methods should now look like:
[Fact]
public void CannonShouldHaveAValidId()
{
    var cannon = Cannon.GetInstance();
 
    Assert.NotNull(cannon.Id);
}
 
[Fact]
public void CannonShouldUseDefaultValueForIdWhenNoValueIsProvided()
{
    var cannon = Cannon.GetInstance();
 
    Assert.Equal("Human", cannon.Id);
}
 
[Fact]
public void CannonShouldUseProvidedValueForIdWhenAValueIsProvided()
{
    var cannon = Cannon.GetInstance();
    cannon.Id = "Test Value";
 
    Assert.Equal("Test Value", cannon.Id);
}
 
[Fact]
public void CannonCannotBeCreatedMoreThanOnce()
{
    var cannon = Cannon.GetInstance();
    var cannon2 = Cannon.GetInstance();
 
    Assert.Equal(cannon, cannon2);
}
  • Run the tests again

This time they pass GREEN so time to refactor. We are going to change our Singleton code because although the code works (and is pretty much 100% the same as the singleton code in the HEAD FIRST DESIGN PATTERNS Book) it is not thread safe in C# (see http://msdn.microsoft.com/en-us/library/ff650316.aspx) So we replace the original singleton code with:

namespace CannonAttack
{
    public sealed class Cannon
    {
        private readonly string CANNONID = "Human";
        private string CannonId;
        private static Cannon cannonInstance;
        static readonly object padlock = new object();
 
        private Cannon()
        {
        }
 
        public static  Cannon GetInstance()
        {
            lock(padlock)
            {
                if (cannonInstance == null)
                {
                    cannonInstance = new ();
                }
 
                return cannonInstance;
            }
        }
 
        public string Id
        {
            get
            {
                return string.IsNullOrWhiteSpace(CannonId) ? CANNONID : CannonId;
            }
            set
            {
                CannonId = value;
            }
        }
    }
}

The block inside the lock ensures that only one thread enters this block at any given time. Given the importance of determining if there is an instance or not, we should definitely use the lock.

  • Run the tests again and they should all still pass

That's the end of the second iteration and I think you should be getting the hang of it by now, so lets get onto the 3rd iteration.

Iteration 3 - Angling for something...

We will add another couple of test methods. This time we want to ensure that an incorrect angle (say 95 degrees) will not hit. So we need a Shoot method and a return type (lets keep it simple and make it a Boolean for now).

  • Add the following test below the last test:
[Fact]
public void CannonShouldNotFireWithAnAngleTooLarge()
{
    var cannon = Cannon.GetInstance();
 
    Assert.False(cannon.Shoot(95, 100));
}
 
[Fact]
public void CannonShouldNotFireWithAnAngleTooSmall()
{
    var cannon = Cannon.GetInstance();
 
    Assert.False(cannon.Shoot(-1, 100));
}
  • Of course the compiler will complain and we get it to compile by putting the insertion point on Shoot and using the CTRL+. keyboard shortcut to view the Intellisense menu and choosing Generate method 'Shoot'
  • Run all tests from Test Explorer to see that all of the previous tests pass, but the two new tests fail
  • Open Cannon.cs in the CannonAttack project and replace the generated Shoot method with:
public bool Shoot(int angle, int velocity)
{
    if (angle > 90 || angle < 0)
    {
        return false;
    }
 
    return true;
}
  • Run all of the tests again and you'll see they all pass so we're back to the GREEN stage again so it's time to refactor
  • We need to add two extra readonly integers to the class
    • We will add them as public so they are exposed to the console application eventually
public static readonly int MAXANGLE = 90;
public static readonly int MINANGLE = 1;
  • Now refactor the Shoot method to use the new constants
public (bool, string) Shoot(int angle, int velocity)
{
    if (angle > MAXANGLE || angle < MINANGLE)
    {
        return (false, "Angle Incorrect");
    }
 
    return (true, "Angle OK");
}

We have changed the interface of the method so it returns a Tuple indicating if its a hit (BOOL) and also a message (STRING) containing the display text. The Tuple is a feature of C# 4.0. used to group a number of types together and we have used it in conjunction with the type inference of the var to give a neat and quick way to handle the messages from our shoot method. See the following article for further information:http://www.davidhayden.me/2009/12/tuple-in-c-4-c-4-examples.html

  • To handle the change to the Shoot method, we change our two newest tests to:
[Fact]
public void CannonShouldNotFireWithAnAngleTooLarge()
{
    var cannon = Cannon.GetInstance();
 
    var shot = cannon.Shoot(95, 100);
 
    Assert.False(shot.Item1);
}
 
[Fact]
public void CannonShouldNotFireWithAnAngleTooSmall()
{
    var cannon = Cannon.GetInstance();
 
    var shot = cannon.Shoot(0, 100);
 
    Assert.False(shot.Item1);
}

[editor's note: in newer versions of C# you can further refactor this, as shown below]

[Fact]
public void CannonShouldNotFireWithAnAngleTooLarge()
{
    var cannon = Cannon.GetInstance();
 
    var (shotStatus, _) = cannon.Shoot(95, 100);
 
    Assert.False(shotStatus);
}
 
[Fact]
public void CannonShouldNotFireWithAnAngleTooSmall()
{
    var cannon = Cannon.GetInstance();
 
    var (shotStatus, _) = cannon.Shoot(0, 100);
 
    Assert.False(shotStatus);
}

At this point we can refactor our tests, which is an important (though frequently overlooked) part of TDD. Since we're always going to create the Cannon instance the same way, we can move that into the constructor of our CannonAttackTest class so it is automatically run before each test.

The entire CannonAttackTest class now looks as you see below. I'm going to stop here and work on a Part 2 later because this post is long and I have other stuff I need to get done today.

using CannonAttack;
 
namespace CannonAttackTest
{
    public class CannonAttackTest
    {
        private readonly Cannon _cannon;
 
        public CannonAttackTest()
        {
            _cannon = Cannon.GetInstance();
        }
 
        [Fact]
        public void CannonShouldHaveAValidId()
        {
            Assert.NotNull(_cannon.Id);
        }
 
        [Fact]
        public void CannonShouldUseDefaultValueForIdWhenNoValueIsProvided()
        {
            Assert.Equal("Human", _cannon.Id);
        }
 
        [Fact]
        public void CannonShouldUseProvidedValueForIdWhenAValueIsProvided()
        {
            _cannon.Id = "Test Value";
 
            Assert.Equal("Test Value", _cannon.Id);
        }
 
        [Fact]
        public void CannonCannotBeCreatedMoreThanOnce()
        {
            var cannon2 = Cannon.GetInstance();
            Assert.Equal(_cannon, cannon2);
        }
 
        [Fact]
        public void CannonShouldNotFireWithAnAngleTooLarge()
        {
            var (shotStatus, _) = _cannon.Shoot(95, 100);
 
            Assert.False(shotStatus);
        }
 
        [Fact]
        public void CannonShouldNotFireWithAnAngleTooSmall()
        {
            var (shotStatus, _) = _cannon.Shoot(0, 100);
 
            Assert.False(shotStatus);
        }
    }
}