Automate UI testing for Dynamics 365 using EasyRepro and a VSTS Build Agent

Why test my Dynamics 365 UI?

We know how configurable Dynamics 365 is; so much so that no two Dynamics organisations are created equal. Companies work in different ways, leverage varying business processes and come in many shapes and sizes. This is great, because Dynamics is flexible enough to cope, and the developer community loves tackling new projects.

However, these very same companies are always changing, and so too are their Dynamics 365 organisations. This is where the developers come in; they configure each Dynamics organisation to meet their customer's needs, and this process is inevitably ongoing. Reconfiguring and adding to an ever evolving system comes with risk:

• Do these new configurations meet the customer's specification?
• Have these changes introduced regression to the existing system?

If your previous Dynamics customisations have tests written for them, these questions can all be answered during development.

As well as mitigating risks, UI tests come with benefits to enhance the usability of the Dynamics 365 organisation being configuring. This means happy stakeholders all around, from developers to end users.

One core benefit is enforcing form style guidelines. Should section headers all be in capital letters? Should the Created On field always be visible at the bottom of every form? Should form headers always contain four fields? Enforce it with UI testing!

What's automated UI testing?

UI tests that aren't automated, consist of a team reading through a list of behavioural expectations and then clicking through the Dynamics organisation manually and asserting a pass or fail to each expectation. These tests can be very click intensive and time-consuming, time that could be better spent by the QA team testing complex features end-to-end.

Automated UI testing is getting a computer to run these tests for you. Selenium is a popular tool built for this purpose: it automates web browsers allowing for websites to be tested by interacting with DOM elements.

Dynamics 365-specific commands have been written for us by Microsoft, in the open source project EasyRepro. In the project's own words:

The purpose of this library is to provide Dynamics customers the ability to facilitate automated UI testing for their projects. These API's provide an easy to use set of commands that make setting up UI testing quick and easy. The functionality provided covers the core CRM commands that end users would perform on a typical workday and working to extend that coverage to more functionality.

Ok, how do I test my Dynamics 365 UI?

Using EasyRepro! See its GitHub page for examples spanning forms, charts, navigation, dashboards and more. I've added an example here below which demonstrates a basic test of navigating through Dynamics and opening a contact form:

Notice how readable the functions of XrmBrowser are such as SwitchView and OpenRecord. This is how good tests should be: human readable and descriptive similar to a design document.

Running UI tests locally takes ages, how do I automate it?

Yes, you'll notice that when your UI tests are run (presumably in Visual Studio), a browser window appears as if by magic and your mouse starts moving to sign into Dynamics, and begin running each test individually. If only these tests could run else where and you could carry on with your work..

Well they can, at least, ever since this commit to GitHub which adds headless compatibility to EasyRepro. Headless Chrome is still just Chrome, but without the UI shell. This is necessary because VSTS Build Agents, which we'll be using to act as a remote machine to run our automated tests, don't have a UI for us to use. Chrome without Chrome!

So with these tools, we can configure a headless Chrome browser to run in a VSTS Build Agent, while we get on with other work.

How do I configure my VSTS Build task?

1. Ensure your code is checked into VSTS along with your test code project
2. Restore your NuGet packages and compile your code (as shown in the image below)
3. Install Chrome silently (without installation prompts) on the build agent

4. Run your UI tests by referencing their test assemblies

You'll also need to configure your EasyRepro Test Options

Your EasyRepro test methods require an XrmBrowser object, which itself requires a BrowserOptions object.

So, create your TestSettings class and specify Chrome as BrowserOptions.BrowserType:

Then pass your TestSettings.Options to XrmBrowser in your test classes:

And you're done, you're now equipped to automate your UI tests for Dynamics 365 both locally and from a VSTS Build Agent, using Microsoft's EasyRepro and Selenium.

If your current place of work doesn't make use of tests or UI testing, you now also have a basis of discussion to convince others that UI testing is worthwhile! (As if the incidence of regression bugs wasn't enough!)