Selenium is commonly used for E2E testing or Acceptance testing, but now there’s a new kid on the block, Cypress. Cypress does the same browser-centric testing that Selenium does, but it runs in your code loop giving you more access and control over what you’re running in your browser. And everything is written in JavaScript. This link has a list of the major benefits they espouse for this open source project.
In this post I’ll be converting the Selenium tests in the Onion-DevOps-Architecture project to Cypress.
The Existing C# Test
In the OnionDevOpsArchitecture solution’s \src\AcceptanceTests\GetAllExpenseReportsTester.cs file, there is a function that is run a handful of times for the acceptance tests during Azure DevOps CI builds (these tests are not run by the local build script). The code below shows the test navigating to the page, adding a new expense report, and verifying that it was added.
public void ShouldBeAbleToAddNewExpenseReport(string expenseReportNumber)
{
void ClickLink(string linkText)
{
_driver.FindElement(By.LinkText(linkText)).Click();
}
void TypeText(string elementName, string text)
{
var numberTextBox = _driver.FindElement(By.Name(elementName));
numberTextBox.SendKeys(text);
}
Console.WriteLine($"Navigating to {_appUrl}");
_driver.Navigate().GoToUrl(_appUrl + "/");
_driver.Manage().Window.Maximize();
TakeScreenshot($"{expenseReportNumber}-Step1Arrange");
ClickLink("Add New");
TypeText(nameof(ExpenseReport.Number), expenseReportNumber);
TypeText(nameof(ExpenseReport.Title), "some title");
TypeText(nameof(ExpenseReport.Description), "some desc");
TakeScreenshot($"{expenseReportNumber}-Step2Act");
_driver.FindElement(By.TagName("form")).Submit();
TakeScreenshot($"{expenseReportNumber}-Step3Assert");
var numberCells = _driver.FindElements(
By.CssSelector($"td[data-expensereport-property=\"{nameof(ExpenseReport.Number)}\"][data-value=\"{expenseReportNumber}\"]"));
numberCells.Count.ShouldBeGreaterThan(0);
numberCells[0].Text.ShouldBe(expenseReportNumber);
}
E2E Testing with Cypress
Installation and Setup
Since the OnionDevOpsArchitecture solution doesn’t currently use npm (their recommended method from here), I used their Direct Download method, but that left much of the configuration to me. So, being a lazy developer, I took the easy way and did npm install cypress --save-dev
Being new to Cypress, you’ll want to open the UI to play with it, perhaps running some of the examples. To run it, do this ./node_modules/.bin/cypress open. The first time it runs, it will create an examples folder, which you’ll want to .gitignore or eventually remove.
Then in the UI you can run the tests by clicking Run all specs button in the upper right. I won’t go into using the cool UI that allows you to see everything as it happens, time travel, etc. since they have some good doc on their site
In addition to the examples they install, they also have a recipes repo with more goodies.
I edited the package.json file to add a script to run Cypress headless so it can be invoked from the command line and in the CI/CD pipelines.
"scripts": {
"integration": "cypress run"
},
To avoid hardcoding the base URL in the tests, I edited the cypress.json file and set it there, which is part of their best practices.
{
"baseUrl": "http://localhost:51345/"
}
New Test
Now for the port of the existing C# test above to Cypress into src\cypress\integration\GetAllExpenseReports.spec.js. You’ll see this looks like a typical scripting test file with describe and it keywords for running tests. You can also run code before or after your tests, and add custom commands to Cypress (like login()).
/// <reference types="Cypress" />
describe('Onion Expense Tests', () => {
const expenseReportNumbers = ["000001", "000010", "000100", "001000", "010000", "100000"]
expenseReportNumbers.forEach((expenseReportNumber) => {
it(`'ShouldBeAbleToAddNewExpenseReport ${expenseReportNumber}`, () => {
cy.visit('/')
cy.screenshot(`${expenseReportNumber}-Step1Arrange`)
cy.contains('Onion DevOps Architecture')
cy.contains('Add New').click()
cy.get('#Number').type(expenseReportNumber)
cy.get('#Title').type('some title')
cy.get('#Description').type('some desc')
cy.screenshot(`${expenseReportNumber}-Step2Act`)
cy.contains('Save').click()
cy.screenshot(`${expenseReportNumber}-Step3Assert`)
cy.get(`td[data-expensereport-property="Number"][data-value="${expenseReportNumber}"]`)
})
})
})
Everything uses the global cy object, and makes extensive use of the fluent coding model by chaining commands together such as get->type, or get->click. The code is pretty straightforward, following the C# step-by-step, visiting the site, adding an expense report, and verifying it.
When running the headless test (npm run integration), Cypress will log the test results and names of any screen shot and and video files it created. The last bit of the output will look like this and $LASTEXITCODE will be 0 if everything passed.
And in a screenshots/GetAllExpenseReports.spec.js folder will be a bunch of pngs with names we specified. Note that if there is an error Cypress will, by default, take a screenshot at the time of the error, named along the lines of Onion Expense Tests -- ShouldBeAbleToAddNewExpenseReport 000001 (failed).png Also by default it creates video for the tests (GetAllExpenseReports.spec.js.mp4).
Using the Cypress UI
So, I lied about not writing about the UI, but I would like to briefly mention it in the terms of creating and debugging tests. From the root npm folder, open the UI with ./node_modules/.bin/cypress open and it will list all the specs in the project that can be run with a click.
The cy.pause() and cy.debug() commands will pause a running test so you can interact with it. Also when a test finishes, you have an active browser window in the UI and can open the developer tools from the menu or Ctrl+Shift+I. When you open a script it will re-run the test every time you save it, so you can use it to incrementally build your test making sure each step works.
Although they recommend using a robust method to identify a controls (which this code does not use), you can find the selector for any item on the screen with the UI. At the end of a test run you can use the target icon to open the “Selector Playground” that will show the selector for any given item on the page to use with cy.get()
E2E Testing Tools – Final Thoughts
I hope this gave you some food for thought about E2E testing tools. For this rather simple case, I found Cypress to be easier and more intuitive to use than Selenium. Before making a decision to switch, I’d recommend playing around with more of the features.
Although they say it’s faster than Selenium, running the six C# Selenium tests from VS2017 took 12 seconds, but the headless Cypress tests took 29-37 seconds, and that is just the test time without overhead. Removing the cy.screenshot() calls got that time to 9 seconds. Since Cypress will create screenshots on error anyway, looks like avoiding them is a good practice.
Originally published May 28, 2019. Information refreshed July 6, 2022.
