Categories
Programming

Cypress E2E testing best practices + gotchas

Cypress E2E testing best practices + gotchas

Cypress is a new testing framework that we just moved to, so it is common to have growing pains and learning the ins and outs as we work with it more. Here are some common practices that we found to prevent any flakiness in the tests.

  • Always clean up any entities in the test suite before and after the test suite

E.g in the `beforeEach() or before()`

before(() => {

   cy.serviceVersion(‘DELETE’, ‘service-version-new’).then(() => {

     cy.servicePackage(‘DELETE’, ‘service-package-new’)

   })

 })

Also make sure to delete it in the `afterEach() or after()` as well.

afterEach(() => {

   cy.serviceVersion(‘DELETE’, ‘service-version-new’).then(() => {

     cy.servicePackage(‘DELETE’, ‘service-package-new’)

   })

 })

This will minimize any lingering entities and make the tests more independent and deterministic

  • Make sure the element you are selecting has a unique selector

Cypress will error if it finds multiple elements selected, so make sure that the element that you are selecting has a unique data-testid or selector so that it doesn’t potentially return more than one value.

       cy.get(‘[data-testid=”i-am-very-unique”]’).click()

  • Make sure you assert on elements that are unique to the page you are testing

This goes in hand with point 2, but the Cypress test environment is very eager so as soon as it finds an element matching the selector, it will act on it, even if it hasn’t navigated to the proper page yet – it will attempt to assert it on the current page if there is an matching element. So make sure the selector is unique to the page you are testing.

     cy.get(‘[data-testid=”i-only-exist-on-the-page-you-want-to-test”]’).click()

  • Be careful of using `.contains()` – it may not work the way you expect it to

In cypress, .contains() will yield a DOM element which you can chain other commands to, but it will operate under the scope of that selected DOM element. This can cause undesired assertions if we are inadvertently chaining a lot of assertions. If we just want to make a straight assertion on the DOM, we should use .should(‘contain’) instead

cy.get(‘foo’).contains(‘bar’) // everything chained after this will operate under the foo element that contained bar

cy.get(‘foo’).should(‘contain’, ‘bar’) // a straight assertion that doesn’t yield unexpected side effects when chaining

  • Try to assert on an element on the page before navigating or clicking somewhere

In Cypress, it will eagerly try to click or navigate as soon as it finds the element on the page before data is fully rendered – this leads to unexpected consequences. Instead of a .wait we can also assert on something in the page which will cause Cypress to wait until that element loads before the next step. 

cy.get(‘[data-testid=”Plugin-card”] .empty-state-content’).should(‘exist’) // this causes Cypress to wait until the KEmptyState has loaded which was a result of an XHR request returning an empty dataset. This ensures that Cypress waited for that request to finish before moving to next step.

cy.get(‘[data-testid=”entity-button”]’).click() // and then click on the entity button. Had we not done the previous assertion, Cypress would have clicked this immediately and any props or params that we got from any XHR requests would not have been set correctly.

 

Categories
Programming

Unit Tests in Angular 2

Recently at my new company, Spigit, I’ve started on working on unit testing. Now, previously I’ve done unit testing and end to end testing using NighwatchJS at Walmart Labs.
And now with my new project, I have to use Angular 2.

After working with Angular 2 for a few months now and ReactJS last year, my feelings towards the two are: ReactJS is more flexible and lightweight, and Angular 2 comes with more out of the box features. It’s kind of like the old BackboneJS vs Angular 1 argument. Do you want a more lightweight, flexible framework, or do you want to go all in with one?

With ReactJS, you only have the “view” layer and you still have to use Flux or Redux to actually make service or API calls. Then you have to add Flow for type checking.
With Angular 2, you have that all out of the box with a comprehensive library + TypeScript which has built in type checking and is ES6 natively (but not exactly the same as ES6 as I found out later on). But Angular 2 is more heavy and you might not be using its full capability, so it’s not as flexible as ReactJS. It’s just like how Angular 1.x provided everything out of the box whereas BackboneJS required you to use UnderscoreJS, jQuery etc on top of it (which some companies expanded upon more with MarionetteJS, HandlebarsJS, EpoxyJS, ThoraxJS, Lodash etc adding various capabilities onto Backbone).

Out of the two, ReactJS has a higher learning curve I feel. Having the markup in the JS files (JSX) and dealing with the component lifecycle and passing in props is a very different way of coding than the past.
Angular 2 (and Angular 4, etc) is also very different than AngularJS (that’s Angular 1.x btw), but its not quite the same chasm of difference that React is. AngularJS devs can migrate to Angular 2 fairly easily as there’s a guide for it, it still uses templates just like Backbone or AngularJS and you still have things like directives and pipes, but now everything is a component (let’s face it we live in an ES6 world now and you should be familiar with ES6 and dealing with importing/exporting components instead of the old MVC structure). Other AngularJS functions like `$scope.apply` have equivalents in Angular 2 like `changeDetectorRef.detectChanges()` But still out of the two new age popular frameworks, Angular 2 is easier to pick up. I technically prefer VueJS to either of them but it’s not quite as popular yet.

So my last 6 months with Walmart Labs I was mostly doing end to end testing using the NightwatchJS framework which uses the Selenium driver to automate browser testing. It puts less pressure on manual QA testing. We did testing on four major browsers: IE11, Firefox (last 3 versions), Chrome (last 3 versions), and Safari. We found some quirks with testing in certain browsers. For example on IE11, we would have to use clearValue twice to actually clear a value.


// For IE11
client.clearValue('input[type=text]');
client.clearValue('input[type=text]');

Which was weird, and also oddly enough on Firefox, clicking a button wouldn’t work unless the button was actually in the viewport. So we have to scroll down to it first.


// For Firefox
client.moveToElement('#main', 10, 10);
client.click("#main ul li a.first");

Very odd issues. But most people use Chrome as the standard and we only noticed the other failures since we integrated our CI with Admiral and SauceLabs.

Out of E2E testing and unit testing, Both are important, but when choosing which one to start out with, I feel unit testing is the more fundamental one to start out with. If you have a great QA team already, then you can make do without E2E testing since all it really does is automate integration testing. Unit tests also help catch regressions but on a more fundamental component level, whereas E2E catches regressions on the UI level. QAs can actually write E2E tests. But QAs usually don’t write unit tests; that’s the developers job since they know their own code the best.

ReactJS uses Mocha for unit testing (and Sinon for spies), while Angular uses Jasmine which come with their own spies. So starting out on Angular 2’s testing guide, its a big long ass intimidating page to read, and there’s tons of testing functions in there. I recommend reading the first half as the latter half are for very niche scenarios. There’s also a great plnkr link at the bottom of the page which contains ALL the unit tests in their whole, and I recommend looking at that because just showing code snippets is really not good enough. Sometimes you might be missing imports or stubs or declarations that you don’t see in the code snippets.

One of issues I ran into on the guide is using stubs vs mock classes. When we are testing components, we have to choose between using one of those if we have a function that reaches out to a service. We don’t want to call the real service. So, if we want more flexibility we can use a mock class, but the problem is Angular’s example doesn’t really work for me… when I put in the mock class as a provider using `useClass` and then using `overrideComponents`, Angular complains about it in my IDE.. I posted a question on stack overflow about it, but no one seems to reply.
So, I’m stuck using stubs instead, which is fine if the component doesn’t do too many service calls, but you lose out on the power of mock classes, but oh well. With Stubs we have to use `useValue` instead. Oh, and another gotcha I encountered… Jasmine spies by default only intercept the function but don’t call through to the real function. You have to use `spyOn(foo, ‘bar’).and.callThrough();` to actually go into the real function, which is useful when you have multiple spies on functions that call other functions.

Now I know there’s a lot of Angular 2 testing articles out there… and a lot of them have different strategies.. this is the one that worked for me…


// In addition to importing all the modules and services / providers you are using here, you need the core libraries...
import { fakeAsync, tick, async, ComponentFixture, TestBed } from '@angular/core/testing';
import { DebugElement, CUSTOM_ELEMENTS_SCHEMA, NO_ERRORS_SCHEMA } from '@angular/core';
import { By } from '@angular/platform-browser';


describe('HeaderComponent', () => {

let comp: HeaderComponent;
let fixture: ComponentFixture;
let sessionServiceStub: any;
let utilsServiceStub: any;

beforeEach(async(() => {
utilsServiceStub = {
getPageName: jasmine.createSpy('getPageName').and.callFake(
() => ''
)
};
sessionServiceStub = {
getUserInfo: jasmine.createSpy('getUserInfo').and.callFake(() => {
return {
displayName: 'Jack'
};
}),
getSiteInfo: jasmine.createSpy('getSiteInfo').and.callFake(
() => Promise.resolve(true).then(() => {})
)
};
TestBed.configureTestingModule({
declarations: [HeaderComponent],
imports: [TranslateModule.forRoot()],
providers: [
{ provide: SessionService, useValue: sessionServiceStub },
{ provide: UtilsService, useValue: utilsServiceStub }
],
schemas: [CUSTOM_ELEMENTS_SCHEMA, NO_ERRORS_SCHEMA]
}).compileComponents();
}));

beforeEach(() => {
fixture = TestBed.createComponent(HeaderComponent);
comp = fixture.componentInstance;
comp.nav = [
{
pageName: 'Home',
tabLabel: 'tablabels.home'
},
{
pageName: 'PostIdea',
tabLabel: 'tablabels.post_idea'
},
{
pageName: 'ViewIdeas',
tabLabel: 'tablabels.view_ideas'
}
];
});

describe('component onInit', () => {
it('can instantiate it', () => {
expect(comp).not.toBeNull();
});

it('should init the header logo', () => {
const headerLogoEl = fixture.debugElement.query(By.css('.header-logo'));
expect(headerLogoEl.nativeElement.textContent).not.toBeNull();
});

it('should init the header menu', () => {
const headerMenuEl = fixture.debugElement.query(By.css('.header-menu'));
expect(headerMenuEl.nativeElement.textContent).not.toBeNull();
});

it('should init the right services on init', () => {
fixture.detectChanges();
expect(utilsServiceStub.getPageName.calls.count()).toBe(1);
expect(sessionServiceStub.getUserInfo.calls.count()).toBe(1);
expect(sessionServiceStub.getSiteInfo.calls.count()).toBe(1);
});
});

describe('buildMenu()', () => {
it('should init the menu holder', () => {
const navBarEl = fixture.debugElement.query(By.css('.navbar-spigit'));
spyOn(comp, 'buildMenu');
navBarEl.triggerEventHandler('showMoreMenu', null);
expect(comp.buildMenu).toHaveBeenCalled();
const menuHolder = fixture.debugElement.query(By.css('.more-menu-holder'));
const menuHolderContent = menuHolder.nativeElement.textContent;
expect(menuHolderContent).not.toBeNull();
});

it('should show dropdown menu when moreMenuConfig is visible', () => {
let dropdownMenu = fixture.debugElement.query(By.css('.dropdown-menu'));
expect(dropdownMenu).toBeNull();
comp.moreMenuConfig = {
visible : true,
mouseOutIntervalTime : 300,
mouseOutInterval: null,
moreMenuButtonVisible: true
};
// dropdown menu should be visible now
fixture.detectChanges();
dropdownMenu = fixture.debugElement.query(By.css('.dropdown-menu'));
expect(dropdownMenu).not.toBeNull();
});
});
});

and if you’re using Angular Bootstrap.. it actually does async stuff under the hood so you have to be careful about selecting DOM elements if they are using Bootstrap…


it('should call onScroll when scrolling on menu', async(() => {
fixture.detectChanges();
fixture.whenStable().then(() => { // wait for promise to return
fixture.detectChanges();
const menuEl = fixture.debugElement.query(By.css('.dropdown-menu'));
menuEl.triggerEventHandler('scroll', {
target: {
scrollHeight: 100,
scrollTop: 0,
clientHeight: 0
}
});
expect(comp.onScroll).toHaveBeenCalled();
expect(comp.getCommunities).toHaveBeenCalled();
})
}));

And sometimes when you are testing service calls, you also want to test for functions or code that executes in the callback of the promise as well… then you need `fakeAsync` for the job:


it('should get mini profile', fakeAsync(() => {
nameElmt.triggerEventHandler('mouseover', null); // mouseover triggers async service call
fixture.detectChanges();
tick(); // this essentially forwards the promise to the 'then' callback function
expect(userServiceStub.getMiniProfile.calls.count()).toBe(1);
}));

That’s just some sample code for testing components… now for testing services it gets more difficult since the Angular 2 testing guide doesn’t explain this very thoroughly… this is how I did it


import * as data from './data.json';

Now in TypeScript we can’t actually import json without it complaining about it missing a module definition.. unlike in native ES6. So we have to use a workaround hack .. we have to create another separate file in the same directory as the json file with the same name (including .json) so we have a file called `data.json.ts` and inside it the only content we have is

export default '';

And that’s it! Now the importing works. It’s weird but yes we have to do that until TypeScript natively supports importing JSON.

then in the test file…


describe('PageService', () => {

let utilsServiceStub: any;
const mockResponse:any = data; // Typescript will not recognize anything in the json data unless we force a type on it

beforeEach(async(() => {
utilsServiceStub = {
getPageName: jasmine.createSpy('getPageName').and.callFake((param: string) => '')
};
TestBed.configureTestingModule({
imports: [HttpModule],
providers: [
PageService,
MockBackend,
{ provide: UtilsService, useValue: utilsServiceStub },
{ provide: XHRBackend, useClass: MockBackend }
],
schemas: [CUSTOM_ELEMENTS_SCHEMA, NO_ERRORS_SCHEMA]
}).compileComponents();
}));

describe('getPagePromise()', () => {
it ('should return appropiate response', async(
inject([PageService, MockBackend], (pageService, mockBackend) => {

mockBackend.connections.subscribe((connection: MockConnection) => {
connection.mockRespond(new Response(new ResponseOptions({
body: JSON.stringify(mockResponse)
})));
});

pageService.getPagePromise().then((pageData) => {
expect(utilsServiceStub.getPageName.calls.count()).toBe(1, 'getPageName called once');
expect(pageData.ideaStages.length).toBe(6);
expect(pageData.numLifecycleStages).toBe(7);
expect(pageData.creatorName).toEqual('Viviana Arandia');
expect(pageData.ideaTitle).toEqual('ONE POPULAR STAR WARS PLANET THAT ALMOST SHOWED UP IN ROGUE ONE');
expect(pageData.hasViewed).toBe(false);
});

})));
});
});

Sometimes … there is a function that deals with DOM and things become very difficult to test with native functions like setTimeOut() for example.


keyPressHandler(event): void {
if (event.keyCode === 13 && this.searchInput.nativeElement.value) {
this.submitSearch();
}
}

clearMoreBtnTimeout() {
clearInterval(this.moreMenuConfig.mouseOutInterval);
}

In this example, its hard to test because we can’t simulate native events easily or native DOM elements that easily. Instead what we can do is pass in an optional test variable that denotes its being used for a test, and that way we can reach better code coverage without having to deal with potentially nasty DOM/event simulations.


keyPressHandler(event, test?): void {
if (event.keyCode === 13 && this.searchInput.nativeElement.value) {
this.submitSearch();
} else if (test) {
this.submitSearch();
}
}

clearMoreBtnTimeout(test?) {
// clear morebutton over timeout
if (!test) {
clearInterval(this.moreMenuConfig.mouseOutInterval);
} else {
this.moreMenuConfig.mouseOutInterval = 0;
}
}

Ok, now we can test it like this:


describe('clearMoreBtnTimeout()', () => {

it('should set mouseOutInterval to 0', () => {
component.clearMoreBtnTimeout(true);
expect(component.moreMenuConfig.mouseOutInterval).toEqual(0);
});

});

describe('keyPressHandler()', () => {

it('should call submitSearch', () => {
spyOn(component, 'submitSearch');
component.keyPressHandler({}, true);
expect(component.submitSearch).toHaveBeenCalled();
});

});

And just like that, we can up our code coverage for those pesky sections of code that were hard to unit test before.

So those are my component and service tests… let me know if that helps anyone! the existing Angular 2 unit testing articles are very confusing and contradictory for me so.. this is what worked for me.

Categories
Tech

Javascript Framework/Library Reference

Base libraries (animation, dom manipulation, event handling, ajax, utility)
jQuery (this library is the most popular base library)
Zepto
Dojo
Prototype
MooTools

Utility Libraries
Underscore
Lodash
Sizzle
Modernizr (feature detection)
Yepnope (conditional loader)

MVC Frameworks
Angular
Ember
Backbone
SproutCore
ExtJS
Batman
Spine
Meteor
Exoskeleton
ActiveJS
StapesJS
JS MVC

View/Data binding frameworks
Knockout
CanJS
React
Rivets

Backbone Libraries
Thorax
Epoxy
Chaplin
Marionette
Quilt
Backpack
Rendr
Backbone Boilerplate

Templating Libraries
Handlebars
Mustache

Widget libraries (animation, ui)
jQueryUI
YUI
Bootstrap
Scriptaculous
Google Closure
ShadowboxJS
Lightbox

Mobile libraries
jQuery Mobile
Dojo Mobile
Sencha Touch

Testing Libraries
Chai
Jasmine
QUnit
Mocha
Sinon
PhantomJS
CasperJS
Istanbul
Karma

Build development Libraries
Grunt (task runner)
Bower (package manager)
RequireJS (module loader)
Yeoman (scaffolding)

Special use libraries (data-driven, graphics, maps, etc)
D3 (document manipulation)
Raphael (vector graphics)
MediaElementJS (HTML5 video/audio)
JPlayer (HTML5 video/audio)
Leaflet (maps)
ChartJS (charts)

NodeJS Libraries
Express
MongooseJS
BookshelfJS
SocketIO
Sequelize
Sails
Prerender