Skip to content
This repository has been archived by the owner on Mar 21, 2021. It is now read-only.

Should we add/migrate to Cypress ? #401

Closed
1 task done
clementdessoude opened this issue Jun 21, 2019 · 10 comments · May be fixed by #631
Closed
1 task done

Should we add/migrate to Cypress ? #401

clementdessoude opened this issue Jun 21, 2019 · 10 comments · May be fixed by #631

Comments

@clementdessoude
Copy link
Contributor

Overview of the feature request

Hi all ! As we had some issues with flaky protractor tests, I started to look if Cypress could help, and finished by coding a draft for supporting Cypress.

I don't know how you feel about that: as @deepu105 said, it is already difficult to maintain all of the options of the generator, so adding one seems difficult. So, I'll leave it up to you to decide what should be done with this PR (#399), and if we should migrate to Cypress or not.

Anyway, it was fun to develop! 😄

Motivation for or Use Case

Cypress looks more fitted to do e2e testing with Vue.js than Protractor: IMO, it seems easier to get rid of some flaky tests and code looks cleaner.

Related issues or PR

#399

  • Checking this box is mandatory (this is just to show you read everything)
@vishal423
Copy link
Contributor

+1 as long as we also support protractor. It will allow consumers to choose protractor if they require cross browser testing support. I am also thinking to refactor current suite of protractor tests to align with style guide, but, not sure how much time I can devote to it.

@deepu105
Copy link
Member

I'm against supporting 2 options and I have some reservations when it comes to Cypress(as I mentioned in the other threads

I have used cypress in our company. While I liked many of its features. I have few issues though.

  1. I'm not in favor of providing 2 options for e2e
  2. The last I checked cypress cant run tests in parallel and it's very slow overall compared to Protractor(Maybe things changed now). It becomes really slow if you have a lot of e2e tests
  3. Though it promises not to be flaky, I have experienced otherwise(so this really depends on the tests and not a framework)
  4. browser compatibility

@atomfrede
Copy link
Member

atomfrede commented Jun 21, 2019

It is hard to maintain multiple options, but as vue-cli supports nightwatch and cypress policy 1 might be applicable as protractor was designed for angular if I am not mistaken.

So yes supporting multiple e2e solutions is not great (but right now vue is a blueprint and not in core). When it comes to browser compatibility we might have a look at nightwatch (also I have very good experience with cypress, but our e2e test suite is quite small and simple).

@pascalgrimaud
Copy link
Member

  • e2e with Protractor+Vue.js were broken for months, that's why it was suggested to try Cypress at our last JHipster day at Ippon -> see Protractor tests: random failures #285
  • I spent night on this, to find a solution but failed
  • thanks to @vishal423 who managed to fix the issue, I hope
  • I agree, supporting both option is probably impossible, we won't have the bandwidth for that
  • for now, it's a blueprint, so I think we can try some stuff and see if it works well
  • it's an official blueprint but we need to discuss about same option between main generator and blueprint
  • for me, if we plan to merge to main generator, we must keep the same option. If not, we should be free here to decide which option is the best for Vue.js

Ping @sahbi-ktifa @hdurix

@hdurix
Copy link
Member

hdurix commented Jun 24, 2019

Hello, here are my thoughts:

  • we should not keep 2 different e2e frameworks, both protractor or cypress because it would be too much work for the core team
  • protractor was great for AngularJS and Angular because it is built for and the big advantage is the wait for promises to be resolved
  • the code for protractor contains a lot of hack for react and vuejs, for instance:
    await waitUntilDisplayed(toast);
    // Success toast should appear
    expect(await toast.isPresent()).to.be.true;

    await entityComponentsPage.waitUntilEntityCountIs(nbButtonsBeforeCreate + 1);
    expect(await entityComponentsPage.countEntity()).to.eq(nbButtonsBeforeCreate + 1);

and the code behind waitUntilDisplayed is really tricky

  • this code is generated with the app, that means a lot of code hard to maintain for the end user (= technical debt)
  • cypress would directly improve this code in 2 ways:
    • can wait for a backend route to be finished
    • wait until the assertion is true
  • IMHO the same code with cypress would be a lot more readable, something like:
    // Success toast should appear
    expect(await toast.isPresent()).to.be.true;

    cy.wait('@entitycreate');
    expect(await entityComponentsPage.countEntity()).to.eq(nbButtonsBeforeCreate + 1);

with no custom code generated

  • For now only Chrome & Electron are available but Cross Browser could appear
  • For me, choosing a browser is not so useful for running e2e tests, I prefer to have them to be sure I don't have any regression.
  • Running protractor in a CI can be very painful, no need for that with the built-in headless mode in Cypress
  • cypress does not seem longer than protractor: 6m16s on the cypress branch created by @clement26695 vs 21m57s on the master branch
  • I wasn't a big fan on implementing cypress tests on JHipster because I thought it would take too much time but Clement already migrated them all!
  • @deepu105 I agree that our tests could be flaky because of their code, maybe we could improve them
  • It would help us a lot if we would eventually have only one template of e2e test for the 3 frameworks, it would be easier on cypress because in protractor, implementing angular is really different from the 2 others
  • We use a lot protractor to check our new code doesn't break anything in the JHipster matrix (especially on Vue.js I think) but we should keep in mind this code is also generated for end user (so reduce number of hacks is a big plus).

The only sure thing is that moving from protractor should not be a quick decision made here in this thread. Do you think we could ask to our end users on Twitter what framework do they prefer (Protractor / Cypress / no preference / No e2e tests)?

@vishal423
Copy link
Contributor

With v4.0 cypress release, Firefox and the new Edge browsers are also supported. Maybe it's time to revisit this issue and target in v7?

@atomfrede
Copy link
Member

👍

@DanielFran
Copy link
Member

Support for Firefox and Microsoft Edge has been added: cypress-io/cypress#310

@geyuqiu
Copy link

geyuqiu commented Mar 9, 2020

made a list of pros and cons

@nassimerrahoui nassimerrahoui mentioned this issue Jun 15, 2020
4 tasks
@qmonmert
Copy link
Contributor

@pascalgrimaud I think you can close this ticket. JHipster supports Cypress now :)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants