Integration tests using cypress

Test client integration (client loads)
Test auth integration (client logs in)
Test backend integration (accept request)
  - broken
Test autoupdate integration (sends au to client)

Add manual cypress tests
Add dockered cypress tests

Add Readme
Add test execution to makefile
Add test execution to github-actions

TODO:
- Create user for tests
- flush db after every test
This commit is contained in:
Sean 2021-09-16 17:16:51 +02:00
parent 97ea71c8db
commit 85c0e50c21
21 changed files with 3612 additions and 5 deletions

22
.github/workflows/test-integration.yml vendored Normal file
View File

@ -0,0 +1,22 @@
---
name: Run integration tests (cypress)
on: [push, pull_request]
jobs:
run-cypress:
name: "Runs integration tests in cypress"
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout@v2
with:
submodules: recursive
- name: Build and run OS4 Dev
run: make run-dev ARGS="-d"
- name: Run integration tests (cypress docker)
run: make cypress-docker
- name: Stop OS4
run: make stop-dev

9
.gitignore vendored
View File

@ -20,6 +20,13 @@ dev-commands/export.json
/docker/secrets/auth_*_key /docker/secrets/auth_*_key
docker/secrets/*.env docker/secrets/*.env
# Integration testing
/integration/results
/integration/cypress/downloads
/integration/cypress/screenshots
/integration/cypress/videos
/integration/node_modules
# Old OS3 files and folders # Old OS3 files and folders
.coverage .coverage
.mypy_cache .mypy_cache
@ -36,7 +43,7 @@ tests
.venv/ .venv/
.virtualenv/ .virtualenv/
.vscode/ .vscode/
package-lock.json client/package-lock.json
server/ server/
# OS3+-Submodules # OS3+-Submodules
/autoupdate/ /autoupdate/

View File

@ -1,5 +1,11 @@
run-system-tests: run-integration-tests:
echo "TODO: write complete system tests" @echo "Start OpenSlides Dev"
make run-dev ARGS="-d"
@echo "Start integration tests"
make cypress-docker
docker-compose -f integration/docker-compose.yml up
@echo "Stop OpenSlides Dev"
make stop-dev
run-service-tests: run-service-tests:
git submodule foreach 'make run-tests' git submodule foreach 'make run-tests'
@ -9,7 +15,7 @@ build-dev:
make -C proxy build-dev make -C proxy build-dev
run-dev: | build-dev run-dev: | build-dev
docker-compose -f docker/docker-compose.dev.yml up docker-compose -f docker/docker-compose.dev.yml up $(ARGS)
stop-dev: stop-dev:
docker-compose -f docker/docker-compose.dev.yml down --volumes --remove-orphans docker-compose -f docker/docker-compose.dev.yml down --volumes --remove-orphans
@ -27,3 +33,13 @@ services-to-master:
# #
# [1] ...or main, or whatever branch the OS4 one is. See .gitmodules. # [1] ...or main, or whatever branch the OS4 one is. See .gitmodules.
git submodule foreach -q --recursive 'git checkout $(git config -f $$toplevel/.gitmodules submodule.$$name.branch || echo master); git pull upstream $$(git config -f $$toplevel/.gitmodules submodule.$$name.branch || echo master)' git submodule foreach -q --recursive 'git checkout $(git config -f $$toplevel/.gitmodules submodule.$$name.branch || echo master); git pull upstream $$(git config -f $$toplevel/.gitmodules submodule.$$name.branch || echo master)'
cypress-open:
cd integration; npm run cypress:open
cypress-run:
cd integration; npm run cypress:run
cypress-docker:
docker-compose -f integration/docker-compose.yml build
docker-compose -f integration/docker-compose.yml up

9
integration/Dockerfile Normal file
View File

@ -0,0 +1,9 @@
FROM cypress/included:8.4.0
WORKDIR /app
COPY ./cypress ./cypress/
COPY ./cypress-docker.json ./cypress.json
COPY entrypoint.sh /usr/local/bin/entrypoint
ENTRYPOINT entrypoint

44
integration/README.md Normal file
View File

@ -0,0 +1,44 @@
# Integration tests
To test the integration of our OpenSlides services we are using cypress.
There are not too many of them, but they are sufficient tell that our services are properly integrated.
- The client can be accessed
- Authentication works (admin admin can be logged in)
- The backend accepts requests (of some sort)
- The auto update transmits data to the client
(this list is not exhaustive)
## Writing tests and using cypress
To write and execute tests meaningfully, you will want to install cypress locally.
Inside the `/integration` directory, execute
$ npm install
$ npm run cypress:open
(or `make cypress-open` from the main directory)
The cypress runner will open using electron and executes the tests for you.
## Run in docker and CI
Start OpenSlides (Usually in dev setup). From the main directory run
$ make run-dev
in the `/integration` just fire
$ docker-compose build
$ docker-compose up
(or `make cypress-docker` from the main directory)
Cypress will run dockered and report errors inside the CLI.
Screenshots and videos of the tests can be found in the `/integration/results` folder.
You can streamline the whole process by using
$ make run-system-tests
From the main directory.
This can take while, since the docker of OS4 has to be build first.

View File

@ -0,0 +1,3 @@
{
"baseUrl": "https://host.docker.internal:8000"
}

6
integration/cypress.json Normal file
View File

@ -0,0 +1,6 @@
{
"baseUrl": "https://localhost:8000",
"screenshotsFolder": "results/screenshots",
"videosFolder": "results/videos",
"downloadsFolder": "results/downloads"
}

View File

@ -0,0 +1,5 @@
{
"name": "Using fixtures to represent data",
"email": "hello@cypress.io",
"body": "Fixtures are a great way to mock data for responses to routes"
}

View File

@ -0,0 +1,7 @@
describe("Load client", () => {
it("successfully loads", () => {
cy.visit("/");
cy.url().should("include", "/login");
cy.contains("OpenSlides");
});
});

View File

@ -0,0 +1,13 @@
describe("Login using UI", () => {
it("Login over login-form, set cookie", () => {
const username = "admin"
const password = "admin"
cy.visit("/login");
cy.get("#mat-input-0").type(username);
cy.get("#mat-input-1").type(`${password}{enter}`);
cy.url().should("not.include", "login");
cy.getCookie("refreshId").should("exist");
});
});

View File

@ -0,0 +1,27 @@
/**
* Some bugs
*/
// describe("Create a new committee", () => {
// beforeEach(() => {
// cy.login();
// cy.visit("/committees");
// });
// it("Creates a committee", () => {
// cy.visit("/committees/create");
// const committeeName = `Cypress Committee ${Date.now().toString()}`;
// cy.get("#mat-input-0").type(committeeName);
// cy.intercept({
// method: "POST",
// url: "/system/action/handle_request",
// }).as("handle_request");
// cy.get("form").submit();
// cy.wait("@handle_request");
// cy.url().should("not.include", "create");
// // only on clean db
// //cy.contains(committeeName);
// });
// });

View File

@ -0,0 +1,30 @@
describe("Get autoupdates for committees detail view", () => {
let committeeName;
let committeeId;
beforeEach(() => {
cy.login();
committeeName = `Cypress ${Date.now().toString()}`;
const committeeData = {
organization_id: 1,
name: committeeName,
manager_ids: [1],
};
cy.os4request("committee.create", committeeData).then((res) => {
committeeId = res.id;
});
});
it("Receives a name change", () => {
cy.visit(`/committees/${committeeId}`);
cy.contains(committeeName);
const updatedName = committeeName + "update";
const committeeData = {
id: committeeId,
name: updatedName,
};
cy.os4request("committee.update", committeeData).then(() => {
cy.contains(updatedName);
});
});
});

View File

@ -0,0 +1,43 @@
describe("Update a committee", () => {
let committeeName;
let committeeId;
beforeEach(() => {
cy.login();
committeeName = `Cypress ${Date.now().toString()}`;
const committeeData = {
organization_id: 1,
name: committeeName,
manager_ids: [1],
};
cy.os4request("committee.create", committeeData).then((res) => {
committeeId = res.id;
});
cy.visit("/committees/");
});
it("Has new Committee", () => {
cy.visit(`/committees/${committeeId}`);
cy.url().should("include", committeeId);
cy.get("h1").contains(committeeName);
});
/**
* Some bugs
*/
// it("Can just update a new Committee", () => {
// cy.visit(`/committees/${committeeId}/edit-committee`);
// cy.url().should("include", `${committeeId}/edit-committee`);
// cy.get(".title-slot").contains("Edit committee");
// cy.get("#mat-input-0").type("edit");
// // cy.intercept({
// // method: "POST",
// // url: "/system/action/handle_request",
// // }).as("au");
// // cy.wait("@au");
// cy.get("form").submit();
// cy.url().should("not.include", `edit-committee`);
// });
});

View File

@ -0,0 +1,22 @@
/// <reference types="cypress" />
// ***********************************************************
// This example plugins/index.js can be used to load plugins
//
// You can change the location of this file or turn off loading
// the plugins file with the 'pluginsFile' configuration option.
//
// You can read more here:
// https://on.cypress.io/plugins-guide
// ***********************************************************
// This function is called when a project is opened or re-opened (e.g. due to
// the project's config changing)
/**
* @type {Cypress.PluginConfig}
*/
// eslint-disable-next-line no-unused-vars
module.exports = (on, config) => {
// `on` is used to hook into various events Cypress emits
// `config` is the resolved Cypress config
}

View File

@ -0,0 +1,92 @@
// ***********************************************
// This example commands.js shows you how to
// create various custom commands and overwrite
// existing commands.
//
// For more comprehensive examples of custom
// commands please read more here:
// https://on.cypress.io/custom-commands
// ***********************************************
//
//
// -- This is a parent command --
// Cypress.Commands.add('login', (email, password) => { ... })
//
//
// -- This is a child command --
// Cypress.Commands.add('drag', { prevSubject: 'element'}, (subject, options) => { ... })
//
//
// -- This is a dual command --
// Cypress.Commands.add('dismiss', { prevSubject: 'optional'}, (subject, options) => { ... })
//
//
// -- This will overwrite an existing command --
// Cypress.Commands.overwrite('visit', (originalFn, url, options) => { ... })
/**
* Login
*/
Cypress.Commands.add("login", (username = "admin", password = "admin") => {
cy.request({
method: "POST",
url: "/system/auth/login/",
body: {
username,
password,
},
})
.as("loginResponse")
.then((response) => {
Cypress.env("authToken", response.headers.authentication);
return response;
})
.its("status")
.should("eq", 200)
});
/**
* Create models
*/
Cypress.Commands.add("os4request", (osAction, body) => {
cy.request({
method: "POST",
url: "/system/action/handle_request",
body: [
{
action: osAction,
data: [
{
...body,
},
],
},
],
})
.should((response) => {
expect(response.status).to.eq(200);
})
.its("body")
.should("contain", {
success: true,
})
.then((body) => {
return body.results[0][0]
});
});
/**
* Extend "request" with auth header
*/
Cypress.Commands.overwrite("request", (originalFn, ...options) => {
const optionsObject = options[0];
const token = Cypress.env("authToken");
if (!!token && optionsObject === Object(optionsObject)) {
optionsObject.headers = {
authentication: token,
...optionsObject.headers,
};
return originalFn(optionsObject);
}
return originalFn(...options);
});

View File

@ -0,0 +1,20 @@
// ***********************************************************
// This example support/index.js is processed and
// loaded automatically before your test files.
//
// This is a great place to put global configuration and
// behavior that modifies Cypress.
//
// You can change the location of this file or turn off
// automatically serving support files with the
// 'supportFile' configuration option.
//
// You can read more here:
// https://on.cypress.io/configuration
// ***********************************************************
// Import commands.js using ES2015 syntax:
import './commands'
// Alternatively you can use CommonJS syntax:
// require('./commands')

View File

@ -0,0 +1,10 @@
version: "3"
services:
cypress:
build: .
image: cypress
volumes:
- ./results/videos :/app/cypress/videos
- ./results/screenshots :/app/cypress/screenshots
extra_hosts:
host.docker.internal: host-gateway

22
integration/entrypoint.sh Executable file
View File

@ -0,0 +1,22 @@
#!/bin/bash
# -------------------------------------------------------------------
# Copyright (C) 2020 by Intevation GmbH
# Author(s):
# Sean Engelhardt <sean.engelhardt@intevation.de>
#
# This program is distributed under the MIT license, as described
# in the LICENSE file included with the distribution.
# SPDX-License-Identifier: MIT
# -------------------------------------------------------------------
HOST="https://host.docker.internal:8000"
echo "wait until OpenSlides is up"
until [[ $(curl -k -s -o /dev/null -w %{http_code} $HOST) -eq 200 ]];
do
sleep 5
done
echo ready
exec npx cypress run

3200
integration/package-lock.json generated Normal file

File diff suppressed because it is too large Load Diff

9
integration/package.json Normal file
View File

@ -0,0 +1,9 @@
{
"scripts": {
"cypress:open": "cypress open",
"cypress:run": "cypress run"
},
"devDependencies": {
"cypress": "8.4.0"
}
}

@ -1 +1 @@
Subproject commit 55e35cedbe8ea77047635772d3486e7d3c38fa03 Subproject commit 9aa339decc73053c25714622ea71401eae77d2d5