diff --git a/ADVANCED.rst b/ADVANCED.rst new file mode 100644 index 000000000..e94bae53a --- /dev/null +++ b/ADVANCED.rst @@ -0,0 +1,64 @@ +Advanced configuration +====================== + +Docker Swarm Mode +----------------- + +OpenSlides may also be deployed in Swarm mode. Distributing instances over +multiple nodes may increase performance and offer failure resistance. + +An example configuration file, ``docker-stack.yml.m4``, is provided. Unlike +the Docker Compose setup, this configuration will most likely need to be +customized, especially its placement constraints and database-related +preferences. + +Before deploying an instance on Swarm, please see `Database Configuration`_ and +`Backups`_, and review your ``docker-stack.yml`` + + +Database Configuration +---------------------- + +It is fairly easy to get an OpenSlides instance up an running; however, for +production setups it is strongly advised to review the database configuration. + +By default, the primary database cluster will archive all WAL files in its +volume. Regularly pruning old data is left up to the host system, i.e., you. +Alternatively, you may disable WAL archiving by setting +``PGNODE_WAL_ARCHIVING=off`` in ``.env`` before starting the instance. + +The provided ``docker-stack.yml.m4`` file includes additional database +services which can act as hot standby clusters with automatic failover +functionality. To take advantage of this setup, the database services need to +be configured with proper placement constraints. Before relying on this setup, +please familiarize yourself with `repmgr `_. + + +Backups +------- + +All important data is stored in the database. Additionally, the project +directory should be included in backups to ensure a smooth recovery. + +The primary database usually runs in the ``pgnode1`` service (but see `Database +Configuration`_ above). + +In some cases, it may be sufficient to generate SQL dumps with ``pg_dump`` +through ``docker exec`` to create backups. However, for proper incremental +backups, the host system can backup the cluster's data directory and WAL +archives. + +The cluster's data directory is available as a volume on the host system. +Additionally, the database archives its WAL files in the same volume by +default. This way, the host system can include the database volume in its +regular filesystem-based backup routine and create efficient database backups +suitable for point-in-time recovery. + +The `former management repository +`_ provides the +script `openslides-pg-mgr.sh` which can enable Postgres' backup mode in all +OpenSlides database containers. + +In Swarm mode, the primary database cluster may get placed on a number of +nodes. It is, therefore, crucial to restrict the placement of database +services to nodes on which appropriate backups have been configured. diff --git a/DEVELOPMENT.rst b/DEVELOPMENT.rst index 482eab1bf..547cafe40 100644 --- a/DEVELOPMENT.rst +++ b/DEVELOPMENT.rst @@ -10,9 +10,10 @@ Check requirements - ``git`` - ``make`` -Note about migrating from previous OpenSlides3 development -setups: You must set the ``OPENSLIDES_USER_DATA_DIR`` variable in -your ``server/personal_data/var/settings.py`` to ``'/app/personal_data/var'`` +**Note about migrating from development +setups before version 3.4**: You must set the ``OPENSLIDES_USER_DATA_DIR`` variable in +your ``server/personal_data/var/settings.py`` to ``'/app/personal_data/var'``. Another +way is to just delete this file. It is recreated with the right paths afterwards. Get OpenSlides source code @@ -21,31 +22,35 @@ Get OpenSlides source code Clone current master version from `OpenSlides GitHub repository `_:: - git clone https://github.com/OpenSlides/OpenSlides.git + git clone https://github.com/OpenSlides/OpenSlides.git --recurse-submodules cd OpenSlides -TODO: submodules. +When updating the repository, submodules must be updated explicitly, too:: + + git submodule update Start the development setup '''''''''''''''''''''''''''''' +Use `make` to start the setup:: make run-dev +All your data (database, config, mediafiles) is stored in ``server/personal_data/var``. +To stop the setup press Ctrl+C. To clean up the docker containers run:: -All you data (database, config, mediafiles) is stored in ``server/personal_data/var``. + make stop-dev Running the test cases -------------------------- - +''''''''''''''''''''''' For all services in submodules check out the documentation there. -Server tests andscripts -''''''''''''''''''''''' +Server tests and scripts +------------------------- You need to have python (>=3.8) and python-venv installed. Change your workdirectory to the server:: cd server -Setup an python virtual environment. If you have already done it, you can skip this step: +Setup an python virtual environment. If you have already done it, you can skip this step:: python3 -m venv .venv source .venv/bin/activate @@ -55,7 +60,7 @@ Make sure you are using the correct python version (e.g. try with explicit minor source .venv/bin/activate -To deactivate it type ``deactivate``. Running all tests and linters: +To deactivate it type ``deactivate``. Running all tests and linters:: black openslides/ tests/ flake8 openslides/ tests/ @@ -64,7 +69,7 @@ To deactivate it type ``deactivate``. Running all tests and linters: pytest tests/ Client tests -'''''''''''' +------------- You need `node` and `npm` installed. Change to the client's directory. For the first time, install all dependencies:: cd client/ diff --git a/README.rst b/README.rst index b104bc9c1..6c56ca192 100644 --- a/README.rst +++ b/README.rst @@ -5,44 +5,47 @@ What is OpenSlides? =================== -OpenSlides is a free, Web-based presentation and assembly system for +OpenSlides is a free, web-based presentation and assembly system for managing and projecting agenda, motions, and elections of assemblies. See https://openslides.com for more information. Installation ============ -The main deployment method is using Docker and docker-compose. You only need to +The main deployment method is using Docker and Docker Compose. You only need to have these tools installed and no further dependencies. If you want a simpler setup or are interested in developing, please refer to `development instructions -`_. - -Note: This is temporary and will be replaced with nice scripts... +`_. First, you have to clone this repository:: - $ git clone https://github.com/OpenSlides/OpenSlides.git - $ cd OpenSlides/docker/ + git clone https://github.com/OpenSlides/OpenSlides.git --recurse-submodules + cd OpenSlides/docker/ -TODO: submodules. +**Note about migrating from version 3.3 or earlier**: With OpenSlides 3.4 submodules +and a Docker setup were introduced. If you ran into problems try to delete your +``settings.py``. If you have an old checkout you need to check out the current master +first and initialize all submodules:: + + git submodule update --init You need to build the Docker images for the client and server with this script:: - $ ./build.sh all + ./build.sh all You must define a Django secret key in ``secrets/django.env``, for example:: - $ printf "DJANGO_SECRET_KEY='%s'\n" \ + printf "DJANGO_SECRET_KEY='%s'\n" \ "$(tr -dc 'a-zA-Z0-9' < /dev/urandom | head -c 64)" > secrets/django.env We also strongly recommend that you set a secure admin password but it is not strictly required. If you do not set an admin password, the default login credentials will be displayed on the login page. Setting the admin password:: - $ cp secrets/adminsecret.env.example secrets/adminsecret.env - $ vi secrets/adminsecret.env + cp secrets/adminsecret.env.example secrets/adminsecret.env + vi secrets/adminsecret.env Afterwards, generate the configuration file:: @@ -50,12 +53,12 @@ Afterwards, generate the configuration file:: Finally, you can start the instance using ``docker-compose``:: - $ docker-compose up - $ # or: - $ docker-compose up -d - $ docker-compose logs - $ # ... - $ docker-compose down + docker-compose up + # or: + docker-compose up -d + docker-compose logs + # ... + docker-compose down More settings ------------- @@ -63,74 +66,13 @@ More settings When generating the ``docker-compose.yml``, more settings can be adjusted in the ``docker/.env`` file. All changes for the backend are passed into djangos ``settings.py``. You can find more information about most settings `here -`_. To generate +`_. To generate the ``docker-compose.yml`` use this command:: - $ ( set -a; source .env; m4 docker-stack.yml.m4 ) > docker-stack.yml - - -Docker Swarm Mode ------------------ - -OpenSlides may also be deployed in Swarm mode. Distributing instances over -multiple nodes may increase performance and offer failure resistance. - -An example configuration file, ``docker-stack.yml.m4``, is provided. Unlike -the Docker Compose setup, this configuration will most likely need to be -customized, especially its placement constraints and database-related -preferences. - -Before deploying an instance on Swarm, please see `Database Configuration`_ and -`Backups`_, and review your ``docker-stack.yml`` - - -Database Configuration ----------------------- - -It is fairly easy to get an OpenSlides instance up an running; however, for -production setups it is strongly advised to review the database configuration. - -By default, the primary database cluster will archive all WAL files in its -volume. Regularly pruning old data is left up to the host system, i.e., you. -Alternatively, you may disable WAL archiving by setting -``PGNODE_WAL_ARCHIVING=off`` in ``.env`` before starting the instance. - -The provided ``docker-stack.yml.m4`` file includes additional database -services which can act as hot standby clusters with automatic failover -functionality. To take advantage of this setup, the database services need to -be configured with proper placement constraints. Before relying on this setup, -please familiarize yourself with `repmgr `_. - - -Backups -------- - -All important data is stored in the database. Additionally, the project -directory should be included in backups to ensure a smooth recovery. - -The primary database usually runs in the ``pgnode1`` service (but see `Database -Configuration`_ above). -. -In some cases, it may be sufficient to generate SQL dumps with ``pg_dump`` -through ``docker exec`` to create backups. However, for proper incremental -backups, the host system can backup the cluster's data directory and WAL -archives. - -The cluster's data directory is available as a volume on the host system. -Additionally, the database archives its WAL files in the same volume by -default. This way, the host system can include the database volume in its -regular filesystem-based backup routine and create efficient database backups -suitable for point-in-time recovery. - -The `former management repository -`_ provides the -script `openslides-pg-mgr.sh` which can enable Postgres' backup mode in all -OpenSlides database containers. - -In Swarm mode, the primary database cluster may get placed on a number of -nodes. It is, therefore, crucial to restrict the placement of database -services to nodes on which appropriate backups have been configured. + ( set -a; source .env; m4 docker-stack.yml.m4 ) > docker-stack.yml +For an advanced database setup refer to the `advanced configuration +`_. Bugs, features and development ================================ @@ -138,7 +80,10 @@ Bugs, features and development Feel free to open issues here on GitHub! Please use the right templates for bugs and features, and use them correctly. Pull requests are also welcome; for a general overview of the development setup refer the `development instructions -`_. +`_. + +For security relevant issues **do not** create public issues and refer to +our `security policy `_. Used software ============= diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 000000000..f59f3a1b0 --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,14 @@ +# Security Policy + +## Supported Versions + +Currently the recent versions (a year old at max) of the current major +release are supported to get security fixes. For older versions we urge +you to update to the most recent version of OpenSlides. + +## Reporting a Vulnerability + +Please let us know about the vulnerability by an email to +[security@openslides.com](mailto:security@openslides.com). Do not create +public issues for such issues. We will get in touch with you after reporting +the issue.