OpenSlides3+: External Autoupdate Service
- Removing channels. Going back to a wsgi deployment - Removed server projector code - Autoupdate throttling is now in the client - New communication stack in the client - Adopted all deployment methods: Docker stack and docker compose (prod and dev) - Added autoupdate service as submodule
This commit is contained in:
parent
1145ae1460
commit
e225a57f97
2
.github/workflows/run-tests.yml
vendored
2
.github/workflows/run-tests.yml
vendored
@ -40,7 +40,7 @@ jobs:
|
||||
run: mypy openslides/ tests/
|
||||
|
||||
- name: test using pytest
|
||||
run: pytest --cov --cov-fail-under=75
|
||||
run: pytest --cov --cov-fail-under=74
|
||||
|
||||
install-client-dependencies:
|
||||
runs-on: ubuntu-latest
|
||||
|
9
.gitignore
vendored
9
.gitignore
vendored
@ -18,14 +18,12 @@
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
# Virtual Environment
|
||||
.virtualenv*/*
|
||||
.virtualenv*
|
||||
.venv
|
||||
server/.venv
|
||||
|
||||
## Compatibility
|
||||
# OS4-Submodules
|
||||
# OS4-Submodules and aux-directories
|
||||
/openslides-*/
|
||||
/haproxy/
|
||||
/docker/keys/
|
||||
/docs/
|
||||
# OS3+-Submodules
|
||||
@ -79,7 +77,8 @@ cypress.json
|
||||
|
||||
## Deployment
|
||||
# Docker build artifacts
|
||||
/docker/docker-compose.yml
|
||||
docker/docker-compose.yml
|
||||
*-version.txt
|
||||
*.pem
|
||||
# secrets
|
||||
docker/secrets/*.env
|
||||
|
3
.gitmodules
vendored
Normal file
3
.gitmodules
vendored
Normal file
@ -0,0 +1,3 @@
|
||||
[submodule "autoupdate"]
|
||||
path = autoupdate
|
||||
url = https://github.com/OpenSlides/openslides3-autoupdate-service.git
|
211
DEVELOPMENT.rst
211
DEVELOPMENT.rst
@ -2,187 +2,82 @@
|
||||
OpenSlides Development
|
||||
========================
|
||||
|
||||
This instruction helps you to setup a development environment for OpenSlides. A
|
||||
simple dev setup will be configured without the need of the docker-compose
|
||||
setup. There are only the server running without a cache and a sqlite database
|
||||
and the client as an development server.
|
||||
|
||||
|
||||
1. Installation on GNU/Linux or Mac OS X
|
||||
----------------------------------------
|
||||
|
||||
a. Check requirements
|
||||
Check requirements
|
||||
'''''''''''''''''''''
|
||||
|
||||
Make sure that you have installed `Python (>= 3.6) <https://www.python.org/>`_,
|
||||
`Node.js (>=10.x) <https://nodejs.org/>`_ and `Git <http://git-scm.com/>`_ on
|
||||
your system. You also need build-essential packages and header files and a
|
||||
static library for Python.
|
||||
- ``docker``
|
||||
- ``docker-compose``
|
||||
- ``git``
|
||||
- ``make``
|
||||
|
||||
For Debian based systems (Ubuntu, etc) run::
|
||||
|
||||
$ sudo apt-get install git nodejs npm build-essential python3-dev
|
||||
Note about migrating from previous OpenSlides3 development
|
||||
setups: You must set the ``OPENSLIDES_USER_DATA_DIR`` variable in
|
||||
your ``server/personal_data/var/settings.py`` to ``'/app/personal_data/var'``
|
||||
|
||||
|
||||
b. Get OpenSlides source code
|
||||
Get OpenSlides source code
|
||||
'''''''''''''''''''''''''''''
|
||||
|
||||
Clone current master version from `OpenSlides GitHub repository
|
||||
<https://github.com/OpenSlides/OpenSlides/>`_::
|
||||
|
||||
$ git clone https://github.com/OpenSlides/OpenSlides.git
|
||||
$ cd OpenSlides
|
||||
git clone https://github.com/OpenSlides/OpenSlides.git
|
||||
cd OpenSlides
|
||||
|
||||
TODO: submodules.
|
||||
|
||||
Start the development setup
|
||||
''''''''''''''''''''''''''''''
|
||||
|
||||
make run-dev
|
||||
|
||||
|
||||
c. Setup a virtual Python environment (optional)
|
||||
''''''''''''''''''''''''''''''''''''''''''''''''
|
||||
All you data (database, config, mediafiles) is stored in ``server/personal_data/var``.
|
||||
|
||||
You can setup a virtual Python environment using the virtual environment
|
||||
(venv) package for Python to install OpenSlides as non-root user. This will
|
||||
allow for encapsulated dependencies. They will be installed in the virtual
|
||||
environment and not globally on your system.
|
||||
|
||||
Setup and activate the virtual environment::
|
||||
|
||||
$ python3 -m venv .virtualenv
|
||||
$ source .virtualenv/bin/activate
|
||||
|
||||
You can exit the environment with::
|
||||
|
||||
$ deactivate
|
||||
|
||||
d. Server
|
||||
'''''''''
|
||||
|
||||
Go into the server's directory::
|
||||
|
||||
$ cd server/
|
||||
|
||||
Install all required Python packages::
|
||||
|
||||
$ pip install --upgrade setuptools pip
|
||||
$ pip install --requirement requirements.txt
|
||||
|
||||
Create a settings file, run migrations and start the server::
|
||||
|
||||
$ python manage.py createsettings
|
||||
$ python manage.py migrate
|
||||
$ python manage.py runserver
|
||||
|
||||
All you data (database, config, mediafiles) are stored in ``personal_data/var``.
|
||||
To get help on the command line options run::
|
||||
|
||||
$ python manage.py --help
|
||||
|
||||
Later you might want to restart the server with one of the following commands.
|
||||
|
||||
To run the OpenSlides server execute::
|
||||
|
||||
$ python manage.py runserver
|
||||
|
||||
When debugging something email related change the email backend to console::
|
||||
|
||||
$ python manage.py runserver --debug-email
|
||||
|
||||
The server is available under http://localhost:8000. Especially the rest interface
|
||||
might be important during development: http://localhost:8000/rest/ (The trailing
|
||||
slash is important!).
|
||||
|
||||
e. Client
|
||||
'''''''''
|
||||
|
||||
Go in the client's directory::
|
||||
|
||||
$ cd client/
|
||||
|
||||
Install all dependencies and start the development server::
|
||||
|
||||
$ npm install
|
||||
$ npm start
|
||||
|
||||
After a while, the client is available under http://localhost:4200.
|
||||
|
||||
|
||||
2. Installation on Windows
|
||||
--------------------------
|
||||
|
||||
Follow the instructions above (Installation on GNU/Linux or Mac OS X) but care
|
||||
of the following variations.
|
||||
|
||||
To get Python download and run the latest `Python 3.7 32-bit (x86) executable
|
||||
installer <https://www.python.org/downloads/windows/>`_. Note that the 32-bit
|
||||
installer is required even on a 64-bit Windows system. If you use the 64-bit
|
||||
installer, step d. of the instruction might fail unless you installed some
|
||||
packages manually.
|
||||
|
||||
In some cases you have to install `MS Visual C++ 2015 build tools
|
||||
<https://www.microsoft.com/en-us/download/details.aspx?id=48159>`_ before you
|
||||
install the required python packages for OpenSlides (unfortunately Twisted
|
||||
needs it).
|
||||
|
||||
To setup and activate the virtual environment in step c. use::
|
||||
|
||||
> .virtualenv\Scripts\activate.bat
|
||||
|
||||
All other commands are the same as for GNU/Linux and Mac OS X.
|
||||
|
||||
|
||||
3. Running the test cases
|
||||
Running the test cases
|
||||
-------------------------
|
||||
|
||||
a. Running server tests
|
||||
For all services in submodules check out the documentation there.
|
||||
|
||||
Server tests andscripts
|
||||
'''''''''''''''''''''''
|
||||
You need to have python (>=3.8) and python-venv installed. Change your workdirectory to the server::
|
||||
|
||||
To run some server tests see `.travis.yml
|
||||
<https://github.com/OpenSlides/OpenSlides/blob/master/.travis.yml>`_.
|
||||
cd server
|
||||
|
||||
b. Client tests and commands
|
||||
''''''''''''''''''''''''''''
|
||||
Setup an python virtual environment. If you have already done it, you can skip this step:
|
||||
|
||||
Change to the client's directory to run every client related command. Run
|
||||
client tests::
|
||||
python3 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip install -U -r requirements.txt
|
||||
|
||||
$ npm test
|
||||
Make sure you are using the correct python version (e.g. try with explicit minor version: ``python3.8``). Activate it::
|
||||
|
||||
source .venv/bin/activate
|
||||
|
||||
To deactivate it type ``deactivate``. Running all tests and linters:
|
||||
|
||||
black openslides/ tests/
|
||||
flake8 openslides/ tests/
|
||||
mypy openslides/ tests/
|
||||
isort -rc openslides/ tests/
|
||||
pytest tests/
|
||||
|
||||
Client tests
|
||||
''''''''''''
|
||||
You need `node` and `npm` installed. Change to the client's directory. For the first time, install all dependencies::
|
||||
|
||||
cd client/
|
||||
npm install
|
||||
|
||||
Run client tests::
|
||||
|
||||
npm test
|
||||
|
||||
Fix the code format and lint it with::
|
||||
|
||||
$ npm run prettify-write
|
||||
$ npm run lint
|
||||
npm run cleanup
|
||||
|
||||
To extract translations run::
|
||||
|
||||
$ npm run extract
|
||||
|
||||
When updating, adding or changing used packages from npm, please update the
|
||||
README.md using following command::
|
||||
|
||||
$ npm run licenses
|
||||
|
||||
|
||||
4. Notes for running OpenSlides in larger setups
|
||||
------------------------------------------------
|
||||
|
||||
For productive setups refer to the docker-compose setup described in the main
|
||||
`README <https://github.com/OpenSlides/OpenSlides/blob/master/README.rst>`_.
|
||||
|
||||
While develpment it might be handy to use a cache and another database.
|
||||
PostgreSQL is recommended and Redis necessary as a cache. Both can be set up in
|
||||
the ``settings.py``. Please consider reading the `OpenSlides configuration
|
||||
<https://github.com/OpenSlides/OpenSlides/blob/master/server/SETTINGS.rst>`_ page
|
||||
to find out about all configurations, especially when using OpenSlides for big
|
||||
assemblies.
|
||||
|
||||
If you followed the instructions and installed the pip requirements form the
|
||||
``requirements.py`` all needed dependencies for another worker are installed.
|
||||
Instead of running ``python manage.py runserver`` you can use daphne or gunicorn
|
||||
(the latter is used in the prod setup)::
|
||||
|
||||
$ export DJANGO_SETTINGS_MODULE=settings
|
||||
$ export PYTHONPATH=personal_data/var/
|
||||
$ daphne -b 0.0.0.0 -p 8000 openslides.asgi:application
|
||||
|
||||
The last line may be interchangeable with gunicorn and uvicorn as protocol
|
||||
server::
|
||||
|
||||
$ gunicorn -w 4 -b 0.0.0.0:8000 -k uvicorn.workers.UvicornWorker openslides.asgi:application
|
||||
|
||||
npm run extract
|
||||
|
17
Makefile
Normal file
17
Makefile
Normal file
@ -0,0 +1,17 @@
|
||||
build-dev:
|
||||
make -C haproxy build-dev
|
||||
git submodule foreach 'make build-dev'
|
||||
docker-compose -f docker/docker-compose.dev.yml build
|
||||
|
||||
run-dev: | build-dev
|
||||
UID=$$(id -u $${USER}) GID=$$(id -g $${USER}) docker-compose -f docker/docker-compose.dev.yml up
|
||||
|
||||
stop-dev:
|
||||
docker-compose -f docker/docker-compose.dev.yml down
|
||||
|
||||
reload-haproxy:
|
||||
docker-compose -f docker/docker-compose.dev.yml kill -s HUP haproxy
|
||||
|
||||
get-server-shell:
|
||||
docker-compose -f docker/docker-compose.dev.yml run server bash
|
||||
|
@ -25,6 +25,8 @@ First, you have to clone this repository::
|
||||
$ git clone https://github.com/OpenSlides/OpenSlides.git
|
||||
$ cd OpenSlides/docker/
|
||||
|
||||
TODO: submodules.
|
||||
|
||||
You need to build the Docker images for the client and server with this
|
||||
script::
|
||||
|
||||
|
1
autoupdate
Submodule
1
autoupdate
Submodule
@ -0,0 +1 @@
|
||||
Subproject commit b187dd439bd7456e105901ca96cd0995862d4419
|
@ -10,8 +10,8 @@ RUN npm install -g @angular/cli@^10
|
||||
RUN ng config -g cli.warnings.versionMismatch false
|
||||
|
||||
USER openslides
|
||||
COPY package.json .
|
||||
COPY package-lock.json .
|
||||
COPY package.json package-lock.json ./
|
||||
RUN npm -v
|
||||
RUN npm ci
|
||||
COPY browserslist *.json ./
|
||||
COPY src ./src
|
||||
|
11
client/docker/Dockerfile.dev
Normal file
11
client/docker/Dockerfile.dev
Normal file
@ -0,0 +1,11 @@
|
||||
FROM node:13
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY package.json .
|
||||
RUN npm install
|
||||
RUN npm run postinstall
|
||||
|
||||
COPY . .
|
||||
|
||||
CMD npm start
|
@ -25,29 +25,6 @@ http {
|
||||
gzip_proxied expired no-cache no-store private auth;
|
||||
gzip_types text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript;
|
||||
|
||||
location /apps {
|
||||
proxy_pass http://server:8000;
|
||||
}
|
||||
location /media/ {
|
||||
proxy_pass http://media:8000;
|
||||
}
|
||||
location /rest {
|
||||
proxy_pass http://server:8000;
|
||||
}
|
||||
location /ws {
|
||||
proxy_pass http://server:8000;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "Upgrade";
|
||||
}
|
||||
location /server-version.txt {
|
||||
proxy_pass http://server:8000;
|
||||
}
|
||||
|
||||
location = /basic_status {
|
||||
stub_status;
|
||||
}
|
||||
|
||||
location / {
|
||||
try_files $uri $uri/ /index.html;
|
||||
}
|
||||
|
@ -29,7 +29,7 @@
|
||||
"dataGroups": [
|
||||
{
|
||||
"name": "api",
|
||||
"urls": ["/rest/*", "/apps/*"],
|
||||
"urls": ["/rest/*", "/apps/*", "/system/*", "/stats"],
|
||||
"cacheConfig": {
|
||||
"maxSize": 0,
|
||||
"maxAge": "0u",
|
||||
|
@ -11,6 +11,7 @@
|
||||
"scripts": {
|
||||
"ng": "ng",
|
||||
"start": "ng serve --proxy-config proxy.conf.json --host=0.0.0.0",
|
||||
"start-https": "ng serve --ssl --ssl-cert localhost.pem --ssl-key localhost-key.pem --proxy-config proxy.conf.json --host=0.0.0.0",
|
||||
"start-es5": "ng serve --proxy-config proxy.conf.json --host=0.0.0.0 --configuration es5",
|
||||
"build": "ng build --prod",
|
||||
"build-to-dir": "npm run build -- --output-path",
|
||||
|
@ -15,5 +15,9 @@
|
||||
"target": "ws://localhost:8000",
|
||||
"secure": false,
|
||||
"ws": true
|
||||
},
|
||||
"/system/": {
|
||||
"target": "https://localhost:8002",
|
||||
"secure": false
|
||||
}
|
||||
}
|
||||
|
@ -10,10 +10,9 @@ import { CountUsersService } from './core/ui-services/count-users.service';
|
||||
import { DataStoreUpgradeService } from './core/core-services/data-store-upgrade.service';
|
||||
import { LoadFontService } from './core/ui-services/load-font.service';
|
||||
import { LoginDataService } from './core/ui-services/login-data.service';
|
||||
import { OfflineService } from './core/core-services/offline.service';
|
||||
import { OperatorService } from './core/core-services/operator.service';
|
||||
import { OverlayService } from './core/ui-services/overlay.service';
|
||||
import { PingService } from './core/core-services/ping.service';
|
||||
import { PrioritizeService } from './core/core-services/prioritize.service';
|
||||
import { RoutingStateService } from './core/ui-services/routing-state.service';
|
||||
import { ServertimeService } from './core/core-services/servertime.service';
|
||||
import { ThemeService } from './core/ui-services/theme.service';
|
||||
@ -75,17 +74,16 @@ export class AppComponent {
|
||||
appRef: ApplicationRef,
|
||||
servertimeService: ServertimeService,
|
||||
router: Router,
|
||||
offlineService: OfflineService,
|
||||
operator: OperatorService,
|
||||
loginDataService: LoginDataService,
|
||||
constantsService: ConstantsService, // Needs to be started, so it can register itself to the WebsocketService
|
||||
constantsService: ConstantsService,
|
||||
themeService: ThemeService,
|
||||
overlayService: OverlayService,
|
||||
countUsersService: CountUsersService, // Needed to register itself.
|
||||
configService: ConfigService,
|
||||
loadFontService: LoadFontService,
|
||||
dataStoreUpgradeService: DataStoreUpgradeService, // to start it.
|
||||
prioritizeService: PrioritizeService,
|
||||
pingService: PingService,
|
||||
routingState: RoutingStateService,
|
||||
votingBannerService: VotingBannerService // needed for initialisation
|
||||
) {
|
||||
|
164
client/src/app/core/core-services/autoupdate-throttle.service.ts
Normal file
164
client/src/app/core/core-services/autoupdate-throttle.service.ts
Normal file
@ -0,0 +1,164 @@
|
||||
import { EventEmitter, Injectable } from '@angular/core';
|
||||
|
||||
import { Observable, Subject } from 'rxjs';
|
||||
|
||||
import { AutoupdateFormat } from 'app/core/definitions/autoupdate-format';
|
||||
import { trailingThrottleTime } from 'app/core/rxjs/trailing-throttle-time';
|
||||
import { Identifiable } from 'app/shared/models/base/identifiable';
|
||||
import { ConstantsService } from './constants.service';
|
||||
import { HttpService } from './http.service';
|
||||
|
||||
interface ThrottleSettings {
|
||||
AUTOUPDATE_DELAY?: number;
|
||||
}
|
||||
|
||||
@Injectable({
|
||||
providedIn: 'root'
|
||||
})
|
||||
export class AutoupdateThrottleService {
|
||||
private readonly _autoupdatesToInject = new Subject<AutoupdateFormat>();
|
||||
|
||||
public get autoupdatesToInject(): Observable<AutoupdateFormat> {
|
||||
return this._autoupdatesToInject.asObservable();
|
||||
}
|
||||
|
||||
private readonly receivedAutoupdate = new EventEmitter<void>();
|
||||
|
||||
private pendingAutoupdates = [];
|
||||
|
||||
private disabledUntil: number | null = null;
|
||||
|
||||
private delay = 0;
|
||||
|
||||
private maxSeenChangeId = 0;
|
||||
|
||||
private get isActive(): boolean {
|
||||
return this.delay !== 0;
|
||||
}
|
||||
|
||||
public constructor(private constantsService: ConstantsService, private httpService: HttpService) {
|
||||
this.constantsService.get<ThrottleSettings>('Settings').subscribe(settings => {
|
||||
// This is a one-shot. If the delay was set one time >0, it cannot be changed afterwards.
|
||||
// A change is more complicated since you have to unsubscribe, clean pending autoupdates,
|
||||
// subscribe again and make sure, that no autoupdate is missed.
|
||||
if (this.delay === 0 && settings.AUTOUPDATE_DELAY) {
|
||||
this.delay = 1000 * settings.AUTOUPDATE_DELAY;
|
||||
console.log(`Configured autoupdate delay: ${this.delay}ms`);
|
||||
this.receivedAutoupdate
|
||||
.pipe(trailingThrottleTime(this.delay))
|
||||
.subscribe(() => this.processPendingAutoupdates());
|
||||
} else if (this.delay === 0) {
|
||||
console.log('No autoupdate delay');
|
||||
}
|
||||
});
|
||||
|
||||
this.httpService.responseChangeIds.subscribe(changeId => this.disableUntil(changeId));
|
||||
}
|
||||
|
||||
public newAutoupdate(autoupdate: AutoupdateFormat): void {
|
||||
if (autoupdate.to_change_id > this.maxSeenChangeId) {
|
||||
this.maxSeenChangeId = autoupdate.to_change_id;
|
||||
}
|
||||
|
||||
if (!this.isActive) {
|
||||
this._autoupdatesToInject.next(autoupdate);
|
||||
} else if (this.disabledUntil !== null) {
|
||||
this._autoupdatesToInject.next(autoupdate);
|
||||
if (autoupdate.to_change_id >= this.disabledUntil) {
|
||||
this.disabledUntil = null;
|
||||
console.log('Throttling autoupdates again');
|
||||
}
|
||||
} else {
|
||||
this.pendingAutoupdates.push(autoupdate);
|
||||
this.receivedAutoupdate.emit();
|
||||
}
|
||||
}
|
||||
|
||||
public disableUntil(changeId: number): void {
|
||||
// Wait for an autoupdate with to_id >= changeId.
|
||||
console.log(this.delay);
|
||||
if (!this.isActive) {
|
||||
return;
|
||||
}
|
||||
this.processPendingAutoupdates();
|
||||
// Checking with maxSeenChangeId is for the following race condition:
|
||||
// If the autoupdate comes before the response, it must not be throttled.
|
||||
// But flushing pending autoupdates is important since *if* the autoupdate
|
||||
// was early, it is in the pending queue.
|
||||
if (changeId <= this.maxSeenChangeId) {
|
||||
return;
|
||||
}
|
||||
console.log('Disable autoupdate until change id', changeId);
|
||||
this.disabledUntil = changeId;
|
||||
}
|
||||
|
||||
private processPendingAutoupdates(): void {
|
||||
const autoupdates = this.pendingAutoupdates;
|
||||
if (autoupdates.length === 0) {
|
||||
return;
|
||||
}
|
||||
this.pendingAutoupdates = [];
|
||||
|
||||
const autoupdate = this.mergeAutoupdates(autoupdates);
|
||||
this._autoupdatesToInject.next(autoupdate);
|
||||
}
|
||||
|
||||
private mergeAutoupdates(autoupdates: AutoupdateFormat[]): AutoupdateFormat {
|
||||
const mergedAutoupdate: AutoupdateFormat = {
|
||||
changed: {},
|
||||
deleted: {},
|
||||
from_change_id: autoupdates[0].from_change_id,
|
||||
to_change_id: autoupdates[autoupdates.length - 1].to_change_id,
|
||||
all_data: false
|
||||
};
|
||||
|
||||
let lastToChangeId = null;
|
||||
for (const au of autoupdates) {
|
||||
if (lastToChangeId === null) {
|
||||
lastToChangeId = au.to_change_id;
|
||||
} else {
|
||||
if (au.from_change_id !== lastToChangeId) {
|
||||
console.warn('!!!', autoupdates, au);
|
||||
}
|
||||
lastToChangeId = au.to_change_id;
|
||||
}
|
||||
|
||||
this.applyAutoupdate(au, mergedAutoupdate);
|
||||
}
|
||||
|
||||
return mergedAutoupdate;
|
||||
}
|
||||
|
||||
private applyAutoupdate(from: AutoupdateFormat, into: AutoupdateFormat): void {
|
||||
if (from.all_data) {
|
||||
into.all_data = true;
|
||||
into.changed = from.changed;
|
||||
into.deleted = from.deleted;
|
||||
return;
|
||||
}
|
||||
|
||||
for (const collection of Object.keys(from.deleted)) {
|
||||
for (const id of from.deleted[collection]) {
|
||||
if (into.changed[collection]) {
|
||||
into.changed[collection] = into.changed[collection].filter(obj => (obj as Identifiable).id !== id);
|
||||
}
|
||||
if (!into.deleted[collection]) {
|
||||
into.deleted[collection] = [];
|
||||
}
|
||||
into.deleted[collection].push(id);
|
||||
}
|
||||
}
|
||||
|
||||
for (const collection of Object.keys(from.changed)) {
|
||||
for (const obj of from.changed[collection]) {
|
||||
if (into.deleted[collection]) {
|
||||
into.deleted[collection] = into.deleted[collection].filter(id => id !== (obj as Identifiable).id);
|
||||
}
|
||||
if (!into.changed[collection]) {
|
||||
into.changed[collection] = [];
|
||||
}
|
||||
into.changed[collection].push(obj);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
@ -1,57 +1,16 @@
|
||||
import { Injectable } from '@angular/core';
|
||||
|
||||
import { AutoupdateFormat } from '../definitions/autoupdate-format';
|
||||
import { AutoupdateThrottleService } from './autoupdate-throttle.service';
|
||||
import { BaseModel } from '../../shared/models/base/base-model';
|
||||
import { CollectionStringMapperService } from './collection-string-mapper.service';
|
||||
import { CommunicationManagerService, OfflineError } from './communication-manager.service';
|
||||
import { DataStoreService, DataStoreUpdateManagerService } from './data-store.service';
|
||||
import { HttpService } from './http.service';
|
||||
import { Mutex } from '../promises/mutex';
|
||||
import { WebsocketService, WEBSOCKET_ERROR_CODES } from './websocket.service';
|
||||
|
||||
export interface AutoupdateFormat {
|
||||
/**
|
||||
* All changed (and created) items as their full/restricted data grouped by their collection.
|
||||
*/
|
||||
changed: {
|
||||
[collectionString: string]: object[];
|
||||
};
|
||||
|
||||
/**
|
||||
* All deleted items (by id) grouped by their collection.
|
||||
*/
|
||||
deleted: {
|
||||
[collectionString: string]: number[];
|
||||
};
|
||||
|
||||
/**
|
||||
* The lower change id bond for this autoupdate
|
||||
*/
|
||||
from_change_id: number;
|
||||
|
||||
/**
|
||||
* The upper change id bound for this autoupdate
|
||||
*/
|
||||
to_change_id: number;
|
||||
|
||||
/**
|
||||
* Flag, if this autoupdate contains all data. If so, the DS needs to be resetted.
|
||||
*/
|
||||
all_data: boolean;
|
||||
}
|
||||
|
||||
export function isAutoupdateFormat(obj: any): obj is AutoupdateFormat {
|
||||
const format = obj as AutoupdateFormat;
|
||||
return (
|
||||
obj &&
|
||||
typeof obj === 'object' &&
|
||||
format.changed !== undefined &&
|
||||
format.deleted !== undefined &&
|
||||
format.from_change_id !== undefined &&
|
||||
format.to_change_id !== undefined &&
|
||||
format.all_data !== undefined
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Handles the initial update and automatic updates using the {@link WebsocketService}
|
||||
* Handles the initial update and automatic updates
|
||||
* Incoming objects, usually BaseModels, will be saved in the dataStore (`this.DS`)
|
||||
* This service usually creates all models
|
||||
*/
|
||||
@ -61,32 +20,49 @@ export function isAutoupdateFormat(obj: any): obj is AutoupdateFormat {
|
||||
export class AutoupdateService {
|
||||
private mutex = new Mutex();
|
||||
|
||||
/**
|
||||
* Constructor to create the AutoupdateService. Calls the constructor of the parent class.
|
||||
* @param websocketService
|
||||
* @param DS
|
||||
* @param modelMapper
|
||||
*/
|
||||
private streamCloseFn: () => void | null = null;
|
||||
|
||||
private lastMessageContainedAllData = false;
|
||||
|
||||
public constructor(
|
||||
private websocketService: WebsocketService,
|
||||
private DS: DataStoreService,
|
||||
private modelMapper: CollectionStringMapperService,
|
||||
private DSUpdateManager: DataStoreUpdateManagerService
|
||||
private DSUpdateManager: DataStoreUpdateManagerService,
|
||||
private communicationManager: CommunicationManagerService,
|
||||
private autoupdateThrottle: AutoupdateThrottleService
|
||||
) {
|
||||
this.websocketService.getOberservable<AutoupdateFormat>('autoupdate').subscribe(response => {
|
||||
this.storeResponse(response);
|
||||
});
|
||||
this.communicationManager.startCommunicationEvent.subscribe(() => this.startAutoupdate());
|
||||
|
||||
// Check for too high change id-errors. If this happens, reset the DS and get fresh data.
|
||||
this.websocketService.errorResponseObservable.subscribe(error => {
|
||||
if (error.code === WEBSOCKET_ERROR_CODES.CHANGE_ID_TOO_HIGH) {
|
||||
this.doFullUpdate();
|
||||
this.autoupdateThrottle.autoupdatesToInject.subscribe(autoupdate => this.storeAutoupdate(autoupdate));
|
||||
}
|
||||
|
||||
public async startAutoupdate(changeId?: number): Promise<void> {
|
||||
this.stopAutoupdate();
|
||||
|
||||
try {
|
||||
this.streamCloseFn = await this.communicationManager.subscribe<AutoupdateFormat>(
|
||||
'/system/autoupdate',
|
||||
autoupdate => {
|
||||
this.autoupdateThrottle.newAutoupdate(autoupdate);
|
||||
},
|
||||
() => ({ change_id: (changeId ? changeId : this.DS.maxChangeId).toString() })
|
||||
);
|
||||
} catch (e) {
|
||||
if (!(e instanceof OfflineError)) {
|
||||
console.error(e);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public stopAutoupdate(): void {
|
||||
if (this.streamCloseFn) {
|
||||
this.streamCloseFn();
|
||||
this.streamCloseFn = null;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle the answer of incoming data via {@link WebsocketService}.
|
||||
* Handle the answer of incoming data, after it was throttled.
|
||||
*
|
||||
* Detects the Class of an incomming model, creates a new empty object and assigns
|
||||
* the data to it using the deserialize function. Also models that are flagged as deleted
|
||||
@ -94,8 +70,9 @@ export class AutoupdateService {
|
||||
*
|
||||
* Handles the change ids of all autoupdates.
|
||||
*/
|
||||
private async storeResponse(autoupdate: AutoupdateFormat): Promise<void> {
|
||||
private async storeAutoupdate(autoupdate: AutoupdateFormat): Promise<void> {
|
||||
const unlock = await this.mutex.lock();
|
||||
this.lastMessageContainedAllData = autoupdate.all_data;
|
||||
if (autoupdate.all_data) {
|
||||
await this.storeAllData(autoupdate);
|
||||
} else {
|
||||
@ -138,17 +115,10 @@ export class AutoupdateService {
|
||||
} else {
|
||||
// autoupdate fully in the future. we are missing something!
|
||||
console.log('Autoupdate in the future', maxChangeId, autoupdate.from_change_id, autoupdate.to_change_id);
|
||||
this.requestChanges();
|
||||
this.startAutoupdate(); // restarts it.
|
||||
}
|
||||
}
|
||||
|
||||
public async injectAutoupdateIgnoreChangeId(autoupdate: AutoupdateFormat): Promise<void> {
|
||||
const unlock = await this.mutex.lock();
|
||||
console.debug('inject autoupdate', autoupdate);
|
||||
await this.injectAutupdateIntoDS(autoupdate, false);
|
||||
unlock();
|
||||
}
|
||||
|
||||
private async injectAutupdateIntoDS(autoupdate: AutoupdateFormat, flush: boolean): Promise<void> {
|
||||
const updateSlot = await this.DSUpdateManager.getNewUpdateSlot(this.DS);
|
||||
|
||||
@ -187,35 +157,21 @@ export class AutoupdateService {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Sends a WebSocket request to the Server with the maxChangeId of the DataStore.
|
||||
* The server should return an autoupdate with all new data.
|
||||
*/
|
||||
public requestChanges(): void {
|
||||
console.log(`requesting changed objects with DS max change id ${this.DS.maxChangeId}`);
|
||||
this.websocketService.send('getElements', { change_id: this.DS.maxChangeId });
|
||||
}
|
||||
|
||||
/**
|
||||
* Does a full update: Requests all data from the server and sets the DS to the fresh data.
|
||||
*/
|
||||
public async doFullUpdate(): Promise<void> {
|
||||
const oldChangeId = this.DS.maxChangeId;
|
||||
const response = await this.websocketService.sendAndGetResponse<{}, AutoupdateFormat>('getElements', {});
|
||||
|
||||
const updateSlot = await this.DSUpdateManager.getNewUpdateSlot(this.DS);
|
||||
let allModels: BaseModel[] = [];
|
||||
for (const collection of Object.keys(response.changed)) {
|
||||
if (this.modelMapper.isCollectionRegistered(collection)) {
|
||||
allModels = allModels.concat(this.mapObjectsToBaseModels(collection, response.changed[collection]));
|
||||
if (this.lastMessageContainedAllData) {
|
||||
console.log('full update requested. Skipping, last message already contained all data');
|
||||
} else {
|
||||
console.error(`Unregistered collection "${collection}". Ignore it.`);
|
||||
console.log('requesting full update.');
|
||||
// The mutex is needed, so the DS is not cleared, if there is
|
||||
// another autoupdate running.
|
||||
const unlock = await this.mutex.lock();
|
||||
this.stopAutoupdate();
|
||||
await this.DS.clear();
|
||||
this.startAutoupdate();
|
||||
unlock();
|
||||
}
|
||||
}
|
||||
|
||||
await this.DS.set(allModels, response.to_change_id);
|
||||
this.DSUpdateManager.commit(updateSlot, response.to_change_id, true);
|
||||
|
||||
console.log(`Full update done from ${oldChangeId} to ${response.to_change_id}`);
|
||||
}
|
||||
}
|
||||
|
@ -0,0 +1,186 @@
|
||||
import { HttpParams } from '@angular/common/http';
|
||||
import { EventEmitter, Injectable } from '@angular/core';
|
||||
|
||||
import { Observable } from 'rxjs';
|
||||
|
||||
import { HttpService } from './http.service';
|
||||
import { OfflineBroadcastService, OfflineReason } from './offline-broadcast.service';
|
||||
import { OperatorService } from './operator.service';
|
||||
import { SleepPromise } from '../promises/sleep';
|
||||
import {
|
||||
CommunicationError,
|
||||
ErrorType,
|
||||
Stream,
|
||||
StreamContainer,
|
||||
StreamingCommunicationService,
|
||||
verboseErrorType
|
||||
} from './streaming-communication.service';
|
||||
|
||||
type HttpParamsGetter = () => HttpParams | { [param: string]: string | string[] };
|
||||
|
||||
const MAX_STREAM_FAILURE_RETRIES = 3;
|
||||
|
||||
export class OfflineError extends Error {
|
||||
public constructor() {
|
||||
super('');
|
||||
this.name = 'OfflineError';
|
||||
}
|
||||
}
|
||||
|
||||
interface StreamConnectionWrapper {
|
||||
id: number;
|
||||
url: string;
|
||||
messageHandler: (message: any) => void;
|
||||
params: HttpParamsGetter;
|
||||
stream?: Stream<any>;
|
||||
hasErroredAmount: number;
|
||||
}
|
||||
|
||||
@Injectable({
|
||||
providedIn: 'root'
|
||||
})
|
||||
export class CommunicationManagerService {
|
||||
private communicationAllowed = false;
|
||||
|
||||
private readonly _startCommunicationEvent = new EventEmitter<void>();
|
||||
|
||||
public get startCommunicationEvent(): Observable<void> {
|
||||
return this._startCommunicationEvent.asObservable();
|
||||
}
|
||||
|
||||
private readonly _stopCommunicationEvent = new EventEmitter<void>();
|
||||
|
||||
public get stopCommunicationEvent(): Observable<void> {
|
||||
return this._stopCommunicationEvent.asObservable();
|
||||
}
|
||||
|
||||
private streamContainers: { [id: number]: StreamContainer } = {};
|
||||
|
||||
public constructor(
|
||||
private streamingCommunicationService: StreamingCommunicationService,
|
||||
private offlineBroadcastService: OfflineBroadcastService,
|
||||
private http: HttpService,
|
||||
private operatorService: OperatorService
|
||||
) {
|
||||
this.offlineBroadcastService.goOfflineObservable.subscribe(() => this.closeConnections());
|
||||
}
|
||||
|
||||
public async subscribe<T>(
|
||||
url: string,
|
||||
messageHandler: (message: T) => void,
|
||||
params?: HttpParamsGetter
|
||||
): Promise<() => void> {
|
||||
if (!params) {
|
||||
params = () => null;
|
||||
}
|
||||
|
||||
const streamContainer = new StreamContainer(url, messageHandler, params);
|
||||
return await this.connectWithWrapper(streamContainer);
|
||||
}
|
||||
|
||||
public startCommunication(): void {
|
||||
if (this.communicationAllowed) {
|
||||
console.error('Illegal state! Do not emit this event multiple times');
|
||||
} else {
|
||||
this.communicationAllowed = true;
|
||||
this._startCommunicationEvent.emit();
|
||||
}
|
||||
}
|
||||
|
||||
private async connectWithWrapper(streamContainer: StreamContainer): Promise<() => void> {
|
||||
console.log('connect', streamContainer, streamContainer.stream);
|
||||
const errorHandler = (type: ErrorType, error: CommunicationError, message: string) =>
|
||||
this.handleError(streamContainer, type, error, message);
|
||||
this.streamingCommunicationService.subscribe(streamContainer, errorHandler);
|
||||
this.streamContainers[streamContainer.id] = streamContainer;
|
||||
return () => this.close(streamContainer);
|
||||
}
|
||||
|
||||
private async handleError(
|
||||
streamContainer: StreamContainer,
|
||||
type: ErrorType,
|
||||
error: CommunicationError,
|
||||
message: string
|
||||
): Promise<void> {
|
||||
console.log('handle Error', streamContainer, streamContainer.stream, verboseErrorType(type), error, message);
|
||||
streamContainer.stream.close();
|
||||
streamContainer.stream = null;
|
||||
|
||||
streamContainer.hasErroredAmount++;
|
||||
if (streamContainer.hasErroredAmount > MAX_STREAM_FAILURE_RETRIES) {
|
||||
this.goOffline(streamContainer, OfflineReason.ConnectionLost);
|
||||
} else if (type === ErrorType.Client && error.type === 'ErrorAuth') {
|
||||
this.goOffline(streamContainer, OfflineReason.WhoAmIFailed);
|
||||
} else {
|
||||
// retry it after some time:
|
||||
console.log(
|
||||
`Retry no. ${streamContainer.hasErroredAmount} of ${MAX_STREAM_FAILURE_RETRIES} for ${streamContainer.url}`
|
||||
);
|
||||
try {
|
||||
await this.delayAndCheckReconnection();
|
||||
await this.connectWithWrapper(streamContainer);
|
||||
} catch (e) {
|
||||
// delayAndCheckReconnection can throw an OfflineError,
|
||||
// which are just an 'abord mission' signal. Here, those errors can be ignored.
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private async delayAndCheckReconnection(): Promise<void> {
|
||||
const delay = Math.floor(Math.random() * 3000 + 2000);
|
||||
console.log(`retry again in ${delay} ms`);
|
||||
|
||||
await SleepPromise(delay);
|
||||
|
||||
// do not continue, if we are offline!
|
||||
if (this.offlineBroadcastService.isOffline()) {
|
||||
console.log('we are offline?');
|
||||
throw new OfflineError();
|
||||
}
|
||||
|
||||
// do not continue, if we are offline!
|
||||
if (!this.shouldRetryConnecting()) {
|
||||
console.log('operator changed, do not rety');
|
||||
throw new OfflineError(); // TODO: This error is not really good....
|
||||
}
|
||||
}
|
||||
|
||||
public closeConnections(): void {
|
||||
for (const streamWrapper of Object.values(this.streamContainers)) {
|
||||
if (streamWrapper.stream) {
|
||||
streamWrapper.stream.close();
|
||||
}
|
||||
}
|
||||
this.streamContainers = {};
|
||||
this.communicationAllowed = false;
|
||||
this._stopCommunicationEvent.emit();
|
||||
}
|
||||
|
||||
private goOffline(streamContainer: StreamContainer, reason: OfflineReason): void {
|
||||
delete this.streamContainers[streamContainer.id];
|
||||
this.closeConnections(); // here we close the connections early.
|
||||
this.offlineBroadcastService.goOffline(reason);
|
||||
}
|
||||
|
||||
private close(streamConnectionWrapper: StreamConnectionWrapper): void {
|
||||
if (this.streamContainers[streamConnectionWrapper.id]) {
|
||||
this.streamContainers[streamConnectionWrapper.id].stream.close();
|
||||
delete this.streamContainers[streamConnectionWrapper.id];
|
||||
}
|
||||
}
|
||||
|
||||
// Checks the operator: If we do not have a valid user,
|
||||
// do not even try to connect again..
|
||||
private shouldRetryConnecting(): boolean {
|
||||
return this.operatorService.guestsEnabled || !!this.operatorService.user;
|
||||
}
|
||||
|
||||
public async isCommunicationServiceOnline(): Promise<boolean> {
|
||||
try {
|
||||
const response = await this.http.get<{ healthy: boolean }>('/system/health');
|
||||
return !!response.healthy;
|
||||
} catch (e) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
}
|
@ -1,9 +1,11 @@
|
||||
import { Injectable } from '@angular/core';
|
||||
|
||||
import { environment } from 'environments/environment';
|
||||
import { BehaviorSubject, Observable } from 'rxjs';
|
||||
import { filter } from 'rxjs/operators';
|
||||
|
||||
import { WebsocketService } from './websocket.service';
|
||||
import { CommunicationManagerService } from './communication-manager.service';
|
||||
import { HttpService } from './http.service';
|
||||
|
||||
/**
|
||||
* constants have a key associated with the data.
|
||||
@ -36,24 +38,15 @@ export class ConstantsService {
|
||||
*/
|
||||
private subjects: { [key: string]: BehaviorSubject<any> } = {};
|
||||
|
||||
/**
|
||||
* @param websocketService
|
||||
*/
|
||||
public constructor(private websocketService: WebsocketService) {
|
||||
// The hook for recieving constants.
|
||||
websocketService.getOberservable<Constants>('constants').subscribe(constants => {
|
||||
this.constants = constants;
|
||||
public constructor(communicationManager: CommunicationManagerService, private http: HttpService) {
|
||||
communicationManager.startCommunicationEvent.subscribe(async () => {
|
||||
console.log('start communication');
|
||||
this.constants = await this.http.get<Constants>(environment.urlPrefix + '/core/constants/');
|
||||
console.log('constants:', this.constants);
|
||||
Object.keys(this.subjects).forEach(key => {
|
||||
this.subjects[key].next(this.constants[key]);
|
||||
});
|
||||
});
|
||||
|
||||
// We can request constants, if the websocket connection opens.
|
||||
// On retries, the `refresh()` method is called by the OpenSlidesService, so
|
||||
// here we do not need to take care about this.
|
||||
websocketService.noRetryConnectEvent.subscribe(() => {
|
||||
this.refresh();
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
@ -66,14 +59,4 @@ export class ConstantsService {
|
||||
}
|
||||
return this.subjects[key].asObservable().pipe(filter(x => !!x));
|
||||
}
|
||||
|
||||
/**
|
||||
* Refreshed the constants
|
||||
*/
|
||||
public refresh(): Promise<void> {
|
||||
if (!this.websocketService.isConnected) {
|
||||
return;
|
||||
}
|
||||
this.websocketService.send('constants', {});
|
||||
}
|
||||
}
|
||||
|
@ -361,12 +361,12 @@ export class DataStoreService {
|
||||
*
|
||||
* @returns The max change id.
|
||||
*/
|
||||
public async initFromStorage(): Promise<number> {
|
||||
public async initFromStorage(): Promise<void> {
|
||||
// This promise will be resolved with cached datastore.
|
||||
const store = await this.storageService.get<JsonStorage>(DataStoreService.cachePrefix + 'DS');
|
||||
if (!store) {
|
||||
await this.clear();
|
||||
return this.maxChangeId;
|
||||
return;
|
||||
}
|
||||
|
||||
const updateSlot = await this.DSUpdateManager.getNewUpdateSlot(this);
|
||||
@ -395,7 +395,6 @@ export class DataStoreService {
|
||||
this.DSUpdateManager.dropUpdateSlot();
|
||||
await this.clear();
|
||||
}
|
||||
return this.maxChangeId;
|
||||
}
|
||||
|
||||
/**
|
||||
@ -670,6 +669,6 @@ export class DataStoreService {
|
||||
public print(): void {
|
||||
console.log('Max change id', this.maxChangeId);
|
||||
console.log(JSON.stringify(this.jsonStore));
|
||||
console.log(this.modelStore);
|
||||
console.log(JSON.parse(JSON.stringify(this.modelStore)));
|
||||
}
|
||||
}
|
||||
|
@ -2,22 +2,14 @@ import { HttpClient, HttpErrorResponse, HttpHeaders } from '@angular/common/http
|
||||
import { Injectable } from '@angular/core';
|
||||
|
||||
import { TranslateService } from '@ngx-translate/core';
|
||||
import { Subject } from 'rxjs';
|
||||
|
||||
import { AutoupdateFormat, AutoupdateService, isAutoupdateFormat } from './autoupdate.service';
|
||||
import { AutoupdateFormat } from '../definitions/autoupdate-format';
|
||||
import { AutoupdateThrottleService } from './autoupdate-throttle.service';
|
||||
import { HTTPMethod } from '../definitions/http-methods';
|
||||
import { OpenSlidesStatusService } from './openslides-status.service';
|
||||
import { formatQueryParams, QueryParams } from '../definitions/query-params';
|
||||
|
||||
/**
|
||||
* Enum for different HTTPMethods
|
||||
*/
|
||||
export enum HTTPMethod {
|
||||
GET = 'get',
|
||||
POST = 'post',
|
||||
PUT = 'put',
|
||||
PATCH = 'patch',
|
||||
DELETE = 'delete'
|
||||
}
|
||||
|
||||
export interface ErrorDetailResponse {
|
||||
detail: string | string[];
|
||||
args?: string[];
|
||||
@ -33,12 +25,12 @@ function isErrorDetailResponse(obj: any): obj is ErrorDetailResponse {
|
||||
}
|
||||
|
||||
interface AutoupdateResponse {
|
||||
autoupdate: AutoupdateFormat;
|
||||
change_id: number;
|
||||
data?: any;
|
||||
}
|
||||
|
||||
function isAutoupdateReponse(obj: any): obj is AutoupdateResponse {
|
||||
return obj && typeof obj === 'object' && isAutoupdateFormat((obj as AutoupdateResponse).autoupdate);
|
||||
return obj && typeof obj === 'object' && typeof (obj as AutoupdateResponse).change_id === 'number';
|
||||
}
|
||||
|
||||
/**
|
||||
@ -53,6 +45,8 @@ export class HttpService {
|
||||
*/
|
||||
private defaultHeaders: HttpHeaders;
|
||||
|
||||
public readonly responseChangeIds = new Subject<number>();
|
||||
|
||||
/**
|
||||
* Construct a HttpService
|
||||
*
|
||||
@ -65,8 +59,7 @@ export class HttpService {
|
||||
public constructor(
|
||||
private http: HttpClient,
|
||||
private translate: TranslateService,
|
||||
private OSStatus: OpenSlidesStatusService,
|
||||
private autoupdateService: AutoupdateService
|
||||
private OSStatus: OpenSlidesStatusService
|
||||
) {
|
||||
this.defaultHeaders = new HttpHeaders().set('Content-Type', 'application/json');
|
||||
}
|
||||
@ -205,8 +198,8 @@ export class HttpService {
|
||||
|
||||
private processResponse<T>(responseData: T): T {
|
||||
if (isAutoupdateReponse(responseData)) {
|
||||
this.autoupdateService.injectAutoupdateIgnoreChangeId(responseData.autoupdate);
|
||||
responseData = responseData.data;
|
||||
this.responseChangeIds.next(responseData.change_id);
|
||||
return responseData.data;
|
||||
}
|
||||
return responseData;
|
||||
}
|
||||
|
@ -2,8 +2,9 @@ import { Injectable } from '@angular/core';
|
||||
|
||||
import { Observable, Subject } from 'rxjs';
|
||||
|
||||
import { CommunicationManagerService, OfflineError } from './communication-manager.service';
|
||||
import { HttpService } from './http.service';
|
||||
import { OperatorService } from './operator.service';
|
||||
import { WebsocketService } from './websocket.service';
|
||||
|
||||
/**
|
||||
* Encapslates the name and content of every message regardless of being a request or response.
|
||||
@ -17,7 +18,12 @@ interface NotifyBase<T> {
|
||||
/**
|
||||
* The content to send.
|
||||
*/
|
||||
content: T;
|
||||
message: T;
|
||||
}
|
||||
|
||||
function isNotifyBase(obj: object): obj is NotifyResponse<any> {
|
||||
const base = obj as NotifyBase<any>;
|
||||
return !!obj && base.message !== undefined && base.name !== undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
@ -26,15 +32,18 @@ interface NotifyBase<T> {
|
||||
* channel names.
|
||||
*/
|
||||
export interface NotifyRequest<T> extends NotifyBase<T> {
|
||||
channel_id: string;
|
||||
to_all?: boolean;
|
||||
|
||||
/**
|
||||
* User ids (or `true` for all users) to send this message to.
|
||||
*/
|
||||
users?: number[] | boolean;
|
||||
to_users?: number[];
|
||||
|
||||
/**
|
||||
* An array of channels to send this message to.
|
||||
*/
|
||||
replyChannels?: string[];
|
||||
to_channels?: string[];
|
||||
}
|
||||
|
||||
/**
|
||||
@ -45,12 +54,12 @@ export interface NotifyResponse<T> extends NotifyBase<T> {
|
||||
* This is the channel name of the one, who sends this message. Can be use to directly
|
||||
* answer this message.
|
||||
*/
|
||||
senderChannelName: string;
|
||||
sender_channel_id: string;
|
||||
|
||||
/**
|
||||
* The user id of the user who sends this message. It is 0 for Anonymous.
|
||||
*/
|
||||
senderUserId: number;
|
||||
sender_user_id: number;
|
||||
|
||||
/**
|
||||
* This is validated here and is true, if the senderUserId matches the current operator's id.
|
||||
@ -59,6 +68,20 @@ export interface NotifyResponse<T> extends NotifyBase<T> {
|
||||
sendByThisUser: boolean;
|
||||
}
|
||||
|
||||
function isNotifyResponse(obj: object): obj is NotifyResponse<any> {
|
||||
const response = obj as NotifyResponse<any>;
|
||||
// Note: we do not test for sendByThisUser, since it is set later in our code.
|
||||
return isNotifyBase(obj) && response.sender_channel_id !== undefined && response.sender_user_id !== undefined;
|
||||
}
|
||||
|
||||
interface ChannelIdResponse {
|
||||
channel_id: string;
|
||||
}
|
||||
|
||||
function isChannelIdResponse(obj: object): obj is ChannelIdResponse {
|
||||
return !!obj && (obj as ChannelIdResponse).channel_id !== undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Handles all incoming and outgoing notify messages via {@link WebsocketService}.
|
||||
*/
|
||||
@ -78,18 +101,41 @@ export class NotifyService {
|
||||
[name: string]: Subject<NotifyResponse<any>>;
|
||||
} = {};
|
||||
|
||||
/**
|
||||
* Constructor to create the NotifyService. Registers itself to the WebsocketService.
|
||||
* @param websocketService
|
||||
*/
|
||||
public constructor(private websocketService: WebsocketService, private operator: OperatorService) {
|
||||
websocketService.getOberservable<NotifyResponse<any>>('notify').subscribe(notify => {
|
||||
notify.sendByThisUser = notify.senderUserId === (this.operator.user ? this.operator.user.id : 0);
|
||||
private channelId: string;
|
||||
|
||||
public constructor(
|
||||
private communicationManager: CommunicationManagerService,
|
||||
private http: HttpService,
|
||||
private operator: OperatorService
|
||||
) {
|
||||
this.communicationManager.startCommunicationEvent.subscribe(() => this.startListening());
|
||||
this.communicationManager.stopCommunicationEvent.subscribe(() => (this.channelId = null));
|
||||
}
|
||||
|
||||
private async startListening(): Promise<void> {
|
||||
try {
|
||||
await this.communicationManager.subscribe<NotifyResponse<any> | ChannelIdResponse>(
|
||||
'/system/notify',
|
||||
notify => {
|
||||
if (isChannelIdResponse(notify)) {
|
||||
this.channelId = notify.channel_id;
|
||||
} else if (isNotifyResponse(notify)) {
|
||||
notify.sendByThisUser =
|
||||
notify.sender_user_id === (this.operator.user ? this.operator.user.id : 0);
|
||||
this.notifySubject.next(notify);
|
||||
if (this.messageSubjects[notify.name]) {
|
||||
this.messageSubjects[notify.name].next(notify);
|
||||
}
|
||||
});
|
||||
} else {
|
||||
console.error('Unknwon notify message', notify);
|
||||
}
|
||||
}
|
||||
);
|
||||
} catch (e) {
|
||||
if (!(e instanceof OfflineError)) {
|
||||
console.log(e);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
@ -97,8 +143,8 @@ export class NotifyService {
|
||||
* @param name The name of the notify message
|
||||
* @param content The payload to send
|
||||
*/
|
||||
public sendToAllUsers<T>(name: string, content: T): void {
|
||||
this.send(name, content);
|
||||
public async sendToAllUsers<T>(name: string, content: T): Promise<void> {
|
||||
await this.send(name, content, true);
|
||||
}
|
||||
|
||||
/**
|
||||
@ -107,8 +153,11 @@ export class NotifyService {
|
||||
* @param content The payload to send.
|
||||
* @param users Multiple user ids.
|
||||
*/
|
||||
public sendToUsers<T>(name: string, content: T, ...users: number[]): void {
|
||||
this.send(name, content, users);
|
||||
public async sendToUsers<T>(name: string, content: T, ...users: number[]): Promise<void> {
|
||||
if (users.length < 1) {
|
||||
throw new Error('You have to provide at least one user');
|
||||
}
|
||||
await this.send(name, content, false, users);
|
||||
}
|
||||
|
||||
/**
|
||||
@ -117,35 +166,48 @@ export class NotifyService {
|
||||
* @param content The payload to send.
|
||||
* @param channels Multiple channels to send this message to.
|
||||
*/
|
||||
public sendToChannels<T>(name: string, content: T, ...channels: string[]): void {
|
||||
public async sendToChannels<T>(name: string, content: T, ...channels: string[]): Promise<void> {
|
||||
if (channels.length < 1) {
|
||||
throw new Error('You have to provide at least one channel');
|
||||
}
|
||||
this.send(name, content, null, channels);
|
||||
await this.send(name, content, false, null, channels);
|
||||
}
|
||||
|
||||
/**
|
||||
* General send function for notify messages.
|
||||
* @param name The name of the notify message
|
||||
* @param content The payload to send.
|
||||
* @param message The payload to send.
|
||||
* @param users Either an array of IDs or `true` meaning of sending this message to all online users clients.
|
||||
* @param channels An array of channels to send this message to.
|
||||
*/
|
||||
public send<T>(name: string, content: T, users?: number[] | boolean, channels?: string[]): void {
|
||||
private async send<T>(
|
||||
name: string,
|
||||
message: T,
|
||||
toAll?: boolean,
|
||||
users?: number[],
|
||||
channels?: string[]
|
||||
): Promise<void> {
|
||||
if (!this.channelId) {
|
||||
throw new Error('No channel id!');
|
||||
}
|
||||
|
||||
const notify: NotifyRequest<T> = {
|
||||
name: name,
|
||||
content: content
|
||||
message: message,
|
||||
channel_id: this.channelId
|
||||
};
|
||||
if (typeof users === 'boolean' && users !== true) {
|
||||
throw new Error('You just can give true as a boolean to send this message to all users.');
|
||||
if (toAll === true) {
|
||||
notify.to_all = true;
|
||||
}
|
||||
if (users !== null) {
|
||||
notify.users = users;
|
||||
if (users) {
|
||||
notify.to_users = users;
|
||||
}
|
||||
if (channels !== null) {
|
||||
notify.replyChannels = channels;
|
||||
if (channels) {
|
||||
notify.to_channels = channels;
|
||||
}
|
||||
this.websocketService.send('notify', notify);
|
||||
|
||||
console.debug('send notify', notify);
|
||||
await this.http.post<unknown>('/system/notify/send', notify);
|
||||
}
|
||||
|
||||
/**
|
||||
|
@ -0,0 +1,37 @@
|
||||
import { EventEmitter, Injectable } from '@angular/core';
|
||||
|
||||
import { BehaviorSubject, Observable } from 'rxjs';
|
||||
|
||||
export enum OfflineReason {
|
||||
WhoAmIFailed,
|
||||
ConnectionLost
|
||||
}
|
||||
|
||||
@Injectable({
|
||||
providedIn: 'root'
|
||||
})
|
||||
export class OfflineBroadcastService {
|
||||
public readonly isOfflineSubject = new BehaviorSubject<boolean>(false);
|
||||
public get isOfflineObservable(): Observable<boolean> {
|
||||
return this.isOfflineSubject.asObservable();
|
||||
}
|
||||
|
||||
private readonly _goOffline = new EventEmitter<OfflineReason>();
|
||||
public get goOfflineObservable(): Observable<OfflineReason> {
|
||||
return this._goOffline.asObservable();
|
||||
}
|
||||
|
||||
public constructor() {}
|
||||
|
||||
public goOffline(reason: OfflineReason): void {
|
||||
this._goOffline.emit(reason);
|
||||
}
|
||||
|
||||
public isOffline(): boolean {
|
||||
return this.isOfflineSubject.getValue();
|
||||
}
|
||||
|
||||
public isOnline(): boolean {
|
||||
return !this.isOffline();
|
||||
}
|
||||
}
|
@ -1,10 +1,9 @@
|
||||
import { Injectable } from '@angular/core';
|
||||
|
||||
import { marker as _ } from '@biesbjerg/ngx-translate-extract-marker';
|
||||
import { TranslateService } from '@ngx-translate/core';
|
||||
import { BehaviorSubject, Observable } from 'rxjs';
|
||||
|
||||
import { BannerDefinition, BannerService } from '../ui-services/banner.service';
|
||||
import { CommunicationManagerService } from './communication-manager.service';
|
||||
import { OfflineBroadcastService, OfflineReason } from './offline-broadcast.service';
|
||||
import { OpenSlidesService } from './openslides.service';
|
||||
import { OperatorService, WhoAmI } from './operator.service';
|
||||
|
||||
/**
|
||||
* This service handles everything connected with being offline.
|
||||
@ -16,63 +15,111 @@ import { BannerDefinition, BannerService } from '../ui-services/banner.service';
|
||||
providedIn: 'root'
|
||||
})
|
||||
export class OfflineService {
|
||||
/**
|
||||
* BehaviorSubject to receive further status values.
|
||||
*/
|
||||
private offline = new BehaviorSubject<boolean>(false);
|
||||
private bannerDefinition: BannerDefinition = {
|
||||
text: _('Offline mode'),
|
||||
icon: 'cloud_off'
|
||||
};
|
||||
private reason: OfflineReason | null;
|
||||
|
||||
public constructor(private banner: BannerService, translate: TranslateService) {
|
||||
translate.onLangChange.subscribe(() => {
|
||||
this.bannerDefinition.text = translate.instant(this.bannerDefinition.text);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Determines of you are either in Offline mode or not connected via websocket
|
||||
*
|
||||
* @returns whether the client is offline or not connected
|
||||
*/
|
||||
public isOffline(): Observable<boolean> {
|
||||
return this.offline;
|
||||
}
|
||||
|
||||
/**
|
||||
* Sets the offline flag. Restores the DataStoreService to the last known configuration.
|
||||
*/
|
||||
public goOfflineBecauseFailedWhoAmI(): void {
|
||||
if (!this.offline.getValue()) {
|
||||
console.log('offline because whoami failed.');
|
||||
}
|
||||
this.goOffline();
|
||||
}
|
||||
|
||||
/**
|
||||
* Sets the offline flag, because there is no connection to the server.
|
||||
*/
|
||||
public goOfflineBecauseConnectionLost(): void {
|
||||
if (!this.offline.getValue()) {
|
||||
console.log('offline because connection lost.');
|
||||
}
|
||||
this.goOffline();
|
||||
public constructor(
|
||||
private OpenSlides: OpenSlidesService,
|
||||
private offlineBroadcastService: OfflineBroadcastService,
|
||||
private operatorService: OperatorService,
|
||||
private communicationManager: CommunicationManagerService
|
||||
) {
|
||||
this.offlineBroadcastService.goOfflineObservable.subscribe((reason: OfflineReason) => this.goOffline(reason));
|
||||
}
|
||||
|
||||
/**
|
||||
* Helper function to set offline status
|
||||
*/
|
||||
private goOffline(): void {
|
||||
this.offline.next(true);
|
||||
this.banner.addBanner(this.bannerDefinition);
|
||||
public goOffline(reason: OfflineReason): void {
|
||||
if (this.offlineBroadcastService.isOffline()) {
|
||||
return;
|
||||
}
|
||||
this.reason = reason;
|
||||
|
||||
if (reason === OfflineReason.ConnectionLost) {
|
||||
console.log('offline because connection lost.');
|
||||
} else if (reason === OfflineReason.WhoAmIFailed) {
|
||||
console.log('offline because whoami failed.');
|
||||
} else {
|
||||
console.error('No such offline reason', reason);
|
||||
}
|
||||
|
||||
this.offlineBroadcastService.isOfflineSubject.next(true);
|
||||
this.checkStillOffline();
|
||||
}
|
||||
|
||||
private checkStillOffline(): void {
|
||||
const timeout = Math.floor(Math.random() * 3000 + 2000);
|
||||
console.log(`Try to go online in ${timeout} ms`);
|
||||
|
||||
setTimeout(async () => {
|
||||
let online: boolean;
|
||||
let whoami: WhoAmI | null = null;
|
||||
|
||||
if (this.reason === OfflineReason.ConnectionLost) {
|
||||
online = await this.communicationManager.isCommunicationServiceOnline();
|
||||
console.log('is communication online? ', online);
|
||||
} else if (this.reason === OfflineReason.WhoAmIFailed) {
|
||||
const result = await this.operatorService.whoAmI();
|
||||
online = result.online;
|
||||
whoami = result.whoami;
|
||||
console.log('is whoami reachable?', online);
|
||||
}
|
||||
|
||||
if (online) {
|
||||
await this.goOnline(whoami);
|
||||
// TODO: check all other reasons -> e.g. if the
|
||||
// connection was lost, the operator must be checked and the other way
|
||||
// around the comminucation must be started!!
|
||||
|
||||
// stop trying.
|
||||
} else {
|
||||
// continue trying.
|
||||
this.checkStillOffline();
|
||||
}
|
||||
}, timeout);
|
||||
}
|
||||
|
||||
/**
|
||||
* Function to return to online-status.
|
||||
*
|
||||
* First, we have to check, if all other sources (except this.reason) are online, too.
|
||||
* This results in definetly having a whoami response at this point.
|
||||
* If this is the case, we need to setup everything again:
|
||||
* 1) check the operator. If this allowes for an logged in state (or anonymous is OK), do
|
||||
* step 2, otherwise done.
|
||||
* 2) enable communications.
|
||||
*/
|
||||
public goOnline(): void {
|
||||
this.offline.next(false);
|
||||
this.banner.removeBanner(this.bannerDefinition);
|
||||
private async goOnline(whoami?: WhoAmI): Promise<void> {
|
||||
console.log('go online!', this.reason, whoami);
|
||||
if (this.reason === OfflineReason.ConnectionLost) {
|
||||
// now we have to check whoami
|
||||
const result = await this.operatorService.whoAmI();
|
||||
if (!result.online) {
|
||||
console.log('whoami down.');
|
||||
this.reason = OfflineReason.WhoAmIFailed;
|
||||
this.checkStillOffline();
|
||||
return;
|
||||
}
|
||||
whoami = result.whoami;
|
||||
} else if (this.reason === OfflineReason.WhoAmIFailed) {
|
||||
const online = await this.communicationManager.isCommunicationServiceOnline();
|
||||
if (!online) {
|
||||
console.log('communication down.');
|
||||
this.reason = OfflineReason.ConnectionLost;
|
||||
this.checkStillOffline();
|
||||
return;
|
||||
}
|
||||
}
|
||||
console.log('we are online!');
|
||||
|
||||
// Ok, we are online now!
|
||||
const isLoggedIn = await this.OpenSlides.checkWhoAmI(whoami);
|
||||
console.log('logged in:', isLoggedIn);
|
||||
if (isLoggedIn) {
|
||||
this.communicationManager.startCommunication();
|
||||
}
|
||||
console.log('done');
|
||||
|
||||
this.offlineBroadcastService.isOfflineSubject.next(false);
|
||||
}
|
||||
}
|
||||
|
@ -3,12 +3,11 @@ import { Router } from '@angular/router';
|
||||
|
||||
import { BehaviorSubject } from 'rxjs';
|
||||
|
||||
import { AutoupdateService } from './autoupdate.service';
|
||||
import { ConstantsService } from './constants.service';
|
||||
import { CommunicationManagerService } from './communication-manager.service';
|
||||
import { DataStoreService } from './data-store.service';
|
||||
import { OperatorService } from './operator.service';
|
||||
import { OfflineBroadcastService, OfflineReason } from './offline-broadcast.service';
|
||||
import { OperatorService, WhoAmI } from './operator.service';
|
||||
import { StorageService } from './storage.service';
|
||||
import { WebsocketService } from './websocket.service';
|
||||
|
||||
/**
|
||||
* Handles the bootup/showdown of this application.
|
||||
@ -35,29 +34,19 @@ export class OpenSlidesService {
|
||||
return this.booted.value;
|
||||
}
|
||||
|
||||
/**
|
||||
* Constructor to create the OpenSlidesService. Registers itself to the WebsocketService.
|
||||
* @param storageService
|
||||
* @param operator
|
||||
* @param websocketService
|
||||
* @param router
|
||||
* @param autoupdateService
|
||||
* @param DS
|
||||
*/
|
||||
public constructor(
|
||||
private storageService: StorageService,
|
||||
private operator: OperatorService,
|
||||
private websocketService: WebsocketService,
|
||||
private router: Router,
|
||||
private autoupdateService: AutoupdateService,
|
||||
private DS: DataStoreService,
|
||||
private constantsService: ConstantsService
|
||||
private communicationManager: CommunicationManagerService,
|
||||
private offlineBroadcastService: OfflineBroadcastService
|
||||
) {
|
||||
// Handler that gets called, if the websocket connection reconnects after a disconnection.
|
||||
// There might have changed something on the server, so we check the operator, if he changed.
|
||||
websocketService.retryReconnectEvent.subscribe(() => {
|
||||
/*websocketService.retryReconnectEvent.subscribe(() => {
|
||||
this.checkOperator();
|
||||
});
|
||||
});*/
|
||||
|
||||
this.bootup();
|
||||
}
|
||||
@ -68,20 +57,24 @@ export class OpenSlidesService {
|
||||
*/
|
||||
public async bootup(): Promise<void> {
|
||||
// start autoupdate if the user is logged in:
|
||||
let response = await this.operator.whoAmIFromStorage();
|
||||
const needToCheckOperator = !!response;
|
||||
let whoami = await this.operator.whoAmIFromStorage();
|
||||
const needToCheckOperator = !!whoami;
|
||||
|
||||
if (!response) {
|
||||
response = await this.operator.whoAmI();
|
||||
if (!whoami) {
|
||||
const response = await this.operator.whoAmI();
|
||||
if (!response.online) {
|
||||
this.offlineBroadcastService.goOffline(OfflineReason.WhoAmIFailed);
|
||||
}
|
||||
whoami = response.whoami;
|
||||
}
|
||||
|
||||
if (!response.user && !response.guest_enabled) {
|
||||
if (!whoami.user && !whoami.guest_enabled) {
|
||||
if (!location.pathname.includes('error')) {
|
||||
this.redirectUrl = location.pathname;
|
||||
}
|
||||
this.redirectToLoginIfNotSubpage();
|
||||
} else {
|
||||
await this.afterLoginBootup(response.user_id);
|
||||
await this.afterLoginBootup(whoami.user_id);
|
||||
}
|
||||
|
||||
if (needToCheckOperator) {
|
||||
@ -121,7 +114,7 @@ export class OpenSlidesService {
|
||||
await this.DS.clear();
|
||||
await this.storageService.set('lastUserLoggedIn', userId);
|
||||
}
|
||||
await this.setupDataStoreAndWebSocket();
|
||||
await this.setupDataStoreAndStartCommunication();
|
||||
// Now finally booted.
|
||||
this.booted.next(true);
|
||||
}
|
||||
@ -129,23 +122,16 @@ export class OpenSlidesService {
|
||||
/**
|
||||
* Init DS from cache and after this start the websocket service.
|
||||
*/
|
||||
private async setupDataStoreAndWebSocket(): Promise<void> {
|
||||
const changeId = await this.DS.initFromStorage();
|
||||
// disconnect the WS connection, if there was one. This is needed
|
||||
// to update the connection parameters, namely the cookies. If the user
|
||||
// is changed, the WS needs to reconnect, so the new connection holds the new
|
||||
// user information.
|
||||
if (this.websocketService.isConnected) {
|
||||
await this.websocketService.close(); // Wait for the disconnect.
|
||||
}
|
||||
await this.websocketService.connect(changeId); // Request changes after changeId.
|
||||
private async setupDataStoreAndStartCommunication(): Promise<void> {
|
||||
await this.DS.initFromStorage();
|
||||
this.communicationManager.startCommunication();
|
||||
}
|
||||
|
||||
/**
|
||||
* Shuts down OpenSlides. The websocket connection is closed and the operator is not set.
|
||||
* Shuts down OpenSlides.
|
||||
*/
|
||||
public async shutdown(): Promise<void> {
|
||||
await this.websocketService.close();
|
||||
this.communicationManager.closeConnections();
|
||||
this.booted.next(false);
|
||||
}
|
||||
|
||||
@ -167,29 +153,37 @@ export class OpenSlidesService {
|
||||
await this.bootup();
|
||||
}
|
||||
|
||||
public async checkOperator(requestChanges: boolean = true): Promise<void> {
|
||||
const response = await this.operator.whoAmI();
|
||||
if (!response.online) {
|
||||
this.offlineBroadcastService.goOffline(OfflineReason.WhoAmIFailed);
|
||||
}
|
||||
await this.checkWhoAmI(response.whoami, requestChanges);
|
||||
}
|
||||
|
||||
/**
|
||||
* Verify that the operator is the same as it was before. Should be alled on a reconnect.
|
||||
*
|
||||
* @returns true, if the user is still logged in
|
||||
*/
|
||||
private async checkOperator(requestChanges: boolean = true): Promise<void> {
|
||||
const response = await this.operator.whoAmI();
|
||||
public async checkWhoAmI(whoami: WhoAmI, requestChanges: boolean = true): Promise<boolean> {
|
||||
let isLoggedIn = false;
|
||||
// User logged off.
|
||||
if (!response.user && !response.guest_enabled) {
|
||||
this.websocketService.cancelReconnectenRetry();
|
||||
if (!whoami.user && !whoami.guest_enabled) {
|
||||
await this.shutdown();
|
||||
this.redirectToLoginIfNotSubpage();
|
||||
} else {
|
||||
isLoggedIn = true;
|
||||
if (
|
||||
(this.operator.user && this.operator.user.id !== response.user_id) ||
|
||||
(!this.operator.user && response.user_id)
|
||||
(this.operator.user && this.operator.user.id !== whoami.user_id) ||
|
||||
(!this.operator.user && whoami.user_id)
|
||||
) {
|
||||
// user changed
|
||||
await this.DS.clear();
|
||||
await this.reboot();
|
||||
} else if (requestChanges) {
|
||||
// User is still the same, but check for missed autoupdates.
|
||||
this.autoupdateService.requestChanges();
|
||||
this.constantsService.refresh();
|
||||
}
|
||||
}
|
||||
|
||||
return isLoggedIn;
|
||||
}
|
||||
}
|
||||
|
@ -10,7 +10,6 @@ import { CollectionStringMapperService } from './collection-string-mapper.servic
|
||||
import { DataStoreService } from './data-store.service';
|
||||
import { Deferred } from '../promises/deferred';
|
||||
import { HttpService } from './http.service';
|
||||
import { OfflineService } from './offline.service';
|
||||
import { OnAfterAppsLoaded } from '../definitions/on-after-apps-loaded';
|
||||
import { OpenSlidesStatusService } from './openslides-status.service';
|
||||
import { StorageService } from './storage.service';
|
||||
@ -207,7 +206,6 @@ export class OperatorService implements OnAfterAppsLoaded {
|
||||
public constructor(
|
||||
private http: HttpService,
|
||||
private DS: DataStoreService,
|
||||
private offlineService: OfflineService,
|
||||
private collectionStringMapper: CollectionStringMapperService,
|
||||
private storageService: StorageService,
|
||||
private OSStatus: OpenSlidesStatusService
|
||||
@ -306,18 +304,19 @@ export class OperatorService implements OnAfterAppsLoaded {
|
||||
*
|
||||
* @returns The response of the WhoAmI request.
|
||||
*/
|
||||
public async whoAmI(): Promise<WhoAmI> {
|
||||
public async whoAmI(): Promise<{ whoami: WhoAmI; online: boolean }> {
|
||||
let online = true;
|
||||
try {
|
||||
const response = await this.http.get(environment.urlPrefix + '/users/whoami/');
|
||||
if (isWhoAmI(response)) {
|
||||
await this.updateCurrentWhoAmI(response);
|
||||
} else {
|
||||
this.offlineService.goOfflineBecauseFailedWhoAmI();
|
||||
online = false;
|
||||
}
|
||||
} catch (e) {
|
||||
this.offlineService.goOfflineBecauseFailedWhoAmI();
|
||||
online = false;
|
||||
}
|
||||
return this.currentWhoAmI;
|
||||
return { whoami: this.currentWhoAmI, online };
|
||||
}
|
||||
|
||||
/**
|
||||
|
@ -1,17 +0,0 @@
|
||||
import { inject, TestBed } from '@angular/core/testing';
|
||||
|
||||
import { E2EImportsModule } from '../../../e2e-imports.module';
|
||||
import { PingService } from './ping.service';
|
||||
|
||||
describe('PingService', () => {
|
||||
beforeEach(() => {
|
||||
TestBed.configureTestingModule({
|
||||
imports: [E2EImportsModule],
|
||||
providers: [PingService]
|
||||
});
|
||||
});
|
||||
|
||||
it('should be created', inject([PingService], (service: PingService) => {
|
||||
expect(service).toBeTruthy();
|
||||
}));
|
||||
});
|
@ -1,86 +0,0 @@
|
||||
import { Injectable } from '@angular/core';
|
||||
|
||||
import { ConstantsService } from './constants.service';
|
||||
import { Deferred } from '../promises/deferred';
|
||||
import { TimeoutPromise } from '../promises/timeout-promise';
|
||||
import { WebsocketService } from './websocket.service';
|
||||
|
||||
interface OpenSlidesSettings {
|
||||
PING_INTERVAL?: number;
|
||||
PING_TIMEOUT?: number;
|
||||
}
|
||||
|
||||
@Injectable({
|
||||
providedIn: 'root'
|
||||
})
|
||||
export class PingService {
|
||||
/**
|
||||
* The interval.
|
||||
*/
|
||||
private pingInterval: any;
|
||||
|
||||
private intervalTime = 30000;
|
||||
|
||||
private timeoutTime = 5000;
|
||||
|
||||
private lastLatency: number | null = null;
|
||||
|
||||
public constructor(private websocketService: WebsocketService, private constantsService: ConstantsService) {
|
||||
this.setup();
|
||||
}
|
||||
|
||||
private async setup(): Promise<void> {
|
||||
const gotConstants = new Deferred();
|
||||
|
||||
this.constantsService.get<OpenSlidesSettings>('Settings').subscribe(settings => {
|
||||
this.intervalTime = settings.PING_INTERVAL || 30000;
|
||||
this.timeoutTime = settings.PING_TIMEOUT || 5000;
|
||||
gotConstants.resolve();
|
||||
});
|
||||
await gotConstants;
|
||||
|
||||
// Connects the ping-pong mechanism to the opening and closing of the connection.
|
||||
this.websocketService.closeEvent.subscribe(() => this.stopPing());
|
||||
this.websocketService.generalConnectEvent.subscribe(() => this.startPing());
|
||||
if (this.websocketService.isConnected) {
|
||||
this.startPing();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Starts the ping-mechanism
|
||||
*/
|
||||
private startPing(): void {
|
||||
if (this.pingInterval) {
|
||||
return;
|
||||
}
|
||||
|
||||
this.pingInterval = setInterval(async () => {
|
||||
const start = performance.now();
|
||||
try {
|
||||
await TimeoutPromise(
|
||||
this.websocketService.sendAndGetResponse('ping', this.lastLatency),
|
||||
this.timeoutTime
|
||||
);
|
||||
this.lastLatency = performance.now() - start;
|
||||
if (this.lastLatency > 1000) {
|
||||
console.warn(`Ping took ${this.lastLatency / 1000} seconds.`);
|
||||
}
|
||||
} catch (e) {
|
||||
console.warn(`The server didn't respond to ping within ${this.timeoutTime / 1000} seconds.`);
|
||||
this.stopPing();
|
||||
this.websocketService.simulateAbnormalClose();
|
||||
}
|
||||
}, this.intervalTime);
|
||||
}
|
||||
|
||||
/**
|
||||
* Clears the ping interval
|
||||
*/
|
||||
private stopPing(): void {
|
||||
if (this.pingInterval) {
|
||||
clearInterval(this.pingInterval);
|
||||
this.pingInterval = null;
|
||||
}
|
||||
}
|
||||
}
|
@ -1,17 +0,0 @@
|
||||
import { inject, TestBed } from '@angular/core/testing';
|
||||
|
||||
import { E2EImportsModule } from '../../../e2e-imports.module';
|
||||
import { PrioritizeService } from './prioritize.service';
|
||||
|
||||
describe('PrioritizeService', () => {
|
||||
beforeEach(() => {
|
||||
TestBed.configureTestingModule({
|
||||
imports: [E2EImportsModule],
|
||||
providers: [PrioritizeService]
|
||||
});
|
||||
});
|
||||
|
||||
it('should be created', inject([PrioritizeService], (service: PrioritizeService) => {
|
||||
expect(service).toBeTruthy();
|
||||
}));
|
||||
});
|
@ -1,46 +0,0 @@
|
||||
import { Injectable } from '@angular/core';
|
||||
|
||||
import { ConstantsService } from './constants.service';
|
||||
import { DataStoreService } from './data-store.service';
|
||||
import { OpenSlidesStatusService } from './openslides-status.service';
|
||||
import { OperatorService } from './operator.service';
|
||||
import { WebsocketService } from './websocket.service';
|
||||
|
||||
interface OpenSlidesSettings {
|
||||
PRIORITIZED_GROUP_IDS?: number[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Cares about prioritizing a client. Checks, if the operator is in one of
|
||||
* some prioritized groups. These group ids come from the server. If the prio-
|
||||
* ritization changes, the websocket connection gets reconnected.
|
||||
*/
|
||||
@Injectable({
|
||||
providedIn: 'root'
|
||||
})
|
||||
export class PrioritizeService {
|
||||
private prioritizedGroupIds: number[] = [];
|
||||
|
||||
public constructor(
|
||||
constantsService: ConstantsService,
|
||||
private websocketService: WebsocketService,
|
||||
private DS: DataStoreService,
|
||||
private openSlidesStatusService: OpenSlidesStatusService,
|
||||
private operator: OperatorService
|
||||
) {
|
||||
constantsService.get<OpenSlidesSettings>('Settings').subscribe(settings => {
|
||||
this.prioritizedGroupIds = settings.PRIORITIZED_GROUP_IDS || [];
|
||||
this.checkPrioritization();
|
||||
});
|
||||
operator.getUserObservable().subscribe(() => this.checkPrioritization());
|
||||
}
|
||||
|
||||
private checkPrioritization(): void {
|
||||
const opPrioritized = this.operator.isInGroupIdsNonAdminCheck(...this.prioritizedGroupIds);
|
||||
if (this.openSlidesStatusService.isPrioritizedClient !== opPrioritized) {
|
||||
console.log('Alter prioritization:', opPrioritized);
|
||||
this.openSlidesStatusService.isPrioritizedClient = opPrioritized;
|
||||
this.websocketService.reconnect(this.DS.maxChangeId);
|
||||
}
|
||||
}
|
||||
}
|
@ -3,8 +3,8 @@ import { Injectable } from '@angular/core';
|
||||
import { BehaviorSubject, Observable, Subject } from 'rxjs';
|
||||
import { auditTime } from 'rxjs/operators';
|
||||
|
||||
import { WebsocketService } from 'app/core/core-services/websocket.service';
|
||||
import { Projector, ProjectorElement } from 'app/shared/models/core/projector';
|
||||
import { CommunicationManagerService, OfflineError } from './communication-manager.service';
|
||||
|
||||
export interface SlideData<T = { error?: string }, P extends ProjectorElement = ProjectorElement> {
|
||||
data: T;
|
||||
@ -15,13 +15,13 @@ export interface SlideData<T = { error?: string }, P extends ProjectorElement =
|
||||
export type ProjectorData = SlideData[];
|
||||
|
||||
interface AllProjectorData {
|
||||
[id: number]: ProjectorData | { error: string };
|
||||
[id: number]: ProjectorData;
|
||||
}
|
||||
|
||||
/**
|
||||
* Received data from server.
|
||||
*/
|
||||
interface ProjectorWebsocketMessage {
|
||||
interface ProjectorDataMessage {
|
||||
/**
|
||||
* The `change_id` of the current update.
|
||||
*/
|
||||
@ -63,38 +63,63 @@ export class ProjectorDataService {
|
||||
*/
|
||||
private currentChangeId = 0;
|
||||
|
||||
/**
|
||||
* Constructor.
|
||||
*
|
||||
* @param websocketService
|
||||
*/
|
||||
public constructor(private websocketService: WebsocketService) {
|
||||
// Dispatch projector data.
|
||||
this.websocketService.getOberservable('projector').subscribe((update: ProjectorWebsocketMessage) => {
|
||||
if (this.currentChangeId > update.change_id) {
|
||||
return;
|
||||
}
|
||||
Object.keys(update.data).forEach(_id => {
|
||||
const id = parseInt(_id, 10);
|
||||
if (this.currentProjectorData[id]) {
|
||||
this.currentProjectorData[id].next(update.data[id] as ProjectorData);
|
||||
}
|
||||
});
|
||||
this.currentChangeId = update.change_id;
|
||||
});
|
||||
private streamCloseFn: () => void | null = null;
|
||||
|
||||
// The service need to re-register, if the websocket connection was lost.
|
||||
this.websocketService.generalConnectEvent.subscribe(() => this.updateProjectorDataSubscription());
|
||||
public constructor(private communicationManager: CommunicationManagerService) {
|
||||
this.communicationManager.startCommunicationEvent.subscribe(() => this.updateProjectorDataSubscription());
|
||||
|
||||
// With a bit of debounce, update the needed projectors.
|
||||
this.updateProjectorDataDebounceSubject.pipe(auditTime(10)).subscribe(() => {
|
||||
const allActiveProjectorIds = Object.keys(this.openProjectorInstances)
|
||||
.map(id => parseInt(id, 10))
|
||||
.filter(id => this.openProjectorInstances[id] > 0);
|
||||
this.websocketService.send('listenToProjectors', { projector_ids: allActiveProjectorIds });
|
||||
this.requestProjectors(allActiveProjectorIds);
|
||||
});
|
||||
}
|
||||
|
||||
public async requestProjectors(allActiveProjectorIds: number[]): Promise<void> {
|
||||
this.cancelCurrentServerSubscription();
|
||||
|
||||
if (allActiveProjectorIds.length === 0) {
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
this.streamCloseFn = await this.communicationManager.subscribe<ProjectorDataMessage>(
|
||||
'/system/projector',
|
||||
message => {
|
||||
this.handleMesage(message);
|
||||
},
|
||||
() => ({ projector_ids: allActiveProjectorIds.join(',') })
|
||||
);
|
||||
} catch (e) {
|
||||
if (!(e instanceof OfflineError)) {
|
||||
console.log(e);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public cancelCurrentServerSubscription(): void {
|
||||
if (this.streamCloseFn) {
|
||||
this.streamCloseFn();
|
||||
this.streamCloseFn = null;
|
||||
}
|
||||
}
|
||||
|
||||
private handleMesage(message: ProjectorDataMessage): void {
|
||||
if (this.currentChangeId > message.change_id) {
|
||||
console.log('Projector: Change id too low:', this.currentChangeId, message.change_id);
|
||||
return;
|
||||
}
|
||||
Object.keys(message.data).forEach(_id => {
|
||||
const id = parseInt(_id, 10);
|
||||
if (this.currentProjectorData[id]) {
|
||||
this.currentProjectorData[id].next(message.data[id] as ProjectorData);
|
||||
}
|
||||
});
|
||||
this.currentChangeId = message.change_id;
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets an observable for the projector data.
|
||||
*
|
||||
|
@ -40,12 +40,6 @@ export interface ProjectorTitle {
|
||||
providedIn: 'root'
|
||||
})
|
||||
export class ProjectorService {
|
||||
/**
|
||||
* Constructor.
|
||||
*
|
||||
* @param DS
|
||||
* @param dataSend
|
||||
*/
|
||||
public constructor(
|
||||
private DS: DataStoreService,
|
||||
private http: HttpService,
|
||||
|
@ -0,0 +1,281 @@
|
||||
import {
|
||||
HttpClient,
|
||||
HttpDownloadProgressEvent,
|
||||
HttpEvent,
|
||||
HttpHeaderResponse,
|
||||
HttpHeaders,
|
||||
HttpParams
|
||||
} from '@angular/common/http';
|
||||
import { Injectable } from '@angular/core';
|
||||
|
||||
import { Observable, Subscription } from 'rxjs';
|
||||
|
||||
import { HTTPMethod } from '../definitions/http-methods';
|
||||
|
||||
const HEADER_EVENT_TYPE = 2;
|
||||
const PROGRESS_EVENT_TYPE = 3;
|
||||
const FINISH_EVENT_TYPE = 4;
|
||||
|
||||
export type Params = HttpParams | { [param: string]: string | string[] };
|
||||
export enum ErrorType {
|
||||
Client,
|
||||
Server, // or network errors, they are the same.
|
||||
Unknown
|
||||
}
|
||||
export function verboseErrorType(type: ErrorType): string {
|
||||
switch (type) {
|
||||
case ErrorType.Client:
|
||||
return 'Client';
|
||||
case ErrorType.Server:
|
||||
return 'Server';
|
||||
case ErrorType.Unknown:
|
||||
return 'Unknown';
|
||||
}
|
||||
}
|
||||
export interface CommunicationError {
|
||||
type: string;
|
||||
msg: string;
|
||||
}
|
||||
export function isCommunicationError(obj: any): obj is CommunicationError {
|
||||
const _obj = obj as CommunicationError;
|
||||
return typeof obj === 'object' && typeof _obj.msg === 'string' && typeof _obj.type === 'string';
|
||||
}
|
||||
interface CommunicationErrorWrapper {
|
||||
error: CommunicationError;
|
||||
}
|
||||
export function isCommunicationErrorWrapper(obj: any): obj is CommunicationErrorWrapper {
|
||||
return typeof obj === 'object' && isCommunicationError(obj.error);
|
||||
}
|
||||
export type ErrorHandler = (type: ErrorType, error: CommunicationError, message: string) => void;
|
||||
|
||||
export class StreamConnectionError extends Error {
|
||||
public constructor(public code: number, message: string) {
|
||||
super(message);
|
||||
this.name = 'StreamConnectionError';
|
||||
}
|
||||
}
|
||||
|
||||
export class StreamContainer {
|
||||
public readonly id = Math.floor(Math.random() * (900000 - 1) + 100000); // [100000, 999999]
|
||||
|
||||
public hasErroredAmount = 0;
|
||||
|
||||
public stream?: Stream<any>;
|
||||
|
||||
public constructor(
|
||||
public url: string,
|
||||
public messageHandler: (message: any) => void,
|
||||
public params: () => Params
|
||||
) {}
|
||||
}
|
||||
|
||||
export class Stream<T> {
|
||||
private subscription: Subscription = null;
|
||||
|
||||
private hasError = false;
|
||||
private reportedError = false;
|
||||
|
||||
private _statuscode: number;
|
||||
public get statuscode(): number {
|
||||
return this._statuscode;
|
||||
}
|
||||
|
||||
private _errorContent: CommunicationError;
|
||||
public get errorContent(): CommunicationError {
|
||||
return this._errorContent;
|
||||
}
|
||||
|
||||
/**
|
||||
* This is the index where we checked, if there is a \n in the read buffer (event.partialText)
|
||||
* This position is always >= contentStartIndex and is > contentStartIndex, if the message
|
||||
* was too big to fit into one buffer. So we have just a partial message.
|
||||
*
|
||||
* The difference between this index and contentStartIndex is that this index remembers the position
|
||||
* we checked for a \n which lay in the middle of the next JOSN-packet.
|
||||
*/
|
||||
private checkedUntilIndex = 0;
|
||||
|
||||
/**
|
||||
* This index holds always the position of the current JOSN-packet, that we are receiving.
|
||||
*/
|
||||
private contentStartIndex = 0;
|
||||
|
||||
private closed = false;
|
||||
|
||||
public constructor(
|
||||
observable: Observable<HttpEvent<string>>,
|
||||
private messageHandler: (message: T) => void,
|
||||
private errorHandler: ErrorHandler
|
||||
) {
|
||||
this.subscription = observable.subscribe(
|
||||
(event: HttpEvent<string>) => {
|
||||
if (this.closed) {
|
||||
return;
|
||||
}
|
||||
if (event.type === HEADER_EVENT_TYPE) {
|
||||
const headerResponse = event as HttpHeaderResponse;
|
||||
this._statuscode = headerResponse.status;
|
||||
if (headerResponse.status >= 400) {
|
||||
this.hasError = true;
|
||||
}
|
||||
} else if ((<HttpEvent<string>>event).type === PROGRESS_EVENT_TYPE) {
|
||||
this.handleMessage(event as HttpDownloadProgressEvent);
|
||||
} else if ((<HttpEvent<string>>event).type === FINISH_EVENT_TYPE) {
|
||||
this.errorHandler(ErrorType.Server, null, 'The stream was closed');
|
||||
}
|
||||
},
|
||||
error => {
|
||||
this.errorHandler(ErrorType.Server, error, 'Network error');
|
||||
},
|
||||
() => {
|
||||
console.log('The stream was completed');
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
private handleMessage(event: HttpDownloadProgressEvent): void {
|
||||
if (this.hasError) {
|
||||
if (!this.reportedError) {
|
||||
this.reportedError = true;
|
||||
// try to get the `error` key from object
|
||||
this._errorContent = this.tryParseError(event.partialText);
|
||||
this.errorHandler(this.getErrorTypeFromStatusCode(), this._errorContent, 'Reported error by server');
|
||||
}
|
||||
return;
|
||||
}
|
||||
// Maybe we get multiple messages, so continue, until the complete buffer is checked.
|
||||
while (this.checkedUntilIndex < event.loaded) {
|
||||
// check if there is a \n somewhere in [checkedUntilIndex, ...]
|
||||
const LF_index = event.partialText.indexOf('\n', this.checkedUntilIndex);
|
||||
|
||||
if (LF_index >= 0) {
|
||||
// take string in [contentStartIndex, LF_index-1]. This must be valid JSON.
|
||||
// In substring, the last character is exlusive.
|
||||
const content = event.partialText.substring(this.contentStartIndex, LF_index);
|
||||
|
||||
// move pointer: next JSON starts at LF_index + 1
|
||||
this.checkedUntilIndex = LF_index + 1;
|
||||
this.contentStartIndex = LF_index + 1;
|
||||
|
||||
const parsedContent = this.tryParseJson(content);
|
||||
if (isCommunicationError(parsedContent)) {
|
||||
if (this.hasError && this.reportedError) {
|
||||
return;
|
||||
}
|
||||
this.hasError = true;
|
||||
this._errorContent = parsedContent;
|
||||
// Do not trigger the error handler, if the connection-retry-routine is still handling this issue
|
||||
if (!this.reportedError) {
|
||||
this.reportedError = true;
|
||||
console.error(this._errorContent);
|
||||
this.errorHandler(
|
||||
this.getErrorTypeFromStatusCode(),
|
||||
this._errorContent,
|
||||
'Reported error by server'
|
||||
);
|
||||
}
|
||||
return;
|
||||
} else {
|
||||
// check, if we didn't get a keep alive
|
||||
if (Object.keys(parsedContent).length > 0) {
|
||||
console.debug('received', parsedContent);
|
||||
this.messageHandler(parsedContent);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
this.checkedUntilIndex = event.loaded;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private getErrorTypeFromStatusCode(): ErrorType {
|
||||
if (!this.statuscode) {
|
||||
return ErrorType.Unknown;
|
||||
}
|
||||
if (this.statuscode >= 400 && this.statuscode < 500) {
|
||||
return ErrorType.Client;
|
||||
}
|
||||
if (this.statuscode >= 500) {
|
||||
return ErrorType.Server;
|
||||
}
|
||||
return ErrorType.Unknown;
|
||||
}
|
||||
|
||||
private tryParseJson(json: string): T | CommunicationError {
|
||||
try {
|
||||
return JSON.parse(json) as T;
|
||||
} catch (e) {
|
||||
return this.tryParseError(json);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* This one is a bit tricky. Error can be:
|
||||
* - string with HTML, e.g. provided by proxies if the service is unreachable
|
||||
* - string with json of form {"error": {"type": ..., "msg": ...}}
|
||||
*/
|
||||
private tryParseError(error: any): CommunicationError {
|
||||
if (typeof error === 'string') {
|
||||
try {
|
||||
error = JSON.parse(error);
|
||||
} catch (e) {
|
||||
return { type: 'Unknown Error', msg: error };
|
||||
}
|
||||
}
|
||||
|
||||
if (isCommunicationErrorWrapper(error)) {
|
||||
return error.error;
|
||||
} else if (isCommunicationError(error)) {
|
||||
return error;
|
||||
}
|
||||
|
||||
// we have something else.... ??
|
||||
console.error('Unknown error', error);
|
||||
throw new Error('Unknown error: ' + error.toString());
|
||||
}
|
||||
|
||||
public close(): void {
|
||||
this.subscription.unsubscribe();
|
||||
this.subscription = null;
|
||||
this.closed = true;
|
||||
}
|
||||
}
|
||||
|
||||
@Injectable({
|
||||
providedIn: 'root'
|
||||
})
|
||||
export class StreamingCommunicationService {
|
||||
public constructor(private http: HttpClient) {}
|
||||
|
||||
public subscribe<T>(streamContainer: StreamContainer, errorHandler: ErrorHandler): void {
|
||||
const options: {
|
||||
body?: any;
|
||||
headers?: HttpHeaders | { [header: string]: string | string[] };
|
||||
observe: 'events';
|
||||
params?: HttpParams | { [param: string]: string | string[] };
|
||||
reportProgress?: boolean;
|
||||
responseType: 'text';
|
||||
withCredentials?: boolean;
|
||||
} = {
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'ngsw-bypass': 'yes'
|
||||
},
|
||||
responseType: 'text',
|
||||
observe: 'events',
|
||||
reportProgress: true
|
||||
};
|
||||
const params = streamContainer.params();
|
||||
if (params) {
|
||||
options.params = params;
|
||||
}
|
||||
const observable = this.http.request(HTTPMethod.GET, streamContainer.url, options);
|
||||
|
||||
if (streamContainer.stream) {
|
||||
console.error('Illegal state!');
|
||||
}
|
||||
|
||||
const stream = new Stream<T>(observable, streamContainer.messageHandler, errorHandler);
|
||||
streamContainer.stream = stream;
|
||||
}
|
||||
}
|
@ -9,7 +9,6 @@ import { DataStoreService, DataStoreUpdateManagerService } from './data-store.se
|
||||
import { HttpService } from './http.service';
|
||||
import { OpenSlidesStatusService } from './openslides-status.service';
|
||||
import { OpenSlidesService } from './openslides.service';
|
||||
import { WebsocketService } from './websocket.service';
|
||||
|
||||
interface HistoryData {
|
||||
[collection: string]: BaseModel[];
|
||||
@ -31,7 +30,6 @@ export class TimeTravelService {
|
||||
* Constructs the time travel service
|
||||
*
|
||||
* @param httpService To fetch the history data
|
||||
* @param webSocketService to disable websocket connection
|
||||
* @param modelMapperService to cast history objects into models
|
||||
* @param DS to overwrite the dataStore
|
||||
* @param OSStatus Sets the history status
|
||||
@ -39,7 +37,6 @@ export class TimeTravelService {
|
||||
*/
|
||||
public constructor(
|
||||
private httpService: HttpService,
|
||||
private webSocketService: WebsocketService,
|
||||
private modelMapperService: CollectionStringMapperService,
|
||||
private DS: DataStoreService,
|
||||
private OSStatus: OpenSlidesStatusService,
|
||||
@ -100,7 +97,8 @@ export class TimeTravelService {
|
||||
* Clears the DataStore and stops the WebSocket connection
|
||||
*/
|
||||
private async stopTime(history: History): Promise<void> {
|
||||
await this.webSocketService.close();
|
||||
// await this.webSocketService.close();
|
||||
// TODO
|
||||
await this.DS.set(); // Same as clear, but not persistent.
|
||||
this.OSStatus.enterHistoryMode(history);
|
||||
}
|
||||
|
@ -1,17 +0,0 @@
|
||||
import { inject, TestBed } from '@angular/core/testing';
|
||||
|
||||
import { E2EImportsModule } from '../../../e2e-imports.module';
|
||||
import { WebsocketService } from './websocket.service';
|
||||
|
||||
describe('WebsocketService', () => {
|
||||
beforeEach(() => {
|
||||
TestBed.configureTestingModule({
|
||||
imports: [E2EImportsModule],
|
||||
providers: [WebsocketService]
|
||||
});
|
||||
});
|
||||
|
||||
it('should be created', inject([WebsocketService], (service: WebsocketService) => {
|
||||
expect(service).toBeTruthy();
|
||||
}));
|
||||
});
|
@ -1,551 +0,0 @@
|
||||
import { EventEmitter, Injectable, NgZone } from '@angular/core';
|
||||
import { MatSnackBarRef, SimpleSnackBar } from '@angular/material/snack-bar';
|
||||
import { Router } from '@angular/router';
|
||||
|
||||
import { compress, decompress } from 'lz4js';
|
||||
import { Observable, Subject } from 'rxjs';
|
||||
import { take } from 'rxjs/operators';
|
||||
|
||||
import { OfflineService } from './offline.service';
|
||||
import { OpenSlidesStatusService } from './openslides-status.service';
|
||||
import { formatQueryParams, QueryParams } from '../definitions/query-params';
|
||||
|
||||
/**
|
||||
* The generic message format in which messages are send and recieved by the server.
|
||||
*/
|
||||
interface BaseWebsocketMessage {
|
||||
type: string;
|
||||
content: any;
|
||||
}
|
||||
|
||||
/**
|
||||
* Outgoing messages must have an id.
|
||||
*/
|
||||
interface OutgoingWebsocketMessage extends BaseWebsocketMessage {
|
||||
id: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Incomming messages may have an `in_response`, if they are an answer to a previously
|
||||
* submitted request.
|
||||
*/
|
||||
interface IncommingWebsocketMessage extends BaseWebsocketMessage {
|
||||
in_response?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* The format of a messages content, if the message type is "error"
|
||||
*/
|
||||
interface WebsocketErrorContent {
|
||||
code: number;
|
||||
message: string;
|
||||
}
|
||||
|
||||
function isWebsocketErrorContent(obj: any): obj is WebsocketErrorContent {
|
||||
return !!obj && obj.code !== undefined && obj.message !== undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* All (custom) error codes that are used to pass error information
|
||||
* from the server to the client
|
||||
*/
|
||||
export const WEBSOCKET_ERROR_CODES = {
|
||||
NOT_AUTHORIZED: 100,
|
||||
CHANGE_ID_TOO_HIGH: 101,
|
||||
WRONG_FORMAT: 102
|
||||
};
|
||||
|
||||
/**
|
||||
* Service that handles WebSocket connections. Other services can register themselfs
|
||||
* with {@method getOberservable} for a specific type of messages. The content will be published.
|
||||
*/
|
||||
@Injectable({
|
||||
providedIn: 'root'
|
||||
})
|
||||
export class WebsocketService {
|
||||
/**
|
||||
* The reference to the snackbar entry that is shown, if the connection is lost.
|
||||
*/
|
||||
private connectionErrorNotice: MatSnackBarRef<SimpleSnackBar>;
|
||||
|
||||
/**
|
||||
* Subjects that will be called, if a reconnect after a retry (e.g. with a previous
|
||||
* connection loss) was successful.
|
||||
*/
|
||||
private readonly _retryReconnectEvent: EventEmitter<void> = new EventEmitter<void>();
|
||||
|
||||
/**
|
||||
* Getter for the retry reconnect event.
|
||||
*/
|
||||
public get retryReconnectEvent(): EventEmitter<void> {
|
||||
return this._retryReconnectEvent;
|
||||
}
|
||||
|
||||
/**
|
||||
* Subjects that will be called, if connect took place, but not a retry reconnect.
|
||||
* THis is the complement from the generalConnectEvent to the retryReconnectEvent.
|
||||
*/
|
||||
private readonly _noRetryConnectEvent: EventEmitter<void> = new EventEmitter<void>();
|
||||
|
||||
/**
|
||||
* Getter for the no-retry connect event.
|
||||
*/
|
||||
public get noRetryConnectEvent(): EventEmitter<void> {
|
||||
return this._noRetryConnectEvent;
|
||||
}
|
||||
|
||||
/**
|
||||
* Listeners will be nofitied, if the wesocket connection is establiched.
|
||||
*/
|
||||
private readonly _generalConnectEvent: EventEmitter<void> = new EventEmitter<void>();
|
||||
|
||||
/**
|
||||
* Getter for the connect event.
|
||||
*/
|
||||
public get generalConnectEvent(): EventEmitter<void> {
|
||||
return this._generalConnectEvent;
|
||||
}
|
||||
|
||||
/**
|
||||
* Listeners will be nofitied, if the wesocket connection is closed.
|
||||
*/
|
||||
private readonly _closeEvent: EventEmitter<void> = new EventEmitter<void>();
|
||||
|
||||
/**
|
||||
* Getter for the close event.
|
||||
*/
|
||||
public get closeEvent(): EventEmitter<void> {
|
||||
return this._closeEvent;
|
||||
}
|
||||
|
||||
/**
|
||||
* The subject for all websocket *message* errors (no connection errors).
|
||||
*/
|
||||
private readonly _errorResponseSubject = new Subject<WebsocketErrorContent>();
|
||||
|
||||
/**
|
||||
* The error response obersable for all websocket message errors.
|
||||
*/
|
||||
public get errorResponseObservable(): Observable<WebsocketErrorContent> {
|
||||
return this._errorResponseSubject.asObservable();
|
||||
}
|
||||
|
||||
/**
|
||||
* Saves, if the connection is open
|
||||
*/
|
||||
private _connectionOpen = false;
|
||||
|
||||
/**
|
||||
* Whether the WebSocket connection is established
|
||||
*/
|
||||
public get isConnected(): boolean {
|
||||
return this._connectionOpen;
|
||||
}
|
||||
|
||||
private sendQueueWhileNotConnected: (string | ArrayBuffer)[] = [];
|
||||
|
||||
/**
|
||||
* The websocket.
|
||||
*/
|
||||
private websocket: WebSocket | null;
|
||||
private websocketId: string | null;
|
||||
|
||||
/**
|
||||
* Subjects for types of websocket messages. A subscriber can get an Observable by {@function getOberservable}.
|
||||
*/
|
||||
private subjects: { [type: string]: Subject<any> } = {};
|
||||
|
||||
/**
|
||||
* Callbacks for a waiting response. If any callback returns true, the message/error will not be propagated with the
|
||||
* responsible subjects for the message type.
|
||||
*/
|
||||
private responseCallbacks: {
|
||||
[id: string]: [(val: any) => boolean, (error: WebsocketErrorContent) => boolean];
|
||||
} = {};
|
||||
|
||||
/**
|
||||
* Saves, if the WS Connection should be closed (e.g. after an explicit `close()`). Prohibits
|
||||
* retry connection attempts.
|
||||
*/
|
||||
private shouldBeClosed = true;
|
||||
|
||||
/**
|
||||
* Counter for delaying the offline message.
|
||||
*/
|
||||
private retryCounter = 0;
|
||||
|
||||
/**
|
||||
* The timeout in the onClose-handler for the next reconnect retry.
|
||||
*/
|
||||
private retryTimeout: any = null;
|
||||
|
||||
/**
|
||||
* Constructor that handles the router
|
||||
*
|
||||
* @param zone
|
||||
* @param router
|
||||
* @param openSlidesStatusService
|
||||
* @param offlineService
|
||||
*/
|
||||
public constructor(
|
||||
private zone: NgZone,
|
||||
private router: Router,
|
||||
private openSlidesStatusService: OpenSlidesStatusService,
|
||||
private offlineService: OfflineService
|
||||
) {}
|
||||
|
||||
/**
|
||||
* Creates a new WebSocket connection and handles incomming events.
|
||||
*
|
||||
* Uses NgZone to let all callbacks run in the angular context.
|
||||
*/
|
||||
public async connect(changeId: number | null = null, retry: boolean = false): Promise<void> {
|
||||
const websocketId = Math.random().toString(36).substring(7);
|
||||
this.websocketId = websocketId;
|
||||
|
||||
if (this.websocket) {
|
||||
this.websocket.close();
|
||||
this.websocket = null;
|
||||
}
|
||||
|
||||
if (!retry) {
|
||||
this.shouldBeClosed = false;
|
||||
}
|
||||
|
||||
const queryParams: QueryParams = {};
|
||||
|
||||
if (changeId !== null) {
|
||||
queryParams.change_id = changeId;
|
||||
}
|
||||
|
||||
// Create the websocket
|
||||
let socketPath = location.protocol === 'https:' ? 'wss://' : 'ws://';
|
||||
socketPath += window.location.host;
|
||||
if (this.openSlidesStatusService.isPrioritizedClient) {
|
||||
socketPath += '/prioritize';
|
||||
}
|
||||
socketPath += '/ws/';
|
||||
socketPath += formatQueryParams(queryParams);
|
||||
|
||||
this.websocket = new WebSocket(socketPath);
|
||||
this.websocket.binaryType = 'arraybuffer';
|
||||
|
||||
// connection established. If this connect attept was a retry,
|
||||
// The error notice will be removed and the reconnectSubject is published.
|
||||
this.websocket.onopen = (event: Event) => {
|
||||
if (this.websocketId !== websocketId) {
|
||||
return;
|
||||
}
|
||||
this.zone.run(() => {
|
||||
this.retryCounter = 0;
|
||||
|
||||
if (this.shouldBeClosed) {
|
||||
this.offlineService.goOnline();
|
||||
return;
|
||||
}
|
||||
|
||||
this._connectionOpen = true;
|
||||
if (retry) {
|
||||
this.offlineService.goOnline();
|
||||
this._retryReconnectEvent.emit();
|
||||
} else {
|
||||
this._noRetryConnectEvent.emit();
|
||||
}
|
||||
this._generalConnectEvent.emit();
|
||||
this.sendQueueWhileNotConnected.forEach(entry => {
|
||||
this.websocket.send(entry);
|
||||
});
|
||||
this.sendQueueWhileNotConnected = [];
|
||||
});
|
||||
};
|
||||
|
||||
this.websocket.onmessage = (event: MessageEvent) => {
|
||||
if (this.websocketId !== websocketId) {
|
||||
return;
|
||||
}
|
||||
this.zone.run(() => {
|
||||
this.handleMessage(event.data);
|
||||
});
|
||||
};
|
||||
|
||||
this.websocket.onclose = (event: CloseEvent) => {
|
||||
if (this.websocketId !== websocketId) {
|
||||
return;
|
||||
}
|
||||
this.zone.run(() => {
|
||||
this.onclose();
|
||||
});
|
||||
};
|
||||
|
||||
this.websocket.onerror = (event: ErrorEvent) => {
|
||||
if (this.websocketId !== websocketId) {
|
||||
return;
|
||||
}
|
||||
// place for proper error handling and debugging.
|
||||
// Required to get more information about errors
|
||||
this.zone.run(() => {
|
||||
console.warn('WS error event:', event);
|
||||
});
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Handles an incomming message.
|
||||
*
|
||||
* @param data The message
|
||||
*/
|
||||
private handleMessage(data: string | ArrayBuffer): void {
|
||||
if (data instanceof ArrayBuffer) {
|
||||
const compressedSize = data.byteLength;
|
||||
const decompressedBuffer: Uint8Array = decompress(new Uint8Array(data));
|
||||
console.debug(
|
||||
`Recieved ${compressedSize / 1024} KB (${
|
||||
decompressedBuffer.byteLength / 1024
|
||||
} KB uncompressed), ratio ${decompressedBuffer.byteLength / compressedSize}`
|
||||
);
|
||||
data = this.arrayBufferToString(decompressedBuffer);
|
||||
}
|
||||
|
||||
const message: IncommingWebsocketMessage = JSON.parse(data);
|
||||
console.debug('Received', message);
|
||||
const type = message.type;
|
||||
const inResponse = message.in_response;
|
||||
const callbacks = this.responseCallbacks[inResponse];
|
||||
if (callbacks) {
|
||||
delete this.responseCallbacks[inResponse];
|
||||
}
|
||||
|
||||
if (type === 'error') {
|
||||
if (!isWebsocketErrorContent(message.content)) {
|
||||
console.error('Websocket error without standard form!', message);
|
||||
return;
|
||||
}
|
||||
|
||||
// Print this to the console.
|
||||
const error = message.content;
|
||||
const errorDescription =
|
||||
Object.keys(WEBSOCKET_ERROR_CODES).find(key => WEBSOCKET_ERROR_CODES[key] === error.code) ||
|
||||
'unknown code';
|
||||
console.error(`Websocket error with code=${error.code} (${errorDescription}):`, error.message);
|
||||
|
||||
// call the error callback, if there is any. If it returns true (means "handled"),
|
||||
// the errorResponseSubject will not be called
|
||||
if (inResponse && callbacks && callbacks[1] && callbacks[1](error)) {
|
||||
return;
|
||||
}
|
||||
this._errorResponseSubject.next(error);
|
||||
return;
|
||||
}
|
||||
|
||||
// Try to fire a response callback directly. If it returnes true, the message is handeled
|
||||
// and not distributed further
|
||||
if (inResponse && callbacks && callbacks[0](message.content)) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (this.subjects[type]) {
|
||||
// Pass the content to the registered subscribers.
|
||||
this.subjects[type].next(message.content);
|
||||
} else {
|
||||
console.warn(
|
||||
`Got unknown websocket message type "${type}" (inResponse: ${inResponse}) with content`,
|
||||
message.content
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Closes the connection error notice
|
||||
*/
|
||||
private onclose(): void {
|
||||
if (this.websocket) {
|
||||
this.websocketId = null; // set to null, so now further events will be
|
||||
// registered with the line below.
|
||||
this.websocket.close(); // Cleanup old connection
|
||||
this.websocket = null;
|
||||
}
|
||||
this._connectionOpen = false;
|
||||
// 1000 is a normal close, like the close on logout
|
||||
this._closeEvent.emit();
|
||||
if (!this.shouldBeClosed) {
|
||||
// Do not show the message snackbar on the projector
|
||||
// tests for /projector and /projector/<id>
|
||||
const onProjector = this.router.url.match(/^\/projector(\/[0-9]+\/?)?$/);
|
||||
if (this.retryCounter <= 3) {
|
||||
this.retryCounter++;
|
||||
}
|
||||
|
||||
if (!this.connectionErrorNotice && !onProjector && this.retryCounter > 3) {
|
||||
this.offlineService.goOfflineBecauseConnectionLost();
|
||||
}
|
||||
|
||||
// A random retry timeout between 2000 and 5000 ms.
|
||||
const timeout = Math.floor(Math.random() * 3000 + 2000);
|
||||
this.retryTimeout = setTimeout(() => {
|
||||
this.retryTimeout = null;
|
||||
this.connect(null, true);
|
||||
}, timeout);
|
||||
}
|
||||
}
|
||||
|
||||
public cancelReconnectenRetry(): void {
|
||||
if (this.retryTimeout) {
|
||||
clearTimeout(this.retryTimeout);
|
||||
this.retryTimeout = null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Closes the websocket connection.
|
||||
*/
|
||||
public async close(): Promise<void> {
|
||||
this.shouldBeClosed = true;
|
||||
this.offlineService.goOnline();
|
||||
if (this.websocket) {
|
||||
this.websocket.close();
|
||||
this.websocket = null;
|
||||
await this.closeEvent.pipe(take(1)).toPromise();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Simulates an abnormal close.
|
||||
*
|
||||
* Internally does not set `shouldBeClosed`, so a reconnect is forced.
|
||||
*/
|
||||
public simulateAbnormalClose(): void {
|
||||
this.onclose();
|
||||
}
|
||||
|
||||
/**
|
||||
* closes and reopens the connection. If the connection was closed before,
|
||||
* it will be just opened.
|
||||
*
|
||||
* @param options The options for the new connection
|
||||
*/
|
||||
public async reconnect(changeId: number | null = null): Promise<void> {
|
||||
await this.close();
|
||||
await this.connect(changeId);
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns an observable for messages of the given type.
|
||||
* @param type the message type
|
||||
*/
|
||||
public getOberservable<T>(type: string): Observable<T> {
|
||||
if (!this.subjects[type]) {
|
||||
this.subjects[type] = new Subject<T>();
|
||||
}
|
||||
return this.subjects[type].asObservable();
|
||||
}
|
||||
|
||||
/**
|
||||
* Sends a message to the server with the content and the given type.
|
||||
*
|
||||
* @param type the message type
|
||||
* @param content the actual content
|
||||
* @param success an optional success callback for a response. If it returns true, the message will not be
|
||||
* propagated through the recieve subjects.
|
||||
* @param error an optional error callback for a response. If it returns true, the error will not be propagated
|
||||
* with the error subject.
|
||||
* @param id an optional id for the message. If not given, a random id will be generated and returned.
|
||||
* @returns the message id
|
||||
*/
|
||||
public send<T, R>(
|
||||
type: string,
|
||||
content: T,
|
||||
success?: (val: R) => boolean,
|
||||
error?: (error: WebsocketErrorContent) => boolean,
|
||||
id?: string
|
||||
): string {
|
||||
if (!this.websocket) {
|
||||
return;
|
||||
}
|
||||
|
||||
const message: OutgoingWebsocketMessage = {
|
||||
type: type,
|
||||
content: content,
|
||||
id: id
|
||||
};
|
||||
|
||||
// create message id if not given. Required by the server.
|
||||
if (!message.id) {
|
||||
message.id = '';
|
||||
const possible = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz';
|
||||
for (let i = 0; i < 8; i++) {
|
||||
message.id += possible.charAt(Math.floor(Math.random() * possible.length));
|
||||
}
|
||||
}
|
||||
|
||||
if (success) {
|
||||
this.responseCallbacks[message.id] = [success, error];
|
||||
}
|
||||
|
||||
const jsonMessage = JSON.stringify(message);
|
||||
const bytesMessage = this.stringToBuffer(jsonMessage);
|
||||
|
||||
const compressedMessage: ArrayBuffer = compress(bytesMessage);
|
||||
const ratio = bytesMessage.byteLength / compressedMessage.byteLength;
|
||||
|
||||
const toSend = ratio > 1 ? compressedMessage : jsonMessage;
|
||||
|
||||
if (this.isConnected) {
|
||||
this.websocket.send(toSend);
|
||||
} else {
|
||||
this.sendQueueWhileNotConnected.push(toSend);
|
||||
}
|
||||
|
||||
return message.id;
|
||||
}
|
||||
|
||||
/**
|
||||
* Sends a message and waits for the response
|
||||
*
|
||||
* @param type the message type
|
||||
* @param content the actual content
|
||||
* @param id an optional id for the message. If not given, a random id will be generated and returned.
|
||||
*/
|
||||
public sendAndGetResponse<T, R>(type: string, content: T, id?: string): Promise<R> {
|
||||
return new Promise<R>((resolve, reject) => {
|
||||
this.send<T, R>(
|
||||
type,
|
||||
content,
|
||||
val => {
|
||||
resolve(val);
|
||||
return true;
|
||||
},
|
||||
val => {
|
||||
reject(val);
|
||||
return true;
|
||||
},
|
||||
id
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Converts an ArrayBuffer to a String.
|
||||
*
|
||||
* @param buffer - Buffer to convert
|
||||
* @returns String
|
||||
*/
|
||||
private arrayBufferToString(buffer: Uint8Array): string {
|
||||
return Array.from(buffer)
|
||||
.map(code => String.fromCharCode(code))
|
||||
.join('');
|
||||
}
|
||||
|
||||
/**
|
||||
* Converts a String to an ArrayBuffer.
|
||||
*
|
||||
* @param str - String to convert.
|
||||
* @returns bufferView.
|
||||
*/
|
||||
private stringToBuffer(str: string): Uint8Array {
|
||||
const bufferView = new Uint8Array();
|
||||
for (let i = 0; i < str.length; i++) {
|
||||
bufferView[i] = str.charCodeAt(i);
|
||||
}
|
||||
return bufferView;
|
||||
}
|
||||
}
|
43
client/src/app/core/definitions/autoupdate-format.ts
Normal file
43
client/src/app/core/definitions/autoupdate-format.ts
Normal file
@ -0,0 +1,43 @@
|
||||
export interface AutoupdateFormat {
|
||||
/**
|
||||
* All changed (and created) items as their full/restricted data grouped by their collection.
|
||||
*/
|
||||
changed: {
|
||||
[collectionString: string]: object[];
|
||||
};
|
||||
|
||||
/**
|
||||
* All deleted items (by id) grouped by their collection.
|
||||
*/
|
||||
deleted: {
|
||||
[collectionString: string]: number[];
|
||||
};
|
||||
|
||||
/**
|
||||
* The lower change id bond for this autoupdate
|
||||
*/
|
||||
from_change_id: number;
|
||||
|
||||
/**
|
||||
* The upper change id bound for this autoupdate
|
||||
*/
|
||||
to_change_id: number;
|
||||
|
||||
/**
|
||||
* Flag, if this autoupdate contains all data. If so, the DS needs to be resetted.
|
||||
*/
|
||||
all_data: boolean;
|
||||
}
|
||||
|
||||
export function isAutoupdateFormat(obj: any): obj is AutoupdateFormat {
|
||||
const format = obj as AutoupdateFormat;
|
||||
return (
|
||||
obj &&
|
||||
typeof obj === 'object' &&
|
||||
format.changed !== undefined &&
|
||||
format.deleted !== undefined &&
|
||||
format.from_change_id !== undefined &&
|
||||
format.to_change_id !== undefined &&
|
||||
format.all_data !== undefined
|
||||
);
|
||||
}
|
10
client/src/app/core/definitions/http-methods.ts
Normal file
10
client/src/app/core/definitions/http-methods.ts
Normal file
@ -0,0 +1,10 @@
|
||||
/**
|
||||
* Enum for different HTTPMethods
|
||||
*/
|
||||
export enum HTTPMethod {
|
||||
GET = 'get',
|
||||
POST = 'post',
|
||||
PUT = 'put',
|
||||
PATCH = 'patch',
|
||||
DELETE = 'delete'
|
||||
}
|
@ -14,16 +14,16 @@
|
||||
* ```
|
||||
*/
|
||||
export class Deferred<T = void> extends Promise<T> {
|
||||
/**
|
||||
* The promise to wait for
|
||||
*/
|
||||
public readonly promise: Promise<T>;
|
||||
|
||||
/**
|
||||
* custom resolve function
|
||||
*/
|
||||
private _resolve: (val?: T) => void;
|
||||
|
||||
private _wasResolved;
|
||||
public get wasResolved(): boolean {
|
||||
return this._wasResolved;
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates the promise and overloads the resolve function
|
||||
*/
|
||||
@ -32,6 +32,7 @@ export class Deferred<T = void> extends Promise<T> {
|
||||
super(resolve => {
|
||||
preResolve = resolve;
|
||||
});
|
||||
this._wasResolved = false;
|
||||
this._resolve = preResolve;
|
||||
}
|
||||
|
||||
@ -40,5 +41,6 @@ export class Deferred<T = void> extends Promise<T> {
|
||||
*/
|
||||
public resolve(val?: T): void {
|
||||
this._resolve(val);
|
||||
this._wasResolved = true;
|
||||
}
|
||||
}
|
||||
|
10
client/src/app/core/promises/sleep.ts
Normal file
10
client/src/app/core/promises/sleep.ts
Normal file
@ -0,0 +1,10 @@
|
||||
/**
|
||||
* Wraps a promise and let it reject after the given timeout (in ms), if it was
|
||||
* not resolved before this timeout.
|
||||
*
|
||||
* @param delay The time to sleep in miliseconds
|
||||
* @returns a new Promise
|
||||
*/
|
||||
export function SleepPromise(delay: number): Promise<void> {
|
||||
return new Promise((resolve, _) => setTimeout(resolve, delay));
|
||||
}
|
6
client/src/app/core/rxjs/trailing-throttle-time.ts
Normal file
6
client/src/app/core/rxjs/trailing-throttle-time.ts
Normal file
@ -0,0 +1,6 @@
|
||||
import { MonoTypeOperatorFunction } from 'rxjs';
|
||||
import { throttleTime } from 'rxjs/operators';
|
||||
|
||||
export function trailingThrottleTime<T = unknown>(time: number): MonoTypeOperatorFunction<T> {
|
||||
return throttleTime<T>(time, undefined, { leading: false, trailing: true });
|
||||
}
|
@ -1,7 +1,10 @@
|
||||
import { Injectable } from '@angular/core';
|
||||
|
||||
import { marker as _ } from '@biesbjerg/ngx-translate-extract-marker';
|
||||
import { BehaviorSubject } from 'rxjs';
|
||||
|
||||
import { OfflineBroadcastService } from '../core-services/offline-broadcast.service';
|
||||
|
||||
export interface BannerDefinition {
|
||||
type?: string;
|
||||
class?: string;
|
||||
@ -20,8 +23,26 @@ export interface BannerDefinition {
|
||||
providedIn: 'root'
|
||||
})
|
||||
export class BannerService {
|
||||
private offlineBannerDefinition: BannerDefinition = {
|
||||
text: _('Offline mode'),
|
||||
icon: 'cloud_off'
|
||||
};
|
||||
|
||||
public activeBanners: BehaviorSubject<BannerDefinition[]> = new BehaviorSubject<BannerDefinition[]>([]);
|
||||
|
||||
public constructor(/*translate: TranslateService, */ offlineBroadcastService: OfflineBroadcastService) {
|
||||
/*translate.onLangChange.subscribe(() => {
|
||||
this.offlineBannerDefinition.text = translate.instant(this.offlineBannerDefinition.text);
|
||||
});*/
|
||||
offlineBroadcastService.isOfflineObservable.subscribe(offline => {
|
||||
if (offline) {
|
||||
this.addBanner(this.offlineBannerDefinition);
|
||||
} else {
|
||||
this.removeBanner(this.offlineBannerDefinition);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Adds a banner to the list of active banners. Skip the banner if it's already in the list
|
||||
* @param toAdd the banner to add
|
||||
|
@ -42,24 +42,24 @@ export class CountUsersService {
|
||||
public constructor(private notifyService: NotifyService, operator: OperatorService) {
|
||||
// Listen for requests to send an answer.
|
||||
this.notifyService.getMessageObservable<CountUserRequest>(REQUEST_NAME).subscribe(request => {
|
||||
if (request.content.token) {
|
||||
if (request.message.token) {
|
||||
this.notifyService.sendToChannels<CountUserResponse>(
|
||||
RESPONSE_NAME,
|
||||
{
|
||||
token: request.content.token,
|
||||
token: request.message.token,
|
||||
data: {
|
||||
userId: this.currentUserId
|
||||
}
|
||||
},
|
||||
request.senderChannelName
|
||||
request.sender_channel_id
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
// Listen for responses and distribute them through `activeCounts`
|
||||
this.notifyService.getMessageObservable<CountUserResponse>(RESPONSE_NAME).subscribe(response => {
|
||||
if (response.content.data && response.content.token && this.activeCounts[response.content.token]) {
|
||||
this.activeCounts[response.content.token].next(response.content.data);
|
||||
if (response.message.data && response.message.token && this.activeCounts[response.message.token]) {
|
||||
this.activeCounts[response.message.token].next(response.message.data);
|
||||
}
|
||||
});
|
||||
|
||||
|
@ -7,7 +7,7 @@ import { distinctUntilChanged } from 'rxjs/operators';
|
||||
import { largeDialogSettings } from 'app/shared/utils/dialog-settings';
|
||||
import { SuperSearchComponent } from 'app/site/common/components/super-search/super-search.component';
|
||||
import { DataStoreUpgradeService } from '../core-services/data-store-upgrade.service';
|
||||
import { OfflineService } from '../core-services/offline.service';
|
||||
import { OfflineBroadcastService } from '../core-services/offline-broadcast.service';
|
||||
import { OpenSlidesService } from '../core-services/openslides.service';
|
||||
import { OperatorService } from '../core-services/operator.service';
|
||||
|
||||
@ -59,7 +59,7 @@ export class OverlayService {
|
||||
private operator: OperatorService,
|
||||
OpenSlides: OpenSlidesService,
|
||||
upgradeService: DataStoreUpgradeService,
|
||||
offlineService: OfflineService
|
||||
offlineBroadcastService: OfflineBroadcastService
|
||||
) {
|
||||
// Subscribe to the current user.
|
||||
operator.getViewUserObservable().subscribe(user => {
|
||||
@ -79,10 +79,7 @@ export class OverlayService {
|
||||
this.checkConnection();
|
||||
});
|
||||
// Subscribe to check if we are offline
|
||||
offlineService
|
||||
.isOffline()
|
||||
.pipe(distinctUntilChanged())
|
||||
.subscribe(offline => {
|
||||
offlineBroadcastService.isOfflineObservable.pipe(distinctUntilChanged()).subscribe(offline => {
|
||||
this.isOffline = offline;
|
||||
this.checkConnection();
|
||||
});
|
||||
|
@ -211,15 +211,15 @@ export class C4DialogComponent implements OnInit, OnDestroy {
|
||||
search: {
|
||||
recievedSearchRequest: {
|
||||
handle: (notify: NotifyResponse<{ name: string }>) => {
|
||||
this.replyChannel = notify.senderChannelName;
|
||||
this.partnerName = notify.content.name;
|
||||
this.replyChannel = notify.sender_channel_id;
|
||||
this.partnerName = notify.message.name;
|
||||
return 'waitForResponse';
|
||||
}
|
||||
},
|
||||
recievedSearchResponse: {
|
||||
handle: (notify: NotifyResponse<{ name: string }>) => {
|
||||
this.replyChannel = notify.senderChannelName;
|
||||
this.partnerName = notify.content.name;
|
||||
this.replyChannel = notify.sender_channel_id;
|
||||
this.partnerName = notify.message.name;
|
||||
// who starts?
|
||||
const startPlayer = Math.random() < 0.5 ? Player.thisPlayer : Player.partner;
|
||||
const startPartner: boolean = startPlayer === Player.partner;
|
||||
@ -232,10 +232,10 @@ export class C4DialogComponent implements OnInit, OnDestroy {
|
||||
waitForResponse: {
|
||||
recievedACK: {
|
||||
handle: (notify: NotifyResponse<{}>) => {
|
||||
if (notify.senderChannelName !== this.replyChannel) {
|
||||
if (notify.sender_channel_id !== this.replyChannel) {
|
||||
return null;
|
||||
}
|
||||
return notify.content ? 'myTurn' : 'foreignTurn';
|
||||
return notify.message ? 'myTurn' : 'foreignTurn';
|
||||
}
|
||||
},
|
||||
waitTimeout: {
|
||||
@ -243,7 +243,7 @@ export class C4DialogComponent implements OnInit, OnDestroy {
|
||||
},
|
||||
recievedRagequit: {
|
||||
handle: (notify: NotifyResponse<{}>) => {
|
||||
return notify.senderChannelName === this.replyChannel ? 'search' : null;
|
||||
return notify.sender_channel_id === this.replyChannel ? 'search' : null;
|
||||
}
|
||||
}
|
||||
},
|
||||
@ -270,10 +270,10 @@ export class C4DialogComponent implements OnInit, OnDestroy {
|
||||
foreignTurn: {
|
||||
recievedTurn: {
|
||||
handle: (notify: NotifyResponse<{ col: number }>) => {
|
||||
if (notify.senderChannelName !== this.replyChannel) {
|
||||
if (notify.sender_channel_id !== this.replyChannel) {
|
||||
return null;
|
||||
}
|
||||
const col: number = notify.content.col;
|
||||
const col: number = notify.message.col;
|
||||
if (!this.colFree(col)) {
|
||||
return null;
|
||||
}
|
||||
@ -455,7 +455,7 @@ export class C4DialogComponent implements OnInit, OnDestroy {
|
||||
*/
|
||||
public enter_waitForResponse(): void {
|
||||
this.caption = 'Wait for response...';
|
||||
this.notifyService.send('c4_search_response', { name: this.getPlayerName() });
|
||||
this.notifyService.sendToChannels('c4_search_response', { name: this.getPlayerName() }, this.replyChannel);
|
||||
if (this.waitTimout) {
|
||||
clearTimeout(<any>this.waitTimout);
|
||||
}
|
||||
|
@ -28,6 +28,7 @@ import { ViewportService } from 'app/core/ui-services/viewport.service';
|
||||
import { BaseProjectableViewModel } from 'app/site/base/base-projectable-view-model';
|
||||
import { BaseViewModel } from 'app/site/base/base-view-model';
|
||||
import { BaseViewModelWithContentObject } from 'app/site/base/base-view-model-with-content-object';
|
||||
import { isProjectable } from 'app/site/base/projectable';
|
||||
|
||||
export interface CssClassDefinition {
|
||||
[key: string]: boolean;
|
||||
@ -458,8 +459,8 @@ export class ListViewTableComponent<V extends BaseViewModel | BaseViewModelWithC
|
||||
}
|
||||
|
||||
public isElementProjected = (context: PblNgridRowContext<V>) => {
|
||||
const model = context.$implicit as V;
|
||||
if (this.allowProjector && this.projectorService.isProjected(this.getProjectable(model))) {
|
||||
const projectableViewModel = this.getProjectable(context.$implicit as V);
|
||||
if (projectableViewModel && this.allowProjector && this.projectorService.isProjected(projectableViewModel)) {
|
||||
return 'projected';
|
||||
}
|
||||
};
|
||||
@ -578,8 +579,10 @@ export class ListViewTableComponent<V extends BaseViewModel | BaseViewModelWithC
|
||||
* @param viewModel The model of the table
|
||||
* @returns a view model that can be projected
|
||||
*/
|
||||
public getProjectable(viewModel: V): BaseProjectableViewModel {
|
||||
return (viewModel as BaseViewModelWithContentObject)?.contentObject ?? viewModel;
|
||||
public getProjectable(viewModel: V): BaseProjectableViewModel | null {
|
||||
const actualViewModel: BaseProjectableViewModel =
|
||||
(viewModel as BaseViewModelWithContentObject)?.contentObject ?? viewModel;
|
||||
return isProjectable(actualViewModel) ? actualViewModel : null;
|
||||
}
|
||||
|
||||
/**
|
||||
|
@ -5,7 +5,7 @@ import { TranslateService } from '@ngx-translate/core';
|
||||
import { Subject, Subscription } from 'rxjs';
|
||||
|
||||
import { BaseComponent } from 'app/base.component';
|
||||
import { OfflineService } from 'app/core/core-services/offline.service';
|
||||
import { OfflineBroadcastService } from 'app/core/core-services/offline-broadcast.service';
|
||||
import { ProjectorDataService, SlideData } from 'app/core/core-services/projector-data.service';
|
||||
import { ProjectorRepositoryService } from 'app/core/repositories/projector/projector-repository.service';
|
||||
import { ConfigService } from 'app/core/ui-services/config.service';
|
||||
@ -166,7 +166,7 @@ export class ProjectorComponent extends BaseComponent implements OnDestroy {
|
||||
private projectorDataService: ProjectorDataService,
|
||||
private projectorRepository: ProjectorRepositoryService,
|
||||
private configService: ConfigService,
|
||||
private offlineService: OfflineService,
|
||||
private offlineBroadcastService: OfflineBroadcastService,
|
||||
private elementRef: ElementRef
|
||||
) {
|
||||
super(titleService, translate);
|
||||
@ -207,7 +207,9 @@ export class ProjectorComponent extends BaseComponent implements OnDestroy {
|
||||
}
|
||||
});
|
||||
|
||||
this.offlineSubscription = this.offlineService.isOffline().subscribe(isOffline => (this.isOffline = isOffline));
|
||||
this.offlineSubscription = this.offlineBroadcastService.isOfflineObservable.subscribe(
|
||||
isOffline => (this.isOffline = isOffline)
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
|
@ -13,6 +13,7 @@
|
||||
</os-head-bar>
|
||||
|
||||
<mat-card class="spacer-bottom-60" [ngClass]="isEditing ? 'os-form-card' : 'os-card'">
|
||||
<mat-card [ngClass]="isEditing ? 'os-form-card' : 'os-card'">
|
||||
<ng-container *ngIf="!isEditing">
|
||||
<div class="app-content">
|
||||
<h1>{{ startContent.general_event_welcome_title | translate }}</h1>
|
||||
|
@ -1639,7 +1639,7 @@ export class MotionDetailComponent extends BaseViewComponentDirective implements
|
||||
*/
|
||||
private listenToEditNotification(): Subscription {
|
||||
return this.notifyService.getMessageObservable(this.NOTIFICATION_EDIT_MOTION).subscribe(message => {
|
||||
const content = <MotionEditNotification>message.content;
|
||||
const content = <MotionEditNotification>message.message;
|
||||
if (this.operator.viewUser.id !== content.senderId && content.motionId === this.motion.id) {
|
||||
let warning = '';
|
||||
|
||||
@ -1656,7 +1656,7 @@ export class MotionDetailComponent extends BaseViewComponentDirective implements
|
||||
if (content.type === MotionEditNotificationType.TYPE_BEGIN_EDITING_MOTION) {
|
||||
this.sendEditNotification(
|
||||
MotionEditNotificationType.TYPE_ALSO_EDITING_MOTION,
|
||||
message.senderUserId
|
||||
message.sender_user_id
|
||||
);
|
||||
}
|
||||
break;
|
||||
|
@ -58,8 +58,6 @@ export abstract class BasePollDialogComponent<
|
||||
votes: this.getVoteData(),
|
||||
publish_immediately: this.publishImmediately
|
||||
};
|
||||
console.log('answer: ', answer);
|
||||
|
||||
this.dialogRef.close(answer);
|
||||
}
|
||||
|
||||
|
@ -10,7 +10,7 @@ import { TranslateService } from '@ngx-translate/core';
|
||||
import { filter } from 'rxjs/operators';
|
||||
|
||||
import { navItemAnim } from '../shared/animations';
|
||||
import { OfflineService } from 'app/core/core-services/offline.service';
|
||||
import { OfflineBroadcastService } from 'app/core/core-services/offline-broadcast.service';
|
||||
import { OverlayService } from 'app/core/ui-services/overlay.service';
|
||||
import { UpdateService } from 'app/core/ui-services/update.service';
|
||||
import { BaseComponent } from '../base.component';
|
||||
@ -84,7 +84,7 @@ export class SiteComponent extends BaseComponent implements OnInit {
|
||||
public constructor(
|
||||
title: Title,
|
||||
protected translate: TranslateService,
|
||||
offlineService: OfflineService,
|
||||
offlineBroadcastService: OfflineBroadcastService,
|
||||
private updateService: UpdateService,
|
||||
private router: Router,
|
||||
public operator: OperatorService,
|
||||
@ -99,7 +99,7 @@ export class SiteComponent extends BaseComponent implements OnInit {
|
||||
super(title, translate);
|
||||
overlayService.showSpinner(translate.instant('Loading data. Please wait ...'));
|
||||
|
||||
offlineService.isOffline().subscribe(offline => {
|
||||
offlineBroadcastService.isOfflineObservable.subscribe(offline => {
|
||||
this.isOffline = offline;
|
||||
});
|
||||
|
||||
|
3
dc-dev.sh
Executable file
3
dc-dev.sh
Executable file
@ -0,0 +1,3 @@
|
||||
#!/bin/bash
|
||||
cd "$(dirname $0)"
|
||||
docker-compose -f docker/docker-compose.dev.yml $@
|
@ -17,10 +17,14 @@ DEFAULT_DOCKER_REGISTRY=
|
||||
|
||||
# Docker Images
|
||||
# -------------
|
||||
DOCKER_OPENSLIDES_HAPROXY_NAME=
|
||||
DOCKER_OPENSLIDES_HAPROXY_TAG=
|
||||
DOCKER_OPENSLIDES_BACKEND_NAME=
|
||||
DOCKER_OPENSLIDES_BACKEND_TAG=
|
||||
DOCKER_OPENSLIDES_FRONTEND_NAME=
|
||||
DOCKER_OPENSLIDES_FRONTEND_TAG=
|
||||
DOCKER_OPENSLIDES_AUTOUPDATE_NAME=
|
||||
DOCKER_OPENSLIDES_AUTOUPDATE_TAG=
|
||||
|
||||
# Database
|
||||
# --------
|
||||
@ -37,6 +41,7 @@ PGBOUNCER_PLACEMENT_CONSTR=
|
||||
# -------------------
|
||||
OPENSLIDES_BACKEND_SERVICE_REPLICAS=
|
||||
OPENSLIDES_FRONTEND_SERVICE_REPLICAS=
|
||||
OPENSLIDES_AUTOUPDATE_SERVICE_REPLICAS=
|
||||
REDIS_RO_SERVICE_REPLICAS=
|
||||
MEDIA_SERVICE_REPLICAS=
|
||||
|
||||
|
@ -6,7 +6,9 @@ declare -A TARGETS
|
||||
TARGETS=(
|
||||
[client]="$(dirname "${BASH_SOURCE[0]}")/../client/docker/"
|
||||
[server]="$(dirname "${BASH_SOURCE[0]}")/../server/docker/"
|
||||
[media-service]="https://github.com/OpenSlides/openslides-media-service.git"
|
||||
[haproxy]="$(dirname "${BASH_SOURCE[0]}")/../haproxy/"
|
||||
[autoupdate]="$(dirname "${BASH_SOURCE[0]}")/../autoupdate/"
|
||||
[media]="https://github.com/OpenSlides/openslides-media-service.git"
|
||||
[pgbouncer]="https://github.com/OpenSlides/openslides-docker-compose.git#:pgbouncer"
|
||||
[postfix]="https://github.com/OpenSlides/openslides-docker-compose.git#:postfix"
|
||||
[repmgr]="https://github.com/OpenSlides/openslides-docker-compose.git#:repmgr"
|
||||
@ -17,7 +19,7 @@ DOCKER_TAG="latest"
|
||||
CONFIG="/etc/osinstancectl"
|
||||
OPTIONS=()
|
||||
BUILT_IMAGES=()
|
||||
DEFAULT_TARGETS=(server client)
|
||||
DEFAULT_TARGETS=(server client haproxy autoupdate)
|
||||
|
||||
usage() {
|
||||
cat << EOF
|
||||
|
47
docker/docker-compose.dev.yml
Normal file
47
docker/docker-compose.dev.yml
Normal file
@ -0,0 +1,47 @@
|
||||
version: "3"
|
||||
services:
|
||||
client:
|
||||
image: os3-client-dev
|
||||
build:
|
||||
context: ../client
|
||||
dockerfile: docker/Dockerfile.dev
|
||||
volumes:
|
||||
- ../client:/app
|
||||
|
||||
server:
|
||||
image: os3-server-dev
|
||||
user: $UID:$GID
|
||||
build:
|
||||
context: ../server
|
||||
dockerfile: docker/Dockerfile.dev
|
||||
volumes:
|
||||
- ../server:/app
|
||||
depends_on:
|
||||
- redis
|
||||
|
||||
autoupdate:
|
||||
image: os3-autoupdate-dev
|
||||
environment:
|
||||
- MESSAGE_BUS_HOST=redis
|
||||
- MESSAGE_BUS_PORT=6379
|
||||
- WORKER_HOST=server
|
||||
- WORKER_PORT=8000
|
||||
depends_on:
|
||||
- server
|
||||
- redis
|
||||
volumes:
|
||||
- ../autoupdate/cmd:/root/cmd
|
||||
- ../autoupdate/internal:/root/internal
|
||||
redis:
|
||||
image: redis:latest
|
||||
|
||||
haproxy:
|
||||
image: os3-haproxy-dev
|
||||
volumes:
|
||||
- ../haproxy/src:/usr/local/etc/haproxy
|
||||
depends_on:
|
||||
- client
|
||||
- server
|
||||
- autoupdate
|
||||
ports:
|
||||
- "8000:8000"
|
@ -13,12 +13,22 @@ define(`read_env', `esyscmd(`printf "\`%s'" "$$1"')')
|
||||
dnl return env variable if set; otherwise, return given alternative value
|
||||
define(`ifenvelse', `ifelse(read_env(`$1'),, `$2', read_env(`$1'))')
|
||||
|
||||
define(`HAPROXY_IMAGE',
|
||||
ifenvelse(`DEFAULT_DOCKER_REGISTRY', openslides)/dnl
|
||||
ifenvelse(`DOCKER_OPENSLIDES_HAPROXY_NAME', openslides-haproxy):dnl
|
||||
ifenvelse(`DOCKER_OPENSLIDES_HAPROXY_TAG', latest))
|
||||
define(`BACKEND_IMAGE',
|
||||
ifenvelse(`DOCKER_OPENSLIDES_BACKEND_NAME', openslides/openslides-server):dnl
|
||||
ifenvelse(`DEFAULT_DOCKER_REGISTRY', openslides)/dnl
|
||||
ifenvelse(`DOCKER_OPENSLIDES_BACKEND_NAME', openslides-server):dnl
|
||||
ifenvelse(`DOCKER_OPENSLIDES_BACKEND_TAG', latest))
|
||||
define(`FRONTEND_IMAGE',
|
||||
ifenvelse(`DOCKER_OPENSLIDES_FRONTEND_NAME', openslides/openslides-client):dnl
|
||||
ifenvelse(`DEFAULT_DOCKER_REGISTRY', openslides)/dnl
|
||||
ifenvelse(`DOCKER_OPENSLIDES_FRONTEND_NAME', openslides-client):dnl
|
||||
ifenvelse(`DOCKER_OPENSLIDES_FRONTEND_TAG', latest))
|
||||
define(`AUTOUPDATE_IMAGE',
|
||||
ifenvelse(`DEFAULT_DOCKER_REGISTRY', openslides)/dnl
|
||||
ifenvelse(`DOCKER_OPENSLIDES_AUTOUPDATE_NAME', openslides-autoupdate):dnl
|
||||
ifenvelse(`DOCKER_OPENSLIDES_AUTOUPDATE_TAG', latest))
|
||||
|
||||
define(`PRIMARY_DB', `ifenvelse(`PGNODE_REPMGR_PRIMARY', pgnode1)')
|
||||
|
||||
@ -41,11 +51,9 @@ x-osserver:
|
||||
&default-osserver
|
||||
image: BACKEND_IMAGE
|
||||
networks:
|
||||
- front
|
||||
- back
|
||||
restart: always
|
||||
x-osserver-env: &default-osserver-env
|
||||
AMOUNT_REPLICAS: ifenvelse(`REDIS_RO_SERVICE_REPLICAS', 1)
|
||||
AUTOUPDATE_DELAY: ifenvelse(`AUTOUPDATE_DELAY', 1)
|
||||
DEMO_USERS: "ifenvelse(`DEMO_USERS',)"
|
||||
CONNECTION_POOL_LIMIT: ifenvelse(`CONNECTION_POOL_LIMIT', 100)
|
||||
@ -54,7 +62,6 @@ x-osserver-env: &default-osserver-env
|
||||
DATABASE_PORT: ifenvelse(`DATABASE_PORT', 5432)
|
||||
DATABASE_USER: "ifenvelse(`DATABASE_USER', openslides)"
|
||||
DEFAULT_FROM_EMAIL: "ifenvelse(`DEFAULT_FROM_EMAIL', noreply@example.com)"
|
||||
DJANGO_LOG_LEVEL: "ifenvelse(`DJANGO_LOG_LEVEL', INFO)"
|
||||
EMAIL_HOST: "ifenvelse(`EMAIL_HOST', postfix)"
|
||||
EMAIL_HOST_PASSWORD: "ifenvelse(`EMAIL_HOST_PASSWORD',)"
|
||||
EMAIL_HOST_USER: "ifenvelse(`EMAIL_HOST_USER',)"
|
||||
@ -69,13 +76,11 @@ x-osserver-env: &default-osserver-env
|
||||
JITSI_ROOM_PASSWORD: "ifenvelse(`JITSI_ROOM_PASSWORD',)"
|
||||
JITSI_ROOM_NAME: "ifenvelse(`JITSI_ROOM_NAME',)"
|
||||
OPENSLIDES_LOG_LEVEL: "ifenvelse(`OPENSLIDES_LOG_LEVEL', INFO)"
|
||||
REDIS_CHANNLES_HOST: "ifenvelse(`REDIS_CHANNLES_HOST', redis-channels)"
|
||||
REDIS_CHANNLES_PORT: ifenvelse(`REDIS_CHANNLES_PORT', 6379)
|
||||
DJANGO_LOG_LEVEL: "ifenvelse(`DJANGO_LOG_LEVEL', INFO)"
|
||||
REDIS_HOST: "ifenvelse(`REDIS_HOST', redis)"
|
||||
REDIS_PORT: ifenvelse(`REDIS_PORT', 6379)
|
||||
REDIS_SLAVE_HOST: "ifenvelse(`REDIS_SLAVE_HOST', redis-slave)"
|
||||
REDIS_SLAVE_PORT: ifenvelse(`REDIS_SLAVE_PORT', 6379)
|
||||
REDIS_SLAVE_WAIT_TIMEOUT: ifenvelse(`REDIS_SLAVE_WAIT_TIMEOUT', 10000)
|
||||
RESET_PASSWORD_VERBOSE_ERRORS: "ifenvelse(`RESET_PASSWORD_VERBOSE_ERRORS', False)"
|
||||
x-pgnode: &default-pgnode
|
||||
image: ifenvelse(`DEFAULT_DOCKER_REGISTRY', openslides)/openslides-repmgr:latest
|
||||
@ -90,15 +95,27 @@ x-pgnode-env: &default-pgnode-env
|
||||
REPMGR_WAL_ARCHIVE: "ifenvelse(`PGNODE_WAL_ARCHIVING', on)"
|
||||
|
||||
services:
|
||||
haproxy:
|
||||
image: HAPROXY_IMAGE
|
||||
depends_on:
|
||||
- server
|
||||
- client
|
||||
- autoupdate
|
||||
- media
|
||||
networks:
|
||||
- front
|
||||
- back
|
||||
ports:
|
||||
- "127.0.0.1:ifenvelse(`EXTERNAL_HTTP_PORT', 8000):8000"
|
||||
|
||||
server:
|
||||
<< : *default-osserver
|
||||
# Below is the default command. You can uncomment it to override the
|
||||
# number of workers, for example:
|
||||
# command: "gunicorn -w 8 --preload -b 0.0.0.0:8000
|
||||
# -k uvicorn.workers.UvicornWorker openslides.asgi:application"
|
||||
# command: "gunicorn -w 8 --preload -b 0.0.0.0:8000 openslides.wsgi"
|
||||
#
|
||||
# Uncomment the following line to use daphne instead of gunicorn:
|
||||
# command: "daphne -b 0.0.0.0 -p 8000 openslides.asgi:application"
|
||||
# command: "daphne -b 0.0.0.0 -p 8000 openslides.wsgi"
|
||||
depends_on:
|
||||
- server-setup
|
||||
environment:
|
||||
@ -127,17 +144,24 @@ services:
|
||||
- pgbouncer
|
||||
- redis
|
||||
- redis-slave
|
||||
- redis-channels
|
||||
|
||||
client:
|
||||
image: FRONTEND_IMAGE
|
||||
restart: always
|
||||
depends_on:
|
||||
- server
|
||||
networks:
|
||||
- front
|
||||
ports:
|
||||
- "127.0.0.1:ifenvelse(`EXTERNAL_HTTP_PORT', 8000):80"
|
||||
- back
|
||||
|
||||
autoupdate:
|
||||
image: AUTOUPDATE_IMAGE
|
||||
restart: always
|
||||
depends_on:
|
||||
- redis
|
||||
- server
|
||||
environment:
|
||||
MESSAGE_BUS_HOST: redis
|
||||
WORKER_HOST: server
|
||||
networks:
|
||||
- back
|
||||
|
||||
pgnode1:
|
||||
<< : *default-pgnode
|
||||
@ -189,9 +213,7 @@ ifelse(read_env(`PGNODE_3_ENABLED'), 1, `'
|
||||
image: redis:alpine
|
||||
restart: always
|
||||
networks:
|
||||
back:
|
||||
aliases:
|
||||
- rediscache
|
||||
- back
|
||||
redis-slave:
|
||||
image: redis:alpine
|
||||
restart: always
|
||||
@ -199,18 +221,11 @@ ifelse(read_env(`PGNODE_3_ENABLED'), 1, `'
|
||||
depends_on:
|
||||
- redis
|
||||
networks:
|
||||
back:
|
||||
aliases:
|
||||
- rediscache-slave
|
||||
- back
|
||||
ifelse(read_env(`REDIS_RO_SERVICE_REPLICAS'),,,deploy:
|
||||
replicas: ifenvelse(`REDIS_RO_SERVICE_REPLICAS', 1))
|
||||
redis-channels:
|
||||
image: redis:alpine
|
||||
restart: always
|
||||
networks:
|
||||
back:
|
||||
media:
|
||||
image: ifenvelse(`DEFAULT_DOCKER_REGISTRY', openslides)/openslides-media-service:latest
|
||||
image: ifenvelse(`DEFAULT_DOCKER_REGISTRY', openslides)/openslides-media:latest
|
||||
environment:
|
||||
- CHECK_REQUEST_URL=server:8000/check-media/
|
||||
- CACHE_SIZE=ifenvelse(`CACHE_SIZE', 10)
|
||||
@ -218,8 +233,7 @@ ifelse(read_env(`PGNODE_3_ENABLED'), 1, `'
|
||||
- CACHE_DATA_MAX_SIZE_KB=ifenvelse(`CACHE_DATA_MAX_SIZE_KB', 10240)
|
||||
restart: always
|
||||
networks:
|
||||
front:
|
||||
back:
|
||||
- back
|
||||
# Override command to run more workers per task
|
||||
# command: ["gunicorn", "-w", "4", "--preload", "-b",
|
||||
# "0.0.0.0:8000", "src.mediaserver:app"]
|
||||
|
@ -13,12 +13,22 @@ define(`read_env', `esyscmd(`printf "\`%s'" "$$1"')')
|
||||
dnl return env variable if set; otherwise, return given alternative value
|
||||
define(`ifenvelse', `ifelse(read_env(`$1'),, `$2', read_env(`$1'))')
|
||||
|
||||
define(`HAPROXY_IMAGE',
|
||||
ifenvelse(`DEFAULT_DOCKER_REGISTRY', openslides)/dnl
|
||||
ifenvelse(`DOCKER_OPENSLIDES_HAPROXY_NAME', openslides-haproxy):dnl
|
||||
ifenvelse(`DOCKER_OPENSLIDES_HAPROXY_TAG', latest))
|
||||
define(`BACKEND_IMAGE',
|
||||
ifenvelse(`DOCKER_OPENSLIDES_BACKEND_NAME', openslides/openslides-server):dnl
|
||||
ifenvelse(`DEFAULT_DOCKER_REGISTRY', openslides)/dnl
|
||||
ifenvelse(`DOCKER_OPENSLIDES_BACKEND_NAME', openslides-server):dnl
|
||||
ifenvelse(`DOCKER_OPENSLIDES_BACKEND_TAG', latest))
|
||||
define(`FRONTEND_IMAGE',
|
||||
ifenvelse(`DOCKER_OPENSLIDES_FRONTEND_NAME', openslides/openslides-client):dnl
|
||||
ifenvelse(`DEFAULT_DOCKER_REGISTRY', openslides)/dnl
|
||||
ifenvelse(`DOCKER_OPENSLIDES_FRONTEND_NAME', openslides-client):dnl
|
||||
ifenvelse(`DOCKER_OPENSLIDES_FRONTEND_TAG', latest))
|
||||
define(`AUTOUPDATE_IMAGE',
|
||||
ifenvelse(`DEFAULT_DOCKER_REGISTRY', openslides)/dnl
|
||||
ifenvelse(`DOCKER_OPENSLIDES_AUTOUPDATE_NAME', openslides-autoupdate):dnl
|
||||
ifenvelse(`DOCKER_OPENSLIDES_AUTOUPDATE_TAG', latest))
|
||||
|
||||
define(`PRIMARY_DB', `ifenvelse(`PGNODE_REPMGR_PRIMARY', pgnode1)')
|
||||
|
||||
@ -41,10 +51,8 @@ x-osserver:
|
||||
&default-osserver
|
||||
image: BACKEND_IMAGE
|
||||
networks:
|
||||
- front
|
||||
- back
|
||||
x-osserver-env: &default-osserver-env
|
||||
AMOUNT_REPLICAS: ifenvelse(`REDIS_RO_SERVICE_REPLICAS', 3)
|
||||
AUTOUPDATE_DELAY: ifenvelse(`AUTOUPDATE_DELAY', 1)
|
||||
DEMO_USERS: "ifenvelse(`DEMO_USERS',)"
|
||||
CONNECTION_POOL_LIMIT: ifenvelse(`CONNECTION_POOL_LIMIT', 100)
|
||||
@ -53,7 +61,6 @@ x-osserver-env: &default-osserver-env
|
||||
DATABASE_PORT: ifenvelse(`DATABASE_PORT', 5432)
|
||||
DATABASE_USER: "ifenvelse(`DATABASE_USER', openslides)"
|
||||
DEFAULT_FROM_EMAIL: "ifenvelse(`DEFAULT_FROM_EMAIL', noreply@example.com)"
|
||||
DJANGO_LOG_LEVEL: "ifenvelse(`DJANGO_LOG_LEVEL', INFO)"
|
||||
EMAIL_HOST: "ifenvelse(`EMAIL_HOST', postfix)"
|
||||
EMAIL_HOST_PASSWORD: "ifenvelse(`EMAIL_HOST_PASSWORD',)"
|
||||
EMAIL_HOST_USER: "ifenvelse(`EMAIL_HOST_USER',)"
|
||||
@ -68,13 +75,11 @@ x-osserver-env: &default-osserver-env
|
||||
JITSI_ROOM_PASSWORD: "ifenvelse(`JITSI_ROOM_PASSWORD',)"
|
||||
JITSI_ROOM_NAME: "ifenvelse(`JITSI_ROOM_NAME',)"
|
||||
OPENSLIDES_LOG_LEVEL: "ifenvelse(`OPENSLIDES_LOG_LEVEL', INFO)"
|
||||
REDIS_CHANNLES_HOST: "ifenvelse(`REDIS_CHANNLES_HOST', redis-channels)"
|
||||
REDIS_CHANNLES_PORT: ifenvelse(`REDIS_CHANNLES_PORT', 6379)
|
||||
DJANGO_LOG_LEVEL: "ifenvelse(`DJANGO_LOG_LEVEL', INFO)"
|
||||
REDIS_HOST: "ifenvelse(`REDIS_HOST', redis)"
|
||||
REDIS_PORT: ifenvelse(`REDIS_PORT', 6379)
|
||||
REDIS_SLAVE_HOST: "ifenvelse(`REDIS_SLAVE_HOST', redis-slave)"
|
||||
REDIS_SLAVE_PORT: ifenvelse(`REDIS_SLAVE_PORT', 6379)
|
||||
REDIS_SLAVE_WAIT_TIMEOUT: ifenvelse(`REDIS_SLAVE_WAIT_TIMEOUT', 10000)
|
||||
RESET_PASSWORD_VERBOSE_ERRORS: "ifenvelse(`RESET_PASSWORD_VERBOSE_ERRORS', False)"
|
||||
x-pgnode: &default-pgnode
|
||||
image: ifenvelse(`DEFAULT_DOCKER_REGISTRY', openslides)/openslides-repmgr:latest
|
||||
@ -90,6 +95,18 @@ x-pgnode-env: &default-pgnode-env
|
||||
REPMGR_WAL_ARCHIVE: "ifenvelse(`PGNODE_WAL_ARCHIVING', on)"
|
||||
|
||||
services:
|
||||
haproxy:
|
||||
image: HAPROXY_IMAGE
|
||||
networks:
|
||||
- front
|
||||
- back
|
||||
ports:
|
||||
- "0.0.0.0:ifenvelse(`EXTERNAL_HTTP_PORT', 8000):8000"
|
||||
deploy:
|
||||
restart_policy:
|
||||
condition: on-failure
|
||||
delay: 5s
|
||||
|
||||
server:
|
||||
<< : *default-osserver
|
||||
# Below is the default command. You can uncomment it to override the
|
||||
@ -128,15 +145,26 @@ services:
|
||||
client:
|
||||
image: FRONTEND_IMAGE
|
||||
networks:
|
||||
- front
|
||||
ports:
|
||||
- "0.0.0.0:ifenvelse(`EXTERNAL_HTTP_PORT', 8000):80"
|
||||
- back
|
||||
deploy:
|
||||
replicas: ifenvelse(`OPENSLIDES_FRONTEND_SERVICE_REPLICAS', 1)
|
||||
restart_policy:
|
||||
condition: on-failure
|
||||
delay: 5s
|
||||
|
||||
autoupdate:
|
||||
image: AUTOUPDATE_IMAGE
|
||||
environment:
|
||||
MESSAGE_BUS_HOST: redis
|
||||
WORKER_HOST: server
|
||||
networks:
|
||||
- back
|
||||
deploy:
|
||||
replicas: ifenvelse(`OPENSLIDES_AUTOUPDATE_SERVICE_REPLICAS', 1)
|
||||
restart_policy:
|
||||
condition: on-failure
|
||||
delay: 5s
|
||||
|
||||
pgnode1:
|
||||
<< : *default-pgnode
|
||||
environment:
|
||||
@ -206,9 +234,7 @@ ifelse(read_env(`PGNODE_3_ENABLED'), 1, `'
|
||||
redis:
|
||||
image: redis:alpine
|
||||
networks:
|
||||
back:
|
||||
aliases:
|
||||
- rediscache
|
||||
- back
|
||||
deploy:
|
||||
replicas: 1
|
||||
restart_policy:
|
||||
@ -218,38 +244,26 @@ ifelse(read_env(`PGNODE_3_ENABLED'), 1, `'
|
||||
image: redis:alpine
|
||||
command: ["redis-server", "--save", "", "--slaveof", "redis", "6379"]
|
||||
networks:
|
||||
back:
|
||||
aliases:
|
||||
- rediscache-slave
|
||||
- back
|
||||
deploy:
|
||||
replicas: ifenvelse(`REDIS_RO_SERVICE_REPLICAS', 3)
|
||||
restart_policy:
|
||||
condition: on-failure
|
||||
delay: 5s
|
||||
redis-channels:
|
||||
image: redis:alpine
|
||||
networks:
|
||||
back:
|
||||
deploy:
|
||||
replicas: 1
|
||||
replicas: ifenvelse(`REDIS_RO_SERVICE_REPLICAS', 1)
|
||||
restart_policy:
|
||||
condition: on-failure
|
||||
delay: 5s
|
||||
media:
|
||||
image: ifenvelse(`DEFAULT_DOCKER_REGISTRY', openslides)/openslides-media-service:latest
|
||||
image: ifenvelse(`DEFAULT_DOCKER_REGISTRY', openslides)/openslides-media:latest
|
||||
environment:
|
||||
- CHECK_REQUEST_URL=server:8000/check-media/
|
||||
- CACHE_SIZE=ifenvelse(`CACHE_SIZE', 10)
|
||||
- CACHE_DATA_MIN_SIZE_KB=ifenvelse(`CACHE_DATA_MIN_SIZE_KB', 0)
|
||||
- CACHE_DATA_MAX_SIZE_KB=ifenvelse(`CACHE_DATA_MAX_SIZE_KB', 10240)
|
||||
deploy:
|
||||
replicas: ifenvelse(`MEDIA_SERVICE_REPLICAS', 8)
|
||||
replicas: ifenvelse(`MEDIA_SERVICE_REPLICAS', 2)
|
||||
restart_policy:
|
||||
condition: on-failure
|
||||
delay: 10s
|
||||
networks:
|
||||
front:
|
||||
back:
|
||||
- back
|
||||
# Override command to run more workers per task
|
||||
# command: ["gunicorn", "-w", "4", "--preload", "-b",
|
||||
# "0.0.0.0:8000", "src.mediaserver:app"]
|
||||
|
5
haproxy/Dockerfile
Normal file
5
haproxy/Dockerfile
Normal file
@ -0,0 +1,5 @@
|
||||
FROM haproxy:2.0-alpine
|
||||
COPY src/haproxy.common.cfg /usr/local/etc/haproxy/haproxy.common.cfg
|
||||
COPY src/haproxy.prod.cfg /usr/local/etc/haproxy/haproxy.prod.cfg
|
||||
COPY src/combined.pem /usr/local/etc/haproxy/combined.pem
|
||||
CMD ["haproxy", "-f", "/usr/local/etc/haproxy/haproxy.common.cfg", "-f", "/usr/local/etc/haproxy/haproxy.prod.cfg"]
|
5
haproxy/Dockerfile.dev
Normal file
5
haproxy/Dockerfile.dev
Normal file
@ -0,0 +1,5 @@
|
||||
FROM haproxy:2.0-alpine
|
||||
COPY src/haproxy.common.cfg /usr/local/etc/haproxy/haproxy.common.cfg
|
||||
COPY src/haproxy.dev.cfg /usr/local/etc/haproxy/haproxy.dev.cfg
|
||||
COPY src/combined.pem /usr/local/etc/haproxy/combined.pem
|
||||
CMD ["haproxy", "-f", "/usr/local/etc/haproxy/haproxy.common.cfg", "-f", "/usr/local/etc/haproxy/haproxy.dev.cfg"]
|
3
haproxy/Makefile
Normal file
3
haproxy/Makefile
Normal file
@ -0,0 +1,3 @@
|
||||
build-dev:
|
||||
./prepare-cert.sh
|
||||
docker build -t os3-haproxy-dev -f Dockerfile.dev .
|
6
haproxy/build.sh
Executable file
6
haproxy/build.sh
Executable file
@ -0,0 +1,6 @@
|
||||
#!/bin/bash
|
||||
|
||||
cd "$(dirname "${BASH_SOURCE[0]}")"
|
||||
./prepare-cert.sh
|
||||
docker build --tag "${img:-openslides/openslides-${service_name}:latest}" \
|
||||
--pull "${OPTIONS[@]}" .
|
27
haproxy/prepare-cert.sh
Executable file
27
haproxy/prepare-cert.sh
Executable file
@ -0,0 +1,27 @@
|
||||
#!/bin/bash
|
||||
|
||||
set -e
|
||||
cd "$(dirname "$0")"
|
||||
|
||||
# check, if we already generated a cert
|
||||
combined="src/combined.pem"
|
||||
|
||||
if [[ ! -f $combined ]]; then
|
||||
echo "Creating certificates..."
|
||||
cd src
|
||||
if type 2>&1 >/dev/null openssl ; then
|
||||
echo "Using openssl to generate a certificate."
|
||||
echo "You will need to accept an security exception for the"
|
||||
echo "generated certificate in your browser manually."
|
||||
openssl req -x509 -newkey rsa:4096 -nodes -days 3650 \
|
||||
-subj "/C=DE/O=Selfsigned Test/CN=localhost" \
|
||||
-keyout localhost-key.pem -out localhost.pem
|
||||
else
|
||||
echo >&2 "FATAL: No valid certificate generation tool found!"
|
||||
exit -1
|
||||
fi
|
||||
cat localhost.pem localhost-key.pem > combined.pem
|
||||
echo "done"
|
||||
else
|
||||
echo "Certificate exists."
|
||||
fi
|
31
haproxy/src/haproxy.common.cfg
Normal file
31
haproxy/src/haproxy.common.cfg
Normal file
@ -0,0 +1,31 @@
|
||||
global
|
||||
log stdout format raw local0 debug
|
||||
|
||||
defaults
|
||||
option http-use-htx
|
||||
option dontlognull
|
||||
timeout connect 3s
|
||||
timeout client 10s
|
||||
timeout client-fin 10s
|
||||
timeout server 10s
|
||||
timeout server-fin 10s
|
||||
timeout check 2s
|
||||
timeout tunnel 10s
|
||||
timeout queue 2s
|
||||
log global
|
||||
option httplog
|
||||
|
||||
resolvers docker_resolver
|
||||
nameserver dns 127.0.0.11:53
|
||||
|
||||
backend backend_server
|
||||
mode http
|
||||
# Do not pass the auth-header from /stats to OS. It confuses the server...
|
||||
http-request del-header authorization
|
||||
timeout server 4m
|
||||
server server server:8000 resolvers docker_resolver check alpn http/1.1
|
||||
|
||||
backend backend_autoupdate
|
||||
mode http
|
||||
timeout server 1h
|
||||
server autoupdate autoupdate:8002 resolvers docker_resolver check ssl verify none alpn h2
|
20
haproxy/src/haproxy.dev.cfg
Normal file
20
haproxy/src/haproxy.dev.cfg
Normal file
@ -0,0 +1,20 @@
|
||||
frontend https
|
||||
mode http
|
||||
bind *:8000 ssl crt /usr/local/etc/haproxy/combined.pem alpn h2,http/1.1
|
||||
default_backend backend_client
|
||||
|
||||
acl autoupdate path_beg -i /system
|
||||
use_backend backend_autoupdate if autoupdate
|
||||
|
||||
acl server path_beg -i /apps /media/ /rest /server-version.txt
|
||||
use_backend backend_server if server
|
||||
|
||||
stats enable
|
||||
stats uri /stats
|
||||
stats refresh 10s
|
||||
stats auth admin:admin
|
||||
|
||||
backend backend_client
|
||||
mode http
|
||||
timeout tunnel 1h
|
||||
server client client:4200 resolvers docker_resolver no-check
|
26
haproxy/src/haproxy.prod.cfg
Normal file
26
haproxy/src/haproxy.prod.cfg
Normal file
@ -0,0 +1,26 @@
|
||||
frontend https
|
||||
mode http
|
||||
bind *:8000 ssl crt /usr/local/etc/haproxy/combined.pem alpn h2,http/1.1
|
||||
default_backend backend_client
|
||||
|
||||
acl autoupdate path_beg -i /system
|
||||
use_backend backend_autoupdate if autoupdate
|
||||
|
||||
acl server path_beg -i /apps /rest /server-version.txt
|
||||
use_backend backend_server if server
|
||||
|
||||
acl media path_beg -i /media/
|
||||
use_backend backend_media if media
|
||||
|
||||
stats enable
|
||||
stats uri /stats
|
||||
stats refresh 10s
|
||||
stats auth admin:admin
|
||||
|
||||
backend backend_client
|
||||
mode http
|
||||
server client client:80 resolvers docker_resolver check
|
||||
|
||||
backend backend_media
|
||||
mode http
|
||||
server media media:8000 resolvers docker_resolver check
|
@ -3,3 +3,6 @@
|
||||
**/.venv
|
||||
tests/
|
||||
personal_data/
|
||||
**/.mypy_cache
|
||||
**/.pytest_cache
|
||||
**/tests/file
|
||||
|
@ -70,5 +70,4 @@ COPY manage.py /app/
|
||||
COPY openslides /app/openslides
|
||||
COPY docker/server-version.txt /app/openslides/core/static/server-version.txt
|
||||
ENTRYPOINT ["/usr/local/sbin/entrypoint"]
|
||||
CMD ["gunicorn", "-w", "8", "--preload", "-b", "0.0.0.0:8000", "-k", \
|
||||
"uvicorn.workers.UvicornWorker", "openslides.asgi:application"]
|
||||
CMD ["gunicorn", "-w", "8", "--preload", "-t", "240", "-b", "0.0.0.0:8000", "openslides.wsgi"]
|
||||
|
30
server/docker/Dockerfile.dev
Normal file
30
server/docker/Dockerfile.dev
Normal file
@ -0,0 +1,30 @@
|
||||
FROM python:3.7-slim AS base
|
||||
|
||||
# Variables relevant for CMD
|
||||
ENV DJANGO_SETTINGS_MODULE settings
|
||||
ENV PYTHONPATH personal_data/var/
|
||||
|
||||
WORKDIR /app
|
||||
RUN apt-get -y update && apt-get install --no-install-recommends -y \
|
||||
postgresql-client \
|
||||
wait-for-it \
|
||||
gcc \
|
||||
git \
|
||||
libxml2-dev \
|
||||
libxmlsec1-dev \
|
||||
libxmlsec1-openssl \
|
||||
pkg-config
|
||||
|
||||
RUN rm -rf /var/lib/apt/lists/*
|
||||
|
||||
COPY requirements /app/requirements
|
||||
COPY requirements.txt /app/requirements.txt
|
||||
|
||||
RUN pip install -r requirements.txt -r requirements/saml.txt && \
|
||||
rm -rf /root/.cache/pip
|
||||
|
||||
EXPOSE 8000
|
||||
COPY docker/entrypoint-dev /usr/local/sbin/
|
||||
COPY . .
|
||||
ENTRYPOINT ["/usr/local/sbin/entrypoint-dev"]
|
||||
CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]
|
@ -37,7 +37,6 @@ done
|
||||
# Wait for redis
|
||||
wait-for-it redis:6379
|
||||
wait-for-it redis-slave:6379
|
||||
wait-for-it redis-channels:6379
|
||||
|
||||
echo 'running migrations'
|
||||
python manage.py migrate
|
||||
|
14
server/docker/entrypoint-dev
Executable file
14
server/docker/entrypoint-dev
Executable file
@ -0,0 +1,14 @@
|
||||
#!/bin/bash
|
||||
|
||||
set -e
|
||||
|
||||
wait-for-it -t 0 redis:6379
|
||||
|
||||
if [[ ! -f "/app/personal_data/var/settings.py" ]]; then
|
||||
echo "Create settings"
|
||||
python manage.py createsettings
|
||||
fi
|
||||
|
||||
python -u manage.py migrate
|
||||
|
||||
exec "$@"
|
@ -54,7 +54,7 @@ DEBUG = False
|
||||
RESET_PASSWORD_VERBOSE_ERRORS = get_env("RESET_PASSWORD_VERBOSE_ERRORS", True, bool)
|
||||
|
||||
# OpenSlides specific settings
|
||||
AUTOUPDATE_DELAY = get_env("AUTOUPDATE_DELAY", 1, int)
|
||||
AUTOUPDATE_DELAY = get_env("AUTOUPDATE_DELAY", 1, float)
|
||||
DEMO_USERS = get_env("DEMO_USERS", default=None)
|
||||
DEMO_USERS = json.loads(DEMO_USERS) if DEMO_USERS else None
|
||||
|
||||
@ -99,25 +99,10 @@ REDIS_HOST = get_env("REDIS_HOST", "redis")
|
||||
REDIS_PORT = get_env("REDIS_PORT", 6379, int)
|
||||
REDIS_SLAVE_HOST = get_env("REDIS_SLAVE_HOST", "redis-slave")
|
||||
REDIS_SLAVE_PORT = get_env("REDIS_SLAVE_PORT", 6379, int)
|
||||
REDIS_CHANNLES_HOST = get_env("REDIS_CHANNLES_HOST", "redis-channels")
|
||||
REDIS_CHANNLES_PORT = get_env("REDIS_CHANNLES_PORT", 6379, int)
|
||||
REDIS_SLAVE_WAIT_TIMEOUT = get_env("REDIS_SLAVE_WAIT_TIMEOUT", 10000, int)
|
||||
|
||||
# Django Channels
|
||||
CHANNEL_LAYERS = {
|
||||
"default": {
|
||||
"BACKEND": "channels_redis.core.RedisChannelLayer",
|
||||
"CONFIG": {
|
||||
"hosts": [(REDIS_CHANNLES_HOST, REDIS_CHANNLES_PORT)],
|
||||
"capacity": 10000,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
# Collection Cache
|
||||
REDIS_ADDRESS = f"redis://{REDIS_HOST}:{REDIS_PORT}/0"
|
||||
REDIS_READ_ONLY_ADDRESS = f"redis://{REDIS_SLAVE_HOST}:{REDIS_SLAVE_PORT}/0"
|
||||
AMOUNT_REPLICAS = get_env("AMOUNT_REPLICAS", 1, int)
|
||||
CONNECTION_POOL_LIMIT = get_env("CONNECTION_POOL_LIMIT", 100, int)
|
||||
|
||||
# Session backend
|
||||
@ -135,8 +120,6 @@ ENABLE_SAML = get_env("ENABLE_SAML", False, bool)
|
||||
if ENABLE_SAML:
|
||||
INSTALLED_APPS += ["openslides.saml"]
|
||||
|
||||
# TODO: More saml stuff...
|
||||
|
||||
# Controls if electronic voting (means non-analog polls) are enabled.
|
||||
ENABLE_ELECTRONIC_VOTING = get_env("ENABLE_ELECTRONIC_VOTING", False, bool)
|
||||
|
||||
|
@ -15,7 +15,6 @@ class AgendaAppConfig(AppConfig):
|
||||
from ..utils.access_permissions import required_user
|
||||
from ..utils.rest_api import router
|
||||
from . import serializers # noqa
|
||||
from .projector import register_projector_slides
|
||||
from .signals import (
|
||||
get_permission_change_data,
|
||||
listen_to_related_object_post_delete,
|
||||
@ -23,9 +22,6 @@ class AgendaAppConfig(AppConfig):
|
||||
)
|
||||
from .views import ItemViewSet, ListOfSpeakersViewSet
|
||||
|
||||
# Define projector elements.
|
||||
register_projector_slides()
|
||||
|
||||
# Connect signals.
|
||||
post_save.connect(
|
||||
listen_to_related_object_post_save,
|
||||
|
@ -1,309 +0,0 @@
|
||||
from collections import defaultdict
|
||||
from typing import Any, Dict, List, Union
|
||||
|
||||
from ..users.projector import get_user_name
|
||||
from ..utils.projector import (
|
||||
ProjectorAllDataProvider,
|
||||
ProjectorElementException,
|
||||
get_config,
|
||||
get_model,
|
||||
register_projector_slide,
|
||||
)
|
||||
|
||||
|
||||
# Important: All functions have to be prune. This means, that thay can only
|
||||
# access the data, that they get as argument and do not have any
|
||||
# side effects.
|
||||
|
||||
|
||||
async def get_sorted_agenda_items(
|
||||
agenda_items: Dict[int, Dict[str, Any]]
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Returns all sorted agenda items by id first and then weight, resulting in
|
||||
ordered items, if some have the same weight.
|
||||
"""
|
||||
return sorted(
|
||||
sorted(agenda_items.values(), key=lambda item: item["id"]),
|
||||
key=lambda item: item["weight"],
|
||||
)
|
||||
|
||||
|
||||
async def get_flat_tree(
|
||||
agenda_items: Dict[int, Dict[str, Any]], parent_id: int = 0
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Build the item tree from all_data_provider.
|
||||
|
||||
Only build the tree from elements unterneath parent_id.
|
||||
|
||||
Returns a list of two element tuples where the first element is the item title
|
||||
and the second a List with children as two element tuples.
|
||||
"""
|
||||
|
||||
# Build a dict from an item_id to all its children
|
||||
children: Dict[int, List[int]] = defaultdict(list)
|
||||
|
||||
for item in await get_sorted_agenda_items(agenda_items):
|
||||
if item["type"] == 1: # only normal items
|
||||
children[item["parent_id"] or 0].append(item["id"])
|
||||
|
||||
tree = []
|
||||
|
||||
def build_tree(item_ids: List[int], depth: int) -> None:
|
||||
for item_id in item_ids:
|
||||
item = agenda_items[item_id]
|
||||
title_information = item["title_information"]
|
||||
title_information["_agenda_item_number"] = item["item_number"]
|
||||
tree.append(
|
||||
{
|
||||
"title_information": title_information,
|
||||
"collection": item["content_object"]["collection"],
|
||||
"depth": depth,
|
||||
}
|
||||
)
|
||||
build_tree(children[item_id], depth + 1)
|
||||
|
||||
build_tree(children[parent_id], 0)
|
||||
return tree
|
||||
|
||||
|
||||
async def item_list_slide(
|
||||
all_data_provider: ProjectorAllDataProvider,
|
||||
element: Dict[str, Any],
|
||||
projector_id: int,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Item list slide.
|
||||
|
||||
Returns all root items or all children of an item.
|
||||
"""
|
||||
# fetch all items, so they are cached:
|
||||
all_agenda_items = await all_data_provider.get_collection("agenda/item")
|
||||
|
||||
only_main_items = element.get("only_main_items", True)
|
||||
if only_main_items:
|
||||
agenda_items = []
|
||||
for item in await get_sorted_agenda_items(all_agenda_items):
|
||||
if item["parent_id"] is None and item["type"] == 1:
|
||||
title_information = item["title_information"]
|
||||
title_information["_agenda_item_number"] = item["item_number"]
|
||||
agenda_items.append(
|
||||
{
|
||||
"title_information": title_information,
|
||||
"collection": item["content_object"]["collection"],
|
||||
}
|
||||
)
|
||||
else:
|
||||
agenda_items = await get_flat_tree(all_agenda_items)
|
||||
|
||||
return {"items": agenda_items}
|
||||
|
||||
|
||||
async def list_of_speakers_slide(
|
||||
all_data_provider: ProjectorAllDataProvider,
|
||||
element: Dict[str, Any],
|
||||
projector_id: int,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
List of speakers slide.
|
||||
|
||||
Returns all usernames, that are on the list of speaker of a slide.
|
||||
"""
|
||||
list_of_speakers_id = element.get("id")
|
||||
|
||||
if list_of_speakers_id is None:
|
||||
raise ProjectorElementException("id is required for list of speakers slide")
|
||||
|
||||
return await get_list_of_speakers_slide_data(all_data_provider, list_of_speakers_id)
|
||||
|
||||
|
||||
async def get_list_of_speakers_slide_data(
|
||||
all_data_provider: ProjectorAllDataProvider, list_of_speakers_id: int
|
||||
) -> Dict[str, Any]:
|
||||
list_of_speakers = await get_model(
|
||||
all_data_provider, "agenda/list-of-speakers", list_of_speakers_id
|
||||
)
|
||||
|
||||
title_information = list_of_speakers["title_information"]
|
||||
# try to get the agenda item for the content object (which must not exist)
|
||||
content_object = await get_model(
|
||||
all_data_provider,
|
||||
list_of_speakers["content_object"]["collection"],
|
||||
list_of_speakers["content_object"]["id"],
|
||||
)
|
||||
agenda_item_id = content_object.get("agenda_item_id")
|
||||
if agenda_item_id is not None:
|
||||
agenda_item = await all_data_provider.get("agenda/item", agenda_item_id)
|
||||
if agenda_item is not None:
|
||||
title_information["_agenda_item_number"] = agenda_item["item_number"]
|
||||
|
||||
# Partition speaker objects to waiting, current and finished
|
||||
speakers_waiting = []
|
||||
speakers_finished = []
|
||||
current_speaker = None
|
||||
for speaker in list_of_speakers["speakers"]:
|
||||
user = await get_user_name(all_data_provider, speaker["user_id"])
|
||||
formatted_speaker = {
|
||||
"user": user,
|
||||
"marked": speaker["marked"],
|
||||
"point_of_order": speaker["point_of_order"],
|
||||
"weight": speaker["weight"],
|
||||
"end_time": speaker["end_time"],
|
||||
}
|
||||
|
||||
if speaker["begin_time"] is None and speaker["end_time"] is None:
|
||||
speakers_waiting.append(formatted_speaker)
|
||||
elif speaker["begin_time"] is not None and speaker["end_time"] is None:
|
||||
current_speaker = formatted_speaker
|
||||
else:
|
||||
speakers_finished.append(formatted_speaker)
|
||||
|
||||
# sort speakers
|
||||
speakers_waiting = sorted(speakers_waiting, key=lambda s: s["weight"])
|
||||
speakers_finished = sorted(speakers_finished, key=lambda s: s["end_time"])
|
||||
|
||||
number_of_last_speakers = await get_config(
|
||||
all_data_provider, "agenda_show_last_speakers"
|
||||
)
|
||||
number_of_next_speakers = await get_config(
|
||||
all_data_provider, "agenda_show_next_speakers"
|
||||
)
|
||||
|
||||
if number_of_last_speakers == 0:
|
||||
speakers_finished = []
|
||||
else:
|
||||
# Take the last speakers
|
||||
speakers_finished = speakers_finished[-number_of_last_speakers:]
|
||||
|
||||
if number_of_next_speakers != -1:
|
||||
speakers_waiting = speakers_waiting[:number_of_next_speakers]
|
||||
|
||||
return {
|
||||
"waiting": speakers_waiting,
|
||||
"current": current_speaker,
|
||||
"finished": speakers_finished,
|
||||
"content_object_collection": list_of_speakers["content_object"]["collection"],
|
||||
"title_information": title_information,
|
||||
"closed": list_of_speakers["closed"],
|
||||
}
|
||||
|
||||
|
||||
async def get_current_list_of_speakers_id_for_projector(
|
||||
all_data_provider: ProjectorAllDataProvider, projector: Dict[str, Any]
|
||||
) -> Union[int, None]:
|
||||
"""
|
||||
Search for elements, that do have a list of speakers:
|
||||
Try to get a model by the collection and id in the element. This
|
||||
model needs to have a 'list_of_speakers_id'. This list of speakers
|
||||
must exist. The first matching element is taken.
|
||||
"""
|
||||
elements = projector["elements"]
|
||||
list_of_speakers_id = None
|
||||
for element in elements:
|
||||
if "id" not in element:
|
||||
continue
|
||||
collection = element["name"]
|
||||
id = element["id"]
|
||||
model = await all_data_provider.get(collection, id)
|
||||
if model is None:
|
||||
continue
|
||||
|
||||
if "list_of_speakers_id" not in model:
|
||||
continue
|
||||
|
||||
list_of_speakers_id = model["list_of_speakers_id"]
|
||||
los_exists = await all_data_provider.exists(
|
||||
"agenda/list-of-speakers", list_of_speakers_id
|
||||
)
|
||||
if not los_exists:
|
||||
continue
|
||||
|
||||
break
|
||||
|
||||
return list_of_speakers_id
|
||||
|
||||
|
||||
async def get_reference_projector(
|
||||
all_data_provider: ProjectorAllDataProvider, projector_id: int
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Returns the reference projector to the given projector (by id)
|
||||
"""
|
||||
this_projector = await get_model(all_data_provider, "core/projector", projector_id)
|
||||
|
||||
reference_projector_id = this_projector["reference_projector_id"] or projector_id
|
||||
return await get_model(all_data_provider, "core/projector", reference_projector_id)
|
||||
|
||||
|
||||
async def current_list_of_speakers_slide(
|
||||
all_data_provider: ProjectorAllDataProvider,
|
||||
element: Dict[str, Any],
|
||||
projector_id: int,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
The current list of speakers slide. Creates the data for the given projector.
|
||||
"""
|
||||
reference_projector = await get_reference_projector(all_data_provider, projector_id)
|
||||
list_of_speakers_id = await get_current_list_of_speakers_id_for_projector(
|
||||
all_data_provider, reference_projector
|
||||
)
|
||||
if list_of_speakers_id is None: # no element found
|
||||
return {}
|
||||
|
||||
return await get_list_of_speakers_slide_data(all_data_provider, list_of_speakers_id)
|
||||
|
||||
|
||||
async def current_speaker_chyron_slide(
|
||||
all_data_provider: ProjectorAllDataProvider,
|
||||
element: Dict[str, Any],
|
||||
projector_id: int,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Returns the username for the current speaker.
|
||||
"""
|
||||
# get projector for color information
|
||||
projector = await get_model(all_data_provider, "core/projector", projector_id)
|
||||
|
||||
slide_data = {
|
||||
"background_color": projector["chyron_background_color"],
|
||||
"font_color": projector["chyron_font_color"],
|
||||
}
|
||||
|
||||
reference_projector = await get_reference_projector(all_data_provider, projector_id)
|
||||
list_of_speakers_id = await get_current_list_of_speakers_id_for_projector(
|
||||
all_data_provider, reference_projector
|
||||
)
|
||||
if list_of_speakers_id is None: # no element found
|
||||
return slide_data
|
||||
|
||||
# get list of speakers to search current speaker
|
||||
list_of_speakers = await get_model(
|
||||
all_data_provider, "agenda/list-of-speakers", list_of_speakers_id
|
||||
)
|
||||
|
||||
# find current speaker
|
||||
current_speaker = None
|
||||
for speaker in list_of_speakers["speakers"]:
|
||||
if speaker["begin_time"] is not None and speaker["end_time"] is None:
|
||||
current_speaker = await get_user_name(all_data_provider, speaker["user_id"])
|
||||
break
|
||||
|
||||
if current_speaker is not None:
|
||||
slide_data["current_speaker"] = current_speaker
|
||||
|
||||
return slide_data
|
||||
|
||||
|
||||
def register_projector_slides() -> None:
|
||||
register_projector_slide("agenda/item-list", item_list_slide)
|
||||
register_projector_slide("agenda/list-of-speakers", list_of_speakers_slide)
|
||||
register_projector_slide(
|
||||
"agenda/current-list-of-speakers", current_list_of_speakers_slide
|
||||
)
|
||||
register_projector_slide(
|
||||
"agenda/current-list-of-speakers-overlay", current_list_of_speakers_slide
|
||||
)
|
||||
register_projector_slide(
|
||||
"agenda/current-speaker-chyron", current_speaker_chyron_slide
|
||||
)
|
@ -13,7 +13,6 @@ class AssignmentsAppConfig(AppConfig):
|
||||
from ..utils.access_permissions import required_user
|
||||
from ..utils.rest_api import router
|
||||
from . import serializers # noqa
|
||||
from .projector import register_projector_slides
|
||||
from .signals import get_permission_change_data
|
||||
from .views import (
|
||||
AssignmentOptionViewSet,
|
||||
@ -22,9 +21,6 @@ class AssignmentsAppConfig(AppConfig):
|
||||
AssignmentVoteViewSet,
|
||||
)
|
||||
|
||||
# Define projector elements.
|
||||
register_projector_slides()
|
||||
|
||||
# Connect signals.
|
||||
permission_change.connect(
|
||||
get_permission_change_data,
|
||||
|
@ -1,113 +0,0 @@
|
||||
from typing import Any, Dict, List
|
||||
|
||||
from ..users.projector import get_user_name
|
||||
from ..utils.projector import (
|
||||
ProjectorAllDataProvider,
|
||||
get_model,
|
||||
get_models,
|
||||
register_projector_slide,
|
||||
)
|
||||
from .models import AssignmentPoll
|
||||
|
||||
|
||||
async def assignment_slide(
|
||||
all_data_provider: ProjectorAllDataProvider,
|
||||
element: Dict[str, Any],
|
||||
projector_id: int,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Assignment slide.
|
||||
"""
|
||||
assignment = await get_model(
|
||||
all_data_provider, "assignments/assignment", element.get("id")
|
||||
)
|
||||
|
||||
assignment_related_users: List[Dict[str, Any]] = [
|
||||
{"user": await get_user_name(all_data_provider, aru["user_id"])}
|
||||
for aru in sorted(
|
||||
assignment["assignment_related_users"], key=lambda aru: aru["weight"]
|
||||
)
|
||||
]
|
||||
|
||||
return {
|
||||
"title": assignment["title"],
|
||||
"phase": assignment["phase"],
|
||||
"open_posts": assignment["open_posts"],
|
||||
"description": assignment["description"],
|
||||
"assignment_related_users": assignment_related_users,
|
||||
"number_poll_candidates": assignment["number_poll_candidates"],
|
||||
}
|
||||
|
||||
|
||||
async def assignment_poll_slide(
|
||||
all_data_provider: ProjectorAllDataProvider,
|
||||
element: Dict[str, Any],
|
||||
projector_id: int,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Poll slide.
|
||||
"""
|
||||
poll = await get_model(
|
||||
all_data_provider, "assignments/assignment-poll", element.get("id")
|
||||
)
|
||||
assignment = await get_model(
|
||||
all_data_provider, "assignments/assignment", poll["assignment_id"]
|
||||
)
|
||||
|
||||
poll_data = {
|
||||
key: poll[key]
|
||||
for key in (
|
||||
"title",
|
||||
"type",
|
||||
"pollmethod",
|
||||
"min_votes_amount",
|
||||
"max_votes_amount",
|
||||
"description",
|
||||
"state",
|
||||
"onehundred_percent_base",
|
||||
"majority_method",
|
||||
)
|
||||
}
|
||||
|
||||
# Add options:
|
||||
poll_data["options"] = []
|
||||
options = await get_models(
|
||||
all_data_provider, "assignments/assignment-option", poll["options_id"]
|
||||
)
|
||||
for option in sorted(options, key=lambda option: option["weight"]):
|
||||
option_data: Dict[str, Any] = {
|
||||
"user": {
|
||||
"short_name": await get_user_name(all_data_provider, option["user_id"])
|
||||
}
|
||||
}
|
||||
if poll["state"] == AssignmentPoll.STATE_PUBLISHED:
|
||||
option_data["yes"] = float(option["yes"])
|
||||
option_data["no"] = float(option["no"])
|
||||
option_data["abstain"] = float(option["abstain"])
|
||||
poll_data["options"].append(option_data)
|
||||
|
||||
if poll["state"] == AssignmentPoll.STATE_PUBLISHED:
|
||||
poll_data["amount_global_yes"] = (
|
||||
float(poll["amount_global_yes"]) if poll["amount_global_yes"] else None
|
||||
)
|
||||
poll_data["amount_global_no"] = (
|
||||
float(poll["amount_global_no"]) if poll["amount_global_no"] else None
|
||||
)
|
||||
poll_data["amount_global_abstain"] = (
|
||||
float(poll["amount_global_abstain"])
|
||||
if poll["amount_global_abstain"]
|
||||
else None
|
||||
)
|
||||
poll_data["votesvalid"] = float(poll["votesvalid"])
|
||||
poll_data["votesinvalid"] = float(poll["votesinvalid"])
|
||||
poll_data["votescast"] = float(poll["votescast"])
|
||||
|
||||
return {
|
||||
"assignment": {"title": assignment["title"]},
|
||||
"poll": poll_data,
|
||||
}
|
||||
|
||||
|
||||
def register_projector_slides() -> None:
|
||||
register_projector_slide("assignments/assignment", assignment_slide)
|
||||
register_projector_slide("assignments/assignment-poll", assignment_poll_slide)
|
@ -16,11 +16,9 @@ class CoreAppConfig(AppConfig):
|
||||
def ready(self):
|
||||
# Import all required stuff.
|
||||
# Let all client websocket message register
|
||||
from ..utils import websocket_client_messages # noqa
|
||||
from ..utils.rest_api import router
|
||||
from . import serializers # noqa
|
||||
from .config import config
|
||||
from .projector import register_projector_slides
|
||||
from .signals import (
|
||||
autoupdate_for_many_to_many_relations,
|
||||
cleanup_unused_permissions,
|
||||
@ -41,9 +39,6 @@ class CoreAppConfig(AppConfig):
|
||||
# Collect all config variables before getting the constants.
|
||||
config.collect_config_variables_from_apps()
|
||||
|
||||
# Define projector elements.
|
||||
register_projector_slides()
|
||||
|
||||
# Connect signals.
|
||||
post_permission_creation.connect(
|
||||
delete_django_app_permissions, dispatch_uid="delete_django_app_permissions"
|
||||
@ -126,6 +121,7 @@ class CoreAppConfig(AppConfig):
|
||||
|
||||
# Client settings
|
||||
client_settings_keys = [
|
||||
"AUTOUPDATE_DELAY",
|
||||
"PRIORITIZED_GROUP_IDS",
|
||||
"PING_INTERVAL",
|
||||
"PING_TIMEOUT",
|
||||
|
@ -1,68 +0,0 @@
|
||||
from typing import Any, Dict
|
||||
|
||||
from ..utils.projector import (
|
||||
ProjectorAllDataProvider,
|
||||
get_config,
|
||||
get_model,
|
||||
register_projector_slide,
|
||||
)
|
||||
|
||||
|
||||
async def countdown_slide(
|
||||
all_data_provider: ProjectorAllDataProvider,
|
||||
element: Dict[str, Any],
|
||||
projector_id: int,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Countdown slide.
|
||||
|
||||
Returns the full_data of the countdown element.
|
||||
|
||||
element = {
|
||||
name: 'core/countdown',
|
||||
id: 5, # Countdown ID
|
||||
}
|
||||
"""
|
||||
countdown = await get_model(all_data_provider, "core/countdown", element.get("id"))
|
||||
return {
|
||||
"description": countdown["description"],
|
||||
"running": countdown["running"],
|
||||
"countdown_time": countdown["countdown_time"],
|
||||
"warning_time": await get_config(
|
||||
all_data_provider, "agenda_countdown_warning_time"
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
async def message_slide(
|
||||
all_data_provider: ProjectorAllDataProvider,
|
||||
element: Dict[str, Any],
|
||||
projector_id: int,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Message slide.
|
||||
|
||||
Returns the full_data of the message element.
|
||||
|
||||
element = {
|
||||
name: 'core/projector-message',
|
||||
id: 5, # ProjectorMessage ID
|
||||
}
|
||||
"""
|
||||
return await get_model(
|
||||
all_data_provider, "core/projector-message", element.get("id")
|
||||
)
|
||||
|
||||
|
||||
async def clock_slide(
|
||||
all_data_provider: ProjectorAllDataProvider,
|
||||
element: Dict[str, Any],
|
||||
projector_id: int,
|
||||
) -> Dict[str, Any]:
|
||||
return {}
|
||||
|
||||
|
||||
def register_projector_slides() -> None:
|
||||
register_projector_slide("core/countdown", countdown_slide)
|
||||
register_projector_slide("core/projector-message", message_slide)
|
||||
register_projector_slide("core/clock", clock_slide)
|
@ -1,7 +1,6 @@
|
||||
from typing import Any
|
||||
|
||||
from ..core.config import config
|
||||
from ..utils.projector import projector_slides
|
||||
from ..utils.rest_api import (
|
||||
Field,
|
||||
IdPrimaryKeyRelatedField,
|
||||
@ -62,10 +61,6 @@ def elements_validator(value: Any) -> None:
|
||||
raise ValidationError(
|
||||
{"detail": "Every dictionary must have a key 'name'."}
|
||||
)
|
||||
if element["name"] not in projector_slides:
|
||||
raise ValidationError(
|
||||
{"detail": "Unknown projector element {0}.", "args": [element["name"]]}
|
||||
)
|
||||
|
||||
|
||||
def elements_array_validator(value: Any) -> None:
|
||||
|
@ -4,7 +4,8 @@ from . import views
|
||||
|
||||
|
||||
urlpatterns = [
|
||||
url(r"^servertime/$", views.ServerTime.as_view(), name="core_servertime"),
|
||||
url(r"^servertime/$", views.ServertimeView.as_view(), name="core_servertime"),
|
||||
url(r"^constants/$", views.ConstantsView.as_view(), name="core_constants"),
|
||||
url(r"^version/$", views.VersionView.as_view(), name="core_version"),
|
||||
url(
|
||||
r"^history/information/$",
|
||||
|
@ -22,6 +22,7 @@ from ..utils.arguments import arguments
|
||||
from ..utils.auth import GROUP_ADMIN_PK, anonymous_is_enabled, has_perm, in_some_groups
|
||||
from ..utils.autoupdate import inform_changed_data
|
||||
from ..utils.cache import element_cache
|
||||
from ..utils.constants import get_constants
|
||||
from ..utils.plugins import (
|
||||
get_plugin_description,
|
||||
get_plugin_license,
|
||||
@ -569,7 +570,7 @@ class CountdownViewSet(ModelViewSet):
|
||||
# Special API views
|
||||
|
||||
|
||||
class ServerTime(utils_views.APIView):
|
||||
class ServertimeView(utils_views.APIView):
|
||||
"""
|
||||
Returns the server time as UNIX timestamp.
|
||||
"""
|
||||
@ -580,6 +581,19 @@ class ServerTime(utils_views.APIView):
|
||||
return now().timestamp()
|
||||
|
||||
|
||||
class ConstantsView(utils_views.APIView):
|
||||
"""
|
||||
Returns the server time as UNIX timestamp.
|
||||
"""
|
||||
|
||||
http_method_names = ["get"]
|
||||
|
||||
def get_context_data(self, **context):
|
||||
if not self.request.user.is_authenticated and not anonymous_is_enabled():
|
||||
self.permission_denied(self.request)
|
||||
return get_constants()
|
||||
|
||||
|
||||
class VersionView(utils_views.APIView):
|
||||
"""
|
||||
Returns a dictionary with the OpenSlides version and the version of all
|
||||
|
@ -16,7 +16,6 @@ INSTALLED_APPS = [
|
||||
"django.contrib.sessions",
|
||||
"django.contrib.staticfiles",
|
||||
"rest_framework",
|
||||
"channels",
|
||||
"openslides.agenda",
|
||||
"openslides.topics",
|
||||
"openslides.motions",
|
||||
@ -122,13 +121,5 @@ PASSWORD_HASHERS = [
|
||||
MEDIA_URL = "/media/"
|
||||
|
||||
|
||||
# Django Channels
|
||||
# http://channels.readthedocs.io/en/latest/
|
||||
|
||||
ASGI_APPLICATION = "openslides.routing.application"
|
||||
|
||||
CHANNEL_LAYERS = {"default": {"BACKEND": "channels.layers.InMemoryChannelLayer"}}
|
||||
|
||||
|
||||
# Enable updating the last_login field for users on every login.
|
||||
ENABLE_LAST_LOGIN_FIELD = False
|
||||
|
@ -11,9 +11,7 @@ class MediafilesAppConfig(AppConfig):
|
||||
# Import all required stuff.
|
||||
from openslides.core.signals import permission_change
|
||||
from openslides.utils.rest_api import router
|
||||
|
||||
from . import serializers # noqa
|
||||
from .projector import register_projector_slides
|
||||
from .signals import get_permission_change_data
|
||||
from .views import MediafileViewSet
|
||||
|
||||
@ -25,9 +23,6 @@ class MediafilesAppConfig(AppConfig):
|
||||
"The MEDIA_URL setting must start and end with a slash"
|
||||
)
|
||||
|
||||
# Define projector elements.
|
||||
register_projector_slides()
|
||||
|
||||
# Connect signals.
|
||||
permission_change.connect(
|
||||
get_permission_change_data,
|
||||
|
@ -1,29 +0,0 @@
|
||||
from typing import Any, Dict
|
||||
|
||||
from ..utils.projector import (
|
||||
ProjectorAllDataProvider,
|
||||
get_model,
|
||||
register_projector_slide,
|
||||
)
|
||||
|
||||
|
||||
async def mediafile_slide(
|
||||
all_data_provider: ProjectorAllDataProvider,
|
||||
element: Dict[str, Any],
|
||||
projector_id: int,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Slide for Mediafile.
|
||||
"""
|
||||
mediafile = await get_model(
|
||||
all_data_provider, "mediafiles/mediafile", element.get("id")
|
||||
)
|
||||
return {
|
||||
"path": mediafile["path"],
|
||||
"mimetype": mediafile["mimetype"],
|
||||
"media_url_prefix": mediafile["media_url_prefix"],
|
||||
}
|
||||
|
||||
|
||||
def register_projector_slides() -> None:
|
||||
register_projector_slide("mediafiles/mediafile", mediafile_slide)
|
@ -12,10 +12,8 @@ class MotionsAppConfig(AppConfig):
|
||||
# Import all required stuff.
|
||||
from openslides.core.signals import permission_change
|
||||
from openslides.utils.rest_api import router
|
||||
|
||||
from ..utils.access_permissions import required_user
|
||||
from . import serializers # noqa
|
||||
from .projector import register_projector_slides
|
||||
from .signals import create_builtin_workflows, get_permission_change_data
|
||||
from .views import (
|
||||
CategoryViewSet,
|
||||
@ -31,9 +29,6 @@ class MotionsAppConfig(AppConfig):
|
||||
WorkflowViewSet,
|
||||
)
|
||||
|
||||
# Define projector elements.
|
||||
register_projector_slides()
|
||||
|
||||
# Connect signals.
|
||||
post_migrate.connect(
|
||||
create_builtin_workflows, dispatch_uid="motion_create_builtin_workflows"
|
||||
|
@ -1,437 +0,0 @@
|
||||
import re
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
from ..users.projector import get_user_name
|
||||
from ..utils.projector import (
|
||||
ProjectorAllDataProvider,
|
||||
ProjectorElementException,
|
||||
get_config,
|
||||
get_model,
|
||||
register_projector_slide,
|
||||
)
|
||||
from .models import MotionPoll
|
||||
|
||||
|
||||
motion_placeholder_regex = re.compile(r"\[motion:(\d+)\]")
|
||||
|
||||
|
||||
async def get_state(
|
||||
all_data_provider: ProjectorAllDataProvider,
|
||||
motion: Dict[str, Any],
|
||||
state_id_key: str,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Returns a state element from one motion. Raises an error if the state does not exist.
|
||||
"""
|
||||
state = await all_data_provider.get("motions/state", motion[state_id_key])
|
||||
if state is None:
|
||||
raise ProjectorElementException(
|
||||
f"motion {motion['id']} can not be on the state with id {motion[state_id_key]}"
|
||||
)
|
||||
return state
|
||||
|
||||
|
||||
async def get_amendment_merge_into_motion_diff(all_data_provider, amendment):
|
||||
"""
|
||||
HINT: This implementation should be consistent to showInDiffView() in ViewMotionAmendedParagraph.ts
|
||||
"""
|
||||
if amendment["state_id"] is None:
|
||||
return 0
|
||||
|
||||
state = await get_state(all_data_provider, amendment, "state_id")
|
||||
if state["merge_amendment_into_final"] == -1:
|
||||
return 0
|
||||
if state["merge_amendment_into_final"] == 1:
|
||||
return 1
|
||||
|
||||
if amendment["recommendation_id"] is None:
|
||||
return 0
|
||||
recommendation = await get_state(all_data_provider, amendment, "recommendation_id")
|
||||
if recommendation["merge_amendment_into_final"] == 1:
|
||||
return 1
|
||||
|
||||
return 0
|
||||
|
||||
|
||||
async def get_amendment_merge_into_motion_final(all_data_provider, amendment):
|
||||
"""
|
||||
HINT: This implementation should be consistent to showInFinalView() in ViewMotionAmendedParagraph.ts
|
||||
"""
|
||||
if amendment["state_id"] is None:
|
||||
return 0
|
||||
|
||||
state = await get_state(all_data_provider, amendment, "state_id")
|
||||
if state["merge_amendment_into_final"] == 1:
|
||||
return 1
|
||||
|
||||
return 0
|
||||
|
||||
|
||||
async def get_amendments_for_motion(motion, all_data_provider):
|
||||
amendment_data = []
|
||||
for amendment_id in motion["amendments_id"]:
|
||||
amendment = await all_data_provider.get("motions/motion", amendment_id)
|
||||
|
||||
merge_amendment_into_final = await get_amendment_merge_into_motion_final(
|
||||
all_data_provider, amendment
|
||||
)
|
||||
merge_amendment_into_diff = await get_amendment_merge_into_motion_diff(
|
||||
all_data_provider, amendment
|
||||
)
|
||||
|
||||
# Add change recommendations to the amendments:
|
||||
change_recommendations = [] # type: ignore
|
||||
for change_recommendation_id in amendment["change_recommendations_id"]:
|
||||
cr = await get_model(
|
||||
all_data_provider,
|
||||
"motions/motion-change-recommendation",
|
||||
change_recommendation_id,
|
||||
)
|
||||
if cr is not None and not cr["internal"] and not cr["rejected"]:
|
||||
change_recommendations.append(cr)
|
||||
|
||||
amendment_data.append(
|
||||
{
|
||||
"id": amendment["id"],
|
||||
"identifier": amendment["identifier"],
|
||||
"title": amendment["title"],
|
||||
"amendment_paragraphs": amendment["amendment_paragraphs"],
|
||||
"change_recommendations": change_recommendations,
|
||||
"merge_amendment_into_diff": merge_amendment_into_diff,
|
||||
"merge_amendment_into_final": merge_amendment_into_final,
|
||||
}
|
||||
)
|
||||
return amendment_data
|
||||
|
||||
|
||||
async def get_amendment_base_motion(amendment, all_data_provider):
|
||||
motion = await get_model(
|
||||
all_data_provider, "motions/motion", amendment.get("parent_id")
|
||||
)
|
||||
|
||||
return {
|
||||
"identifier": motion["identifier"],
|
||||
"title": motion["title"],
|
||||
"text": motion["text"],
|
||||
}
|
||||
|
||||
|
||||
async def get_amendment_base_statute(amendment, all_data_provider):
|
||||
statute = await get_model(
|
||||
all_data_provider,
|
||||
"motions/statute-paragraph",
|
||||
amendment.get("statute_paragraph_id"),
|
||||
)
|
||||
return {"title": statute["title"], "text": statute["text"]}
|
||||
|
||||
|
||||
async def extend_reference_motion_dict(
|
||||
all_data_provider: ProjectorAllDataProvider,
|
||||
recommendation: Optional[str],
|
||||
referenced_motions: Dict[int, Dict[str, str]],
|
||||
) -> None:
|
||||
"""
|
||||
Extends a dict of motion ids mapped to their title information.
|
||||
The client can replace the placeholders in the recommendation correctly.
|
||||
"""
|
||||
if recommendation is None:
|
||||
return
|
||||
|
||||
# Collect all meantioned motions via [motion:<id>]
|
||||
referenced_ids = [
|
||||
int(id) for id in motion_placeholder_regex.findall(recommendation)
|
||||
]
|
||||
for id in referenced_ids:
|
||||
# Put every referenced motion into the referenced_motions dict
|
||||
referenced_motion = await all_data_provider.get("motions/motion", id)
|
||||
if id not in referenced_motions and referenced_motion is not None:
|
||||
referenced_motions[id] = {
|
||||
"title": referenced_motion["title"],
|
||||
"identifier": referenced_motion["identifier"],
|
||||
}
|
||||
|
||||
|
||||
async def motion_slide(
|
||||
all_data_provider: ProjectorAllDataProvider,
|
||||
element: Dict[str, Any],
|
||||
projector_id: int,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Motion slide.
|
||||
|
||||
The returned dict can contain the following fields:
|
||||
* identifier
|
||||
* title
|
||||
* text
|
||||
* submitters
|
||||
* amendment_paragraphs
|
||||
* is_child
|
||||
* show_meta_box
|
||||
* show_referring_motions
|
||||
* reason
|
||||
* modified_final_version
|
||||
* recommendation
|
||||
* recommendation_extension
|
||||
* recommender
|
||||
* change_recommendations
|
||||
"""
|
||||
# Get motion
|
||||
mode = element.get(
|
||||
"mode", await get_config(all_data_provider, "motions_recommendation_text_mode")
|
||||
)
|
||||
|
||||
# populate cache:
|
||||
|
||||
motion = await get_model(all_data_provider, "motions/motion", element.get("id"))
|
||||
|
||||
# Add submitters
|
||||
submitters = [
|
||||
await get_user_name(all_data_provider, submitter["user_id"])
|
||||
for submitter in sorted(
|
||||
motion["submitters"], key=lambda submitter: submitter["weight"]
|
||||
)
|
||||
]
|
||||
|
||||
# Get some needed config values
|
||||
show_meta_box = not await get_config(
|
||||
all_data_provider, "motions_disable_sidebox_on_projector"
|
||||
)
|
||||
show_referring_motions = not await get_config(
|
||||
all_data_provider, "motions_hide_referring_motions"
|
||||
)
|
||||
line_length = await get_config(all_data_provider, "motions_line_length")
|
||||
line_numbering_mode = await get_config(
|
||||
all_data_provider, "motions_default_line_numbering"
|
||||
)
|
||||
motions_preamble = await get_config(all_data_provider, "motions_preamble")
|
||||
|
||||
# Query all change-recommendation and amendment related things.
|
||||
amendments = [] # type: ignore
|
||||
base_motion = None
|
||||
base_statute = None
|
||||
if motion["statute_paragraph_id"]:
|
||||
base_statute = await get_amendment_base_statute(motion, all_data_provider)
|
||||
elif motion["parent_id"] is not None and motion["amendment_paragraphs"]:
|
||||
base_motion = await get_amendment_base_motion(motion, all_data_provider)
|
||||
else:
|
||||
amendments = await get_amendments_for_motion(motion, all_data_provider)
|
||||
|
||||
change_recommendations = [] # type: ignore
|
||||
for change_recommendation_id in motion["change_recommendations_id"]:
|
||||
cr = await get_model(
|
||||
all_data_provider,
|
||||
"motions/motion-change-recommendation",
|
||||
change_recommendation_id,
|
||||
)
|
||||
if cr is not None and not cr["internal"]:
|
||||
change_recommendations.append(cr)
|
||||
|
||||
# The base return value. More fields will get added below.
|
||||
return_value = {
|
||||
"identifier": motion["identifier"],
|
||||
"title": motion["title"],
|
||||
"submitters": submitters,
|
||||
"preamble": motions_preamble,
|
||||
"amendment_paragraphs": motion["amendment_paragraphs"],
|
||||
"base_motion": base_motion,
|
||||
"base_statute": base_statute,
|
||||
"is_child": bool(motion["parent_id"]),
|
||||
"show_meta_box": show_meta_box,
|
||||
"show_referring_motions": show_referring_motions,
|
||||
"change_recommendations": change_recommendations,
|
||||
"amendments": amendments,
|
||||
"line_length": line_length,
|
||||
"line_numbering_mode": line_numbering_mode,
|
||||
}
|
||||
|
||||
if not await get_config(all_data_provider, "motions_disable_text_on_projector"):
|
||||
return_value["text"] = motion["text"]
|
||||
|
||||
if not await get_config(all_data_provider, "motions_disable_reason_on_projector"):
|
||||
return_value["reason"] = motion["reason"]
|
||||
|
||||
if mode == "final":
|
||||
return_value["modified_final_version"] = motion["modified_final_version"]
|
||||
|
||||
# Add recommendation, if enabled in config (and the motion has one)
|
||||
if (
|
||||
not await get_config(
|
||||
all_data_provider, "motions_disable_recommendation_on_projector"
|
||||
)
|
||||
and motion["recommendation_id"]
|
||||
):
|
||||
recommendation_state = await get_state(
|
||||
all_data_provider, motion, "recommendation_id"
|
||||
)
|
||||
return_value["recommendation"] = recommendation_state["recommendation_label"]
|
||||
if recommendation_state["show_recommendation_extension_field"]:
|
||||
recommendation_extension = motion["recommendation_extension"]
|
||||
# All title information for referenced motions in the recommendation
|
||||
referenced_motions: Dict[int, Dict[str, str]] = {}
|
||||
await extend_reference_motion_dict(
|
||||
all_data_provider, recommendation_extension, referenced_motions
|
||||
)
|
||||
return_value["recommendation_extension"] = recommendation_extension
|
||||
return_value["referenced_motions"] = referenced_motions
|
||||
if motion["statute_paragraph_id"]:
|
||||
return_value["recommender"] = await get_config(
|
||||
all_data_provider, "motions_statute_recommendations_by"
|
||||
)
|
||||
else:
|
||||
return_value["recommender"] = await get_config(
|
||||
all_data_provider, "motions_recommendations_by"
|
||||
)
|
||||
|
||||
if show_referring_motions:
|
||||
# Add recommendation-referencing motions
|
||||
return_value[
|
||||
"recommendation_referencing_motions"
|
||||
] = await get_recommendation_referencing_motions(
|
||||
all_data_provider, motion["id"]
|
||||
)
|
||||
|
||||
return return_value
|
||||
|
||||
|
||||
async def get_recommendation_referencing_motions(
|
||||
all_data_provider: ProjectorAllDataProvider, motion_id: int
|
||||
) -> Optional[List[Dict[str, Any]]]:
|
||||
"""
|
||||
Returns all title information for motions, that are referencing
|
||||
the given motion (by id) in their recommendation. If there are no
|
||||
motions, None is returned (instead of []).
|
||||
"""
|
||||
recommendation_referencing_motions = []
|
||||
all_motions = await all_data_provider.get_collection("motions/motion")
|
||||
for motion in all_motions.values():
|
||||
# Motion must have a recommendation and a recommendaiton extension
|
||||
if not motion["recommendation_id"] or not motion["recommendation_extension"]:
|
||||
continue
|
||||
|
||||
# The recommendation must allow the extension field (there might be left-overs
|
||||
# in a motions recommendation extension..)
|
||||
recommendation = await get_state(all_data_provider, motion, "recommendation_id")
|
||||
if not recommendation["show_recommendation_extension_field"]:
|
||||
continue
|
||||
|
||||
# Find referenced motion ids
|
||||
referenced_ids = [
|
||||
int(id)
|
||||
for id in motion_placeholder_regex.findall(
|
||||
motion["recommendation_extension"]
|
||||
)
|
||||
]
|
||||
|
||||
# if one of the referenced ids is the given motion, add the current motion.
|
||||
if motion_id in referenced_ids:
|
||||
recommendation_referencing_motions.append(
|
||||
{"title": motion["title"], "identifier": motion["identifier"]}
|
||||
)
|
||||
return recommendation_referencing_motions or None
|
||||
|
||||
|
||||
async def motion_block_slide(
|
||||
all_data_provider: ProjectorAllDataProvider,
|
||||
element: Dict[str, Any],
|
||||
projector_id: int,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Motion block slide.
|
||||
"""
|
||||
motion_block = await get_model(
|
||||
all_data_provider, "motions/motion-block", element.get("id")
|
||||
)
|
||||
|
||||
# All motions in this motion block
|
||||
motions = []
|
||||
|
||||
# All title information for referenced motions in the recommendation
|
||||
referenced_motions: Dict[int, Dict[str, str]] = {}
|
||||
|
||||
# iterate motions.
|
||||
for motion_id in motion_block["motions_id"]:
|
||||
motion = await all_data_provider.get("motions/motion", motion_id)
|
||||
# primarily to please mypy, should theoretically not happen
|
||||
if motion is None:
|
||||
raise RuntimeError(
|
||||
f"motion {motion_id} of block {element.get('id')} could not be found"
|
||||
)
|
||||
|
||||
motion_object = {
|
||||
"title": motion["title"],
|
||||
"identifier": motion["identifier"],
|
||||
}
|
||||
|
||||
recommendation_id = motion["recommendation_id"]
|
||||
if recommendation_id is not None:
|
||||
recommendation = await get_state(
|
||||
all_data_provider, motion, "recommendation_id"
|
||||
)
|
||||
motion_object["recommendation"] = {
|
||||
"name": recommendation["recommendation_label"],
|
||||
"css_class": recommendation["css_class"],
|
||||
}
|
||||
if recommendation["show_recommendation_extension_field"]:
|
||||
recommendation_extension = motion["recommendation_extension"]
|
||||
await extend_reference_motion_dict(
|
||||
all_data_provider, recommendation_extension, referenced_motions
|
||||
)
|
||||
motion_object["recommendation_extension"] = recommendation_extension
|
||||
|
||||
motions.append(motion_object)
|
||||
|
||||
return {
|
||||
"title": motion_block["title"],
|
||||
"motions": motions,
|
||||
"referenced_motions": referenced_motions,
|
||||
}
|
||||
|
||||
|
||||
async def motion_poll_slide(
|
||||
all_data_provider: ProjectorAllDataProvider,
|
||||
element: Dict[str, Any],
|
||||
projector_id: int,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Poll slide.
|
||||
"""
|
||||
poll = await get_model(all_data_provider, "motions/motion-poll", element.get("id"))
|
||||
motion = await get_model(all_data_provider, "motions/motion", poll["motion_id"])
|
||||
|
||||
poll_data = {
|
||||
key: poll[key]
|
||||
for key in (
|
||||
"title",
|
||||
"type",
|
||||
"pollmethod",
|
||||
"state",
|
||||
"onehundred_percent_base",
|
||||
"majority_method",
|
||||
)
|
||||
}
|
||||
|
||||
if poll["state"] == MotionPoll.STATE_PUBLISHED:
|
||||
option = await get_model(
|
||||
all_data_provider, "motions/motion-option", poll["options_id"][0]
|
||||
) # there can only be exactly one option
|
||||
poll_data["options"] = [
|
||||
{
|
||||
"yes": float(option["yes"]),
|
||||
"no": float(option["no"]),
|
||||
"abstain": float(option["abstain"]),
|
||||
}
|
||||
]
|
||||
poll_data["votesvalid"] = poll["votesvalid"]
|
||||
poll_data["votesinvalid"] = poll["votesinvalid"]
|
||||
poll_data["votescast"] = poll["votescast"]
|
||||
|
||||
return {
|
||||
"motion": {"title": motion["title"], "identifier": motion["identifier"]},
|
||||
"poll": poll_data,
|
||||
}
|
||||
|
||||
|
||||
def register_projector_slides() -> None:
|
||||
register_projector_slide("motions/motion", motion_slide)
|
||||
register_projector_slide("motions/motion-block", motion_block_slide)
|
||||
register_projector_slide("motions/motion-poll", motion_poll_slide)
|
@ -1,15 +0,0 @@
|
||||
from channels.routing import ProtocolTypeRouter, URLRouter
|
||||
from django.conf.urls import url
|
||||
|
||||
from openslides.utils.consumers import CloseConsumer, SiteConsumer
|
||||
from openslides.utils.middleware import AuthMiddlewareStack
|
||||
|
||||
|
||||
application = ProtocolTypeRouter(
|
||||
{
|
||||
# WebSocket chat handler
|
||||
"websocket": AuthMiddlewareStack(
|
||||
URLRouter([url(r"^ws/$", SiteConsumer), url(".*", CloseConsumer)])
|
||||
)
|
||||
}
|
||||
)
|
@ -11,13 +11,9 @@ class TopicsAppConfig(AppConfig):
|
||||
|
||||
from ..utils.rest_api import router
|
||||
from . import serializers # noqa
|
||||
from .projector import register_projector_slides
|
||||
from .signals import get_permission_change_data
|
||||
from .views import TopicViewSet
|
||||
|
||||
# Define projector elements.
|
||||
register_projector_slides()
|
||||
|
||||
# Connect signals.
|
||||
permission_change.connect(
|
||||
get_permission_change_data, dispatch_uid="topics_get_permission_change_data"
|
||||
|
@ -1,32 +0,0 @@
|
||||
from typing import Any, Dict
|
||||
|
||||
from ..utils.projector import (
|
||||
ProjectorAllDataProvider,
|
||||
get_model,
|
||||
register_projector_slide,
|
||||
)
|
||||
|
||||
|
||||
async def topic_slide(
|
||||
all_data_provider: ProjectorAllDataProvider,
|
||||
element: Dict[str, Any],
|
||||
projector_id: int,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Topic slide.
|
||||
|
||||
The returned dict can contain the following fields:
|
||||
* title
|
||||
* text
|
||||
"""
|
||||
topic = await get_model(all_data_provider, "topics/topic", element.get("id"))
|
||||
item = await get_model(all_data_provider, "agenda/item", topic["agenda_item_id"])
|
||||
return {
|
||||
"title": topic["title"],
|
||||
"text": topic["text"],
|
||||
"item_number": item["item_number"],
|
||||
}
|
||||
|
||||
|
||||
def register_projector_slides() -> None:
|
||||
register_projector_slide("topics/topic", topic_slide)
|
@ -15,13 +15,9 @@ class UsersAppConfig(AppConfig):
|
||||
from ..core.signals import permission_change, post_permission_creation
|
||||
from ..utils.rest_api import router
|
||||
from . import serializers # noqa
|
||||
from .projector import register_projector_slides
|
||||
from .signals import create_builtin_groups_and_admin, get_permission_change_data
|
||||
from .views import GroupViewSet, PersonalNoteViewSet, UserViewSet
|
||||
|
||||
# Define projector elements.
|
||||
register_projector_slides()
|
||||
|
||||
# Connect signals.
|
||||
post_permission_creation.connect(
|
||||
create_builtin_groups_and_admin,
|
||||
|
@ -1,44 +0,0 @@
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
from ..utils.projector import (
|
||||
ProjectorAllDataProvider,
|
||||
get_model,
|
||||
register_projector_slide,
|
||||
)
|
||||
|
||||
|
||||
async def user_slide(
|
||||
all_data_provider: ProjectorAllDataProvider,
|
||||
element: Dict[str, Any],
|
||||
projector_id: int,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
User slide.
|
||||
|
||||
The returned dict can contain the following fields:
|
||||
* user
|
||||
"""
|
||||
return {"user": await get_user_name(all_data_provider, element.get("id"))}
|
||||
|
||||
|
||||
async def get_user_name(
|
||||
all_data_provider: ProjectorAllDataProvider, user_id: Optional[int]
|
||||
) -> str:
|
||||
"""
|
||||
Returns the short name for an user_id.
|
||||
"""
|
||||
user = await get_model(all_data_provider, "users/user", user_id)
|
||||
|
||||
name_parts: List[str] = []
|
||||
for name_part in ("title", "first_name", "last_name"):
|
||||
if user[name_part]:
|
||||
name_parts.append(user[name_part])
|
||||
if not name_parts:
|
||||
name_parts.append(user["username"])
|
||||
if user["structure_level"]:
|
||||
name_parts.append(f"({user['structure_level']})")
|
||||
return " ".join(name_parts)
|
||||
|
||||
|
||||
def register_projector_slides() -> None:
|
||||
register_projector_slide("users/user", user_slide)
|
@ -4,13 +4,12 @@ from collections import defaultdict
|
||||
from typing import Any, Dict, Iterable, List, Optional, Tuple, Union
|
||||
|
||||
from asgiref.sync import async_to_sync
|
||||
from channels.layers import get_channel_layer
|
||||
from django.db.models import Model
|
||||
from mypy_extensions import TypedDict
|
||||
|
||||
from .auth import UserDoesNotExist
|
||||
from .cache import ChangeIdTooLowError, element_cache, get_element_id
|
||||
from .projector import get_projector_data
|
||||
from .stream import stream
|
||||
from .timing import Timing
|
||||
from .utils import get_model_from_collection_string, is_iterable, split_element_id
|
||||
|
||||
@ -112,8 +111,7 @@ class AutoupdateBundle:
|
||||
save_history(self.element_iterator)
|
||||
|
||||
# Update cache and send autoupdate using async code.
|
||||
change_id = async_to_sync(self.dispatch_autoupdate)()
|
||||
return change_id
|
||||
return async_to_sync(self.dispatch_autoupdate)()
|
||||
|
||||
@property
|
||||
def element_iterator(self) -> Iterable[AutoupdateElement]:
|
||||
@ -121,7 +119,7 @@ class AutoupdateBundle:
|
||||
for elements in self.autoupdate_elements.values():
|
||||
yield from elements.values()
|
||||
|
||||
async def update_cache(self) -> int:
|
||||
async def get_data_for_cache(self) -> Dict[str, Optional[Dict[str, Any]]]:
|
||||
"""
|
||||
Async helper function to update the cache.
|
||||
|
||||
@ -136,7 +134,7 @@ class AutoupdateBundle:
|
||||
"no_delete_on_restriction", False
|
||||
)
|
||||
cache_elements[element_id] = full_data
|
||||
return await element_cache.change_elements(cache_elements)
|
||||
return cache_elements
|
||||
|
||||
async def dispatch_autoupdate(self) -> int:
|
||||
"""
|
||||
@ -145,25 +143,12 @@ class AutoupdateBundle:
|
||||
Return the change_id
|
||||
"""
|
||||
# Update cache
|
||||
change_id = await self.update_cache()
|
||||
cache_elements = await self.get_data_for_cache()
|
||||
change_id = await element_cache.change_elements(cache_elements)
|
||||
|
||||
# Send autoupdate
|
||||
channel_layer = get_channel_layer()
|
||||
await channel_layer.group_send(
|
||||
"autoupdate", {"type": "msg_new_change_id", "change_id": change_id}
|
||||
)
|
||||
|
||||
# Send projector
|
||||
projector_data = await get_projector_data()
|
||||
channel_layer = get_channel_layer()
|
||||
await channel_layer.group_send(
|
||||
"projector",
|
||||
{
|
||||
"type": "msg_projector_data",
|
||||
"data": projector_data,
|
||||
"change_id": change_id,
|
||||
},
|
||||
)
|
||||
autoupdate_payload = {"elements": cache_elements, "change_id": change_id}
|
||||
await stream.send("autoupdate", autoupdate_payload)
|
||||
|
||||
return change_id
|
||||
|
||||
@ -286,14 +271,12 @@ class AutoupdateBundleMiddleware:
|
||||
if status_ok or status_redirect:
|
||||
change_id = bundle.done()
|
||||
|
||||
# inject the autoupdate, if there is an autoupdate and the status is
|
||||
# inject the change id, if there was an autoupdate and the response status is
|
||||
# ok (and not redirect; redirects do not have a useful content)
|
||||
if change_id is not None and status_ok:
|
||||
user_id = request.user.pk or 0
|
||||
# Inject the autoupdate in the response.
|
||||
# The complete response body will be overwritten!
|
||||
_, autoupdate = async_to_sync(get_autoupdate_data)(change_id, user_id)
|
||||
content = {"autoupdate": autoupdate, "data": response.data}
|
||||
content = {"change_id": change_id, "data": response.data}
|
||||
# Note: autoupdate may be none on skipped ones (which should not happen
|
||||
# since the user has made the request....)
|
||||
response.content = json.dumps(content)
|
||||
|
@ -163,6 +163,8 @@ class ElementCache:
|
||||
logger.info("Saving cache data into the cache...")
|
||||
await self.cache_provider.add_to_full_data(mapping)
|
||||
logger.info("Done saving the cache data.")
|
||||
await self.cache_provider.set_cache_ready()
|
||||
logger.info("Done: Cache is ready now.")
|
||||
|
||||
def _build_cache_get_elementid_model_mapping(
|
||||
self, config_only: bool = False
|
||||
|
@ -8,11 +8,7 @@ from django.core.exceptions import ImproperlyConfigured
|
||||
from typing_extensions import Protocol
|
||||
|
||||
from . import logging
|
||||
from .redis import (
|
||||
read_only_redis_amount_replicas,
|
||||
read_only_redis_wait_timeout,
|
||||
use_redis,
|
||||
)
|
||||
from .redis import use_redis
|
||||
from .schema_version import SchemaVersion
|
||||
from .utils import split_element_id, str_dict_to_bytes
|
||||
|
||||
@ -54,6 +50,9 @@ class ElementCacheProvider(Protocol):
|
||||
async def data_exists(self) -> bool:
|
||||
...
|
||||
|
||||
async def set_cache_ready(self) -> None:
|
||||
...
|
||||
|
||||
async def get_all_data(self) -> Dict[bytes, bytes]:
|
||||
...
|
||||
|
||||
@ -127,6 +126,7 @@ class RedisCacheProvider:
|
||||
full_data_cache_key: str = "full_data"
|
||||
change_id_cache_key: str = "change_id"
|
||||
schema_cache_key: str = "schema"
|
||||
cache_ready_key: str = "cache_ready"
|
||||
|
||||
# All lua-scripts used by this provider. Every entry is a Tuple (str, bool) with the
|
||||
# script and an ensure_cache-indicator. If the indicator is True, a short ensure_cache-script
|
||||
@ -325,6 +325,7 @@ class RedisCacheProvider:
|
||||
"""
|
||||
async with get_connection() as redis:
|
||||
tr = redis.multi_exec()
|
||||
tr.delete(self.cache_ready_key)
|
||||
tr.delete(self.change_id_cache_key)
|
||||
tr.delete(self.full_data_cache_key)
|
||||
tr.hmset_dict(self.full_data_cache_key, data)
|
||||
@ -342,11 +343,16 @@ class RedisCacheProvider:
|
||||
Returns True, when there is data in the cache.
|
||||
"""
|
||||
async with get_connection(read_only=True) as redis:
|
||||
return await redis.exists(self.full_data_cache_key) and bool(
|
||||
await redis.zrangebyscore(
|
||||
self.change_id_cache_key, withscores=True, count=1, offset=0
|
||||
)
|
||||
)
|
||||
return (await redis.get(self.cache_ready_key)) is not None
|
||||
# return await redis.exists(self.full_data_cache_key) and bool(
|
||||
# await redis.zrangebyscore(
|
||||
# self.change_id_cache_key, withscores=True, count=1, offset=0
|
||||
# )
|
||||
# )
|
||||
|
||||
async def set_cache_ready(self) -> None:
|
||||
async with get_connection(read_only=False) as redis:
|
||||
await redis.set(self.cache_ready_key, "ok")
|
||||
|
||||
@ensure_cache_wrapper()
|
||||
async def get_all_data(self) -> Dict[bytes, bytes]:
|
||||
@ -495,7 +501,11 @@ class RedisCacheProvider:
|
||||
async def get_schema_version(self) -> Optional[SchemaVersion]:
|
||||
""" Retrieves the schema version of the cache or None, if not existent """
|
||||
async with get_connection(read_only=True) as redis:
|
||||
try:
|
||||
schema_version = await redis.hgetall(self.schema_cache_key)
|
||||
except aioredis.errors.ReplyError:
|
||||
await redis.delete(self.schema_cache_key)
|
||||
return None
|
||||
if not schema_version:
|
||||
return None
|
||||
|
||||
@ -543,15 +553,6 @@ class RedisCacheProvider:
|
||||
raise CacheReset()
|
||||
else:
|
||||
raise e
|
||||
if not read_only and read_only_redis_amount_replicas is not None:
|
||||
reported_amount = await redis.wait(
|
||||
read_only_redis_amount_replicas, read_only_redis_wait_timeout
|
||||
)
|
||||
if reported_amount != read_only_redis_amount_replicas:
|
||||
logger.warn(
|
||||
f"WAIT reported {reported_amount} replicas of {read_only_redis_amount_replicas} "
|
||||
+ f"requested after {read_only_redis_wait_timeout} ms!"
|
||||
)
|
||||
return result
|
||||
|
||||
async def _eval(
|
||||
@ -584,6 +585,7 @@ class MemoryCacheProvider:
|
||||
self.set_data_dicts()
|
||||
|
||||
def set_data_dicts(self) -> None:
|
||||
self.ready = False
|
||||
self.full_data: Dict[str, str] = {}
|
||||
self.change_id_data: Dict[int, Set[str]] = {}
|
||||
self.locks: Dict[str, str] = {}
|
||||
@ -594,6 +596,7 @@ class MemoryCacheProvider:
|
||||
|
||||
async def clear_cache(self) -> None:
|
||||
self.set_data_dicts()
|
||||
self.ready = False
|
||||
|
||||
async def reset_full_cache(
|
||||
self, data: Dict[str, str], default_change_id: int
|
||||
@ -606,7 +609,11 @@ class MemoryCacheProvider:
|
||||
self.full_data.update(data)
|
||||
|
||||
async def data_exists(self) -> bool:
|
||||
return bool(self.full_data) and self.default_change_id >= 0
|
||||
return self.ready
|
||||
# return bool(self.full_data) and self.default_change_id >= 0
|
||||
|
||||
async def set_cache_ready(self) -> None:
|
||||
self.ready = True
|
||||
|
||||
async def get_all_data(self) -> Dict[bytes, bytes]:
|
||||
return str_dict_to_bytes(self.full_data)
|
||||
|
@ -1,102 +0,0 @@
|
||||
import asyncio
|
||||
from asyncio import Task
|
||||
from typing import Optional, cast
|
||||
|
||||
from django.conf import settings
|
||||
|
||||
from .autoupdate import get_autoupdate_data
|
||||
from .cache import element_cache
|
||||
from .websocket import ChangeIdTooHighException, ProtocollAsyncJsonWebsocketConsumer
|
||||
|
||||
|
||||
AUTOUPDATE_DELAY = getattr(settings, "AUTOUPDATE_DELAY", None)
|
||||
|
||||
|
||||
class ConsumerAutoupdateStrategy:
|
||||
def __init__(self, consumer: ProtocollAsyncJsonWebsocketConsumer) -> None:
|
||||
self.consumer = consumer
|
||||
# client_change_id = None: unknown -> set on first autoupdate or request_change_id
|
||||
# client_change_id is int: the change_id, the client knows about, so the next
|
||||
# update must be from client_change_id+1 .. <next clange_id>
|
||||
self.client_change_id: Optional[int] = None
|
||||
self.next_send_time = None
|
||||
self.timer_task_handle: Optional[Task[None]] = None
|
||||
self.lock = asyncio.Lock()
|
||||
|
||||
async def request_change_id(
|
||||
self, change_id: int, in_response: Optional[str] = None
|
||||
) -> None:
|
||||
"""
|
||||
The change id is not inclusive, so the client is on change_id and wants
|
||||
data from change_id+1 .. now
|
||||
"""
|
||||
# This resets the server side tracking of the client's change id.
|
||||
async with self.lock:
|
||||
await self.stop_timer()
|
||||
|
||||
max_change_id = await element_cache.get_current_change_id()
|
||||
self.client_change_id = change_id
|
||||
|
||||
if self.client_change_id == max_change_id:
|
||||
# The client is up-to-date, so nothing will be done
|
||||
return None
|
||||
|
||||
if self.client_change_id > max_change_id:
|
||||
message = (
|
||||
f"Requested change_id {self.client_change_id} is higher than the "
|
||||
+ f"highest change_id {max_change_id}."
|
||||
)
|
||||
raise ChangeIdTooHighException(message, in_response=in_response)
|
||||
|
||||
await self.send_autoupdate(in_response=in_response)
|
||||
|
||||
async def new_change_id(self, change_id: int) -> None:
|
||||
async with self.lock:
|
||||
if self.client_change_id is None:
|
||||
# The -1 is to send this autoupdate as the first one to he client.
|
||||
# Remember: the client_change_id is the change_id the client knows about
|
||||
self.client_change_id = change_id - 1
|
||||
|
||||
if AUTOUPDATE_DELAY is None: # feature deactivated, send directly
|
||||
await self.send_autoupdate()
|
||||
elif self.timer_task_handle is None:
|
||||
await self.start_timer()
|
||||
|
||||
async def get_running_loop(self) -> asyncio.AbstractEventLoop:
|
||||
if hasattr(asyncio, "get_running_loop"):
|
||||
return asyncio.get_running_loop() # type: ignore
|
||||
else:
|
||||
return asyncio.get_event_loop()
|
||||
|
||||
async def start_timer(self) -> None:
|
||||
loop = await self.get_running_loop()
|
||||
self.timer_task_handle = loop.create_task(self.timer_task())
|
||||
|
||||
async def stop_timer(self) -> None:
|
||||
if self.timer_task_handle is not None:
|
||||
self.timer_task_handle.cancel()
|
||||
self.timer_task_handle = None
|
||||
|
||||
async def timer_task(self) -> None:
|
||||
try:
|
||||
await asyncio.sleep(AUTOUPDATE_DELAY)
|
||||
except asyncio.CancelledError:
|
||||
return
|
||||
|
||||
async with self.lock:
|
||||
await self.send_autoupdate()
|
||||
self.timer_task_handle = None
|
||||
|
||||
async def send_autoupdate(self, in_response: Optional[str] = None) -> None:
|
||||
# here, 1 is added to the change_id, because the client_change_id is the id the client
|
||||
# *knows* about -> the client needs client_change_id+1 since get_autoupdate_data is
|
||||
# inclusive [change_id .. max_change_id].
|
||||
max_change_id, autoupdate = await get_autoupdate_data(
|
||||
cast(int, self.client_change_id) + 1, self.consumer.user_id
|
||||
)
|
||||
if autoupdate is not None:
|
||||
self.client_change_id = max_change_id
|
||||
# It will be send, so we can set the client_change_id
|
||||
await self.consumer.send_json(
|
||||
type="autoupdate", content=autoupdate, in_response=in_response
|
||||
)
|
@ -1,175 +0,0 @@
|
||||
import time
|
||||
from typing import Any, Dict, List, Optional, cast
|
||||
from urllib.parse import parse_qs
|
||||
|
||||
from channels.generic.websocket import AsyncWebsocketConsumer
|
||||
|
||||
from . import logging
|
||||
from .auth import UserDoesNotExist, async_anonymous_is_enabled
|
||||
from .cache import element_cache
|
||||
from .consumer_autoupdate_strategy import ConsumerAutoupdateStrategy
|
||||
from .utils import get_worker_id
|
||||
from .websocket import BaseWebsocketException, ProtocollAsyncJsonWebsocketConsumer
|
||||
|
||||
|
||||
logger = logging.getLogger("openslides.websocket")
|
||||
|
||||
|
||||
class SiteConsumer(ProtocollAsyncJsonWebsocketConsumer):
|
||||
"""
|
||||
Websocket Consumer for the site.
|
||||
"""
|
||||
|
||||
groups = ["site"]
|
||||
|
||||
ID_COUNTER = 0
|
||||
"""
|
||||
ID counter for assigning each instance of this class an unique id.
|
||||
"""
|
||||
|
||||
def __init__(self, *args: Any, **kwargs: Any) -> None:
|
||||
self.projector_hash: Dict[int, int] = {}
|
||||
SiteConsumer.ID_COUNTER += 1
|
||||
self._id = get_worker_id() + "-" + str(SiteConsumer.ID_COUNTER)
|
||||
self.autoupdate_strategy = ConsumerAutoupdateStrategy(self)
|
||||
super().__init__(*args, **kwargs)
|
||||
|
||||
async def connect(self) -> None:
|
||||
"""
|
||||
A user connects to the site.
|
||||
|
||||
If it is an anonymous user and anonymous is disabled, the connection is closed.
|
||||
|
||||
Sends the startup data to the user.
|
||||
"""
|
||||
self.user_id = self.scope["user"]["id"]
|
||||
|
||||
self.connect_time = time.time()
|
||||
# self.scope['user'] is the full_data dict of the user. For an
|
||||
# anonymous user is it the dict {'id': 0}
|
||||
change_id = None
|
||||
if not await async_anonymous_is_enabled() and not self.user_id:
|
||||
await self.accept() # workaround for #4009
|
||||
await self.close()
|
||||
logger.debug(f"connect: denied ({self._id})")
|
||||
return
|
||||
|
||||
query_string = cast(
|
||||
Dict[bytes, List[bytes]], parse_qs(self.scope["query_string"])
|
||||
)
|
||||
if b"change_id" in query_string:
|
||||
try:
|
||||
change_id = int(query_string[b"change_id"][0])
|
||||
except ValueError:
|
||||
await self.accept() # workaround for #4009
|
||||
await self.close()
|
||||
logger.debug(f"connect: wrong change id ({self._id})")
|
||||
return
|
||||
|
||||
await self.accept()
|
||||
|
||||
if change_id is not None:
|
||||
logger.debug(f"connect: change id {change_id} ({self._id})")
|
||||
try:
|
||||
await self.request_autoupdate(change_id)
|
||||
except BaseWebsocketException as e:
|
||||
await self.send_exception(e)
|
||||
else:
|
||||
logger.debug(f"connect: no change id ({self._id})")
|
||||
|
||||
await self.channel_layer.group_add("autoupdate", self.channel_name)
|
||||
|
||||
async def disconnect(self, close_code: int) -> None:
|
||||
"""
|
||||
A user disconnects. Remove it from autoupdate.
|
||||
"""
|
||||
await self.channel_layer.group_discard("autoupdate", self.channel_name)
|
||||
active_seconds = int(time.time() - self.connect_time)
|
||||
logger.debug(
|
||||
f"disconnect code={close_code} active_secs={active_seconds} ({self._id})"
|
||||
)
|
||||
|
||||
async def msg_new_change_id(self, event: Dict[str, Any]) -> None:
|
||||
"""
|
||||
Send changed or deleted elements to the user.
|
||||
"""
|
||||
change_id = event["change_id"]
|
||||
try:
|
||||
await self.autoupdate_strategy.new_change_id(change_id)
|
||||
except UserDoesNotExist:
|
||||
# Maybe the user was deleted, but a websocket connection is still open to the user.
|
||||
# So we can close this connection and return.
|
||||
await self.close()
|
||||
|
||||
async def msg_projector_data(self, event: Dict[str, Any]) -> None:
|
||||
"""
|
||||
The projector has changed.
|
||||
"""
|
||||
all_projector_data = event["data"]
|
||||
change_id = event["change_id"]
|
||||
|
||||
projector_data: Dict[int, Dict[str, Any]] = {}
|
||||
for projector_id in self.listen_projector_ids:
|
||||
data = all_projector_data.get(projector_id, [])
|
||||
new_hash = hash(str(data))
|
||||
if new_hash != self.projector_hash.get(projector_id):
|
||||
projector_data[projector_id] = data
|
||||
self.projector_hash[projector_id] = new_hash
|
||||
|
||||
if projector_data:
|
||||
await self.send_projector_data(projector_data, change_id=change_id)
|
||||
|
||||
async def msg_notify(self, event: Dict[str, Any]) -> None:
|
||||
"""
|
||||
Send a notify message to the user.
|
||||
"""
|
||||
item = event["incomming"]
|
||||
|
||||
users = item.get("users")
|
||||
reply_channels = item.get("replyChannels")
|
||||
if (
|
||||
(isinstance(users, bool) and users)
|
||||
or (isinstance(users, list) and self.user_id in users)
|
||||
or (
|
||||
isinstance(reply_channels, list) and self.channel_name in reply_channels
|
||||
)
|
||||
or (users is None and reply_channels is None)
|
||||
):
|
||||
item["senderChannelName"] = event["senderChannelName"]
|
||||
item["senderUserId"] = event["senderUserId"]
|
||||
await self.send_json(type="notify", content=item)
|
||||
|
||||
async def request_autoupdate(
|
||||
self, change_id: int, in_response: Optional[str] = None
|
||||
) -> None:
|
||||
await self.autoupdate_strategy.request_change_id(
|
||||
change_id, in_response=in_response
|
||||
)
|
||||
|
||||
async def send_projector_data(
|
||||
self,
|
||||
data: Dict[int, Dict[str, Any]],
|
||||
change_id: Optional[int] = None,
|
||||
in_response: Optional[str] = None,
|
||||
) -> None:
|
||||
"""
|
||||
Sends projector data to the consumer.
|
||||
"""
|
||||
if change_id is None:
|
||||
change_id = await element_cache.get_current_change_id()
|
||||
|
||||
content = {"change_id": change_id, "data": data}
|
||||
await self.send_json(type="projector", content=content, in_response=in_response)
|
||||
|
||||
|
||||
class CloseConsumer(AsyncWebsocketConsumer):
|
||||
""" Auto-closes the connection """
|
||||
|
||||
groups: List[str] = []
|
||||
|
||||
def __init__(self, args: Dict[str, Any], **kwargs: Any) -> None:
|
||||
logger.info(f'Closing connection to unknown websocket url {args["path"]}')
|
||||
|
||||
async def connect(self) -> None:
|
||||
await self.accept()
|
||||
await self.close()
|
@ -1,73 +0,0 @@
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
from channels.auth import (
|
||||
AuthMiddleware,
|
||||
CookieMiddleware,
|
||||
SessionMiddleware,
|
||||
_get_user_session_key,
|
||||
)
|
||||
from django.conf import settings
|
||||
from django.contrib.auth import BACKEND_SESSION_KEY, HASH_SESSION_KEY
|
||||
from django.utils.crypto import constant_time_compare
|
||||
|
||||
from .cache import element_cache
|
||||
|
||||
|
||||
class CollectionAuthMiddleware(AuthMiddleware):
|
||||
"""
|
||||
Like the channels AuthMiddleware but returns a user dict id instead of
|
||||
a django Model as user.
|
||||
"""
|
||||
|
||||
def populate_scope(self, scope: Dict[str, Any]) -> None:
|
||||
# Make sure we have a session
|
||||
if "session" not in scope:
|
||||
raise ValueError(
|
||||
"AuthMiddleware cannot find session in scope. SessionMiddleware must be above it."
|
||||
)
|
||||
# Add it to the scope if it's not there already
|
||||
if "user" not in scope:
|
||||
scope["user"] = {}
|
||||
|
||||
async def resolve_scope(self, scope: Dict[str, Any]) -> None:
|
||||
scope["user"].update(await get_user(scope))
|
||||
|
||||
|
||||
async def get_user(scope: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""
|
||||
Returns a user id from a channels-scope-session.
|
||||
|
||||
If no user is retrieved, return {'id': 0}.
|
||||
"""
|
||||
# This code is basicly from channels.auth:
|
||||
# https://github.com/django/channels/blob/d5e81a78e96770127da79248349808b6ee6ec2a7/channels/auth.py#L16
|
||||
if "session" not in scope:
|
||||
raise ValueError(
|
||||
"Cannot find session in scope. You should wrap your consumer in SessionMiddleware."
|
||||
)
|
||||
session = scope["session"]
|
||||
user: Optional[Dict[str, Any]] = None
|
||||
try:
|
||||
user_id = _get_user_session_key(session)
|
||||
backend_path = session[BACKEND_SESSION_KEY]
|
||||
except KeyError:
|
||||
pass
|
||||
else:
|
||||
if backend_path in settings.AUTHENTICATION_BACKENDS:
|
||||
user = await element_cache.get_element_data("users/user", user_id)
|
||||
if user:
|
||||
# Verify the session
|
||||
session_hash = session.get(HASH_SESSION_KEY)
|
||||
session_hash_verified = session_hash and constant_time_compare(
|
||||
session_hash, user["session_auth_hash"]
|
||||
)
|
||||
if not session_hash_verified:
|
||||
session.flush()
|
||||
user = None
|
||||
return user or {"id": 0}
|
||||
|
||||
|
||||
# Handy shortcut for applying all three layers at once
|
||||
AuthMiddlewareStack = lambda inner: CookieMiddleware( # noqa
|
||||
SessionMiddleware(CollectionAuthMiddleware(inner))
|
||||
)
|
@ -1,172 +0,0 @@
|
||||
"""
|
||||
General projector code.
|
||||
|
||||
Functions that handel the registration of projector elements and the rendering
|
||||
of the data to present it on the projector.
|
||||
"""
|
||||
|
||||
from collections import defaultdict
|
||||
from typing import Any, Awaitable, Callable, Dict, List, Optional
|
||||
|
||||
from . import logging
|
||||
from .cache import element_cache
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ProjectorElementException(Exception):
|
||||
"""
|
||||
Exception for errors in one element on the projector.
|
||||
"""
|
||||
|
||||
|
||||
class ProjectorAllDataProvider:
|
||||
NON_EXISTENT_MARKER = object()
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.cache: Any = defaultdict(dict) # fuu you mypy
|
||||
self.fetched_collection: Dict[str, bool] = {}
|
||||
|
||||
async def get(self, collection: str, id: int) -> Optional[Dict[str, Any]]:
|
||||
cache_data = self.cache[collection].get(id)
|
||||
if cache_data is None:
|
||||
data: Any = await element_cache.get_element_data(collection, id)
|
||||
if data is None:
|
||||
data = ProjectorAllDataProvider.NON_EXISTENT_MARKER
|
||||
self.cache[collection][id] = data
|
||||
|
||||
cache_data = self.cache[collection][id]
|
||||
if cache_data == ProjectorAllDataProvider.NON_EXISTENT_MARKER:
|
||||
return None
|
||||
return cache_data
|
||||
|
||||
async def get_collection(self, collection: str) -> Dict[int, Dict[str, Any]]:
|
||||
if not self.fetched_collection.get(collection, False):
|
||||
collection_data = await element_cache.get_collection_data(collection)
|
||||
self.cache[collection] = collection_data
|
||||
self.fetched_collection[collection] = True
|
||||
return self.cache[collection]
|
||||
|
||||
async def exists(self, collection: str, id: int) -> bool:
|
||||
model = await self.get(collection, id)
|
||||
return model is not None
|
||||
|
||||
|
||||
ProjectorSlide = Callable[
|
||||
[ProjectorAllDataProvider, Dict[str, Any], int], Awaitable[Dict[str, Any]]
|
||||
]
|
||||
|
||||
|
||||
projector_slides: Dict[str, ProjectorSlide] = {}
|
||||
|
||||
|
||||
def register_projector_slide(name: str, slide: ProjectorSlide) -> None:
|
||||
"""
|
||||
Registers a projector slide.
|
||||
|
||||
Has to be called in the app.ready method.
|
||||
"""
|
||||
projector_slides[name] = slide
|
||||
|
||||
|
||||
async def get_projector_data(
|
||||
projector_ids: List[int] = None,
|
||||
) -> Dict[int, List[Dict[str, Any]]]:
|
||||
"""
|
||||
Calculates and returns the data for one or all projectors.
|
||||
|
||||
The keys of the returned data are the projector ids as int. When converted
|
||||
to json, the numbers will changed to strings like "1".
|
||||
|
||||
The data for each projector is a list of elements.
|
||||
|
||||
Each element is a dict where the keys are "elements", "data". "elements"
|
||||
contains the projector elements. It is the same as the projector elements in
|
||||
the database. "data" contains all necessary data to render the projector
|
||||
element. The key can also be "error" if there is a generall error for the
|
||||
slide. In this case the values "elements" and "data" are optional.
|
||||
|
||||
The returned value looks like this:
|
||||
|
||||
projector_data = {
|
||||
1: [
|
||||
{
|
||||
"element": {
|
||||
"name": "agenda/item-list",
|
||||
},
|
||||
"data": {
|
||||
"items": []
|
||||
},
|
||||
},
|
||||
],
|
||||
}
|
||||
"""
|
||||
if projector_ids is None:
|
||||
projector_ids = []
|
||||
|
||||
projector_data: Dict[int, List[Dict[str, Any]]] = {}
|
||||
all_data_provider = ProjectorAllDataProvider()
|
||||
projectors = await all_data_provider.get_collection("core/projector")
|
||||
|
||||
for projector_id, projector in projectors.items():
|
||||
if projector_ids and projector_id not in projector_ids:
|
||||
# only render the projector in question.
|
||||
continue
|
||||
|
||||
if not projector["elements"]:
|
||||
# Skip empty elements.
|
||||
continue
|
||||
|
||||
projector_data[projector_id] = []
|
||||
for element in projector["elements"]:
|
||||
projector_slide = projector_slides[element["name"]]
|
||||
try:
|
||||
data = await projector_slide(all_data_provider, element, projector_id)
|
||||
except ProjectorElementException as err:
|
||||
data = {"error": str(err)}
|
||||
projector_data[projector_id].append({"data": data, "element": element})
|
||||
|
||||
return projector_data
|
||||
|
||||
|
||||
async def get_config(all_data_provider: ProjectorAllDataProvider, key: str) -> Any:
|
||||
"""
|
||||
Returns a config value from all_data_provider.
|
||||
Triggers the cache early: It access `get_colelction` instead of `get`. It
|
||||
allows for all successive queries for configs to be cached.
|
||||
"""
|
||||
from ..core.config import config
|
||||
|
||||
config_id = (await config.async_get_key_to_id())[key]
|
||||
|
||||
configs = await all_data_provider.get_collection(config.get_collection_string())
|
||||
return configs[config_id]["value"]
|
||||
|
||||
|
||||
async def get_model(
|
||||
all_data_provider: ProjectorAllDataProvider, collection: str, id: Any
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Tries to get the model identified by the collection and id.
|
||||
If the id is invalid or the model not found, ProjectorElementExceptions will be raised.
|
||||
"""
|
||||
if id is None:
|
||||
raise ProjectorElementException(f"id is required for {collection} slide")
|
||||
|
||||
model = await all_data_provider.get(collection, id)
|
||||
if model is None:
|
||||
raise ProjectorElementException(f"{collection} with id {id} does not exist")
|
||||
return model
|
||||
|
||||
|
||||
async def get_models(
|
||||
all_data_provider: ProjectorAllDataProvider, collection: str, ids: List[Any]
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Tries to fetch all given models. Models are required to be all of the collection `collection`.
|
||||
"""
|
||||
logger.info(
|
||||
f"Note: a call to `get_models` with {collection}/{ids}. This might be cache-intensive"
|
||||
)
|
||||
return [await get_model(all_data_provider, collection, id) for id in ids]
|
@ -10,15 +10,13 @@ logger = logging.getLogger(__name__)
|
||||
# Defaults
|
||||
use_redis = False
|
||||
use_read_only_redis = False
|
||||
read_only_redis_amount_replicas = None
|
||||
read_only_redis_wait_timeout = None
|
||||
|
||||
try:
|
||||
import aioredis
|
||||
except ImportError:
|
||||
pass
|
||||
else:
|
||||
from .redis_connection_pool import ConnectionPool
|
||||
from .redis_connection_pool import ConnectionPool # type: ignore
|
||||
|
||||
# set use_redis to true, if there is a value for REDIS_ADDRESS in the settings
|
||||
redis_address = getattr(settings, "REDIS_ADDRESS", "")
|
||||
@ -33,13 +31,6 @@ else:
|
||||
if use_read_only_redis:
|
||||
logger.info(f"Redis read only address {redis_read_only_address}")
|
||||
read_only_pool = ConnectionPool({"address": redis_read_only_address})
|
||||
|
||||
read_only_redis_amount_replicas = getattr(settings, "AMOUNT_REPLICAS", 1)
|
||||
logger.info(f"AMOUNT_REPLICAS={read_only_redis_amount_replicas}")
|
||||
read_only_redis_wait_timeout = getattr(
|
||||
settings, "REDIS_SLAVE_WAIT_TIMEOUT", 1000
|
||||
)
|
||||
logger.info(f"REDIS_SLAVE_WAIT_TIMEOUT={read_only_redis_wait_timeout}")
|
||||
else:
|
||||
logger.info("Redis is not configured.")
|
||||
|
||||
|
@ -1,8 +1,10 @@
|
||||
# type: ignore
|
||||
import asyncio
|
||||
import sys
|
||||
import types
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
import aioredis
|
||||
from channels_redis.core import ConnectionPool as ChannelRedisConnectionPool
|
||||
from django.conf import settings
|
||||
|
||||
from . import logging
|
||||
@ -13,6 +15,128 @@ connection_pool_limit = getattr(settings, "CONNECTION_POOL_LIMIT", 100)
|
||||
logger.info(f"CONNECTION_POOL_LIMIT={connection_pool_limit}")
|
||||
|
||||
|
||||
# Copied from https://github.com/django/channels_redis/blob/master/channels_redis/core.py
|
||||
# and renamed..
|
||||
|
||||
AIOREDIS_VERSION = tuple(map(int, aioredis.__version__.split(".")))
|
||||
|
||||
|
||||
def _wrap_close(loop, pool):
|
||||
"""
|
||||
Decorate an event loop's close method with our own.
|
||||
"""
|
||||
original_impl = loop.close
|
||||
|
||||
def _wrapper(self, *args, **kwargs):
|
||||
# If the event loop was closed, there's nothing we can do anymore.
|
||||
if not self.is_closed():
|
||||
self.run_until_complete(pool.close_loop(self))
|
||||
# Restore the original close() implementation after we're done.
|
||||
self.close = original_impl
|
||||
return self.close(*args, **kwargs)
|
||||
|
||||
loop.close = types.MethodType(_wrapper, loop)
|
||||
|
||||
|
||||
class ChannelRedisConnectionPool:
|
||||
"""
|
||||
Connection pool manager for the channel layer.
|
||||
It manages a set of connections for the given host specification and
|
||||
taking into account asyncio event loops.
|
||||
"""
|
||||
|
||||
def __init__(self, host):
|
||||
self.host = host
|
||||
self.conn_map = {}
|
||||
self.in_use = {}
|
||||
|
||||
def _ensure_loop(self, loop):
|
||||
"""
|
||||
Get connection list for the specified loop.
|
||||
"""
|
||||
if loop is None:
|
||||
loop = asyncio.get_event_loop()
|
||||
|
||||
if loop not in self.conn_map:
|
||||
# Swap the loop's close method with our own so we get
|
||||
# a chance to do some cleanup.
|
||||
_wrap_close(loop, self)
|
||||
self.conn_map[loop] = []
|
||||
|
||||
return self.conn_map[loop], loop
|
||||
|
||||
async def pop(self, loop=None):
|
||||
"""
|
||||
Get a connection for the given identifier and loop.
|
||||
"""
|
||||
conns, loop = self._ensure_loop(loop)
|
||||
if not conns:
|
||||
if sys.version_info >= (3, 8, 0) and AIOREDIS_VERSION >= (1, 3, 1):
|
||||
conn = await aioredis.create_redis(**self.host)
|
||||
else:
|
||||
conn = await aioredis.create_redis(**self.host, loop=loop)
|
||||
conns.append(conn)
|
||||
conn = conns.pop()
|
||||
if conn.closed:
|
||||
conn = await self.pop(loop=loop)
|
||||
return conn
|
||||
self.in_use[conn] = loop
|
||||
return conn
|
||||
|
||||
def push(self, conn):
|
||||
"""
|
||||
Return a connection to the pool.
|
||||
"""
|
||||
loop = self.in_use[conn]
|
||||
del self.in_use[conn]
|
||||
if loop is not None:
|
||||
conns, _ = self._ensure_loop(loop)
|
||||
conns.append(conn)
|
||||
|
||||
def conn_error(self, conn):
|
||||
"""
|
||||
Handle a connection that produced an error.
|
||||
"""
|
||||
conn.close()
|
||||
del self.in_use[conn]
|
||||
|
||||
def reset(self):
|
||||
"""
|
||||
Clear all connections from the pool.
|
||||
"""
|
||||
self.conn_map = {}
|
||||
self.in_use = {}
|
||||
|
||||
async def close_loop(self, loop):
|
||||
"""
|
||||
Close all connections owned by the pool on the given loop.
|
||||
"""
|
||||
if loop in self.conn_map:
|
||||
for conn in self.conn_map[loop]:
|
||||
conn.close()
|
||||
await conn.wait_closed()
|
||||
del self.conn_map[loop]
|
||||
|
||||
for k, v in self.in_use.items():
|
||||
if v is loop:
|
||||
self.in_use[k] = None
|
||||
|
||||
async def close(self):
|
||||
"""
|
||||
Close all connections owned by the pool.
|
||||
"""
|
||||
conn_map = self.conn_map
|
||||
in_use = self.in_use
|
||||
self.reset()
|
||||
for conns in conn_map.values():
|
||||
for conn in conns:
|
||||
conn.close()
|
||||
await conn.wait_closed()
|
||||
for conn in in_use:
|
||||
conn.close()
|
||||
await conn.wait_closed()
|
||||
|
||||
|
||||
class InvalidConnection(Exception):
|
||||
pass
|
||||
|
||||
|
@ -83,40 +83,15 @@ DATABASES = {
|
||||
}
|
||||
|
||||
|
||||
# Set use_redis to True to activate redis as cache-, asgi- and session backend.
|
||||
use_redis = False
|
||||
|
||||
if use_redis:
|
||||
# Django Channels
|
||||
|
||||
# https://channels.readthedocs.io/en/latest/topics/channel_layers.html#configuration
|
||||
CHANNEL_LAYERS = {
|
||||
"default": {
|
||||
"BACKEND": "channels_redis.core.RedisChannelLayer",
|
||||
"CONFIG": {
|
||||
"hosts": [("localhost", 6379)],
|
||||
"capacity": 100000,
|
||||
},
|
||||
},
|
||||
}
|
||||
# Collection Cache
|
||||
|
||||
# Can be:
|
||||
# a Redis URI — "redis://host:6379/0?encoding=utf-8";
|
||||
# a (host, port) tuple — ('localhost', 6379);
|
||||
# or a unix domain socket path string — "/path/to/redis.sock".
|
||||
REDIS_ADDRESS = "redis://127.0.0.1"
|
||||
# REDIS_READ_ONLY_ADDRESS
|
||||
AMOUNT_REPLICAS = 1
|
||||
REDIS_ADDRESS = "redis://redis:6379/0"
|
||||
|
||||
# Session backend
|
||||
|
||||
# Redis configuration for django-redis-sessions.
|
||||
# https://github.com/martinrusev/django-redis-sessions
|
||||
|
||||
SESSION_ENGINE = 'redis_sessions.session'
|
||||
SESSION_REDIS = {
|
||||
'host': '127.0.0.1',
|
||||
'host': 'redis',
|
||||
'port': 6379,
|
||||
'db': 0,
|
||||
'prefix': 'session',
|
||||
@ -144,21 +119,17 @@ ENABLE_ELECTRONIC_VOTING = False
|
||||
|
||||
# Internationalization
|
||||
# https://docs.djangoproject.com/en/1.10/topics/i18n/
|
||||
|
||||
TIME_ZONE = 'Europe/Berlin'
|
||||
|
||||
|
||||
# Static files (CSS, JavaScript, Images)
|
||||
# https://docs.djangoproject.com/en/1.10/howto/static-files/
|
||||
|
||||
STATICFILES_DIRS = [os.path.join(OPENSLIDES_USER_DATA_DIR, 'static')] + STATICFILES_DIRS
|
||||
|
||||
STATIC_ROOT = os.path.join(OPENSLIDES_USER_DATA_DIR, 'collected-static')
|
||||
|
||||
|
||||
# Files
|
||||
# https://docs.djangoproject.com/en/1.10/topics/files/
|
||||
|
||||
MEDIA_ROOT = os.path.join(OPENSLIDES_USER_DATA_DIR, 'media', '')
|
||||
|
||||
|
||||
@ -169,7 +140,6 @@ MEDIA_ROOT = os.path.join(OPENSLIDES_USER_DATA_DIR, 'media', '')
|
||||
|
||||
# Logging
|
||||
# see https://docs.djangoproject.com/en/2.2/topics/logging/
|
||||
|
||||
LOGGING = {
|
||||
'version': 1,
|
||||
'disable_existing_loggers': False,
|
||||
|
50
server/openslides/utils/stream.py
Normal file
50
server/openslides/utils/stream.py
Normal file
@ -0,0 +1,50 @@
|
||||
import json
|
||||
from typing import Any
|
||||
|
||||
from django.conf import settings
|
||||
from typing_extensions import Protocol
|
||||
|
||||
from . import logging
|
||||
from .redis import use_redis
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
REDIS_STREAM_MAXLEN = getattr(settings, "REDIS_STREAM_MAXLEN", None)
|
||||
logger.info(f"Redis stream maxlen {REDIS_STREAM_MAXLEN}")
|
||||
|
||||
if use_redis:
|
||||
from .redis import get_connection
|
||||
|
||||
|
||||
class Stream(Protocol):
|
||||
def __init__(self) -> None:
|
||||
...
|
||||
|
||||
async def send(self, stream_name: str, payload: Any) -> None:
|
||||
...
|
||||
|
||||
|
||||
class RedisStream:
|
||||
async def send(self, stream_name: str, payload: Any) -> None:
|
||||
fields = {
|
||||
"content": json.dumps(payload, separators=(",", ":")),
|
||||
}
|
||||
async with get_connection() as redis:
|
||||
await redis.xadd(stream_name, fields, max_len=REDIS_STREAM_MAXLEN)
|
||||
|
||||
|
||||
class NoopStream:
|
||||
async def send(self, stream_name: str, payload: Any) -> None:
|
||||
pass
|
||||
|
||||
|
||||
def load_stream() -> Stream:
|
||||
if use_redis:
|
||||
return RedisStream()
|
||||
else:
|
||||
logger.error("You have to configure redis to let OpenSlides work properly!")
|
||||
return NoopStream()
|
||||
|
||||
|
||||
stream = load_stream()
|
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue
Block a user