Big Mode for OpenSlides
Uses django channels instead of tornado for the autoupdate. Therefore tornado is nolonger a dependency of OpenSlides (but channels). This uses websockets instead of SockJS. Use the flag insecure in the start command to provide static files serving. Use a new session backend that has a ForeignKey to User.
This commit is contained in:
Oskar Hahn
parent
dbbaeb245c
commit
fe64941aab
13
CHANGELOG
13
CHANGELOG
@ -4,12 +4,17 @@
|
||||
|
||||
https://openslides.org/
|
||||
|
||||
Version 2.0.1 (unreleased)
|
||||
==========================
|
||||
[https://github.com/OpenSlides/OpenSlides/milestones/2.0.1]
|
||||
Version 2.1 (unreleased)
|
||||
========================
|
||||
[https://github.com/OpenSlides/OpenSlides/milestones/2.1]
|
||||
|
||||
Core:
|
||||
- Used Django Channels instead of Tornado.
|
||||
- Added support for big assemblies with lots of users.
|
||||
|
||||
Other:
|
||||
- Remove config cache to support multiple threads or processes.
|
||||
- Removed config cache to support multiple threads or processes.
|
||||
- Fixed bug, that the last change of a config value was not send via autoupdate.
|
||||
|
||||
|
||||
Version 2.0 (2016-04-18)
|
||||
|
||||
50
README.rst
50
README.rst
@ -79,15 +79,14 @@ To start OpenSlides simply run::
|
||||
|
||||
$ openslides
|
||||
|
||||
If you run this command the first time, a new database and the admin
|
||||
account (Username: `admin`, Password: `admin`) will be created. Please
|
||||
change the password after first login!
|
||||
If you run this command the first time, a new database and the admin account
|
||||
(Username: `admin`, Password: `admin`) will be created. Please change the
|
||||
password after first login!
|
||||
|
||||
OpenSlides will start using the integrated Tornado webserver. It will also
|
||||
try to open the webinterface in your default webbrowser. The server will
|
||||
try to listen on the local ip address on port 8000. That means that the
|
||||
server will be available to everyone on your local network (at least for
|
||||
commonly used network configurations).
|
||||
OpenSlides will start a webserver. It will also try to open the webinterface in
|
||||
your default webbrowser. The server will try to listen on the local ip address
|
||||
on port 8000. That means that the server will be available to everyone on your
|
||||
local network (at least for commonly used network configurations).
|
||||
|
||||
If you use a virtual environment (see step b.), do not forget to activate
|
||||
the environment before restart after you have closed the terminal::
|
||||
@ -122,6 +121,32 @@ If you want to contribute to OpenSlides, have a look at `OpenSlides website
|
||||
<https://github.com/OpenSlides/OpenSlides/blob/master/DEVELOPMENT.rst>`_.
|
||||
|
||||
|
||||
Installation for big assemblies
|
||||
===============================
|
||||
|
||||
The installation steps described above install OpenSlides in a way, that does
|
||||
not support hundreds of concurrent clients. To install OpenSlides in for big
|
||||
assemblies, some config variables have to be changed in the OpenSlides settings
|
||||
usualy called settings.py.
|
||||
|
||||
The configuration values, that have to be altered are:
|
||||
|
||||
* CACHES
|
||||
* CHANNEL_LAYERS
|
||||
* DATABASES
|
||||
|
||||
Please see:
|
||||
* http://channels.readthedocs.io/en/latest/deploying.html
|
||||
* https://docs.djangoproject.com/en/1.9/topics/cache/
|
||||
* https://github.com/sebleier/django-redis-cache
|
||||
* https://docs.djangoproject.com/en/1.9/ref/settings/#databases
|
||||
|
||||
You should use a webserver like Apache HTTP Server or nginx to serve the static
|
||||
and media files and as proxy server in front of OpenSlides server. You also
|
||||
should use a database like PostgreSQL and Redis as channels backend and cache
|
||||
backend.
|
||||
|
||||
|
||||
Used software
|
||||
=============
|
||||
|
||||
@ -140,6 +165,8 @@ OpenSlides uses the following projects or parts of them:
|
||||
|
||||
* `html5lib <https://github.com/html5lib/html5lib-python>`_, License: MIT
|
||||
|
||||
* `Django Channels <https://github.com/andrewgodwin/channels/>`_, License: MIT
|
||||
|
||||
* `django-jsonfield <https://github.com/bradjasper/django-jsonfield/>`_,
|
||||
License: MIT
|
||||
|
||||
@ -157,12 +184,6 @@ OpenSlides uses the following projects or parts of them:
|
||||
|
||||
* `Six <http://pythonhosted.org/six/>`_, License: MIT
|
||||
|
||||
* `sockjs-tornado <https://github.com/mrjoes/sockjs-tornado>`_,
|
||||
License: MIT
|
||||
|
||||
* `Tornado <http://www.tornadoweb.org/en/stable/>`_, License: Apache
|
||||
License v2.0
|
||||
|
||||
* `Whoosh <https://bitbucket.org/mchaput/whoosh/wiki/Home>`_, License: BSD
|
||||
|
||||
* Several JavaScript packages (see ``bower.json``)
|
||||
@ -200,7 +221,6 @@ OpenSlides uses the following projects or parts of them:
|
||||
* `open-sans-fontface <https://github.com/FontFaceKit/open-sans>`_, License: Apache License version 2.0
|
||||
* `pdfjs-dist <http://mozilla.github.io/pdf.js/>`_, License: Apache-2.0
|
||||
* `roboto-condensed <https://github.com/davidcunningham/roboto-condensed>`_, License: Apache 2.0
|
||||
* `sockjs <https://github.com/sockjs/sockjs-client>`_, License: MIT
|
||||
* `tinymce <http://www.tinymce.com>`_, License: LGPL-2.1
|
||||
* `tinymce-i18n <https://github.com/OpenSlides/tinymce-i18n>`_, License: LGPL-2.1
|
||||
|
||||
|
||||
@ -31,7 +31,6 @@
|
||||
"ngBootbox": "~0.1.3",
|
||||
"open-sans-fontface": "https://github.com/OpenSlides/open-sans.git#1.4.2.post1",
|
||||
"roboto-condensed": "~0.3.0",
|
||||
"sockjs": "~0.3.4",
|
||||
"tinymce-i18n": "OpenSlides/tinymce-i18n#a186ad61e0aa30fdf657e88f405f966d790f0805"
|
||||
},
|
||||
"overrides": {
|
||||
|
||||
@ -185,7 +185,13 @@ def start(args):
|
||||
|
||||
# Start the webserver
|
||||
# Tell django not to reload. OpenSlides uses the reload method from tornado
|
||||
execute_from_command_line(['manage.py', 'runserver', '%s:%s' % (args.host, args.port), '--noreload'])
|
||||
# Use insecure to serve static files, even when DEBUG is False.
|
||||
execute_from_command_line([
|
||||
'manage.py',
|
||||
'runserver',
|
||||
'{}:{}'.format(args.host, args.port),
|
||||
'--noreload',
|
||||
'--insecure'])
|
||||
|
||||
|
||||
def createsettings(args):
|
||||
|
||||
@ -12,7 +12,7 @@ def listen_to_related_object_post_save(sender, instance, created, **kwargs):
|
||||
"""
|
||||
if created and hasattr(instance, 'get_agenda_title'):
|
||||
Item.objects.create(content_object=instance)
|
||||
inform_changed_data(False, instance)
|
||||
inform_changed_data(instance)
|
||||
|
||||
|
||||
def listen_to_related_object_post_delete(sender, instance, **kwargs):
|
||||
|
||||
9
openslides/asgi.py
Normal file
9
openslides/asgi.py
Normal file
@ -0,0 +1,9 @@
|
||||
from channels.asgi import get_channel_layer
|
||||
|
||||
from .utils.main import setup_django_settings_module
|
||||
|
||||
# Loads the openslides setting. You can use your own settings by setting the
|
||||
# environment variable DJANGO_SETTINGS_MODULE
|
||||
setup_django_settings_module()
|
||||
|
||||
channel_layer = get_channel_layer()
|
||||
@ -1,82 +0,0 @@
|
||||
import errno
|
||||
import os
|
||||
import socket
|
||||
import sys
|
||||
from datetime import datetime
|
||||
|
||||
from django.conf import settings
|
||||
from django.core.management.commands.runserver import Command as _Command
|
||||
from django.utils import six
|
||||
from django.utils.encoding import force_text, get_system_encoding
|
||||
|
||||
from openslides.utils.autoupdate import run_tornado
|
||||
|
||||
|
||||
class Command(_Command):
|
||||
"""
|
||||
Runserver command from django core, but starts the tornado webserver.
|
||||
|
||||
Only the line to run tornado has changed from the django default
|
||||
implementation.
|
||||
|
||||
The Code is from django 1.9
|
||||
"""
|
||||
help = 'Starts the Tornado webserver.'
|
||||
|
||||
# TODO: do not start tornado when the settings says so
|
||||
|
||||
def inner_run(self, *args, **options):
|
||||
# If an exception was silenced in ManagementUtility.execute in order
|
||||
# to be raised in the child process, raise it now.
|
||||
# OPENSLIDES: We do not use the django autoreload command
|
||||
# autoreload.raise_last_exception()
|
||||
|
||||
# OPENSLIDES: This line is not needed by tornado
|
||||
# threading = options.get('use_threading')
|
||||
shutdown_message = options.get('shutdown_message', '')
|
||||
quit_command = 'CTRL-BREAK' if sys.platform == 'win32' else 'CONTROL-C'
|
||||
|
||||
self.stdout.write("Performing system checks...\n\n")
|
||||
self.check(display_num_errors=True)
|
||||
self.check_migrations()
|
||||
now = datetime.now().strftime('%B %d, %Y - %X')
|
||||
if six.PY2:
|
||||
now = now.decode(get_system_encoding())
|
||||
self.stdout.write(now)
|
||||
self.stdout.write((
|
||||
"Django version %(version)s, using settings %(settings)r\n"
|
||||
"Starting development server at http://%(addr)s:%(port)s/\n"
|
||||
"Quit the server with %(quit_command)s.\n"
|
||||
) % {
|
||||
"version": self.get_version(),
|
||||
"settings": settings.SETTINGS_MODULE,
|
||||
"addr": '[%s]' % self.addr if self._raw_ipv6 else self.addr,
|
||||
"port": self.port,
|
||||
"quit_command": quit_command,
|
||||
})
|
||||
|
||||
try:
|
||||
handler = self.get_handler(*args, **options)
|
||||
run_tornado(
|
||||
self.addr,
|
||||
int(self.port),
|
||||
handler,
|
||||
ipv6=self.use_ipv6)
|
||||
except socket.error as e:
|
||||
# Use helpful error messages instead of ugly tracebacks.
|
||||
ERRORS = {
|
||||
errno.EACCES: "You don't have permission to access that port.",
|
||||
errno.EADDRINUSE: "That port is already in use.",
|
||||
errno.EADDRNOTAVAIL: "That IP address can't be assigned to.",
|
||||
}
|
||||
try:
|
||||
error_text = ERRORS[e.errno]
|
||||
except KeyError:
|
||||
error_text = force_text(e)
|
||||
self.stderr.write("Error: %s" % error_text)
|
||||
# Need to use an OS exit because sys.exit doesn't work in a thread
|
||||
os._exit(1)
|
||||
except KeyboardInterrupt:
|
||||
if shutdown_message:
|
||||
self.stdout.write(shutdown_message)
|
||||
sys.exit(0)
|
||||
35
openslides/core/migrations/0002_session.py
Normal file
35
openslides/core/migrations/0002_session.py
Normal file
@ -0,0 +1,35 @@
|
||||
import django.db.models.deletion
|
||||
from django.conf import settings
|
||||
from django.db import migrations, models
|
||||
|
||||
|
||||
class Migration(migrations.Migration):
|
||||
|
||||
dependencies = [
|
||||
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
|
||||
('sessions', '0001_initial'),
|
||||
('core', '0001_initial'),
|
||||
]
|
||||
|
||||
operations = [
|
||||
migrations.CreateModel(
|
||||
name='Session',
|
||||
fields=[
|
||||
('session_ptr',
|
||||
models.OneToOneField(
|
||||
auto_created=True,
|
||||
on_delete=django.db.models.deletion.CASCADE,
|
||||
parent_link=True,
|
||||
primary_key=True,
|
||||
serialize=False,
|
||||
to='sessions.Session')),
|
||||
('user', models.ForeignKey(null=True, on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
|
||||
],
|
||||
options={
|
||||
'verbose_name_plural': 'sessions',
|
||||
'abstract': False,
|
||||
'verbose_name': 'session',
|
||||
},
|
||||
bases=('sessions.session',),
|
||||
),
|
||||
]
|
||||
@ -1,5 +1,6 @@
|
||||
from django.conf import settings
|
||||
from django.contrib.contenttypes.models import ContentType
|
||||
from django.contrib.sessions.models import Session as DjangoSession
|
||||
from django.db import models
|
||||
from jsonfield import JSONField
|
||||
|
||||
@ -252,3 +253,19 @@ class ChatMessage(RESTModelMixin, models.Model):
|
||||
|
||||
def __str__(self):
|
||||
return 'Message {}'.format(self.timestamp)
|
||||
|
||||
|
||||
class Session(DjangoSession):
|
||||
"""
|
||||
Model like the Django db session, which saves the user as ForeignKey instead
|
||||
of an encoded value.
|
||||
"""
|
||||
user = models.ForeignKey(
|
||||
settings.AUTH_USER_MODEL,
|
||||
on_delete=models.CASCADE,
|
||||
null=True)
|
||||
|
||||
@classmethod
|
||||
def get_session_store_class(cls):
|
||||
from .session_backend import SessionStore
|
||||
return SessionStore
|
||||
|
||||
23
openslides/core/session_backend.py
Normal file
23
openslides/core/session_backend.py
Normal file
@ -0,0 +1,23 @@
|
||||
from django.contrib.sessions.backends.db import \
|
||||
SessionStore as DjangoSessionStore
|
||||
|
||||
|
||||
class SessionStore(DjangoSessionStore):
|
||||
"""
|
||||
Like the Django db Session store, but saves the user into the db field.
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def get_model_class(cls):
|
||||
# Avoids a circular import
|
||||
from .models import Session
|
||||
return Session
|
||||
|
||||
def create_model_instance(self, data):
|
||||
"""
|
||||
Set the user from data to the db field. Set to None, if its a session
|
||||
from an anonymous user.
|
||||
"""
|
||||
model = super().create_model_instance(data)
|
||||
model.user_id = data['_auth_user_id'] if '_auth_user_id' in data else None
|
||||
return model
|
||||
@ -41,7 +41,7 @@ angular.module('OpenSlidesApp.core', [
|
||||
function (DS, $rootScope) {
|
||||
var socket = null;
|
||||
var recInterval = null;
|
||||
$rootScope.connected = true;
|
||||
$rootScope.connected = false;
|
||||
|
||||
var Autoupdate = {
|
||||
messageReceivers: [],
|
||||
@ -55,7 +55,7 @@ angular.module('OpenSlidesApp.core', [
|
||||
}
|
||||
};
|
||||
var newConnect = function () {
|
||||
socket = new SockJS(location.origin + "/sockjs");
|
||||
socket = new WebSocket('ws://' + location.host + '/ws/');
|
||||
clearInterval(recInterval);
|
||||
socket.onopen = function () {
|
||||
$rootScope.connected = true;
|
||||
@ -168,22 +168,20 @@ angular.module('OpenSlidesApp.core', [
|
||||
'autoupdate',
|
||||
'dsEject',
|
||||
function (DS, autoupdate, dsEject) {
|
||||
autoupdate.onMessage(function(data) {
|
||||
autoupdate.onMessage(function(json) {
|
||||
// TODO: when MODEL.find() is called after this
|
||||
// a new request is fired. This could be a bug in DS
|
||||
|
||||
// TODO: Do not send the status code to the client, but make the decission
|
||||
// on the server side. It is an implementation detail, that tornado
|
||||
// sends request to wsgi, which should not concern the client.
|
||||
var data = JSON.parse(json);
|
||||
console.log("Received object: " + data.collection + ", " + data.id);
|
||||
var instance = DS.get(data.collection, data.id);
|
||||
if (data.status_code == 200) {
|
||||
if (data.action == 'changed') {
|
||||
if (instance) {
|
||||
// The instance is in the local db
|
||||
dsEject(data.collection, instance);
|
||||
}
|
||||
DS.inject(data.collection, data.data);
|
||||
} else if (data.status_code == 404) {
|
||||
} else if (data.action == 'deleted') {
|
||||
if (instance) {
|
||||
// The instance is in the local db
|
||||
dsEject(data.collection, instance);
|
||||
|
||||
@ -8,6 +8,9 @@ AUTH_USER_MODEL = 'users.User'
|
||||
|
||||
AUTHENTICATION_BACKENDS = ('openslides.users.auth.CustomizedModelBackend',)
|
||||
|
||||
# Uses a db session backend, that saves the user_id directly in the db
|
||||
SESSION_ENGINE = 'openslides.core.session_backend'
|
||||
|
||||
SESSION_COOKIE_NAME = 'OpenSlidesSessionID'
|
||||
|
||||
LANGUAGES = (
|
||||
@ -74,6 +77,7 @@ INSTALLED_APPS = (
|
||||
'django.contrib.staticfiles',
|
||||
'django.contrib.humanize',
|
||||
'rest_framework',
|
||||
'channels',
|
||||
'openslides.poll', # TODO: try to remove this line
|
||||
'openslides.agenda',
|
||||
'openslides.motions',
|
||||
@ -93,26 +97,9 @@ CACHES = {
|
||||
# See https://docs.djangoproject.com/en/1.5/ref/settings/#allowed-hosts
|
||||
ALLOWED_HOSTS = ['*']
|
||||
|
||||
# Use Haystack with Whoosh for full text search
|
||||
HAYSTACK_CONNECTIONS = {
|
||||
'default': {
|
||||
'ENGINE': 'haystack.backends.whoosh_backend.WhooshEngine'
|
||||
},
|
||||
}
|
||||
|
||||
# Haystack updates search index after each save/delete action by apps
|
||||
HAYSTACK_SIGNAL_PROCESSOR = 'openslides.utils.haystack_processor.OpenSlidesProcessor'
|
||||
|
||||
# Adds all automaticly collected plugins
|
||||
INSTALLED_PLUGINS = collect_plugins()
|
||||
|
||||
# Set this True to use tornado as single wsgi server. Set this False to use
|
||||
# other webserver like Apache or Nginx as wsgi server.
|
||||
USE_TORNADO_AS_WSGI_SERVER = True
|
||||
|
||||
OPENSLIDES_WSGI_NETWORK_LOCATION = ''
|
||||
|
||||
|
||||
TEST_RUNNER = 'openslides.utils.test.OpenSlidesDiscoverRunner'
|
||||
|
||||
# Config for the REST Framework
|
||||
@ -122,3 +109,11 @@ REST_FRAMEWORK = {
|
||||
'openslides.users.auth.RESTFrameworkAnonymousAuthentication',
|
||||
)
|
||||
}
|
||||
|
||||
# Config for channels
|
||||
CHANNEL_LAYERS = {
|
||||
'default': {
|
||||
'BACKEND': 'asgiref.inmemory.ChannelLayer',
|
||||
'ROUTING': 'openslides.routing.channel_routing',
|
||||
},
|
||||
}
|
||||
|
||||
9
openslides/routing.py
Normal file
9
openslides/routing.py
Normal file
@ -0,0 +1,9 @@
|
||||
from channels.routing import route
|
||||
|
||||
from openslides.utils.autoupdate import send_data, ws_add, ws_disconnect
|
||||
|
||||
channel_routing = [
|
||||
route("websocket.connect", ws_add, path='/ws/'),
|
||||
route("websocket.disconnect", ws_disconnect),
|
||||
route("autoupdate.send_data", send_data),
|
||||
]
|
||||
@ -1,211 +1,132 @@
|
||||
import itertools
|
||||
import json
|
||||
import os
|
||||
import posixpath
|
||||
from importlib import import_module
|
||||
from urllib.parse import unquote
|
||||
|
||||
from django.conf import settings
|
||||
from django.core.wsgi import get_wsgi_application
|
||||
from sockjs.tornado import SockJSConnection, SockJSRouter
|
||||
from tornado.httpserver import HTTPServer
|
||||
from tornado.ioloop import IOLoop
|
||||
from tornado.options import parse_command_line
|
||||
from tornado.web import (
|
||||
Application,
|
||||
FallbackHandler,
|
||||
HTTPError,
|
||||
StaticFileHandler,
|
||||
)
|
||||
from tornado.wsgi import WSGIContainer
|
||||
from asgiref.inmemory import ChannelLayer
|
||||
from channels import Channel, Group
|
||||
from channels.auth import channel_session_user, channel_session_user_from_http
|
||||
from django.apps import apps
|
||||
from django.utils import timezone
|
||||
|
||||
from ..users.auth import AnonymousUser, get_user
|
||||
from ..users.auth import AnonymousUser
|
||||
from ..users.models import User
|
||||
from .access_permissions import BaseAccessPermissions
|
||||
|
||||
RUNNING_HOST = None
|
||||
RUNNING_PORT = None
|
||||
|
||||
|
||||
class DjangoStaticFileHandler(StaticFileHandler):
|
||||
def get_logged_in_users():
|
||||
"""
|
||||
Handels static data by using the django finders.
|
||||
Helper to get all logged in users.
|
||||
|
||||
Only needed in the "small" version with tornado as wsgi server.
|
||||
Only works with the OpenSlides session backend.
|
||||
"""
|
||||
|
||||
def initialize(self):
|
||||
"""Overwrite some attributes."""
|
||||
# NOTE: root is never actually used and default_filename is not
|
||||
# supported (must always be None)
|
||||
self.root = ''
|
||||
self.default_filename = None
|
||||
|
||||
@classmethod
|
||||
def get_absolute_path(cls, root, path):
|
||||
from django.contrib.staticfiles import finders
|
||||
normalized_path = posixpath.normpath(unquote(path)).lstrip('/')
|
||||
absolute_path = finders.find(normalized_path)
|
||||
return absolute_path
|
||||
|
||||
def validate_absolute_path(self, root, absolute_path):
|
||||
# differences from base implementation:
|
||||
# - we ignore self.root since our files do not necessarily have
|
||||
# a shared root prefix
|
||||
# - we do not handle self.default_filename (we do not use it and it
|
||||
# does not make much sense here anyway)
|
||||
if absolute_path is None or not os.path.exists(absolute_path):
|
||||
raise HTTPError(404)
|
||||
if not os.path.isfile(absolute_path):
|
||||
raise HTTPError(403, 'The requested resource is not a file.')
|
||||
return absolute_path
|
||||
return User.objects.exclude(session=None).filter(session__expire_date__gte=timezone.now()).distinct()
|
||||
|
||||
|
||||
class OpenSlidesSockJSConnection(SockJSConnection):
|
||||
def get_model_from_collection_string(collection_string):
|
||||
"""
|
||||
SockJS connection for OpenSlides.
|
||||
Returns a model class which belongs to the argument collection_string.
|
||||
"""
|
||||
waiters = set()
|
||||
|
||||
def on_open(self, info):
|
||||
self.waiters.add(self)
|
||||
self.connection_info = info
|
||||
|
||||
def on_close(self):
|
||||
OpenSlidesSockJSConnection.waiters.remove(self)
|
||||
|
||||
@classmethod
|
||||
def send_object(cls, json_container):
|
||||
def model_generator():
|
||||
"""
|
||||
Sends an OpenSlides object to all connected clients (waiters).
|
||||
Yields all models of all apps.
|
||||
"""
|
||||
# Load JSON
|
||||
container = json.loads(json_container)
|
||||
for app_config in apps.get_app_configs():
|
||||
for model in app_config.get_models():
|
||||
yield model
|
||||
|
||||
# Search our AccessPermission class.
|
||||
for access_permissions in BaseAccessPermissions.get_all():
|
||||
if access_permissions.get_dispatch_uid() == container.get('dispatch_uid'):
|
||||
break
|
||||
for model in model_generator():
|
||||
try:
|
||||
model_collection_string = model.get_collection_string()
|
||||
except AttributeError:
|
||||
# Skip models which do not have the method get_collection_string.
|
||||
pass
|
||||
else:
|
||||
raise ValueError('Invalid container. A valid dispatch_uid is missing.')
|
||||
|
||||
# Loop over all waiters
|
||||
for waiter in cls.waiters:
|
||||
# Read waiter's former cookies and parse session cookie to get user instance.
|
||||
try:
|
||||
session_cookie = waiter.connection_info.cookies[settings.SESSION_COOKIE_NAME]
|
||||
except KeyError:
|
||||
# There is no session cookie so use anonymous user here.
|
||||
user = AnonymousUser()
|
||||
else:
|
||||
# Get session from session store and use it to retrieve the user.
|
||||
engine = import_module(settings.SESSION_ENGINE)
|
||||
session = engine.SessionStore(session_cookie.value)
|
||||
fake_request = type('FakeRequest', (), {})()
|
||||
fake_request.session = session
|
||||
user = get_user(fake_request)
|
||||
|
||||
# Two cases: models instance was changed or deleted
|
||||
if container.get('action') == 'changed':
|
||||
data = access_permissions.get_restricted_data(container.get('full_data'), user)
|
||||
if data is None:
|
||||
# There are no data for the user so he can't see the object. Skip him.
|
||||
break
|
||||
output = {
|
||||
'status_code': 200, # TODO: Refactor this. Use strings like 'change' or 'delete'.
|
||||
'collection': container['collection_string'],
|
||||
'id': container['rest_pk'],
|
||||
'data': data}
|
||||
elif container.get('action') == 'deleted':
|
||||
output = {
|
||||
'status_code': 404, # TODO: Refactor this. Use strings like 'change' or 'delete'.
|
||||
'collection': container['collection_string'],
|
||||
'id': container['rest_pk']}
|
||||
else:
|
||||
raise ValueError('Invalid container. A valid action is missing.')
|
||||
|
||||
# Send output to the waiter (client).
|
||||
waiter.send(output)
|
||||
if model_collection_string == collection_string:
|
||||
# The model was found.
|
||||
break
|
||||
else:
|
||||
# No model was found in all apps.
|
||||
raise ValueError('Invalid message. A valid collection_string is missing.')
|
||||
return model
|
||||
|
||||
|
||||
def run_tornado(addr, port, *args, **kwargs):
|
||||
# Connected to websocket.connect
|
||||
@channel_session_user_from_http
|
||||
def ws_add(message):
|
||||
"""
|
||||
Starts the tornado webserver as wsgi server for OpenSlides.
|
||||
Adds the websocket connection to a group specific to the connecting user.
|
||||
|
||||
It runs in one thread.
|
||||
The group with the name 'user-None' stands for all anonymous users.
|
||||
"""
|
||||
# Save the port and the addr in a global var
|
||||
global RUNNING_HOST, RUNNING_PORT
|
||||
RUNNING_HOST = addr
|
||||
RUNNING_PORT = port
|
||||
|
||||
# Don't try to read the command line args from openslides
|
||||
parse_command_line(args=[])
|
||||
|
||||
# Setup WSGIContainer
|
||||
app = WSGIContainer(get_wsgi_application())
|
||||
|
||||
# Collect urls
|
||||
sock_js_router = SockJSRouter(OpenSlidesSockJSConnection, '/sockjs')
|
||||
other_urls = [
|
||||
(r'%s(.*)' % settings.STATIC_URL, DjangoStaticFileHandler),
|
||||
(r'%s(.*)' % settings.MEDIA_URL, StaticFileHandler, {'path': settings.MEDIA_ROOT}),
|
||||
('.*', FallbackHandler, dict(fallback=app))]
|
||||
|
||||
# Start the application
|
||||
debug = settings.DEBUG
|
||||
tornado_app = Application(sock_js_router.urls + other_urls, autoreload=debug, debug=debug)
|
||||
server = HTTPServer(tornado_app)
|
||||
server.listen(port=port, address=addr)
|
||||
IOLoop.instance().start()
|
||||
|
||||
# Reset the global vars
|
||||
RUNNING_HOST = None
|
||||
RUNNING_PORT = None
|
||||
Group('user-{}'.format(message.user.id)).add(message.reply_channel)
|
||||
|
||||
|
||||
def inform_changed_data(is_delete, *args):
|
||||
# Connected to websocket.disconnect
|
||||
@channel_session_user
|
||||
def ws_disconnect(message):
|
||||
Group('user-{}'.format(message.user.id)).discard(message.reply_channel)
|
||||
|
||||
|
||||
def send_data(message):
|
||||
"""
|
||||
Informs all users about changed data.
|
||||
|
||||
The first argument is whether the object or the objects are deleted.
|
||||
The other arguments are the changed or deleted Django/OpenSlides model
|
||||
instances.
|
||||
The argument message has to be a dict with the keywords collection_string
|
||||
(string), pk (positive integer), id_deleted (boolean) and dispatch_uid
|
||||
(string).
|
||||
"""
|
||||
if settings.USE_TORNADO_AS_WSGI_SERVER:
|
||||
for instance in args:
|
||||
try:
|
||||
root_instance = instance.get_root_rest_element()
|
||||
except AttributeError:
|
||||
# Instance has no method get_root_rest_element. Just skip it.
|
||||
pass
|
||||
else:
|
||||
access_permissions = root_instance.get_access_permissions()
|
||||
container = {
|
||||
'dispatch_uid': access_permissions.get_dispatch_uid(),
|
||||
'collection_string': root_instance.get_collection_string(),
|
||||
'rest_pk': root_instance.get_rest_pk()}
|
||||
if is_delete and instance == root_instance:
|
||||
# A root instance is deleted.
|
||||
container['action'] = 'deleted'
|
||||
else:
|
||||
# A non root instance is deleted or any instance is just changed.
|
||||
container['action'] = 'changed'
|
||||
root_instance.refresh_from_db()
|
||||
container['full_data'] = access_permissions.get_full_data(root_instance)
|
||||
OpenSlidesSockJSConnection.send_object(json.dumps(container))
|
||||
for access_permissions in BaseAccessPermissions.get_all():
|
||||
if access_permissions.get_dispatch_uid() == message['dispatch_uid']:
|
||||
break
|
||||
else:
|
||||
raise ValueError('Invalid message. A valid dispatch_uid is missing.')
|
||||
|
||||
if not message['is_deleted']:
|
||||
Model = get_model_from_collection_string(message['collection_string'])
|
||||
instance = Model.objects.get(pk=message['pk'])
|
||||
full_data = access_permissions.get_full_data(instance)
|
||||
|
||||
# Loop over all logged in users and the anonymous user.
|
||||
for user in itertools.chain(get_logged_in_users(), [AnonymousUser()]):
|
||||
channel = Group('user-{}'.format(user.id))
|
||||
output = {
|
||||
'collection': message['collection_string'],
|
||||
'id': instance.get_rest_pk(),
|
||||
'action': 'deleted' if message['is_deleted'] else 'changed'}
|
||||
if not message['is_deleted']:
|
||||
data = access_permissions.get_restricted_data(full_data, user)
|
||||
if data is None:
|
||||
# There are no data for the user so he can't see the object. Skip him.
|
||||
continue
|
||||
output['data'] = data
|
||||
channel.send({'text': json.dumps(output)})
|
||||
|
||||
|
||||
def inform_changed_data(instance, is_deleted=False):
|
||||
try:
|
||||
root_instance = instance.get_root_rest_element()
|
||||
except AttributeError:
|
||||
# Instance has no method get_root_rest_element. Just ignore it.
|
||||
pass
|
||||
# TODO: Implement big variant with Apache or Nginx as WSGI webserver.
|
||||
else:
|
||||
try:
|
||||
Channel('autoupdate.send_data').send({
|
||||
'collection_string': root_instance.get_collection_string(),
|
||||
'pk': root_instance.pk,
|
||||
'is_deleted': is_deleted and instance == root_instance,
|
||||
'dispatch_uid': root_instance.get_access_permissions().get_dispatch_uid()})
|
||||
except ChannelLayer.ChannelFull:
|
||||
pass
|
||||
|
||||
|
||||
def inform_changed_data_receiver(sender, instance, **kwargs):
|
||||
"""
|
||||
Receiver for the inform_changed_data function to use in a signal.
|
||||
"""
|
||||
inform_changed_data(False, instance)
|
||||
inform_changed_data(instance)
|
||||
|
||||
|
||||
def inform_deleted_data_receiver(sender, instance, **kwargs):
|
||||
"""
|
||||
Receiver for the inform_changed_data function to use in a signal.
|
||||
"""
|
||||
inform_changed_data(True, instance)
|
||||
inform_changed_data(instance, is_deleted=True)
|
||||
|
||||
@ -2,10 +2,10 @@
|
||||
Settings file for OpenSlides
|
||||
|
||||
For more information on this file, see
|
||||
https://docs.djangoproject.com/en/1.7/topics/settings/
|
||||
https://docs.djangoproject.com/en/1.9/topics/settings/
|
||||
|
||||
For the full list of settings and their values, see
|
||||
https://docs.djangoproject.com/en/1.7/ref/settings/
|
||||
https://docs.djangoproject.com/en/1.9/ref/settings/
|
||||
"""
|
||||
|
||||
import os
|
||||
@ -26,7 +26,6 @@ SECRET_KEY = %(secret_key)r
|
||||
# SECURITY WARNING: Don't run with debug turned on in production!
|
||||
|
||||
DEBUG = %(debug)s
|
||||
TEMPLATE_DEBUG = DEBUG
|
||||
|
||||
|
||||
# OpenSlides plugins
|
||||
@ -41,7 +40,7 @@ INSTALLED_APPS += INSTALLED_PLUGINS
|
||||
|
||||
# Database
|
||||
# Change this to use MySQL or PostgreSQL.
|
||||
# See https://docs.djangoproject.com/en/1.7/ref/settings/#databases
|
||||
# See https://docs.djangoproject.com/en/1.9/ref/settings/#databases
|
||||
|
||||
DATABASES = {
|
||||
'default': {
|
||||
@ -51,6 +50,25 @@ DATABASES = {
|
||||
}
|
||||
|
||||
|
||||
# Big Mode
|
||||
# Uncomment the following lines to activate redis as channel and cache backend.
|
||||
# You have to install a redis server and the python packages asgi_redis and
|
||||
# django-redis for this to work.
|
||||
# See https://channels.readthedocs.io/en/latest/backends.html#redis
|
||||
# https://niwinz.github.io/django-redis/latest/#_user_guide
|
||||
|
||||
# CHANNEL_LAYERS['default']['BACKEND'] = 'asgi_redis.RedisChannelLayer'
|
||||
# CACHES = {
|
||||
# "default": {
|
||||
# "BACKEND": "django_redis.cache.RedisCache",
|
||||
# "LOCATION": "redis://127.0.0.1:6379/1",
|
||||
# "OPTIONS": {
|
||||
# "CLIENT_CLASS": "django_redis.client.DefaultClient",
|
||||
# }
|
||||
# }
|
||||
# }
|
||||
|
||||
|
||||
# Some other settings
|
||||
|
||||
TIME_ZONE = 'Europe/Berlin'
|
||||
@ -63,5 +81,4 @@ TEMPLATE_DIRS = (
|
||||
|
||||
STATICFILES_DIRS = [os.path.join(OPENSLIDES_USER_DATA_PATH, 'static')] + STATICFILES_DIRS
|
||||
|
||||
|
||||
SEARCH_INDEX = os.path.join(OPENSLIDES_USER_DATA_PATH, 'search_index')
|
||||
|
||||
@ -15,6 +15,7 @@
|
||||
"gulp-jshint": "~2.0.0",
|
||||
"gulp-rename": "~1.2.2",
|
||||
"gulp-uglify": "~1.5.2",
|
||||
"jshint": "~2.9.2",
|
||||
"main-bower-files": "~2.11.1",
|
||||
"po2json": "~0.4.1",
|
||||
"sprintf-js": "~1.0.3",
|
||||
|
||||
@ -1,6 +1,7 @@
|
||||
# Requirements for OpenSlides in production in alphabetical order
|
||||
Django>=1.8,<1.10
|
||||
beautifulsoup4>=4.4,<4.5
|
||||
channels>=0.14,<0.15
|
||||
djangorestframework>=3.2.0,<3.4.0
|
||||
html5lib>=0.9,<1.0
|
||||
jsonfield>=0.9.19,<1.1
|
||||
|
||||
@ -1,18 +1,15 @@
|
||||
import json
|
||||
from unittest.mock import patch
|
||||
|
||||
from django.core.urlresolvers import reverse
|
||||
from rest_framework import status
|
||||
from rest_framework.test import APIClient
|
||||
|
||||
from openslides import __version__ as version
|
||||
from openslides.core.config import ConfigHandler, ConfigVariable
|
||||
from openslides.core.config import ConfigVariable, config
|
||||
from openslides.core.models import CustomSlide, Projector
|
||||
from openslides.utils.rest_api import ValidationError
|
||||
from openslides.utils.test import TestCase
|
||||
|
||||
config = ConfigHandler()
|
||||
|
||||
|
||||
class ProjectorAPI(TestCase):
|
||||
"""
|
||||
@ -75,18 +72,19 @@ class VersionView(TestCase):
|
||||
'version': 'unknown'}]})
|
||||
|
||||
|
||||
@patch('openslides.core.config.config', config)
|
||||
@patch('openslides.core.views.config', config)
|
||||
class ConfigViewSet(TestCase):
|
||||
"""
|
||||
Tests requests to deal with config variables.
|
||||
"""
|
||||
def setUp(self):
|
||||
# Save the old value of the config object and add the test values
|
||||
# TODO: Can be changed to setUpClass when Django 1.8 is no longer supported
|
||||
self._config_values = config.config_variables.copy()
|
||||
config.update_config_variables(set_simple_config_view_integration_config_test())
|
||||
|
||||
def tearDown(self):
|
||||
# Reset the config variables
|
||||
config.config_variables = {}
|
||||
config.config_variables = self._config_values
|
||||
|
||||
def test_retrieve(self):
|
||||
self.client.login(username='admin', password='admin')
|
||||
|
||||
0
tests/integration/utils/__init__.py
Normal file
0
tests/integration/utils/__init__.py
Normal file
46
tests/integration/utils/autoupdate.py
Normal file
46
tests/integration/utils/autoupdate.py
Normal file
@ -0,0 +1,46 @@
|
||||
from datetime import timedelta
|
||||
|
||||
from django.utils import timezone
|
||||
|
||||
from openslides.core.models import Session
|
||||
from openslides.users.models import User
|
||||
from openslides.utils.autoupdate import get_logged_in_users
|
||||
from openslides.utils.test import TestCase
|
||||
|
||||
|
||||
class TestGetLoggedInUsers(TestCase):
|
||||
def test_call(self):
|
||||
"""
|
||||
Test to call the function with:
|
||||
* A user that session has not expired
|
||||
* A user that session has expired
|
||||
* A user that has no session
|
||||
* An anonymous user that session hot not expired
|
||||
|
||||
Only the user with the session that has not expired should be returned
|
||||
"""
|
||||
user1 = User.objects.create(username='user1')
|
||||
user2 = User.objects.create(username='user2')
|
||||
User.objects.create(username='user3')
|
||||
|
||||
# Create a session with a user, that expires in 5 hours
|
||||
Session.objects.create(user=user1, expire_data=timezone.now() + timedelta(hours=5))
|
||||
|
||||
# Create a session with a user, that is expired before 5 hours
|
||||
Session.objects.create(user=user2, expire_data=timezone.now() + timedelta(hours=-5))
|
||||
|
||||
# Create a session with an anonymous user, that expires in 5 hours
|
||||
Session.objects.create(user=None, expire_data=timezone.now() + timedelta(hours=5))
|
||||
|
||||
self.assertEqual(list(get_logged_in_users()), [user1])
|
||||
|
||||
def test_unique(self):
|
||||
"""
|
||||
Test the function with a user that has two not expired session.
|
||||
The user should be returned only once.
|
||||
"""
|
||||
user1 = User.objects.create(username='user1')
|
||||
Session.objects.create(user=user1, expire_data=timezone.now() + timedelta(hours=1))
|
||||
Session.objects.create(user=user1, expire_data=timezone.now() + timedelta(hours=2))
|
||||
|
||||
self.assertEqual(list(get_logged_in_users()), [user1])
|
||||
@ -1,6 +1,4 @@
|
||||
from unittest.mock import patch
|
||||
|
||||
from openslides.core.config import ConfigHandler, ConfigVariable
|
||||
from openslides.core.config import ConfigVariable, config
|
||||
from openslides.core.exceptions import ConfigError, ConfigNotFound
|
||||
from openslides.utils.test import TestCase
|
||||
|
||||
@ -8,12 +6,12 @@ from openslides.utils.test import TestCase
|
||||
class TestConfigException(Exception):
|
||||
pass
|
||||
|
||||
config = ConfigHandler()
|
||||
|
||||
|
||||
@patch('openslides.core.config.config', config)
|
||||
class HandleConfigTest(TestCase):
|
||||
def setUp(self):
|
||||
# Save the old value of the config object and add the test values
|
||||
# TODO: Can be changed to setUpClass when Django 1.8 is no longer supported
|
||||
self._config_values = config.config_variables.copy()
|
||||
config.update_config_variables(set_grouped_config_view())
|
||||
config.update_config_variables(set_simple_config_view())
|
||||
config.update_config_variables(set_simple_config_view_multiple_vars())
|
||||
@ -22,7 +20,7 @@ class HandleConfigTest(TestCase):
|
||||
|
||||
def tearDown(self):
|
||||
# Reset the config variables
|
||||
config.config_variables = {}
|
||||
config.config_variables = self._config_values
|
||||
|
||||
def get_config_var(self, key):
|
||||
return config[key]
|
||||
|
||||
Loading…
Reference in New Issue
Block a user