Difference between revisions of "Work-Adventure/install"
(→Parts) |
(moved systemd link from TODO to the main section) |
||
(12 intermediate revisions by 2 users not shown) | |||
Line 17: | Line 17: | ||
This section lists the parts as you find them on their respective GITHUB repos. | This section lists the parts as you find them on their respective GITHUB repos. | ||
− | + | '''''IMPORTANT''''': | |
It is important to realize that this installation-guide replaces the default docker-compose.yaml files shipped with the projects and introduces a third one, enabling splitting off 'traefik' and 'coturn' from Work-adventure, making it easier to re-use these and manage them seperately from either Jitsi and/or Work-Adventure. | It is important to realize that this installation-guide replaces the default docker-compose.yaml files shipped with the projects and introduces a third one, enabling splitting off 'traefik' and 'coturn' from Work-adventure, making it easier to re-use these and manage them seperately from either Jitsi and/or Work-Adventure. | ||
Line 54: | Line 54: | ||
If you are familiar with Traefik and docker, you can likely skip all this. If it is your first run-in with either, this chapter is likely essential to make any sense of what is going on in the install-instructions further on; though they will likely work well enough, regardless. | If you are familiar with Traefik and docker, you can likely skip all this. If it is your first run-in with either, this chapter is likely essential to make any sense of what is going on in the install-instructions further on; though they will likely work well enough, regardless. | ||
− | + | Docker is an easy way to deploy projects in the form of 'microservices'; little pieces of application functionality, handily packaged in containerized piece, easily assembled and made to work. Great. The issue is that if you want to internet-traffick to all these pieces you quickly run into the need to put a reverse-proxy service in between 'the outside world' and the docker-containers. | |
+ | |||
+ | You may want to: | ||
+ | * Route http-requests for different host/domains to different containers | ||
+ | * Have one site with multiple parts served under seperate paths, requiring routing to different containers depending on just the path. | ||
+ | * Have some way of handling SSL-requests without having to put certificates into containers | ||
+ | * Have some way of doing ACME-requests for LetsEncrypt | ||
+ | * Provide an easy way to view , monitor and debug routing | ||
+ | * Have some way of keeping the required ROUTING-CONFIG together WITH THE CONTAINER so that starting/stopping services will adjust routing | ||
+ | |||
+ | For the first four things, an NGINX or even an Apache setup would still be able to help you out. The last point'd already be a lot more work to properly take care of, if possible at all. | ||
+ | |||
+ | For this reason, and some others not even mentioned yet, a lot of serious containerization-deployments end up using Traefik; a piece of reverse-proxy software that: | ||
+ | |||
+ | * Is able to attach to a containerization-system like Docker, Kubernetes, etc, and receive config-data dynamically as containers are spun up or down | ||
+ | * Is configurable via 'labels' , supplied in the docker-compose.yaml file for a project, making it possible to have routing-config together with project-config | ||
+ | * Has built-in HTTP→HTTPS rewriting middleware available | ||
+ | * Has built-in ACME functionality for LetsEncrypt, as well as other CertificateResolvers for other SSL-services | ||
+ | * Is able to dynamically request SSL-certs whenever a new hostname is referred to in a docker-compose.yaml file of a spun-up project. | ||
+ | * Has a dashboard available to view and debug routing paths | ||
+ | * Can also be used for simple 'port-forwarding' , also of UDP traffic | ||
+ | * Has built-in exporters for external telemetry tools like Prometheus | ||
+ | * '''Can, itself, be deployed as a docker-container and be configured through a docker-compose.yaml file''' | ||
+ | |||
+ | Keep the points above in mind when going through the install-instructions for the 'traefik-infra' project, below. it is beyond the scope of this guide to explain all the options provided there, but a lot of it might be self-evident , or easily understood with a short look in the Traefik documentation. | ||
== Installation instructions== | == Installation instructions== | ||
+ | |||
+ | The installation will consist of: | ||
+ | * Setting up your docker environment and your network/dns setup | ||
+ | * Setting up Traefik (and coturn) seperately from jitsi/workadventure | ||
+ | * Setting up WorkAdventure | ||
+ | * Setting up Jitsi | ||
===Pre-requisites and config=== | ===Pre-requisites and config=== | ||
Line 178: | Line 208: | ||
CERT_LOCATION=/etc/traefik/acme | CERT_LOCATION=/etc/traefik/acme | ||
</nowiki> | </nowiki> | ||
− | + | '''''IMPORTANT:''''' | |
The last thing to do is to tell Docker that we want it to create a network-bridge called <code>web</code> that we have referred to in our config. | The last thing to do is to tell Docker that we want it to create a network-bridge called <code>web</code> that we have referred to in our config. | ||
− | <code> # docker create | + | <code> # docker network create web</code> |
Awesome. At this point, if all is well, you should be able to start the traefik-container using the command below, and you should be able to find the Traefik dashboard on admin.your.domain.tld | Awesome. At this point, if all is well, you should be able to start the traefik-container using the command below, and you should be able to find the Traefik dashboard on admin.your.domain.tld | ||
Line 187: | Line 217: | ||
<nowiki> | <nowiki> | ||
# docker-compose up | # docker-compose up | ||
− | <nowiki> | + | </nowiki> |
If all went well, the only thing left to do is to create a password for the an admin-user. Edit 'password' in the following command and cut-n-paste the output to the docker-compose.yml file at the obvious spot provided; currently commented out. | If all went well, the only thing left to do is to create a password for the an admin-user. Edit 'password' in the following command and cut-n-paste the output to the docker-compose.yml file at the obvious spot provided; currently commented out. | ||
Line 222: | Line 252: | ||
DEBUG_MODE=false | DEBUG_MODE=false | ||
# Use jitsi.meet if you want to test out workadventure jitsi integration before you have managed to set up your own jitsi | # Use jitsi.meet if you want to test out workadventure jitsi integration before you have managed to set up your own jitsi | ||
− | JITSI_URL=meet.your.domain.tld | + | # For now we use public meet.jit.si , reverse commenting out lines below when you have your own jitsi up |
− | + | #JITSI_URL=meet.your.domain.tld | |
+ | JITSI_URL=meet.jit.si | ||
# If your Jitsi environment has authentication set up, you MUST set JITSI_PRIVATE_MODE to "true" and you MUST pass a SECRET_JITSI_KEY to generate the JWT secret | # If your Jitsi environment has authentication set up, you MUST set JITSI_PRIVATE_MODE to "true" and you MUST pass a SECRET_JITSI_KEY to generate the JWT secret | ||
JITSI_PRIVATE_MODE=false | JITSI_PRIVATE_MODE=false | ||
Line 684: | Line 715: | ||
# docker-compose up | # docker-compose up | ||
</nowiki> | </nowiki> | ||
+ | |||
+ | '''''Note''''': | ||
+ | If this works, you can now go back to the Work-adventure setup and edit the JITSI_URL parameter to point it to your <code>meet.your.domain.tld</code> and test if things still work the way you want them to. | ||
+ | |||
+ | == Run docker-compose as systemd units == | ||
+ | |||
+ | To have automatic startups of traefik, jitsi and wa, run them via Systemd → [[Work-Adventure/install/Add docker-compose as systemd unit|Add docker-compose as systemd unit]]. | ||
==How to manage maps== | ==How to manage maps== | ||
==Other things== | ==Other things== | ||
==TODO points== | ==TODO points== | ||
− | |||
* Investigate logging: Logspout | * Investigate logging: Logspout | ||
* Fix CORS issues with linking to external web-pages in map (requires extra Apache config in 'wa-front' container | * Fix CORS issues with linking to external web-pages in map (requires extra Apache config in 'wa-front' container | ||
Line 703: | Line 740: | ||
==Reference material== | ==Reference material== | ||
* [[https://github.com/thecodingmachine/workadventure/issues/738 Work-adventure github issue introducing better/easier configuration]] | * [[https://github.com/thecodingmachine/workadventure/issues/738 Work-adventure github issue introducing better/easier configuration]] | ||
+ | * [[https://bloggeek.me/webrtc-turn/ A lot of info on TURN/STUN]] What it does, how it works, how to test it. |
Latest revision as of 03:05, 20 March 2021
Contents
Introduction
Since early 2021, Techinc hosts an 'instance' of Work-Adventure on play.social.techinc.nl.
[Work-Adventure] can be described as a spatially-enabled meet/chat space in an environment that is akin to 16-bit RPG games from the 90's. It allows text-based messaging, has direct web-rtc enabled video/audio chats available for up to 4 people in eachother's vincinity, as well as providign mechanisms to have 'rooms' of more participants by leveraging a [JITSI] server of choice.
The software has been designed by [The Coding Machine] and is released on their [github repo] as open source. The release is fully functional, apart from lacking an authentication/management backend that would allow restricting access, moderation, administrator-rights, etc. The API is documented however, and creation of a custom backend seems relatively easy to do.
By default, installing the github-provided release is most easily done via their included 'docker-compose.yaml', as listed on the README.MD file in the repo. While the mechanism works well, it lacks instructions on how to configure and run the software in a publically-accessible context instead of just on 'workadventure.localhost' as the README.MD explains you how to do.
The challenge for newcomers to docker comes with the way that the software is composed from multiple moving parts that are all required to be started alongside eachother. This is true of Work-Adventure itself which starts 4 service-containers all on its own; but becomes even more clear when you also want to host the JITSI-instance locally (instead of relying on meet.jit.si) and you want to use ssl-certs to make things work properly, as webrtc and CORS is rather picky about non-https content loaded into https-enabled sites.
All of the above can make an installation of work-adventure (with, or without Jitsi locally) a challenging endevour. This guide hopes to give some aid in understanding and configuring the infra.
Parts
This section lists the parts as you find them on their respective GITHUB repos.
IMPORTANT: It is important to realize that this installation-guide replaces the default docker-compose.yaml files shipped with the projects and introduces a third one, enabling splitting off 'traefik' and 'coturn' from Work-adventure, making it easier to re-use these and manage them seperately from either Jitsi and/or Work-Adventure.
Work-adventure
Work-adventure is found [on github] as a project deployed through 'docker-compose'. You do not need to know exactly the role of each container to get it to work with the install-guide supplied later, but it is handy to understand 'reverse-proxy', 'front' and 'maps', mainly. There is a '.env.template' file that should be copied to '.env' and edited for settings. (more about this later).
By default, the docker-compose file provisions the following containers:
- reverse-proxy: a 'traefik:v2.0' container from the public docker repo's. This is an auto-configuring proxy-service that can be crucial in getting things to run smoothly. Understanding what it does can be crucial to getting work-adventure and jitsi to work. This guide contains info about this, later.
- front: based on a 'node-js' image from TheCodingMachine docker-repo. Runs code for the interface part of the game. Has sprites for characters, login-screens, menu's, etc.
- pusher: The pusher component is in charge of accepting WebSocket and HTTP connections from the front and forwarding them to the correct "api" server (or to the "admin").
- maps: Serves the 'map' files via an apache server. No active components
- back: This provides the api.your.domain.tld endpoint, handling most of the user-interaction
- uploader: The uploader component is in charge of accepting incoming files that can be downloaded by other users. It is currently used by administrators of maps to send sounds/recordings to everyone on a map. (not sure if this is implemented/used/essential)
- messages: A container without a need of being reachable from the outside world; internally used to distribute/sync ProtoBuf messaging format between instances (i think)
- coturn: provides a TURN-server to allow NAT-traversal for those behind restricted/crippled networks. (not enabled yet on techinc infra; we use a public one)
JITSI
Jitsi is easily installed using [their github-repo for a docker-compose install]. There is an 'env.example' file that should be copied to .env and edited (more about this later).
Their docker-compose file provides the following containers:
- web: The frontend-service that clients talk to
- prosody: The XMPP service that carries text-messages and internal system-messages between parts.
- jicofo: Jitsi COnference FOcus; a part responsible in organizing video-connections between participants and the JitsiVideoBridge (jvb)
- jvb: The Jitsi Video Bridge server serves as a router for video-data; reducing bandwidth-load on participants.
Of note is that the jitsi-setup does not provide a config for use with Traefik by default, though there are some examples provided in 'examples' for traefik and kubernetes. Inspiration from these have been used to create a working docker-compose file using traefik v2.0 in a way that works well with Work-adventure
Understanding Docker and Traefik
If you are familiar with Traefik and docker, you can likely skip all this. If it is your first run-in with either, this chapter is likely essential to make any sense of what is going on in the install-instructions further on; though they will likely work well enough, regardless.
Docker is an easy way to deploy projects in the form of 'microservices'; little pieces of application functionality, handily packaged in containerized piece, easily assembled and made to work. Great. The issue is that if you want to internet-traffick to all these pieces you quickly run into the need to put a reverse-proxy service in between 'the outside world' and the docker-containers.
You may want to:
- Route http-requests for different host/domains to different containers
- Have one site with multiple parts served under seperate paths, requiring routing to different containers depending on just the path.
- Have some way of handling SSL-requests without having to put certificates into containers
- Have some way of doing ACME-requests for LetsEncrypt
- Provide an easy way to view , monitor and debug routing
- Have some way of keeping the required ROUTING-CONFIG together WITH THE CONTAINER so that starting/stopping services will adjust routing
For the first four things, an NGINX or even an Apache setup would still be able to help you out. The last point'd already be a lot more work to properly take care of, if possible at all.
For this reason, and some others not even mentioned yet, a lot of serious containerization-deployments end up using Traefik; a piece of reverse-proxy software that:
- Is able to attach to a containerization-system like Docker, Kubernetes, etc, and receive config-data dynamically as containers are spun up or down
- Is configurable via 'labels' , supplied in the docker-compose.yaml file for a project, making it possible to have routing-config together with project-config
- Has built-in HTTP→HTTPS rewriting middleware available
- Has built-in ACME functionality for LetsEncrypt, as well as other CertificateResolvers for other SSL-services
- Is able to dynamically request SSL-certs whenever a new hostname is referred to in a docker-compose.yaml file of a spun-up project.
- Has a dashboard available to view and debug routing paths
- Can also be used for simple 'port-forwarding' , also of UDP traffic
- Has built-in exporters for external telemetry tools like Prometheus
- Can, itself, be deployed as a docker-container and be configured through a docker-compose.yaml file
Keep the points above in mind when going through the install-instructions for the 'traefik-infra' project, below. it is beyond the scope of this guide to explain all the options provided there, but a lot of it might be self-evident , or easily understood with a short look in the Traefik documentation.
Installation instructions
The installation will consist of:
- Setting up your docker environment and your network/dns setup
- Setting up Traefik (and coturn) seperately from jitsi/workadventure
- Setting up WorkAdventure
- Setting up Jitsi
Pre-requisites and config
The setup instructions are written with the following assumptions in mind:
- you have a domain-name you have control over
- You have publically reachable IPv4 space (usage over IPV6 not tested)
- You have a VM or physical hardware you want to dedicate to running jitsi+WA infra
It is likely other scenarios can be made to work, like hosting it on a DSL-line at home, with proper port-forwarding on your modem, or wanting to run the infra on a machine that also runs web-based services outside of a docker-context. These are definately possible, but are hard to cater for in this guide. Understanding the parts and how they go together will enable you to get things to work , however.
First steps:
- Set a wildcard DNS-record for *.whatever.your.domain.is to the publically reachable IP of the machine you want to host it on. Using a CNAME record works fine. It might be good to set TTL to 5mins at first
- Create a directory for all the parts we'll be installing (This guide assumes /opt for all of this)
- Install docker and docker-compose with
apt-get install docker docker-compose
if under a debian-like OS
Running Traefik and configuring docker-networking
By default, a container running traefik is started through the docker-compose.yaml file included with the work-adventure repo.
It is suggested to run any infra-structure that is shared between Jitsi and Work-adventure separately from either of them.
Currently the obvious candidates are:
- reverse-proxy: traefik, also handles LetsEncrypt
- coturn: a TURN-server for NAT-breaking, primarily used by Jitsi, but possibly also WorkAdventure (or other apps you might want to run on your instance)
We'll create a /opt/traefik-infra directory for this purpose
# mkdir /opt/traefik-infra
Now go into the traefik-infra directory and create 'docker-compose.yaml' file with the following contents:
version: '3' services: reverse-proxy: image: traefik:v2.3 command: - --log.level=WARN #- --api.insecure=true - --api.dashboard=true - --providers.docker.network=web - --providers.docker - --entryPoints.web.address=:80 - --entrypoints.web.http.redirections.entryPoint.to=websecure - --entrypoints.web.http.redirections.entryPoint.scheme=https - --entryPoints.websecure.address=:443 - --entryPoints.video.address=:10000/udp - --certificatesresolvers.le.acme.email=${ACME_EMAIL} - --certificatesresolvers.le.acme.storage=/etc/traefik/acme/acme.json # used during the challenge - --certificatesresolvers.le.acme.httpchallenge.entrypoint=web ports: - "80:80" - "443:443" # The Web UI (enabled by --api.insecure=true) - "8080:8080" volumes: - /var/run/docker.sock:/var/run/docker.sock - "${CERT_LOCATION}:/etc/traefik/acme" restart: unless-stopped labels: - "traefik.enable=true" - "traefik.http.services.traefik.loadbalancer.server.port=888" - "traefik.http.routers.traefik.rule=Host(`admin.${DOMAIN}`)" - "traefik.http.routers.traefik.entrypoints=websecure" - "traefik.http.routers.traefik.tls.certresolver=le" - "traefik.http.routers.traefik.service=api@internal" - "traefik.http.routers.traefik.middlewares=traefik-auth" # You should modify the line below to set a password for your Traefik-dashboard #- "traefik.http.middlewares.traefik-auth.basicauth.users=admin:" networks: web: meet.jitsi: # Still working on a useful COTURN setup. Included for later use. # coturn: # image: coturn/coturn:4.5.2 # command: # - turnserver # - --log-file=stdout # - --external-ip=$$(detect-external-ip) # - --listening-port=3478 # - --min-port=10000 # - --max-port=10010 # - --tls-listening-port=5349 # - --listening-ip=0.0.0.0 # - --realm=coturn.${DOMAIN} # - --server-name=coturn.${DOMAIN} # - --lt-cred-mech # # Enable Coturn "REST API" to validate temporary passwords. # # - --use-auth-secret # # - --static-auth-secret=SomeStaticAuthSecret # # - --userdb=/var/lib/turn/turndb # - --user=workadventure:WorkAdventure123 # # use real-valid certificate/privatekey files # # - --cert=/root/letsencrypt/fullchain.pem # # - --pkey=/root/letsencrypt/privkey.pem # network_mode: host networks: meet.jitsi: web: external: true
Create the file .env
with the following contents:
DOMAIN=some.domain.tld # These are not used yet TURN_SERVER= TURN_USER= TURN_PASSWORD= # The email address used by Let's encrypt to send renewal warnings (compulsory) ACME_EMAIL=my_letsencrypt_contact@email.address # The directory on your OS that Traefik will store all ACME-produced certs in CERT_LOCATION=/etc/traefik/acme
IMPORTANT:
The last thing to do is to tell Docker that we want it to create a network-bridge called web
that we have referred to in our config.
# docker network create web
Awesome. At this point, if all is well, you should be able to start the traefik-container using the command below, and you should be able to find the Traefik dashboard on admin.your.domain.tld
# docker-compose up
If all went well, the only thing left to do is to create a password for the an admin-user. Edit 'password' in the following command and cut-n-paste the output to the docker-compose.yml file at the obvious spot provided; currently commented out.
echo $(htpasswd -nb admin password) | sed -e s/\\$/\\$\\$/g
There are also ways of using password-files or other mechanisms to provide authentication for the Traefik-dashboard (or other places). This is left as an excersize for the reader. Good luck.
Once Traefik is up-and-running like this, move to setting up WorkAdventure
Work-Adventure
Installing work-adventure is as simple as:
- Pulling the [workadventure github repo]
- Copying the .env.example file to .env and editing it (changing settings and adding some new ones)
- Replacing the docker-compose.yaml file by an edited with the following changes:
- Has understanding for the '$DOMAIN' environment variable
- Has 'traefik' removed from it (we are running one already)
- Has 'coturn' remove from it (we are running one already)
First, get the github repo:
# cd /opt # https://github.com/thecodingmachine/workadventure.git
Now we create a /opt/workadventure/.env
file, supporting the DOMAIN variable that allows setting all the host-names properly in one go. Be sure to edit the 'your.domain.tld' part in all three places.
#The base domain DOMAIN=your.domain.tld DEBUG_MODE=false # Use jitsi.meet if you want to test out workadventure jitsi integration before you have managed to set up your own jitsi # For now we use public meet.jit.si , reverse commenting out lines below when you have your own jitsi up #JITSI_URL=meet.your.domain.tld JITSI_URL=meet.jit.si # If your Jitsi environment has authentication set up, you MUST set JITSI_PRIVATE_MODE to "true" and you MUST pass a SECRET_JITSI_KEY to generate the JWT secret JITSI_PRIVATE_MODE=false JITSI_ISS= SECRET_JITSI_KEY= # The URL used by default, in the form: "/_/global/map/url.json" START_ROOM_URL=/_/global/maps.your.domain.tld/Floor0/floor0.json # These are used for the admin backend which is yet-to-be-created (not supplied with the open source version) #ADMIN_API_TOKEN=123 #ADMIN_API_URL=to-be-determined # # # URL of the TURN server (needed to "punch a hole" through some networks for P2P connections (future work) #TURN_SERVER= #TURN_USER= #TURN_PASSWORD= # # If your Turn server is configured to use the Turn REST API, you should put the shared auth secret here. # If you are using Coturn, this is the value of the "static-auth-secret" parameter in your coturn config file. # Keep empty if you are sharing hard coded / clear text credentials. #TURN_STATIC_AUTH_SECRET=
Once that has been done, make a backup of the supplied docker-compose.yaml file that comes with workadventure and replace it with the following one:
version: "3.3" services: front: image: thecodingmachine/workadventure-front:master environment: DEBUG_MODE: "$DEBUG_MODE" JITSI_URL: $JITSI_URL JITSI_PRIVATE_MODE: "$JITSI_PRIVATE_MODE" API_URL: pusher.${DOMAIN} TURN_SERVER: "${TURN_SERVER}" TURN_USER: "${TURN_USER}" TURN_PASSWORD: "${TURN_PASSWORD}" START_ROOM_URL: "${START_ROOM_URL}" labels: - "traefik.http.routers.front.rule=Host(`play.${DOMAIN}`)" - "traefik.http.routers.front.entryPoints=web" - "traefik.http.services.front.loadbalancer.server.port=80" - "traefik.http.routers.front-ssl.rule=Host(`play.${DOMAIN}`)" - "traefik.http.routers.front-ssl.entryPoints=websecure" - "traefik.http.routers.front-ssl.tls=true" - "traefik.http.routers.front-ssl.service=front" - "traefik.http.routers.front-ssl.tls.certresolver=le" restart: unless-stopped networks: web: pusher: image: thecodingmachine/workadventure-pusher:master command: yarn run runprod environment: SECRET_JITSI_KEY: "$SECRET_JITSI_KEY" SECRET_KEY: yourSecretKey API_URL: back:50051 JITSI_URL: $JITSI_URL JITSI_ISS: $JITSI_ISS labels: - "traefik.http.routers.pusher.rule=Host(`pusher.${DOMAIN}`)" - "traefik.http.routers.pusher.entryPoints=web" - "traefik.http.services.pusher.loadbalancer.server.port=8080" - "traefik.http.routers.pusher-ssl.rule=Host(`pusher.${DOMAIN}`)" - "traefik.http.routers.pusher-ssl.entryPoints=websecure" - "traefik.http.routers.pusher-ssl.tls=true" - "traefik.http.routers.pusher-ssl.service=pusher" - "traefik.http.routers.pusher-ssl.tls.certresolver=le" restart: unless-stopped networks: web: back: image: thecodingmachine/workadventure-back:master command: yarn run runprod environment: SECRET_JITSI_KEY: "$SECRET_JITSI_KEY" ADMIN_API_TOKEN: "$ADMIN_API_TOKEN" ADMIN_API_URL: "$ADMIN_API_URL" JITSI_URL: $JITSI_URL JITSI_ISS: $JITSI_ISS labels: - "traefik.http.routers.back.rule=Host(`api.${DOMAIN}`)" - "traefik.http.routers.back.entryPoints=web" - "traefik.http.services.back.loadbalancer.server.port=8080" - "traefik.http.routers.back-ssl.rule=Host(`api.${DOMAIN}`)" - "traefik.http.routers.back-ssl.entryPoints=websecure" - "traefik.http.routers.back-ssl.tls=true" - "traefik.http.routers.back-ssl.service=back" - "traefik.http.routers.back-ssl.tls.certresolver=le" restart: unless-stopped networks: web: maps: image: thecodingmachine/nodejs:12-apache environment: DEBUG_MODE: "$DEBUG_MODE" HOST: "0.0.0.0" NODE_ENV: development #APACHE_DOCUMENT_ROOT: dist/ #APACHE_EXTENSIONS: headers #APACHE_EXTENSION_HEADERS: 1 STARTUP_COMMAND_0: sudo a2enmod headers STARTUP_COMMAND_1: yarn install STARTUP_COMMAND_2: yarn run dev & volumes: - ./maps:/var/www/html labels: - "traefik.http.routers.maps.rule=Host(`maps.${DOMAIN}`)" - "traefik.http.routers.maps.entryPoints=web" - "traefik.http.services.maps.loadbalancer.server.port=80" - "traefik.http.routers.maps-ssl.rule=Host(`maps.${DOMAIN}`)" - "traefik.http.routers.maps-ssl.entryPoints=websecure" - "traefik.http.routers.maps-ssl.tls=true" - "traefik.http.routers.maps-ssl.service=maps" - "traefik.http.routers.maps-ssl.tls.certresolver=le" networks: web: networks: web: external: true
Once the .env and docker-compose.yaml files are in place, you should be able to start the project. It will fetch the required repo's and after a short time compiling some code and doing some checks, you should be able to visit your work-adventure instance on play.your.domain.tld:
# cd /opt/workadventure # docker-compose up
Check to see if basic functionality works. There are some meeting-rooms in the default map you can join to test the jitsi-integration with meet.jit.si If this seems to generally work, move on to setting up your own jitsi
Jitsi
A general guide for running JITSI via DOCKER can be found [here]
Installing Jitsi is as simple as:
- Pulling the [Jitsi Docker repo] from github
- Copying the env.example file to .env and extending it with a 'HOSTNAME' and editing PUBLIC_URL
- Running a script that creates random passwords for between the JITSI parts (./gen-passwords.sh)
- Creating a set of CONFIG dirs (see guide-url above)
- Replacing docker-compose.yaml file with one adjusted for our setup
First things first; fetch the git-repo of the dockerized jitsi-meet:
# cd /opt # git clone https://github.com/jitsi/docker-jitsi-meet.git
Now copy the env.example file to .env and edit it; mainly the adding HOSTNAME and changing PUBLIC_URL
... # System time zone TZ=UTC HOSTNAME=meet.your.domain.tld # Public URL for the web service (required) PUBLIC_URL=https://meet.your.domain.tld ...
Once that's done, we run the provided shell-script that sets random passwords for all the jitsi-parts to use between themselves.
# ./gen-passwords.sh
Jitsi's dockerized setup stores settings in the /root/.jitsi-meet-cfg/
on the host. Create the required directories with the command below:
mkdir -p ~/.jitsi-meet-cfg/{web/letsencrypt,transcripts,prosody/config,prosody/prosody-plugins-custom,jicofo,jvb,jigasi,jibri}
Great, almost there. The most important part is the docker-compose.yml file which has been edited to work with our traefik setup. Make a backup of the original and replace it with the one below:
version: '3' services: # Frontend web: image: jitsi/web volumes: - ${CONFIG}/web:/config - ${CONFIG}/web/letsencrypt:/etc/letsencrypt - ${CONFIG}/transcripts:/usr/share/jitsi-meet/transcripts environment: - ENABLE_LETSENCRYPT - ENABLE_HTTP_REDIRECT - ENABLE_HSTS - ENABLE_XMPP_WEBSOCKET - DISABLE_HTTPS - LETSENCRYPT_DOMAIN - LETSENCRYPT_EMAIL - LETSENCRYPT_USE_STAGING - PUBLIC_URL - TZ - AMPLITUDE_ID - ANALYTICS_SCRIPT_URLS - ANALYTICS_WHITELISTED_EVENTS - BRIDGE_CHANNEL - CALLSTATS_CUSTOM_SCRIPT_URL - CALLSTATS_ID - CALLSTATS_SECRET - CHROME_EXTENSION_BANNER_JSON - CONFCODE_URL - CONFIG_EXTERNAL_CONNECT - DEPLOYMENTINFO_ENVIRONMENT - DEPLOYMENTINFO_ENVIRONMENT_TYPE - DEPLOYMENTINFO_USERREGION - DIALIN_NUMBERS_URL - DIALOUT_AUTH_URL - DIALOUT_CODES_URL - DROPBOX_APPKEY - DROPBOX_REDIRECT_URI - DYNAMIC_BRANDING_URL - ENABLE_AUDIO_PROCESSING - ENABLE_AUTH - ENABLE_CALENDAR - ENABLE_FILE_RECORDING_SERVICE - ENABLE_FILE_RECORDING_SERVICE_SHARING - ENABLE_GUESTS - ENABLE_IPV6 - ENABLE_LIPSYNC - ENABLE_NO_AUDIO_DETECTION - ENABLE_P2P - ENABLE_PREJOIN_PAGE - ENABLE_WELCOME_PAGE - ENABLE_CLOSE_PAGE - ENABLE_RECORDING - ENABLE_REMB - ENABLE_REQUIRE_DISPLAY_NAME - ENABLE_SIMULCAST - ENABLE_STATS_ID - ENABLE_STEREO - ENABLE_SUBDOMAINS - ENABLE_TALK_WHILE_MUTED - ENABLE_TCC - ENABLE_TRANSCRIPTIONS - ETHERPAD_PUBLIC_URL - ETHERPAD_URL_BASE - GOOGLE_ANALYTICS_ID - GOOGLE_API_APP_CLIENT_ID - INVITE_SERVICE_URL - JICOFO_AUTH_USER - MATOMO_ENDPOINT - MATOMO_SITE_ID - MICROSOFT_API_APP_CLIENT_ID - NGINX_RESOLVER - NGINX_WORKER_PROCESSES - NGINX_WORKER_CONNECTIONS - PEOPLE_SEARCH_URL - RESOLUTION - RESOLUTION_MIN - RESOLUTION_WIDTH - RESOLUTION_WIDTH_MIN - START_AUDIO_ONLY - START_AUDIO_MUTED - DISABLE_AUDIO_LEVELS - ENABLE_NOISY_MIC_DETECTION - START_BITRATE - DESKTOP_SHARING_FRAMERATE_MIN - DESKTOP_SHARING_FRAMERATE_MAX - START_VIDEO_MUTED - TESTING_CAP_SCREENSHARE_BITRATE - TESTING_OCTO_PROBABILITY - XMPP_AUTH_DOMAIN - XMPP_BOSH_URL_BASE - XMPP_DOMAIN - XMPP_GUEST_DOMAIN - XMPP_MUC_DOMAIN - XMPP_RECORDER_DOMAIN - TOKEN_AUTH_URL networks: # traefik: change the following line to your external docker network web: meet.jitsi: aliases: - ${XMPP_DOMAIN} labels: - "traefik.http.routers.jitsiweb-http.entrypoints=web" - "traefik.http.routers.jitsiweb-http.rule=Host(`${HOSTNAME}`)" - "traefik.http.services.jitsiweb-http.loadbalancer.server.port=80" - "traefik.http.routers.jitsiweb.entrypoints=websecure" - "traefik.http.routers.jitsiweb.rule=Host(`${HOSTNAME}`)" - "traefik.http.routers.jitsiweb.tls=true" - "traefik.http.routers.jitsiweb.tls.certresolver=le" # traefik: change the following line to your external docker network - "traefik.docker.network=web" # XMPP server prosody: image: jitsi/prosody expose: - '5222' - '5347' - '5280' volumes: - ${CONFIG}/prosody:/config environment: - AUTH_TYPE - ENABLE_AUTH - ENABLE_GUESTS - ENABLE_LOBBY - ENABLE_XMPP_WEBSOCKET - GLOBAL_MODULES - GLOBAL_CONFIG - LDAP_URL - LDAP_BASE - LDAP_BINDDN - LDAP_BINDPW - LDAP_FILTER - LDAP_AUTH_METHOD - LDAP_VERSION - LDAP_USE_TLS - LDAP_TLS_CIPHERS - LDAP_TLS_CHECK_PEER - LDAP_TLS_CACERT_FILE - LDAP_TLS_CACERT_DIR - LDAP_START_TLS - XMPP_DOMAIN - XMPP_AUTH_DOMAIN - XMPP_GUEST_DOMAIN - XMPP_MUC_DOMAIN - XMPP_INTERNAL_MUC_DOMAIN - XMPP_MODULES - XMPP_MUC_MODULES - XMPP_INTERNAL_MUC_MODULES - XMPP_RECORDER_DOMAIN - XMPP_CROSS_DOMAIN - JICOFO_COMPONENT_SECRET - JICOFO_AUTH_USER - JICOFO_AUTH_PASSWORD - JVB_AUTH_USER - JVB_AUTH_PASSWORD - JIGASI_XMPP_USER - JIGASI_XMPP_PASSWORD - JIBRI_XMPP_USER - JIBRI_XMPP_PASSWORD - JIBRI_RECORDER_USER - JIBRI_RECORDER_PASSWORD - JWT_APP_ID - JWT_APP_SECRET - JWT_ACCEPTED_ISSUERS - JWT_ACCEPTED_AUDIENCES - JWT_ASAP_KEYSERVER - JWT_ALLOW_EMPTY - JWT_AUTH_TYPE - JWT_TOKEN_AUTH_MODULE - LOG_LEVEL - PUBLIC_URL - TZ networks: meet.jitsi: aliases: - ${XMPP_SERVER} web: # Focus component jicofo: image: jitsi/jicofo volumes: - ${CONFIG}/jicofo:/config environment: - AUTH_TYPE - BRIDGE_AVG_PARTICIPANT_STRESS - BRIDGE_STRESS_THRESHOLD - ENABLE_AUTH - ENABLE_AUTO_OWNER - ENABLE_CODEC_VP8 - ENABLE_CODEC_VP9 - ENABLE_CODEC_H264 - ENABLE_RECORDING - ENABLE_SCTP - JICOFO_COMPONENT_SECRET - JICOFO_AUTH_USER - JICOFO_AUTH_PASSWORD - JICOFO_ENABLE_BRIDGE_HEALTH_CHECKS - JICOFO_CONF_INITIAL_PARTICIPANT_WAIT_TIMEOUT - JICOFO_CONF_SINGLE_PARTICIPANT_TIMEOUT - JICOFO_ENABLE_HEALTH_CHECKS - JICOFO_SHORT_ID - JICOFO_RESERVATION_ENABLED - JICOFO_RESERVATION_REST_BASE_URL - JIBRI_BREWERY_MUC - JIBRI_REQUEST_RETRIES - JIBRI_PENDING_TIMEOUT - JIGASI_BREWERY_MUC - JIGASI_SIP_URI - JVB_BREWERY_MUC - MAX_BRIDGE_PARTICIPANTS - OCTO_BRIDGE_SELECTION_STRATEGY - TZ - XMPP_DOMAIN - XMPP_AUTH_DOMAIN - XMPP_INTERNAL_MUC_DOMAIN - XMPP_MUC_DOMAIN - XMPP_SERVER depends_on: - prosody networks: meet.jitsi: web: # Video bridge jvb: image: jitsi/jvb ports: - '${JVB_PORT}:${JVB_PORT}/udp' - '${JVB_TCP_MAPPED_PORT}:${JVB_TCP_PORT}' volumes: - ${CONFIG}/jvb:/config environment: - DOCKER_HOST_ADDRESS - XMPP_AUTH_DOMAIN - XMPP_INTERNAL_MUC_DOMAIN - XMPP_SERVER - JVB_AUTH_USER - JVB_AUTH_PASSWORD - JVB_BREWERY_MUC - JVB_PORT - JVB_TCP_HARVESTER_DISABLED - JVB_TCP_PORT - JVB_TCP_MAPPED_PORT - JVB_STUN_SERVERS - JVB_ENABLE_APIS - JVB_WS_DOMAIN - JVB_WS_SERVER_ID - PUBLIC_URL - TZ depends_on: - prosody labels: traefik.udp.routers.jvb.entrypoints: video traefik.udp.routers.jvb.service: jvb traefik.udp.services.jvb.loadbalancer.server.port: 10000 networks: meet.jitsi: web: # Custom network so all services can communicate using a FQDN networks: meet.jitsi: web: external: true
Awesome. You should now be able to start the project with docker-compose and be able to navigate to meet.your.domain.tld and check it out after a brief while allowing things to get started:
# docker-compose up
Note:
If this works, you can now go back to the Work-adventure setup and edit the JITSI_URL parameter to point it to your meet.your.domain.tld
and test if things still work the way you want them to.
Run docker-compose as systemd units
To have automatic startups of traefik, jitsi and wa, run them via Systemd → Add docker-compose as systemd unit.
How to manage maps
Other things
TODO points
- Investigate logging: Logspout
- Fix CORS issues with linking to external web-pages in map (requires extra Apache config in 'wa-front' container
- Setup CoTurn fully. Biggest issue there seems getting SSL to work on the UDP service since Traefik cannot be configured to handle this. WIll need some way to export Traefik-created LE-keys into coturn container
- Set up telemetry AND monitoring for Traefik, Jitsi, Coturn and Work-adventure.
- Traefik Performance metrics via [Prometheus]
- Jitsi performance metrics via [Prometheus-jitsi-meet-exporter] (seems Dreamer has something to do with this. Awesome!)
- Work-adventure 'backend' manager (see back/src/Controller/PrometheusController.ts and back/src/Services/GaugeManager.ts )
- Number of users/conferences active; export via API for use in IRC-bot/webpage
- Create a 'catch-page' on social.techinc.nl with info on WA and Jitsi; quick-links to some jitsi-rooms so as to not have to go through WA to join a meeting
- Investigate 'federation' of WA-installs (seems people are working on this)
- Set up some way to allow people to help work on the techinc map (and/or others)
Reference material
- [Work-adventure github issue introducing better/easier configuration]
- [A lot of info on TURN/STUN] What it does, how it works, how to test it.