Nova Resource:Striker

From Wikitech
Jump to: navigation, search
Project Name striker
Details,
admins/members
openstack-browser
Monitoring

Striker is a web application to help Toolforge maintainers manage their tools.

See also

Edit documentation

Server admin log

2017-08-13

  • 20:13 bd808: Got a clean scap3 deploy after manually deleting __pycache__ directories on deploy target
  • 19:48 bd808: Cherry-picked 370139 fro final testing
  • 15:58 bd808: Hacked keystone admin endpoint config manually -- openstack endpoint set --url http://striker-deploy03.striker.eqiad.wmflabs:5000/v3/ 35205d0

2017-07-31



Setting up the project

The project uses MediaWiki-Vagrant and its striker role to provision a VM running:

  • An "SUL" wiki
  • An LDAP wiki
  • An OpenStack Keystone service

The role is hackedcustomized locally to disable the striker service itself:

diff --git i/puppet/modules/role/manifests/striker.pp w/puppet/modules/role/manifests/striker.pp
index a9dc91a..c038fcd 100644
--- i/puppet/modules/role/manifests/striker.pp
+++ w/puppet/modules/role/manifests/striker.pp
@@ -99,14 +99,14 @@ class role::striker(
     include ::role::oathauth
     include ::role::oauth
     include ::role::titleblacklist
-    include ::apache::mod::wsgi_py3
+#    include ::apache::mod::wsgi_py3
     include ::memcached
     include ::mysql::large_prefix

-    file { "${log_dir}/striker":
-        ensure => 'directory',
-        mode   => '0777',
-    }
+#    file { "${log_dir}/striker":
+#        ensure => 'directory',
+#        mode   => '0777',
+#    }

     # Setup Striker
     $app_dir = "${deploy_dir}/striker"
@@ -212,14 +212,14 @@ class role::striker(
         unless  => "${venv}/bin/python manage.py collectstatic --noinput --dry-run| grep -q '^0 static'",
     }

-    apache::site { $vhost_name:
-        ensure   => present,
-        # Load before MediaWiki wildcard vhost for Labs.
-        priority => 40,
-        content  => template('role/striker/apache.conf.erb'),
-        require  => Class['apache::mod::wsgi_py3'],
-        notify   => Service['apache2'],
-    }
+#    apache::site { $vhost_name:
+#        ensure   => present,
+#        # Load before MediaWiki wildcard vhost for Labs.
+#        priority => 40,
+#        content  => template('role/striker/apache.conf.erb'),
+#        require  => Class['apache::mod::wsgi_py3'],
+#        notify   => Service['apache2'],
+#    }

     # Setup devwiki
     $admin_email = 'admin@local.wmftest.net'

The striker uwsgi service is deployed to an OpenStack VM using the same role::striker::web Puppet class used in production. This role expects the actual deployment to be done via scap, so the project also needs a deployment server. Instructions for setting up a VM to do this can be found at User:BryanDavis/Scap3_in_a_Labs_project. The same VM that runs MediaWiki-Vagrant can also serve as the deploy server.

A few hiera settings made via horizon are needed to setup role::striker::web:

memcached::ip: 127.0.0.1
memcached::port: 11211
memcached::size: 256
nginx::variant: light
striker::apache::docroot: /srv/deployment/striker/deploy/public_html
striker::apache::port: 80
striker::apache::server_name: striker.wmflabs.org
striker::apache::servers:
- http://127.0.0.1:8081
striker::uwsgi::config:
  cache:
    LOCATION: 127.0.0.1:11211
  db:
    ENGINE: django.db.backends.mysql
    HOST: striker-deploy03.striker.eqiad.wmflabs
    NAME: striker
    PORT: 3306
    USER: striker
  debug:
    DEBUG: false
  https:
    REQUIRE_HTTPS: true
    SSL_CANONICAL_HOST: striker.wmflabs.org
  ldap:
    BASE_DN: dc=wmftest,dc=net
    BIND_USER: cn=writer,dc=wmftest,dc=net
    SERVER_URI: ldap://striker-deploy03.striker.eqiad.wmflabs:1389
    STAFF_GROUP_DN: cn=wmf,ou=groups,dc=wmftest,dc=net
    SUPERUSER_GROUP_DN: cn=tools.admin,ou=servicegroups,dc=wmftest,dc=net
    TLS: false
    TOOLS_MAINTAINER_BASE_DN: ou=people,dc=wmftest,dc=net
    TOOLS_TOOL_BASE_DN: ou=servicegroups,dc=wmftest,dc=net
    USER_SEARCH_BASE: ou=People,dc=wmftest,dc=net
  logging:
    FILE_FILENAME: /srv/log/striker/striker.log
    HANDLERS: file logstash
    LEVEL: DEBUG
    LOGSTASH_HOST: deployment-logstash2.eqiad.wmflabs
    LOGSTASH_PORT: 11514
  oauth:
    CONSUMER_KEY: dc94ba54f485921af07b7b53a62d083c
    MWURL: https://devwiki-striker.wmflabs.org/w/index.php
  openstack:
    URL: http://striker-deploy03.striker.eqiad.wmflabs:5000/v3
  phabricator:
    REPO_ADMIN_GROUP: PHID-PROJ-i4vx7xul4ozakbdlt52z
    SERVER_URL: http://phabricator-striker.wmflabs.org
    USER: StrikerBot
  static:
    STATIC_ROOT: /srv/deployment/striker/deploy/public_html/static
  wikitech:
    SERVER_URL: https://ldapauth-striker.wmflabs.org
  xff:
    TRUSTED_PROXY_LIST: 10. 127.0.0.1
    USE_XFF_HEADER: true
striker::uwsgi::port: 8081
striker::uwsgi::secret_config:
  db:
    PASSWORD: striker
  ldap:
    BIND_PASSWORD: vagrant_writer
  oauth:
    CONSUMER_SECRET: follow the tutorial linked below
  openstack:
    PASSWORD: striker-vagrant
    USER: admin
  phabricator:
    TOKEN: follow the tutorial linked below
  secrets:
    SECRET_KEY: 'do not use this secret, make your own'
  wikitech:
    ACCESS_SECRET: follow the tutorial linked below
    ACCESS_TOKEN: follow the tutorial linked below
    CONSUMER_SECRET: follow the tutorial linked below
    CONSUMER_TOKEN: follow the tutorial linked below

Many of these values are deployment dependent, but can be found by following the installation instructions included with the role. A few additional manual changes are needed to make things work correctly with the Striker uwsgi service separated from the MediaWiki-Vagrant VM:

  • MySQL grants need to be made for the `striker`@`%` user. These should be identical to the grants that will already exist for the `striker`@`127.0.0.1` user.
  • The advertised URL for the public keystone service needs to be changed from http://127.0.0.1:5000/v3/ to http://vm-hosting-mwvagrant.project.eqiad.wmflabs:5000/v3/ using openstack endpoint set --url.