Nova Resource:Striker

From Wikitech
Jump to navigation Jump to search
Project Name striker
Details,
admins/members
openstack-browser
Monitoring

Striker is a web application to help Toolforge maintainers manage their tools.

See also

Edit documentation

Server admin log

2019-02-13

  • 19:15 andrewbogott: deleting striker-puppet01 and striker-deploy04

2019-01-16

  • 17:26 andrewbogott: deleting VM striker-mwvdev

2018-12-27

  • 22:50 bd808: Deleted striker-deploy03 for T204563
  • 04:26 bd808: Shutdown striker-deploy03 instance for T204563
  • 00:24 bd808: Promoted striker-deploy04 to active deploy server

2018-10-19

  • 19:39 andrewbogott: migrating project to eqiad1

2018-08-14

  • 14:42 bstorm_: deployed updated keystone URL

2018-08-02

  • 19:17 bd808: Building new deploy server as striker-... (more)



Setting up the project

The project uses MediaWiki-Vagrant and its striker role to provision a VM running:

  • An "SUL" wiki
  • An LDAP wiki
  • An OpenStack Keystone service

The role is hackedcustomized locally to disable the striker service itself:

diff --git i/puppet/modules/role/manifests/striker.pp w/puppet/modules/role/manifests/striker.pp
index 5fc8531c..c2d2046a 100644
--- i/puppet/modules/role/manifests/striker.pp
+++ w/puppet/modules/role/manifests/striker.pp
@@ -218,13 +218,13 @@ class role::striker(
         unless  => "/usr/bin/mysql -qfsANe \"${populate_unless}\" | /usr/bin/tail -1 | /bin/grep -vq 0",
     }

-    apache::site { $vhost_name:
-        ensure   => present,
-        # Load before MediaWiki wildcard vhost for Labs.
-        priority => 40,
-        content  => template('role/striker/apache.conf.erb'),
-        notify   => Service['apache2'],
-    }
+#   apache::site { $vhost_name:
+#       ensure   => present,
+#       # Load before MediaWiki wildcard vhost for Labs.
+#       priority => 40,
+#       content  => template('role/striker/apache.conf.erb'),
+#       notify   => Service['apache2'],
+#   }

     # Setup Phabricator
     class { '::phabricator':

/srv/mediawiki-vagrant/puppet/hieradata/local.yaml settings:

mwv::tld: -striker.wmflabs.org
role::mediawiki::hostname: devwiki-striker.wmflabs.org
mediawiki::multiwiki::base_domain: -striker.wmflabs.org
$ cd /srv/mediawiki-vagrant
$ vagrant roles enable striker
$ vagrant forward-port 3306 3306    # mysql
$ vagrant forward-port 1389 389     # ldap
$ vagrant forward-port 5000 5000    # openstack
$ vagrant forward-port 35357 35357  # openstack
$ vagrant up --provision

Use Horizon to setup several proxies:

  • devwiki-striker.wmflabs.org
  • phabricator-striker.wmflabs.org
  • ldapauth-striker.wmflabs.org

The striker uwsgi service is deployed to an OpenStack VM using the same role::striker::web Puppet class used in production. This role expects the actual deployment to be done via scap, so the project also needs a deployment server. Instructions for setting up a VM to do this can be found at User:BryanDavis/Scap3 in a Cloud VPS project. The same VM that runs MediaWiki-Vagrant can also serve as the deploy server.

A few hiera settings made via horizon are needed to setup role::striker::web:

memcached::ip: 127.0.0.1
memcached::port: 11211
memcached::size: 256
nginx::variant: light
striker::apache::docroot: /srv/deployment/striker/deploy/public_html
striker::apache::port: 80
striker::apache::server_name: striker.wmflabs.org
striker::apache::servers:
- http://127.0.0.1:8081
striker::uwsgi::config:
  cache:
    LOCATION: 127.0.0.1:11211
  db:
    ENGINE: django.db.backends.mysql
    HOST: striker-deploy03.striker.eqiad.wmflabs
    NAME: striker
    PORT: 3306
    USER: striker
  debug:
    DEBUG: false
  https:
    REQUIRE_HTTPS: true
    SSL_CANONICAL_HOST: striker.wmflabs.org
  ldap:
    BASE_DN: dc=wmftest,dc=net
    BIND_USER: cn=writer,dc=wmftest,dc=net
    SERVER_URI: ldap://striker-support01.striker.eqiad.wmflabs:1389
    STAFF_GROUP_DN: cn=wmf,ou=groups,dc=wmftest,dc=net
    SUPERUSER_GROUP_DN: cn=tools.admin,ou=servicegroups,dc=wmftest,dc=net
    TLS: false
    TOOLS_MAINTAINER_BASE_DN: ou=people,dc=wmftest,dc=net
    TOOLS_TOOL_BASE_DN: ou=servicegroups,dc=wmftest,dc=net
    USER_SEARCH_BASE: ou=People,dc=wmftest,dc=net
  logging:
    FILE_FILENAME: /srv/log/striker/striker.log
    HANDLERS: file
    LEVEL: DEBUG
  oauth:
    CONSUMER_KEY: dc94ba54f485921af07b7b53a62d083c
    MWURL: https://devwiki-striker.wmflabs.org/w/index.php
  openstack:
    URL: http://striker-support01.striker.eqiad.wmflabs:5000/v3
  phabricator:
    REPO_ADMIN_GROUP: PHID-PROJ-i4vx7xul4ozakbdlt52z
    SERVER_URL: http://phabricator-striker.wmflabs.org
    USER: StrikerBot
  static:
    STATIC_ROOT: /srv/deployment/striker/deploy/public_html/static
  wikitech:
    SERVER_URL: https://ldapauth-striker.wmflabs.org
  xff:
    TRUSTED_PROXY_LIST: 10. 127.0.0.1
    USE_XFF_HEADER: true
striker::uwsgi::port: 8081
striker::uwsgi::secret_config:
  db:
    PASSWORD: striker
  ldap:
    BIND_PASSWORD: vagrant_writer
  oauth:
    CONSUMER_SECRET: follow the tutorial linked below
  openstack:
    PASSWORD: striker-vagrant
    USER: admin
  phabricator:
    TOKEN: follow the tutorial linked below
  secrets:
    SECRET_KEY: 'do not use this secret, make your own'
  wikitech:
    ACCESS_SECRET: follow the tutorial linked below
    ACCESS_TOKEN: follow the tutorial linked below
    CONSUMER_SECRET: follow the tutorial linked below
    CONSUMER_TOKEN: follow the tutorial linked below

Many of these values are deployment dependent, but can be found by following the installation instructions included with the role. A few additional manual changes are needed to make things work correctly with the Striker uwsgi service separated from the MediaWiki-Vagrant VM:

  • MySQL grants need to be made for the `striker`@`%` user. These should be identical to the grants that will already exist for the `striker`@`127.0.0.1` user.
    • NOTE: you may need to set bind-address = 0.0.0.0 in /etc/mysql/mariadb.conf.d/50-server.cnf to allow remote connections to the service.
  • The advertised URL for the keystone services need to be changed to use public hostnames:
    $ /usr/local/bin/use-openstack endpoint list
    +----------------------------------+-----------+--------------+--------------+---------+-----------+---------------------------------------------------------+
    | ID                               | Region    | Service Name | Service Type | Enabled | Interface | URL                                                     |
    +----------------------------------+-----------+--------------+--------------+---------+-----------+---------------------------------------------------------+
    | 7cf84775dbd24987951490c9886e12a2 | RegionOne | keystone     | identity     | True    | public    | http://127.0.0.1:5000/v3/ |
    | b2cb723b0157468eaf7ae58ba780a72c | RegionOne | keystone     | identity     | True    | admin     | http://127.0.0.1:35357/v3/                              |
    +----------------------------------+-----------+--------------+--------------+---------+-----------+---------------------------------------------------------+
    $ /usr/local/bin/use-openstack endpoint set --url http://striker-support01.striker.eqiad.wmflabs:5000/v3/ 7cf84775dbd24987951490c9886e12a2
    $ /usr/local/bin/use-openstack endpoint set --url http://striker-support01.striker.eqiad.wmflabs:35357/v3/ b2cb723b0157468eaf7ae58ba780a72c