Fundraising/ContributionTracking

From Wikitech

ContributionTracking refers to donation metadata saved to a table which currently resides not in the fundraising civicrm database, but in the drupal database for the host drupal instance. The placement of this table in the drupal database is clearly very silly, and as such it needs to be moved elsewhere.

Payments-wiki formerly used a direct DB connection to the backend to insert new rows and get the autoincrement ID to use as the contribution_tracking_id. That DB connection was managed by an extension called ContributionTracking. The ContributionTracking extension no longer exists, and ID generation has been moved to a sequence generator in Redis, to fully decouple the payments-wiki front end from the back-end database.

UPDATE: The ContributionTracking table has been moved from drupal to the civicrm database. The table's name in the civicrm database is civicrm_contribution_tracking. The ContributionTracking entity controls the DB connection from CiviCRM and is being managed by the wmf-civicrm CiviCRM extension.

NOTE: THE BELOW PLAN IS OUTDATED

Plan To Change The Contribution Tracking Table

Moving to its own database

  1. Create a new ContributionTracking database
  2. Import a dump of the old ContributionTracking table to the new database
  3. Alter the auto increment on the ContributionTracking's id, to be significantly higher than the current highest id
  4. Grant mysql privileges to user@host that need various levels of access to the new ContributionTracking database
  5. Change the references to the ContributionTracking db. This process will have to wait for a cluster deploy.
    1. Disable the queue consumer
    2. Update the references and credentials in the following places:
      1. On the Cluster
      2. Payments Cluster
      3. All Drupal/Civi Modules
    3. Re-enable the Queue Consumer
  6. Watch the old database to determine that nothing is still updating the old table
  7. Once you have determined that there is no movement on the old table, import all records that were created between the initial import, and the switch-over.
  8. Kill the old table

Schema Changes to the new ContributionTracking DB

  1. Analyze the ContributionTracking extension, determine what must be changed so the actual schema change can be done code-first or DB-first, and make the necessary changes
  2. Push just those changes out to the cluster
  3. Do the same analysis and make the same changes for extensions on the payments cluster which require ContributionTracking.
  4. Push those changes out to the payments cluster.
  5. Do the same analysis and make the same changes for all drupal/civi modules.
  6. Push those changes out to aluminium.
  7. Get the schema change ready to go
  8. Get the old crufty code cleanup ready to go
  9. Do the schema change and old crufty code cleanup in any order (shouldn't matter if you did the rest of this properly).

Al files referring to db1008

br/cnTopCities.py
br/FRStats.py
br/getvars.py
br/mins.py
br/MVData.py
br/r/test.R
org.wikimedia.civicrm-dev/sites/default/civicrm.settings.php
org.wikimedia.civicrm-dev/sites/default/settings.php
org.wikimedia.civicrm/fundcore_gateway/paypal
org.wikimedia.civicrm/IPNListener_Recurring.php
org.wikimedia.civicrm/sites/default/civicrm.settings.php
org.wikimedia.civicrm/sites/default/settings.php
org.wikimedia.community-analytics/community-analytics/classes/classes/DataLoader.py
org.wikimedia.community-analytics/community-analytics/classes/DataLoader.py
org.wikimedia.community-analytics/community-analytics/config/settings.py
org.wikimedia.community-analytics/community-analytics/sandbox/src/SOPA_miner.py
org.wikimedia.community-analytics/community-analytics/sandbox/src/zip_to_congress.py
org.wikimedia.fundraising-analytics/fundraiser-analysis/classes/DataLoader.py
org.wikimedia.fundraising-analytics/fundraiser-analysis/web_reporting/json_reporting/views.py
org.wikimedia.fundraising-analytics/Fundraiser_Tools/classes/DataLoader.py
org.wikimedia.fundraising-analytics/Fundraiser_Tools/classes/SOPA_miner.py
org.wikimedia.fundraising-analytics/Fundraiser_Tools/config/settings.py
org.wikimedia.fundraising-analytics/Fundraiser_Tools/scripts/config/settings.py
org.wikimedia.fundraising-analytics/Fundraiser_Tools/scripts/settings.py
org.wikimedia.fundraising-analytics/Fundraiser_Tools/web_reporting/json_reporting/views.py
org.wikimedia.fundraising/IPNListener_Standalone.php