Jump to content

Fundraising/Internal Endpoints/Drupal modules

From Wikitech

These modules interface with CiviCRM. They are found in the wikimedia/fundraising/crm repo, under sites/all/modules.

Module Description Source Documentation
Audit modules (drupal) Compares automatically-downloaded nightly audit files against the CRM database, and logs discrepancies. wikimedia/fundraising/crm
contribution_tracking Applies the contribution_tracking table schema wikimedia/fundraising/crm Wikitech:Fundraising
Communications Generalized mailing job management and templating. Wraps the sendmail library (currently PHPMailer). wikimedia/fundraising/crm
Donations queue consumer Takes donations added to the queue by IPN listeners or the front-end payments servers and inserts them into CiviCRM Fundraising tech/Message queues
Exchange Rates This module handles converting foreign currencies into US Dollars. It consists of two major components. One function handles doing the conversions based on exchange rates stored in the drupal.exchange_rates table. This function is called from the queue2civicrm module when it is processing donations from the queue. Another function, which is activated by a cron job, looks up the exchange rates on the web and updates the exchange_rates table. The exchange rates are currently pulled from oanda.com, with http://www.ecb.int/stats/eurofxref/eurofxref-daily.xml used as a back-up source.
Fredge queue consumer Exposes per-gateway throughput counts and processing time statistics as seen on https://grafana.wikimedia.org/dashboard/db/fundraising-overview. [Broken Link T211982]
ganglia_reporter (Theoretically) exposes per-gateway throughput counts.


globalcollect_audit

Compares automatically-downloaded nightly audit files against the CRM database, and logs discrepancies.

large_donation

Send an email to the Major Gifts team when any online donation exceeds a configured dollar amount threshold.

log_audit (Theoretically) audits nightly PayPal IPN logs against CRM database.
Metrics reporter
offline2civicrm Import check and chargeback spreadsheets into the CRM.
paypal_audit (Deprecated) audits Payflow Pro logs.
queue2civicrm This module pulls online contributions from the ActiveMQ queue 'donations' and writes them to the CiviCRM database via the CiviCRM API. It is activated by a periodic Jenkins job, which fires an included Drush command (found in queue2civicrm/queue_consume.drush.inc) to replace the former cron method. Output from the execution is captured in the Jenkins console logs.

Queue2civicrm also contains a hook (queue2civicrm_import) which is invoked at the end of a singular transaction, passing transactional information on to other modules that will perform other actions on the transaction. The following modules make use of this hook:

It also contains a hook which is invoked at the end of processing a batch of messages, which allows for other modules to perform batch processing. This hook is made use of in the following modules:

The settings for queue2civicrm can be configured at http://<web root for drupal>/admin/settings/queue2civicrm. You can configure the connection information to ActiveMQ, what queue to consume from and the number of messages to consume at once. Also, you can test the ActiveMQ connection and test pushing messages into the queue.

Formerly, this module interacted directly with the PayflowPro gateway to validate transactions and passed queued items ready for consumption to a custom API for inserting them into CiviCRM. This module no longer performs any gateway-specific actions and uses the built-in CiviCRM API for inserting contributions.

Execution

There are a number of ways to execute queue consumption with queue2civicrm and queue2civicrm/recurring. The easiest way is with Drush. From the Drupal root,
$ drush --user=1 qc
This executes the queue2civicrm Drush script, located in queue2civicrm/queue_consume.drush.inc
recurring

The recurring module is a sub-module to queue2civicrm and shares much of the functionality. As is it is right now, it is built particularly to handle raw IPN messages sent from PayPal pertaining to recurring (or in PayPal's language, 'subscription') contributions. Because CiviCRM does not have an API for recurring contributions and due to the immaturity of some aspects of the Civi APIs, this module performs a lot of custom queries to both the CiviCRM and Drupal databases.

This module can be configured from the queue2civicrm configuration.

It processes messages out of its own queue (as configured in the interface). Queue consumption is invoked by making use of the hook at the end of batch processing in queue2civicrm.

Recurring queue consumer
recurring_globalcollect

GlobalCollect does not actively recur subscriptions. This module is run daily, and will query the database for GC recurring donations which should be charged that day.

thank_you This module sends a thank you message to online donors. There's a job polling for new donations with a null no_thank_you and an empty thankyou_date field. The body of the thank you message is taken from the modules/thank_you/templates directory, and an interface to configure or test mailings can be found in Administration » Configuration » Thank you settings. Generate new Thank You email templates
wmf_civicrm

Compatibility and utility layer on top of the CiviCRM API.

wmf_common

Shared convenience library.

wmf_communication Generalized mailing job management and templating. Wraps the sendmail library (currently PHPMailer).
wmf_eoy_receipt Create an tax receipt listing all of a donor's contributions for that year.
wmf_unsubscribe

Perform donor mailing list opt-out.

wmf_unsubscribe_qc

Read from the queue and call opt-out routines from wmf_unsubscribe.

OrphanSlayer Remove orphan transactions from the Pending table of the Smashpig database Fundraising tech/orphan slayer