Fundraising/Data and flow/Queues/Queue performance log

Tue 01 Nov 2022 11:58:53 PM UTC

Queue speed in 3 hour test was slow.

1) Redis cache hits still seemed higher than I expected - I created an issue to add some notes on re this 2) There were high deadlocks. The deduper job was running, which I thought should have been throttled. On looking the throttling depends on a drush variable threshold which stands for 'max number of contacts created in the last (5) minutes for the job to be permitted to continue. I checked what the threshold was and found it was not set. Until the threshold_period the threshold has no default value. This meant the deduper throttling was effectively turned off.

I set the threshold to be 250 (drush @wmff vset threshold=250) - so if 250 contacts have been created in the last 5 minutes deduper will not run. I don't know if that threshold is 'right' - we update more often than we create. Christine did suggest changing to 'modfied_date' which might be better, although it would pick up bulk changes that are not ongoing. We can monitor... however, the goal is that the deduper job should not run during time of high load & it was running