Fundraising/Data and flow/PSP integrations/Adyen Checkout

From Wikitech
Adyen Checkout
Original Name Adyen Checkout
Our Name Adyen Checkout
Current Name Adyen Checkout
Payment Methods Credit Card, iDEAL, Apple Pay, Google Pay
Countries Main Card Processor
Documentation https://docs.adyen.com/online-payments/web-components/integrated-before-5-0-0
Production Console https://ca-live.adyen.com/ca/ca/login.shtml
Test Console https://ca-test.adyen.com/ca/ca/login.shtml
Contact ?

Overview

Adyen is one of our credit card processors.

In 2021, we updated our Adyen code to use their 'Checkout' integration, version 4. Checkout integration sources Adyen-hosted JS on our payments page and renders payment fields in the same form as the donor data, encrypting the data before submitting it to our server. We are using the 'components' style Checkout integration rather than the 'drop-in' style. Both styles use the same Adyen JS objects, but with components we instantiate a slightly lower-level JS object that is specific to the payment method.

A month after we deployed our new integration, Adyen released Checkout version 5 with some breaking changes and a simplified control flow. We have not made plans to update to version 5.

All calls to the Adyen API are routed through the SmashPig library. There are PaymentProcessor subclasses for each payment method.

Documentation for the former (Hosted Payment Pages) integration is at https://www.mediawiki.org/wiki/Fundraising_tech/Adyen

Payment Methods

Credit Cards

https://docs.adyen.com/payment-methods/cards/web-component

Credit cards are available with payment_method=cc which adds Adyen's fields (Card number, Expiry date, and CVC / CVV) below ours in the payments form.

Credit Card form

3DS

https://docs.adyen.com/online-payments/3d-secure

When 3DS is enabled we collect more information about the donation and send it to Adyen in the browserInfo parameter.

https://docs.adyen.com/online-payments/3d-secure/redirect-3ds2/web-component

Example browserInfo: 

   "browserInfo":{
      "userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/70.0.3538.110 Safari\/537.36",
      "acceptHeader":"text\/html,application\/xhtml+xml,application\/xml;q=0.9,image\/webp,image\/apng,*\/*;q=0.8",
      "language":"nl-NL",
      "colorDepth":24,
      "screenHeight":723,
      "screenWidth":1536,
      "timeZoneOffset":0,
      "javaEnabled":true
   },

Adyen will determine on their end if the 3DS redirect is needed and if so return an action object on the response. This contains information on the 3DS redirect.

https://docs.adyen.com/online-payments/api-only/handling-redirects#3d-secure

iDEAL

https://docs.adyen.com/payment-methods/ideal/web-component

iDEAL is bank transfer payment method only available in the Netherlands (payment_method=rtbt&country=NL).

The supported banking systems can be down so Adyen handles creating the list of available banks (referred to as issuers by Adyen) which we get by calling getPaymentMethods.

The donor chooses a bank and is then redirected to their bank's website to complete the donation.

When that is complete they are redirected to our thank you page.

List of available banks (issuers)
List of available banks (issuers)
List of available banks (issuers) Example redirect to donors bank

iDEAL Recurring

https://docs.adyen.com/payment-methods/ideal/web-component#recurring-ideal-payments

iDEAL recurring has a unique flow as Adyen sends the recurring token on an IPN after the donor finishes the transaction. The donation isn't immediately put onto the donations queue.

From the donor's perspective the flow is exactly the same as a one time donation but on our end we don't complete the donation until we get a RECURRING_CONTRACT IPN that has the recurring token on it.

RECURRING_CONTRACT IPNs are set for every new recurring so we filter out just the iDEAL recurrings, find the matching pending database entry, add the recurring token to the message and send it to the donations queue.

Recurring iDEAL donations are of the type SEPA Direct Debit.

p.s. iDEAL do not need separate capture, but the recurring iDEAL is using SEPA, and have the config from Adyen side that control if auto capture or no.

Apple Pay

Apple Pay is available on Safari with payment_method=apple

It replaces the credit card form with a Donate to Apple Pay button as seen below. When the donor clicks on it it brings up a sheet (I think this is what its called?) that lets the donor choose which card and contact information to use.

Using Apple Pay in Safari)
Using Apple Pay in Safari)
Using Apple Pay in Safari Choosing card and contact information

In App (Native) Donations

We have two endpoints to handle in app donations, getPaymentMethods and submitPayment

getPaymentMethods

This takes in the country and returns the available payment methods. The app caches this data for ~1 month.

params

country (two letter country code - US)

/api.php?action=getPaymentMethods&format=json&country=US

Returns:

response (all the payment methods available for that country)

Example response: country=US

{"response":{"paymentMethods":[{"brands":["visa","mc","amex","discover","cup","maestro","diners","jcb"],"name":"Credit Card","type":"scheme"},{"brands":["visa","mc","amex","discover","maestro"],"configuration":{"merchantId":"000000000201454","merchantName":"WikiMedia"},"name":"Apple Pay","type":"applepay"},{"issuers":[{"id":"jp","name":"Apple Pay"},{"id":"ap","name":"Google Pay"},{"id":"cs","name":"\u010cesk\u00e1 spo\u0159itelna"},{"id":"kb","name":"Komer\u010dn\u00ed banka"},{"id":"c","name":"P\u0142atno\u015b\u0107 online kart\u0105 p\u0142atnicz\u0105"},{"id":"dpcz","name":""}],"name":"Online Banking CZ","type":"onlineBanking_CZ"},{"configuration":{"merchantId":"50","gatewayMerchantId":"WikimediaDonations"},"name":"Google Pay","type":"paywithgoogle"}]}}

submitPayment

This takes in a token generated in the app as well as the rest of the donation parameters and creates the payment.

Params

amount

app_version

banner

city

country

currency ( USD, CAD, etc)

donor_country (2 letter code US)

email

first_name

full_name (we can handle this on our end if the separated name fields arent available)

language

last_name

opt_in (If they agreed to emails or not , 0 or 1 )

pay_the_fee (if they agreed to pay the fee, 0 or 1 )

payment_method (applepay, googlepay)

payment_network (visa, mastercard, etc)

payment_token (from apple)

postal_code

recurring (1 for yes, 0 or null for no)

state_province (2 letter code CO)

street_address (123 Street Drive)

Returns

order_id

gateway_trasaction_id (not sure if you need this, ADYEN123)

status (success or error)

error_message: (will return the message localized in the language passed through)

Example response

Success

{"response":{"status":"success","gateway_transaction_id":"123456TEST82","order_id":"1235.1"}}

Error

{"response":{"status":"error","error_message":"There was an error processing your request. Error reference: 1234.5","gateway_transaction_id":null,"order_id":”1234.5”}}

Apple Pay Testing Links (TBC)

Google Pay

Spike: https://phabricator.wikimedia.org/T244513

Sandbox test:

google test image

  • Use https://groups.google.com/forum/#!forum/googlepay-test-mode-stub-data to join test group, then you can test google pay with test card implemented by google already
  • Currently support payment methods are inside the `/src/payments/extensions/DonationInterface/adyen_gateway/checkout/adyen_checkout.adapter.php` as getGoogleAllowedNetwork, includes 'AMEX', 'DISCOVER', 'JCB', 'MASTERCARD', 'VISA'

Recurring

Recurring Adyen donations are tokenized and are charged on our schedule by the SmashPig Civi extension's recurring payment charge job. Adyen recurring donations require two identifiers:

  • Standard payment token, stored in the civicrm_payment_token table and called recurring_payment_token in queue messages and SmashPig parameters, referred to as 'storedPaymentMethodId' in the Adyen API
  • Shopper identifier, stored in civicrm_contribution_recur as invoice_id and called processor_contact_id in queue messages and SmashPig parameters, referred to as shopperReference in the Adyen API

Auto Rescue

Project phab: https://phabricator.wikimedia.org/T340159

https://docs.adyen.com/online-payments/auto-rescue/cards/

Config

There are two config settings, enabling and the amount of days to rescue for, defaults in Smashpig adyen's main.yaml

enable-auto-rescue: true

max-days-to-rescue: 16

Subscription Models

We have three different situations of recurring/non recurring models in the Adyen console

One time donation with no monthly convert

Shopper interaction - Ecommerce


One time donation with monthly convert or the first payment of a recurring

Shopper interaction - Ecommerce

Recurring contract RECURRING,ONECLICK

Recurring processing model Subscription

Subsequent recurring charges (not the first one)

Shopper interaction ContAuth

Recurring contract RECURRING

Recurring processing model Subscription

IPN messages

https://docs.adyen.com/development-resources/webhooks

We continue to receive Adyen IPN messages in SOAP format, as with the HPP integration. Because with Checkout we are able to capture card payments directly from payments-wiki, we have disabled the asynchronous capture flow that we were using with the HPP integration. Code for async capture is still present in SmashPig but is disabled in config/adyen/main.yaml by setting capture-from-ipn-listener: false. We should probably remove that code and config variable as part of T291448.

The IPN messages have an eventCode that we use to identify them and then either do something or ignore the notification.

List of possible eventCodes here: https://docs.adyen.com/development-resources/webhooks/understand-notifications

RECURRING_CONTRACT

This notification comes in when a new recurring payment is created. Right now we are ignoring all of them except recurring iDEAL.

OFFER_CLOSED

iDEAL only When a Pending/Received payment is not authorized.

When Adyen returns a Pending or Received result from a redirect: "The shopper has completed the payment but the final result is not yet known. It may take minutes or hours for the iDEAL payments network to confirm this."

The final result of the payment: "You will receive the final result of the payment in an AUTHORISATION notification. If the transaction is not authorised within 30 minutes, the offer will close. Optionally, you can be informed of this through an OFFER_CLOSED notification"

Audits

We are using two different reports for the Adyen audit. The original report used was the weekly file that comes out on friday, this is called the Settlement Detail Report. The nightly file was added afterwards and is called the Payment Accounting Report.

We get an IPN message with the report path and download it using a DownloadReportJob under the jobs-adyen job queue runner.

Files are named settlement_detail_report_batch_1.csv where the number increases and payments_accounting_report_2023_01_1.csv.

We process them with the wmf_audits drupal module that calls functions from SmashPig to parse the CSVs.

Adding data to the reports

Adyen lets us control what columns are in the reports. To see this configuration you need to be on our company account level in the Adyen console. Once you are on the company account, Settings->Report columns should appear on the left menu.

File scheduling

The schedules for the report creation are on the company account level under Reports on the left menu. There are also reports available on the merchant level account but they are not all there. Each report can then be set to Automatic generation, some have specific schedules you can set and some are determined by Adyen.

Pending Table

The DonationInterface Adyen Checkout adapter as of November 2021 sends duplicated messages to the Pending queue. It might not need to send any messages for card payments, as that queue is mostly used for pending transaction resolution, and that scenario is much less likely under Adyen Checkout due to us making the payment authorization and capture in the same HTTP request where we get the donor data.

Jobs Queue

The IPN listener sends 'RecordCaptureJob' messages to the Jobs-Adyen queue. These jobs combine information from the IPN message with information from the pending queue and send information to the donations queue.

There is also RecurringContractJob which are used for creating new iDEAL recurrings. They combine the recurring token from the IPN message from the original donation information in the pending queue and send the combined information to the donations queue.

For overall payment queue can go to server, then run `drush --user=1 -v -r /srv/civi-sites/wmff/drupal qc` to consume the queues

Testing

https://www.mediawiki.org/wiki/Fundraising_tech/Testing#Adyen

Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=Fundraising/Data_and_flow/PSP_integrations/Adyen_Checkout&oldid=2163304"