Error / Incident
This alert means that our terraform tests failed to apply, this is, create all the resources.
We run the terraform tests from the machine
tf-bastion.tf-infra-test.eqiad1.wikimedia.cloud, that regularly try to apply (create) and then destroy a collection of resources on cloud vps.
Ssh to the
tf-bastion.tf-infra-test.eqiad1.wikimedia.cloud and try running the creation manually, that will give you more logs.
NOTE: if you are not member of the project, you can try sshing as root, or using the console cookbook (
dcaro@urcuchillay$ ssh tf-bastion.tf-infra-test.eqiad1.wikimedia.cloud dcaro@tf-bastion$ sudo -i root@tf-bastion:~# cd tf-infra-test/ root@tf-bastion:~/tf-infra-test# terraform apply data.openstack_images_image_v2.debian: Reading... ... Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes ...
If you know which resource failed, you can try also applying that specific resource like this:
root@tf-bastion:~/tf-infra-test# terraform apply --target=openstack_db_instance_v1.postgresql
Add here any new common issues you find.
Quota errors when creating the trove databases
There's several quotas attached to trove databases:
- Trove's regular quotas
It might be that you are hitting your own project's
database quotas or torve is hitting it's own.
Quota usage and requests out of sync
It happened once that the quota that trove registers as 'requested' and the one it registers as 'used' get out of sync, it seems that the requests get lost somehow and it does not decrease the 'requested' counter.
You'll see that as a regular quota error, to debug and fix see Portal:Cloud_VPS/Admin/Trove#Reserved_quota_does_not_go_down.
Communication and support
Support and administration of the WMCS resources is provided by the Wikimedia Foundation Cloud Services team and Wikimedia movement volunteers. Please reach out with questions and join the conversation:
Add here any new tasks for incidents you might encounter.