So we have been heavily using WebHooks to synchronize data between our e-commerce system and Pipedrive. It is fairly standard that when an operation happens, the status is then reported as a status code in the table of WebHooks in the UI as such:
Obviously 200 being OK, 500 being server error - on our side. Easily understandable. However - recently a new kind of message started appearing:
Along with some WebHooks just not being called at all without any notice anywhere in the system - notification, e-mail…we checked out logs, we checked our load balancers and there simply aren’t ANY records of this Error call. The support is absolutely hopeless, giving statements that “they will check”, “they are waiting” and literally nothing is happening. Meanwhile, status page is lit perfectly green, not indicating any suspicious activity, while WebHooks are going bonkers.
What does this mean, other than - “this functionality should not be in any production systems”? Is there any hope we could see the logs for the WebHooks?
From looking into our logs, it seems there several timeouts (10s) to one of your webhooks which caused it to get banned (and thus the error). Would you be able to take a look on your end as to what may have caused the timeouts?
After several weeks of back and forth the problem has been (hopefully) identified, although it needs to be addressed from your side as well:
When these status codes appear:
it’s perfectly clear to the engineer/sysadmin what had happened - 200 is a success, 500 is a problem on the server of the client.
However, when this appears:
where do we even start? Oh, that’s right, the documentation. Except - there’s nothing explaining what this cryptic “Error” could possibly mean - is it an error on our side? Is it an error on your side? Is it a code issue, auth issue, permission issue? Just dumping a simple “Error” with support knowing absolutely nothing (because, they - obviously - read the documentation, which has nothing on this topic) is like turning off the lights and locking the door.
Thankfully, I come with solutions as well:
1. The “correct” solution
Status code 408 Request Timeout when the webhook request times out while trying to reach the client’s URL. Here is an RFC documentation if you’re pondering the legitimity of this approach.
2. The less correct solution, but still better than nothing
Adding a section in the documentation stating that when a webhook is timing out or being banned, a big red “Error” will appear in the webhook status dashboard.
Either way will save loads of time and energy of everyone involved. Thanks.
Thank you for your valuable feedback. I’m talking with our engineering team and our technical writer to help clear up the documentation as well as see if we can make the error messages more coherent or at least informational.
I agree, as it stands, it is quite difficult to grasp what the actual error is.
I’m talking with our engineering team and our technical writer to help clear up the documentation as well as see if we can make the error messages more coherent or at least informational.