Scope and URL mismatch for Delete a Follower from Organization

Hi, yet another Scope and URL mismatch - this time for /api/v1/organizations/1/followers/97

My OAuth has contacts:full scope and I can successfully add a follower to an organization.
I can also delete a follower from an organization using API Key
But I cannot delete the follower from organization using the same OAuth I have for creating the follower.
Is there any other scope or OAuth setting needed for deleting a follower from Organization?
Thanks