Here is the context:
The problem:
In addition, I have disabled cache for the authorizer.
What might have caused this issue?
This could be fixed in two ways that are described in buggy's answer: https://forum.serverless.com/t/rest-api-with-custom-authorizer-how-are-you-dealing-with-authorization-and-policy-cache/3310
Short version:
I've tried each solution and they both solved the issue with "User is not authorized to access this resource" for me.