Invalid Response Format error is issued in the following situation:
(<method name>) method has an invalid Response Format property. Either change it, or remove the output parameters from the method
You have an invalid value in the 'Response Format' property of a REST API method, or you don't have any value set at all.
Double-click on the error line to take you directly to the REST API method's 'Response Format' property where the problem was detected.