Testing/validating RAML vs implementation


#1

I’m looking to test and validate that my API specs match how the API code has been implemented. I’ve played with the abao project some, but I don’t think it fully does everything I want. So I’m curious what others are doing. Ideally, the tool I’m looking for needs to validate the following

  • Does the endpoint exist
  • Correct responses are returned
  • Responses match the specified schema format
  • Request parameters match what’s required
  • and more.

In my testing with Abao, I haven’t seen it validate the response content structure matching that of the RAML. And I wasn’t sure if it verified the request part as well or not - haven’t delved that far yet. Are the hook files required to do that? From my viewpoint, it looks like the hook files are more or less just to setup data to test against, which I’m just looking for schema and code validation - the data piece isn’t as important (yet).

I very much like the concept of the project and it’s direction. (I had been researching a similar option for Swagger or other API documentation formats but hadn’t really seen anything that would compare the API spec to the actual code/server.)
Thoughts?


#2

ABAO seems to be the best tool for what you are asking, have you reached out to the team to determine if they are adding features and/or can otherwise instruct how to achieve your needs with what they have now? Another idea… as I believe is is open source, you could fork the code and look to add some your needs yourself maybe and contribute it back? I have not used ABAO yet but looking to see if it fits our needs down the road as well. I am hoping the team will update it soon for RAML 1.0 support.


#3

Have you tried testing your API using the code genereated by scraml? Because it generates a typesafe DSL, you will get failed tests as soon as the RAML specification doesn’t match your implementation, see this article. It’s still on RAML 0.8 (using json-schema), but work is under way to support 1.0.


#4

Hello,

I wrote a testing tool called Vigia in ruby. It does support RAML files and it can be easily extended using RSpec both normal and shared examples.

you can check it out on github: https://github.com/nogates/vigia