Examples validations in RAML parsers


#1

I have a couple of questions for the community regarding examples validations in RAML

  • Should RAML parsers validate examples?
    RAML defines that schemas must be either of, an XSD, a JSON schema, or the name of a top level schema
    But there is no restriction for examples, they an be anything, random text, documentation…
  • How should a parser infer the type of the example? Since there is no restriction, any inference may be wrong.
  • What if the body has not defined a schema, but has an example? ==> Do not validate, should a parser warn users here?
  • Is the whole RAML invalid if an example is malformed / does not conform to the schema? Should parsers throw an error or a warning on an invalid example?

#2

I think that examples should be validated and throw an error if they are invalid given the following conditions:

  • a schema is provided
  • the schema is json-schema and media type is either application/json or ends with +json
  • the schema is xsd and media type is either text/xml or application/xml or ends with +xml

I don’t think we should raise warnings if these conditions are not met, we may potentially get lots of undesired ones


#3

I mostly agree with your points, but for this to work, the spec should say that examples MUST conform to the schema provided (a point that is important and is missing currently).


#4

I feel like declaring the RAML invalid if a schema is malformed might be taking it a little bit far. This might be a case where we want to throw a warning instead of an error? I don’t feel too strongly about which direction we should go, but lets try to get some more opinions if possible!

Also - whatever direction RAML takes in 0.9, let’s ensure consistency in the behavior between body examples and parameter examples.


#5

Agreed. Let’s add it to 0.9 version of the spec.


#6

The spec clearly states which version of JSON schema and XML schema all parsers must support in the following statement:

All parsers of RAML MUST be able to interpret JSON Schema
[JSON_SCHEMA] and XML Schema [XML_SCHEMA].

It doesn’t say that it is invalid to use other schema types or rendering them invalid if they don’t conform, but I think it is an oversight rather than the intent here. The more accurate the information provided in the RAML the better, and validating schemas and examples are steps in that direction. RAML files in the wild with broken schemas and broken examples only hurt the creditability of the specification.

Client generators will break, API servers will break, and more importantly, people doing cut and paste from examples will curse the API in question. Let’s be proactive here and ensure people document their APIs the proper way. After all you are not obligated to add an schema, but if you do let’s make sure we validate that schema conforms to something that everybody understands.

Let’s get this addressed for 0.9.


#7

Hi
Also I thought a getSchema method should be provided for Response object.
For the below example taken from raml spec, I would want a method to expose the schema for the 200 response.
Any thoughts?

#%RAML 0.8
baseUri: https://api.example.com
title: Filesystem API
version: 0.1
schemas:
  - !include path-to-canonical-schemas/canonicalSchemas.raml
  - File:       !include path-to-schemas/filesystem/file.xsd
    FileUpdate: !include path-to-schemas/filesystem/fileupdate.xsd
    Files:      !include path-to-schemas/filesystem/files.xsd
    Dir:        !include path-to-schemas/filesystem/dir.xsd
    Dirs:       !include path-to-schemas/filesystem/dirs.xsd
/files:
  get:
    responses:
      200:
        body:
          application/xml:
            schema: Files

#8

There is a stand-alone example validator, ramlev.