How to unify all SDK generators


#1

I am hoping we can come up with some form of set of rules or guidelines for those wanting to add a language or framework within a language to the generator process. It would be difficult if not impossible for a company to use .NET, Java, Ruby and Python SDK generators that we build here if each of the generated SDKs have different namings for methods/functions, etc. Naturally having the community build additional generators is a bonus for all of us wanting to have RAML to , but it doesn’t benefit anyone if a .NET generator puts out a completely different SDK syntax than the Java generator. I say this without having good knowledge of .NET, python or ruby… so I suspect there may be some circumstances that may not make the SDKs all “nearly” identical. But things like naming of methods/functions, parameter passing, etc should be consistent.

There is another caveat I am not quite sure how to address. Models. RAML supports XSD and Json Schema. How do we handle the pojo generation of XSD and Json Schemas in different languages such that to stick to the above SDK consistency across language generators, the POJO generation process does not hamper the ability to stay consistent?

I’ve dabbled a wee bit with python POJO generation and at least with XSDs it was possibly to have similar named/created POJOs in python and java. I suspect that .NET will be similar, and I am assuming Ruby and possibly Go as well? Anyone care to enlighten this thread if you know about the respective language and how it handles XSD or Json Schema to Object creation?

Also, maybe not on this thread… but it would be good to start listing out the various “libraries” for each language that can be used for POJO generation. For example, in Java XSDs are typically converted to POJOs with JAXB (often the Jackson library). Json Schema seems to be best supported with a project call JsonSchema2pojo. The Jax-Rs project is using that and I have used it as well for my initial foray into Java SDK generation.

Lastly (for now…) how does this nodejs project “Call out” to the various languages to generate POJOs? I don’t know if the current project has support for converting XSD/JSonSchema models into JSON objects for it’s generate javascript or not? But we’ll definitely need some way for each added language/framework generator to be able to call out to the command line to execute the POJO generator portion. Or… do we not do that at all (since this is not compiling code…just generating output) and leave that to some other process? I personally would like to know that this project can generate SDKs completely… possibly including bundling them up as artifacts ready to be shipped, maybe as zip files or tar.gz or something? But maybe that is asking too much of this project?

Look forward to the coming discussions!


#2

Lastly (for now…) how does this nodejs project “Call out” to the various languages to generate POJOs?

You can easily spawn a child process inline by writing a handlebars helper, or we could add a pre-process step to generation of data too. It doesn’t exist right now, but it’s easy to add.

I personally would like to know that this project can generate SDKs completely… possibly including bundling them up as artifacts ready to be shipped, maybe as zip files or tar.gz or something?

There’s nothing wrong with adding that, but it might not be the right place since any tool can bundle it into a zip. The more options we add, the more we need to maintain. Right now, we have folder output and that folder could then be zipped. We should definitely be generating all assets needed for bundling though. For example, the JavaScript project comes with a package.json and .gitignore.

There is another caveat I am not quite sure how to address. Models. RAML supports XSD and Json Schema.

The JavaScript client doesn’t actually use this information when parsing the API responses since it’s weakly typed and we want to get as much information from the response as possible. For example, testing outputs that don’t conform to the RAML, etc. Of course, certain languages won’t have this ability - but just an FYI.


#3

This sounds awfully similar to what we do at apimatic.io


#4

I agree zeeshan. It would be fantastic if you and/or your team could contribute to this project as well. It will benefit everyone in the RAML community.


#5

@zeeshan Are you guys also chaining methods, or how do you generate method names? Can you share your experience with us?


#6

Hi, in my opinion I would prefer to have and API were I can add new generators or modify the templates, parsers and helpers, just like #blakeembrey said, so you can customize what really needs. I already had a generator that works on this way https://github.com/gextech/raml2code.

Our client generator doesn’t do the chain it generates on this way:
getCatalogAndSizeAndFrom
getAndAnd(<@Path pathVariables)

it is verbose but it works great with retrofit http://square.github.io/retrofit/


#7

Hi @atomswork

I am all for using the same template engine if it works. I am unclear right now how to use handlebars.js to build out subdirectories and such at this point, so I would definitely say that we need to provide a LOT more documentation around this particular process. Sooner than later too, so we can get more people involved right away. The use of handlebars.js seems fitting if it can be used to template any language and code generator. I don’t know it nearly well enough nor can I find much in the way of resources presently that explain advance concepts like using handlebars.js to generate files in various sub directories, and how you handle things like compile-time generated objects ( pojos in the java world) such that you would include them in the generated sdk. Using a tool like Java Poet would allow for XSD or JSon Schema generated pojos to be properly referenced when parsing the RAML file and first converting XSD or Json Schema to pojos, then referencing those pojos when creating the classes that handle them as request bodies or response bodies. But with a templating system like what handlebars.js provides, I am not sure how you ensure the right naming and other issues when part of the handlebars.js templates will need to “shoe in” code that is generated by another tool. It’s almost as if I would need to generate the other POJOs first, then after I know the exact package/class names, can then write the templates to use them. If those POJO names change… they wouldn’t get automatically updated in the handlebars.js template code…that would need to be a manual process… something I am hoping to avoid. I am not saying it’s not possible, just that I can’t find anything on handlebars.js that explains how to use it the way the codegen project is. I love the idea, just can’t seem to grasp it well enough just yet.

Do you have some good resources on handlebars.js and how to use it for a noob that doesn’t understand node/npm/handlebars.js?


#8

Hi,

Yeah, handlebars works really well indeed in https://github.com/gextech/raml2code it is the core of the project, we solved the problem of the path with gulp.js.

Basically raml2code it’s a gulp plugin and it’s main responsibility it’s to orchestrate the input streams (RAML file) and the output (Files generated).

Please review this file:

Also all the handlebars things is done whit this:


It’s main responsibility it to handle partials, templates, helper sand partial, and a parser.

The combination of partials, templates, helper sand partial, and a parser is a generator.

For example this is our generator for pojos:
https://github.com/gextech/raml2code/blob/master/src/generators/groovy/jaxrsInterface.coffee

If you see:

generator.template = require("./tmpl/jaxrsResources.hbs") –> Template
generator.parser = (data) -> Parser

The parser function return a object where each key represent a File
result[“POJO.groovy”] = model

and then raml2code write to file here:

And all this stuff is done, to make easier to write generators, and just worry about parsers and templates.


#9

Nicely done! I suspect you need someone that knows JavaScript + HandlebarsJS very well to achieve this though?


#10

Hi… looking at your source… you have this:

{{#if extra.importPojos}}
import {{extra.importPojos}}.*;
{{/if}}

Does this get replaced with 1 or more lines in this one call? I ask because you have the import outside of the {{ }} so it doesn’t strike me that multiple import statements would be inserted here.

What would be really good info to know is some good tutorials on Handlebars.js that maybe you (and @christian_vogel and others) used. I get the gist of the code, but there is still a bit of a learning curve for someone that hasn’t done any sort of JS code (ok I lie…I did back in the mid 90s before the likes of prototype, jquery, etc came about).

Thanks.


#11

Could you please elaborate more, this are metadata that it’s needed to provide a the java-client with the package where the domain objects are.


#12

I’ll try… what I mean is… is that one construct a “loop” that fills in multiple import statements… or does it only end up with 1 import in the output file? I really should know/understand Handlebars.js before asking these as I suspect that understanding would answer this… I just was looking through the source and was really curious how you generate POJOs without something like XJC, and then how you use the generated POJOs within the generated SDK classes.


#13

it its just a conditional


#14

Ok… so I am guessing you build up a string of import statements and that conditional will replace the token with all the import statements? Anyway, I see Handlebars.js 3.0 came out recently. However, it seems that the latest book I can find dates back to 2013… I fear it is not the ideal book to use to learn Handlebars.js with because it’s out dated by quite a bit. The online tutorials are limited. Any other links/tutorials you can recommend to pick up Handlebars.js?