Public roadmap for each upcoming TAP specification version

[kinow]

Create a public roadmap of features/issues to be addressed in each upcoming TAP specification version.

[kinow]

We should decide where this information will be available. My initial guess is that a good place would be somewhere in the website, but open for suggestions.

[jonathanKingston]

I should have actually told people my plan rather than just assume people knew.

My plan was to make a milestone and pin approved issues to it:
https://github.com/TestAnything/Specification/milestones

The website seemed a little formal until there was enough ground work in place to show.

When issues were agreed then it would be worked on in the branch:
https://github.com/TestAnything/Specification/tree/tap-14-specification

My aim was to keep 14 fairly light to get most parsers in line with being stricter and more specified.
I was thinking the more abstract features would be in 15, this could include subtests and other features that would break much older parsers.

[kinow]

Oh, hadn't considered GitHub milestones as an option @jonathanKingston . We can use that :-)

My aim was to keep 14 fairly light to get most parsers in line with being stricter and more specified.

Great idea, I think it's inline with what we are discussing in #2 then. +1 for your plan then. Feel free to close this issue or update it with any other information. Not sure if anybody else would like to comment here too.

[jonathanKingston]

@kinow I will wait for further comments as I would like to get an 'official' approval process down. TAP is an old format and I don't want to cause any upset at all. I don't actually think there is much room for 100's of features in TAP its merit is its simplicity.
I think that once subtests are finally standardised and YAML sub structures there isn't much room for other features so realistically we don't need some committee or anything heavyweight.

My original idea was to lock in the features into the milestones then mail out to all in the TAP mailing lists and other bits. If there is great opposition to the changes then I fear a fork might have to be on the cards (which isn't what I want but I would prefer not to upset the many many people involved in the history of TAP).

[kinow]

I think that once subtests are finally standardised and YAML sub structures there isn't much room for other features so realistically we don't need some committee or anything heavyweight.

Indeed. Once we define subtests and YAML, a new release would be really close I think.

My original idea was to lock in the features into the milestones then mail out to all in the TAP mailing lists and other bits.

Another great idea, my +1.

If there is great opposition to the changes then I fear a fork might have to be on the cards (which isn't what I want but I would prefer not to upset the many many people involved in the history of TAP).

I also don't want a fork, but I think we won't have to think about that :-)

[jonathanKingston]

@kinow great we seem to be on the same/similar page.

What did you think to having a point increase from 13, so 13.1 being a more specified version of the same format. Breaking changed could be pushed into 14.

[kinow]

What did you think to having a point increase from 13, so 13.1 being a more specified version of the same format. Breaking changed could be pushed into 14.

I'd be -1 on this; since we had 13 releases, having the first point increase to be 13.1 would be quite strange (at least for me). Though if there are enough good arguments for it, I'd change my vote to -0 and join the pool and push a new 13.1 release :-)

[jonathanKingston]

I was thinking the same about it potentially causing issues with it being a point release so was going to be followed by a 14.0 or 14.0.0 suggestion but I expect the patch identifier would be pedantic at the rate that TAP moves.

My rationale for 13.1 is that it's a clarification on 13 rather than any features (in fact the only one would be adding in points into the version) and that a version increment that has points would likely cause the same issues with parsers if it were 13.1 and 14.1. So 13.1 seemed the most correct.

I think there would be benefit in full semver versioning to allow for clarifications in future of the specification to go through as patches.

[kinow]

These are good arguments. I still think it would be confusing for users, but maybe after a 13.1 release, users could get more used to this new version schema. Not sure if that's common for specification (I could check some W3C specifications that I use). If there's a consensus on 13.1 based on your arguments, then I'm fine with that.

[jonathanKingston]

@kinow W3C works on the full version basis for things like HTML when they become standardised, they sort of embrace the living standard approach for the specs until they are recommended.
They have embraced the modular approach for CSS but TAP isn't anything the same size or structure as anything W3C are going for.

HTTP, TCP and so on are arguably a closer standards, HTTP embraces point increments where as TCP doesn't. It isn't clear on what the best structure is for version strings here. The only reason for me to deviate is allow 13.1 to be the catch up specification to clear up rustier older implementations and brush up resources to provide better information on how implementations should behave.
The freedom then would allow for further specification in 13.2 for example to add stricter parsing and edge case behaviour clarifications.

Perhaps version 14 being the more specified and 15 being the included features mentioned is easier to stomach.

As I often say, I don't really care about the implementation we just need to pick one and stick with it.

[Leont]

Indeed. Once we define subtests and YAML, a new release would be really close I think.

Agreed.

[Ovid]

A point release will also break things. The Test::Harness::Grammar considers version numbers to be integers. This email thread has some of the historical discussion on that,

Also note that Schwern was arguing that we will need to break compatibility at some point in the future, but that was before TAP was so widespread outside of Perl and today, I think there's not widespread agreement on that point. (And I admit I could be a bit paranoid about the topic).

[kinow]

A point release will also break things. The Test::Harness::Grammar considers version numbers to be integers. This email thread has some of the historical discussion on that,

Now that you mention.. I think I read in the old web site that a TAP header would be "TAP" "Version" < INTEGER >... I think tap4j uses a Regex with some \d+ there.... though that's not so hard to fix there, indeed it would add break things for others.

[jonathanKingston]

@Ovid That was my initial thought that it would break some parsers.

So if it were to be a point release it would have to be to 14.0. Otherwise just 14 would suffice. However I do like the approach suggested in the mail which largely follows the course of other languages and as I mentioned earlier 14.0.0 might be a better suggestion if change were to happen to keep inline with semver which is now pretty widespread.