An analysis of semver compliance in Rust finds accidental semver violations are common even in the most popular crates. Human error is not the cause, and better tooling is the way forward.
semver is a nice way to communicate your intent to other humans, but it’s never been a great way of programmatically communicating those changes. If you want it to communicate something meaningful programmatically, you’d have to have the version generated automatically, and not rely on a human to do so.
Semver was literally made to communicate programmatically. But people keep using it for “human” communication instead, like the whole “1.0 means stable” thing.
It was made to do so, but failed from the start exactly because humans got involved. semver’s ideals can only happen when tooling generates the version number, not humans.
How can a machine decide if something is a patch, minor, or major release? I guess a major release could be defined by a comprehensive unit test suite breaking, but the others are very much something humans should decide.
Imo, the solution is that people need to be less afraid of major release bumps. Do it frequently and it’s not likely to cause issues downstream.
It is quite simple conceptually to decide. Simply examine all the functions of the API between two versions. If the signatures are equivalent, increment the patch version. If there are new signatures, but the existing ones are the same, it’s a minor version. If any function signatures change, it’s a major version.
Then you also need to examine trait impls and such but the basic idea is the same.
That oversimplifies and misses a lot of edge cases, such as:
change the meaning of an existing parameter - e.g. an Integer timeout changes from seconds to milliseconds
dependency changes, and this package exposes exposes types from that dependency
internal refactor changes the order of execution of existing uses (say, a scheduler change in an async library, or event order in a GUI library)
Each of those would be, imo, a breaking change, but an automated semver tool would probably mark them as patch releases. I could come up with more examples, but hopefully the point is clear.
Maybe it’s correct 90% of the time, but the last 10% of the time can be really impactful. I think there should be less sigma against major releases. If in doubt, mark it as a major release.
change the meaning of an existing parameter - e.g. an Integer timeout changes from seconds to milliseconds
Ideally you should change the type if you do such a thing, which would cause it to become a breaking change. In this specific instance, you should take std::time::Duration obviously.
dependency changes, and this package exposes exposes types from that dependency
I think that should be automatically detectable?
But yes you’re right, in general it’s not possible to detect all forms of semver changes. But perhaps at least detecting violations when they weren’t meant to be there would be good.
And yea 100% agree with you, people should use the power that is 2.0.0 way more than they currently do.
Ideally you should change the type if you do such a thing
But that’s not always possible or desirable. For example, maybe you’re largely just passing it through to another library, and that library made a breaking change.
So I don’t think we should have semver always handled automatically, it should instead prompt the developer with its best guess, and the developer would ideally only move from patch -> minor, patch -> major, or minor -> major, and never the other direction. But the developer should always be involved in picking the version. So don’t just throw it into CI and call it a day, but instead have a CLI tool that suggests it and requires developer approval before making the PR for the version bump.
I gave a more thorough response with examples to the other user that replied (link on my instance here, but basically there are cases where you could break someone’s code with a patch release.
I’m completely fine with using tools to help decide what versions to assign, but in general developers should not hesitate to increment the major version if there’s any doubt.
Is that even possible though? Sometimes you need a human to understand if something is a breaking change.
Imagine an API like fnthird_planet_from_sun() ->String, and an update is made where the output changes the value to be lowercase instead of capitalized. That should normally be considered a breaking change.
However, imagine fncurrent_version() ->String. That is by its definition meant to change outputs between versions, so it isn't a breaking change since that's part of its human, documentation based API contract.
Also, what if somw function which returns a String changes, but only one code path that is very hard to hit changes the output? How would a machine find that?
I guess the first example with Earth / version could use some attribute macro so devs can say the output is expected to change across versions, but then there is no way for a program to know what is a breaking change vs expected vs a bug.
To do it 100% probably isn't possible, something something halting problem. However, you'd catch a lot of basic mistakes with proper typing. In your example, the first function should be typed like this: fnthird_planet_from_sun() -> Planet, where Planet is an enum. De/serializing it still has the same problem of interpreting an arbitrary string, but at least for deserializing it, you can be loose in what you allow and just lowercase it before matching it to the enum.
semver is a nice way to communicate your intent to other humans, but it’s never been a great way of programmatically communicating those changes. If you want it to communicate something meaningful programmatically, you’d have to have the version generated automatically, and not rely on a human to do so.
Semver was literally made to communicate programmatically. But people keep using it for “human” communication instead, like the whole “1.0 means stable” thing.
It was made to do so, but failed from the start exactly because humans got involved. semver’s ideals can only happen when tooling generates the version number, not humans.
How can a machine decide if something is a patch, minor, or major release? I guess a major release could be defined by a comprehensive unit test suite breaking, but the others are very much something humans should decide.
Imo, the solution is that people need to be less afraid of major release bumps. Do it frequently and it’s not likely to cause issues downstream.
It is quite simple conceptually to decide. Simply examine all the functions of the API between two versions. If the signatures are equivalent, increment the patch version. If there are new signatures, but the existing ones are the same, it’s a minor version. If any function signatures change, it’s a major version.
Then you also need to examine trait impls and such but the basic idea is the same.
That oversimplifies and misses a lot of edge cases, such as:
Each of those would be, imo, a breaking change, but an automated semver tool would probably mark them as patch releases. I could come up with more examples, but hopefully the point is clear.
Maybe it’s correct 90% of the time, but the last 10% of the time can be really impactful. I think there should be less sigma against major releases. If in doubt, mark it as a major release.
Ideally you should change the type if you do such a thing, which would cause it to become a breaking change. In this specific instance, you should take
std::time::Durationobviously.I think that should be automatically detectable?
But yes you’re right, in general it’s not possible to detect all forms of semver changes. But perhaps at least detecting violations when they weren’t meant to be there would be good.
And yea 100% agree with you, people should use the power that is 2.0.0 way more than they currently do.
But that’s not always possible or desirable. For example, maybe you’re largely just passing it through to another library, and that library made a breaking change.
So I don’t think we should have semver always handled automatically, it should instead prompt the developer with its best guess, and the developer would ideally only move from patch -> minor, patch -> major, or minor -> major, and never the other direction. But the developer should always be involved in picking the version. So don’t just throw it into CI and call it a day, but instead have a CLI tool that suggests it and requires developer approval before making the PR for the version bump.
@sugar_in_your_tea If you’re interested, I recommend looking at how Elm does it. Elm has automatic semver enforcement in its package system.
The long and the short is
missing stuff: major change
new stuff: minor change
patch: internal implementation change.
I gave a more thorough response with examples to the other user that replied (link on my instance here, but basically there are cases where you could break someone’s code with a patch release.
I’m completely fine with using tools to help decide what versions to assign, but in general developers should not hesitate to increment the major version if there’s any doubt.
Is that even possible though? Sometimes you need a human to understand if something is a breaking change.
Imagine an API like
fn third_planet_from_sun() -> String, and an update is made where the output changes the value to be lowercase instead of capitalized. That should normally be considered a breaking change.However, imagine
fn current_version() -> String. That is by its definition meant to change outputs between versions, so it isn't a breaking change since that's part of its human, documentation based API contract.Also, what if somw function which returns a String changes, but only one code path that is very hard to hit changes the output? How would a machine find that?
I guess the first example with Earth / version could use some attribute macro so devs can say the output is expected to change across versions, but then there is no way for a program to know what is a breaking change vs expected vs a bug.
To do it 100% probably isn't possible, something something halting problem. However, you'd catch a lot of basic mistakes with proper typing. In your example, the first function should be typed like this:
fn third_planet_from_sun() -> Planet, wherePlanetis an enum. De/serializing it still has the same problem of interpreting an arbitrary string, but at least for deserializing it, you can be loose in what you allow and just lowercase it before matching it to the enum.