command: make script-binding command scalable

[na-na-hi]

This makes script-binding command and key bindings added by mp.add_key_binding() scalable. The script-message command now has an extra argument with the scale of the key.

[github-actions]

Download the artifacts for this pull request:

Windows

• mpv-i686-w64-mingw32
• mpv-x86_64-w64-mingw32
• mpv-x86_64-windows-msvc

macOS

• mpv-macos-13-intel
• mpv-macos-14-arm
• mpv-macos-15-arm

[avih]

The script-message command now has an extra argument

Did you mean script-binding? I don't think I see at the changes anything specific to script-message, but I do see changes to script-binding.

If it's still script-message, then at which level is the extra argument added?

The script-message command itself - as clients use it, takes any number of arbitrarry arguments, and by convention the first is the message name.

So adding an argument at this level will surely break existing clients which expect the receiver to see exactly the arguments they send.

Or maybe it's only in how script-binding uses script-message?

Please clarify.

[na-na-hi]

Did you mean script-binding? I don't think I see at the changes anything specific to script-message, but I do see changes to script-binding.

It's noted in the changed script-binding documentation. Internally script-binding calls script-message command with several arguments. This makes script-binding calling script-message with an additional argument of scale.

I have updated the commit message to clarify this.

So adding an argument at this level will surely break existing clients which expect the receiver to see exactly the arguments they send.
Or maybe it's only in how script-binding uses script-message?

This only affects how script-binding uses script-message. The script-binding documentation explicitly notes: "Future versions can add more arguments and more key state characters to support more input peculiarities.", so it doesn't guarantee stability, but for this PR the extra argument is added to the end to avoid breaking existing clients.

[na-na-hi]

Added some documentation for scalable keys and scalable commands to better clarify scalable stuffs as a whole.

[guidocella]

We can't actually use this for cursor-centric-zoom can we? Because that needs to be bound to script-message in input.conf and not script-binding to send the zoom argument.

[na-na-hi]

You can add a script key binding for ctrl+wheel_up with mp.add_key_binding() in your script, which will provide the scale information. You can then send a script-message to cursor-centric-zoom with this information, or do the zoom directly if in the same script.

[guidocella]

I mean the user can no longer configure the zoom amount in input.conf. I guess we need to add a script-opt for it, which is more indirect.

[guidocella]

Actually, osd-relative-pan <axis> <amount> which needs to be a script message will have the same issue. Will users that want keybindings to pan with different amounts have to define bindings like

LEFT  {image} no-osd change-list script-opts append positioning-pan_amount=-.2; script-binding positioning/osd-relative-pan-x
DOWN  {image} no-osd change-list script-opts append positioning-pan_amount=+.2; script-binding positioning/osd-relative-pan-y
UP    {image} no-osd change-list script-opts append positioning-pan_amount=-.2; script-binding positioning/osd-relative-pan-y
RIGHT {image} no-osd change-list script-opts append positioning-pan_amount=+.2; script-binding positioning/osd-relative-pan-x
Shift+LEFT  {image} no-osd change-list script-opts append positioning-pan_amount=-.02; script-binding positioning/osd-relative-pan-x
Shift+DOWN  {image} no-osd change-list script-opts append positioning-pan_amount=+.02; script-binding positioning/osd-relative-pan-y
Shift+UP    {image} no-osd change-list script-opts append positioning-pan_amount=-.02; script-binding positioning/osd-relative-pan-y
Shift+RIGHT {image} no-osd change-list script-opts append positioning-pan_amount=+.02; script-binding positioning/osd-relative-pan-x

On top of no-osd not working these are multiple hoops users have to jump through to not make these commands.

[na-na-hi]

Actually, osd-relative-pan <axis> <amount> which needs to be a script message will have the same issue. Will users that want keybindings to pan with different amounts have to define bindings like

The case you mentioned can simply be a script-message with scrolling amount as a parameter. script-binding is only needed to get high resolution scrolling, so I think it's OK to keep it a separate option from the rest. The script can have both the script-message and script-binding to cover both cases.

[guidocella]

In my opinion having both script-message and script-binding for the same function is more complex to write and document than to just write it in C and extremely confusing for users, how are they supposed to understand which one to use? And users that want both high resolution scaling and bindings with different amounts still have to define user-unfriendly bindings like above.
I think we can implement osd relative pan and cursor centric zoom as commands, because commands can both accept arbitrary arguments and factor high resolution scaling. This lets users configure pan and zoom amounts in input.conf like with properties.
Cursor centric zoom is already implemented in C and osd relative pan is trivial.
As a bonus, cursor centric zoom will respect the standard no-osd prefix instead of configuring it indirectly with a script-opt.
Then keep drag to pan and align to cursor as a script, because they are too hard to reimplement in C since they temporarily rebind MOUSE_MOVE/observe mouse-pos, and there is no advantage in making them commands since they don't accept arguments.
We could then name the script misc.lua and reuse it for future key bindings, like #13837 (comment)

[na-na-hi]

In my opinion having both script-message and script-binding for the same function is more complex to write and document than to just write it in C and extremely confusing for users, how are they supposed to understand which one to use? And users that want both high resolution scaling and bindings with different amounts still have to define user-unfriendly bindings like above.

The intended way is to configure keybinds and amounts as script options, and let the script create and destroy key bindings by itself, and doesn't require putting anything into input.conf. input.conf isn't suitable for anything complex or specialized, and it's more confusing to use compared to making them all available as script options.

The only reason they are useful as script messages is for other scripts to call them. I don't think they're needed on their own when script options serve the purpose.

I think we can implement osd relative pan and cursor centric zoom as commands, because commands can both accept arbitrary arguments and factor high resolution scaling. This lets users configure pan and zoom amounts in input.conf like with properties.

Like I said script options is a better choice.

Cursor centric zoom is already implemented in C and osd relative pan is trivial.

I mentioned in your PR how features like animated zooming would be much harder to do in C while being easier in lua. mpv clients are clearly in a better place to implement these features compared in player core.

Then keep drag to pan and align to cursor as a script, because they are too hard to reimplement in C since they temporarily rebind MOUSE_MOVE/observe mouse-pos, and there is no advantage in making them commands since they don't accept arguments.

Yeah, some are commands and others are script messages... totally not confusing which you seem trying so hard to avoid.

I looked into making the script-message scalable but decided against it because it makes little sense when all arguments are strings, but I would rather hack scaling into it (when the first argument looks like a number) if you still insist on the script-message approach than whatever you're suggesting here.

[guidocella]

The intended way is to configure keybinds and amounts as script options, and let the script create and destroy key bindings by itself, and doesn't require putting anything into input.conf. input.conf isn't suitable for anything complex or specialized, and it's more confusing to use compared to making them all available as script options.

I don't think zoom and pan amounts are complex or specialized. All key bindings that modify properties already configure amounts in input.conf, it is nothing new.

I mentioned in your PR how features like animated zooming would be much harder to do in C while being easier in lua. mpv clients are clearly in a better place to implement these features compared in player core.

We can add a command for osd-relative-pan and keep cursor-centric-zoom in a script a middle ground. Wanting key bindings that pan with different amounts should be much more common than key bindings that zoom in different amounts. This is also the case in https://github.com/occivink/mpv-image-viewer/blob/master/input.conf, it has multiple pan amounts and one cursor-centric-zoom amount. I don't know if having to define the bindings in #15316 (comment) just do that seems sane to you. ¯\(ツ)/¯ But if maintainers deem it fine I'm fine with either way.

Yeah, some are commands and others are script messages... totally not confusing which you seem trying so hard to avoid.

Most mpv key bindings are already commands and some script messages, the only difference is that I'm adding these ones at the same time.

I looked into making the script-message scalable but decided against it because it makes little sense when all arguments are strings, but I would rather hack scaling into it (when the first argument looks like a number) if you still insist on the script-message approach than whatever you're suggesting here.

Nah, that is too hacky. You can't know whether numbers for script-message are floats or integers not meant to be scaled. Also ideally the axis would ideally be the first argument to pan.

[na-na-hi]

Added a custom argument for script-binding so custom information can be passed in the key bind and overridden in commands and input.conf.

[avih]

Without expressing an opinion on this whole PR, I'm not fond of overloading mp.add_key_binding with these additional arguments.

I also don't quite understand what arg is or does, and I've read the new docs few times.

According to the new docs, at the prototype it's additional to flags, but at the docs text it's instead of flags, or also inside it?!

The last argument is used for optional parameters. This is a table, which can

I presume this refers to the new args argument, yes?

Then why is args also listed as something which can be inside this table? "arg A user-provided string which is available in the table argument of fn by default. This option only makes sense when complex is set to true."

And, looking at the JS patch only:

        if (key_data.arg)
            key_data.input = key_data.input + " " + key_data.arg;

You do realize that this string is parsed according to input.conf syntax, right? which means it can become several arguments at the callback.

Also, if it's supposed to get back to the caller during callback, like some context argument which mpv itself doesn't care about? (is this what it is?) then, at least with the JS code, but almost certainly also at the lua code, you don't need to append it to the input.conf line and then parse it back during the callback. JS and Lua have closures. You can just keep it in key_data (without doing anything - it's already there), and then do arg: key_data.arg, instead of taking it from the arg callback argument (which you also forgot to add to key_data.cb, so it will try to use the global variable arg, and if it doesn't exist and the user happened to add "use strict";, then it will also throw an exception).

This is way too complex and too unclear for my taste.

[guidocella]

Thanks, this is much saner. I assume this can be extended to pass multiple arguments to script-binding in the future by converting them to a JSON array like I suggested.

I assume adding arg to the signature of mp.add_key_binding is a mistake, since it is passed to script-binding instead. Also passing arg = 'has spaces' to add_key_binding errors. But I'm not sure why this is even needed, is it only a default if no argument is passed to script-binding?

[na-na-hi]

Removed the ability to specify a default arg in mp.add_key_binding since it's not wanted.

Also, if it's supposed to get back to the caller during callback, like some context argument which mpv itself doesn't care about? (is this what it is?)

Yes.

JS and Lua have closures. You can just keep it in key_data (without doing anything - it's already there), and then do arg: key_data.arg, instead of taking it from the arg callback argument

The point of this is to allow the script-binding command specify an extra argument to be used. Closures won't be able to be written in the input.conf syntax unless it's written in escaped JSON.

(which you also forgot to add to key_data.cb, so it will try to use the global variable arg, and if it doesn't exist and the user happened to add "use strict";, then it will also throw an exception).

This is fixed now.

[avih]

A bit better in a quick glance.

You should also add a trailing comma in lists which just keep getting longer, this way at least the next time an item is added, only one line is added and no line is modified.

But overall, I'm not fond of it. This is just too many arguments at the callback, hard for the user to not make mistakes, and becomes increasingly annoying to extend. Not sure what the solution should be, if at all.

[na-na-hi]

You should also add a trailing comma in lists which just keep getting longer, this way at least the next time an item is added, only one line is added and no line is modified.

Added. The trailing comma wasn't here before this PR so I didn't add it.

But overall, I'm not fond of it. This is just too many arguments at the callback, hard for the user to not make mistakes, and becomes increasingly annoying to extend. Not sure what the solution should be, if at all.

The callback parameter is a single table. If the user doesn't care about some of the fields they can simply ignore them.

[avih]

The callback parameter is a single table.

Correct. Thanks.

But even internally, functions with arguments list which just keep getting longer is really meh.

Especially when it's prpagated from one place to another. In this case adding a single argument required updating 4 lists at the code (dispatch input arguments, dispatch call arguments, binding callback arguments, callback object).

This is not nice. You just experience first hand how easy it is to forget to add it on one of the places - with the missing arg at key_data.cb.

Also, unrelated, if I read it right, mp.add_key_binding now does them scalable by default? I've not followed the whole scalable subject, and I definitely don't know the details, but IMO this can break scripts easily, if they expect discrete wheel events but now by default they'd get bombarded with events.

[guidocella]

You have to opt-in with scalable = true in add_key_binding. Also I don't think scalable changes the number of the events, just the amounts.

It would have been nice to make script-binding send 1 script-message JSON argument like mp.input so the order of arguments doesn't matter, but changing it now is a breaking change.

[kasper93]

Nitpick about the organization of commits: Since we don’t use merge commits and instead maintain a linear history, it would be better to organize commits based on functionality rather than structure. After merging, PR boundaries are not visible, and separating documentation changes from functional changes makes it harder to follow the history, revert changes, and understand the context. In general, having 13 commits for a patch set with +89 and −29 changes is excessive. I'm all for small, easy to understand commits, but splitting single change into 5 commits (change + docs + lua + js + lua docs) is not making it easier to follow.

[na-na-hi]

This is not nice. You just experience first hand how easy it is to forget to add it on one of the places - with the missing arg at key_data.cb.

This is just a natural effect of a feature requiring coordination between several components. This only concerns the development process and I think the current process works well.

Also, unrelated, if I read it right, mp.add_key_binding now does them scalable by default?

No, it's opt-in for mp.add_key_binding. However, script-binding command is now scalable, so if a user manually binds scroll wheel to script-binding in input.conf without specifying nonscalable prefix, the command will be triggered more frequently when using a touch pad to scroll.

If you think this is a problem, I can make it nonscalable by default, and introduce a new command prefix scalable which turns the command scalable.

Nitpick about the organization of commits

It was split because it's easier for me to manage in WIP state. I have merged the doc commits into code changes.

[avih]

Also, unrelated, if I read it right, mp.add_key_binding now does them scalable by default?

No, it's opt-in for mp.add_key_binding. However, script-binding command is now scalable, so if a user manually binds scroll wheel to script-binding in input.conf without specifying nonscalable prefix, the command will be triggered more frequently when using a touch pad to scroll.

I see.

So at input.conf without adding any prefix it's scalable by default, but in mp.add_key_binding it's the other way around, i.e. without adding scalable the default is non-scalable (because it adds explicit nonscalable to the string in such case)?

The docs are not wrong, but it doesn't immediately stand out that it's different than the normal binding scalability behavior. It might be worth adding a note at the add_key_binding scalable docs.

(and about the whole PR... well... I'll shut up).

[na-na-hi]

So at input.conf without adding any prefix it's scalable by default, but in mp.add_key_binding it's the other way around, i.e. without adding scalable the default is non-scalable (because it adds explicit nonscalable to the string in such case)?

Yes. I made the behavior of mp.add_key_binding unchanged by default.

The docs are not wrong, but it doesn't immediately stand out that it's different than the normal binding scalability behavior. It might be worth adding a note at the add_key_binding scalable docs

Added a note for this.

[avih]

The docs say:

7. The user-provided string ``<arg>``, or empty string if the argument is not used.

At the C code:

scale_s, cmd->args[1].v.s};

Is this guaranteed to be an empty string if the user didn't add the extra argument?

[na-na-hi]

Is this guaranteed to be an empty string if the user didn't add the extra argument?

Yes.

mpv/player/command.c

Line 7162 in 947b7a5

{"arg", OPT_STRING(v.s), OPTDEF_STR(""),