RFC: find-workspace-symol iteratively

[nemethf]

I had this idea to iteratively search for workspace-symbols. It's somewhat like ido-switch-buffer, but as you type or when you stop typing, eglot displays the current candidates with xref. The implementation works for me, but I'm curious about your thoughts of the user interface. The code can be cleaned up later.

[leungbk]

Would this make sense to upstream into xref? Upstreaming lets this kind of thing work with etags as well. cc @dgutov (I think you're the xref maintainer?)

[dgutov]

I'm not sure. (A gif would help, BTW).

It sounds like a different UI, more like what ivy/counsel/vertico would provide. Whether that would be added cleanly to Xref, I suppose, is yet to be demonstrated with a patch.

[nemethf]

I'm not sure. (A gif would help, BTW).

I've created a .gif (using huge font-sizes somehow), but I'm not sure about its usefulness. It hopefully shows that the user is searching for a workspace symbol, the partially entered symbol name has more than one matches, and the matches have location information attached to them. When there is only one match, RET in the minibuffer leads to the location of the match. When there are many matches, RET jumps to the xref buffer.

It sounds like a different UI, more like what ivy/counsel/vertico would provide. Whether that would be added cleanly to Xref, I suppose, is yet to be demonstrated with a patch.

I haven't used any of ivy/counsel/vertico, so I don't really know. I'm still not sure if it's generally useful.

---

I have another experiment with xref. There's a language server that sends "runnalbe" commands corresponding to different file locations. An xref-after-jump-hook processes the last selected item (xref--current-item) and gives an option to the user to execute the selected command. However, xref--current-item is private. @dgutov, is there another way (an api call) to get the selected xref-item?

Thank you.

[dgutov]

I've created a .gif (using huge font-sizes somehow), but I'm not sure about its usefulness. It hopefully shows that the user is searching for a workspace symbol, the partially entered symbol name has more than one matches, and the matches have location information attached to them.

Sounds similar to xref-find-definitions's behavior.

I was more referring to a different change in behavior, though: the list being updated as you type. That's the common feature of frameworks like Helm/Ivy/Vertico. Not sure what would be the best place to add it. Whether it could be applicable to all xref commands, etc.

An xref-after-jump-hook processes the last selected item (xref--current-item) and gives an option to the user to execute the selected command. However, xref--current-item is private. @dgutov, is there another way (an api call) to get the selected xref-item?

Nope, never considered that the value would be interesting for extensions. You can refer to this name, but it's probably a good idea to rename it (with a compatibility alias, I guess). We can move this to the bug tracker.

[nemethf]

I was more referring to a different change in behavior, though: the list being updated as you type. That's the common feature of frameworks like Helm/Ivy/Vertico. Not sure what would be the best place to add it. Whether it could be applicable to all xref commands, etc.

Yes, one can look at this feature as completing-read with location. Reading (info "(elisp)Programmed Completion") it seems Emacs does not support this. If I read it correctly, company (or company-capf) does support it but only for completion-at-point. If Helm, Ivy, and Vertico have something like this, are there any common api for this? Like, for example, a company-location metadata field. (Even if completing-read does not use location info, it would be good if emacs could standardize a location metadata field for programmed completion.)

[...] We can move this to the bug tracker.

OK, thank you.

[dgutov]

Like, for example, a company-location metadata field.

There is such field, but it works differently: to show the location of the current completion. And anything Company does works when you start typing something, in some buffer. Even if we turn it on in the minibuffer, which command would activate it?

But there is a related feature of Xref which Eglot does not support: identifier completion. When you call C-u xref-find-definitions, or xref-find-references, you are allowed to press TAB and choose the identifier to look for, with completion.

It sounds like this is somewhat similar, in both workflow and purpose. I'm not sure if it's currently feasible to implement xref-backend-identifier-completion-table in terms of workspace search (or whether it would work with many language servers), but it would be very nice to have.

[nemethf]

Like, for example, a company-location metadata field.

There is such field, but it works differently: to show the location of the current completion. And anything Company does works when you start typing something, in some buffer. Even if we turn it on in the minibuffer, which command would activate it?

I have to admit I haven't look at the capabilities of currently available third party completion packages. Nevertheless, the way I understand it, there are two basic kinds of completion. There is completion-at-point, which works in normal buffers. And there's completing-read for the minibuffer. Completion items for completion-at-point can be annotated [1] and company-mode extends the simple annotation with additional metadata fields like location [2]. It seems these extensions are so popular that other completion packages also support them.

I wonder whether it makes sense to use some of these extended metadata fields for completing-read as well. And what is the best UI for presenting the location info in case of completing-read. Maybe if emacs "codifies" company's additional metadata fields, it will inspire 3rd party developers to create innovative interfaces.

But there is a related feature of Xref which Eglot does not support: identifier completion. When you call C-u xref-find-definitions, or xref-find-references, you are allowed to press TAB and choose the identifier to look for, with completion.

It sounds like this is somewhat similar, in both workflow and purpose. I'm not sure if it's currently feasible to implement xref-backend-identifier-completion-table in terms of workspace search (or whether it would work with many language servers), but it would be very nice to have.

I need to look into this, but my guess it is possible to implement a xref-backend-identifier-completion-table for eglot. However, then the workflow would be the following: the user selects a possible symbol to search for from a list of non-annotated strings and then xrefs shows a list of locations for that selected symbol. What I'm interested in is to show locations of the possible symbols/identifiers in the first phase when the users selects the symbol to search for.

(Thank you for your patience, it seems it's hard for me to write about this clearly.)

[dgutov]

I wonder whether it makes sense to use some of these extended metadata fields for completing-read as well. And what is the best UI for presenting the location info in case of completing-read. Maybe if emacs "codifies" company's additional metadata fields, it will inspire 3rd party developers to create innovative interfaces.

I think a bunch of those get used already -- at least Corfu does use them. But like Company, it specializes on in-buffer completion. In both cases that can be changed, but if we look at Ivy/Helm/Vertico, none of them do that. Though they could indeed make use of company-location: it's a simple enough interface.

But I guess one problem, when we talk about completing-read, is that it doesn't use CAPF: it only gets passed the completion table. Maybe extending the metadata fields to contain location is how this could be implemented. I'm not crazy about the duplication, though.

I need to look into this, but my guess it is possible to implement a xref-backend-identifier-completion-table for eglot. However, then the workflow would be the following: the user selects a possible symbol to search for from a list of non-annotated strings and then xrefs shows a list of locations for that selected symbol. What I'm interested in is to show locations of the possible symbols/identifiers in the first phase when the users selects the symbol to search for.

IMHO the former is more important than the latter. But of course your priorities are your own.

(Thank you for your patience, it seems it's hard for me to write about this clearly.)

No problem, it's usually not easy to narrow down the potential set of all changes to something actionable.

[nemethf]

[...] my guess it is possible to implement a xref-backend-identifier-completion-table for eglot. [...]

My idea was to fetch the possible candidates for C-u M-. with the workspace/symbol LSP request and feed to result to a completion table. This is unfortunately not viable with the sever I try this out. It answers the request with a partial result: it doesn't the the full list of matching symbols, only a subset. Additionally, it uses flex matching internally, so Emacs cannot use
its own filtering infrastructure. And finally, symbols are different from the identifiers that M-. can jump to with the textDocument/definition LSP request.

The server in question is rust-analyzer, but it seems it doesn't violate the protocol specification when serving these request, so my conclusion is it's not possible to reliably create a completion-table for the xref commands.

However, Eglot maps xref-find-apropos to workspace/symbol, which is OK, but workspace/symbol has more potential as demonstrated at the beginning of this discussion or with the code below.

---

(stream is not among the possible completions. SyntaxTreeParams matches because of SynTaxtREepAraMs.)

the corresponding code for future reference:

(defun eglot-find-workspace-symbol (pattern)
  "Find a workspace symbol for PATTERN."
  (interactive (list (completing-read "WS-symbol: " (my-compilation-fn))))
  (let ((xref-backend-functions 'eglot-xref-backend)
    (xref-find-apropos pattern)))

(defun my-compilation-fn ()
  (let* ((source-buffer (current-buffer))
         (symbols (lambda (pattern)
                    (mapcar
                     (eglot--lambda ((SymbolInformation) name)
                       name)
                     (with-current-buffer source-buffer
                       (jsonrpc-request (eglot--current-server-or-lose)
                                        :workspace/symbol
                                        `(:query ,pattern))))))
         (metadata nil)
         (fn (lambda (prob pred action)
               (cond
                ((eq action 'metadata) metadata)               ; metadata
                ((eq action 'lambda)                           ; test-completion
                 (funcall symbols prob))
                ((eq (car-safe action) 'boundaries) nil)       ; boundaries
                ((null action)                                 ; try-completion
                 (let ((nb_matches (length (funcall symbols prob))))
                   (cond
                    ((= 0 nb_matches) nil)
                    ((= 1 nb_matches) t)
                    (t prob))))
                ((eq action t)                                 ; all-completions
                 (funcall symbols prob))))))
    fn)))

[dgutov]

This is unfortunately not viable with the sever I try this out. It answers the request with a partial result: it doesn't the the full list of matching symbols, only a subset.

Could it still be an improvement over what we have now?

Even if the completion table produced this way is incomplete, it could contain whatever is reachable this way (perhaps global constants; maybe fully-qualifier method names as well), and the symbol at point as additional member (perhaps specially annotated for the user and the program to be able to distinguish it).