Try at implementing TransportLayer

[zetashift]

Hiya! I want to attempt to write a GUI for neovim using Pixie and for that I need to have a nice neovim client, I was hoping to get this working but as soon as I dove in, I found out that this is pretty hard for a beginner!

So I'm hoping you can give some tips on how to implement the transport layer succesfully, if not then that's okay too! :P

First off I'm tackling named pipes, but as far as I know these are just special files on Unix so I'm just using stdlib's file methods to get these working, except I don't know how readPipe should be implemented.

I also updated the nimble file.

[jangko]

Thank you for your interest in this project.

The reason I'm pausing my work on this is at that time there was no good enough async lib around. If you are going to integrate it with a GUI app, it will be a horrible experience for user if you are using synchronous operation.

It should be async or multi threaded. but using thread in nim is also really hard because of nim memory model. async should be more realistic.

you can take a look at https://github.com/status-im/nim-chronos. it has async socket and pipe.

I found out that this is pretty hard for a beginner!

Don't worry, you will become more skilled as you working on more projects.
I work with protocols and communications to earn my living, and it is still hard for me too.
sometimes we even joking around with our workmates about something hard to solve and compare it debugging async code.

[zetashift]

Hiya thanks for the awesome explanation, I'll try to use chronos, synchronous does indeed sound bad, I was hoping to get a minimal working version first and then update as I go.

you can take a look at https://github.com/status-im/nim-chronos. it has async socket and pipe.

I did take a look at chronos and the json rpc implementation, but the chronos README also this under TODO:

Pipe/Subprocess Transports.

So I thought it might have been better to use: https://github.com/cheatfate/asynctools

sometimes we even joking around with our workmates about something hard to solve and compare it debugging async code.

Haha, that's good to know, it's such low level intricate stuff! Fascinating nonetheless.

[zetashift]

So I'm having issues generating nim-chronos documentation, but it is it correct that for creating pipes I need to use this function: https://github.com/status-im/nim-chronos/blob/master/chronos/handles.nim#L157 ?

Nim-chronos pipes are anonymous on Linux and for neovim I think I need named pipes specifically:

https://github.com/cheatfate/asynctools/blob/master/asynctools/asyncipc.nim#L56 might be better to use

[rosshadden]

I just tried using nvim_client and ran into the waitNamedPipeW function not working natively on Linux, which it looks like you are trying to fix here. I'm also rather new to Nim but am trying to get something working. I will report back if I come up with anything notable or run into specific trouble.

[zetashift]

I just tried using nvim_client and ran into the waitNamedPipeW function not working natively on Linux, which it looks like you are trying to fix here. I'm also rather new to Nim but am trying to get something working. I will report back if I come up with anything notable or run into specific trouble.

I think AsyncIPC from asynctools is the way to go, but I'm really new to all this async stuff in general, so I have some stuff written up but I'm pretty sure it's broken :(.

[rosshadden]

They just merged cheatfate/asynctools#27 yesterday. That PR into asynctools combined with your changes from a few days ago make the tests at least pass for me now (nimble test), which is more than I can say for when I first started poking around here. 🎉

Edit: Nevermind. I just realized the tests are trying to run nvim.exe which obviously fails, so the actual tests get skipped.

[rosshadden]

I found that initTransportLayer, since it just discards, always returns a TransportLayer with the first TransportKind enum value. Which in our case is tkPipe. Just to force my way through and on to the next problem, I moved tkStdio to the top of the enum definition.

Changing that, fully qualifying the poUsePath and poEvalCommand pragmas because another import adds them as well, and changing the nvim.exe --embed to nvim --embed, the tests then get up to nvim.get_current_tabpage() which hangs indefinitely.

[zetashift]

Changing that, fully qualifying the poUsePath and poEvalCommand pragmas because another import adds them as well, and changing the nvim.exe --embed to nvim --embed, the tests then get up to nvim.get_current_tabpage() which hangs indefinitely.

I see there is more to do then only fixing up tkPipe! It's gonna be interesting.

[rosshadden]

@zetashift Checking back in for the new year. Have you made any progress on this?