More efficient synchronization?

[n0lavar]

I am using manual executor to run async on my main thread and looks like mutex work take a lot of time
It would be nice if you add lock-free queue or something like this to improve performance

[David-Haim]

I'm not sure a lock free queue here will solve anything.
Lock free containers are very problematic, and they don't always provide better performance than traditional containers that are synchronized with a lock.

In the manual_executor case, many producers can enqueue tasks to run and many consumers can dequeue those tasks and run them, so we need an unbounded MPMC lock free queue. those unbounded mpmc lock free queues work by allocating every element separately and cas-ing those elements into the queue. so we're basically trading using a lock with allocating memory for every push operation. not sure it will give any performance gain on windows (with windows' crappy implementation of malloc/HeapAlloc).

Moreover, manual_executor gives the user the ability to wait until tasks are available or until a number of tasks are executed. if we remove the lock, then we also remove the underlying condition variable, and there is no way for us to wait for events any more. we will have to run the executor running methods in a busy loop or something to gain the same functionality.

Over all, I think that if applications run into executor related performance problems that are caused by high contention, then maybe the tasks that are given to those executors are too small to execute, and serving a longer-running tasks might help tremendously to reduce contention.

Also, executor::post is more efficient than executor::submit (but you don't get a result in return), maybe you can switch some of the tasks to use that?

[n0lavar]

I use only co_await and resume_on in my code and I don't really want to limit users of my engine with "please use only long raged tasks"... Async execution in the end of each frame on main thread is really widely used pattern in gamedev
Maybe some tricks with r/w/rw mutexes could help?
Another option is separate executor with lock-free queue which can be used with custom and fast general purpose allocator

[n0lavar]

We can also move mutex and deque types to the traits so user can define types he want, for example stub mutex and lock free container