New async/await handler support breaks next(false) functionality in current async handlers

[gmahomarf]

• Used appropriate template for the issue type
• Searched both open and closed issues for duplicates of this issue
• Title adequately and concisely reflects the feature or the bug

Restify Version: 10.0.0
Node.js Version: 16.8.1

Expected behaviour

Given a handler that does async work, I should be able to call next(false); and have the chain stop processing there.

Actual behaviour

The handler arity checks prevent me from having handlers that use next and are async

Repro case

Code similar to this is used in one of our projects using restify v8. It breaks when trying to update to v10:

server.use(async (req, res, next) => {
  const result = await someAsyncWork();
  if (shouldStop(result)) {
    res.send({something: 'here'});
    next(false);
    return;
  }

  // ... more work
  next();
});

I am aware I could make my handler synchronous, then do someAsyncWork().then(result => {...}) but async/await syntax was chosen for cleanliness and readability.

Cause

node-restify/lib/chain.js

Lines 77 to 101 in 2053ef6

	if (handler.length <= 2) {
	// arity <= 2, must be AsyncFunction
	assert.equal(
	handler.constructor.name,
	'AsyncFunction',
	`Handler [${handlerId}] is missing a third argument (the ` +
	'"next" callback) but is not an async function. Middleware ' +
	'handlers can be either async/await or callback-based.' +
	'Callback-based (non-async) handlers should accept three ' +
	'arguments: (req, res, next). Async handler functions should ' +
	'accept maximum of 2 arguments: (req, res).'
	);
	} else {
	// otherwise shouldn't be AsyncFunction
	assert.notEqual(
	handler.constructor.name,
	'AsyncFunction',
	`Handler [${handlerId}] accepts a third argument (the 'next" ` +
	'callback) but is also an async function. Middleware ' +
	'handlers can be either async/await or callback-based. Async ' +
	'handler functions should accept maximum of 2 arguments: ' +
	'(req, res). Non-async handlers should accept three ' +
	'arguments: (req, res, next).'
	);
	}

Are you willing and able to fix this?

This probably requires reworking the async chain stuff, so no.

[phil-warner]

Agree. This is making me consider switching framework.

[kolbma]

Kind of this also broke the express middleware support, although there has been no usage of async in own code.
Have tried to change everything in my code to async, but then also there is this compatibility problem with chained express middleware.

[ghermeto]

Calling next is not allowed when using async functions. That said, I agree Restify must provide a way to stop chain processing, similarly to next(false);.

Right now, all values are being discarded when returning a AsyncFunction. Maybe false should be passed through.

node-restify/lib/chain.js

Lines 188 to 198 in adf24c1

	function resolve(value) {
	if (value && req.log) {
	// logs resolved value
	req.log.warn(
	{ value },
	'Discarded returned value from async handler'
	);
	}
	return next();
	}

[gmahomarf]

Calling next is not allowed when using async functions. That said, I agree Restify must provide a way to stop chain processing, similarly to next(false);.

Yes, I'm well aware it's not allowed. That's the whole point of this issue, as is the inability to stop chain processing.

[phil-warner]

@ghermeto - have you agreed an approach to tackling this issue? We'd like to plan our upgrade path and that depends partly on which direction Restify takes here.

[cjroebuck]

You can workaround this by wrapping all async handlers, like the package https://www.npmjs.com/package/@gilbertco/restify-async-wrap does.

export const wrap = (fn: any) => (req: Request, res: Response, next: Next) => {
  fn(req, res, next).catch(next);
};

then

app.post("/v1/users",wrap(someAsyncHandler))

[kolbma]

@cjroebuck

Do you have any idea how to use https://github.com/express-validator/express-validator middleware after the restify changes? Your wrapper doesn't work.

[cjroebuck]

@kolbma I'm using zod now to validate request inputs, and its just another middleware function on my routes. I'm sure it would be possible to still use express-validator if you wrap the middleware it exports in your own function and call it from your route.

[kolbma]

In an async handler you have to throw an error instead of calling next(false) (@ghermeto, @gmahomarf).
And for response.redirect(...) see #1916.

But as I've written before there has been introduced some more new incompatibility with existing express middleware.

E.g. I'm using express-validator middleware and with the newer restify versions I've to use a wrapper (thanks to @cjroebuck) like (TS)...

async (req) => { await checkSchema(CHECK_SCHEMA).run(req); }

With restify v8 I could simply use checkSchema(CHECK_SCHEMA) (api) in the handler chain, like it is documented for express.

Do you have an idea what and where might be the problem to look for?

[gmahomarf]

In an async handler you have to throw an error instead of calling next(false) (@ghermeto, @gmahomarf). And for response.redirect(...) see #1916.

No, because I don't want to throw an error. I want to stop the handler chain.

[kolbma]

@gmahomarf Ok, maybe one workaround would be to use if (response.writeableEnded()) { return; } in all the following handlers. Sure only possible if you can modify them or also with wrapping.

I think #1941 would stop the handler chain if you have already handled request and response.
Is there any use-case where this doesn't work?

IMO next() makes not much sense in async environment, because you don't want to get/check for races in one handler between multiple next-statements in no guaranteed order. And next is callback-syntax.

[gmahomarf]

I don't want any errors when all I'm trying to do is stop the chain. I also don't like using next in async handlers, but there's currently no other way to stop the chain. That's the whole point of this issue.

[kolbma]

But you have to handle the request somehow or the tcp connection to the client is closed silently after timeout. So if you don't want to return an error you have at least to return some other response and there you close the response-writer and the handler chain would be stopped by itself.

[phil-warner]

@kolbma this isn't the case if your code is running on a schedule, when no client is connected. We need the option to gracefully stop the chain.

[kolbma]

@phil-warner Could you explain what you mean by running code on a schedule? Sounds for me at the moment like async job scheduling in a bad way. The handler chain should be finished if the request is finished or you are open for Denial of Service.

[phil-warner]

@kolbma sorry - that was inaccurate. I misread the code.

Nonetheless, this should be supported without a workaround that people have to come to this thread to find.