Deno: deno run does not exit

[gr2m]

Follow up to #2075

What happened?

@laughedelic was able to narrow down the problem to the throttle plugin.

import { Octokit } from "https://cdn.skypack.dev/@octokit/core";
import { throttling } from "https://cdn.skypack.dev/@octokit/plugin-throttling";
const MyOctokit = Octokit.plugin(throttling);
const octokit = new MyOctokit({
  auth: Deno.env.get("GITHUB_TOKEN"),
  throttle: {
    onRateLimit: (retryAfter: any, options: any, octokit: any) => {
      return true;
    },
    onAbuseLimit: (retryAfter: any, options: any, octokit: any) => {
      octokit.log.warn(`Abuse detected for request ${options.method} ${options.url}`);
    },
  },
});

octokit.log.warn("foo");

It prints foo and hangs. If I remove everything related throttling, it prints the message and exits.

A workaround for this hanging is to add Deno.exit(0) at the end.

What did you expect to happen?

The deno process should exit, just like Node does

What the problem might be

The throttle plugin is using bottleneck by @SGrondin. Maybe it's a known issue?

I'd appreciate if someone with more deno experience could try to reproduce the hanging problem with just Bottleneck. The sourcecode where we use bottleneck is here: https://github.com/octokit/plugin-throttling.js/

[wperron]

Hey, Deno maintainer here 👋 I was actually just trying to use octokit to extract our release download stats and hit this issue. So far my script is as simple as it gets :

const octokit = new Octokit({ auth: Deno.env.get("GITHUB_TOKEN") });
const repo = await octokit.rest.repos.get({
  owner: "denoland",
  repo: "deno",
});

I also traced it down to plugin-throttling, I'll try to get a minimal repro with just Bottleneck next week. Don't hesitate to contact me too if you happen to be working on it.

[gr2m]

thank you so much @wperron for looking into it! Please let me know if there is anything that I can help with

[wperron]

I believe I found the reason for the hang, probably around the _startAutoCleanup function in Bottleneck (here).

In Deno, just like in the browser setInterval returns an IntervalID as opposed to the Timeout object returned by Node.js's setInterval. I have made the following little script to highlight the differences in Node.js and Deno

// Nodejs: `node this.js`
// Deno: `deno run this.js`
const base = setInterval(() => {}, 1);
if (typeof base.unref === "function") {
  base.unref();
} else {
  console.log("can't unref");
}

Without using the unref they both hang forever. unref allows Node to exit when it reaches the end of the script, regardless of the interval running in the background. I have a suspicion that the hang is also present when using octokit directly in the browser as an ES Module, but that due to the nature of how browsers work, it's simply not being noticed. I'm having issues with Skypack at the moment so Octokit simply won't load. I'll try again soon and confirm whether or not my hypothesis is correct.

[st3fan]

@wperron I'm still running into this today. As a workaround I'm ending my program with a Deno.exit(0) now but I was wondering if there is anything I could do or contribute to avoid that. Any ideas?

[michielbdejong]

Hey @gr2m nice to see you here! :) I'm also running into this issue; maybe the use of deno test helps us to trace it:

• Save this as test.ts:

import { Octokit } from "https://esm.sh/octokit@4.0.2?dts";
Deno.test("hello", async () => {
  const octokit = new Octokit({ auth: Deno.env.get("GITHUB_TOKEN") });
  await octokit.rest.repos.get({
    owner: "denoland",
    repo: "deno",
  });
});

• Run:

$ deno test -A --trace-leaks test.ts
Check file:///Users/michiel/test.ts
running 1 test from ./test.ts
hello ... FAILED (373ms)

 ERRORS 

hello => ./test.ts:2:6
error: Leaks detected:
  - An interval was started in this test, but never completed. This is often caused by not calling `clearInterval`. The operation was started here:
    at Object.queueUserTimer (ext:core/01_core.js:738:9)
    at setInterval (ext:deno_web/02_timers.js:83:15)
    at u._startAutoCleanup (https://esm.sh/v135/bottleneck@2.19.5/denonext/light.js:3:14348)
    at new u (https://esm.sh/v135/bottleneck@2.19.5/denonext/light.js:3:12962)
    at j (https://esm.sh/v135/@octokit/plugin-throttling@9.3.0/denonext/plugin-throttling.mjs:2:1925)
    at Array._ (https://esm.sh/v135/@octokit/plugin-throttling@9.3.0/denonext/plugin-throttling.mjs:2:2397)
    at new h (https://esm.sh/v135/@octokit/core@6.1.2/denonext/core.mjs:2:1840)
    at new <anonymous> (https://esm.sh/v135/@octokit/core@6.1.2/denonext/core.mjs:2:692)
    at file:///Users/michiel/test.ts:3:19
    at innerWrapped (ext:cli/40_test.js:174:11)
  - An interval was started in this test, but never completed. This is often caused by not calling `clearInterval`. The operation was started here:
    at Object.queueUserTimer (ext:core/01_core.js:738:9)
    at setInterval (ext:deno_web/02_timers.js:83:15)
    at u._startAutoCleanup (https://esm.sh/v135/bottleneck@2.19.5/denonext/light.js:3:14348)
    at new u (https://esm.sh/v135/bottleneck@2.19.5/denonext/light.js:3:12962)
    at j (https://esm.sh/v135/@octokit/plugin-throttling@9.3.0/denonext/plugin-throttling.mjs:2:1991)
    at Array._ (https://esm.sh/v135/@octokit/plugin-throttling@9.3.0/denonext/plugin-throttling.mjs:2:2397)
    at new h (https://esm.sh/v135/@octokit/core@6.1.2/denonext/core.mjs:2:1840)
    at new <anonymous> (https://esm.sh/v135/@octokit/core@6.1.2/denonext/core.mjs:2:692)
    at file:///Users/michiel/test.ts:3:19
    at innerWrapped (ext:cli/40_test.js:174:11)
  - An interval was started in this test, but never completed. This is often caused by not calling `clearInterval`. The operation was started here:
    at Object.queueUserTimer (ext:core/01_core.js:738:9)
    at setInterval (ext:deno_web/02_timers.js:83:15)
    at u._startAutoCleanup (https://esm.sh/v135/bottleneck@2.19.5/denonext/light.js:3:14348)
    at new u (https://esm.sh/v135/bottleneck@2.19.5/denonext/light.js:3:12962)
    at j (https://esm.sh/v135/@octokit/plugin-throttling@9.3.0/denonext/plugin-throttling.mjs:2:2067)
    at Array._ (https://esm.sh/v135/@octokit/plugin-throttling@9.3.0/denonext/plugin-throttling.mjs:2:2397)
    at new h (https://esm.sh/v135/@octokit/core@6.1.2/denonext/core.mjs:2:1840)
    at new <anonymous> (https://esm.sh/v135/@octokit/core@6.1.2/denonext/core.mjs:2:692)
    at file:///Users/michiel/test.ts:3:19
    at innerWrapped (ext:cli/40_test.js:174:11)
  - An interval was started in this test, but never completed. This is often caused by not calling `clearInterval`. The operation was started here:
    at Object.queueUserTimer (ext:core/01_core.js:738:9)
    at setInterval (ext:deno_web/02_timers.js:83:15)
    at u._startAutoCleanup (https://esm.sh/v135/bottleneck@2.19.5/denonext/light.js:3:14348)
    at new u (https://esm.sh/v135/bottleneck@2.19.5/denonext/light.js:3:12962)
    at j (https://esm.sh/v135/@octokit/plugin-throttling@9.3.0/denonext/plugin-throttling.mjs:2:2150)
    at Array._ (https://esm.sh/v135/@octokit/plugin-throttling@9.3.0/denonext/plugin-throttling.mjs:2:2397)
    at new h (https://esm.sh/v135/@octokit/core@6.1.2/denonext/core.mjs:2:1840)
    at new <anonymous> (https://esm.sh/v135/@octokit/core@6.1.2/denonext/core.mjs:2:692)
    at file:///Users/michiel/test.ts:3:19
    at innerWrapped (ext:cli/40_test.js:174:11)

 FAILURES 

hello => ./test.ts:2:6

FAILED | 0 passed | 1 failed (377ms)

error: Test failed

It should probably be possible to extract a minimal snippet of code from plugin-throttling.js that calls Bottleneck and shows this problem.

[michielbdejong]

Got it; opened an upstream issue: SGrondin/bottleneck#225

[wolfy1339]

The bottleneck package hasn't seen any new commits in the last 4 years.

At this point, it's best to find a new library that doesn't have their problem.