Github Aid

This is a Chromium and Firefox extension for viewing github Repository file and folder sizes, in addition to Bulk downloading them.

Note

TODO: The download feature has yet to be implemented

Downloads

Obligatory screenshots

Why?

• because two of the popular extensions which did previously work, suddenly stopped working since december 2023.

  • github-repo-size
  • enhanced-github

• the codebase of both uses NodeJs, which comes along with a lot of boilerplate

• both used manifest v2, which will be deprecated in chromium sometime later this year

• neither looks good and usable at the same time in mobile display

• wanted to do a fun weekend project, try out some GraphQL apis, and be done with the annoyance of constant non-functioning extensions

Problems you may encounter:

1. The buttons are not appearing.

• Fixes:
  • Refresh the page. Github is an SPA (single-page application), so there are times when the background script does get reloaded upon navigation from one page to another.
  • Make sure that you did not disable all buttons in the option page's layout section. If you're having touble, consider clicking on the red "reset" button to clear any misconfigurations and return to factory settings.
  • Make sure that storage permission has been granted to this extension, otherwise the script will fail to load the default configuration, and terminate fatally.

2. One of the buttons flashes in red, and nothing happens.

• This is because the request query sent to github's server has failed, possibly due to one of the following reasons:
  • Github's server is overloaded, and it decided to reject your request (quite common). In this case, try switching your api method in the options page, and reload your webpage and retry.
  • You are viewing a private repo, and the access token that you have (or may lack) is not authorized for pulling up data of that private repository.
  • Your Github Access Token key is invalid, or has expired.
  • You (and your Token) have reached their per hour request limit, or that you've been sending too many requests too quickly. The request data rate for github is: 50 requests per hour for non-authorized tokenless users, and 5000 per hour for authorized users. Moreover, the REST api in this extension used 2 requests for fetching file and folder sizes.
  • There might be a bug in the code, or github's api might have changed. In that case, I'd appreciate if you could report the issue on github: https://github.com/omar-azmi/github_aid_ts/issues

How the internals differ

This project provides a good minimal-boilerplate example of how one can generate a web-extension, while:

• only using typescript, and no NodeJs boilerplate or 3rd-party typescript dependencies
• uncoupling all of the following three major parts of the code:
  • library code (responsible for fetching and parsing data from api.github.com), coded under /src/lib/
  • user interface code (responsible for configuring and customizing user options), coded under /src/html/
  • background script (responsible for injecting this extension's UI into github.com), coded under /src/js/
• being able to transpile, bundle, minify, and package all in one go. all thanks to the incredible esbuild tool, and its Deno compatibility plugin esbuild_deno_loader

If you look at a typical extension development codebase, you'll encounter:

• a lot of node-specific files
• ton of external developmental dependencies
• lots of config files for the dependencies
• lots of tasks defined by the dependencies that need to be run sequentially for the build process to work
• having to strictly adhere to a certain directory structure set by the dependencies (or their config files)
• a nighmare when trying to reference a simple typescript relative import
  • for instance, you'll need to use the ".js" extension suffix in some occasions. other times the extension part has to be dropped
  • same directory relative import cannot be done, and has to be done relative to the project's root (where package.json lies)
  • having to adjust to framework specific import prefix characters

all in all, that will lead to a highly coupled codebase, and it'll be incredibly difficult to extract just one part of it for testing, or reusability elsewhere.

The Deno build process

Building it yourself

to build this yourself, make sure that you first have Deno installed. then, in your terminal (or cmd), run:

deno task build-all

and you should get an unpacked distribution of the extension in the /dist/github_aid_ts-v* directory.

other available commands (tasks):

command	general description
deno task clean	delete the /dist/ folder
deno task clean --js-only	delete only the javascript files under the /dist/ folder
deno task build-1	copy all non-typescript source files to the /dist/ folder
deno task build-2	bundle endpoint typescript files inside of /src/, and mirror the resulting javascript files to the /dist/ folder
add an additional --log-only flag to any of the commands above	execute the task without actually writing or deleting any files
deno task build-all-log-only	runs deno task build-1 --log-only and then deno task build-2 --log-only

Build diagram

---
title: "Build process"
---
flowchart LR
	StartNode(["deno task build-all"])
	StartNode -->|"1"| 877701(["deno task build-1"])
	StartNode -->|"2"| 247711(["deno task build-2"])
	247711 --> 557486
	877701 --> 977090

	subgraph 557486["./build_2.ts"]
		418522["
			collect each '.ts' file under
			the script endpoints:
			- '/src/html/*.ts'
			- '/src/js/*.ts'
		"] --> 287901

		287901 --> 176317["
			write the resulting js-code files from
			memory to the '/dist/' directory, mirroring
			their original (entry-point) location.
			unfortunately, the non-endpoint
			split-code js files end up directly
			under '/dist/', which may look ugly
			from a distribution perspective
		"]

		subgraph 287901["./build_tools.ts --> doubleCompileFiles()"]
		direction LR
			728262["
				run esbuild bundling with:
				- deno plugin, for resolving import paths
				- minification + tree-shaking, to
				remove unimported exported objects
				- code-splitting, to preserve common
				imports across different endpoints
				- writing disabled, so that the bundled
				file results are stored in-memory
			"] --> 558940["
				now that the standalone bundled code files
				are free of external dependencies,
				on each individual output js code file:
				- run esbuild transform with minification,
				to remove dead-code within the same file
			"]
		end
	end
	
	557486 --> EndNode(["end"])

	subgraph 977090["./build_1.ts"]
		708084["
			copy all non-typescript
			files from '/src/' to '/dist/'
		"]
	end

Loading

General import map

---
title: "Import graph"
---
flowchart TD
	591514(("deps.ts")) --> 530727((("option.ts\n(endpoint)")))
	423910(("typedefs.ts")) --> 709575((("content_script.ts\n(endpoint)")))
	405595(("modify_ui.ts")) --> 709575
	591514 --> 709575
	subgraph 325333["/src/lib/"]
		591514 --> 650261(("gh_rest_api.ts"))
		423910 --> 650261
		591514 --> 517060(("gh_graphql_api.ts"))
		423910 --> 517060
		650261 --> 405595
		517060 --> 405595
		591514 --> 405595
		423910 --> 405595
	end
	subgraph 201358["/src/js/"]
		709575 -->|"dynamic import as\nchromeURL('/js/content_script.js')"| 798822((("content_script\n_extension_adapter.ts\n(endpoint)")))
	end
	subgraph 843058["/src/html/"]
		530727 -->|"imported as\n'./option.js'"| 635635[["option.html"]]
	end

Loading

Rationale for Code-Splitting, and its caveates

What is Code-Splitting, and how does it differ from regular bundling?

suppose you have a total of 5 source code files (A, B, C, D, and E), with the following dependency graph:

---
title: "source dependency graph"
---
flowchart TD
	222161(("C")) --> 339264((("A")))
	842277(("D")) --> 339264
	842277 --> 633740((("B")))
	394730(("E")) --> 633740
	842277 --> 222161
	842277 --> 394730
	subgraph 977297["bundling endpoints"]
		633740
		339264
	end

Loading

if you were to bundle the endpoints A and B without code-splitting, you'd get two independently operated pieces of bundled code:

---
title: "bundled endpoints"
---
flowchart TD
	subgraph 280910["bundle B"]
		735571["Union(\nIntersect(D, B.imports),\nIntersect(D, E.imports)\n)"] --> 985393["Intersect(E, B.imports)"]
		735571 --> 633740((("B")))
		985393 --> 633740
	end
	subgraph 977297["bundle A"]
		384214["Union(\nIntersect(D, A.imports),\nIntersect(D, C.imports)\n)"] --> 592871["Intersect(C, A.imports)"]
		592871 --> 339264((("A")))
		384214 --> 339264
	end

Loading

this is where a potential problem can occur: if there was a runtime-unique shared piece of code inside of D, then inconsistencies will arise when the two pieces of bundled codes try to interoperate.
a few examples of runtime-unique shared pieces of code would be:

• a javascript Symbol. both bundleA and bundleB will define the symbol separately, thereby becoming incompatible
• mutative side-effects inside of D. if, for instance, D.injectButton was a function that would inject a button into your html and then set a D.button_injected = true flag inside so that duplicates of buttons are not made in future calls. If both bundleA and bundleB call D.injectButton several times, then we'd get two buttons instead of one, because now, the flags are defined independently.
  had we imported from D instead of bundling, we would've expected to see a single injected button.

if we bundle with code-splitting enabled, we would get the following output dependency graph, and our runtime-unique shared pieces of code will interoperate correctly:

---
title: "Diagram Title"
---
flowchart
	842277(("split D = Union(
		Intersect(D, A.imports),
		Intersect(D, C.imports),
		Intersect(D, B.imports),
		Intersect(D, E.imports)
	)")) --> 912727["Intersect(C, A.imports)"]
	842277 --> 339264((("A")))
	842277 --> 633740((("B")))
	842277 --> 517056["Intersect(E, B.imports)"]
	subgraph 650622["split-bundle A"]
		912727 --> 339264
	end
	subgraph 367445["split-bundle B"]
		517056 --> 633740
	end

Loading

bundling with code-splitting also comes with the advantage of smaller total output file size.

Caveates of Code-Splitting specific to browser-extensions development

first, you need to know that the extension is supposed to be a single javascript file that is executed after your designated injection-target website has loaded (this being "github.com" for this extension).
the javascript file to be ran is specified in the manifest file (/src/manifest.json), under manifest["content_scripts"][0]["js"][0].
the global context available to this javascript of yours is simply the same as the one available to the target webpage (i.e. you are in the window environment), in addition to having a limited number of browser-extension features, such as: chrome.storage and chrome.runtime (or in the case of firefox: browser.storage and browser.runtime).
moreover, by web-security design, you cannot do static imports in any content_script (so import {x, y, z} from "./abc.js" is disallowed).
however, you can dynamically import using const {x, y, z} = await import("./abc.js"), but it will require that the target webpage has access to that imported file, which it currently doesn't, because the file sits locally in your browser.
hence, you will need to give the target website access to your imported javascript files, which is done by specifying the manifest["web_accessible_resources"] entry in the manifest file.
here's how it should look:

// manifest.json
{
	// ...
	"web_accessible_resources": [
		{
			"resources": ["*.js"],
			"matches": ["<all_urls>"]
		}
	],
	// ...
}

with that, you've solved the problem that you initiated, and made your extension less secure along the way.
alternatively you can choose to bundle without code-splitting, and avoid all the fuss. but where's the fun in that?
see this stackexchange answer, where I found this information from.
also see /src/js/content_script_extension_adapter.ts for the dynamic imports being done in this extension.

License

see license.md. but for a quick summary:

• you may not execute a crawling script onto this project's contents
• you may not feed any part of the project's contents to an AI training system, or use it as an AI chat prompt
• you may not copy-paste this project and make it your own
• you may not hold me liable for any damages this code might result in

How to get a Github Access Token

• First of all, you'll need to be logged into your github account (Duh!).
• Navigate to this github page: Generate new token (classic) (https://github.com/settings/tokens/new).
• Set an Expiration date for your token, (you'll probably want to choose No expiration).
• In the Select scopes section, under the repo checkbox:
  • enable only the public_repo checkbox if you will NOT be viewing your private repository's stats.
  • enable the whole repo group checkbox otherwise.
• Scroll to the bottom and click on the Generate token button.
• You will now be presented with the access token. MAKE SURE TO COPY AND SAVE IT SECURELY RIGHT NOW! This is a one time preview of you key, and it will disappear forever after you've closed the dialog.
• Paste the token into this browser extension's options page.

For a visual guide, see one of:

• https://www.geeksforgeeks.org/how-to-generate-personal-access-token-in-github/
• https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens

But remember NOT to check any scope boxes besides the repo one. Having anything else checked is dangerous if your access key gets leaked, and someone decides to maliciously delete your projects, or hold them for ransom. (as if that has ever happened)