The proposal of using monorepo

[SukkaW]

Why should we use monorepo?

For example, Hexo is dropping Node.js 8 at this moment. That means every CI config under every repo has to be updated. It is painful.

If we are using monorepo, we could write only one ci config for all packages (run eslint & all unit test cases at once).

Also, with monorepo we could maintain a global eslintrc / mocharc for all packages.

What should be included in monorepos

Only for those repos that are not hexo plugins, which means:

• repos like hexo-util, hexo-i18n, hexo-pagination, hexo-server, hexo-cli, etc. should be merged into monorepos.
• repos like hexo-renderer-marked, hexo-filter-nofollow, hexo-uglify, etc. should remain separated.

When should we start

Right after Hexo 5.0.0 released.

What monorepo manager should we use?

lerna.

Any play ground?

https://github.com/hexojs/hexo-monorepo

@hexojs/core and owners of @hexojs should have access to the repo.

[tomap]

I've seen Lerna used at work, and I was not convinced.
But I thought about it in numerous cases, especially all the plumbing you are mentionning (nodejs versions, eslint, tests, readme, ...)
Si why not. Do you have experience in bootstrapping a lerna project?

[SukkaW]

@tommy351 has some exprience on migrating to monorepo. Hopefully he could give us some suggestions.

[tommy351]

I think it will take some time to setup the environment for testing and releasing and the build time on CI will be longer. But it would be great to be able to share configs and utils across the project.

[SukkaW]

Next.js uses lerna.
React uses yarn workspace.
Babel used to use lerna, and it is now using yarn workspace, too.
Microsoft has Rush.

So which one should we use?

@tomap @yoshinorin @stevenjoezhang @jiangtj @himself65

[ertrzyiks]

I wouldn't recommend using lerna nowadays, it's maintenance is seems to fade out and we can get a lot with yarn 3 workspaces alone.

[yoshinorin]

I wouldn't recommend using lerna nowadays, it's maintenance is seems to fade out

+1

IMHO, I think it is desirable if we can do it without adding any tools or dependencies.
Therefore, I think npm workspace or yarn workspace is preferable. However, I haven't actually used it, so I'm not sure we can reach our goal with them or not.

[tomap]

I have two questions:
will yarn also become a dependency of hexo user if we use yarn workspace?
Isn't the sandbox supposed to help us test that?
What are the things that you want to validate?

[ertrzyiks]

@tomap, nothing changes on the end users side. We would keep the package hierarchy in NPM registry as it is now. The migration to yarn workspaces and monorepo will affect developers of the packages that became part of the monorepo only.

I didn't get the second question, so I assume I miss some context.

I allowed myself to chime in this thread because monorepo is a hot topic for me both professionally and personally. I recently migrated a few of my pet projects to yarn workspaces using yarn 3. One of the migration was from rush.js. I don't publish any packages to NPM though.

[SukkaW]

Isn't the sandbox supposed to help us test that? What are the things that you want to validate?

Well here is a thing. We just have so many repos to maintain. hexo-utils, hexo-pagination, hexo-front-matters...

When we want to fix a bug inside hexo, sometimes we need to fix it in hexo-utils first. And before we release a newer version of hexo-utils and bump its version in hexo's package.json, we can't get out hands on the hexo. Monorepo should help us fix the workflow here.

IMHO, this reason alone is already enough for us to migrate to the monorepo approach.

[tomap]

Sure, so now, plan and then do!

1. Do some tests.
2. Once ready, convert hexo into a monorepo but with only hexo in it, ready to receive other projects.
3. Add projects one at a time. Each time, release before doing it again.

More specific steps?

[SukkaW]

Sure, so now, plan and then do!

1. Do some tests.
2. Once ready, convert hexo into a monorepo but with only hexo in it, ready to receive other projects.
3. Add projects one at a time. Each time, release before doing it again.

More specific steps?

0. I'd like to continue using the hexojs/hexo repo and retain the master branch as our default branch.
1. Create a new branch from our current master branch, and it will be our archive branch (maybe name it 6.0.0 or v6)

• May not be necessary as we will retain all our commit history anyway.

2. Create another new branch (name it migrate-to-monorepo?), where we try out the monorepo approach. I suggest we start with only hexo itself and hexo-utils.
3. We should practice changes, releases, dependency updates workflow a bit.
4. Finally, we merge it back in the master branch, with commit history untouched.
5. Delete the migrate-to-monorepo branch
6. After we finish making hexo and hexo-utils into the monorepo and push to the master branch, we can put other packages in, like hexo-pagination, hexo-log, hexo-front-matters, hexo-server, etc.

I'd like to use yarn workspaces with the latest version of yarn (yarn v3), as pnpm workspaces need Microsoft Rush as its backend while the Lerna is kinda inactive.

[SukkaW]

I am monitoring the development of https://turborepo.org/

Maybe we can use the npm + turborepo approach.

---

And here is how we should go with:

• We will use Lerna for package bootstrapping and publishing.
  • Although lerna hasn't been active for a year, it is still used by many large projects like facebook/jest and typescript-eslint.
  • Lerna has interactive CLI for managing and publishing. E.g. during lerna publish, it will automatically detect if a package has been changed, and interactively ask you if a version bump is required.
• We will use turbo for tasks running.
  • turbo has smart tasks scheduling. E.g. turbo can realize hexo-util is a dependency of hexo, so hexo-util:test should be run before hexo:test, while lerna run can't do that.
  • turbo is faster than lerna:
    

    https://github.com/SukkaW/hexo/tree/monorepo

[yoshinorin]

@SukkaW
Is there anything that we can do?
Personally, I would like to migrate to monorepo even if a little bit aggressive.

[so1ve]

Is lerna still necessary now? We can use pnpm / yarn / even npm's native workspaces.

@SukkaW If you want, I can do some work under the v7 milestone.

[SukkaW]

Is lerna still necessary now? We can use pnpm / yarn / even npm's native workspaces.

@SukkaW If you want, I can do some work under the v7 milestone.

Sure! That would be awesome!

[so1ve]

So... turborepo and npm? I don't think turbo is necessary either. Hexo is not large and pnpm works well in most scenarios. I prefer using pnpm(personally)

[SukkaW]

I don't think turbo is necessary either. Hexo is not large and pnpm works well in most scenarios.

There are 75 repos under @hexojs org, and at least one-third or half of them will eventually be merged into the monorepo. I believe turbo is a must-have.

[D-Sketon]

Currently hexo, hexo-util, hexo-cli, hexo-i18n, hexo-fs, hexo-front-matter, hexo-log and warehouse have fully migrated to ts. Do other repositories in monorepo need to be migrated as well?

[yoshinorin]

IMHO, warehouse doesn't need to be included in the monorepo. I believe it can be used as a separate JSON Database, independent of Hexo modules.

And if migrating to a monorepo, it would be better to merge the history of each repositories. This is not only beneficial for tracking history but also respects all of contributors to the plugins.

I think this can be achieved simply by merging the branch of another repository into the repository we want to merge like below.

https://stackoverflow.com/questions/1425892/how-do-you-merge-two-git-repositories

[uiolee]

it would be better to merge the history of each repositories

But that will make the repository's size larger