Roadmap

[ChristopherPHolder]

Motivation

Every PR could potentially include a performance regression.
Therefore need a way to continuously measure the performance to inform developers about performance slowdown before the PR gets merged.

To accomplish this, we need a way to compare the measures of the current branch against the measure of the target branch.

Suggested solution

The solution will use @push-based/user-flow and wrap it in a GitHub action to interact with the PR.

There are different building blocks necessary to implement this feature:

Legend:

• ‼ - important for second contact DX: Implement DX for second contact user-flow#45

Building blocks

• report visualization - a comment in the PR that a human can interpret e.g. MD table
• report comparison - logic that compares 2 measures and calculates the diff
  • report comparison visualization - a comment in the PR that a human can interpret e.g. MD table
• assertions - a way to let the gh action fail if the budget is exceeded
  • performance budgets - A definition of a data structure that can be reflected as JSON

Implementation

We have to split into 2 responsibilities/packages here:

• 💻 the CLI - @push-based/user-flow
• ⚙ the CI Action - CI related code e.g. GitHub Action @push-based/user-flow-gh-action

To implement the building blocks we need to solve a couple of core problems:

• ‼⚙ interact with a PR and maintain a comment - Introduce composite action for a clean project architecture and tooling #80
  • ‼⚙ decide on comment update and re-run logic - feat: Implement the comment interaction #10
• 💻 pure function to compare 2 (reduced?) reports
  Compare and visualise two reports in MD format user-flow#94
  • 💻 handling git checkout and target branch
• 💻 performance budgets - Lighthouse budgets and/or custom version
  • 💻 LH budgets technically is a separate LH audit => RFC: Add support for Performance Budgets GoogleChrome/lighthouse#6053
  • 💻 custom budgets will be applicable to all measurement modes (navigation, timespan, snapshot), LH budgets just to navigation
• ‼💻 visualization of report -> --format md
  • 💻 have the different measures on the vertical axis is the way to go as there can be a lot.
• 💻 visualization of comparison
  • 💻show the diff in the same unit as the measure e.g. 7 / 2 ⬇
  • 💻 Implement assert command
  • ⚙ aggregate reports into one text piece
• ‼💻Setup GH action automatically - feat: extend init param with file creation for gh action user-flow#198

Documentation

• It should link to the user-flow repo https://github.com/push-based/user-flow
• How to set it up?
• Use Cases
  • Report visualization
  • Report comparison visualization
  • Setup assertions

References

• GH comments
• Comparison visualizations:
  • 
  • 
  • 

[BioPhoton]

[UserFlowName] - Report Visualization

Date/Time: 2022-11-03 15:06

Report Summary

Step Name	Gather Mode	Performance	Accessibility	Best Practices	Seo	Pwa
Navigate to coffee cart	navigation	90	97	100	83	30
Select coffee	timespan	11/11	-	7/7	-	-
Coffee selected	snapshot	Ø 3/3	15/16	5/5	7/9	-
Checkout order	timespan	13/13	-	7/7	-	-
Order checked out	snapshot	Ø 3/3	17/18	5/5	7/9	-
Submit order	timespan	13/13	-	7/7	-	-
Order submitted	snapshot	Ø 3/3	15/16	5/5	7/9	-

Settings

table here

[BioPhoton]

[UserFlowName] - Report Comparison Visualization

before 80 now 90

Date/Time: 2022-11-03 15:06

Comparison Summary

Step Name	Gather Mode	Performance	Accessibility	Best Practices	Seo	Pwa
Navigate to coffee cart	navigation	90 (10⏫)	97	100	83	30
Select coffee	timespan	10/11 (1⏬)	-	7/7	-	-
Coffee selected	snapshot	Ø 3/3	16/16 (1⏫)	5/5	7/9	-
Checkout order	timespan	13/13	-	5/7 (2⏬)	-	-
Order checked out	snapshot	Ø 3/3	17/18	5/5	7/9	-
Submit order	timespan	13/13	-	6/7 (1⏬)	-	-
Order submitted	snapshot	Ø 3/3	15/16	5/5	7/9	-

Settings

table here

[BioPhoton]

[UserFlowName] - Report Comparison Visualization v2

before 80 now 90

Date/Time: 2022-11-03 15:06

Comparison Summary

Step Name	Gather Mode	Performance	Accessibility	Best Practices	Seo	Pwa
Navigate to coffee cart	navigation	90 (10💚)	97	100	83	30
Select coffee	timespan	10/11 (1🔻)	-	7/7	-	-
Coffee selected	snapshot	Ø 3/3	16/16 (1💚)	5/5	7/9	-
Checkout order	timespan	13/13	-	5/7 (2🔻)	-	-
Order checked out	snapshot	Ø 3/3	17/18	5/5	7/9	-
Submit order	timespan	13/13	-	6/7 (1🔻)	-	-
Order submitted	snapshot	Ø 3/3	15/16	5/5	7/9	-

Settings

table here

[BioPhoton]

For the following tasks we need to do more research:

• handling git checkout and target branch => maybe better in the CLI?

[BioPhoton]

Related to:

• feat: Implement the comment interaction #10
• environment variables and CI mode user-flow#15

[ChristopherPHolder]

Related to: push-based/user-flow#94