Skip to content

Commit

Permalink
Mason performance issue felangel/mason#1257
Browse files Browse the repository at this point in the history
  • Loading branch information
absar committed Mar 7, 2024
1 parent 73d8fdc commit d1b907c
Show file tree
Hide file tree
Showing 16 changed files with 1,028 additions and 0 deletions.
58 changes: 58 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Miscellaneous
*.class
*.log
*.pyc
*.swp
.DS_Store
.atom/
.buildlog/
.history
.svn/
migrate_working_dir/

# IntelliJ related
*.iml
*.ipr
*.iws
.idea/

# The .vscode folder contains launch configuration and tasks you configure in
# VS Code which you may wish to be included in version control, so this line
# is commented out by default.
#.vscode/

# Flutter/Dart/Pub related
**/doc/api/
**/ios/Flutter/.last_build_id
.dart_tool/
.flutter-plugins
.flutter-plugins-dependencies
.pub-cache/
.pub/
/build/

# Symbolication related
app.*.symbols

# Obfuscation related
app.*.map.json

# Android Studio will place build artifacts here
/android/app/debug
/android/app/profile
/android/app/release

# Mason
.mason/

# Since the flutter project is only used to generate code, we should not check-in generated files
/lib/

# Exceptions to above rules
!bricks/*/lib
!/bricks/code_gen/__brick__/integration_test/
!/bricks/code_gen/__brick__/test/

mason-lock.json
/integration_test/
/test/
27 changes: 27 additions & 0 deletions .metadata
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# This file tracks properties of this Flutter project.
# Used by Flutter tool to assess capabilities and perform upgrades etc.
#
# This file should be version controlled and should not be manually edited.

version:
revision: "41456452f29d64e8deb623a3c927524bcf9f111b"
channel: "stable"

project_type: app

# Tracks metadata for the flutter migrate command
migration:
platforms:
- platform: root
create_revision: 41456452f29d64e8deb623a3c927524bcf9f111b
base_revision: 41456452f29d64e8deb623a3c927524bcf9f111b

# User provided section

# List of Local paths (relative to this file) that should be
# ignored by the migrate tool.
#
# Files that are not part of the templates will be ignored by default.
unmanaged_files:
- 'lib/main.dart'
- 'ios/Runner.xcodeproj/project.pbxproj'
57 changes: 57 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# mason_performance_issue_1257
https://github.com/felangel/mason/issues/1257

Generating files using Mason from a list which is not a simple string rather a `List<Map<String, dynamic>>` takes exponentially greater time.
For example `{{#updateMethods}}{{{usecaseName.snakeCase()}}}.dart{{/updateMethods}}` takes over 10 minutes to generate 4 files on Windows 10 i7 processor with 16gb RAM, the process consumes only around 25% CPU and negligible disk and memory activity, however if you change the name extraction from triple `{` to double e.g. `{{#updateMethods}}{{usecaseName.snakeCase()}}.dart{{/updateMethods}}` then it generates in 5 seconds, however it only generates a single file by appending all names, with all the content meant for the 4 files.
On further investigation we found that increasing number of `context.vars` might be the reason it slows down, e.g. in the config file `/bricks/templates/wallet.json` add more `properties`.
In the pre_gen.dart if you reduce `context.vars` by commenting these lines then the time is reduced exponentially, solving this issue will help adopt Mason for slightly complex use cases by reducing
generation time from hours to seconds:
```
'notNullProperties': notNullProperties(properties),
'notNullViewProperties': notNullViewProperties(properties),
```

Setup:
Latest mason CLI, Flutter 3.16.9
Windows 10

Steps:
`mason get`
`mason make code_gen -c ./bricks/templates/wallet.json --on-conflict overwrite`

Sample input data:
```json
{
"project_name": "Project1",
"feature_name": "Wallet",
"feature_directory": "wallet",
"properties": [
{ "name": "property1", "entityDataType": "UniqueId?", "viewDataType": "UniqueId?"},
{ "name": "property2", "entityDataType": "double", "viewDataType": "double"},
{ "name": "property3", "entityDataType": "double", "viewDataType": "double"},
{ "name": "property4", "entityDataType": "double", "viewDataType": "double"},
{ "name": "property5", "entityDataType": "double", "viewDataType": "double"},
{ "name": "property6", "entityDataType": "double", "viewDataType": "double"},
{ "name": "property7", "entityDataType": "DateTime?", "viewDataType": "DateTime?"},
{ "name": "property8", "entityDataType": "DateTime?", "viewDataType": "DateTime?"},
{ "name": "property9", "entityDataType": "DateTime?", "viewDataType": "DateTime?"},
{ "name": "property10", "entityDataType": "DateTime?", "viewDataType": "DateTime?"},
{ "name": "property11", "entityDataType": "DateTime?", "viewDataType": "DateTime?"},
{ "name": "property12", "entityDataType": "DateTime?", "viewDataType": "DateTime?"},
{ "name": "property13", "entityDataType": "DateTime?", "viewDataType": "DateTime?"},
{ "name": "property14", "entityDataType": "DateTime?", "viewDataType": "DateTime?"},
{ "name": "property15", "entityDataType": "DateTime?", "viewDataType": "DateTime?"},
{ "name": "property16", "entityDataType": "int", "viewDataType": "int"},
{ "name": "property17", "entityDataType": "int", "viewDataType": "int"},
{ "name": "property18", "entityDataType": "int", "viewDataType": "int"},
{ "name": "property19", "entityDataType": "int", "viewDataType": "int"},
{ "name": "property20", "entityDataType": "int", "viewDataType": "int"}
],
"updateMethods": [
{ "usecaseName": "UpdateBalance", "paramsToUpdate": [{"paramName": "balance", "paramType": "double"}]},
{ "usecaseName": "RewardA", "paramsToUpdate": [{"paramName": "balance", "paramType": "double"}, {"paramName": "rewardedCoins", "paramType": "double"}, {"paramName": "rewardedAt", "paramType": "DateTime"}]},
{ "usecaseName": "RewardB", "paramsToUpdate": [{"paramName": "balance", "paramType": "double"}, {"paramName": "rewardedCoins", "paramType": "double"}, {"paramName": "rewardedAt", "paramType": "DateTime"}]},
{ "usecaseName": "UpdateName", "paramsToUpdate": [{"paramName": "name", "paramType": "String"}]}
]
}
```
16 changes: 16 additions & 0 deletions analysis_options.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
include: package:flutter_lints/flutter.yaml
analyzer:
language:
strict-raw-types: true
exclude:
- "lib/common/localization/generated/*"
- "bricks/code_gen/__brick__/*"

linter:
rules:
- always_declare_return_types
- prefer_single_quotes
- sort_child_properties_last
- unawaited_futures
- unsafe_html
- use_super_parameters
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import 'package:dartz/dartz.dart' show Either, Left;

import 'package:{{project_name}}/features/{{feature_directory}}/domain/entity/{{feature_name.snakeCase()}}.dart';

class Save{{feature_name.pascalCase()}} {

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{{#updateMethods}}import 'package:dartz/dartz.dart' show Either, Left, Right;

class {{usecaseName}} {
final {{feature_name.pascalCase()}}Repository _repository;
final LogService _logger;

{{usecaseName}}(this._repository, this._logger);

@override
Future<Either<Failure, {{feature_name.pascalCase()}}>> call() async {
try {
late {{feature_name.pascalCase()}} {{feature_name.camelCase()}};
Failure? errorResult;

final result = await _repository.{{usecaseName.camelCase()}}({{#paramsToUpdate}}
{{paramName}}: {{feature_name.camelCase()}}.{{paramName}},{{/paramsToUpdate}}
);
return result.fold(
(failure) => Left(failure),
(success) => Right({{feature_name.camelCase()}}),
);
} catch (e, s) {
_logger.e('$_logPrefix error', e, s);
return const Left(SomethingWrongContactSupportError(secondaryMessage: _logPrefix));
}
}
}
{{/updateMethods}}
16 changes: 16 additions & 0 deletions bricks/code_gen/brick.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: code_gen
description: A brick to generate a feature
version: 0.0.1
repository:

vars:
project_name:
default: mason_performance_issue_1257
type: string
description: The project name
prompt: What is the project name?
feature_name:
default: wallet
type: string
description: The feature name
prompt: What is the feature name?
4 changes: 4 additions & 0 deletions bricks/code_gen/hooks/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
build/
.dart_tool
.packages
pubspec.lock
89 changes: 89 additions & 0 deletions bricks/code_gen/hooks/pre_gen.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
import 'package:mason/mason.dart';

import 'property.dart';
import 'update_method_properties.dart';

Future<void> run(HookContext context) async {
final hasProperties =
context.vars['properties'] != null && (context.vars['properties'] as List).isNotEmpty;

final hasUpdateMethods =
context.vars['updateMethods'] != null && (context.vars['updateMethods'] as List).isNotEmpty;

final properties = <Map<String, dynamic>>[];

if (hasProperties) {
final seededProperties = context.vars['properties'] as List;
for (final property in seededProperties) {
Property.addProperty(
properties,
Property.fromMap(property as Map<String, dynamic>),
);
}
}

final updateMethods = <Map<String, dynamic>>[];

if (hasUpdateMethods) {
final seededUpdateMethods = context.vars['updateMethods'] as List;
for (final method in seededUpdateMethods) {
UpdateMethodProperties.addMethod(
UpdateMethodProperties.fromMap(method as Map<String, dynamic>),
updateMethods,
);
}
}

context.vars.removeWhere((key, value) => ['properties', 'updateMethods'].contains(key));
context.vars = {
...context.vars,
'properties': properties,
'first2Properties': firstNProperties(properties, 2),
'first2PropertiesReversed': firstNProperties(properties, 2, reversed: true),
'first3Properties': firstNProperties(properties, 3),
'first3PropertiesReversed': firstNProperties(properties, 3, reversed: true),
'first4Properties': firstNProperties(properties, 4),
'notNullProperties': notNullProperties(properties),
'notNullViewProperties': notNullViewProperties(properties),
'hasProperties': properties.isNotEmpty,
'updateMethods': updateMethods,
'hasUpdateMethods': updateMethods.isNotEmpty,
};
}

List<Map<String, dynamic>> firstNProperties(
List<Map<String, dynamic>> properties,
int n, {
bool reversed = false,
}) {
var result = properties.take(n).map((e) => Map<String, dynamic>.from(e)).toList();
return (reversed ? result.reversed.toList() : result)..last['isLastProperty'] = true;
}

List<Map<String, dynamic>> notNullProperties(
List<Map<String, dynamic>> properties, {
bool reversed = false,
}) {
final result = properties
.where((it) => !(it['isNullable'] as bool))
.map((e) => Map<String, dynamic>.from(e))
.toList();
return (reversed ? result.reversed.toList() : result)..last['isLastProperty'] = true;
}

List<Map<String, dynamic>> notNullViewProperties(
List<Map<String, dynamic>> properties, {
bool reversed = false,
}) {
final result = properties
.where((it) => !(it['isViewNullable'] as bool))
.map((e) => Map<String, dynamic>.from(e))
.toList();
return (reversed ? result.reversed.toList() : result)..last['isLastProperty'] = true;
}

Future<void> preGenHookWrapper(HookContext context,
{Function(Map<String, dynamic> value)? preGenHookChanged}) async {
await run(context);
preGenHookChanged?.call(context.vars);
}
45 changes: 45 additions & 0 deletions bricks/code_gen/hooks/property.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
import 'dart:convert';

class Property {
const Property({
required this.name,
required this.entityDataType,
required this.viewDataType,
this.isNullable = false,
this.isViewNullable = false,
});

final String name;
final String entityDataType;
final String viewDataType;

final bool isNullable;
final bool isViewNullable;

factory Property.fromMap(Map<String, dynamic> map) {
return Property(
name: map['name'] as String,
entityDataType: map['entityDataType'] as String,
viewDataType: map['viewDataType'] as String,
);
}

factory Property.fromJson(String source) =>
Property.fromMap(json.decode(source) as Map<String, dynamic>);

static void addProperty(
List<Map<String, dynamic>> properties,
Property property,
) {
properties
..forEach((e) => e['isLastProperty'] = false)
..add({
'name': property.name,
'entityDataType': property.entityDataType,
'viewDataType': property.viewDataType,
'isNullable': property.isNullable,
'isViewNullable': property.isViewNullable,
'isLastProperty': true,
});
}
}
9 changes: 9 additions & 0 deletions bricks/code_gen/hooks/pubspec.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
name: mason_performance_issue_1257_hooks

environment:
sdk: '>=3.2.0 <4.0.0'

dependencies:
mason: ^0.1.0-dev
yaml: ^3.1.2
dart_style: ^2.3.4
Loading

0 comments on commit d1b907c

Please sign in to comment.