v0.12.90

This is a pre-release in preparation for v0.13.

This release adds an all-new template and variables system built on top of the workflow configuration mechanism. It also brings numerous improvements throughout AliECS, including reorganized path handling, task cleanup facilities, FLP Suite workflow versioning, an environment fast destroy operation, as well as API extensions for interfacing with the variables system.

• Variables system:
  • [coconut] Print variables in coconut env show
  • [coconut] Variables output in role query
  • [coconut] Acquire --extra-vars when creating environment
  • [coconut] Correctly handle env create with empty extra-vars
  • [coconut] Extract a new componentcfg package from configurationutil.go
  • [coconut] Add root role column in coconut env list
  • [coconut] Sort printed maps by key
  • [common] Make gera.Map implement yaml.Unmarshaler
  • [core] New hierarchical property map
  • [core] Add WrappedAndFlattened to hierarchical map
  • [core] Extend o2control.proto to accept environment properties
  • [core] Update proto and regenerate
  • [core] Add StringMap to gera package
  • [core] Add support for defaults and vars in confsys
  • [core] Implement variables KV in workflow
  • [core] Add properties to TaskInfo in protofile
  • [core] Task properties
  • [core] Use the full roleBase unmarshal path in aggregatorRole
  • [core] Output vars in GetEnvironment
  • [core][OCTRL-73] Add properties to running task
  • [core][OCTRL-73] Add properties to buildcommand properties
  • [core] Correctly unmarshal properties map into task.Class
  • [core] Add Copy method to gera.StringWrapMap
  • [core] Add defaults map to task template
  • [core] gera.FlattenStack convenience function
• Reorganized path handling:
  • [config] Add trailing slash to viper defaultRepo field
  • [core] Manage a single writable directory path
  • [misc] Update default executor path to a generic one
  • [repos] Consistent trailing slash use for repos
• Mesos lost task cleanup on startup:
  • [core] Persistent FID with reconciliation for previous running tasks
  • [core] Repos and runcounter.txt are stored inside coreWorkingDir
  • [core] Sanitize viper paths to remove trailing slash
  • [core] Use filepath.Join to concatenate paths
  • [core] Use path/filepath to remove trailing '/'
  • [core] Remove repositoriesPath from configuration
  • [core] Cleaner reposPath approach in service
  • [core] Use GetReposPath in repos manager
  • [core] Use service to get reposPath
• Mesos task cleanup on incoming UNIX signals:
  • [core] Handle UNIX signals
  • [core] Use taskman to cleanup and to get tasks
  • [core] Use proper exit status
• Support for per-repo default revision (FLP Suite workflow versioning):
  • [core+coconut] Introduce global default branch for repos
  • [core+coconut] Add default bracnh option for new repos and improve API and coconut
  • [core] Support perisistent default revisions per repo
  • [core] Update backend when a default-revision is explicitly set
  • [docs] Improve coconut repo add help dialogue
• Template engine:
  • [core] Implement new template engine
  • [core] Correctly handle precedence of template resolution
  • [core] Template support in constraints and objStack
  • [core] Add support for wrapping slices in template
  • [core] Parametrize basic task arguments and env vars
  • [core] Run template system on Task.properties
  • [core] Implement GetKeysByPrefix for all configuration.Sources
  • [core] Implement confsys.GetComponentConfiguration
  • [core] Implement To/FromJson and GetConfig functions for template system
  • [core] Template Task.Value and User
  • [core] Allow variables to control the expansion of an iteratorRole
  • [core] Fix issue with user provided vars not accessible
  • [core] Ensure correct template output when the result is a slice
  • [core] Fix bug with nil TaskCommandInfo preventing deployment
  • [core] Facilities for pushing runtime variables back up the stack
  • [core] Propagate task result to parent taskRole
  • [core] Include task class defaults in varStack for TaskCommandInfo
• Environment fast destroy feature:
  • [coconut][OCTRL-206] Add option in coconut to allow destroy env in running state
  • [core][OCTRL-206] Update core to destroy running env if specified
• Miscellaneous:
  • [build] Make "make generate" independent from $PATH and $GOPATH
  • [build] Improve Makefile output
  • [core] New proposed protofile for DCS integration
  • [core] Move NewUnixTimestamp to common/utils
  • [core] Suppress periodic output unless veryVerbose=true
  • [executor] Lower logrus severity of task stout/stderr
  • [executor] Make incoming MESSAGE handling concurrent
  • [executor] Ensure correct log level for task stdout/stderr

v0.12.2

This release fixes a bug which resulted in missing log messages.

• Miscellaneous:
  • [core] Fill out logrus fields

v0.12.1

This release fixes a build system issue which resulted in a broken core binary.

• Miscellaneous:
  • [build] Bump dependencies
  • [build] Fix viper dependency and go mod tidy

v0.12.0

This release brings support for basic tasks and other improvements.

• Basic tasks:
  • [core] Add BASIC control mode
  • [core] Normalize interface of task.Task
  • [core] Fix CommandInfo for basic tasks and handle BASIC_TASK_TERMINATED
  • [core] Make transition reacting to END_OF_STREAM asynchronous
  • [core] SafeToStop mechanism in task.Task
  • [executor] Implement BasicTask transitioner
  • [executor] Fix crash on rerunning basic task within existing env
  • [executor] Implement basic task handlers and transitioner
  • [occ][executor] Add BASIC_TASK_TERMINATED event and update END_OF_STREAM
• Executor refactor:
  • [executor] Refactor for lockless concurrency and ControllableTask
  • [executor] Further refactor Transitioner-RPC relationship
  • [executor] Remove taskCmd from executor task base
• Miscellaneous:
  • [build] Support building on Mac
  • [coconut + core] Improvements in template list feature
  • [common] Improve log processing
  • [core] Tone down debug output
  • [core] Cleanup on failed deployment
  • [core] Correctly report deployment error
  • [misc] Add fx3-copy-executors utility script for quick lab deployment
  • [misc] Add protoString utility function to utils
  • [misc] Migrate from dep to Go Modules
  • [misc] Refresh coconut documentation

v0.11.1

This release brings a gRPC API fix to facilitate the implementation of the repo refresh feature in the AliECS GUI.

• Miscellaneous:
  • [core] Update repo rpcs to return errors
  • [core] Regenerate proto

v0.11.0

This release brings API improvements to the Git-based workflow configuration mechanism in order to support git-based workflow configuration in the AliECS GUI. It also adds a task configuration import and query feature to the coconut configuration command which can be used to manage task configuration in Consul, in preparation for the upcoming task configuration push mechanism.

• Workflow template API:
  • [coconut][core] Implement coconut template queries + API
  • [core] Task class always identified by hash
  • [core] Regenerate proto
  • [core] Update defaults for templ list API
• Task configuration Consul import:
  • [coconut] Update documentation
  • [coconut] Generate coconut user documentation
  • [coconut] coconut conf list command
  • [coconut] coconut conf show command
  • [coconut] coconut conf history command
  • [coconut] coconut conf import command
• Miscellaneous:
  • [executor] Fix bug which prevented configuration payload to be printed
  • [executor] Improve configuration kv output

v0.10.1

Merge pull request #61 from ktf/fix-warnings

Fix warnings on macOS

v0.10.0

This release brings a major refactor of the configuration system, including a new git-based workflow configuration mechanism.

• git-based workflow configuration:
  • [coconut] Add repo add, remove, default and list commands
  • [coconut] Allow for indexed repo operations & improvements
  • [core] Support a dir structure for tasks and workflows
  • [core] Add support for revisions (branch, tags, long hashes)
  • [core] Add RefreshRepos()
  • [core] Add full task path for task identifiers
  • [core] Allow workflows with cross-repo tasks
  • [core] Discover and add repos according to default repos dir structure
  • [core] Change workflowConfigurationUri to globalConfigurationUri
  • [core] Add support for persistent default_repo consul/file backend
  • [core] Add godocs for control repo API
  • [core] Add default workflows repo
• Miscellaneous:
  • [coconut] Add documentation generator
  • [coconut] Improve and generate command reference documentation
  • [misc] Rewrite and improve user documentation
  • [occ] Add example configuration for testing OCC with peanut

v0.9.4

This release adds support for an O² role string in OCC and AliECS core.

• O² role support:
  • [core] Push O2_ROLE env var to task
  • [occ] Add readout-specific documentation on configure argument
  • [occ] Expose O² role in RuntimeControlledObject

v0.9.3.2

This release brings a single build system fix related to Boost include path handling.

• Build:
  • [build] Rely on target-provided Boost include path