COBREXA 2.0

[stelmo]

Let's add thoughts and ideas for COBREXA 2.0 here:

Breaking changes (v2.0)

• Currently we use reactions and fluxes interchangeably and actually mean variables. We should not do this, as future models will certainly predict more than just fluxes, e.g. enzyme constrained models. See Sort out the reactions vs fluxes vs variables #613 as well as Change reaction constraints using un-mangled reaction id #633
• Add a result type wrapper to allow analysis functions pipes Result wrapper #763
• Make function discoverability easier by reorganizing everything into modules Organize everything into submodules #663
• Consistently name kwargs Consistent argument/kwarg naming #623
• Rename MetabolicModel to AbstractMetabolicModel, this is better Julia style. See Rename MetabolicModel to AbstractMetabolicModel #687
• Rename analysis functions: GeckoModel and SMomentModel should just be EnzymeConstrainedModels
• Change the names of CoreModel, CoreCoupledModel, and StandardModel
• Combine CoreModel and CoreCoupledModel MatrixModel should get 🪓ed
• Rename all grr to gra
• Deprecate and remove moment modifications. See Delete moment algorithm modification #686
• Remove the @add_reaction macro - just use the add_reaction functions like a human Remove @add_reactions #697
• Rename reaction_gene_association to reaction_gene_associations Rename reaction_gene_association to reaction_gene_associations #721
• Improve the model construction functions. Make it easier to construct a sensible model. In particular, using non-human readable IDs will basically ruin lots of our existing looks_like functions and tutorials etc. Full-text search for metabolite/reaction/gene IDs #652 starts along this path
• Fix and make community model construction more robust add_model_with_exchanges produces an invalid model #591
• Make convenience functions for enzyme constrained models with ObjectModel Add accessors for enzyme associated data #724
• Remove sparse matrix from show and pretty printing.. it usually doesn't display nicely and just clutters everything Compress model pretty print #698
• Fix references to GeneAssociations when they should be GeneAssociationsDNF e.g. in MatrixModel docstrings
• Add with_ functions or wrappers for model extensions. E.g. a smooth way to add coupling constraints, that could be nested etc.
• Pick between reverse and backward keywords (currently mixed usage) Replace uses of :reverse with :backward to be consistent #722
• After module re-org, no functions are automatically exported. E.g. to use load_model I first need to do using COBREXA.IO. Maybe we should export everything by default to make the user experience "faster"? Add convenience macro that loads all the functions in modules #716
• Improve user experience with sampling Sampling size parameters are not very intuitive #657 #665
• Convert MMDF to a wrapper max_min_driving_force and moment should be a ModelWrapper #614
• Remove ability to construct reactions like this: 1.0 * nadh + 4.0 * h_c + q8 ← 1.0 * q8h2 + 1.0 * nad + 3.0 * h_p Delete reaction overloads #718
• Add ReactionForward, ReactionBackward, ReactionBidirectional as overloads to Reaction instead of the current symbol passing approach Remove switching argument for reaction constructors #740
• Make pFBA is pipe-able pFBA pipe #747
• Check that other analysis functions are pipe-able
• Add fields listed in Carey 2020 to ObjectModel to make the structure conform to community standards Add model annotation details #750
• Ensure all the with_ functions have analogs for all all ObjectModel reconstruction functions. E.g. currently with_added_reactions does not exist for ObjectModel Add more reconstruction functions to ObjectModel #739
• change_constraint has kwarg lower_bound instead of lower - pick one for these types of functions Homogenize bound kwargs #746
• name constraints in gecko to make modification of opt problem easier
• Deprecate crowding (they won't work with a general metabolic model) Remove crowding (obsolete via SMOMENT) #725
• Allow loopless to be applied to any model e.g. enzyme constrained models Generalize loopless modification #745
• Parse gene reaction rules more robustly Incorrect gene reaction rule parsing #711
• Extend summary functions with semantics
• Make GRR DNF access optional for models that do not require it, for example SBML can evaluate KOs directly on the boolean formula. (IMPORTANT.)
• Convert knockout to a wrapper to actually work with reactions instead of plain variables.
• Be able to import and run enzyme constrained analysis on these matlab models through a general purpose enzymewrappermodel (Chalmer matlab EC models). Not necessary, construct EC models sensibly
• Consistently use the same delimiter for IDs and make it a const: const delim = "#"
• Rework community fba Fix cFBA implementation #764
• Add ability to add multiple capacity constraints to SimplifiedEnzymeModel Extend simplified enzyme model to include multiple capacity constraints #772
• Add traits somehow with https://github.com/oxinabox/Tricks.jl/ With Julia 1.10 this would be in the base language, delay until then
• Delete ObjectModel 🪓
• Delete MatrixModel 🪓
• Add methods to build a model directly with SBMLModel
• Add support for ME models (read in SBTab files) Add support for ME models #664

Close longstanding issues

• Enzyme models with mangled IDs Change reaction constraints using un-mangled reaction id #633
• Model conversion issues Conversion from MATModel produces opaque errors if fields are missing #709
• Each function should have an example and test in the docstring add many doctests, convert all examples to doctests #73

Non-breaking changes (v.2.1+)

• Add units used by enzyme constrained functions to the docstrings to remove ambiguity (probably also for the community stuff)
• better error messages for model reading, indicate which reaction/gene/metabolite breaks the IO or convert function
• Pretty printing for ModelWithResult type, see Neaten the summarize functions #790
• Need a way to save ObjectModels that have been extended with e.g. kcats etc... ObjectModel needs to be 🪓ed
• Add precompile statements to take advantage of native code caching
• Use ReadableRegex instead of normal regex Not worth the effort
• Implement smetana for community analysis see here for the method

[stelmo]

Here is a potential option we can consider for a dramatic change:

#=
Is this a better way to encapsulate the default cobrexa model types?

Note, I assume we have renamed our abstract type to AbstractMetabolicModel,
thus JSONModel, SBMLModel, MATModel <: AbstractMetabolicModel
=#

abstract type AbstractReaction end
abstract type AbstractMetabolite end
abstract type AbstractGene end

# this should be a subtype of AbstractMetabolicModel but I don't know how to code that in...
struct MetabolicModel{R, M, G} where {R<:AbstractReaction, M<:AbstractMetabolite, G<:AbstractGene}
    id::String
    reactions::Vector{R}
    metabolites::Vector{M}
    genes::Vector{G}
end

# Implement CoreModel like this:
const CoreModel = MetabolicModel{CoreReaction, CoreMetabolite, CoreGene}

struct CoreReaction <: AbstractReaction 
    id :: String 
    column :: SparseVec
    lb :: Float64
    ub :: Float64
    objective_coefficient :: Maybe{Float64}
    coupled :: SparseVec
end

struct CoreMetabolite <: AbstractMetabolite
    id :: String # matches order of column in CoreReaction 
end

struct CoreGene <: AbstractGene
    id :: String # matches order of 
end

# Implement StandardModel like this:
const StandardModel = MetabolicModel{GeneralizedReaction{???}, ...} # here ??? depends on what you want 

struct GeneralizedReaction{T} <: AbstractReaction 
    id :: String
    metadata :: Vector{Pair{Symbol, T}}
end

#= 
metadata1 = [
    Pair(:name => "Pyruvate formate lyase"),
    Pair(:metabolite => ("accoa_c", 1)),
    Pair(:metabolite => ("coa_c", -1)),

    Pair(:upper_bound => 1000.0),

    Pair(:isozyme => (b0902, b0903)),
    Pair(:isozyme => (b0902, b3114)),
]

r1 = GeneralizedReaction("rxn1", metadata1)
=#

#= 
metadata2 = [
    Pair(:name => "Pyruvate formate lyase"),
    Pair(:metabolite => ("accoa_c", 1)),
    Pair(:metabolite => ("coa_c", -1)),

    Pair(:lower_bound => 0.0),


    Pair(:isozyme => ((1, b0902), (2, b0903))),
    Pair(:isozyme => ((3, b0902), (2, b3114))),
]

r2 = GeneralizedReaction("rxn2", metadata2)
=#

#= 
Then r1 and r2 have different types depending on what your model needs. 
=#

[exaexa]

Just a reminder to self -- @mihal-sysbio pointed out we should care about naming our variant of Gecko somehow better (which is reasonable to avoid confusion). It might be GeckoLite (need to verify).

[stelmo]

Let's just called it the EnzymeConstrainedModel... It seems they want GECKO to mean the pipeline where they estimate kcats for the enzyme constrained models instead of the algorithm to solve the problem. Since we won't support Moment anymore there is no conflict with using such a broad term.

[stelmo]

This will be addressed by #719 and #720

[mihai-sysbio]

FYI GECKO has full and light forms already, and it's at v3 now.

[exaexa]

CODE FREEZE FOR 2.0: Feb 10th

[stelmo]

Probably a good idea to push this another week or two

[exaexa]

ok pushed. Feb 17th. :D

[stelmo]

Push another week to 24th

[exaexa]

spoiler: we redone next completely.

[mihai-sysbio]

Another thing to consider adding: more documentation. It was rather difficult for me to understand there are different types of Models that what can and cannot be done with each so that I do right conversion.
Also if versions appear, perhaps it would be possible to keep old documentation available online, e.g., with a dropdown tag with the release version.
On a more philosophical note, should model types be related to file formats or to the type purpose? I'm thinking of JSONModel and SBMLModel which both can take in a genome-scale metabolic model, but also that SBML can handle many other model types.

[exaexa]

This is very true. We're vastly reworking the whole thing in next branch. ConstraintTrees should make the necessity of multiple different model structures basically obsolete. We'll preserve the filetype models, but only as information sources/sinks.

For the documentation, there's a decision to have 100% coverage ONLY by stuff being rendered in literate notebooks, so hopefully that will help. We already did almost the same (except for a few corner cases) for CTs here: https://cobrexa.github.io/ConstraintTrees.jl/stable/

[exaexa]

PS the "old documentation" should be present, the dropdown should be in the lower left corner in the docs.