0.4.0

NEW FEATURES

1. resub and resuball function can reference regex patterns from template vars
2. exact_space match variable indicator
3. Added group record function
4. Added group set function
5. Added support to groups for absolute path
6. Added support for filter list to ttp.add_template method, to allow filtering of child templates that need to be loaded from text file that contains several templates or from template that has child tags defined, filter contains names of child templates to be loaded
7. Added filter support for parent templates names, now they can be filtered as well - if no name of template specified in filter, that template content will be ignored and no results produced
8. Added gpvlookup to lookup match results against glob patterns
9. Added support for geoip2 db lookups - geoip_lookup match variable function
10. Added support for lookup tag databse loaders
11. Added match variable geoip_lookup function
12. Added support for group contains_val and exclude_val to reference variable;
13. Added support for match variable equal, non_equal and cidr_match to reference variable;
14. Added cli tool -v/--vars keyword to supply json string-dictionary of variables to add to ttp parser object
15. Added colouring to terminal returner using colorama
16. Added group validate function that used Cerberus to check group result
17. Modified outputter validate function to have features similar to group validate function
18. Match variable function equal/notequal/exclude/append/prepend/contains/replaceall can reference Template variables
19. Added TTP variable get_timestamp_iso function
20. Added support for flat_list result structure to flatten overall results
21. Added group 'expand' function to expand dot separated variables names to nested dictionary
22. Added 'clear_result' method to ttp parser object
23. Added outputter support for N2G formatter to produce xml diagrams
24. Added group lookup function
25. added support for "doc" tag - TTP will not complain about it and this tag can be used to put template description in it
26. Added support for group functions to reference list of functions in template variable using chain or functions attribute
27. Added default name to templates - "root_template", added template_name attribute to add_input, set_input, clear_input methods; get_inout_load method now returns dictionary of templates and input names.
28. Added syslog returner to outputs using logging syslog handler to return results to syslog server over udp
29. Added support for netmiko as an input source function
30. Added support for "headers" indicator to dynamically construct regexes line to match tabulated data using exact number of spaces
31. Added support for Nornir as an input source
32. Added reference to variables for match variable conditional functions to introduce dynamic filtering capabilities

CHANGES

1. Modified variable and pattern line escaping behavior, to fix bug with several match variables with digits in name e.g. d1_1, d1_2
2. Modified behavior of DEFAULTS handling in results object to add them at the very end and make them the least preferred in comparison to results values.
3. Modified per_template results method handling to flatten results structure by converting overall result to dictionary for named groups and using single list for anonymous group
4. Results object processgrp now adds default to results instead of save_results method
5. Updated template object form results to handle anonymous and named groups so that results for both groups included in output
6. Deleted get_input_commands_list and get_input_commands_dict parser objects, as they no longer deemed to be useful
7. Data on add associated with templates straight away instead of being saved in ttp.__data list. As a result, templates must be added before data will be added, if data added before template, that data will not be processed, as there are not templates/inputs to associate it with.
8. Outputters now load tag data and merge it with attributes if its a dictionary.

TESTS

1. test1-48 for variable with digits in name and for exact indicator
2. Test 1-50-x for group record and set functions
3. Test 1-51 to test group absolute path handling
4. Test 1-52 to test let vs set behavior when defaults must be overridden by match values
5. Test 1-53 for exact_space indicator
6. Test 29-1 and 29-2 gpvlookup tests
7. Test 1-50 for group macro attribute with comma separated macro names
8. Added tests 1-56, 1-57 to test match cidr_match and group contains_val functions referencing to variable
9. Test 1-58 to test group validate function
10. Results forming for template with anonymous and named group to test change-5
11. Started moving tests to pytest - 31 tests added so far

DOCS

1. Added HOWTO for parsing text data structured in tables
2. Added docs for group record and set functions
3. Added docs for group absolute path
4. Added docs to 'ignore' indicator saying that regex can come from re patterns or from template variables
5. Added docs to cli tool about new keywords options -ot and -s
6. Added docs about exact_space indicator
7. Added docs for gpvlookup match variable function
8. Added docs for geoip_lookup function
9. Updated docs for group contains_val and exclude_val, for match variable equal, non_equal and cidr_match to mention that they can reference variable
10. Added docs for group 'expand' function
11. Updated docstring for 'flat_list' structure
12. Flattened docs structure, created separate section for results structure
13. Wrote docs for headers indicator
14. Updated summary notes to installation page for other packages to be installed for certain features to work
15. Documented N2G output formatter
16. Changed output formatters to include all supported arguments within each of them, rather then having separate section for formatter's attributes.
17. Added new cli tool -v/--vars argument in the docs
18. Added docs about coloring terminal returner output using colorama
19. Modified inputs docs - merged attributes and parameters together
20. Added docs for syslog returner
21. Added docs for netmiko source
22. Added docs about Norninr source
23. Added docs about group validate function

BUGS

1. fixed bug with get_attributes function when empty match variable item give e.g. {{ var | | func }}
2. Fixed bug for variables having digit in name, that digit was substituted with \d+ pattern, resulting in match varibales have the same name, which is prohibited for regex groups
3. Fixed bug for py2.7 struct load for exec() function when text contains "True" of "False" by skipping the error, as exec chokes on it while eval used to load it. Not an issue in py3 though.
4. Fixed problem with set overriding let values due to default values override results dictionary when group method="table" and we have dynamic path in use to form dictionary, override happen due to the fact that each match was written to results straight away together with defaults values and if we have let function and unconditional set (set to defaults) values for same variable, let value was overridden by defaults from set.
5. Fixed handling of absolute path together with anonymous parent group
6. Fixed template form_results function to handle the case when anonymous group produces dictionary results in per_template mode
7. Load struct function was fixed to correctly handle the case when template vars tag has include attribute but has not text defined in XML element
8. Fixed traceback display issue with get_attributes function, log string had wrong formatting
9. Fixed truncate function and added test to test it
   10.Fixed issue with group macro having several macro names separated by comma, added test to test it
10. TTP duplicate logs when used as a module - changed log config function
11. Fixed bug with anonymous template introduced by adding r'' raw strings
12. Fixed csv formatter results handling when they contains integers
13. Fixed this bug:
interface {{ interface }} description {{ description | ORPHRASE }} was returning no results for interfaces without descriptions 15. Fixed CSV formatter to handle "," properly in matches, by vrapping items in '"' or "'" characters