Releases: LD4P/qa_server
v5.0.2
v5.0.1
v5.0.0
Action Required
This version of qa_server is required to work with QA 5.1.0 and later.
Version 5.0.0 supports QA 5.1.0
Version 4.0.0 supports QA >= 5.0.0 < 5.1.0
This is a major release due to changes to the query_search prepend. Any customization of this method will need to be updated.
Change Log
- prepends for query updated to retain response_header in results
v4.0.0
Action Required
This version of qa_server is required to work with QA 5.0.0 and later.
Version 4.0.0 supports QA 5.0.0 and later.
Version < 4.0.0 supports QA < 5.0.0 and earlier.
This is a major release due to the public api change on the prepended classes. If you did not override FindTerm #find or SearchQuery #search, then there are no updates that you need to make. If you did override these classes, take a look at the changes to those methods to understand the changes you will need to make to your customization.
Change Log
- updates prepends for FindTerm and SearchQuery to support new request_header parameter in QA 5.0.0 release
v3.0.3
v3.0.2
v3.0.1
Bug fixes only
- make sure count is counting based on the current set of times
- add in missing translations
- fix performance data table data style for retrieve
- unsupported_action? for performance datatable handles all 0 stats
- use old :load data when split :retrieve and :graph_load are not available
v3.0.0
Overview Changes in Release 3.0.0
This release updates authorities and modifies the processing of performance statistics. The performance statistics changes are breaking which drives the major release.
-
If you modified the performance calculations or display, or use that data in another context, see PR #217 to understand the refactor that was done to that code and data structures.
-
Previously, there were two sub-tasks for each request (e.g. :load, :normalization). To get a better insight into performance issues, :load was split into :retrieve and :graph_load. This means that older data for :load is no longer used in performance calculations or graph generation. The data was not removed from the database, but the column holding the :load data was renamed from
load_time_mstoretrieve_plus_graph_load_time_msto avoid confusion.
Actions Required to Upgrade
Update gem
Update to qa_server v3.0.0
bundle update qa_server
Update database
Pull in one new migration to add the performance_history table. Should add migration:
- _update_performance_history_table.qa_server.rb
rake qa_server:install:migrations
Run migrations
rake db:migrate
Add Prepends
A prepend is added to record the time to read data from the source authority and the time to load the graph.
Update /config/application.rb to include the following prepend registrations. Prepends were added for release 2.2.0. If you don't have any prepends yet, see the release notes for 2.2.0. Otherwise, just add the registration for RDF::Graph with your existing prepends.
config.to_prepare do
# There should be existing prepends for FindTerm and SearchQuery. See release 2.2.0.
RDF::Graph.prepend PrependedRdf::RdfGraph
end
Update configurations (optional)
All configuration changes are optional. See details for configuration changes under Description of Changes below.
Description of Changes
Changes to authorities:
- adds pagination start_record parameter for LD4P cached data
- adds RDA Registry authority
- splits LCNAF into 2 authorities for LD4P cached data focused locnames and real world object focused locnames_rwo
- adds wikidata authority - NOTE: The first pass on fetching through this authority is non-performant.
Changes to performance data processing:
A significant refactor was made on the calculation and display of performance data. If you override any of the performance calculations or display, you will want to take a close look at the changes in PR #217.
Performance graphs are created for all combinations of...
- authority: each authority + all authorities
- time period: :day, :month, :year
- action: :all_actions, :search, :fetch (NEW)
This produces 9 graphs per authority plus 9 more that include data from all authorities.
The performance data table splits details into combinations for...
- authority: each authority + all authorities
- 10th percentile (fastest), average, 90th percentile (slowest)
- action: :all_actions, :search, :fetch (NEW)
- task: :retrieve, :graph_load, :normalization, :full_action (labeled TOTAL) (MODIFIED - separated :load into :retrieve and :graph_load)
where...
- :retrieve includes network costs + cache/authority processing for retrieving data from the cache or directly from an authority
- :graph_load includes the time to load the retrieved data into an RDF::Graph
- :normalization includes any time spent by QA to normalize the data including filtering out results, extracting extended context, and writing the normalized json
- :full_action is the length of time spent in the search method or find method. There is a small amount of overhead for Rails routing and controller processing that is not included in this time.
Changes to Configuration Options
All new configurations are optional.
The performance reporting has new configurations to control the output in the Monitor Status page:
config.performance_y_axis_maximum = 4000(ms)config. performance_retrieve_color = '#ABC3C9'config. performance_graph_load_color = '#E8DCD3'
Removed configuration:
config. performance_load_color
It is recommended that you copy the new configurations to keep your local config file in sync with the configurations.
Copy only the new configs...
FROM: lib/generators/qa_server/templates/config/initializers/qa_server.rb
TO: config/initializers/qa_server.rb
Migrations
A migration (e.g. update_performance_history_table) was added to modify the performance_history table.
- changes action column from a string to an integer. This column is an ActiveRecord enum and should never have been set to string type. ActiveRecord was ignoring that type and treating it like an integer at least for some databases. This is a bug fix.
- rename :load_time_ms to :retrieve_plus_graph_load_time_ms (to avoid confusion with new :graph_load_time_ms)
- add columns :retreive_time_ms, :graph_load_time_ms, and :action_time_ms
- index on
Change Log
- refactor performance data and graphs to include stats by action as well as by authority and time period
- adds pagination start_record parameter for LD4P cached data
- adds RDA Registry authority
- splits LCNAF into 2 authorities for LD4P cached data focused locnames and real world object focused locnames_rwo
- adds wikidata authority - NOTE: The first pass on fetching through this authority is non-performant.
v2.2.3
v2.2.2
Actions Required to Upgrade
Update to qa_server v2.2.2
bundle update qa_server
Changes
All the changes relate to the performance data and graphs shown on the Monitor Status page.
Performance data table changes
- show statistics for each authority
- configure how much data is used to generate the table (e.g.
:day,:month,:year,:all) with a default of:year - show 10th and 90th percentile instead of min and max
Performance graph changes
- show graphs for each authority
- all graphs initially display for the configured time period (e.g.
:day,:month,:year) with a default of:month - display graphs as a stacked bar chart instead of a line chart
- allow for configuration of bar colors (defaults are accessible)
New Configuration Options
All new configurations are optional.
The performance reporting has new configurations to control the output in the Monitor Status page:
config.performance_load_color = '#ABC3C9'config.performance_normalization_color = '#CCBE9F'config.performance_graph_default_time_period = :month # values: :day | :month | :yearconfig.performance_datatable_default_time_period = :year # values: :day | :month | :year | :allconfig.performance_datatable_max_threshold = 1500config.performance_datatable_warning_threshold = 1000
It is recommended that you copy the new configurations to keep your local config file in sync with the configurations.
Copy only the new configs...
FROM: lib/generators/qa_server/templates/config/initializers/qa_server.rb
TO: config/initializers/qa_server.rb
Changed Log
- handle stat calculation and formatting when 0 records or missing stats
- switch performance graph to stacked_bar
- Use 10th and 90th percentiles for datatable
- display day graph for every authority for default time period
- cache performance graphs
- don’t calculate low and high for graphs since they aren’t used
- don’t calcualte full request data for graphs since it isn’t used
- include authority data for performance graphs
- move some of the performance data construction code to services