Skip to content
This repository has been archived by the owner on Apr 22, 2020. It is now read-only.

Get rid of tag value and tag name caches #36

Open
elgris opened this issue Jan 9, 2018 · 0 comments
Open

Get rid of tag value and tag name caches #36

elgris opened this issue Jan 9, 2018 · 0 comments
Labels
Chore zmon-arch Relevant issues. To be tackled during architecture discussions

Comments

@elgris
Copy link

elgris commented Jan 9, 2018

Just as memo that might either help to fix Kairos or to remove unused code.

User story

As a developer of KairosDB fork
I want to remove all functionality related to storing tag names and tag values in Cassandra table string_index
So that there is no dangling and unused functionality and no more unnecessary requests to Cassandra in putDataPoint method

Explanation

Current fork of KairosDB supports storing tag names and tag values in Cassandra table string_index . It keeps them in keys tag_names and tag_values. However, those values are never read. Because actual tags and values are fetched by selecting latest 9999 data row keys for a specific metric. Start looking from org.kairosdb.core.http.rest.MetricsResource#getMeta which serves "/datapoints/query/tags" endpoint. Eventually you'll reach org.kairosdb.datastore.cassandra.CassandraDatastore#queryMetricTags:

    public TagSet queryMetricTags(DatastoreMetricQuery query) {
        TagSetImpl tagSet = new TagSetImpl();
        Collection<DataPointsRowKey> rowKeys = getKeysForQueryIterator(query, 9999);

        MemoryMonitor mm = new MemoryMonitor(20);
        for (DataPointsRowKey key : rowKeys) {
            for (Map.Entry<String, String> tag : key.getTags().entrySet()) {
                tagSet.addTag(tag.getKey(), tag.getValue());
                mm.checkMemoryAndThrowException();
            }
        }

        return (tagSet);
    }

It works so, because every data row key contains associated tags.
Thus, tag_names and tag_values keys in string_index table and all related functionality is, in fact, not needed and we can safely get rid of it.
However, metric_names key in string_index table is still in use. Thus, just dropping whole string_index table is not an option.

This change is specific for ZMON and drags the fork further from original KairosDB.

@rajatparida86 rajatparida86 added the zmon-arch Relevant issues. To be tackled during architecture discussions label Jul 25, 2018
@pitr pitr added the Chore label May 16, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Chore zmon-arch Relevant issues. To be tackled during architecture discussions
Projects
None yet
Development

No branches or pull requests

3 participants