Expressions & Querying by GSI PK

[mcccoleman]

Hello!

I'm in the process of trying to query a particular table through using its global secondary index. I'd like to be able to query by using an expression attribute name & value, but I'm unable to determine how to configure the syntax to do so appropriately. Regardless of what I try to pass in to queryAllWithGsiByPk I get the following validation: ValidationException: ExpressionAttributeValues must not be empty.

I wasn't able to find anything in the documentation on how to do this (please forgive me if I just missed something) - do you happen to know of a way to accomplish what I'm trying to do?

[mikebroberts]

Hello! So that's an error from DynamoDB, and I think it's probably because something's going weird between your table configuration, and what's in your Entity. Are you able to include both of those things here?

[mcccoleman]

Hi @mikebroberts - Thank you so much for your reply.

Here's an example of what I'm trying to run that's resulting in the above error:

[tableClient].operations.queryAllWithGsiByPk(
        {
            d: 'EXAMPLE VALUE',
            ExpressionAttributeValues: {
                ':domain': '#d',
            },
        },
        { gsiId: 'domainIndex' },
    )

I've tried a number of different iterations of this, including explicitly including an ExpressionAttributeNames property.

One additional note - I'm currently not running this against a production db but am instead trying to get it to work in a test environment first. I'm using @shelf/jest-dynamodb to set up my test tables.

[mikebroberts]

Can you share the code that implements the Entity interface? Also - for the first parameter you should just need to provide whatever the pk function in your entity class would take. (The Entity is what you would have passed to entityStore.for(...) , the result of which you call queryAllWithGsiByPk on)

For the second argument - do you have your gsi configured in your table configuration - i.e. what you call createStore with? Can you share that table configuration code?

[mcccoleman]

Hi @mikebroberts thank you so much for your reply & absolutely - please find both included here:

Entity interface

const domainPrimaryKey = ([EntityRecord]: Pick<[EntityRecord], 'domain'>) => [entityRecord].domain

const entity: Entity<[EntityRecord], Pick<[EntityRecord], [primaryKey]>, Pick<[EntityRecord], 'id'>> = {
    type: [EntityRecord],
    parse: parseStatement,
    pk: [primaryKey],
    sk: [idSortKey],
    gsis: {
        ...,
        domainIndex: {
            pk: domainPrimaryKey,
        },
    },
}
export const operations = store.for(entity)

Table configuration

const tableConfig: TableConfig = {
    tableName: [tableName]
    allowScans: false,
    metaAttributeNames,
    gsiNames: {
      ...,
        domainIndex: 'domain-index',
    },
}
const store = createStore(tableConfig, storeContext)

One question on the parameters that I need to pass, generally when I'm trying to run a query on the secondary key I can just pass that secondary key (ie in cases when the secondary index primary key is not a reserved word, unlike the situation above with domain):

[entityClient].operations.queryAllWithGsiByPk({ [gsiPrimaryKey]:[value] }, { gsiId: [indexName] })

Is that what you're saying?

[mikebroberts]

const tableConfig: TableConfig = {
tableName: [tableName]
allowScans: false,
metaAttributeNames,
gsiNames: {
...,
domainIndex: 'domain-index',
},

I think this might be at least part of the problem. The GSI keys need to be on the metaAttributeNames field, not the gsiNames field (I'm sorry the config is complicated, I didn't come up with something cleaner) . E.g. look at the example in documentation here: https://github.com/symphoniacloud/dynamodb-entity-store/blob/main/documentation/GSIs.md?plain=1#L67-L85

[mcccoleman]

const tableConfig: TableConfig = {
tableName: [tableName]
allowScans: false,
metaAttributeNames,
gsiNames: {
...,
domainIndex: 'domain-index',
},

I think this might be at least part of the problem. The GSI keys need to be on the metaAttributeNames field, not the gsiNames field (I'm sorry the config is complicated, I didn't come up with something cleaner) . E.g. look at the example in documentation here: https://github.com/symphoniacloud/dynamodb-entity-store/blob/main/documentation/GSIs.md?plain=1#L67-L85

I could have been clearer in the code I posted above (apologies), but I've defined my metaAttributeNames above the table config & then passed it directly in. Here's what i'm setting as metaAttributeNames

const metaAttributeNames: MetaAttributeNames = {
    pk: [primaryKey],
    gsisById: {
        ...otherIndexes,
        domainIndex: {
            pk: 'domain',
        },
    },
}

One question. you mention the GSI keys shouldn't be on the gsiNames, but from the example it doesn't seem like they're removed. Do you mean they should be on the metaAttributeNames field in addition to gsiNames?

Really thank you so much for going over this all with me, I really do appreciate it.

[mikebroberts]

Hi again,

The physical GSI names should be on the gsiNames property of the table config.

The physical GSI key fields should be on metaAttributeNames.gsisById.GSI_LOGICAL_NAME.

So for this example for table config in the docs:

{
  tableName: 'AnimalsTable',
  gsiNames: {
    gsi: 'GSI'
  },
  metaAttributeNames: {
    gsisById: {
      gsi: {
        pk: 'GSIPK',
        sk: 'GSISK'
      }
    }
    ... etc.
  }
}

• There is a "logical" GSI named gsi
• The "physical" name of this GSI (in the DynamoDB table is AWS) is GSI
• The "physical" names of the underlying key fields (in the DynamoDB table in AWS) are GSIPK and GSISK
• ... but the Entity code never knows the physical field names - it just uses the logical GSI name (gsi).