Creation of GSI taking long time

Dixit Gokhale picture Dixit Gokhale · Dec 21, 2017 · Viewed 7.5k times · Source

I have a table with close to 2 billion rows already created in DynamoDB.

Due to a query requirement, I had to create a Global Secondary Index(GSI) in it. The process of GSI creation started 36 hours ago but still isn't completed. Portal shows Item Count to be around 100 million. So long way to go.

Questions:

  1. Why does it take such a long time when sufficient WCU and RCU are alotted( 30k in fact ).
  2. GSI partition key I've used is something whose values are repetitive, could that be the reason why GSI creation is taking more time (ideal scenario is that we select a partition key which doesn't repeat for items to span across multiple partitions).
  3. Is there a way to abort the creation of GSI while the process is on? it doesn't allow through AWS console.

Thanks.

Answer

F_SO_K picture F_SO_K · Dec 21, 2017

A GSI has its own WCUs and RCUs, distinct and separate to the primary index. Could this be because you dont have enough WCUs on your GSI?

If your global secondary index is taking too long to create (common when adding indexes on an existing large table), you can provision additional write capacity by following these steps:

Open the DynamoDB console.

From the navigation pane, choose Tables, and then select your table from the list.

Choose the Indexes tab.

Increase the write capacity of the index, and then choose Save.

After about a minute, check the OnlineIndexPercentageProgress metric from the Metrics tab to see if the creation of your global secondary index is progressing satisfactorily.

EDIT: Above from the AWS Knowledge Center

'OnlineIndexPercentageProgress' instructions:

Creation of your global secondary index will begin. You can monitor the progress on the Metrics tab:

Choose the Metrics tab.

Choose View all CloudWatch metrics.

In the CloudWatch console, choose DynamoDB. In the Search Metrics box, enter OnlineIndexPercentageProgress. Note: If the search returns an empty list, wait about a minute for metrics to populate.

Choose the name of the index to see the progress.