Hi everyone,
I'm trying to load data from a SQL Server 2017 table with a Nonclustered Columstore Index.
I'm primarly using a columnstore index due to the compression benefits. The underyling rowstore table is >10gb in size (making refreshes in the web service impossbile) and greatly benefits from the columnstore compression (~800mb in the columnstore index).
I've previously sucessfully tested this setup with a clustered columnstore index (CCI); Web service refreshes worked and refresh times were unchanged, great! However, our usecases requires about 1 mil row updates per hour (queryied by ID) which results in terrible write performance on a CCI table as I can't create a clustered index on the primary key.
As storage is not an issue for us, I figured a Nonclustered Columnstore Index (NCCI) would be the silver bullet, giving good update performance whilst still benefitting from the compression in PowerBI.
However, when I try to load an NCCI table in PowerBI, the query does not seem to use the columnstore but rather the rowstore (according to SQL Server execution plan as well as service refresh errors due to table size).
I could come up with 2 workaround:
1. Force usage of NCCI via query hints. Suboptimal as I would like to keep our PowerBI team from having to write raw SQL. Also, not sure if this is possible to do on a "connection-level" / in a way that preserves query folding functionalities.
2. Creating seperate tables for the rowstore and CCI and regularly rebuilding the CCI off of the rowstore. This seems rather ugly as the entire CCI would have to be rebuild every time (instead of the delta being merged as is the case with an NCCI). Besides its hackyness this would be a workable solution depending on the cost of a CCI rebuild; Haven't tested that yet.
Does anyone else here have experience in using NCCIs with PowerBI? Is there some setting I'm missing?
Also, the issue applies to both direct query and import mode. Direct Query is however not really an option for us. We'd migrate to SSAS before we go to direct query.
Thanks a lot!