Quantcast
Channel: Power Query topics
Viewing all articles
Browse latest Browse all 30988

Getting error for huge records - Evaluation resulted in a stack overflow and cannot continue

$
0
0

I have a table with lakhs of records. 'Time' is one of the fields. I have created a function fetchlatest to get the latest record based on 'time'. As you can see in the below code, the function basically groups the table based on unique key (GroupByIds) and then create column 'MaxDateTime'. Then filter out the records which has time = MaxDateTime.

 

I validated the function with a table having thousands of records and it worked fine. But when I did scale testing with lakhs of records, I am getting the error 'We encountered an error during evaluation. Errors- Evaluation resulted in a stack overflow and cannot continue'. When I manually tested each step, it happens at the step Table.Group.

 

Note - I used the below power query in Dataflows. I am assuming this error is thrown by power query and not by dataflows.

 

Could you please suggest me why this error is occuring with large records? How can I rectify this error or can you give me any other workaround to get latest using M code ?

 

 

 

let
  FetchLatest = (Table , AllProperties, GroupByIds) =>
let    
    Properties = List.Difference(AllProperties, GroupByIds),
    GroupRowsById = Table.Group( Table, GroupByIds, {{"MaxDateTime", each List.Max([time]), type datetime}, {"All rows", each _, type table}}),
    ExpandGroupedRows = Table.ExpandTableColumn(GroupRowsById , "All rows", Properties, Properties),
    FilterMaxTimeRow = Table.SelectRows(ExpandGroupedRows, each [time] = [MaxDateTime]),
    RemoveExtraColumns = Table.RemoveColumns(FilterMaxTimeRow ,{"MaxDateTime"}),

    Result = 
    if List.IsEmpty(GroupByIds) = true
    then Table
    else RemoveExtraColumns 
in 
    Result
in
  FetchLatest

 


Viewing all articles
Browse latest Browse all 30988

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>