Quantcast
Channel: Power Query topics
Viewing all 31411 articles
Browse latest View live

Connecting PowerBI to PostgreSQL

$
0
0

Hi All,

I have been attempting to connect PowerBI to PostgreSQL, however I am receiving the error seen below.Capture.PNG

 

 

I have followed the steps outlined below.

Install Npgsql which allows .NET access to PostgresSQL.  I installed it using the Pakagae Manager Console in Visual Studio; detailed instructions are here: https://www.nuget.org/packages/Npgsql/

 

You need to connect over SSL to AWS which isn't setup by default (on my Windows 10 machine anway)

  1. Download the AWS public key from here https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem
  2. Convert that to a certificate using https://www.sslshopper.com/ssl-converter.html(convert to PKCS#7/P7B)
  3. Import that certificate to the Trusted Room Certificate described here http://www.cs.virginia.edu/~gsw2c/GridToolsDir/Documentation/ImportTrustedCertificates.htm (easier to search for ‘certificates’ and launch ‘Manage computer certificates’)

 

Even after completing the following changes I am still receiving the error seen above.

Anyone else experienced something similar?

Thanks,


Merge Query

$
0
0

Hello everyone! I state that I am a beginner with Power Query! I have two tables containing two columns each: table 1: -Telefono1 -Mail1 Table 2 -Telefono2 -Mail2 I would like to filter the rows of table 1 that contain at least one value corresponding to table2. I had thought of linking phone & mail together with the tables and using the MERGE function by entering the approximation but I can't get any results, while if I only use the phone I get a filter. do you have suggestions? Thanks in advance!

BI Integration with Open Edge version 10.0.b

$
0
0

Hi,

 

Can anyone tell me if BI works with Open Edge Version 10.0B database?

Performance dips due to Concurrent usage of report

$
0
0

hello,

 

We have been using Power Bi reports in the direct query mode. Database table structures has been adjusted to make sure we minimum joins and calculations are done on the fly. The schema user has also been allocated proper resources on the DB. The execution time ranges on the DB are on an average 1.6 sec. Individual users of the reports see mostly under 20 sec load times. 

The problem arises when we have concurrent users in the system(ranging from 40- 60 users). The load time for the reports are in minutes. 

 

There is a setting on the pbix file to control the concurrency. 

Maximum connections per data source.(https://docs.microsoft.com/en-us/power-bi/desktop-directquery-about#guidance-for-using-directquery-successfully)

The default setting for this is 10. can anyone please explain how this works?

We also use a gateway cluster to load balance the request from the reports to the datasource.

 

Also any help to tackle concurrency on reports is appreciated. 

 

Thanks,

Anu 

Recent changes to Web.Contents behaviour?

$
0
0

Hello,

 

I've been using Power BI to pull data from a REST API data source (LogMeIn Rescue's HTTP GET method). As of this week, it is no longer working as it originally had, leading me to think something has changed with the way Power BI is handling the query.

 

Essentially, when using their API's HTTP GET method, from a web browser I can send GET requests to set up the report options, such as the timezone, delimiter, and date range for the report (so Eastern Standard Time, comma-delimited, and beginning of current year to yesterday). After, I can send the getReport command to retrieve the report with the options I sent previously. 

 

This used to work in Power BI without any issues, but now it doesn't seem to save/remember the session/cookies of the previous GET methods, so when it goes to request the report, none of the options are applied to it. 

 

Here's a snippet of the query. I have verified that each "step" of the query provides an "OK" response from the API, so everything is still sending correctly in that regard:

 

let
    Year = Date.ToText((Date.AddDays(Date.From(DateTime.FixedLocalNow()),-1)),"yyyy"),
    Month = Date.ToText((Date.AddDays(Date.From(DateTime.FixedLocalNow()),-1)),"MM"),
    Day = Date.ToText((Date.AddDays(Date.From(DateTime.FixedLocalNow()),-1)),"dd"),
    URL = "https://secure.logmeinrescue.com/API",
    authcode = "myauthcode",
    TimeZonePath = "setTimezone.aspx?timezone=-240&authcode=" & ""&authcode&"",
    DelimiterPath = "setDelimiter.aspx?delimiter=,&authcode=" & ""&authcode&"",
    DateRangePath = "setReportDate_v2.aspx?bdate=1/1/" & ""&Year&"" & " 00:00:00" & "&edate=" & ""&Month&"" & "/" & ""&Day&"" & "/" & ""&Year&"" & " 23:59:59" & "&authcode=" & ""&authcode&"",
    ReportPath = "getReport.aspx?node=14153094&nodetype=NODE&authcode=" & ""&authcode&"",

    TimeZone = Csv.Document(Web.Contents(URL,[RelativePath = TimeZonePath]),[Delimiter=",", Columns=35]),
    Delimiter = Csv.Document(Web.Contents(URL,[RelativePath = DelimiterPath]),[Delimiter=",", Columns=35]),
    DateRange = Csv.Document(Web.Contents(URL,[RelativePath = DateRangePath]),[Delimiter=",", Columns=35]),
    Report = Csv.Document(Web.Contents(URL,[RelativePath = ReportPath,Timeout=#duration(0,0,5,0)]),[Delimiter=",", Columns=35]),

    Source = Table.Combine({#"TimeZone",#"Delimiter",#"DateRange",#"Report"}),

Has something changed with the behaviour in which Power BI now handles sessions/cookies for Web.Contents? If so, is there an option I can set to have it use the same session across each Web.Contents?

 

Thanks!

 

 

 

Dataset used as a pandas DataFrame in a power query

$
0
0

I have a dataset that I import from salesforce. I wrote a python script that starts with a loop through the ID column and changes it to a list of strings.

 

The code:

FRID_list = dataset['FRID_c'].tolist()
FRID_listn = [str(z) for z in FRID_list]

 

So if I export the dataset to a csv, I can loop through, but when I try to run it through power query it says it doesn't recognize the column name (FRID_c).

 

image.png

Append tables dynamically

$
0
0
Hi I have appended 4 different tables in one new table in query editor. The data source is Excel. But a user made changes in the column of the original data source in excel and now after refresh the appended new table is showing error. Is there a way in which even if changes have been made to column in data source. It should not affect the appended new table. Thanks

Adding rows in Power Query (M)

$
0
0

Hello how are you?. I have found in the forum a solution for part of a problem that has arisen.

https://community.powerbi.com/t5/Desktop/adding-rows-in-power-query-and-performance-issue/m-p/339558#M151973

I was trying to modify that solution but I can not find a way to do it.
It happens that I have a list of records where there are missing records to complete, which have to take the value of the next record (not the previous one) and in the range of the same Material and Center. I attach the example and how it should look.

 

Missing Records.png

 

I have written this code which is very similar to the solution I had found, but I do not know how to do so that it only compare within the range of Material and Center, and that in the lost record goes the Amount of the next record:

 

let
Origen = Excel.Workbook(File.Contents("C:\Users\walter.krajcar\Documents\Libro1ssss.xlsx"), null, true),
ConMes_Sheet = Origen{[Item="ConMes",Kind="Sheet"]}[Data],
#"Encabezados promovidos" = Table.PromoteHeaders(ConMes_Sheet, [PromoteAllScalars=true]),
#"Tipo cambiado" = Table.TransformColumnTypes(#"Encabezados promovidos",{{"Date", type date}, {"Mes", Int64.Type}, {"Amount", Int64.Type}}),
#"Índice agregado" = Table.AddIndexColumn(#"Tipo cambiado", "Índice", 1, 1),
#"Added Custom" = Table.AddColumn(#"Índice agregado", "Custom", each try #"Índice agregado"[Mes]{[Índice]}-[Mes] otherwise 1),
#"Changed Type1" = Table.TransformColumnTypes(#"Added Custom",{{"Custom", Int64.Type}}),
#"Added Custom1" = Table.AddColumn(#"Changed Type1", "Custom.1", each List.Dates(Date.From([Date]),[Custom],#duration(31,0,0,0))),
#"Expanded Custom.1" = Table.ExpandListColumn(#"Added Custom1", "Custom.1")
in
#"Expanded Custom.1"

 

I'm sorry my English!. 

Thank You Very Much!!

 


SharePoint list limitations

$
0
0

I am building a new reporting package for my company using PowerBI. The main data source is a SharePoint Online list to which multiple users will add project data. I expect this list to grow to over 10,000 rows over the next two years. Are there limitations to using a list? Should I consider connecting the list to a database and querying that? New to this - thanks for any help!

dataflows is truncating my 19 digit number

$
0
0

I am getting data from on-prem SQL Server through dataflows and the last 2 digits of the 19 digit number are being changed to zero. Below is a screenshot of the data in SQL and Power Query after it is moved through data flows.Annotation 2019-06-25 174112.jpgAnnotation 2019-06-25 174224.png

 

help with power query

$
0
0

Hello,

 I have a data set for bike rental  with start date and time and end date and time. now, i want to calculate the difference between the start date and and time and end date and time. for example if a bike starts at 06.00 am on 7/2/2016 and ends on 7/3/2016 at 4.30 am the no of hours between them must be calculated which is 22 hours  and per each hours i want to print a new row with all the data included and the updated hour. for example :

#row1 trip id 1 , start time and date 7/2/2016 06:00 am

#row 2 trip id 1, start time and date 7/2/2016 07:00 am

#row 3 trip id 1, start time and date 7/2/2016 08:00 am 

#row 3 trip id 1, start time and date 7/2/2016 09:00 am ..........so ontrip id 1, till 7/3/2016 04:00am 

 

i have tried using the 

  1. temp ={Number.From([Start_date])..Number.From([End_date])}
  2. A list is generated expanded it and changed to date 
  3. if ( [Temp] = [StartDate]) then [StartTime] else [Temp]
  4. if ([Temp] = [EndDate]) then [EndTime] else [Temp] & #time(23,59,59) but, it dint work. i am attaching a screen shot for reference. Thank you for the help in advance. Capture6.PNGsample data with date and time , excluding other multiple columns

Error with < query

$
0
0

Hi,

I'm trying to add a column to a table by which looks at value in another column and adds "Yes" or "No" depending on if it's less that 0.

The line is

= Table.AddColumn(#"Grouped Rows", "Required Rework", each if [Min Vote] < 0 then "Yes" else "No")

 

where the content of [Min vote] is a number and the column type if also a number. Teh Min Vote column is created in a previous Table.Group aggregate using {"Min Vote", each List.Min([Vote Only]), type number}.

 

However I get an error

Expression.Error: We cannot apply operator < to types Number and Text.
Details:
Operator=<
Left=0
Right=10

 

Any idea why this is giving me an error?

Concatenate values from child records while expanding from merged query

$
0
0

Given a 'parent' table:

ROUTING_ID
1
2

 

and a 'child' table:

ROUTING_ID_FromROUTING_ID_To
125
126
127
225
226

 

How can I get the following output column 'NextRoutings', if possible in the 'ExpandTableColumn' step:

ROUTING_IDNextRoutings
125,26,27
225,26

 

This is NOT working:

= Table.ExpandTableColumn(#"Merged Queries", "qryRoutingsFromTo", List.Accumulate(qryRoutingsFromTo[ROUTING_ID_To], "", (state, current) => if state = "" then Number.ToText(current) else state & "," & Number.ToText(current)), {"ROUTING_ID_To"})

 

I also don't want to expand the table in the 'normal' way and then group it on all columns other than qryRoutingsFromTo[ROUTING_ID_To] and then 'calculcate' the NextRouting column. Reason: I think it can be done directly in the ExpandTableColumn step. I also need to group on a lot of columns.

 

Another solution could be to first group the child table (which is only on one column) and after that to merge with it. But still... can it be done in the expand-step?

Power Query - Create multiple COUNT columns in a single Power Query line of code

$
0
0

Hi,

I have successfuly made a COUNT based on Group By columns. See below for the snippet that works.
#"NoO_Removed Other Columns" = Table.SelectColumns(Source,{"Partition", "Invoice Date", "Branch Name"}),
#"NoO_Grouped Rows" = Table.Group(#"NoO_Removed Other Columns", {"Partition", "Invoice Date", "Branch Name"}, {{"Number of Orders", each List.NonNullCount([Partition]), type number}}),

 

What I would like to do now is COUNT a number of individual columns by the same Group By. I can do this separately and then merge the individual COUNTS together, however I would like to achieve the result using a single line of code. I am having trouble achieving this. Below is my snippet of code that is failing:

 

#"NoO_Removed Other Columns" = Table.SelectColumns(Source,{"Partition", "Invoice Date", "Branch Name", "Postcode", "DMS Place"}),
#"NoO_Grouped Rows" = Table.Group(#"NoO_Removed Other Columns", {"Partition", "Invoice Date", "Branch Name"}
                                                                                                                                                                         {{"Number of Orders", each List.NonNullCount([Partition]), type number}},
                                                                                                                                                                         {{"Count of Postcode", each List.NonNullCount([Postcode]), type number}},
                                                                                                                                                                         {{"Count of DMS Place", each List.NonNullCount([DMS Place]), type number}}
                                                                                                                                                                         ),

 

The error I'm getting is:
-------------------------------------------------------------------------------
Expression.Error: We cannot apply indexing to the type List.
Details:
Value=List
Index=List
-------------------------------------------------------------------------------


Can it actually be done?  If yes, then what do I need to change in my failing snippet of code?

Thanks in advance.

Monthly Updated Report Data in Excel

$
0
0

Hi

 

I have an excel file where the data is hosted locally on my computer. My issue is that each month I get a new excel workbook with the "new month's data" but I don't want to go through the cleaning process. 

So for example spreadsheet 1 comes with data which I clean in PowerBI, import, update field types etc. - now I create my report with various visualisations for this month.

 

In month 2 I have a new spreadsheet with the exact same data layout (spreadsheet 2). I want to add this data into my report and keep last month's data (to allow comparitive analysis).

 

What is the most effecient way to import this into PowerBI (and avoid redoing the cleaning, etc) and update my report with the fields in the new report without having to manually edit each visualisation and drag the relevant field to each visualisation?


AggregateTableColumn with List.Count returns 1 instead of 0

$
0
0

With an aggregation I get an unexpected result. The 'parent' table is as follows:

ROUTING_ID
1
2
3

 

The 'child' table is as follows:

ROUTING_ID_FromROUTING_ID_To
123
124
125
223
224

 

Now I do a AggregateTableColumn with a List.Count, and I get as result:

ROUTING_IDNrOf_ROUTING_ID_To
13
22
31

 

Why has the last row (ROUTING_ID = 3) as number of Routings-To 1 instead of 0?

 

This is the line of code:

#"Aggregated qryRoutingTransitions_Direct2" = Table.AggregateTableColumn(#"Merged Queries3", "qryRoutingTransitions_Direct2", {{"ROUTING_ID_To", List.Count, "NrOfNextRoutings"}}),

Should I include that ROUTING_ID_To must not be null in order to not get counted as 1? Would be strange... If so: how to do that?

Help using Buffer command

$
0
0

Hi everyone,

 

I have three queries which stem from two raw data worksheets. The total number of rows from the raw data worksheets is about 15,000 and the three queries produce is about 10,000 rows of total data. It's currently taking about 30 minutes for each query to refresh (I don't run the queries in parallel) which isn't an exorbitantly long time, but I think it could be faster based on my research of using either the List.Buffer or the Table.Buffer command.

 

I'm fairly new to power query so I'm hoping some one with more experience with the Buffer command can recommend where to place it within the my three queries. I've attached pictures of the queries to this thread. Additionally, if you have any recommendations of other changes I should make to my queries, I'm open to that information!

 

Thanks in advance!

Query for Curriculum Data.pngQuery for Documents for Deletion.pngQuery for Effective Documents.png

How to update data in PowerBI after adding columns in the original Excel file

$
0
0

Hello Power Bi Community,

 

I'm new to Power Bi, but was able to develop a report in PowerBi Desktop using data from Excel File saved in OneDrive. However, after completing the report, I realized that I needed to add two extra columns in my Excel file. I'm now trying to update my data in PowerBI after adding the columns in the original Excel file, but I can't see the columns. I tried changing the data soource from data query, but nothing.    Any tips on how to do that?

 

Thanks!

The best way to get Power BI Pro and Delve Analytics

$
0
0

Just joined a new firm as an manager, can someone help me with the best way to acquire some analaytic tools like delve or power bi

Power BI Integration with Tenable

$
0
0

Hello All – I am looking for someone who has been able to export data out of Tenable and import it into PowerBI.

 

  • What is Tenable -  A cloud based company that helps organizations to manage & measure their cyber exposure
  • What data to export -  Vulnerabilties and Assests


Tenable documentation has shown some ways to export their data - One way is using post REST API and the other is using a Python Script. I have tried using their python script in powerBI and it errors out with a TimeOut error. The script takes more than 40 minutes to run.

Has any one worked out an integration between Tenable & PowerBI before, if so, please let me know.

We really do not want to reinvent the wheels here.. We need some help here..

 

Thanks,

Harsha Beeram

Viewing all 31411 articles
Browse latest View live


Latest Images