Quantcast
Channel: Power Query topics
Viewing all 31288 articles
Browse latest View live

SharePoint Folder Not connecting all Files into Power Query

$
0
0

Hello, 

 

I am trying to combine (3) files in a folder on our Sharepoint site into a Query. When I combine and transform the data, the only file that comes through is the first data file. Each file corresponds to a monthly report pulled from our GPS system database. I have identified that all (3) montly files are identical (rows, columns, headers, etc.). 

 

I have created reports before when combining folders from SharePoint and have never had this issue before. When filtering the data it does not even give me the option of the other (2) files. They are just non-existant. 

 

Please help!

 

How to call SOAP(not REST) sharepoint Lists Service method GetVersionCollection

$
0
0

I have spent quite a few days reading posts on this and I still can't figure it out:

Requirement

  • I need to get all prior versions of a given list item into power bi to report on them. ( I already have all other data)

Background

  • It seems sharepoint has 2 web APIs for listdata, REST and SOAP(older) 
  • the SOAP SP service has a method called GetVersionCollection that takes a listID and itemID that returns the exact data I need
  • In my current model, I have a table that has all of these parameters (listIDs, ItemIDs)
  • As far as I know, the way to do it via REST is to use this page: Web.Page(Web.Contents(https://foo.com/sites/site/_layouts/versions.aspx?list=","xxxxx-xx-xx-xx-xxxxxxx", "&ID=17"}) ) and also web.Contents But the problem with this approach is that the page is a report and has all the data rows split (2 field in 1 row, 2 fields in the next. Like this (below are row id 6 and 5 - the row below has the value)

image.png

 

So, My Question is:  How can I EITHER

Call the SOAP version GetVersionCollection in PBI (not PBI, but something like:

 

$().SPServices({  operation:"GetVersionCollection",async:false,  strlistID:"Projects",  strlistItemID:1,  strFieldName:"Description",  completefunc:function(xData,Status){    $(xData.responseText).find("Version").each(function(i){      console.log("Name: "+ $(this).attr("Description")+" Modified: "+ $(this).attr("Modified"));});}});

https://docs.microsoft.com/en-us/previous-versions/office/developer/sharepoint-2010/bb263650(v%3Doffice.14)

 

OR

Get the REST resultSet split/parsed so it's in one table.

 

Like I said, I have been doing trial and error for quite a while.  Any tips would help.

Expression:Error: Calculated Column trying to convert to fraction instead of decimal

$
0
0

So i am trying to create a new calculated colum to calculate the true sales price of our products using a calculation, it seems simple enough except i am getting an Expression:Error and i cannot for the life of me work out why? The same calculation is working for half the date and there is no difference other then the numbers being used.

The error im getting is as follows: 

 

Expression:Error: We cannot apply operator / to types Text and Numbers.

Details

 Operator=/

 Left= 27.48

 Right=3

 

In my eyes this should be 9.16? Does anyone have any idea why this would be throwing an error? My calculations for the column are below: We have 4 wholesalers whos true sale price is calculated slightly differently. Wholesaler 4 is where im getting the error, but wholesaler 3 & 4 are excactly the same other then the wholesaler names?

 

net_sales_per_unit = 

if[Wholesaler Name] = "wholesaler1" then ([api_cost_price] -[claim_amount_per_unit])
else if[Wholesaler Name] = "wholesaler2" then ([net_sales_before_claim] / [quantity])
else if[Wholesaler Name] = "wholesaler3" then ([5iproducts.sell_price_1] - ([claim_amount] / [quantity]))
else if[Wholesaler Name] = "wholesaler4" then ([5iproducts.sell_price_1] - ([claim_amount] / [quantity]))
else 0

New Column with latest value

$
0
0

Hi, I have a table with 4 columns: Date, Item-Number, Item-Name, Sales. The Item-Name (of the same Item-Number) changes over time.
-> I want to add a column with Power Query M which adds the latest Item-Name for each row, as below. Thanks your help! Mike

 

sample-data.JPG

Looking to load value from a file. The file contains a base URL to be used as a variable

$
0
0

Currently trying to write a custom connector which can be migrated to different environements (ei test, stage, int, qa, prod).

We want to have a supplementary file, for each environment, which will contain a base URL (https://int-aws.com, https://qa-aws.com, https://prod-aws.com). We will then provide the key to be used and connect to the API URL.

 

We're able to hard code the URL (first comment below) and pass that through just fine. However it seems to have issues with the File.Contents. We've tried forward slashes, back slashes, etc. Below is the current iteration for declaring ENV. Error below, after code snippet. The error seems to hint at the key creds provided as not being able to authenticate to the file; to open the file. I wouldn't think you'd need creds to open or access the file. I would have figured File.Contents would be able to read the value in the file and use it.

//Works:

//ENV = "https://int-aws.com/v1/example-API/exampleAPI=1460";

 

//below code not working:

 

ENV = Text.FromBinary(File.Contents("file://C/ENV.txt"));

[DataSource.Kind="exampleAPIConnector", Publish="exampleAPIConnector.Publish"]
shared exampleAPIConnector.Contents = (GetEnvID as text) =>
let
apiKey = Extension.CurrentCredential()[Key],
headers = [
#"ApiKey" = apiKey,
Accept = "application/vnd.api+json",
#"Content-Type" = "application/json"
],
exampleAPI = Web.Contents(ENV, [ Headers = headers, ManualCredentials = true ]),
exampleAPIRecords = Json.Document(exampleAPI),
exampleAPITable = Table.FromRecords(exampleAPIRecords),
exampleAPILevelsExpanded = Table.ExpandRecordColumn(costCenterTable, "levels", {"l0", "l1", "l2", "l3", "l4", "l5", "l6", "l7"})
in
exampleAPILevelsExpanded;

 

 

error:

"Credentials are required to connect to the exampleAPIConnector source. (Source: from \\c\env.txt to {"GetEnvID":"ENV.txt"}.)"

Importing content from HTM files from Sharepoint folder

$
0
0

Hi Guys,

 

I'm trying to set up a data load, in Power BI desktop (and later in the service, as a dataflow), which merges daily extracts, that are stored in a Sharepoint folder, in HTM format.

Each file has a table of the same structure embedded, and Power BI is able to recognize it as Table0. I can load it as data table and transform it, and using Local folder option (from my laptop), I can combine the Table0 from every file no problem.

However, when I try the exact same process, but with a sharepoint folder as a source, Power BI cannot read the content of the HTM files, instead, I get a Navigation to the webpage was canceled message:

sharepoint error.PNG

and at the same time, I get a popup window, this $value file being offered for download:

values.PNG

The $value file actually contans the htm file content, which PBI would need to analyze.

 

Can you help me, what to set or check, to make it actually read the htm content, instead of downloading it?

 

ps.: I have read permission on Sharepoint for the base URL and the files themselves too.

Group data by consecutive dates per client

$
0
0

Hello,

 

This is the general structure of my data, where each row represents a single transaction currently:

 

ClientIDTransaction_DateAmount
aa01/01/2010
aa01/02/2015
aa01/05/2013
bb02/06/2011
bb02/02/204

 

What I would like to do is group transactions per client by consecutive dates, so client "aa" would show 2 transactions -- one for 25

and one for 13 ,because the first two transactions happen on consecutive dates, so they are counted as a single transaction, and client "bb" would show 2 transactions for 11 and 4. 

 

I don't need to preserve the date data, what I'm trying to show is average transaction amounts by client where transactions with consecutive dates count as a single transaction.

 

 

Any help would be really appreciated, thanks!

Security Concern: PowerQuery w/Credentials in M Code

$
0
0

Hi all,

 

We are connecting to our data by way of a Rest API and Power Query (thank you !)..  One area of minor concern is that our company users can download Power BI Projects (PBIX files) from PowerBI.com once they are published. After downloaded the employee can open the Power Query, select Advance Editor, and have a clear view of the connection details.  It is only a low level concern due to the number of users with access, but I don't see anyway of adding even basic password protecting akin to Excel or VBA.  Correct?

 

In an effort to resolve, it looks like I may be able to shield it using a Data Flow  My thought is build a Power Query (with API details) within the Data Flow, not the project.  The Power BI project (which they can still download) would connect directly to the Data Flow, but would not be able to view the API query code.

 

Is that the correct understanding?  As long as they aren't admin, they couldn't download or access the API details behind the data flow?  I understand the trade-off that the Data Flow needs to be schedule refreshes (its cached data), but we don't need real-time data and that may actually be faster.  My larger concern is whether they can see the query code written in the data flow..


Last 6 month data with filter

$
0
0

Hi,

i want last 6 months data but filter each month data like below DAX

CreatedTicket(6Months) =
CALCULATE (
COUNT('TicketsDetailMonthly'[TicketId]),FILTER('TicketsDetailMonthly', MONTH('TicketsDetailMonthly'[CreatedDate])= MONTH(MAX('Date'[Date])) && YEAR('TicketsDetailMonthly'[CreatedDate])= YEAR(MAX('Date'[Date]))),
DATESINPERIOD ('Date'[Date], MAX('Date'[Date]), -6, MONTH ))
 
but this one is filtering only selected month and year data rather than filtering each month month in last 6 months.
Please help.
Thanks

Calling PowerBI Rest API in Power Query

$
0
0

Hi Folks,

I'm trying to pull data from the Power BI REST API into Power Query and need some guidance.   I've seen numerous threads/blogs describing how to go about this, but none of them have a complete solution, only partial. 

 

1. To get data into Power Query from the Power BI REST API, should "Web" be used as the data source?  (Some posts have said to use "blank query") 

 

2.  Whether it's Web or BlankQuery, how to pass Authentication?  Signing into Power BI as usual doesn't seem to do the trick.

 

3. I've seen lots of posts about APIs, other than Power BI, talking about "tokens"... is a token necessary to get data from the PowerBI REST API?

 

4. Do Headers need to be anywhere when calling the REST API?

 

Thanks much for all feedback on this!

Table matrix grouping and transform

$
0
0

I have an permissions matrix that looks like this:

 

IDitemgroup1group2group3
1string_bYNN
2string_xNYY
3string_eY Y
4string_bNYY
5string_xNYN
6string_bYYY
7string_eYNY

 

Groups associate to items. However, the items are not always uniquely-named, and I need to know when there's a permissions mismatch where the item is enabled for one row, but disabled on another. 

 

Adjacent to the table above, I have another column that looks like this:

 

IDDisabled elsewhere
1group1
2group3
3 
4group2, group3
5 
6group1, group2, group3
7 

 

It's driven by about 50 helper columns, which, because of the dynamic nature of the groups, need to use elaborate formulas with lots of indirects, which make the sheet unusable. I need to convert it to a mechanism I can run on command. DAX is definitely preferable to VBA here. 

 

Can anyone suggest how to transform the table to create the result above? I've been hacking away at it and not making much progress. Much appreciated as always, a lot of the functionality of this matrix derives from advice from this forum.

 

A few notes:

 

1500 rows
Up to 50 groups
Unique IDs, sequential
Unique group names, dynamic number of groups
Item strings are not unique
Name/group associations can be "Y", "N", or null

Return a unique list of all strings enclosed within {} from a column

$
0
0

This is a very complex problem with many moving parts. 

I have a table with column named Value like this:

 

Value Hello! This is great. This is {adjective}. My {person} is {adjective} now.

 


The final product I need is a unique table OR list of all strings found enclosed within {} from column Value:

 

Variable adjective person{"adjective", "person"}

 

 

So far, I know that we can find the position of { and } like this:

 

#"Count{" = Table.AddColumn(LastStep, "Count Of {", each List.Count(Text.PositionOf([Value],"{",Occurrence.All ))), #"Count}" = Table.AddColumn(#"Count{", "Count Of }", each List.Count(Text.PositionOf([Value],"}",Occurrence.All ))),

 

 

We can also return the matched string with Text.PositionOf([Value], "{", n) where n is the number of occurence.

 

Now there are 2 ways to proceed:
METHOD 1:
1. Output the matched strings to columns dynamically resized based on the max number of matches in the entire table.
2. Append those columns, remove blanks and duplicates
With this method I can do 2. I need your help with step 1 only.
METHOD 2:
1. Output the matched strings to a list in a single column.

2. Combine the lists from all rows into a big list. Remove duplicates.
I have no idea how to proceed with this method.

Can any experts help me?
     

Determining if a parent in a hierarchy is disabled, for multiple test cases

$
0
0

I have a security matrix in this format:

 

IDL1L2L3L4L5L6L7group1group2
1Menu1      YN
2Menu1Sub1     NY
3Menu1Sub2     Y 
4Menu2      YY
5Menu2Sub1     NN
6Menu2Sub1SubSub1    YY
7Menu2Sub2      Y
8Menu2Sub2SubSub1    YN
9Menu2Sub2SubSub1SubSubSub1   YY

 

L1 to L7 are string values that correspond to clickpaths. If a parent is disabled, the children are implictly disabled.

 

Groups associate with clickpaths. While column names from ID to L7 will be consistent, the group names will vary, and there many be up to 50.  

 

I'm trying to create a column that aggregates groups that, for each row, have an enabled child with a disabled parent. The user implication is you're trying to enable a clickpath, but you need to go enable the parents in the hiearchy for the child to be visible.

 

IDParent Disabled
1 
2group2
3 
4 
5 
6group1, group2
7 
8 
9group2

 

I have a calculated column to do this, but it has 50 helper columns that require a lot of INDIRECTS because the groups are dynamic, and it's making the sheet unusuably slow. Can it be done in DAX? 

 

Some notes:

 

1500 rows
Up to 50 groups
Unique IDs, sequential
Unique group names, dynamic number of groups
Name/group associations can be "Y", "N", or null

 

This is a follow-on to the 'Table matrix grouping and transform' post here (thanks ImkeF!), and hopefully the last formula I'll need help converting:

 

https://community.powerbi.com/t5/Power-Query/Table-matrix-grouping-and-transform/m-p/1048155

Using multiple parameters invoking custom column

$
0
0

Hey there!

So I've got following issue:

I'm fetchting tables from a website. So far so good, but these are categorized into different categories which can be parameterized in the URL. Also those tables have multiple pages, which also can be parameterized in the URL.

I created a table for the categories to use a function to invoke all tables on the categories on a flat table basis.

I also created a list for the pages {1..100} so that I can also use a function to "iterate" over those.

I'm able to invoke the tables to categories, also the tables to the page numbers.

But where I fail is to get both steps in one query. I haven't been able to succeed over two steps - doing one thing and then the other - because I can't expand a function. 

I feel like drawing a blank. Someone please have mercy and help me here 🙂

M experts - Search a list of values in each cell of a column and return a particular value.

$
0
0

M experts please help here!  ,  

I have a column of Paper Materials(below) that contains Material Weight.

for e.g. "kreda mat250gsm" is having "250gsm" as Material Weight. The order/sequence of these 2 things in a cell is not fixed.

Job IdMaterials
12.95 and 2.64 mm M.S wire
23mm PVC
8135 grs hv silk mc
10paper 120 gsm
12PS 2mm
14kreda mat250gsm
14Material: 70gsm FSC woodfree paper
16115gsm Silk
17matte coated paper 250GSM
18mat kuse135gr
2380 gsm Ozone fsc mixed credit
24Cartulina gráfica 400 gr
25Vinil branco blackout 0.10 mm
29SINTRA 3mm
310.6mm Gloss Vinyl Coated Magnetic
34200 gr/qm Bilderdruck matt

 

Then I created a LIST of values(below) that need to be searched in that Material column and based on this I could extract the Material Weight. I created this LIST so that it could be expanded based on more matching values.

Search
mil
mm
gsm
gr

 

Now with these 2 things (1 table and 1 list), I want to create a function that takes this LIST as an input and search the values of this LIST in each cell of the Material column and return the exact material weight like below.

Job IdMaterialsResult
12.95 and 2.64 mm M.S wire2.64 mm
23mm PVC3 mm
8135 grs hv silk mc135 gr
10paper 120 gsm120 gsm
12PS 2mm2 mm
14kreda mat250gsm250 gsm
14Material: 70gsm FSC woodfree paper70 gsm
16115gsm Silk115gsm
17matte coated paper 250GSM250 gsm
18mat kuse135gr135 gr
2380 gsm Ozone fsc mixed credit80 gsm
24Cartulina gráfica 400 gr400 gr
25Vinil branco blackout 0.10 mm0.10 mm
29SINTRA 3mm3 mm
310.6mm Gloss Vinyl Coated Magnetic0.6 mm
34200 gr/qm Bilderdruck matt200 gr

 


Convert excel formula into Power bi

$
0
0

Hi all, 

I'm trying to import an excel calculation which includes multiple filters into Power Bi, but I'm having big trouble. The formula is to calculate an end date based on user input date and two filters selections, and also to avoid holidays and weekend if the end result falls in either of these categories.

Date: user input (range between 1/1/2018 - 12/31/2021)

Period end: Q1, Q2, Q3, Annual

Issuer status: Large Accelerated Filer (LAF), Accelerated Filer (AF), Non Accelerated Filer (NAF), SBC/EGC, Initial filer, Loss corp/Deliquent filer (LAF/AF)


I was able to create a simple calculation in excel - "Period end" and "Issuer status" are using drop-down selection list, and timeline is formula driven based on the selection of the above two fields. 

 

Timeline =IFS(AND(OR(B3="Q1",B3="Q2",B3="Annual"),OR(B4="Large Accelerated Filer (LAF)",B4="Accelerated Filer (AF)")),"129",AND(OR(B3="Q1",B3="Q2",B3="Annual"),OR(B4="Non Accelerated Filer (NAF)",B4="SBC/EGC",B4="Initial filer",B4="Loss corp/Deliquent filer (LAF/AF)")),"134",AND(B3="Q3",B4="Large Accelerated Filer (LAF)"),"60",AND(B3="Q3",B4="Accelerated Filer (AF)"),"75",AND(B3="Q3",OR(B4="Initial Filer",B4="Loss Corp/Deliquent filer (LAF/AF)")),"45",AND(B3="Q3",OR(B4="Non Accelerated Filer (NAF)",B4="SBC/EGC")),"90")

 

Date 4/1/19
Period endQ2
Issuer statusSBC/EGC
Timeline134
Last day to file10/9/2019

 

In Power BI, is there a way to create a dashboard with field "last day to file" when having user input of "Date", "Period end" and "Issuer status"? 

 

Thanks very much!

 

Preview.Error: The type of the current preview value is too complex to display

$
0
0

I have seen older threads related to this however i have not found anything that is definitive as to the cause.  I get this on one table from a SQL Server database. Tried Direct Query & Import.   I have other tables far more complex than this one that import with no issues. 

 

If i open the native query window and enter "select * from dbo.coitem_mst" i get the data with no issue. 

 

I am using version 2.79.  I am hoping someone has some addtional clarity on what causes this error.

 

Screen Shot 2020-04-25 at 15.32.43 PM.png

How to Combine rows of web table by order (every 2 rows)

$
0
0

Hello All,

 

I am using the following code in power BI to get a table of prior version values for a given list item.

 

let Source = Web.Page(Web.Contents("<site>/_layouts/15/versions.aspx?list=xxxx-61cc-405d-xx-c411eaa43b90&ID=17")), Data0 = Source{0}[Data], #"Changed Type" = Table.TransformColumnTypes(Data0,{{"No.", Int64.Type}, {"Modified", type any}, {"Modified By", type any}}), #"Removed Bottom Rows" = Table.RemoveLastN(#"Changed Type",1) in #"Removed Bottom Rows"

 

 

However, The data on the page is split on to 2 lines/record.  I am having trouble deciding the logical list of steps to join them

together.  The table "removed Bottom Rows" looks like:

CurrentTable.PNG

 

Each desired record has it's value ( metric number value) one row below and with the header "metric number value" in the row below.

 

I am trying to get to the set of steps to just get data as [no] , [modified], [modified by] , [value]

 

Also,  many can probably already tell that the data comes from Sharepoint versions.aspx page.  I am open to other ways, butI have another ticket regarding that question and since I am on prem Sharepoint and power BI server some options were off the table.

 

ThanksFor any help you can provide

Left Join Between dates

$
0
0

Hi

 

I've recently started dabling with powerbi. I have a scenario where I want to left join two tables with an aditional condition checking if the date values from table A is between a Start and End Date in Table B. 

 

If the data is not present in Table B then Table A's rows still needs to be returned. In SQL this is a pretty straight forward query, but for the life of me i'm strugling to figure this out in Powerbi.

 

What would the syntax be in either DAX or M Query to achieve the same result set as posted below ?

 

Bellow I have listed the 2 tables , sample query written in SQL and the expected results. I have also attached the scripts to generate the sample set.

 

Thanks in advance 🙂

 

Table : ta

 

ColumnData Type
RDdatetime
Scheduleint
Datasetint

 

Table : tb

 

ColumnData Type
SDdatetime
EDdatetime
Datasetint
RowsCopiedint

 

SQL : Query :

 

select

a.RD as a_eventDate ,

a.Schedule as a_Schedule,

a.Dataset as a_Dataset,

b.SD as b_SD ,

b.ED as b_ED,

b.Dataset as b_Dataset,

b.RowsCopied as b_RowsCopied

from ta as a

left join tb as b on a.Dataset = b.Dataset

and a.RD >= b.SD and a.RD <= b.ED

 

SQL Results :

 

a_eventDatea_Schedulea_Datasetb_SDb_EDb_Datasetb_RowsCopied
2020-01-01 08:00:00112020-01-01 07:45:002020-01-01 08:45:001300
2020-01-01 08:00:00122020-01-01 07:47:002020-01-01 08:25:002100
2020-01-01 08:00:0013NULLNULLNULLNULL
2020-01-01 08:00:00142020-01-01 07:45:002020-01-01 08:35:00410
2020-01-01 08:00:0015NULLNULLNULLNULL
2020-01-01 14:00:0026NULLNULLNULLNULL
2020-01-01 14:00:00222020-01-01 14:00:002020-01-01 14:15:00225
2020-01-02 08:00:00112020-01-02 07:45:002020-01-02 08:45:00130
2020-01-02 08:00:00122020-01-02 07:45:002020-01-02 08:05:00240
2020-01-02 08:00:00132020-01-02 07:45:002020-01-02 08:15:00320
2020-01-02 08:00:0014NULLNULLNULLNULL
2020-01-02 08:00:0015NULLNULLNULLNULL
2020-01-02 14:00:00262020-01-02 14:00:002020-01-02 14:15:00620
2020-01-02 14:00:0022NULLNULLNULLNULL

 

 

Creating the sample data set scripts:

 

create table ta (RD datetime , Schedule int , Dataset int)
create table tb (SD datetime , ED datetime, Dataset int , RowsCopied int)

 

insert into ta values('2020-01-01 8:00' ,1,1 )
insert into ta values('2020-01-01 8:00' ,1,2 )
insert into ta values('2020-01-01 8:00' ,1,3 )
insert into ta values('2020-01-01 8:00' ,1,4 )
insert into ta values('2020-01-01 8:00' ,1,5 )
insert into ta values('2020-01-01 14:00' ,2 ,6 )
insert into ta values('2020-01-01 14:00' ,2 ,2 )
insert into ta values('2020-01-02 8:00' ,1,1 )
insert into ta values('2020-01-02 8:00' ,1,2 )
insert into ta values('2020-01-02 8:00' ,1,3 )
insert into ta values('2020-01-02 8:00' ,1,4 )
insert into ta values('2020-01-02 8:00' ,1,5 )
insert into ta values('2020-01-02 14:00' ,2,6 )
insert into ta values('2020-01-02 14:00' ,2,2 )

insert into tb values('2020-01-01 7:45','2020-01-01 8:45' ,1,300)
insert into tb values('2020-01-01 7:47','2020-01-01 8:25' ,2,100)
insert into tb values('2020-01-01 7:45','2020-01-01 8:35' ,4,10)
insert into tb values('2020-01-02 7:45','2020-01-02 8:45' ,1,30)
insert into tb values('2020-01-02 7:45','2020-01-02 8:05' ,2,40)
insert into tb values('2020-01-02 7:45','2020-01-02 8:15' ,3,20)
insert into tb values('2020-01-02 14:00','2020-01-02 14:15' ,6,20)
insert into tb values('2020-01-01 14:00','2020-01-01 14:15' ,2,25)

 

decimal number smaller than 1 in Excel is rounded to 0 after loading data to PBI

$
0
0

Hi all,
i know that is a common issue but i have not found any solution for it.

i have customers monthly sales data by products and it is common that some of them buy for example 0.5 unit of x product, but when i load this data to PBI all these decimal numbers are automatically rounded to zero and on project it seems they did not buy this x product at all which is not true and misleading.

please help me solve that,
thanks
Amit

Viewing all 31288 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>