Code Monkey home page Code Monkey logo

libpq's Introduction

LibPQ - detach your M code from workbooks and reuse it!

This project offers a collection of functions and useful queries for Power Query Formula Language (also known as M language) and a loader to import source code from external modules into Power Query.

Feature highlights

More information

What other people say about LibPQ

Installation and usage

Copy source code to the filesystem (optional)

Download the contents of this repo and extract it anywhere you want. If you skip this step you will not be able to list all available modules with LibPQ() call and every time you load a module LibPQ will access the network.

Add LibPQ to the workbook

Create two blank queries in your workbook:

  • LibPQ - copy the contents of LibPQ.pq verbatim
  • LibPQPath - provide location of modules source code (local and/or web). See sample at LibPQPath-sample.pq

Import LibPQ modules when writing your queries

Import any of the available modules via LibPQ("modulename") when writing your queries in Advanced Editor.

To import all modules available locally use LibPQ() with no arguments (or null). That will return a record where keys are module names and values are the results of loading those modules (function object or output of a query).

LibPQ modules

All modules are grouped into namespaces, much like the standard library:

  • Date - Functions for working with dates
  • Factory - Functions that create other functions
  • Folder - Functions for manupulating folders and their contents
  • Function - Functions that operate on other functions
  • List - Functions for working with lists
  • Module - LibPQ tools for importing source code
  • Read - Functions that import data
  • Table - Functions for table operations
  • Tests - Unit tests for LibPQ modules
  • Text - Functions for working with text
  • UnitTest - Unit testing framework

Detailed module list is available in documentation, and each module has a clear description in the comment at the top of its source file.

Compatibility with @tycho01's library

LibPQ allows specifying multiple possible locations for source code, that's why you can add @tycho01's library as a fallback.

Any module from that library should be supported, though some will require adding a Load query to your workbook. To create a Load alias for LibPQ you need to:

  • Create a new blank query
  • Replace all its code with text LibPQ
  • Save the query under the name Load

Now the Load function will be available from #shared context, and you should be able to import any module from @tycho01's library.

Contributing

All contributions are welcome! Please check CONTRIBUTING.md for details

Support

Issue tracker

GitHub's issue tracker is the primary venue for asking and answering support questions. Please don't forget to search closed issues for the topic you're interested in!

If you know an answer to the question asked by someone else, please do not hesitate to post it! That would be a great help to the project!

Email

If for some reason you'd rather not use the issue tracker, contacting me via email is OK too. Please use a descriptive subject line to enhance visibility of your message. Also please keep in mind that support through the channels accessible to the public is preferable because one answer can help many people who might read it later.

My email is visible under the GitHub profile and in the commit log.

Community support

You might get faster and more interactive support from other users.

This project does not (yet) have a dedicated community venue, but experienced PowerBI/Power Query users will most likely be able to figure out most common questions. You can try asking at the local PowerBI-related forums, Facebook groups, IRC/Discord/Telegram chats or on Reddit/StackOverflow.

License and copyright

Copyright © 2017-2021 Vitaly Potyarkin

   Licensed under the Apache License, Version 2.0 (the "License");
   you may not use these files except in compliance with the License.
   You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.

libpq's People

Contributors

sio avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

libpq's Issues

Feedback from users

Hello! If you are using LibPQ (or have used, or have tried to use it but something didn't work out for you) - please leave some feedback in comments below.

  • Does LibPQ suit your needs? Were you able to successfully use it in any workbooks/reports?
  • Was the documentation sufficient? What topics were unclear or missing?
  • What LibPQ features do you consider most important?

Any feedback beside the three points above is also greatly appreciated! Thank you!

Path as an argument to LibPQ

Maybe:

Allow optional second argument to LibPQ() that passes the path record to override LibPQPath

I was considering allowing to override the values provided by LibPQPath for some calls, e.g. if you'd want LibPQ("CoolModule") to return one thing in one report and another thing in another one - both in the same workbook. I have not encountered a use case for this idea, so I've just left it at that.

Unit testing

Add a basic framework for writing and executing unit tests

Compatibility with PowerBI service

Looks like LibPQ can not be used in PowerBI service because of providing dynamic arguments to Web.Content(). This page is the place to discuss, develop and test possible workarounds.

Important

I have no account with PowerBI service, so I will need outside help to do the tests

Roadmap

  • Describe the testing sequence (@sio)
  • Test LibPQ 1.2.0 with PowerBI service (@spythan?)
  • Modify loader to use static base URL with Web.Contents (@sio)
  • Test modified version of LibPQ with PowerBI service (not assigned)
  • Write documentation on using LibPQ with Power BI service

Move test code into a separate directory

As the number of tests grows, the test modules create unnecessary clutter in the Modules/ directory.

We need to:

  • Move tests into a separate directory
  • Decide on another prefix for tests, because Tests/Tests.Name.pq seems redundant but tests module names still should not collide with normal module names
  • Update LibPQPath.sample.pq to enable importing tests

Unit Testing - Tests with coding errors disappear from test run results

When a unit test file has an error in it, the entire file's tests disappear from the UnitTest Results. To Reproduce do the following:

  1. With LibPQ configured to run the LibPQ Unit Tests, Run UnitTest.Discover()
  2. Notice that the Test.MicrosoftUnitTestDemo tests are listed.
  3. Edit Tests\Test.MicrosoftUnitTestDemo.pq and comment out line 12 (UnitTesting.Returns123...)
  4. Save and refresh the UnitTest.Discover()

The Test.MicrosoftUnitTestDemo tests are no longer listed. It would be more desirable to list the file as a single row with an error. I've looked through the code, and tried a number of things, without success, so thought it was worthy of an "Issue".

I'm glad to help with resolving, if someone could give me some pointers.

Add support for subtests

Single test (one function): testName = () => Assert
Subtests (list of functions): testName = {()=>Assert, ()=>Assert, ()=>Assert, ...}

Changes can be stitched into the line below. We can invoke single test function or run all test functions in a list of subtests if needed.

{"Result", each try Record.Field(Suite, [Test])()},

LibPQ disabling Power Query intellisense?

Originally posted by @B-D-T in #12 (comment)

I apologize if I'm missing something, but it appears that adding LibPQ breaks the Power Query intellisense. In Excel workbooks where I don't have LibPQ/LibPQPath, I get the intellisense function list, arguments, etc. But for those where I've integrated LibPQ, the intellisense only reads objects that were defined within the existing module.

Almost all my work is done directly in M in the Advanced Editor. Having intellisense makes that process easier, but now that I've discovered the wonderful LibPQ, I don't want to give that up either! Any suggestions?

Release LibPQ as *.mez file

Idea

  1. Create a build script that packages LibPQ into *.mez file
  2. Build and test LibPQ as Power Query Extension
    • Decide where to store LibPQPath (or maybe introduce sensible defaults)
    • Decide whether to include only LibPQ or to package all default modules as well

More information

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.