Data Access

Over the next few weeks, part of the engineering team at Pentaho will be working towards making it easier to get access to your data.  The two use cases we’re addressing in the short term include accessing your SQL data, along with uploading a flat file (CSV, Excel) to drive a report or chart from within Pentaho’s user console.  Our general approach for both of these scenarios is to use Pentaho’s Metadata layer to abstract the querying of data sources.  This allows us to use a common interface and common widgets in our client apps.  To do this, we’ll be extending Pentaho Metadata to include new physical model implementations.  The team has started to prototype some of these capabilities.  We’ve added a web services layer to Pentaho Metadata, and also have started work on the physical models as well as our common widgets, which use our Java / GWT XUL UI framework.

We’re also spending a lot of time thinking about the long term direction of our metadata layer.  I’ve created a community project page for the Metadata project, with links to documentation, binaries and source to make it easier to get involved in the project. Doug will be hosting a live community webex in the next couple of weeks to have a general conversation about where we should take the metadata layer.  We want to make it as easy as possible for folks to start using Pentaho, and we’re going to make that possible through our rich and easy to use metadata layer.

5 Comments so far

  1. Pedro Alves on April 14th, 2009

    Great news! Don’t forget MDX input and kettle transformations!

  2. willgorman on April 14th, 2009

    Hey Pedro, thanks for chiming in. Both are on the roadmap. Part of the community discussion that Doug will be hosting includes working out exactly how Pentaho Metadata and OLAP play together in a friendly manner. In regards to kettle transformations, that’s what will power flat file based data access, so expanding that shouldn’t be difficult!

  3. Andy Grohe on April 15th, 2009

    Looks like a good start.

    I know that I could wait to ask Doug, but I get some questions out here.

    You mentioned “a web services layer” but I do not see any information related to that on the link you provided?

    Does this replace AdhocWebService?

    Will you rework WAQR to use the new services?

    Will the new query and reporting wizard use the new services?

  4. willgorman on April 15th, 2009

    Hi Andy,

    The web services layer is still in the prototype phase, I’ve updated the metadata project page with a link to the webservices svn. Most of the code lives in the metadata project, under the packages org.pentaho.pms.schema.v3.

    Portions of the AdhocWebService and WAQR will be rewritten or phased out and replaced with the data access layer. One of our user stories is to replace the first screen of Pentaho Ad Hoc Reporting with the Data Access wizard.

    I’m not sure what you mean by “new query and reporting wizard”. We are writing the data access UI with XUL so we can reuse it in Report Designer and Report Design Wizard, although this will take place after we’ve updated WAQR and the Dashboard Chart Editor.

  5. Peter Schmidt on April 16th, 2009

    Sounds interesting! I second Pedro’s comments specifically around how does PME and the Mondrian Schema play nicely so to speak. There is so much synergy surrounding the meta data layer that could be produced from PME and the metadata layer (resident in the mondrian schema). Perhaps it is a chicken and the egg scenario. I can write MDX to go against the mondrian schema that then turns it into SQL to retrieve data. I can also do directly against the meta data layer to write a report in turn using SQL to retrieve the data. Ideally, I should be always able to go against the common meta data layer, but should it then send MDX to return data or just SQL? Or should it always send SQL and then the end delivery is different (i.e. mondrian cube vs a report). Not sure, myself. Also, they idea of storing a metric definition once and only once should be supported in the PME.
    Thanks!

Leave a reply