Discover, Analyze, Explore, Pivot, Drilldown, Visualize your data… “How do I know what I think until I see what I say?” [E.M. Forster, G. Wallas, A. Gide]
My best wishes for 2012 to the members of Data Visualization community!
By conservative estimates, which includes registered and active users of Data Visualization (DV) tools, DV specialists from customers of DV vendors, consultants and experts from partners of DV vendors and employees of those vendors, the Data Visualization (DV) community exceeds 2 millions of people in 2011! I am aware of at least 35000 customers of leading DV vendors, at least 3000 DV consultants and experts and at least 2000 employees of leading DV vendors.
With this audience in mind and as the extension of this blog, I started in 2011 the Google+ page "Data Visualization" for DV-related news, posts, articles etc., see it here:
Due the popular demand and the tremendous success of Tableau in 2011 (basically you can say that 2011 was a year of Tableau) I started recently the new blog (as an extension of this blog), called ... "Data Visualization with Tableau", see it here:
I also have some specific best wishes for 2012 to my favorite DV vendors.
To Microsoft: please stop avoiding DV market and build a real DV tool (as oppose to a nice BI stack) and integrate it with MS-Office the same way as you did with Visio.
.
To Qliktech: I wish Qliktech will add a free Desktop Qlikview Reader, a free (limited of course) Qlikview Public Web Service and integrate Qlikview with R Library. I wish Qliktech will consider the consolidation of its offices and moving at least part of R&D into USA (MA or PA). I think that having too much offices and specifically having R&D far away from product management, marketing, consulting and support forces is not healthy. And please consider to hire more engineers as oppose to sales and marketing people.
.
To TIBCO and Spotfire: please improve your partner program and increase the number of VAR and OEM partners. Please consider the consolidation of your offices and moving at least part of your R&D into USA (MA that is). And I really wish that TIBCO will follow the super-successful example from EMC (VMWare!) and spinoff Spotfire with public IPO. Having Spotfire as the part of larger parent corporation slows sales considerably.
.
To Tableau: I wish Tableau will able to maintain its phenomenal 100% Year-over-Year growth in 2012. I wish Tableau will improve their partner program and integrate their products with R Library. And I wish Tableau will open/create API and add scripting to their products.
To Visokio: I wish you more customers, ability to hire more developers and other employees, more profit and please stay on your path!
.
To Microstrategy, SAS, Information Builders, Advizor Solutions, Pagos, Panorama, Actuate, Panopticon, Visual Data Mining and many, many others - my best wishes in 2012!
One of the most popular posts on this blog was a comparison of Data Visualization Tools, which originally was posted more then a year ago where I compared those best tools only qualitatively. However since then I got a lot of requests to compare those tools "quantitatively". Justification for such update were recent releases of Spotfire 4.0, Qlikview 11, Tableau 7.0 and Microsoft's Business Intelligence Stack (mostly SQL Server 2012 and PowerPivot V.2.)
.
However I quickly realized that such "quantitative" comparison cannot be objective. So here it is - the updated and very subjective comparison of best Data Visualization tools, as I see them at the end of 2011. I know that many people will disagree with my assessment, so if you do not like my personal opinion - please disregard it at "your own peril". I am not going to prove "numbers" below - they are just my personal assessments of those 4 technologies - I love all 4 of them. Feel free to make your own comparison and if you can share it with me - I will appreciate it very much.
.
Please keep in mind that I reserve the right to modify this comparison overtime if/when I will learn more about all those technologies, their vendors and usage. Criterias used in comparison below listed in 1st column and they are grouped in 3 groups: business, visualization and technical. Columns 2-5 used for my assessments of 4 technologies, last column used for my subjective weights for each criteria and last row of this worksheet has Total for each Data Visualization technology I evaluated.
I said on this blog many times that 80% of Data Visualization (DV) is ... Data.
SQL Server 2012 is here.
And technology and process of how these Data collected, extracted, transformed and loaded into DV backend and frontend is a key to DV success. It seems to me that one of the best possible technology for building DV backend is around the corner as SQL Server 2012 will be released soon - Release Candidate for it is out...
And famous Microsoft marketing machine is not silent about it. SQL Server 2012 Virtual Launch Event planned for March 7, 2012 and real release probably at the end of March 2012.
Columnstore Index.
I already mentioned on this blog the most interesting feature for me - the introduction of Columnstore Index (CSI) can transform SQL Server into Columnar Database (for DV purposes) and accelerates DV-relevant Queries by 10X or even 100X of times. Oracle does not have it!
.
Some reasonable rules and features applied to CSI: each table can have only one CSI; CSI has Row grouping (about million rows, like paging for columns); table with CSI cannot be replicated. New (unified for small and large memory allocations) memory manager optimized for Columnstore Indexes, supports Windows 8 maximum memory and logical processors.
Power View.
SSRS (Reporting Services) got massive improvements, including new Power View as Builder/Viewer of interactive Reports. I like this feature: "even if a table in the view is based on an underlying table that contains millions of rows, Power View only fetches data for the rows that are visible in the view at any one time" and UI features (some of them are standard for existing Data Visualization tools, like multiple views in Power View reports (see gallery of thumbnails in the bottom of screenshot below):
.
"2 clicks to results", export to PowerPoint etc. See also video here:
.
.
PowerView is still far behind Tableau and Qlikview as a Visualizer, but at least it makes SSRS reports more interactive and development of them easier. Below are some thumbnails of Data Visualization samples produced with PowerView and presented by Microsoft:
Support for Big Data.
SQL Server 2012 has a lot new features like "deep" HADOOP support (including Hive ODBC Driver) for "big data" projects, ODBC drivers for Linux, grouping databases into Availability Group for simultaneous failover, Contained Databases (enable easy migration from one SQL Server instance to another) with contained Database users.
Parallel Data Warehouse, Azure, Data Explorer.
And don't forget PDW (SQL Server-based Parallel Data Warehouse; massive parallel processing (MPP) provides scalability and query performance by running independent servers in parallel with up to 480 cores) and SQL Azure cloud services with it high availability features...
.
New Data Explorer allows discover data in the cloud and import them from standard and new data sources, like OData, Azure Marketplace, HTML etc. and visualize and publish your Data to the cloud.
LocalDB.
LocalDB is a new free lightweight deployment option for SQL Server 2012 Express Edition with fewer prerequisites that installs quickly. It is an embedded SQL Server database for desktop applications (especially for DIY DV apps) or tools. LocalDB has all of the same programability features as SQL Server 2012 Express, but runs in user mode with applications and not as a service. Application that use LocalDB simply open a file. Once a file is opened, you get SQL Server functionality when working with that file, including things like ACID transaction support. It’s not intended for multi-user scenarios or to be used as a server. (If you need that, you should install SQL Server Express.)
BIDS.
SQL Server 2012 is restoring a very desirable feature, which was missing in Visual Studio 2010 for 2+ years - something called BIDS (BI Development Studio was available as part of Visual Studio 2008 and SQL Server 2008). For that a developer needs VS2010 installed with SP1 and then install "SQL Server Data Tools" (currently it is in the state of CTP4, but I guess it will be a real thing when when SQL Server 2012 will be released to production).
SSAS, Tabular Mode, PowerPivot, DAX.
Most important improvement for BI and Data Analytics will be of course the changes in SSAS (SQL Server Analysis Services), including the addition of Tabular Mode, restoration of BIDS (see above), the ability to design local multidimensional cubes with PowerPivot and Excel and then deploy them directly from Excel as SSAS Cubes, the new DAX language shared between PowerPivot and SSAS, and availability of all those Excel Services directly from SSAS without any need for SharePoint. I think those DV tools who will able to connect to those SSAS and PowerPivot Cubes will have a huge advantage. So far only Tableau has it (and Omniscope has it partially).
Backend for Data Visualization.
All of these features making SQL Server 2012 a leading BI stack and backend for Data Visualization applications and tools. I just wish that Microsoft will develop an own DV front-end tool, similar to Tableau or Qlikview and integrate it with Office 201X (like they did with Visio), but I guess that DV market ( approaching $1B in 2012) is too small compare with markets for Microsoft Office and SQL Server.
Pricing.
Now is time for a "bad news". The SQL Server 2012 CAL price will increase by about 27%. New pricing you can see below and I predict you will not like it:
Some of visitors to this blog after reading of my recent post about $300K/employee/year as a KPI (Key Performance Indicator) suggested to me another Indicator of the health of Data Visualization vendors: a number of job openings and specifically a number and percentage of software development openings (I include software testers and software managers into this category) and use it also as a predictor of the future. Fortunately it is a public data and below is what I got today from respective websites:
56(!) positions at Tableau, 14 them of are developers;
46 openings at Qliktech, 4 of them are developers;
21 positions at Spotfire, 3 of them are developers;
3 positions at Visokio, 2 of them are developers.
Considering that Tableau is 4 times less in terms of sales then Qlikview and 3-4 times less (then Qliktech) in terms of workforce, this is an amazing indicator. If Tableau can sustain this speed of growth, we can witness soon the change of Data Visualization landscape, unless Qliktech can find the way to defend its dominant position (50% of DV market).
.
For comparison, you can use Microstrategy's number of openings. While Microstrategy is not a Data Visualization vendor, it is close enough (as BI vendor) for benchmarking purposes: it has 281 openings, 38 of them are developers and current Microstrategy's workforce is about 3069, basically 3 times more then Qliktech's workforce...
.
In light of recent releases of Qlikview 11 and Spotfire 4.0 it makes (soon to be released) Tableau 7.0 is very interesting to compare... Stay tuned!
I expected Qlikview 11 to be released on 11/11/11 but it was released today to Qliktech partners and customers. Since Qliktech is the public company, it releases regularly a lot of information which is not available (for now) from other DV leaders like Tableau and Visokio and more fuzzy from Spotfire, because Spotfire is just a part of larger successful public corporation TIBCO, which has many other products to worry about.
However I guessed a little and estimated for DV Leaders their 2011 sales and number of employees and got an interesting observation, which is true for a few last years: size of sales per employee (of DV leading vendor) is $300k/Year or less. I included for comparison purposes similar numbers for Apple, Microsoft and Google as well as for Microstrategy, which is a public company, established (22+ years) player in BI market, dedicated to BI and recently to Data Visualization (that is DV, thanks to it Visual Insight product).
Table below included 2 records related to Spotfire: 1 based on 2010 annual report from TIBCO (for TIBCO as whole; I know TIBCO sales for 2011 grew from $754M to $920M but do not know the exact number of TIBCO's employees for 2011) and other record is my estimates (of a number of employees and sale) for Spotfire division of TIBCO. Update from 1/11/12: For Tableau's 2011 I used the numbers from John Cook's article here: http://www.geekwire.com/2012/tableau-software-doubles-sales-2011-hires-160-workers ) :
To me this is an interesting phenomena, because Qliktech thanks to its fast growing sales and recent IPO was able to double it's sales in last 2 years while ... doubling it's number of employees so it still has its sales hovering around $300K/employee/year, while Software giants Apple, Microsoft and Google are way above this barrier and Microstrategy is 50% below it. I will also guess that Qliktech will try to break this $300K barrier and be closer to Apple/Microsoft/Google in terms of sales per employee.
Thanks to the public nature of Qliktech we know details of its annual Revenue growth and YoY (Year-over-Year) indicators:
and with estimate of 2011 Revenue about $315M, YoY growth (2011 over 2010) will be around 39.4% which is an excellent result, making it difficult (but still possible) for other DV competitors to catch-up with Qliktech. Best chance for this belongs to Tableau Software, who probably will reach the same size of sales in 2011 as Spotfire (my estimate is around $70M-$75M for both), but for last 2 years Tableau has 100% (or more) YoY revenue growth... Qliktech also published the interesting info about major factors for its sales: Europe (56%), Existing Customers (58%), Licenses (61%), Partners(52%):
which means that the increase of sales in Americas, improving New sales (as oppose to sales to existing customer by using "Land and Expand" approach) and improving revenue from Services and Maintenance may help Qliktech to keep the pace. Qliktech has the tremendous advantage over its DV competitors because it has 1200+ partners, who contributed 52% to Qliktech sales (about $136K per partner and I can guess that Qliktech wish to see at least $200K/year contribution from each partner).
Observing the strengths of other DV competitors, I personally think that Qliktech will benefit from the "imitation" of some of their most popular and successful features in order to keep its dominance in Data Visualization market, including:
free public Qlikview service (with obvious limitations) like free SaaS from Tableau Public and free Spotfire Silver personal edition,
ability to distribute Data Visualization to desktops without Server by making available a free desktop Qlikview Reader (similar to free desktop readers from Tableau and Omniscope/Visokio),
integration with R library (Spotfire and recently Omniscope) to improve analytical power of Qlikview users,
ability to read multidimensional OLAP Cubes (currently only Tableau can do that), especially Cubes from Microsoft SQL Server 2012 Analysis Services and
scalability toward Big Data (currently Spotfire's and Tableau's data engines can use the disk space as Virtual Memory but Qlikview limited by size of RAM)
This is not a never ending "feature war" but rather a potential ability to say to customers: "why go to competitors, if we have all their features and much more"? Time will tell how DV competition will play out, I expect a very interesting 2012 for Data Visualization market and users and I hope that somebody will able to break $300K/employee/year barrier unless the major M&A will change the composition of DV market. I hope that the DV revolution will continue in new year...
I never liked pre-announcements of "new" products, especially if they are in state which will screw my PCs. But almost everybody doing it to us, starting with Microsoft SQL Server 2012 (Denali can be downloaded as "CTP3"), Tableau 7.0, Qlikview 11 (Qliktech partners and customers can download "Release candidate") etc. Just a few months after releasing Spotfire 3.3, TIBCO announced that Spotfire 4.0 will be available in November 2011 with a lot of new features.
.
Some of them sound like buzzwords: ""free dimensional" analytics, collective intelligence, visual and social data discovery etc." (we need that marketing will brainwash us, right?), but some of them can be very useful, like integration with TIBBR (that I like; in fact TIBCO has many other good products and they should be integrated with Spotfire) and SharePoint (sounds like a M$ bending to me, I don't see too much DV money coming from SharePoint hole), support for dynamic icons, sparklines,
stepped linecharts, pop-over filters and legends, better font management, embedded actions and more. Some features I wish will be added, but I guess we need to wait more: I wish to be able to read with Spotfire the SSAS and PowerPivot multidimensional Cubes and support for some other Data Sources, like Tableau 6.1 does...
.
Spotfire and its Web Player Server support now the latest web browsers, .NET 4.0 and it dropped support for obsolete stuff like Internet Explorer 6 and Windows 2003 Server. I mentioned on this blog earlier that I like Spotfire Silver 2.0 and the wealth and depth of Spotfire Analytical Platform (S-Plus, Miner, S+FinMetrics, Spotfire Developer/API, Statistics, Data and Automation Services, Metrics, Network Analysis, Decision Site, Clinical Graphics and more, this list should make Qliktech and Tableau worry or at least try to add similar features...).
.
Spotfire updated their set of Demos to reflect Spotfire 4.0 features: Spotfire Demos and Templates. More to come later, especially when Spotfire 4.0 will be Released (as oppose to be announced).
Data Visualization has at least 3 parts: largest will be a Data, the most important part will be a Story behind those Data and a View (or Visualization) is just an Eye Candy on top of it. However only a View allows users to interact, explore, analyze and drilldown those Data and discover the Actionable Info, which is why Data Visualization (DV) is such a Value for business user in the Big (and even in midsized) Data Universe.
Productivity Gain.
One rarely covered aspect of advanced DV usage is a huge a productivity gain for application developer(s). I recently had an opportunity to estimate a time needed to develop an interactive DV reporting application in 2 different groups of DV & BI environments
Samples of Traditional and Popular BI Platforms.
Open Source toolsets like Jaspersoft 4/ Infobright 4/ MySQL (5.6.3)
MS BI Stack (Visual Studio/C#/.NET/DevExpress/SQL Server 2012)
Tried and True BI like Microstrategy (9.X without Visual Insight)
Samples of Advanced DV tools, ready to be used for prototyping
Spotfire (4.0)
Tableau (6.1 or 7.0)
Qlikview (11.0)
Results proved a productivity gain I observed for many years now: first 3 BI environments need month or more to complete and last 3 DV toolsets required about a day to complete entire application. The same observation done by ... Microstrategy when they added Visual Insight (in attempt to compete with leaders like Qlikview, Tableau, Spotfire and Omniscope) to their portfolio (see below slide from Microstrategy presentation earlier this year, this slide did not count time to prepare the data and assume they are ready to upload):
I used this productivity gain for many years not only for DV production but for Requirement gathering, functional Specifications and mostly importantly for a quick Prototyping. Many years ago I used Visio for interactions with clients and collecting business requirements, see the Visio-produced slide below as an approximate example:
DV is the best prototyping approach for traditional BI
This leads me to a surprising point: modern DV tools can save a lot of development time in traditional BI environment as ... a prototyping and requirement gathering tool. My recent experience is that you can go to development team which is completely committed for historical or other reasons to a traditional BI environment (Oracle OBIEE, IBM Cognos, SAP Business Objects, SAS, Microstrategy etc.) and prototype for such team dozens and hundreds new (or modify existing) reports in a few days or weeks and give it to the team to port it to their traditional environment.
These DV-based prototypes have completely different behavior from previous generation of (mostly MS-Word and PowerPoint based) BRD (Business Requirement Documents), Functional Specification, Design Documents and Visio-based application Mockups and prototypes: they are living interactive applications with real-time data updates, functionality refreshes in a few hours (in most cases at the same day as new request or requirement is collected) and readiness to be deployed into production anytime!
However, my estimate that 9 out of 10 such BI teams, even they will be impressed by prototyping capabilities of DV tools (and some will use them for prototyping!), will stay with their environment for many years due political (can you say job security) or other (strange to me) reasons, but 1 out of 10 teams will seriously consider to switch to Qlikview/Tableau/Spotfire. I see this as a huge marketing opportunity for DV vendors, but I am not sure that they know how to handle such situation...
Example: using Tableau for Storytelling: [youtube=https://www.youtube.com/watch?v=_2u-cQED1ek]
Spreadsheets (VisiCalc or "Visible Calculator" was released by Dan Bricklin and Bob Frankston in October 1979 - 32 years ago - originally for Apple II computer) were one of the very first Business Intelligence (BI) software (sold over 700,000 copies in six years).
For historical purposes I have to mention that VisiCalc actually was not the first spreadsheet program invented (for example I am aware of multi-user spreadsheet software written before VisiCalc in USSR in PL/1 for mainframes with IBM's IMS Database as a backend ), but it is a first commercial spreadsheet introduced on American market and it was a turning point of PC industry.
The "Visible Calculator" went on sale in November of 1979 and was a big hit. It retailed for US$100 and sold so well that many dealers started bundling the Apple II with VisiCalc. The success of VisiCalc turned Apple into a successful company, selling tens of thousands of the pricey 32 KB Apple IIs (no matter how hard Bob Frankston tried, he could not fit VisiCalc in the 16 KB of RAM on the low-end Apple II. VisiCalc would only be available for the much more expensive 32 KB Apple II) to businesses that wanted them only for the spreadsheet. Version of VisiCalc for Atari was even retailed for $200!
VisiCalc was published without any Patent and it is a living prove that Patent System currently is useless for people, abused by large corporations for their own benefit, and it is actually a brake for innovations and it is not protecting inventors. Absence of patent protection for VisiCalc created the Spreadsheet Revolution and Innovations (SuperCalc, Lotus 1-2-3, QuattroPro, Excel, OpenOffice's Calc, Google's Spreadsheets and many others) and tremendously accelerated PC industry.
As Dan Bricklin said it by himself "We all borrowed from each other" and as George Bernard Shaw said: "If you have an apple and I have an apple and we exchange these apples then you and I will still each have one apple.But if you have an idea and I have an idea and we exchange these ideas, then each of us will have two ideas."
Application of Spreadsheets in the BI field began with the integration of OLAP (On-Line Analytical Processing) and Pivot tables. In 1991, Lotus (in addition to 1-2-3) released Improv with Pivoting functionality (also see Quantrix as a reborned [originally in 1994-95] Improv), followed by Microsoft’s release (in Excel 5) of PivotTable in 1993 (trademarked by Microsoft). 500+ millions people currently using Excel and at least 5% of them using it for BI and Data Visualization purposes. PowerPivot added to Excel 2010 speedy and powerful in-memory columnar database which enables millions of end-users to have a self-serviced BI.
Essbase was the first scalable OLAP software to handle large data sets that the early spreadsheet software was incapable of. This is where its name comes from: Extended Spread Sheet Database (Essbase owned by Oracle now). Currently one of the best OLAP and BI software is SSAS (Analysis Services from Microsoft SQL Server 2008 R2 and upcoming SQL Server 2012 with its new Tabular mode) and Excel 2010 with its PowerPivot, PivotTables and Pivot Charts is one of the most popular front-end for SSAS.
There is no doubt that Excel is the most commonly used software for "BI purposes". While Excel is general business software, its flexibility and ease of use makes it popular for data analysis with millions of users worldwide. Excel has an install base of hundreds of millions of desktops: far more than any other "BI platform". It has become a household name.With certain precaution it can be used for a good or at least prototyping Data Visualization (most of charts below created with Excel):
From educational utilization to domestic applications to prototyping (or approximated) Data Visualization and enterprise implementation, Excel has been proven incredibly indispensable. Most people with commercial or corporate backgrounds have developed a proficient Excel skillset. This makes Excel the ultimate self-service BI platform and spreadsheet technologies as a common ground for all viable Data Visualization technologies on market.
is announced on 10/11/11 - one year after 10/10/10, the release date of Qlikview 10! Qliktech also lunched new demo site with 12 demos of Qlikview 11 Data Visualizations: http://demo11.qlikview.com/ . Real release happened (hopefully) before end of 2011, my personal preference for release date will be 11/11/11 but it may be too much to ask...
QlikView 11 introduces the comparative analysis by enabling the interactive comparison of user-defined groupings. Also now with comparative analysis business users have the power of creating any (own) data (sub)sets and decide which dimensions and values would define the data sets. Users can then view the data sets they have created side by side in a single chart or in different charts:
Collaborative Data Visualization and Discovery.
Also Qlikview 11 enables Collaborative Workspaces – QlikView users can invite others – even those who do not have a license – to participate in live, interactive, shared sessions. All participants in a collaborative session interact with the same analytic app and can see others’ interactions live, see
QlikView users can engage each other in discussions about QlikView content. A user can create notes associated with any QlikView object. Other users can then add their own commentary to create a threaded discussion. Users can capture snapshots of their selections and include them in the discussion so others can get back to the same place in the analysis when reviewing notes and comments. QlikView captures the state of the object (the user’s selections), as well as who made each note and comment and when. Qliktech's press release is here:
“Our vision for QlikView 11 builds on the fact that decisions aren’t made in isolation, but through social exchanges driven by real-time debate, dialog, and shared insight,” says Anthony Deighton, CTO and senior Vice President, Products at QlikTech. “QlikView 11’s social business discovery approach allows workgroups and teams to collaborate and make decisions faster by collectively exploring data, anywhere, anytime, on any device. Business users are further empowered with new collaborative and mobile capabilities, and IT managers will appreciate the unified management functionality that allows them to keep control and governance at the core while pushing usage out to the edges of the organization.”
New Features in Qlikview 11
Qlikview now is integrated (I think it is a big deal) with TFS - source control system from Microsoft. This makes me think that may be Donald Farmer (he left Microsoft in January 2011 and joined Qliktech) has an additional assignment to make it possible for Microsoft to buy Qliktech? [Dear Donald - please be careful: Microsoft already ruined ProClarity and some others after buying them]. Free QlikView 11 Personal Edition will be available for free download by the end of year at www.qlikview.com/download.
Oracle's timing for "unveiling Exalytics In-Memory Machine" was unfortunate because it was in a shadow of Steve Jobs. In addition It was a lot of distraction between Larry Ellison's and Mark Benioff's egos.
Oracle is late to Analytics appliance game and have to fight already released products like Netezza/IBM (proven performer), SAP HANA (has large sales pipeline already), family of Teradata Appliances (Teradata Columnar coming in 2 months and sounds very good to me plus it packaged with Information Builders BI) , EMC/Greenplum Data Computing Appliance (doubled the sales during last year!), Microsoft Parallel Data Warehouse Appliance (Based on CTP3 I expect the great things from SQL Server 2011/2012/Denali) etc. They all are in-memory Machine, capable to store and process big data (exabytes? I guess depends on price...), almost all of them already have or will have soon columnar database.
Larry Ellison claimed during Oracle Openworld this week that "Exalytics is 10x faster than…just about everything."
Yes, It runs a software stack that includes parallelized versions of Oracle’s TimesTen in-memory database and memory-optimized Essbase OLAP Server ("BI Foundation"), but it is not a columnar database, so I wonder how Oracle is going to prove Larry's bold claims. However, Oracle TimesTen In-Memory Database for Exalytics supports columnar compression that reduces the memory footprint for in-memory data. Compression ratios of 5X are practical and help expand in-memory capacity (Qlikview, PowerPivot and Spotfire can do much better "columnar compression" then 5 times, claimed by Oracle)
2 quad-data rate (QDR) 40 GB/s InfiniBand ports. When connected to Oracle Exadata, Oracle Exalytics becomes an integral part of the Oracle Exadata private InfiniBand network and has high-speed, low latency access to the database servers. When multiple Oracle Exalytics machines are clustered together, the InfiniBand fabric also serves as the high-speed cluster interconnect.
Exalytics has Two 10 GB/s Ethernet ports for connecting to enterprise data sources
Exalytics has Four 1 GB/s Ethernet ports are available for client access
Exalytics includes 3.6TBs of raw disk capacity. Optionally, clusters of Oracle Exalytics machines can leverage network attached storage.
Hardware portion of it probably below $100000 (I saw a guesstimate of $87000) but most expensive probably will be the Essbase (Business Intelligence Foundation Suite with in-memory Cubes now and ability to replicate entire data warehouse into TimesTen in-memory database) with list price about $450000, so we are talking here about millions of dollars, which is (let's wait and see the final pricing) will definitely reduce the number of potential buyers, especially considering weak Data Visualization and average BI functionality of Oracle's software stack. According to Larry Ellison, Exalytics has 1TB of RAM but can hold five to 10TB of data in memory thanks to COLUMNAR compression.
Oracle Exalytics promotes self service analytics and makes it easier to develop analytics content by introducing a Presentation Suggestion Engine (PSE) which provides recommendations on type of visualizations to use to best represent a data set.
I do not expect anything spectacular from this "PSE". For example Oracle proudly introduced "new micro charts and multi-panel trellis charts to visualize dense multi-dimensional, multi-page data on a single screen. The multi-panel trellis charts are particularly effective at displaying multiple visualizations across a common axis scale for easy comparison, to see a trend and quickly gain insights":
but this micro charts available in much better shape and form for many years from Spotfire, Qlikview, Tableau etc. and relatively recently even from Excel.
In any case, Exalytics suppose to be well integrated with Oracle's Exadata database machine and Exalogic application server. Mr. Ellison did some other bold claims like:
"For a given task, it will cost you less on an Exadata than it would on a plain old commodity server."
"we move data around a hundred times faster than anyone else in this business"
"1,000 Exadata machines have been installed and 3,000 more will be sold this year"
"Java applications' response times are 10 times as fast on Exalogic, and companies can serve many more users at once"
Special Note about Java.
I am not sure why Java is advantage for Oracle. Java is not welcome at Apple (can you say Objective C?), at Microsoft (can you cay C# ?) and recently even at Google (after Oracle sued Google for “misuse” of Java, which reminded me the Sun, disappearing after it sued Microsoft for … “misuse” of … Java). Together those 3 companies have almost all cash (almost $200B if you exclude Oracle as a Java Owner) software companies have worldwide (Apple has $76B+ in a bank, Microsoft has $60B+ and Google has about $40B – may be less after buying Motorola Mobility) and I am simply following the money here. If Oracle wishes to have the Java-based advanced Data Visualization, they are better buy Visokio and integrate their Omniscope with Exalytics and Exalogic instead of the inventing the wheel with PSE.
Do you want the 1st class Data Visualization on your cool Mac without any Virtual Machine with Windows? If so, your best choice will be the Omniscope 2.6 which is finally about to be released (after more then 2 years of delays) by Visokio, located in UK. Of course the Omniscope will run on Windows (most customers use it on Windows anyway) too: all it needs is Java (if needed, a private copy of Java will be installed on your computer as part of Omniscope package). You can get Omniscope Viewer on Linux workstation as well but if you need a full Omniscope 2.6 on Linux, you will have to ask Visokio about special license for you.
Java was the problem for me, when I first heard about Omniscope, but more about that in a Special note at the end of this post. Visokio is a tiny company, started in 2002. Because of its size and private funding it took 3 years to release Omniscope 1.0 in 2005 and another 4 years to release Omniscope 2.5 in 2009,
which is what Visokio currently is still shipping. Visokio obviously have rich customers in financial (13+ clients), publishing and marketing(10+), and many other industries and some of them in love with Apple's Macs, but most customers prefer Windows. Omniscope is a Desktop Java application but completely integrated with internet. It has 4 editions (in both 32-bit and 64-bits versions), which are identical as far a deployment file-set concern, so all you need is buy an appropriate license. The installation process requires about 5 clicks, and user can get started by simply dragging in an Excel file and data will immediately appear and can be explored organically.
Omniscope Editions: Viewer, Desktop, Server, Server Plus.
Free Viewer allows server-less distribution of all Data Visualizations and interact fully (explore, select, filter and drill-down among other interactions) with all data, charts and reports, which are all can be easily exported to PDF, PPT, XLS and JPG files. Omniscope has zero-install "Web Start online version of free Viewer.
Omniscope Desktop/Professional ($4000 with discount for volume orders) in addition to all Viewer functionality, acts as a Development Studio for Data Visualizations (so called IOK applications are secure and compressed files, ready for easy internet delivery) and as a ETL wizard (using Drag-and-Drop Data Manager) for data:
Omniscope Desktop creates, edits and continuously refreshes all involved datasets, formulas, filters, views, layouts, even assumption-driven models, designs and export interactive Flash Data Players, embeddable into websites and into documents. Desktop able to read multidimensional cubes, just like Tableau and PowerPivot, which is a big advantage over Qlikview and Spotfire.
Omniscope Server (about $16000) adds to Desktop functionality: enables 64-bit IOK files behave (even remotely) as Central Datamarts (multi-source data assembly), as Timeslices (auto-refreshable proxies for datasources: one per each datasource), as Master Report IOK (automatically refreshed from Central Datamart IOK) and as Distributed Report IOK(s) (automatically distributed and live-refreshed from Master Report IOK), automates the refreshing of data, enables batch and scheduled distribution of customized IOK files.
Server Plus (about $24000) includes all Server functionality and adds ability to empower selected actions in free Omniscope Viewers (e.g. continuous data refreshing from Datamart IOK files, export to XLS, PPT, PDF, add/edit/save comments and queries etc.), permits unrestricted publishing of IOK visualizations, enables white labeling and branding Viewers and IOK files to customers specifications, allows multiple servers work as one.
Data Engine.
Omniscope is using in-memory Columnar Database, as all best Data Visualizers do but its architecture is different. For example, all datasets are collection of Cells (organized in column, rows and tables). Each Cell with String or Text is a separate Java Object and it leads to a large overhead in terms of memory usage (I always blame Java, which allows only 1.2GB of addressable memory for 32-bit Windows). Some usage statistics prompting that 32-bit Omniscope Desktop/Professional thinks that 5 millions cells is a large dataset and 15 millions cells is a very large dataset. According to Visokio, average client data file is around 40 fields and 50,000 records (2 million cells).
With Omniscope 2.6, experts from Visokio was able to run on 32-bit Windows PC (with 2GB of RAM) the Data Visualization with 70 millions of cells. For comparison with Qlikview I was able to fit 600+ millions of (data) cells into the same 32-bit PC, basically 9 times more data then with Omniscope and overall Omniscope is slower then competitors. As of now, Omniscope will try to use as much memory as possible in order to accelerate performance. I expect in near future the version of Omniscope with large performance and memory management improvements.
64-bit Installations of Omniscope are far more scalable, for example with 8GB of RAM 120 millions of cells was not a problem; largest known installation of Omniscope has 34 million Rows (about half of billion of cells) running on 64-bit Windows/Java PC with 16GB of RAM
In Omniscope 2.6, the DataManager can be used as an entirely new and independent application, allowing you to create and automate ETL workflows, without even loading data into the classic Omniscope interface. You can visually drag sources in, append and merge, and transform with a variety of powerful operations such as Field Organiser which allows you to add formulas. You can then publish, including a Batch Publisher which allows you to specify commands in another IOK file, such as "Publish [this subset] to [email] using [this view template]", etc.
The original foundation of exportable Flash DataPlayer "generation" was totally re-written (for Omniscope 2.6) in ActionScript 3, which increased the scalability of DataPlayer and added new view types/features. DataPlayers available as an experimental feature in Omniscope 2.6, and fully feature-complete in Omniscope 2.7 (I personally think that the time for Flash is gone/over and it is time to port DataPlayers into HTML5).
Visokio is confident that Omniscope 2.7 will come soon after release of Omniscope 2.6 and it will be integrated with super-popular Open Source Statistical R Library, and hopefully will contain HTML5-based DataPlayer, integration with Salesforce etc. If customers will demand, I also expect the Linux version of Omniscope at some future point.
By the way, my recent Poll is confirming that Omniscope is among Data Visualization Leaders and it got respectable 6% of votes so far! You can vote on this poll, just click here!
Special Note about Java.
While Java gave Omniscope the unique ability to run everywhere, it also gave a performance disadvantage to it, compare with my favorites Qlikview, Spotfire, Tableau and PowerPivot (all 4 written as native Windows applications).
Teradata sounds good and smells like money, especially today. I already mentioned that they received U.S. Patent #7966340 on June 21, 2011. The patent is about SQL-MapReduce technology: the data analytic framework that combines the popular MapReduce™ software with the enterprise friendliness of SQL. (Also see article about "multi-structured data sources" from Aster Data).
Today Teradata Columnar is announced (available in December 2011 as a component of Teradata Database 14) and Teradata Database 14 is released. The new columnar capability from Teradata allows users to mix-and-match ("hybrid") columnar and row-based physical storage when it best suits an application. Teradata Columnar is integrated with the row-based storage and relational database software. Only the data in the columns required for a query are pulled into memory for processing, reducing the time-constraining input/output of a row-based approach that would read data from all the columns.
Teradata Columnar brings traditional "columnar" benefit: the flexible data compression. Teradata Columnar dynamically adjusts the compression mechanisms for optimal storage depends on type and size of data involved, automatically chooses from among six types of compression: run length, dictionary, trim, delta on mean, null and UTF8 based on the column demographics.
Again, these are just a good sound bites until Teradata Columnar will be released. Teradata may be trying to out-market Microsoft with its SQL Server 2011 (or Denali; as of today available as CTP3 community release) which already has the Columnstore Index, integrated with row-based storage and relational database.
I am wondering if Tableau will able timely and natively support Teradata Columnar as it supports now the Teradata Database (important for Data Visualization applications):
This is a guest post from, Marc Gedansky, a well-known sales and marketing consultant in the Business Intelligence space. Marc writes and speaks frequently on a variety of issues that influence technology providers and users, and is based in Cambridge, MA. I am fortunate to know Marc as Business Intelligence and Data Visualization expert and as my friend for many years.
Recently I noticed that internet (thanks to big data waves and to easy to use Data Visualization tools) is polluted with a lot of useless Dashboards and I spoke with Marc about this topic. Turned out he has a a very good explanation for it and he was kind enough to share his opinion on this blog as a guest blogger. Marc's post reminded me the old story:
"An admirer asked Michelangelo how he sculpted the famous statue of David that now sits in the Academia Gallery in Florence. How did he craft this masterpiece of form and beauty? Michelangelo’s offered this strikingly simple description: He first fixed his attention on the slab of raw marble. He studied it and then “chipped away all that wasn’t David.”
“Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.” - Antoine de Saint-Exupery
Most dashboards are designed with no clue as to the meaning and/or importance of this quote.
(BTW, even though this is a blog about data visualization, I (M.G.) won’t show any poorly designed dashboard examples, as they are ubiquitous. Trying to find them is about as difficult as trying to find leaves
on the ground in New England during the Fall).
I view dashboards every day; on software company sites, news sites, financial sites, and blogs. Since dashboards can distill so much information and display it in such a small space, they hold the potential of quickly delivering valuable insights; of cutting through the “data clutter” to immediately reveal important trends or truths.
So why then, are most dashboards crammed with so many charts, dials, and graphs that they overwhelm you? Just because you can fit a half-dozen on a screen, why is there a need to do it? (This approach reminds me of my friend Geoff, who, upon hearing that Hellmann’s was coming out with mayonnaise that had half the calories remarked, “great, now I can eat twice as much”.)
I think there can only be two reasons.
1. The designer/developer wants to show off their expertise with Qlikview, or Spotfire, or Tableau, or X product.
2. The designer/developer does not care about the average person, and wants to build smart software for brilliant users.
That attitude reminds me of a meeting I attended at a software company a few years ago. The head of development was upset because he was being asked to make his software “easy to use”. He called it “dumbing down”, and complained that it would be less challenging for his development team to build “software for idiots”. At this point, the President of the company interjected, “if our customers are smart enough to write us a check, then they are smart enough to use our software. And the onus for them to be able to use our software is on us, not on them.”
Update 9/27/11: TIBCO officially released Silver 2.0, see http://www.marketwatch.com/story/tibco-unveils-silver-spotfire-20-to-meet-growing-demand-for-easy-to-use-cloud-based-analytics-solutions-2011-09-27 "TIBCO Silver Spotfire 2.0 gives users the ability to embed live dashboards into their social media applications, including business blogs, online articles, tweets, and live feeds, all without complex development or corporate IT resources... Overall, the software's capabilities foster collaboration, which allows users to showcase and exchange ideas and insights -- either internally or publicly. In addition, it allows users to share solutions and application templates with customers, prospects, and other members of the community."
Spotfire Silver Personal Edition is Free (Trial for one year, can be "renewed" with other email address for free) and allows 50MB (exactly the same amount as Tableau Public) and allows 10 concurrent read-only web users of your content. If you wish more then Personal Edition you can buy Personal Plus ($99/year) or Publisher ($99/month or $1000/year) or Analyst ($399/month) Account.
In any case you will GET for your Account needs a real Spotfire Desktop Client and worry-free and hassle-free web hosting (by TIBCO) of your Data Visualization applications - you do not need to buy any hardware, software or services for web hosting, it is all part of your Spotfire Silver account.
To test Spotfire Silver 2.0 Personal Edition I took Adventure Works dataset from Microsoft (60398 rows, which is 6 times more than Spotfire's own estimate of 10000 rows for 50MB Web storage). Adventure Works dataset requires 42MB as Excel XLS file (or 16M as XLSX with data compression) and only 5.6MB as Spotfire DXP file (Tableau file took approximately the same disk space, because both Spotfire and Tableau are doing a good data compression job). This 5.6MB size of DXP file for Adventure Works is just 11% of web storage allowed by Spotfire (50MB for Personal Edition) to each user of free Spotfire Silver 2.0 Personal Edition.
Spotfire Silver 2.0 is a very good and mature Data Visualization product with excellent Web Client, with Desktop Client development tool and with tutorials online here: https://silverspotfire.tibco.com/us/tutorials . Functionally (and Data Visualization-wise) Spotfire Silver 2.0 has more to offer then Tableau Public. However Tableau Public account will not expire after 1 year of "trial" and will not restrict number of simultaneous users to 10.
Spotfire Silver 2.0 Publisher and Analyst Accounts can compete successfully with Tableau Digital and they have much clear licensing then Tableau Digital (see http://www.tableausoftware.com/products/digital#top-10-features-of-tableau-digital ), which is based on number of "impressions" and can be confusing and more expensive then Spotfire Silver Analyst Edition.
7 months ago I published a poll on LinkedIn and got a lot of responses, 1340 votes (in average 1 vote per hour) and comments. People asked me many times to repeat this poll from time to time. I guess it is time to re-Poll. I added 2 more choices (LinkedIn allows maximum 5 choices in their polls and it is clear not enough for this poll), based on a feedback I got: Omniscope and Visual Insight/Microstrategy. I also got some angry voters complaining that certain vendors are funding this poll. This is completely FALSE, I am unaffiliated with any of vendors, mentioned in this poll and I am working for completely independent (from those vendors) software company, see the About page of this Blog.
Today Tableau 6.1 is released (and client for iPad and Tableau Public for iPad), that includes the full support for incremental Data updates whether they are scheduled or on demand:
New in Tableau 6.1
Incremental Data updates scheduled or on demand
Text parser faster, can parse any text files as data source (no 4GB limit)
Files larger than 2GB can now be published to Tableau Server (more "big data" support)
Impersonation for SQL Server and Teradata; 4 times faster Teradata reading
Tableau Server auto-enables touch, pinch, zoom, gesture UI for Data Views
Tableau iPad app is released, it browses and filters a content on Server
Any Tableau Client sees Server-Published View: web browser, mobile Safari, iPad
Server enforces the same (data and user) security on desktop, browser, iPad
Straight links from an image on a dashboard, Control of Legend Layout etc.
Here is a Quick demo of how to create Data Visualization with Tableau 6.1 Desktop, how easy to publish it on Tableau server 6.1 and how it is instantly visible, accessible and touch optimized on the iPad:
New since Tableau 6.0, more then 60 features, including:
Tableau now has in-memory Data Engine, which greatly improves I/O speed
Support for "big" data
Data blending from multiple sources
Unique support for local PowerPivot Multidimensional Cubes as Data Source
Support for Azure Datamarket and OData (Open Data Protocol) as Data Sources
Support for parameters in Calculations
Motion Charts and Traces (Mark History)
In average 8 times faster of rendering of Data Views (compare with previous version)
Tableau Product Family
Desktop: Personal ($999), Professional ($1999), Digital, Public.
Server: Standard, Core Edition, Digital, Public Edition.
Free Client: Web Browser, Desktop/Offline Tableau Reader.
Free Tableau Reader enables Server-less distribution of Visualizations!
Free Tableau Public served 20+ millions visitors since inception
Tableau Server
Easy to install: 13 minutes + optional 10 minutes for firewall configuration
Tableau has useful command line tools for administration and remote management
Scalability: Tableau Server can run (while load balancing) on multiple machines
Straightforward licensing for Standard Server (min 10 users, $1000/user)
With Core Edition Server License: unlimited number of users, no need for User Login
Digital Server Licensing based on impressions/month, allows unlimited data, Tableau-hosted.
Public Server License: Free, limited (100000 rows from flat files) data, hosted by Tableau.
Widest (and Tableau optimized) Native Support for data sources
Microsoft SSAS and PowerPivot: Excel Add-in for PowerPivot, native SSAS support
Native support for Microsoft SQL Server, Access, Excel, Azure Marketplace DataMarket
Other Enterprise DBMSes: Oracle, IBM DB2, Oracle Essbase
Analytical DBMSes: Vertica, Sybase IQ, ParAccel, Teradata, Aster Data nCluster
Database appliances: EMC/GreenPlum, IBM/Netezza
Many Popular Data Sources: MySQL, PostgreSQL, Firebird, ODBC, OData, Text files etc.
Some old problems I still have with Tableau
No MDI support in Dashboards, all charts share the same window and paint area
Wrong User Interface (compare with Qlikview UI) for Drilldown Functionality
Tableau's approach to Partners is from stone ages
Tableau is 2 generations behind Spotfire in terms of API, Modeling and Analytics
Microsoft finally released SQL Server 11 "Denali" as CTP3 (Community Technology Preview) for public ... Preview. Microsoft is (these are politeness words I can type) stubbornly refusing to have/build own Data Visualization Product. I doubt Crescent "experience" can be considered as a product, especially because it is Silverlight-base, while world already moved to HTML5.
If you have 7 minutes, you can watch Crescent Demo from WPC11, which is showing that while trailing a few years behind DV Leaders and Google, Microsoft is giving to its die hard followers something to cheer about:
I have to admit, that while there is nothing new (for DV expert) in video above, it is a huge progress compare with Excel-based Data Visualizations, which Microsoft tried to promote as a replacement of ProClarity and PerformancePoint Server. Even Microsoft itself positions Crescent (which is 32-bit only!) as a replacement for SSRS Report Builder, so DV Leaders can sleep well another night.
However, Microsoft's BI Stack is the number 4 or 5 on my list of DV Leaders and CTP3 is so rich with new cool functionality, that it deserves to be covered on this blog.
Of course major news is availability of Tabular Data Model, which means VertiPaq in-memory columnar Engine, similar to PowerPivot Engine but running on Server without any SharePoint (which is a slow virus, as far as I am concerned) and without stupid SharePoint UI and limitations and I quote Microsoft: " In contrast with the previous release, where VertiPaq was only available via in PowerPivot for SharePoint, you can now use VertiPaq on a standalone Analysis Services instance with no dependency on SharePoint."!
SSAS (SQL Server Analysis Services) has new (they may existed before, but before CTP3 - ALL who knew that were under NDA) features like memory paging (allows models to be larger than the physical memory of the server, means unlimited scalability and BIG Data support), row level security (user identity used to hide/show visible data), KPI, Partitions; CTP3 removes the maximum 4GB file size limit for string storage file, removes the limit of 2 billion rows per table (each column is still limited to a maximum of 2 billion distinct values, but in columnar database it is much more tolerable restriction!).
New version of PowerPivot is released with support of Tabular Model and I quote: "You can use this version of the add-in to author and publish PowerPivot workbooks from Excel 2010 to Microsoft SQL Server" and it means no SharePoint involvement again! As Marco Russo put it: "Import your existing PowerPivot workbooks in a Tabular project (yes, you can!)" and I agreed 100% with Marco when he said 4 times: Learn DAX!
After 3 years of delays, Microsoft is finally has BIDS for Visual Studio 2010 and that is huge too, I quote again: "The Tabular Model Designer ... is now integrated with Microsoft SQL Server “Denali” (CTP 3) Business Intelligence Development Studio." It means that BIDS now is not just available but is the main unified development interface for both Multidimensional and Tabular Data Models. Now we can forget about Visual Studio 2008 and finally use more modern VS2010!
Another extremely important for Data Visualization feature is not in SSAS but in SQL Server itself: Columnstore index is finally released and I a quote 1 more time again: "The ... SQL Server (CTP 3) introduces a new data warehouse query acceleration feature based on a new type of index called the columnstore. This new index ... improves DW query performance by hundreds to thousands of times in some cases, and can routinely give a tenfold speedup for a broad range of decision support queries... columnstore indexes limit or eliminate the need to rely on pre-built aggregates, including user-defined summary tables, and indexed (materialized) views. Furthermore, columnstore indexes can greatly improve ROLAP performance" (ROLAP can be used for real-time Cubes and real-time Data Visualizations).
All these cool SQL Server 11 new stuff is coming soon into Azure Cloud and this can be scary for any DV vendor, unless it knows (Tableau does; Qliktech and Spotfire still ignore SSAS) how to be friendly with Microsoft.
As we know now the newly coined by Microsoft term BISM (Business Intelligence Semantic Model) was a marketing attempt to have a "unified" umbrella
for 2 different Data Models and Data Engines: Multidimensional Cubes (invented by Mosha Pasumansky 15 years ago and the foundation for SSAS and MDX - SQL Server Analysis Services) and Tabular Model (used in PowerPivot and VertiPaq in-memory columnar Database with new DAX Language which is going to be very important for future Data Visualization projects).
New CTP3-released BIDS 2010 (finally almighty Visual Studio 2010 will have a "Business Intelligence Development Studio" after 3+ years of unjustified delays!) UI-wise will able to handle these 2 Data Models, but it is giving me a clue why Mosha left Microsoft for Google. And lack of DV product is a clue for me why Donald Farmer (face of Microsoft BI) left Microsoft for Qliktech.
Even more: if you need both Data Models to be present, you need to install 2 (TWO!) different instances of "Analysis Services": one with Multidimensional Engine and one with new Tabular (VertiPaq/PowerPivot) Engine. It seems to me not as ONE "BI" architecture but TWO "BI" Architectures, interface-glued on Surface by BIDS 2010 and on back-end by all kind of Data Connectors. Basically Microsoft is in confused BI state now because financially it can afford 2 BI Architectures and NO Data Visualization Product!
I cannot believe I am saying this, but I wish Bill Gates back from retirement (it will be good for Microsoft shares and good for Microsoft market capitalization too - just ask Apple's shareholders about Steve and they will say he is a god)!
In last few days something (3 news covered here in one post below) important for the future of Data Visualization and Big Data Analytics happened. IBM recently had 100th Birthday and almost at the same time their engineers published new invention, based on PCM (Phase-Change Memory).
PCM will not lose data when when power is turned off.
PCM 100 times faster (10 microseconds latency!) then flash and HDD.
PCM can endure at least 10 million write cycles (Flash maxed-out @30000)
PCM is cheap, has huge capacity and will be mass-produced before 2016.
PCM can be used everywhere from huge servers to smartphones
This invention is changing the approach to how to store and access "Big Data" and what portion of "Big Data" need to be in-memory (RAM) for Data Visualization purposes as oppose to outside of RAM (say on hard disk, flash or PCM). IBM may have a keys to Big Data kingdom...
To some people it may be unrelated, but not to me: Teradata just got the Patent on SQL-MapReduce technology they got from Aster Data acquisition. This technology allows also to integrate with Apache Hadoop and derived database systems, used in many Big Data applications.
And last but not least is a recent acknowledgment (for some reason it came from India's branch of IBM Software and I am wondering why, but finally it came "Straight from the horse's mouth"! ) from IBM that Data Visualization is the future of Business Intelligence (I said THIS many years ago and still repeating it from time to time: DV is new BI or in other words: the BI is dead, all hails to DV!). IBM is very proudly saying that Cognos 10 supports "enormous" number of Charts (I guess it will make Qlikview, Spotfire and Tableau people laughing)
and that the most discussed feature in Cognos 10 is Active Reports. This functionality allows the report authors to create interactive reports (apparently it is a big deal for IBM!).
IBM even is spreading rumors for weeks (through people who signed NDA with them) about Cognos TM1-based "new visualization tool", which will "disrupt" DV market... I guess because IBM knows that BI is dead (and IBM wasted $14+B buying 24 BI companies lately) and DV is new BI.
Since IBM improved PCM (see above) and had 100th birthday, I really wish good luck to them, but I wish IBM to stay focused on what they good at instead of spreading all over the high-tech. All these 3 "news" were published yesterday and today and somehow connected in my mind to Data Visualization's future and forced me to publish this "eclectic" post...
As I said many times, BI is just a marketing umbrella for multiple products and technologies and Data Visualization became recently as one of the most important among those. Data Visualization (DV) so far is a very focused technology and article below shows how to publish Excel Data Visualizations and Dashboards on Web. Actually a few Vendors providing tools to publish Excel-based Dashboards on Web, including Microsoft, Google, Zoho, Pagos and 4+ other vendors:
I leave to the reader to decide if other vendors can compete in business of publishing Excel-based Dashbaords on Web, but the author of the artcile below provides a very good 3 criterias of how to select the vendor, tool and technology for it (and when I used it myself it left me only with 2 choices - the same as described in article).
Author: Ugur Kadakal, Ph.D., CEO and founder of Pagos, Inc.
In previous article (see "Excel as BI Platform" here) I discussed Excel’s use as a Business Intelligence platform and why it is exceedingly popular software among business users. In 2nd article ("Dos&Don’ts of Building Successful Dashboards in Excel") I talked about some of the principles to follow when building a dashboard or a report in Excel. Together this is a discussion of why Excel is the most powerful self-service BI platform.
However, one of the most important facets of any BI platform is web enablement and collaboration. It is important for business users to be able to create their own dashboards but it is equally important for them to be able to distribute those dashboards securely over the web. In this article, I will discuss two technologies that enable business users to publish and distribute their Excel based dashboards over the web.
Selection Criteria
The following criteria were selected in order to compare the products:
Ability to convert a workbook with most Excel-supported features into a web based application with little to no programming.
Dashboard management, security and access control capabilities that can be handled by business users.
On-premise, server-based deployment options.
Criteria #3 eliminates online spreadsheet products such as Google Docs or Zoho. As much as I support cloud based technologies, in order for a BI product to be successful it should have on-premise deployment options. Without on-premise you neglect the possibility of integration with other data sources within an organization.
There are other web based Excel conversion products on the market but none of them meet the criteria of supporting most Excel features relevant to BI; therefore, they were not included in this article about how to publish Excel Dashboard on Web .
Below is a Part 2 of the Guest Post by my guest blogger Dr. Kadakal, (CEO of Pagos, Inc.). This article is about of how to build Dashboards and Data Visualizations with Excel. The topic is large, and the first portion of article (published on this blog last week) contains the the general Introduction and the Part 1 “Use of Excel as a BI Platform Today“.
The Part 2 – “Dos and Don’ts of building dashboards in Excel“ is below and Part 3 – “Publishing Excel dashboards to the Internet“ is coming soon. It is easy to fall into a trap with Excel, but if you avoid those risks as described in article below, Excel can become of one of the valuable BI and Data Visualization (DV) tool for user. Dr. Kadakal said to me recently: "if the user doesn't know what he is doing he may end up spending lots of time maintaining the file or create unnecessary calculation errors". So we (Dr. Kadakal and me) hope that article below can save time for visitors of this blog.
BI in my mind is a marketing umbrella for multiple products and technologies, including RDBMS, Data Collection, ETL, DW, Reporting, Multidimensional Cubes, OLAP, Columnar and in-Memory Databases, Predictive and Visual Analytics, Modeling and DV.
Data Visualization (aka DV), on other hand, is a technology, which enabling people to explore, drill-down, visually analyze their data and visually search for data patterns, like trends, clusters, outliers, etc. So BI is marketing super-abused term, while DV so far is focused technology and article below shows how to use Excel as a great Dashboard builder and Data Visualization tool.
Dos&Don’ts of Building Successful Dashboards in Excel
In previous week's post (see also article "Excel as BI Platform" here) I discussed Excel’s use as a Business Intelligence platform and why it is exceedingly popular software among business users. In this article I will talk about some of the principles to follow when building a dashboard or a report in Excel.
One of the greatest advantages of Excel is its flexibility: it puts little or no constraints on the user’s ability to create their ideal dashboard environments. As a result, Excel is being used as a platform for solving practically any business challenge. You will find individuals using Excel to solve a number of business-specific challenges in practically any organization or industry. This makes Excel the ultimate business software.
On the other hand, this same flexibility can lead to errors and long term maintenance issues if not handled properly. There are no constraints on data separation, business logic or the creation of a user interface. Inexperienced users tend to build their Excel files by mixing them up. When these facets of a spreadsheet are not properly separated, it becomes much harder to maintain those workbooks and they become prone to errors.
In this article, I will discuss how you can build successful dashboards and reports by separating data, calculations and the user interface. The rest of this post you can find in this article
It discusses how to prepare Data (both static and external) for dashboards, how to build formulas and calculation models, UI and Input Controls for Dashboards and of course - Pivots,Charts, Sparklines and Conditional Formatting for innovative and powerful Data Visualizations in Excel.
