Omniscope 2.6 (wait is over!)

Do you want the 1st class Data Visualization on your cool Mac without any Virtual Machine with Windows? If so, your best choice will be the Omniscope 2.6 which is finally about to be released (after more then 2 years of delays) by Visokio, located in UK. Of course the Omniscope will run on Windows (most customers use it on Windows anyway) too: all it needs is Java (if needed, a private copy of Java will be installed on your computer as part of Omniscope package). You can get Omniscope Viewer on Linux workstation as well but if you need a full Omniscope 2.6 on Linux, you will have to ask Visokio about special license for you.



Java  was the problem for me, when I first heard about Omniscope, but more about that in a Special note at the end of this post. Visokio is a tiny company, started in 2002. Because of its size and private funding it took 3 years to release Omniscope 1.0 in 2005 and another 4 years to release Omniscope 2.5 in 2009,



which is what Visokio currently is still shipping. Visokio obviously have rich customers in financial (13+ clients), publishing and marketing(10+), and many other  industries and some of them in love with Apple's Macs, but most customers prefer Windows. Omniscope is a Desktop Java application but completely integrated with internet. It has 4 editions (in both 32-bit and 64-bits versions), which are identical as far a deployment file-set concern, so all you need is buy an appropriate license. The installation process requires about 5 clicks, and user can get started by simply dragging in an Excel file and data will immediately appear and can be explored organically.

Omniscope Editions: Viewer, Desktop, Server, Server Plus.

Free Viewer allows server-less distribution of all Data Visualizations and interact fully (explore, select, filter and drill-down among other interactions) with all data, charts and reports, which are all can be easily exported to PDF, PPT, XLS and JPG files. Omniscope has zero-install "Web Start online version of free Viewer.

Omniscope Desktop/Professional ($4000 with discount for volume orders) in addition to all Viewer functionality, acts as a Development Studio for Data Visualizations (so called IOK applications are secure and compressed files, ready for easy internet delivery) and as a ETL wizard (using Drag-and-Drop Data Manager) for data:



Omniscope Desktop creates, edits and continuously refreshes all involved datasets, formulas, filters, views, layouts, even assumption-driven models, designs and export interactive Flash Data Players, embeddable into websites and into documents. Desktop able to read multidimensional cubes, just like Tableau and PowerPivot, which is a big advantage over Qlikview and Spotfire.

Omniscope Server (about $16000) adds to Desktop functionality: enables 64-bit IOK files behave (even remotely) as Central Datamarts (multi-source data assembly), as Timeslices (auto-refreshable proxies for datasources: one per each datasource), as Master Report IOK (automatically refreshed from Central Datamart IOK) and as Distributed Report IOK(s) (automatically distributed and live-refreshed from Master Report IOK), automates the refreshing of data, enables batch and scheduled distribution of customized IOK files.

Server Plus (about $24000) includes all Server functionality and adds ability to empower selected actions in free Omniscope Viewers (e.g. continuous data refreshing from Datamart IOK files, export to XLS, PPT, PDF, add/edit/save comments and queries etc.), permits unrestricted publishing of IOK visualizations, enables white labeling and branding Viewers and IOK files to customers specifications, allows multiple servers work as one.

Data Engine.

Omniscope is using in-memory Columnar Database, as all best Data Visualizers do but its architecture is different. For example, all datasets are collection of Cells (organized in column, rows and tables). Each Cell with String or Text is a separate Java Object and it leads to a large overhead in terms of memory usage (I always blame Java, which allows only 1.2GB of addressable memory for 32-bit Windows). Some usage statistics prompting that 32-bit Omniscope Desktop/Professional thinks that 5 millions cells is a large dataset and 15 millions cells is a very large dataset. According to Visokio, average client data file is around 40 fields and 50,000 records (2 million cells).

With Omniscope 2.6, experts from Visokio was able to run on 32-bit Windows PC (with 2GB of RAM) the Data Visualization with 70 millions of cells. For comparison with Qlikview I was able to fit 600+ millions of (data) cells into the same 32-bit PC, basically 9 times more data then with Omniscope and overall Omniscope is slower then competitors. As of now, Omniscope will try to use as much memory as possible in order to accelerate performance. I expect in near future the version of Omniscope with large performance and memory management improvements.

64-bit Installations of Omniscope are far more scalable, for example with 8GB of RAM 120 millions of cells was not a problem; largest known installation of Omniscope has 34 million Rows (about half of billion of cells) running on 64-bit Windows/Java PC with 16GB of RAM

In Omniscope 2.6, the DataManager can be used as an entirely new and independent application, allowing you to create and automate ETL workflows, without even loading data into the classic Omniscope interface.  You can visually drag sources in, append and merge, and transform with a variety of powerful operations such as Field Organiser which allows you to add formulas.  You can then publish, including a Batch Publisher which allows you to specify commands in another IOK file, such as "Publish [this subset] to [email] using [this view template]", etc.

For full list of Omniscope features please check this: http://www.visokio.com/omniscope-features and for new features in version 2.6 please review this: http://www.visokio.com/omniscope-new-in-2-6 .

The original foundation of exportable Flash DataPlayer "generation" was totally re-written (for Omniscope 2.6) in ActionScript 3, which increased the scalability of DataPlayer  and added new view types/features. DataPlayers available as an experimental feature in Omniscope 2.6, and fully feature-complete in Omniscope 2.7 (I personally think that the time for Flash is gone/over and it is time to port DataPlayers into HTML5).

Visokio is confident that Omniscope 2.7 will come soon after release of Omniscope 2.6 and it will be integrated with super-popular Open Source Statistical R Library, and hopefully will contain HTML5-based DataPlayer, integration with Salesforce etc. If customers will demand, I also expect the Linux version of Omniscope at some future point.

By the way, my recent Poll is confirming that Omniscope is among Data Visualization Leaders and it got respectable 6% of votes so far! You can vote on this poll, just click here!

Special Note about Java.

While Java gave Omniscope the unique ability to run everywhere, it also gave a performance disadvantage to it, compare with my favorites Qlikview, Spotfire, Tableau and PowerPivot (all 4 written as native Windows applications).

Teradata Columnar sounds good

Teradata sounds good and smells like money, especially today. I already mentioned that they received U.S. Patent #7966340 on June 21, 2011. The patent is about SQL-MapReduce technology: the data analytic framework that combines the popular MapReduce™ software with the enterprise friendliness of SQL. (Also see article about "multi-structured data sources" from Aster  Data).

Today Teradata Columnar is announced (available in December 2011 as a component of Teradata Database 14) and Teradata Database 14 is released. The new columnar capability from Teradata allows users to mix-and-match ("hybrid") columnar and row-based physical storage when it best suits an application. Teradata Columnar is integrated with the row-based storage and relational database software. Only the data in the columns required for a query are pulled into memory for processing, reducing the time-constraining input/output of a row-based approach that would read data from all the columns.

Teradata Columnar brings traditional "columnar" benefit: the flexible data compression. Teradata Columnar dynamically adjusts the compression mechanisms for optimal storage depends on type and size of data involved, automatically chooses from among six types of compression: run length, dictionary, trim, delta on mean, null and UTF8 based on the column demographics.

Again, these are just a good sound bites until Teradata Columnar will be released. Teradata may be trying to out-market Microsoft with its SQL Server 2011 (or Denali; as of today available as CTP3 community release) which already has the Columnstore Index, integrated with row-based storage and relational database.

I am wondering if Tableau will able timely and natively support Teradata Columnar as it supports now the Teradata Database (important for Data Visualization applications):

Proliferation of useless Dashboards

This is a guest post from, Marc Gedansky, a well-known sales and marketing consultant in the Business Intelligence space.  Marc writes and speaks frequently on a variety of issues that influence technology providers and users, and is based in Cambridge, MA. I am fortunate to know Marc as Business Intelligence and Data Visualization expert and as my friend for many years.

Recently I noticed that internet (thanks to big data waves and to easy to use Data Visualization tools) is polluted with a lot of useless Dashboards and I spoke with Marc about this topic. Turned out he has a a very good explanation for it and he was kind enough to share his opinion on this blog as a guest blogger. Marc's post reminded me the old story:

"An admirer asked Michelangelo how he sculpted the famous statue of David that now sits in the Academia Gallery in Florence. How did he craft this masterpiece of form and beauty? Michelangelo’s offered this strikingly simple description: He first fixed his attention on the slab of raw marble. He studied it and then “chipped away all that wasn’t David.”

________________________________________

Dashboards – why are so many useless?

Marc Gedansky, http://marc1717.blogspot.com/

“Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.” - Antoine de Saint-Exupery

Most dashboards are designed with no clue as to the meaning and/or importance of this quote.

(BTW, even though this is a blog about data visualization, I (M.G.) won’t show any poorly designed dashboard examples, as they are ubiquitous.  Trying to find them is about as difficult as trying to find leaves



on the ground in New England during the Fall).

I view dashboards every day; on software company sites, news sites, financial sites, and blogs.  Since dashboards can distill so much information and display it in such a small space, they hold the potential of quickly delivering valuable insights; of cutting through the “data clutter” to immediately reveal important trends or truths.

So why then, are most dashboards crammed with so many charts, dials, and graphs that they overwhelm you?  Just because you can fit a half-dozen on a screen, why is there a need to do it?  (This approach reminds me of my friend Geoff, who, upon hearing that Hellmann’s was coming out with mayonnaise that had half the calories remarked, “great, now I can eat twice as much”.)

I think there can only be two reasons.

1. The designer/developer wants to show off their expertise with Qlikview, or Spotfire, or Tableau, or X product.

2. The designer/developer does not care about the average person, and wants to build smart software for brilliant users.

That attitude reminds me of a meeting I attended at a software company a few years ago.  The head of development was upset because he was being asked to make his software “easy to use”.    He called it “dumbing down”, and complained that it would be less challenging for his development team to build “software for idiots”.  At this point, the President of the company interjected, “if our customers are smart enough to write us a check, then they are smart enough to use our software.  And the onus for them to be able to use our software is on us, not on them.”

For Continuation of this post please see it on this blog's page: http://apandre.wordpress.com/dataviews/dashboard/proliferation-of-useless-dashboards/

Old Excel Online Chart Templates with 12-Color Scheme

">

Spotfire Silver 2.0

Spotfire Silver version 2.0 is available now on https://silverspotfire.tibco.com/us/home and it will be officially announced at TIBCO User Conference 2011 (9/27-9/29/11) at http://tucon.tibco.com/

Spotfire Silver available in 4 Editions, see Product Comparison Chart here: https://silverspotfire.tibco.com/us/product-comparison-chart and Feature List at Feature Matrix here: https://silverspotfire.tibco.com/us/get-spotfire/feature-matrix

Update 9/27/11: TIBCO officially released Silver 2.0, see http://www.marketwatch.com/story/tibco-unveils-silver-spotfire-20-to-meet-growing-demand-for-easy-to-use-cloud-based-analytics-solutions-2011-09-27 "TIBCO Silver Spotfire 2.0 gives users the ability to embed live dashboards into their social media applications, including business blogs, online articles, tweets, and live feeds, all without complex development or corporate IT resources... Overall, the software's capabilities foster collaboration, which allows users to showcase and exchange ideas and insights -- either internally or publicly. In addition, it allows users to share solutions and application templates with customers, prospects, and other members of the community."

Spotfire Silver Personal Edition is Free (Trial for one year, can be "renewed" with other email address for free) and allows 50MB (exactly the same amount as Tableau Public) and allows 10 concurrent read-only web users of your content. If you wish more then Personal Edition you can buy Personal Plus ($99/year) or Publisher ($99/month or $1000/year) or Analyst ($399/month) Account.

In any case you will GET for your Account needs a real Spotfire Desktop Client and worry-free and hassle-free web hosting (by TIBCO) of your Data Visualization applications - you do not need to buy any hardware,  software or services for web hosting, it is all part of your Spotfire Silver account.

To test Spotfire Silver 2.0 Personal Edition I took Adventure Works dataset from Microsoft (60398 rows, which is 6 times more than Spotfire's own estimate of 10000 rows for 50MB Web storage). Adventure Works dataset  requires 42MB as Excel XLS file (or 16M as XLSX with data compression) and only 5.6MB as Spotfire DXP file (Tableau file took approximately the same disk space, because both Spotfire and Tableau are doing a good data compression job). This 5.6MB size of DXP file for Adventure Works is just 11% of web storage allowed by Spotfire (50MB for Personal Edition) to each user of free Spotfire Silver 2.0 Personal Edition.

Spotfire Silver 2.0 is a very good and mature Data Visualization product with excellent Web Client, with Desktop Client development tool and with tutorials online here: https://silverspotfire.tibco.com/us/tutorials . Functionally (and Data Visualization-wise) Spotfire Silver 2.0 has more to offer then Tableau Public. However Tableau Public account will not expire after 1 year of "trial" and will not restrict number of simultaneous users to 10.

Spotfire Silver 2.0 Publisher and Analyst Accounts can compete successfully with Tableau Digital and they have much clear licensing then Tableau Digital (see http://www.tableausoftware.com/products/digital#top-10-features-of-tableau-digital ), which is based on number of "impressions" and can be confusing and more expensive then Spotfire Silver Analyst Edition.