Integrate Blockchain Analytics into Business Intelligence Tools

Featured image: Custom SQL query in Tableau

Data forms a crucial part of all business activities today. All the decisions are increasingly being based on metrics rather than emotions or instinct alone. However, it is imperative for the data to be presented in a form that can be translated into actionable decisions by the executives, managers, product development teams and all the other entities running an enterprise.

What is business intelligence and analytics?

Popular BI software comes from specialized companies such as Tableau or Qlik, or from large companies like IBMs Cognos Analytics, Oracle BI, and Microsoft Power BI to name a few.

With the rise of enterprise blockchains, many of the business processes which were previously done on paper can now be done on decentralized ledgers. Blockchain technology is not only used by firms dealing with cryptocurrency investment and financing but also insurance agencies, banking and other FinTech services, logistics companies handling large scale supply chains, and even electronic certificate or document verification services.

As a result of this further digitization of business processes through blockchain, the volume of data available for analysis will increase profoundly.

Blockchain and business analytics

Given these benefits of well-analyzed and communicated data on the status of a company, it is not surprising that a Forbes report has suggested that the proportion of businesses using data analytics went up from 17% in 2015 to 59% in 2018.

Image: fizzy flight insurance Metabase dashboard

What is the purpose of Business Intelligence in a business?

IBM Cognos Analysis studio has been one such online analytical processing tool. Some of the key components of this and many other tools like Tableau and Qlik are the roll-up and drill-down features, as well as several data management and visualization tools. Dashboarding and visualizations are the most popular uses of Business Intelligence as they help in presenting data for both data literate as well as the lay person. They also help to organize large data sets in manageable and accessible forms. Tools like Tableau, Cliq, and Datawrapper provide very sophisticated visualization options and can connect and integrate with many different types of data sources.

Why is Blockchain Data difficult to work with?

Blockchain works on the principle of a public ledger. It is a decentralized network that records data in chunks called blocks which continuously reference the previous block (hence the name “blockchain”). This data is then stored redundantly over all the nodes, i.e. all the participating computers on the network. In this system, the data is usually stored in the form of transaction details and time stamps, as in the case of the Bitcoin blockchain. However, for many other blockchains, especially the Ethereum blockchain, there are smart contracts that enable the storage of data through logical patterns. These so called smart contracts in Ethereum run on the Ethereum Virtual Machine, which is a compiler provided by Ethereum so that these contracts can be executed on all machines.

When a function in the contract code is called, an event is emitted from the code. In this way, the smart contracts on the blockchains generate logs, which are stored hex-encoded for efficiency reasons. Logs can be used for both returning values to the user interface or as asynchronous triggers for some action that needs to be performed by the contract. The data fields in the function calls are essential for understanding how the code is functioning. Thus, a very tedious process of decoding is involved. To add to the complications, the data is entirely in hexadecimal form without any word labels. Moreover, the data itself is serialized, that is to say, for any single data point much more queries will have to be run compared to a centralized system. All these factors cumulatively make it a very daunting affair.

Furthermore, to read time-series blockchain data on smart contracts, it is necessary to set up an archive node, as regular nodes go back only a few weeks. Archive nodes are difficult to operate and maintain, are expensive due to storage requirements and take weeks-long sync time for the full blockchain history. You will have to query it by using some API or in the case of Ethereum via (JSON) RPC, which works prohibitive slow for many analytic queries.

Integrating blockchain data into BI, therefore, essentially involves converting the data into a human-readable format. This has to be done through some kind of matching tools to combine the raw data with text labels and decoded values. Interestingly, all these processes can be automated and be carried out through the use of the Anyblock Index technology that transforms data into a standard enterprise format, namely an Elasticsearch API and a SQL database (PostgreSQL). This not only makes queries easy and lightning-fast but also helps BI tools to aggregate and visualize data.

Anyblock Analytics: Bridging the Gap between Blockchain Data and BI

The first way in which this system can help is by allowing companies to work with blockchain data with their existing BI tools that are already used for other purposes. Simply put, companies can use our Anyblock Index, which will store the blockchain data and that is then connected as a regular data source by the BI tools. These tools will also help select, aggregate and visualize the data.

Our Anyblock Index uses the standard interfaces such as an ElasticSearch API and direct PostgreSQL database access. This makes it widely compatible with all kinds of data analytics tools like Qlik or Tableau.

Our system also integrates with other visualization tools such as Metabase and Grafana, which can help create comprehensive visual representations of the data gathered from the blockchain. These can also be hosted by Anyblock.

Image: Exemplary insight into Metabase dashboard creation

As we have already established the necessity of using creatively communicated and simplified forms of data, visualization tools can be the best medium for such communication. While tables are complex and cumbersome to interpret, converting them to charts or other graphical formats is therefore essential. These tools also allow the visualizations to be interactive. This lets the consumer filter the dashboards according to his individual needs and thus allows to communicate data more personalized.

All of this makes the process of integrating blockchain data with BI tools efficient and most importantly, leads to actionable insights, which after all is one of the primary purposes of collecting the data in the first place. Thus, to be able to provide a blockchain data solution for users of popular tools like Tableau or Qlik without any need to change the software and without additional costs, is the achievement of this Anyblock technology.

And importantly for the enterprise use cases, the Anyblock Index can be used not only for large public blockchains like Ethereum, but such indexes can be created for smaller private enterprise blockchains as well. This virtual appliance solution was already installed on-premise in a German bank IT network that is part of a financial blockchain consortium.

If you don’t have the necessary resources available in-house, the data science team at Anyblock Analytics can also help you to create the most useful professionally created interactive dashboards, to make the data as personally actionable as possible unlike other bland dashboard formats that lack our signature filters.

Freddy Zwanzger — Co-Founder of Anyblock

Interested or questions?

Follow us:
Twitter, LinkedIn,
Medium, YouTube

Originally published at https://www.anyblockanalytics.com on March 19, 2020.

Anyblock Analytics is a German blockchain solution provider. We offer consulting, tools and data to integrate business processes with blockchain.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store