Deep dive into Qlik Cloud And learn more about Connections
- by Michiel Hofsteenge
"Qlik Cloud saved my organization!" Of course, that is one of the best compliments you can get as an organization from a customer. Why this customer is so enthusiastic, I like to explain in my 7-part blog about the possibilities and content of Qlik Cloud. My name is Michiel Hofsteenge, Tech Lead Data Analytics at Cmotions and Qlik Cloud fanatic.
In my third blog in the Deep Dive into Qlik Cloud series, I take a closer look at Connections. Connections to get data to the cloud and connections to get data from various source systems. In short, an interesting and indispensable topic when talking about Qlik Cloud. So sit back and enjoy a piece of free knowledge.
Connections to Qlik
Making connections is perhaps one of the main pain points during configuration when it comes to getting data to the cloud. Here we come to a playing field where various parties have set up to make sense of this. And rightly so! After all, business or privacy-sensitive information is going across the Internet. The help of someone who understands your own environment is then often desirable. There are quite a few techniques available to get data into the cloud.
When it comes to moving (and using) data, it will need to be clear what the purpose of this data is. I am often asked how real-time the data appears on the screen. It all depends on the choices that are made here. I can imagine that data on the occupancy of beds in a hospital would preferably be updated every 5 minutes, but that turnover figures can also wait for a day. The question is what added value (near) real-time data has within the processes for which it is intended.
In addition to making data connections, there are also connections that can be made to various other systems. Most organizations like to work with single sign on. It is also necessary to set up a mail connection to make it possible to subscribe to visualizations, among other things. I will briefly discuss these options as well.
Qlik Data Movement
The first connection variant goes by various names. Recently, it went through another name change to Data Movement. But you can also know it as Data Services or Data integration. Despite all these name changes, it remains a mighty fine product. However, there is so much in this product that I cannot cover everything. In short, Data Movement moves data near real-time from on-premises to the cloud. It catalogs, transforms and steams ready to be used in a data analytics app.
For this, you can use "change data capture." Which in turn means something like checking the log files of a database to see if a new line is being written away and then moving that line to the cloud. By doing this, you don't have to move the entire dataset over but instead reduce the dataset to just the last lines. So because you don't have to move a lot of data at once, this is a lot faster.
In short, it is a data warehouse tool that you can use in the cloud. It helps your organization to access multiple sources in a structured way. Note that an additional license is required to use this product.
Qlik Data Gateway
Another tool I like to use with clients is the Data Gateway. This falls under the same heading of hybrid data delivery as Data Movement. A gateway loosely translated means "gateway. It creates a gateway between your local databases and Qlik Cloud. The data can only pass through that gateway and no other tool can reach that gateway. I can best explain the tool using the image below.
On the left side of the image, you can see the databases (Enterprise Data Sources). A tool (Qlik Data Gateway) is installed in a location behind the firewall and as close to the source as possible. With a few simple actions, the link between the cloud and the Data Gateway can be established. From there, it is very easy to establish the connection from Qlik cloud and retrieve data.
Qlik Data Transfer
One tool that may not deserve the beauty prize, but does what it is supposed to do, is Qlik data Transer. To use it, it also needs to be installed as close to the source as possible. It, unlike Data Gateway, can use an ODBC connection. I often compare this to a world connector, as most sources can be accessed with it.
In addition to uploading data from databases, it can also make Excel or other types of files, which are stored locally, unlockable with the cloud. If the file is updated, then the file also moves directly to the cloud. Therefore, this is the situation for which I use Qlik Data Transfer the most.
Other data connections
Of course, various connections are also possible from Qlik Cloud directly to a source. Depending on the choices and setup of the organization, this can also be chosen. The most common cloud solutions can be accessed directly. For example, if you use SharePoint, we can easily connect to this and access the files. We can also connect to the folders on your computer when they are in an Azure Blob storage. Sounds complicated, but is perfectly doable. Feel free to contact me to discuss the possibilities.
Supporting connections
As indicated in the intro, there are also some connections to make that are supportive to the use of Qlik Cloud. Think of the Active Directory (where all users and rights are managed within the organization) or connecting to the mail server. These connections are not necessary but convenient to set up.
Of course, there are still countless connections possible. My motto is: wherever data is stored, we can get it out. The only question is how to get it done. Next time, I'm going to take you through the next step. If we have made connection now, what do we do with it? Can we track the data? And how secure is that cloud anyway? Stay tuned and we'll get back to you soon.
