COSMOS – "Let's take a look"
We have all heard of 'Big Data' and some of us have discussed the challenges that a seemingly never-ending stream of information quickly generates.
The medical and pharmaceutical industries find themselves in a battle of understanding; trying to make some sense from the mass of information that now becomes routinely available to researchers. Data quality, consistency, integrity and value are now as big a problem as data volume.
The problems are certainly not getting smaller but we are beginning to see a number of initiatives around the world which begin to address the real issues and may start to answer some of the questions. Of course the first problem is to know what the question is.
It is easy to say that smoking is bad for you and few would argue differently but then so too is chocolate. Unfortunately many of the arguments for and against are impacted by factors which only go to make a simple theory much more complicated. Chocolate may be bad for us but when lifestyle choices are considered, regular exercise for example, that choice might offset those negatives unless of course you suffer from diabetes. In reality cause and effect are very hard to identify and even harder to prove.
"All truths are easy to understand once they are discovered; the point is to discover them." – Galileo Galilei
Four years ago the team in Optima Systems Ltd set about a project to build a tool which allowed someone to view and explore data in new ways. The tool had to enable any interested party to take a top down look at the information but then navigate through it, identify patterns, see hot spots and hopefully be drawn towards areas where questions should, could or might be asked. The whole premise was of a tool that could absorb large quantities of information from a wide variety of sources and analyse in real time producing a variety of outputs to aid the inquisitive mind. From time to time the tool had to be driven by non-domain experts and so a fast, responsive and intuitive UI would be essential. In addition this same UI had to have the features demanded by the experts and give them the power they needed to interact with their data.
That project has produced COSMOS; a data analysis and visualisation tool unlike any other. From those who have seen it, it has been described as a 'thesis generator', a 'tool of thought', 'very fast' or simply 'cool!'
COSMOS uses Dyalog APL running on servers in the background which feed data to the browser based front end in response to user interaction. Having APL under the cover gives great power and performance but as importantly, having the Dyalog team behind the development team is a huge advantage. Without APL the project would probably not have developed to where it is now and certainly not in the time available.
In the various worlds COSMOS operates data can be monitored and tracked in real time. Tactical business decisions can be made in response to real time and changing data and it can all be done quickly. Processes using more traditional techniques can, in some cases, take days or even weeks to produce the required results. COSMOS and APL combined do the same job in hours or seconds.
At the time of writing COSMOS was being used in a number of medical trials and showing itself to be essential to their data collection and analysis requirements. The team at Optima hope that within the next few weeks a new and even more exciting opportunity will arise that will allow COSMOS to show its true value and start to make a real difference in the application of some new cancer therapies.