Data: Do We Have Too Much of a Good Thing?

Can there be too much of a good thing?

When it comes to data, the answer may be yes. It seems the more data we have, the more knowledge we lose; the more value goes unrecognised. Over the past decades we’ve gone from wet lab methods, to high-powered spectroscopy, to lab-on-a-chip, and more. The chromatography community knows this all too well. As the science of chromatography has advanced (for example from HPLC to UHPLC) so too have the numerous instruments and software become more sophisticated. The data generated from new tools and techniques have been massive and have proved invaluable to the advancement of science, without a doubt. In fact, organisations’ investment in instruments now exceeds $1 million annually, and 80% of organisations rely on analytical data for decision-making.

But as we’ve all experienced, the massive amounts of data we are generating can be a real pain. Or, more precisely, managing the data can be a real pain. It comes in different formats from different instrument vendors; it’s shared differently and is more and more often spread over multiple departments, geographic locations, and between partner organisations/contractors. If the data generated from method development and optimisation could be re-used effectively, imagine the reduction of workload and increased productivity from your lab. Of course, this isn’t a problem relegated to chromatographers or analytical chemists in general. R&D labs across disciplines are grappling with the glut of information and how to best extract its value.
Is the problem of data overload, and more importantly data management, all that bad, though? Are we really just collectively whining? After all, with the preponderance of data over the years, there have been a number of software management systems and techniques introduced to solve the problem.

At ACD/Labs we set out to gather some empirical evidence on data management. In 2015 we approached scientists, managers, and executives in R&D organisations to complete a survey that would provide a clearer understanding of the knowledge of analytical data management. What we found was unsurprising given what we’d heard from customers at the outset of the project, but at the same time the results were a
little incredulous.

It turns out that an overwhelming 70% of survey respondents use up to 10 different analytical techniques, as well as a variety of instruments from multiple vendors, to collect and analyse their data. And a majority of respondents indicated that once information is extracted after the analysis, there is no one standard method used to share or record the data.
There’s more. Remember that old childhood game where a kid whispers a story in another kid’s ear, and that story is passed along in a circle until the last child in the circle has to repeat it out loud? Then everyone laughs because the story is nothing like it was when they heard it, some pieces of information are missing and others are incorrect......