“Data is the new Oil. Data is just like crude. It's valuable, but if unrefined it cannot really be used. ”
—Social Media as a Stored Value Currency
“This is a generation of kids that grew up with data science around them — Netflix telling them what movies they should watch, Amazon telling them what books they should read — so this is an academic interest with real-world applications,” said Chris Wiggins, a professor of applied mathematics at Columbia who is involved in its new Institute for Data Sciences and Engineering. “And,” he added, “they know it will make them employable.”
—Universities Offer Courses in a Hot New Field - Data Science - NYTimes.com“Forget Instagram’s billion-dollar payday. Forget IPOs, past and future, from Facebook, Groupon, LinkedIn and the like. And ignore, please, the online ramblings of attention-hungry venture capitalists and narcissistic Silicon Valley journalists with the off-putting habit of making their inside-baseball sound like the World Series. Their stories, to paraphrase Shakespeare, are tales told by idiots, full of sound and fury, but signifying very little about the impact of technology on most of our lives. ”
—Paul SmaleraLeveraging the digitized books database
With little fanfare, Google has made a mammoth database culled from nearly 5.2 million digitized books available to the public for free downloads and online searches, opening a new landscape of possibilities for research and education in the humanities.
A simple online tool allows investigating cultural trends quantitatively using the corpus of digitized texts. It gives to anyone the ability to browse cultural trends throughout history, as recorded in book: http://ngrams.googlelabs.com/
With a click you can see that “women,” in comparison with “men,” is rarely mentioned until the early 1970s, when feminism gained a foothold. The lines eventually cross paths about 1986.
Welcome to the internet of data: the semantic web.
“We fundamentally try to collect everything and hang on to it forever.”
—CIA’s chief technology officer Gus Hunt on the agency’s strategy for collecting digital intelligence. via Huffington Post
“An estimated 5 million Americans are already using wearable devices to sync their lives to the cloud, and their ranks are growing rapidly. Like Paulus, they are sending vast amounts of information—collectively referred to as Big Data—to the servers of salivating Silicon Valley executives. In just the first half of last year, venture-capital firms invested $700 million in businesses developing new wearable and embedded devices. According to a study by the consulting giant McKinsey, Big Data could be worth $300 billion annually to the health-care industry alone. But its value to sports-apparel companies, health-food purveyors, and even mattress-makers is also apparent. At this year's Consumer Electronics Show in Las Vegas, 283 vendors showed up to promote digital health products—over 100 more than the number of companies hawking games. One British mobile-research firm estimates that by 2017, 70 million people will be buying wearable devices annually and slapping them on their wrists (and chests, ankles, and necks). And while there is no valuation yet for, say, what Paulus' heart rate is worth per beat, there's reason to think that users of tracking apps and sensor-laden devices are giving the milk away free.”
—Who Really Owns Your Personal Data?: Critical Eye : Details
Big Data . Big Local . Big Compare
The Compare is pleased to announce the introduction of our blog.
The Compare is not a price comparison engine. It is different, it is unique, we use proprietary algorithms to find hidden gems for you to discover and explore.
Data would just be static, lackluster information if we showed simply data. We go beyond data, we provide the energy to find data from data. We’ve invented hyper Comparison Engine, built it from the idea of hyperdata. New functionality based on real world desires - expressed or not - satisfies people. You want it, and we want to be the ones to bring it to you.
By subscribing to our alpha site, you’ll be taking a front row seat at a new and fantastic show. Get all of the progress updates as we continue this journey with you, we appreciate your interest and encourage you to let us know how we are doing.
Image by James Brooks on Flickr
Tracking people using cool biology tools of the digital age
With the advent of super-fast genomic sequencing, “big data” geneaology and forensic testing, we have the ability to gather more genetic and biological data these days than we know what to do with. This is thanks to new technology that makes this analysis both cheap and fast (relatively speaking … I mean, you couldn’t go out and do it with your lunch money).
Some people have decided to focus less on selling you a cheek swab in order to predict whether creased ear lobes are connected to heart disease, and instead are trying to apply these massive amounts of genomic data to a more interesting problem:
Where did certain groups of humans come from? And where are they going?
- In this story from Public Radio International, meet a team of scientists in Utah using hair as a biological tape recorder. By reading how much of certain elements and isotopes a person’s hair has in it from one end to another, they can draw a map of every place that person’s been in the past few years. It’s all thanks to local water supplies. This CSI-worthy tech even helped solve the case of a mystery body in a UK human trafficking case.
- The fall of the Otomí people in central Mexico has been a mystery for centuries. Usurped by Aztec conquerers, no one knew exactly what happened to them. Remember that each of a person’s mitochondria has its own tiny genome that can be used to track your genetic history. Now, thanks to sequencing the tiny genomes of Otomí corpse mitochondria, scientists may have added a little more mystery to the story of their overthrow. Were they wiped out? Or made part of the Aztecs? With further research, the DNA may tell the tale…
- Tapping into the centuries-worth of genealogy data from family history company Geni.com, Whitehead Institute scientists were able to recreate the pattern of European conquest purely from an API. It’s more proof of principle than anything else, but the video below shows the potential anthropological value of digital data when turned into meaningful visuals. Relive a couple hundred years of Western history:
The Weatherman Is Not a Moron
nytimes.comFinancial forecasters almost universally agreed there would be no collapse before 2008. It happened anyway. Earthquakes? The Fukushima disaster and the tsunami that caused it reminded us how awful we are at predicting those, too.
So are weathermen, the most ridiculed of predictors, actually doing it right? Although they miss a day or two here and there, our National Weather Service has been consistently improving for over a century.
In the New York Times, Nate Silver discusses how their acceptance of chaos theory helps them get past simply relying on Big Data:
Why are weather forecasters succeeding when other predictors fail? It’s because long ago they came to accept the imperfections in their knowledge. That helped them understand that even the most sophisticated computers, combing through seemingly limitless data, are painfully ill equipped to predict something as dynamic as weather all by themselves. So as fields like economics began relying more on Big Data, meteorologists recognized that data on its own isn’t enough.
