Reliability of neuroscience research questioned

New research has questioned the reliability of neuroscience studies, saying that conclusions could be misleading due to small sample sizes.


A team led by academics from the University of Bristol reviewed 48 articles on neuroscience meta-analysis which were published in 2011 and concluded that most had an average power of around 20 per cent – a finding which means the chance of the average study discovering the effect being investigated is only one in five.

The paper, being published in Nature Reviews Neuroscience, reveals that small, low-powered studies are ‘endemic’ in neuroscience, producing unreliable research which is inefficient and wasteful.

It focuses on how low statistical power – caused by low sample size of studies, small effects being investigated, or both – can be misleading and produce more false scientific claims than high-powered studies.

It also illustrates how low power reduces a study’s ability to detect any effects and shows that when discoveries are claimed, they are more likely to be false or misleading.

The paper claims there is substantial evidence that a large proportion of research published in scientific literature may be unreliable as a consequence.

Another consequence is that the findings are overestimated because smaller studies consistently give more positive results than larger studies. This was found to be the case for studies using a diverse range of methods, including brain imaging, genetics and animal studies.

Kate Button, from the School of Social and Community Medicine, and Marcus Munafò, from the School of Experimental Psychology, led a team of researchers from Stanford University, the University of Virginia and the University of Oxford.

She said: “There’s a lot of interest at the moment in improving the reliability of science. We looked at neuroscience literature and found that, on average, studies had only around a 20 per cent chance of detecting the effects they were investigating, even if the effects are real. This has two important implications - many studies lack the ability to give definitive answers to the questions they are testing, and many claimed findings are likely to be incorrect or unreliable.”

The study concludes that improving the standard of results in neuroscience, and enabling them to be more easily reproduced, is a key priority and requires attention to well-established methodological principles.

It recommends that existing scientific practices can be improved with small changes or additions to methodologies, such as acknowledging any limitations in the interpretation of results; disclosing methods and findings transparently; and working collaboratively to increase the total sample size and power.

Ten years ago we would warn students, as a matter of course, not to use Wikipedia—that it was partial, incomplete, open to misrepresentation,” says Tom Lawson, dean of the School of Arts at CalArts. “But over time it’s become much more reliable and much more ubiquitous.

I understand fully now, how important a sense of security truly is to the female partner in a relationship. Practically more important to them than anything else when choosing a relationship to be in. She will date a man she wouldn’t look twice at, if he can offer her these things. It’s different from being a gold digger, it appears to be genetically coded into their instincts.

Army Cronyism Results In Halt To M4 Replacement Trials

Army Cronyism Results In Halt To M4 Replacement Trials

Army Quits Tests After Competing Rifle Outperforms M4A1 Carbine



” A competing rifle outperformed the Army’s favored M4A1 carbine in key firings during a competition last year before the service abruptly called off the tests and stuck with its gun, according to a new confidential report.

  The report also says the Armychanged the ammunition midstream to a round “tailored” for the M4A1…

View On WordPress

Questionable narrators

In a discussion on Frankenstein, a professor mentioned that we should go into the reading thinking about unreliable narrators.

That thought has stuck with me in our discussions of Glee.  This concept that there are so many different characters narrating at different times, each telling their own side of the story…

But not always reliably.

This is most obvious in Shooting Star, where Sue tells the principal one story, but a few minutes later the audience is shown what really happened was rather different.

This is another of those ways in which I think it’s not always the Glee writers who are inconsistent, it’s the Glee characters who sometimes/often have their own reasons to distort the truth.

Only the person who has faith in himself is able to be faithful to
others, because only he can be sure that he will be the same at a future time as he is today and, therefore, that he will feel and act as he now expects to. Faith in oneself is a condition of our ability to promise, and since, as Nietzsche said, man can be defined by his capacity to promise, faith is one of the conditions of human existence. What matters in relation to love is the faith in one’s own love; in its ability to produce love in others, and in its reliability.
—  Erich Fromm - The Art of Loving
I finally figured it out!

After spending so long trying to figure this out, I think I’ve got it!

If the heroes of Class of the Titans were the Digidestined, I finally got what their crests and Digimon would be:

Jay - Crest of Hope (mostly cause he’s constantly hoping to end this whole war against time).  I’ve also figured his Digimon would be Patamon because Patamon digivolves into Angemon, Angemon serving as Jay’s guardian angel.

Odie - Crest of Knowledge (cause he’s the brains).  I’ve also thought Odie’s Digimon would be Tentomon or Wormmon because Odie would find either Digimon fascinating.

Herry - Crest of Reliability (admit it, he’s always there for our favorite heroes no matter what).  Thinking about it, Herry’s Digimon would be Gomamon because Gomamon digivolves into Ikkakumon, a really strong Digimon.

Atlanta - Crest of Sincerity (despite Atlanta being a bad ass, she’s as sincere as she comes).  I could totally see Atlanta with Palmon because Palmon is a plant-type Digimon, and Atlanta is all about protecting the environment.

Archie - Crest of Friendship (not kidding you, Archie is probably one of the greatest friends any of the guys have).  Archie’s Digimon is Gabumon, mostly because they both don’t have a liking for water.

Theresa - Crest of Love (she’s likely one of the most motherly of the group).  Her Digimon is Biyomon, reason being Biyomon is very much like her - caring beyond belief.

And, finally,

Neil - Crest of Miracles (I kid you not, it’s because he’s the lucky one).  His Digimon is Veemon, because both are self-absorbed, and the fact they’re both lucky enough to get away unscathed.

Tech Tuesday: Cloud and Reliability

I have often said that Amazon has done more for startups than all of us early stage VCs combined. I really don’t see a startup anymore these days that is not getting going on either Amazon or sometimes Rackspace’s cloud offering (I have yet to seen a startup on Azure or IBM’s cloud). The idea of racking your own server as a first step, which is what we routinely did in the late 90s and then later managed dedicated servers seems positively quaint. So I am with Marc Andreessen here in what seems like a bit of a manufactured fight with Pat Gelsinger.

There is an important thing to keep in mind though: the more magic you use the less you control your own reliability. What do I mean by magic? Anything that powerfully abstracts away the underlying hardware. This starts with something like Amazon’s EBS and goes all the way to PaaS like Heroku. These abstractions really dramatically reduce your admin requirements which is terrific but they also mean your reliability is now at best as good as the platform. The platforms are getting better all the time but if what you are offering needs to have reliability higher than that offered by the platform you can’t deliver and more importantly you cannot really do anything (other than wait) if there is downtime.

For most startups this is an entirely reasonable trade off to make. The reliability of the cloud providers is constantly going up and they are likely to be better at operating many server than you would be yourself. Also, if you want to have better control over your reliability you can still make use of the cloud, you just can use less or no magic. For instance, if all you do is spin up EC2 instances and use those across availability zones I don’t believe there has been a single Amazon outage where all instances would have disappeared across all availability zones (someone please correct me if I am wrong here). For a case like this you have to really architect for being able to route load / computation to available nodes and be mindful of ephemeral instance storage.

In summary then — for most startups cloud reliability will exceed what you could do on your own and for the few where it really matters you can still use the cloud but mostly for giving you lots of instances in lots of places.


   “My last question for you concerns attendance. How reliable are you? With Virginia’s health issues we really need someone we can depend on”

    Translation? My employers can’t depend on me and need someone to take up the slack. It sucks, but it’s necessary and I know it…. and that’s why I sat in on the interviews yesterday and repeatedly cringed when we got to the reliability question but pasted on a big smile and tried to maintain some excitement about hiring two more people to assist with the workload.

   I just hate that the main deciding factor seems to be “Will you actually show up regularly unlike Little Miss Sickie over here who disappears for weeks and months at a time?”

    Because I do… and I can’t blame anyone in the slightest for being concerned or wanting to ensure someone will actually be showing up every day to get the job done. Before my first major flare I never took time off and had saved hundreds of hours of accrued Paid Time Off and unused sick time. Now I’ll frequently be skipping along happy as can be then fall into a flare, end up so sick the situation becomes life threatening, be hospitalized, drop a few dozen pounds within a few weeks, and miss tons of work. Hell, I’m honestly surprised anyone kept me around this long….

    AND IT IS SO FRUSTRATING!!!!!!!!!!!!!

The Case for the Reliability of the New Testament (A Brief Outline)

(1) The New Testament Has Been Faithfully Transmitted

The overarching content of the New Testament Gospels can be tested over time as we examine the writings of those who learned from the apostolic eyewitnesses:

(a) Ancient Sources Confirm the Early New Testament Canon

1. Clement of Rome (c. A.D. 95)
2. Ignatius of Antioch (c. A.D. 115)
3. Polycarp, a disciple of John, (c. A.D. 108)

(b) It Was Recognized in Geographically Independent Areas

1. Irenaeus (in Asia Minor)
2. Origen (in Alexandria, Egypt)
3. Hippolytus (in Rome)
4. Eusebius (in Cæsarea, Palestine)
5. Athanasius (in Alexandria, Eqypt)

(c) The Informational Content of the New Testament is Reflected in the Writings of the Students of the Apostolic Eyewitnesses

(d) The New Testament Documents are Larger in Number and Closer in Proximity to the Events than ANY Other Ancient Record

(2) The New Testament Has Been Verified with Archeology

The “touch point” corroboration of archaeology affirms theNew Testament narratives. We don’t have to verify every detail of the New Testament to be confident in its reliability:

(a) The Gospels and Writings Have Been Verified By Archeology

1. The Census – by the Quirinius Inscription
2. Lysanias – by the Damascus Inscription
3. “The Pavement” – by the Tower of Antonia
4. Pontius Pilate – by the Pontus Pilate Rock
5. Crucifixion – by remains of Yohanan Ben Ha’galgol
6. The Iconium – by the William Ramsay monument
7. “Politarch” – by Thessalonican Inscriptions
8. Sergius Paulus – by the Sergius Paulus Inscription
9. Gallio – by the Delphi Inscriptions

(b) The Gospel of Luke Includes True Accounts of Roman Culture

1. A correct description of ways to gain Roman citizenship
2. An accurate explanation of provincial penal procedure
3. A true depiction of invoking one’s roman citizenship
4. A true description of being in Roman custody

(3) The New Testament Has Been Confirmed by Prophecy

The fulfilled prophecy in the New Testament places the text in a category of its own, elevating it from reliable historical record to Divinely inspired communication:

(a) The New Testament Contains Accurate Predictions From Jesus

1. That Peter Would Deny Him Three Times
2. That Jerusalem and the Temple Would Be Destroyed
3. That the Church Would Survive and Grow
4. That the Gospel Would Be Preached to the World
5. That His Words Would Never Be Forgotten

(b) The New Testament Contains Accurate Predictions Fulfilled by Jesus Himself