A Short Incursion Into Alternate Hadoop Filesystems

Steve Loughran starts with a critical look at Netapp Open solution for Hadoop paper:

Actually it is weirder than I first thought. This is still HDFS, just running on more expensive hardware. You get the (current) HDFS limitations: no native filesystem mounting, a namenode to care about, security on a par with NFS, without the cost savings of pure-SATA-no-licensing-fees. Instead you have to use RAID everywhere, which not only bumps up your cost of storage, puts you at risk of RAID controller failure and errors in the OS drivers for those controller (hence their strict rules about which Linux releases to trust). If you do follow their recommendations and rely on hardware for data integrity, you’ve cut down the probability of node-local job execution, so all FUD about replication traffic is now moot as at least 1/3 more of your tasks will be running remote -possibly even with the Fair Scheduler, which waits for a bit to see if a local slot becomes free. What they are doing then is adding some HA hardware underneath a filesystem that is designed to give strong availability out of medium availability hardware. I have seen such a design before, and thought it sucked then too.  Information week says this is a response to EMC, but it looks more like NetApp’s strategy to stay relevant, and Cloudera are partnering with them as NetApp offered them money and if it sells into more “enterprise customers” then why not? With the extra hardware costs of NetApp the cloudera licenses will look better value, and clearly both NetApp and their customers are in need of the hand-holding that Cloudera can offer.

Then in a follow up post, he looks at a couple of alternatives (Lustre, GPFS, IBRIX, etc):

I’m not against running MapReduce—or the entire Hadoop stack—against alternate filesystems. There are some good cases where it makes sense. Other filesystems offer security, NFS mounting, the ability to be used by other applications and other features. HDFS is designed to scale well on “commodity” hardware, (where servers containing Xeon E5 series parts with 64GB RAM, 10GbE and 8-12 SFF HDDs are considered a subset of “commodity”).

Original title and link: A Short Incursion Into Alternate Hadoop Filesystems (NoSQL database©myNoSQL)


Skate Canada International, 2012, short dance

There was an awkward moment when Moir’s face disappeared in the gauzy coloured skirt of Virtue’s dress.

When someone pointed out that Moir had a “facefull of skirt,” he replied laughing: “That’s normal though, that’s my day to day, I’m used to that.”

Virtue shot back: “On the ice.”

Post FP Interview (GPF 2015) : Sara & Yuzuru

S - Portador del rècord del mundo, 330 puntos! Un aplausa por favor!  (sreams) El pobre, el pobre, no sabe, le he dicho que se siente porque… estaba ya también muy cansado… Here I am with the world record, little person that is next to me that is able to do incredible jumps, seriously Yuzuru I admire you so thank you, thank you for what you just did today. (screams) Le quería preguntar si cree verdaderamente que se puede superar y que se siente al ser el … su única competición. (I think Sara wanted to say “competidor”) Eh Yuzuru, do you think you can improve yourself…anymore, or how, you know, how you confront this moment of being the top…

Y - Score?

S - The top score, yes.

Y - (Laughs) I don’t know, because, I just want, I just want to do the perfect perfect feeling, and perfect jump and every element and… Because I feel like, score, score is score, my performance is my performance. Because… That’s a difference, right ? Bec.. I feel, I feel, I’m feeling really good today … because, because it’s not… I mean not my performance. I, I mean everyone, everyone support me, supporting me and I owe my performance to every audience and and before the TV.

S - Bueno. Yuzuru acaba de decir algo que tiene mucha razón, que en el al fin el acabo, una puntuación es solo una puntuación. y que lo que él busca es hacer su trabajo, sus pasos, sus saltos bien, sentirse completo en la pista, sentirse que de verdaderamente es el jefe de su programa y que está presente aquí con todos vosotros y ha dado las gracias a todo..el apoyo que ha recibido hoy porque se ha sentido muy apoyado…así que… eso va a por vosotros.
My next question, mi siguiente pregunta, cómo entrena esa confianza que transmite en el hielo, cómo la construye? Yuzuru, how do you train that solid confidence, you, you, you could, we could see you from the outside how you look confident, you’re solid on the ice. Do you train it, do you do you eat it, how, where do you find it, you know, where can I buy it, a little bit.

Y - (Thinks) I don’t, I don’t, I don’t have an idea, because I have… I wanna, I wanna get something. Because I’m really really nervous and I feel really heavy pressure today so I wasn’t, I wasn’t confortable aaam… (cute) Yeah please tell me, how, how do I, how do I get confident before my performance?

S - Bueno, pues no lo vamos a saber, no sabemos de donde saca esta confianza porque ni él lo sabe. Acaba de decir que se sentía muy nervioso y que solo sentía también muchísima presión, y que solo quería hacerlo bien. Así que no lo, nunca lo sabremos, es un secreto. So it’s a secret, we don’t know where your confidence comes from.

Y - Noo, no no no it’s not secret ! That I said true. And so please tell me, this this address, this address, please tell me…

S - Thank you, Yuzuru. Then one last question. Una ultima pregunta. Qué le diria a todos esos patinadores pequeños que están viéndole hoy? What would you tell to those little skaters that are watching you today? Right now you’re their big… They look up to you.

Y - Uhhm I was… flower skater in NHK when I was a little boy, and I, I was, I was, I was dreaming to be the kind of Plushenko, or kind of Johnny Weir, Alexei Yagudin…some top skater, so please, please, don’t give up for the skating and the practice, and and dreaming! (look what he’s doing with his hands it’s so cute, imho)

S - Pues aquí, Yuzuru tiene un mensaje para todos vosotros. El se acorde de cuando estaba un patinador pequeño y soñaba con ser un Plushenko o todos esos patinadores de su época…y os dice a todos que por favor no dejéis, nunca, nunca os deis por vencidos y no dejéis de soñar… Mensaje. Thank you very much Yuzuru, muchas gracias a todos, thank you for coming everyone and here we go with the award ceremony.

(Yuzu tries to speak but the mic is already off, so he waves… Stop being so cute!)

There, his hands when he says to never give up on dreaming!

The parts in italics are for when Sara speaks in Spanish and the parts in bold, when she speaks in English.

So I just wrote down what was said during the interview, I let the moments of hesitation and all :) I think it allows the interview to feel ‘natural’ and spoken (as it was) even if it’s written down. Also, I think Sara mixed Catalan and Spanish, so some words are not in Spanish for the Spanish part

I don’t actually know why I did this but it was a lot of fun so I’m posting it.

Sorry if I made mistakes, neither English nor Spanish are my mother languages.


Lenovo Solution for GPFS Storage Server - YouTube - Watch more at

Too much of a Good Thing? IBM Buys Cleversafe

Analysis - IBM Buys Cleversafe - Good Move or Bad?

External image

The industry is memorized by all things flash and sometimes forgets that IT professionals have other challenges to face than just meeting the performance demands of the modern data center. One of those is dealing with unstructured data. Data centers are drowning in data and are looking for solutions that can cost-effectively and reliably store all of it. Despite its slower than expected adoption…

View On WordPress

Mao Asada in practice at GPF Barcelona, 2015.12.09 (picture 1, 2)

Note: this is by far not my best Mao picture from GPF Barcelona technically, but this is the first decent one I took of her, on that first day. And there’s just something about her in this picture - the pose, the perfect raise of her arms, the closed eyes - that to me captures a fraction of what sets her apart among her peers, a grace that belongs to her only, even as she simply glides, taking her first steps on the Catalan ice

My photograph - all rights reserved


In this video Colin McMcMurtrie from the Swiss Center for Scientific Computing shows how to run the IBM General Purpose Filesystem (GPFS) over Obsidian.