libuv

Libuv is the core library behind Node.js. Very similar to libevent or libev, it provides the main elements for event driven systems: watching and waiting for availability in a set of sockets, and some other events like timers or asynchronous messages. However, libuv also comes with some other extras like:

2011年12月はhowtonodeのTim Caswell氏によるluvit(=libuv luajit)が出現した。 登場以降、恐ろしい勢いで実装が進み、nodeと似たような、 徹底的な非同期より速いか同じぐらい性能が出そうってことがすぐ確認された。 LuaはCのコードと非常に連携しやすいのでゲームのサーバを書くのに最適だ。 golangやrustなどのシステム言語が枯れるまでの何年かは、これでかなりつなげるかもしれない。 その場合、重い処理はC/C で書いて、回数の少ない部分はぜんぶLuaで書くという分担になる。 1月には同じくLuaベースのMoai SDKのNative Clientホストが動くようになったのと、 Three-Deeブランチがmasterに取り込まれるなど、1.0で3Dをサポートできる状態に近づいた。 これまでMoai SDKはARMサポートが無かったこともあり、いまはLuaJITベースではなく通常のLuaベースだが、 LuaJITのARMサポートが安定したら順次移行することになる(ロードマップには入っている)。 luvitとMoaiがJSONを使って通信するとオーバーヘッドが大きいので、 MessagePackで通信ができるように必要なモジュール lua-msgpack-nativeを実装した。 組み込みのJSONより2桁以上速かったので文句なし。実際の開発でこれを使うことになるだろう。 現在luvitにおける最大の弱点は、過去のLuaライブラリはすべて同期プログラミング用のものしかないということ。 nodeもそうだったが、必要な非同期ライブラリ、とくにDBまわりがそろうまでは使いにくいだろう。 自分でもlibmysqlclientのAPIを駆使してmysqlバインディングを作れないか試したが、 それはそれはひどかった。。。。のでどうしようかと考えている。Redisのようにきれいなプロトコルでもないし。
This week in Node

This week in Node is a weekly blog post that focuses on the latest changes and updates in the Node.js repository and in NPM modules.

Forking and IPC

Commit 9e26dab150e and 337c48db5fe landed a new method in the child_process module. The fork method allows you to spawn a new Node.js process and pass messages between your current Node.js process and the newly created process using a Inter Process Communication (IPC) channel.

So why is this an important change? This is the first step to proper and hopefully first class support for multiple processing Node.js. When you are doing allot of CPU heavy processing it can take a while before your function is done and is ready to return to the event loop again. Ideally you would use a job server such as Gearman or Resque for heavy processing but that is not always and option. As your application is busy with processing data it can the amount of concurrent your application can handle, but you can now offload all the processing to a different Node.js process and be notified once the work is done.

The API is quite simple and straight forward:

var cp = require('child_process')
  , n = cp.fork(__dirname + '/process.js');

n.on('message', function(m) {
  console.log('PARENT got message:', m);
});

n.send({ hello: 'world' });

And inside process.js:

process.on('message', function(m) {
  // illustrative function ;) 
  hardcoreProcessing(m, function(err, res){
    process.send(res);
  })
});

Wait, what? No Web Worker API

That was the first thing that came to my mind, why don’t we just use the Web Worker API for it. It was designed for stuff like this and it’s already known to some developers even the Node.js website states:

But what about multiple-processor concurrency? Aren’t threads necessary to scale programs to multi-core computers? Processes are necessary to scale to multi-core computers, not memory-sharing threads. The fundamentals of scalable systems are fast networking and non-blocking design—the rest is message passing. In future versions, Node will be able to fork new processes (using the Web Workers API ) which fits well into the current design.

I don’t know how far in the future we currently are but it with the current release it’s actually quite easy to emulate with a few simple lines of code:

if (process.env.NODE_CHANNEL_FD){
  (function(){
    var onmessage;

    // create onmessage api
    Object.defineProperty(GLOBAL, 'onmessage', {
      get: function(){ return onmessage }
    , set: function(fn){
        onmessage = fn;
        process.on('message', function(msg){
          onmessage({data:msg});
        });
      }
    });

    // all we need to do is proxy the process.send
    Object.defineProperty(GLOBAL, 'postMessage', {
      get: function(){ return process.send }
    });
  })()
}

Adding those lines of code would allow you to use onmessage and postMessage as specified in the Web Worker API. But non the less this forking and IPC between different node instances is a more than welcome addition to the node core. See full .fork api here.

Libuv

Integration of Libuv (formerly known as liboio) in to Node.js has started. Libuv provides a high-concurrency and high-performance I/O on all operating systems. This would allow Node.js to be deployed on Windows without using Cygwin or MinGW Unix emulation layers. Because these are emulation layers, the performance has always been really poor and Libuv changes that by using native Windows APIs.

For more information about Libuv check out the github repository and Ry’s Nodeconf 2011 slides.

Long ago in my technical writing, I sought to eliminate gendered pronouns when referring to unnamed people or entities.

Joyent, the folks in charge of Node.js, say that they would have fired Ben Noordhuis if he worked for them. Why? A contributor tried to submit a change removing a needless “he” in the comments of libuv and Ben reverted the change. Why? His direct response to the commit: “Sorry, not interested in trivial changes like that.”

TRIVIALLY EASY. It had no affect on the way the code ran but it had every effect on the community of contributors and potential contributors of the project. Luckily, Bryan Cantrill, SVP of Engineering for Joyent, understands the problem and supported the project’s manager, Isaac Schlueter, in accepting the patch. He’s also stated that he expects Isaac will take other actions since Ben is not an employee.

By publicly talking about what they as a company did and why, they set a tone and expectations about how things will be run in their projects.

This event is not in and of itself a world-changing event, but it is crucial. This is a small and easy second step.

So far as I can tell, Joyent did not follow the first step. Projects must have community guidelines emphasizing diversity and equality along with a solid anti-harassment policy. I looked all over the libuv project, and couldn’t find anything on it. I’d rather they have an unspoken policy they do enforce than a written policy they don’t enforce. It’s far better to have the written policy, constant commitment to that policy, and enforcement.

The development community as a whole needs to demand more of our projects.

Joyent has shown the community that it cares and will act. It’s not perfect, but it’s a hell of a lot better than the development community does as a whole.

Rust 0.6 for Fedora


The previous draft spec wasn’t able to build rust with the official 0.6 sources. The spec file is now updated and you can build your own rust package using:
$ curl -O https://raw.github.com/fabiand/rust-spec/master/rust.spec
$ nice ionice rpmbuild --noclean -ba -v rust.spec

It is assumed that you’ve got a working fedora-package setup and an internet connection.

Changes are mainly that chrpath is used to remove rpaths from files. Rust is still using it’s own libuv and llvm.
LibUV: How Node.js' Asynchronous IO Works Under the Hood

At LXJS 2012, Node.js core committer Bert Belder gave a talk about
libuv, the library Node uses to get cross-platform asynchronous IO. And
if this sort of thing intrigues you, consider Marc’s course (above).

http://blog.nodejs.org/2012/09/30/bert-belder-libuv-lxjs-2012

.js


10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS…
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION

http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci

This week in Node

Welcome to another epic episode of This week in Node, where I blog about the latest news and updates in the Node.js community.

Node 0.4.8

A new Node stable has been released just one day after I posted This week in Node #20. This release brings us more performance for encrypted SSL connections. This is achieved by disabling the compressions in OpenSSL. This improves memory consumption and speed. If you want to apply compression it should now be done in “userland”.

Full list of fixes

  • #974 Properly report traceless errors (isaacs)
  • #983 Better JSON.parse error detection in REPL (isaacs)
  • #836 Agent socket errors bubble up to req only if req exists
  • #1041 Fix event listener leak check timing (koichik)
  • #1038 Fix dns.resolve() with ‘PTR’ throws Error: Unknown type “PTR” (koichik)
  • #1073 Share SSL context between server connections (Fedor Indutny)
  • Disable compression with OpenSSL. Improves memory perf.
  • Implement os.totalmem() and os.freemem() for SunOS (Alexandre Marangone)
  • Fix a special characters in URL regression (isaacs)
  • Fix idle timeouts in HTTPS (Felix Geisendörfer)
  • SlowBuffer.write() with ‘ucs2′ throws ReferenceError. (koichik)
  • http.ServerRequest ‘close’ sometimes gets an error argument (Felix Geisendörfer)
  • Doc improvements
  • cleartextstream.destroy() should close(2) the socket. Previously was being mapped to a shutdown(2) syscall.
  • No longer compile out asserts and debug statements in normal build.
  • Debugger improvements.
  • Upgrade V8 to 3.1.8.16.

Download the latest version from the Node.js site http://nodejs.org/dist/node-v0.4.8.tar.gz

New Core team members

Just announced today, 2 new core team members (Bert and Felix) have been added to Node. Bert Belder (@piscisaureus) has contributed countless bug fixes for the windows, not only fixing bugs but also implementing new features in windows. He has been working been working allot on Libuv the last couple of weeks / months and it’s being integrated in to node as we speak.

Felix (@felixge) has been working and patching allot of different I/O streams which are one the most important parts of Node.js. But not only did he work on Streams, he fixed and worked on allot of more important parts of Node.js.

So congratulations for the both of them thanks for all the bugs that you guys fixed, and will fix in the future!

Ryan Dahl's libuv video Tutorial Revisited

In 2010, Ryan Dahl, the Author of Node.js, created a 53 minutes video to get you started with the libuv Async IO library, it’s an amazing tutorial.

Even when the video was created with a early version of libuv, you can follow the tutorial fluently. I wanted to highlight some of the differences that I noticed to get it working by Apr 2012:

EV Multiplicity Flag Use EV_MULTIPLICITY set to 1 instead of 0

gcc ... -DEV_MULTIPLICITY=1

**Header Files* Includes are in uv/include, compiler arg -Iuv/include should be used and #include “uv.h” instead of “uv/uv.h”

uv_tcp_init

uv_tcp_init now requires uv_loop_t* as first parameter and the handle as the second parameter, so you should use it as: uv_tcp_init(uv_default_loop(), &server);

uv_bind

uv_bind is not generic anymore, in the video Ry is using TCP so you should use the TCP version of bind, that would be uv_tcp_bind, no argument changes.

uv_accept

uv_accept no longer accepts a close callback, you will pass the callback when you invoke uv_close. uv_close is usually called when the uv_read_cb callback is invoked and UV_EOF is detected in the error code.

Also the ->data handle is no longer set by the last argument, you are going to set the pointer yourself after the uv_handle_t is initialized.

Example:

client_handle->data = your_thing;

uv_read_cb

uv_read_cb now takes uv_stream_t* instead of uv_tcp_t*. Simple cast fix.

uv_write

uv_write now takes the handle, the buffers, the count and a callback. You don’t need to use uv_req_init anymore, you just allocate it and release it when on the callback. None of the requests in libuv are reused, you need to free them when the callback is called.

The http-parser library didn’t suffered any change at all.

Happy libuv hacking!

libevent [はてなブックマーク]

libevent

Support & Info Bug Tracking: Bugs Features Mailing List: Info Git Repository: sourceforge github License: 3-clause BSD IRC: #libevent on OFTC Happy Hacking You can make Libevent developers happy by re…

kjw_junichi プログラミング, windows, Linux, osx libuvみたいにWindowsも扱えるライブラリのようだ



from kjw_junichiのブックマーク http://libevent.org/
人間とウェブの未来 - Linuxにlibuvをインストールした時にハマった所 [はてなブックマーク]

人間とウェブの未来 - Linuxにlibuvをインストールした時にハマった所

「ウェブの歴史は人類の歴史の繰り返し」という観点から色々勉強しています。NWM代表としてフリーランスのエンジニアをしています。最近はnode.jsに興味を持ち出しています。 そうなると、やっぱり内部のアーキテクチャが気になってしまって、jsを弄るのと変更してlibuvを使ってCでコード書いたりしています。例えば、Apache2.4系のmpmはLoadableなので、libuvでmpm書いちゃったら…

kjw_junichi プログラミング



from kjw_junichiのブックマーク http://ift.tt/1zBb1XR