Erudite Ruby Distributed Database

I decided to write erudite because I was fascinated by Erlang’s EPMD communications daemon.  I was thinking of a new open source project to work on, and it seemed like a good idea to build a ragged right edge database that would auto-shard, auto-migrate and auto-cluster.  I also needed for it to listen for MySQL traffic and have a DSL for SQL which Ruby helps with.

I chose to write it in Ruby because I felt that Ruby’s dynamic nature would make it easier to work with.

Features

  • Auto sharding
  • Auto data migration based on capacity and usage
  • Partial file storage
  • Self-healing 
  • Eventual Consistency
  • Auto-Discovery of new processes

Did You Get An 18.3% ROI?

amplify.com

The managers of Pride of Austin High Yield Fund 1 distributed an annualized yield of 18.3% to the investors in the fund for the 1ST Quarter of 2011. A similar performance is anticipated by the managers for the second quarter of 2011.
If you would like information on investing into the fund or converting your IRA to a self-directed IRA so that you would be allowed to invest into the fund as part of your retirement strategy then please contact us for a Memorandum for your consideration.

“The future has already arrived. It's just not evenly distributed yet.”

—William Gibson

Central Heating In The Modern World

Central Heating In The Modern World

Amplify’d from www.massageprofit.com

 

See this Amp at http://amplify.com/u/anrut

the ghandi countdown

Martin Random posted:

Trying to out-discipline and out-armor the police is a losing tactic. The police can always beat an inexperienced demonstration crowd when the crowd has numbers because of their superior discipline, experience, coordination, and equipment.

Using the human mic to coordinate the mass of the people and rapidly maneuver, unpredictably disperse and coalesce, or even retreat and then stampede, that is going to be unbeatable when you have numbers.

Imagine trying to head off or kettle a march that can use the human mic to turn on a dime or perform a coordinated about face. Impossible. Imagine everyone in a crowd spontaneously running at full speed to disperse only to coalesce two blocks away. 

Any tactic that requires large numbers of people to coordinate and which can be shouted and quickly understood by untrained or mostly inexperienced people in one or two quick sentences is now possible. Just throwing stuff off the top of my head: Group charging and retreating, coordinated volleys of projectiles, even crazy stuff like “run forward and climb over” to completely overrun a line of shields. 

The human mic is a leap advance in crowd tactics like the first rudimentary central nervous system was in multicellular organisms.


I just wanted to emphasize that this is entirely correct. At the Assembly in Chicago this weekend, I ran into a representative from OWS who is making the trip west to hook up with other major occupations in order to build a world-wide Occupation Alert System, that was secure, authoritative, and could quickly mobilize people anywhere to respond to immediate crises. The elevator pitch he gave at the GA was something like this: “The next time something like Oakland happens, we need everyone to respond immediately, and we can’t wait for days or even hours for the word to spread.”

Talking to the guy afterwards, he was concerned that the pace, scope, and authority of the standard techniques like FB and Twitter did not make for an effective alert and coordination system, especially in response to large crises (and the possibility of direct conflict in the streets). FB and Twitter tend to let information get buried and lost, are difficult to search, convey very little concrete and useful information, and there are all sorts of security holes and noise using those services. Crowdsourcing the media works over the long term and over large scales, but for direct and immediate action you need something more flexible and spritely. 

We are at stage 2 of the Ghandi Countdown: they are laughing at us. If we make it to stage 3, however, (where they actually fight us directly, instead of just lobbing explosives from behind barricades and denying it afterwards) then we will need to organize a new generation of urban guerrilla nonviolent warfare techniques. 

And that takes exactly the form that Martin Random and Petey and others are describing. It means small roving groups that are linked back to some kind of centralized dispatch, which can respond with a flash mob within a few minutes notice. Places like Manhattan and Chicago, which are laid out on a grid system, are absolutely perfect for this kind of distributed action. It means that cops can’t show up 100 strong on their ponies and make a big show of force, but have to be scattered around the city, raising the chances of being outnumbered by the people, and generally reducing their role to glorified traffic cops. 

This wouldn’t be possible before the digital age. This is next gen tactics and we need to start thinking seriously of how to organize it. I saw a lot of people making suggestions to the OWS guy that weren’t any more developed than “A big skype conference call and IRC channel”, but that shit won’t cut it for a million reasons. I did hear the model of TinyChat brought up a few times, but I don’t know what lies down that path. 

Here’s how I think it happens, from my experience on the ground, given my near total lack of technical knowledge. Some system where a collection of about 20-30 people can collectively link as a roving cluster, with one person (with the fullest cell battery, maybe) designated as the com/media guy, who is in contact with a central dispatch that is monitoring the location and activity of all the other pods. OWS has shown that it isn’t too hard to get groups of 3000+ people gathered with about a week’s notice; if that amounts to 100 different protests in 100 different locations instead of one centralized gathering, with each pod coordinated and synced to the distributed collective, then the 200 police officers either have to split up 100 ways also (and be completely ineffective), or push all 200 cops at a single group of 30. That’s a dangerous scenario, but one which shouldn’t be a problem if the com/media guy has it all documented, and some central dispatch can coordinate the nearby pods to contract on that location within a moment’s notice. 

Let me emphasize: if these protests are going to work, then some tactic like this is absolutely essential. The police have a hundreds of years of experience dealing with crowd control of large groups of people, and they are effective at crowd control in nearly 100% of cases, including cases where the crowds turn violent and riotous. They know how to deal with these situations; they have extensive experience and training for exactly this situations. We cannot beat them at their own game. So we are forced to develop the next set of strategies, and that’s collective distributed actions. The last 10 years of watching our troops fight the “autonomous cells” of Al Qaeda have certainly been instructive, even for the X-Box generation, and it will soon be time for us to step up our own organizing efforts on the ground to meet this 21st century challenge.

A system like that would require a lot of coordination and discipline, and I don’t think we have the tech yet to do it properly even if we assume a stable 3/4g internet connection. If things continue to escalate at their current rate, we’ll have to start dealing with deliberate network shut downs (like we saw in Cairo) within the next year. That’ll be a major hurdle to overcome when we get there, but I think by that point we’ll already be locked into a major tech and arms race with the state, which will mean that our grab for power is working. 

Just a few more points on this topic:

1) DHS is already involved in these operations. The FEDs have been fighting urban warfare scenarios like this for a long, long time, and they know very well what they are doing. There are a lot of very smart, very talented people helping out with the occupations as well, but this is not going to be an easy fight, and at least from an organizational standpoint I think it will be the hardest in America since we have the strongest enemy, the lowest public support, and the least personal stake riding on our success than any other occupation.

2) The next gen of internet toys and social modeling will be driven by this kind of arms race. Even if you don’t care about the occupation or want to see it fail, you have to recognize that these are the challenges that drive innovation and technological progress. I guarantee that in 10 years, the big internet fad that is so important that we can’t imagine how we ever got by without it, will have been developed to address the organizational needs of the occupation.

Centralized VCS vs Distributed VCS

Today I actually got the difference between Centralized VCS and Distributed VCS. My company uses SVN which is the CSVS and we are planning to move on to DVCS(Git). 

The main advantage of a distributed system is efficiency in branch structure. In DVCS, the branches can be created locally instead of creating them on the Repo. If developer A and B work on the same project and B wants to play with the code without actually disturbing the A’s repo, B can make a local branch of the repo and try experimenting on it. (SVN also allows branching but the branch is on Repo not local)

Second advantage of DVCS is portability. If you are at home/offline where you cannot VPN and need to work on the code , obviously you cannot commit your code to Repo(in case of SVN). But using Git, you can work offline commit your code to your local branches and when you are connected to the office network, you can merge you branch to Repo.

The only downside I see in Git is two developers can’t see what others are working on. If developer A makes some modifications on the Repo before B actually merges his branch, then that is going to cause some confusion. 

Unanimous multiprocessing of the single chip

As SOC designs the appearance of the component, such as MIPS32 1004K unanimous processing system (CPS) ,Symmetrical multiprocessing (SMP) on slice under single operating system terms Having already become a kind of genuine design alternative, and the systematic framework teacher needs to find out about its advantage and limitation.

The more tasks are, the more need running side by side

The complicated embedded software system of modularization multi-tasking often demonstrates ” Find accidentally ” Erupt simultaneously. The general assignment of this system may include multitask operation, there are different duties in each operation, a series of different input demands that satisfiable. Without time-sharing operating system, each one in these tasks must run on the independent processor. In a timesharing monoprocessor (uniprocessor) Have, the task can run in the alternative time passage. On a multi-processor which adopts SMP operating system, the task can run on a lot of processors suitable for using running side by side.

Concurrent multi-tasking of Fig. 1 

Distributed process

Another kind of formal concurrent processing has already become the customary thing, that is not even considered to be sometimes ” Run side by side ” ,there are for distributed computation on this,the it one net computer / server model for the most common one up until now. The programming of client / server is basically the form in which a kind of control flow is decomposed. A procedure task sends the request connected sum of the job to one or more special tasks in the system, this system is appointed to carry out the concrete work, instead of being carried out all to calculate by task of this procedure. The programming of client / server is usually realized in LAN and on WAN, the communication between SMP SoC inner task is to follow the same example too. People can utilize upper or invalid slice ” Loopback ” The network interface has not communicated through the binary scale of the modified client / server through TCP/IP, or utilize transmitting the local communication protocol that the data buffered in the memorizer more effectively.

In fact, this kind of technology can use singly, or use in conjunction, in order to utilize and give the applied ability based on SMP platform. The array runs side by side in the data that people can even construct a kind of distributed SMP server, among them each array can carry out the assembly line of a control flow. However, in order to realize this scheme effectively, may need very big work capacity and data set.

The support of the system software is very crucial

In some SoC systems, may realize static physics of the processor decomposes the parallelism (for example each input port has a processor kernel) of the task ,In such a SoC system, distributes a plurality of concurrent tasks into different processors to deal with and can finish in the hardware. This will reduce the software expenses and area of position occupied, but can’t offer the flexibility.

Likewise, if embedded application can be resolved into customer end and server end procedure by the static behavior, these procedures are through interconnecting communication on slice, the so head-to-tail and required only system software is carrying out and sharing the information transfer code of the agreement among the processors. The information transfer agreement can offer some abstraction layers, can be used for disposing more or less processors to operate and share the basal application code, but to any given disposition, the load balancing among the processors is silent just like hardware subregion. In order to realize flexible concurrent system programming, need to finish the distributed task of the software in the multi-processor system of a shared resource.

Flexibility and adaptability of SMP system

As its name suggests, there is one SMP operating system to system ” symmetrical ” Meaning. All processors can find the state of identical memorizer, identical I/O device and identical overall operating system. This will make the procedure from a processor to another processor transplant making it extremely simple and effective, simple example,to last load balancing is getting easier. The programming or system management that needn’t be extra, the task program on a set of single CPU which adopts time division, will operate on a SMP system can use CPU at the same time. If the intersection of Linux and SMP dispatcher of dispatcher open and shut off the procedure of the processor, everything goes on in a fair way.

It is unnecessary in order to give play to the parallelism of SMP and revise that Linux running as multiprocessing employs. In most cases, needn’t compile again!; Binary code that the static behavior of safe storehouse of the thread of right and wrong connected of an exception.

SMP Linux environment can offer a lot of tools, helps the system designer to regulate and use the task share mode of the processor. These tasks can let their priority be risen and dropped, can be limited by the arbitrary subset running on the processor. Utilize appropriate nuclear support, they can ask to use different real-time scheduling states.

One kind of UNIX operating systems always allows the utility program to control the relative dispatcher prioritiy of the task, it is like this even if it is a time-sharing system of the monoprocessor. With the appearance of more complicated mechanism of operating the priority of the task, task set or concrete user of system, traditionally fine other shell command (shell command) And system call has already risen to Linux, it should be very essential that this pair judges quality of this operating system.

In addition, dispose the respect in the multi-processor, all Linux tasks have a parameter, have defined what processor groups (set of processor) it is Can manage the task. According to acquiescence, this parameter exists in the full group processor in the system, however, like priority, this kind of CPU affinity can be by the shell command outside taskset, or by the explicit (explicit system) Adjust ” CPU affinity ” used for operating the task .

SMP task on the multi-processor resources of Fig. 2 is distributed

Erlang and Haskell sitting on a Tree

Last night I was reading the paper Haskell for the cloud [pdf], written by the super kick-ass Haskell guru Simon Peyton-Jones.

Two paragraphs got my eye, and I wanted to share them with you

We use the term “cloud” to mean a large number of processors with separate memories that are connected by a network and have independent failure modes. We don’t believe that shared-memory concurrency is appropriate for programming the cloud. This is because an effective programming model must be accompanied by a cost model. In a distributed memory system, the most significant cost, in both energy and time, is data movement. A programmer trying to reduce these costs needs a model in which they are explicit, not one that denies that data movement is even taking place— which is exactly the premise of a simulated shared memory.

In many ways, failure is the defining issue of distributed computation. In a network of hundreds of computers, some of them are likely to fail during the course of an extended computation; if our only recourse were to restart the computation from the beginning, the likelihood of it ever completing would become ever smaller as the system scales up. A programming system for the cloud must therefore be able to tolerate partial failure. Here again, Erlang has a solution that has stood the test of time; we don’t innovate in this area, but adopt Erlang’s solution

This a really interesting paper if you are a functional programming lover.

The NoSQL Fad

quora.com

Adam D’Angelo[1]:

I think the “NoSQL” fad will end when someone finally implements a distributed relational database with relaxed semantics.

I believe that defining these relaxed semantics will actually lead to figuring out the origins of many of the NoSQL solutions—just as an example, relaxing the relational model would lead to options like the document model or the BigTable-like columnar model.

  1. Adam D’Angelo: Quora Founder  

Original title and link: The NoSQL Fad (NoSQL database©myNoSQL)

“Wednesday 19th July 2000, may not go down in popular culture with quite the same magnitude but it's had a similar impact on internet scale business as the Sex Pistols did on music a quarter of a century earlier, for that was the keynote speech by Eric Brewer at the ACM Symposium on the Principles of Distributed Computing (PODC).”

Brewer’s CAP Theorem

The Intelligent Transport Layer - zeromq

zeromq.org

The socket library that acts as a concurrency framework.

ØMQ (also spelled ZeroMQ, 0MQ or ZMQ) is a high-performance asynchronous messaging library aimed at use in scalable distributed or concurrent applications. It provides a message queue, but unlike message-oriented middleware, a ØMQ system can run without a dedicated message broker. The library is designed to have a familiar socket-style API. (http://en.wikipedia.org/wiki/ØMQ)

Science and technology puts out High Speed Link distributed U serial digital I/O modules to insult China

Science and technology introduces and uses the distributed serial communication technological High Speed Link (HSL) of the high speed to insult China Digital input-outputs U series distributed HSL-DI16DO16-UJ, HSL-DI16DO16-US module and HSL-DI16-UL. The common characteristics of this serial module are that the volume is quite exquisite, can save the disposition space of the machine of the apparatus, and offer the various piecing and facilitate the use of conductor arrangement.

Ling Hua HSL-DI16DO16-UJ and HSL-DI16DO16-US offer 16 passway of the intersection of digital input-output and dispositions, offer mix the intersection of wire connection and difference. HSL-DI16DO16-UJ module offers I/O what has been separated mixes the wire connection, its advantage lies in, when a certain I/O among them click because fail and must be changed, can change individually, and does not influence the operation of other I/O; HSL-DI16DO16-US module offers mixing the end of a thread of ox’s corner-shape, the interstar connection lives in the fixing that the advantage lies in ox’s corner-shape piecing can be firmer, prevent the machine from emerging to shake in operation and cause the interstar connection to loss. Users can choose according to the demand.

Ling Hua HSL-DI16-UL offers 16 passways of digital input dispositions, and have signal delay function of pulses (Pulse Stretcher) . This function can let users set for particular delay time, its range is 1 to 127 ms, when entering module in input signal changed fast, can target and according to the time delayed signal which users presume, utility program can soon, there won’t be time, deal with and omit information this too by the intersection of sensor and input signal, very suitable for using in the high machine of unit’s productive capacity per hour.

Does Ling Hua’s U series module offer Windows? Driver program, until products enclose, present the intersection of LinkMaster and testing software that Hua Ling develop by oneself, test while being convenient except that fault, promote operating efficiency. Please browse through relevant information to insult China’s scientific and technological website: http //www.adlinktech.com/IA.

Heat plugs the function and dispels idle hour

In the application system of adopting the power framework of the fault tolerance, all demand to have heat to plug the function in order to respond to the request that zero shut down. This text introduction adopts the design question that the power module that has heat to plug the function should be considered while making up 48V distributed power structure.


Pieces of this look over full text

Loading more posts...