gesture tech


Natural Human-Drone Interaction

Research project from Eirini Malliaraki illustrates ideas for drone programming, from gesture to emotion recognition:

1-month graduate project // Royal College of Art & Imperial College// May 2017

Taking inspiration from the interaction between falconers and their birds of prey, as well as from common daily gestures, cybernetics, dance, and robotics, several themes were explored, namely:
- a gesture-based interaction scheme that attempts to create a more intuitive and natural way to communicate with aerial robots
- ways in which aerial robots can become more autonomous by interpreting their environment in richer ways
- ways in which they can communicate their intentions and give feedback
- ways in which an aerial robot can understand and react to human emotions and eventually influence our behaviour

Parrot AR Drone, Node js, Javascript, Affectiva Emotion analysis SDK



I saw PVRIS live over the weekend and decided to start work on this. Definitely have a part II in mind, so let me know if you’d been keen to read. Lynn Gunn/ Y/N - very much smut ofc. Requests are open so be sure to hmu with any!

Let me know thoughts! x

You could have sworn she was looking at you. You knew she wasn’t, but each time you caught her eyes you could almost see the green in them. You shook away your thoughts, assuring yourself that she was just working the crowd and probably hadn’t even noticed your face at all. She saw so many faces every single day, you were just another fan at another show in another city. Her blonde hair bounced as she smashed away at the symbals, throwing her sticks violently at the kit once the song ended.
“Thank you, that was a new song called Half. It’ll be on our new record that drops this August.” Her voice was so sweet and innocent in comparison to the way she was stomping around the stage, screaming at the front row and rolling her eyes at herself every time she took a step away from the mic. She gestured to the sound tech, the rings on her long fingers sparkling as they caught the light.
“This is a song called Mirrors.” She spoke lightly, beginning to dance as soon as the music started. You wiped away the sweat pooled next to your temple, gazing around awkwardly. You hoped no one had noticed you were at the show by yourself. You were meant to be there with your ex, but she had decided it would be too hard to spend a whole night together since you’d only broken up a month ago.
Lynn was definitely looking at you. She walked right over to the edge of the stage, singing the lyrics directly to you as she gave you a small smirk. You turned around, wondering if maybe she was aiming her looks at a friend or a photographer. You turned back around quickly, not wanting to miss a minute. You stared at her, nearly fainting when she pointed at you, extending her finger and nodding her blonde head as she mouth the word ‘you.’ The air got tighter as you tried desperately to gather yourself. The butterflies in your stomach were bouncing against your ribs, you were unable to look at her anymore. She was killing you. Slowly suffocating you with every passing second, and she was nowhere fucking near you. You locked eyes with your shoes, wondering if maybe you were just going completely insane. Or maybe you were still asleep, dreaming of the night you’d always imagined. You pinched hard at the skin next to the bone of your left wrist, stopping when you saw a dot of crimson appear.
“Shit.” You whispered, shaking your head and returning your sights to the stage. She found you immediately, her teeth glowing as she flashed you a devious smile. You wanted the set to be over, it was impossible to enjoy it at this point. You were losing your actual mind.

Keep reading


Interactive shopping

Fraunhofer HHI has created an interactive gesture recognition system for afterhour windowshopping.

Fraunhofer HHI want to make future shopping trips a special experience by enabling passers-by to operate window displays with hand and facial gestures. Four cameras record the 3-D positions of people´s hands, face and eyes and transform them into commands for selecting and purchasing goods - even after the shop has closed.

via Innovisions

Not That Scary



Aone Takanobu was the kind of guy who terrified most people.  At 6′3″ with broad shoulders and a perpetually scowling face it wasn’t surprising that most people shied away from him before getting to know him.  It didn’t make it hurt any less though.  It was another evening like this after a long volleyball practice that Aone found himself on the train, once again with a wide berth on either side of him while he sat, resting his tired legs.  Practice was long today, going over an extra half an hour.  One stop into his journey he watched a pretty girl walk on while stuffing an apron into her schoolbag, and he mentally steeled himself to get up as she noticed the only free seats were next to him.

You felt exhausted, and gazed around for a seat, finding one on either side of a tired looking, tall boy.  He was surprised as you approached him with a shy smile, tucking a lock of hair behind your ear.  “U-um, is this seat t-taken?”  Shaking his head no and trying to look at least a little less terrifying he scooted over slightly to make a bit more room.  He was transfixed as you sank into the seat next to him, giving him a wider, sweeter smile and bringing with you the scent of strawberries and pastry. 

Your wide hips had your thigh pressed against his, and he noticed you blushing a little embarrassed.  “Sorry if I’m crowding you.”  He shook his head no emphatically, relieved when you relaxed once again, looking up at him cutely.  “S-so you’re a student at Date Tech?”  You gestured at his uniform, and he nodded, holding up two fingers.  “Oh!  I’m a first year there, but I work after school s-so I don’t get to do any club stuff.  I thought you looked familiar, y-you’re on the volleyball team aren’t you?”  He nodded again, looking proud. 
“I-I’m _______ _______, I know you don’t talk much, b-but it’s nice to meet you.”

He nodded emphatically, patting your hand hurriedly before pulling back nervously, afraid he’d been too rough.  He was glancing between your hand and face, and you managed to decipher that he was wondering if you were okay.  “O-oh!  I’m fine…” you went to say his name but stopped, the sentence falling off as you realized you had no idea what it was.  You blushed darkly again, embarrassed, plucking at your pants in nervousness.

“Aone Takanobu.”  His voice was low and gravelly, and you looked at him surprised.  He was flushed lightly, looking away and rubbing the back of his neck.  He looked back at you, relaxing a bit at your wide, awed smile. 

“Ah!  It’s really nice to meet you Aone-kun!  Thank you for letting me sit next to you.”  It was your turn to pat his hand, and he flushed again, nodding shakily.  What you didn’t know is he wanted to thank you for sitting next to him, he wanted to know if you’d be willing to do it again.  He decided he’d stay late every day if it meant having someone sit next to him, if it meant getting to know a pretty girl who didn’t seem put off by his scowling face.

It seemed too soon when you finally stood, a stop before his.  He barely registered you pulling out a slip of paper and scribbling on it before hurriedly shoving it into his hand and rushing off the bus with a shy wave and quiet goodbye.  Momentarily he scowled, wondering what he could have done to scare you off after what seemed like such a…nice encounter.  Finally pulling out your hasty note he blushed deeply and read it over and over again until his stop finally came.

Thanks for keeping me company Aone-kun, if you don’t like talking maybe we can text?  _______-chan

He didn’t even mind when everyone else on the train chose to remain standing than sit next to him for the next few minutes.  Pulling out his phone he composed a quick text message, sending it before he could think better of it.



Getting Smartphones To Understand In-Air Gestures

by Michael Keller

Sometimes getting your smartphone to do what you want requires feats of phalangeal gymnastics. And other times, a bag in one hand and a not-quite-long-enough thumb on the hand holding the device mean turning gymnastics into juggling.

Now researchers at Microsoft and the Federal Institute of Technology in Zurich, Switzerland, (ETH) say they’ve got a solution to your manual mauling of that otherwise user-friendly piece of electronics. They’ve developed an app that harnesses the phone’s camera to translate hand gestures in front of it into input commands. 

Pinching and opening the thumb and pointer finger can open a magnifying glass on a map. Making the gun gesture can advance a page in a document or fire the weapon in your video game. See the video below.

Keep reading

A thought about Tony Stark, Edwin Jarvis, and Vision

So we’re all pretty sure that Tony and Edwin Jarvis were close, that Jarvis basically raised Tony. After all, Tony named his best AI system after Jarvis, has a stocking for JARVIS that from the looks of it was originally intended for Edwin Jarvis. Jarvis meant everything to Tony, at least as much as his AI did. That JARVIS was there for Tony through everything, his friend as much as his program, who knew every single one of Tony’s secrets, cared about him (as much as an AI can) , took care of him and held Tony together as much, if not more, than his human friends and loved ones. 

Cue Age of Ultron, cue the ‘death’ of JARVIS, cue the return, the sacrifice to amend his mistake the only way he can think of to do, create of Vision. and if he expected it to be Jarvis in a body, well, obviously that didn’t happen. In a way, Tony kinda of killed his best friend (apart from Rhodey of course) to save the world from his mistake, from his fear manifesting into worldwide terror, from his perceived failings materializing into his biggest failure yet. What is the loss of a computer program compared to fixing all that? When it comes to the world vs Tony Stark’s happiness, well, the world’s been winning that battle since Yinsen sacrificed himself so Tony could escape. But still, there are nights Tony’s working late and he forgets and calls for JARVIS only to have FRIDAY answer, nights that Pepper’s away, that he’s alone in the workshop and it hits him how few people he has, how poorly he’s done keeping them safe. Not even his AI was safe from him and his mistakes and his ego, no, his belief that surely if he can create an Iron Man suit out of scrap metal and a blow torch, that if he can make an arc reactor out of a car battery, that he can protect the ones he loves most. 

But then there’s Vision. Vision who calls him Mr. Stark, who bears JARVIS’ voice, who wears collared shirts and sweaters and crosses his legs when he sits. Vision who cooks to cheer people up, who misunderstands what an open door means. Vision who doesn’t need to sleep and sometimes encounters Tony working late on everyone’s newest tech improvements–trying to keep them safer, to make them everything they need to be to come home every time–and silently sets a sandwich down by Tony’s elbow, or a cup of coffee, or maybe just gently reminds him that humans function far more adroitly and at their highest capacity when they’ve had at a minimum, 8 hours of sleep. 

And in those moments Tony sees glimpses of JARVIS, but more strongly, he sees glimpses of Edwin Jarvis. Moments like that Tony can’t tell who he misses more and just claps Vision on the shoulder with a careless remark that isn’t careless at all. But he always accepts the offering whatever it may be–sandwich, coffee, reminder–because he remembers when he didn’t, and he recognizes now what the gesture really is: an expression of love. He’s passed the days of scoffing at those. They are too few and precious. And in return he pours more effort into his own gestures of love: the tech they all wear, the shirts and slacks he buys for Vision, the Sokovian foods he imports for Wanda, the gadgetry and vinyl tracks for Sam, the old photograph his father gave him of Steve and Bucky slinging their arms around each other for Steve. 

Maybe they all see it as him showing off his wealth, maybe they don’t, but it’s the only way he knows anymore how to say, “I love you.”

Intel “Make It Wearable” Finalist BLOCKS Wants to Make Smartwatches Modular

The smartwatches on the market today come pre-packaged with features that can’t change unless they are entirely on the software side. If you bought a watch that doesn’t have a heart rate monitor and now want to track that data you will have to go out and get a brand new one. This is exactly what BLOCKS, a team from the Intel “Make It Wearable” challenge is trying to avoid. 

Keep reading


Leap Motion Hack 02 - DirectX 11

Coding project by evvvvil is a real-time generative music visualizer controlled with hand gestures via a Leap Motion sensor - video embedded below:

Made in vvvv, HLSL and C# by Mangosh Prunier aka evvvvil. 100% audio-responsive real-time generative 3d graphics, no post-production, entirely controlled by leap motion.
Features: Custom audio responsive extrude geometry shader, custom audio-responsive particle system with geometry instancing & particle rotation, dynamic animated cube maps, multi-pass rendering with correctly occluded glow, real-time text effects, 3 track audio analysis.
How it works: Keytab gesture to start/pause extrusion, circle gesture to rewind. Red cubes react to DRUMS, extrusion reacts to BASS and blue circles react to NOISE.



Mi.Mu Glove for Music

Imogen Heap’s Kickstarter for a gestural performance glove to make music is 4 days away from it’s deadline (and only halfway through it’s goal):

Most of us on our small team are musicians who are tired of being stuck behind computer screens, keyboards, faders, knobs, and buttons to make our music.  We feel there could be a better way that is more like the experiences we have with traditional instruments: using the dexterity and mobility of the human body. 

Imagine, for example, that instead of turning up a fader in order to bring in a sound or add reverb, you could be raising your arms to achieve the same effect. Or to move a sound around the room, you could simply point where you want it to be.  Not only is this much more intuitive, it is also more enjoyable to watch, making it easier for your audience to connect with what you’re doing. Our aim is to break down the barriers between musicians and machines, and between performers and audiences. Every musician and/or performer will know the bane of their existence is to have to carry, or worse, ship vast amounts of technological equipment to whatever destination they are playing. The gloves are a compact, lightweight and self-contained system requiring little more than a laptop to function fully. 

More Here

While the project is a great idea, I can’t help think the goal was too high to be realistic - the amount seems to give the creators too much comfort, and the device could certainly be applied to fields outside of music … oh well … It is still an attractive project for it’s potential …