• Register
  • Help
Page 1 of 2 12 LastLast
Results 1 to 10 of 14

Topic: Tactile Musical Interfaces in a Virtual World.

Share/Bookmark
  1. #1
    Moderator
    Join Date
    Oct 2000
    Location
    Orcas Island
    Posts
    11,454

    Tactile Musical Interfaces in a Virtual World.

    "Touch is the next frontier in the evolution of virtual reality," says Thenkurussi Kesavadas, director of the Virtual Reality Lab at the University of Buffalo School of Engineering and Applied Sciences (close to you Styxx).

    Researchers from the Virtual Reality Lab are developing new tools for transmitting physical touch to the virtual world. The first applications are remote sculpturing. Using a ModelGlove and a virtual clay sculpting system, researchers can replicate (in real time on a personal computer)" the physical act of sculpting a block of clay or other malleable material."

    The force exerted by hand when depressing and shaping a block of clay can be measured and simulated. "Force-feedback information, as well as information on hand position and speed of fingertip motion, is instantaneously communicated to a personal computer" says Kesavadas.

    I wonder if there will be musical applications for this technology. Perhaps the model glove can play on a virtual fretboard of a violin or the keyworks of a bassoon. Haptic (touch-based) virtual instruments may be more natural that a keyboard. The ability to touch a virtual instrument as well as play it may open up new and exciting possibilities.

    Imagine calling up a holographic saxomophone.

    What are your thoughts? Do you think this is sci-fi stuff or that this is the next step in the evolution of musical instruments?

    Gary Garritan

  2. #2

    Re: Tactile Musical Interfaces in a Virtual World.

    I think that this has very strong potential applications but that everyone will underestimate the incubation time, just like they so often do already. I don't necesarily see the technology phasing out keyboards but I do see it being offered up as another alternative that allows current keyboard users another dimension of editing work. In addition it will allow those who prefer to avoid the keyboard paradigm another interface possibility that is more natural for them.

    So in short: It's not the next step, probably not the next step after or ... or after that... but one day it will most likely have a big impact on the options we have when composing digital music.

    Just my 2 cents. Where is my Nostradamus cap when I need it...

  3. #3
    Moderator/Developer Brian2112's Avatar
    Join Date
    Nov 2003
    Location
    Out of my Mind
    Posts
    1,858

    Re: Tactile Musical Interfaces in a Virtual World.

    Interesting. This might sound a little bizarre (nothing new for me huh? ), but imagine this: Virtual gloves that are hooked to a poll that runs horizontally just under they keyboard. Load up any song on a sequencer. The computer sends the performance to the gloves (which guide your fingers) and the hooks (guiding your position on the keyboard). Someone who has never had a music lesson could sit down and play a piano work as performed (virtually recorded) by a famous piano player. Not only does this instruct (learn by doing) but also trains the brain which develops accordingly. Perhaps I should apply for a patent…
    "So what if some parts of life are a crap shoot? Get out there and shoot the crap." -- Neil Peart
    Hint:1.6180339887498948482 Φ

  4. #4

    Smile Re: Tactile Musical Interfaces in a Virtual World.

    Novel approaches such as physical or spectral modeling (such as GSS) will undoubtedly revolutionize the way we make music. Expressiveness will be closer to real intruments, and this is badly needed. There's a price to pay though. We cannot approach these instruments as we are used to do with, say, GPO. We'll have direct, continuous control of many parameters, such as dynamics, vibrato depth and rate, attacks, articulations, portamento, type of legato, release-triggered notes, noise, etc. This requires a substantial number of midi controllers to act upon in real time. Furthermore, we'll need to become proficient in each particular instrument, both theoretically, knowing playing techniques and nuances, and practically, learning how to recreate these sounds with the available means.

    From these considerations, it may appear obvious that alternative or adjunctive approaches to a midi keyboard and a limited bunch of midi controllers are advisable. Proposed solutions range from very exoteric gestural controllers to three dimensional boards, as the continuum fingerboard of Lippold Haken http://namm.harmony-central.com/WNAM...Continuum.html, to tactile pads
    (http://www.thinkmig.com/stcspecs.html), usb joysticks with midi converters,
    and so forth.

    For the time being, in my opinion, a prudent stepwise approach seems advisable. Many of us are keyboard players, and wouldn't easily accept to discard investments and expertise. Integration of additional midi controllers to the available hardware is probably feasible at a reasonable cost. I would strongly favor a usb midi box, equipped with two foot pedals, two wheels and some sliders. In my experience, quite satisfactory results may be obtained by linking the right pedal to dynamics and the left to vibrato control. The left hand is thus free for keyswitching and/or slider adjustement. Real time playing is definitely feasible even with fairly complex instruments.

  5. #5
    Senior Member Styxx's Avatar
    Join Date
    Mar 2004
    Location
    West Seneca, NY
    Posts
    11,075

    Re: Tactile Musical Interfaces in a Virtual World.

    Retrain the brain. A whole new set of thought patterns. Interesting.

    Gary, maybe I'll take a ride up to the old campus and seek out Thenkurussi Kesavadas to discuss the potential impact on the vertual orchestra!

    Hey, as long as I don't start singing, "You know I'm Bad, Bad, You Know It" when I put on one of those gloves!
    Styxx

  6. #6

    Re: Tactile Musical Interfaces in a Virtual World.

    Such technologies could be applied in a broader sense. Picture a sequencer program that looked and functioned like something out of "The Matrix; Revolutions": the control panels of the team that opens the docking doors in the ship bays at Zion.

    Can I have one, please Gary? Please? Can I?

  7. #7
    Senior Member
    Join Date
    Jan 2002
    Location
    UK- teeming with life....
    Posts
    2,567

    Re: Tactile Musical Interfaces in a Virtual World.

    Gary, This is interesting as I was in contact with SensAble Technologies a few years ago as they were one of the first in this field. I was tickled by their clever name and offered an improvement which they liked but never adopted, which was SensAble Touchnologies. The company www.sensable.com have been working in the area of feedback modelling since 1993!

    More recently I have been experimenting with gloved feedback modelling. I was persuaded of its worth by recent events. For those of you interested I reverse engineered a virtual 3D model of Tom Hopkin's head (less moustache) from his photograph (since removed?) which resided alongside 'Easy Winners' in the early days of GPO. (I think Tom may have had his acknowledgement removed (to preserve his reputation amongst his peers) when he discovered the piece had only three chords)

    It has given me great satisfaction feeling his face through my virtual gloves. However I do need to make some adjustments to reduce the level of my applied force as I have broken the apparatus several times. I don't understand why, but I just seem to 'loose' it occasionally. To heighten the experience I have recorded a continuous loop, using an (atonaly) shrill 'Eek' 'Eek' 'Eek' violin, in true vintage 'Psycho' style.

    More good news!!!>>>.Computer Music magazine (which I received today) has finally reviewed GPO....giving it a score of 9/10 which is very good for them. It's a short review but should increase the number of CubaseSX users (popular in Europe). It's a pity that they didn't say "listen to the demos on the Garritan site" .....that would clinch it for many. There was a link to Garritan.com so most will have the nous to have a look I would have thought. The concluding paragraph of the article reads:

    There isn't room here to go into all of GPO's highlights, but we feel sure it will be a huge hit with budding composers who want truly authentic sounds. It provides not only a pro-sounding sample library that covers an entire orchestra, but also performance elements that give it an extra dimension. Pretty impressive given the price. (Pretty fair conclusion too, I think)

    I was getting miffed that no one had reviewed GPO yet in SOS or CM and I was intending to post to that effect. It's high time. It was annoying that one of the magazines had reviewed EWQL Silver/ Gold which came out after GPO. Gary will understand the reason's for this better than me.

    I also saw the first advert for Creative's great new 0404 sound card which sells for £70 here!!!! sub $100 in the US? Brilliant little card with the same high sound spec and hardware effects of its big brothers.....pity about the lack of GS drivers though....or is it? This card is really going to severly hurt some of Creative's competitors.

    Getting back to the Haptic world; I don't see this technology replacing the hardware instrument interfaces anytime soon. Certainly not keyboards as they are available very cheaply already. For wind and stringed instruments you would still need a rigid instrument body with some means of locating the fingers...unless arms and forearms were also mechanically restrained. It sounds as if it would be very difficult, with current technology, to replicate the fingertip sensitivity which traditional instruments provide....certainly at current costs. Nevertheless it's a fascinating area and promises great things in the mid to longer term. Controlling the Theremin is likely to be the most useful application for 'hands in space' for some time.

    Also, I don't like looking stupid !! (but I've had to get used to it!!)

    Frank

  8. #8
    Moderator
    Join Date
    Oct 2000
    Location
    Orcas Island
    Posts
    11,454

    Re: Tactile Musical Interfaces in a Virtual World.

    Giorgio,

    Novel approaches are needed to control more parameters than are available with the modern keyboard. Keyboards are very limiting. Consider what we are trying to do with keys, mod wheels, sustain pedals and a few controllers. We are trying to play dozens of orchestral instruments - each of which can take a lifetime to master. We're trying to mimic what a violinist does with a bow and on the fretboard or how a wind player blows and intonates. Controlling dynamic, bursts, crescendi, attack, accentuation, vibrato rate, vibrato width, portamento, leaps, slurs, legato, and so many different subtle nuances and techniques, all at the same time and all in real time. The keyboard as it exists now simply is too limiting.

    Perhaps new tactile interfaces will be the next evolution. It doctors can perform surgery remotely, perhaps a remote performance by a virtuoso on a virtual violin is in the realm of possibility.

    In the meantime, before we arrive at that brave new world, supplemental and enhanced MIDI controllers along with intelligent programming could give us the tools we need to provide more expressive performances.

    Gary Garritan

  9. #9

    Re: Tactile Musical Interfaces in a Virtual World.

    Quote Originally Posted by Garritan
    In the meantime, before we arrive at that brave new world, supplemental and enhanced MIDI controllers along with intelligent programming could give us the tools we need to provide more expressive performances.
    On the other side, we also need enhanced software which recognizes an extended range of values for MIDI controller data! Values from 0-127 are a limiting factor. The MIDI spec already allows for controllers to use both course and fine values (for a range of 14-bits, or 0 to 16384), but since no controllers acually transmit that full range, software programmers don't bother coding to it...

    ...so why would a hardware supplier create a controller that uses that full range if no software responds to it! As such, I imagine that this cycle will only be broken by a company that comes out with an integrated software/hardware virtual instrument performance package. Only then will third-party software designers start coding for that controller, giving it a bit more momentum in the marketplace.

    - m

  10. #10

    Re: Tactile Musical Interfaces in a Virtual World.

    This is definitely an interesting topic. I do think that touch technology will play a huge role in music technology. But for now, I would gladly settle for some more innovation in making MIDI controllers that simulate essential features of string or brass other than more knobs and sliders.

    How about this? A tabletop controller box consisting of ribbon-type strips for simulating bowing articulation. Imagine moving your left index finger along a ribbon controller that simulates both the pressure and direction of the bow? I think if we had something like that, it would bring us that much closer to the real thing.

    Have you guys ever had thoughts like this?

Go Back to forum

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •