Realtime Convolution FULLTEXT01

  • Published on
    07-Jan-2016

  • View
    6

  • Download
    0

Embed Size (px)

DESCRIPTION

Realtime Convolution FULLTEXT01

Transcript

  • Real-time Convolution of Two Unknown Signals for Use in a Musical Context

    Antoine Henning BardozLars Eri Myhre

    Master of Science in Electronics

    Supervisor: Jan Tro, IETCo-supervisor: Tor A. Ramstad, IET

    Sigurd Saue, IMyvind Brandtsegg, IM

    Department of Electronics and Telecommunications

    Submission date: June 2013

    Norwegian University of Science and Technology

  • NORWEGIAN UNIVERSITY OF SCIENCE AND TECHNOLOGY

    AbstractFaculty of Information Technology, Mathematics and Electrical Engineering

    Department of Electronics and Telecommunications

    Master of Science

    Cross Convolution of Live Audio Signals for Musical Applications

    by Antoine Henning Bardoz

    Lars Eri Myhre

    This thesis proposes a method for convolution of two real-time audio signals, for

    use in live performances or post-production. In contrast to traditional convolu-

    tion techniques, which require a predefined impulse response as one of the input

    signals, our method allows for convolution of two continuously updated, and un-

    known, signals, allowing two musicians to shape each others timbral and temporal

    contributions.

    The aim was to create an effect that sounded like convolution, offered low output

    delay, as well as giving satisfying feedback to musicians. To achieve this, a hybrid

    of time- and frequency domain techniques has been used, offering the low output

    delay associated with the time domain, and the low CPU load characteristic of

    FFT-based frequency domain processing. To deal with the limitations inherent in

    convolution, namely that to perform ideal convolution of two unending signals, an

    infinite amount of memory and processing power are eventually required, transient

    detection has been applied to segment the signals in a musically relevant way. The

    transient-assisted segmentation also makes the effect more intuitive for users, as

    it increases the users ability to interact rhythmically.

    A GUI was developed, and the effect was implemented as a VST plug-in, to allow

    users to easily apply the effect in DAWs.

    The effect was prototyped in Matlab, and later implemented in Csound and C,

    using the Cabbage framework for the VST.

  • NORGES TEKNISK-NATURVITENSKAPELIGE UNIVERSITET

    SammendragFakultet for informasjonsteknologi, matematikk og elektronikk

    Institutt for elektronikk og telekommunikasjon

    Master i elektronikk

    Krysskonvolusjon av sanntidslydsignaler til musikalske anvendelser

    by Antoine Henning Bardoz

    Lars Eri Myhre

    I denne oppgaven foreslas en fremgangsmate for konvolusjon av to sanntids lydsig-

    naler, til bruk i live-opptredener eller post-produksjon. I motsetning til tradis-

    jonelle konvolusjonsteknikker, som krever en forhandsdefinert impulsrespons som

    ett av inngangssignalene, tillater var metode konvolusjon av to kontinuerlig opp-

    daterte, og ukjente, signaler, slik at to musikere kan forme hverandres klanglige

    og tidsmessige bidrag.

    Malet var a skape en effekt som hres ut som konvolusjon, tilbyr lav utgangs-

    forsinkelse, og gir tilfredsstillende tilbakemelding til musikere. For a oppna dette

    har en kombinasjon av tids- og frekvensdomeneteknikker blitt brukt. Dette kom-

    binerer lav CPU-belastning, takket vre FFT-basert frekvensplanprosessering,

    med den lave forsinkelsen assosiert med tidsdomenet. For a handtere begren-

    sningene forbundet med konvolusjon, nemlig at for a utfre ideell konvolusjon av

    to uendelige signaler, kreves det etter hvert uendelig minne og prosessorkraft, har

    transientdeteksjon blitt brukt til a segmentere signalene pa en musikalsk relevant

    mate. Segmentering ved hjelp av transienter gjr ogsa effekten mer intuitiv for

    brukerne ved a ke deres evne til a samhandle rytmisk.

    Et grafisk brukergrensesnitt ble utviklet, og effekten ble implementer som en VST

    plug-in, slik at brukere enkelt kan benytte effekten i DAWer.

    Effekten ble prototypet i Matlab, og senere implementert i Csound og C. Cabbage-

    rammeverket ble benyttet for VST-implementasjonen.

  • Acknowledgements

    We would like to extend a special thanks Sigurd Saue for giving us valuable sug-

    gestions and technical insight, without which we would truly have been lost.

    In addition we would like to thank Jan Tro for keeping music alive at Glshaugen,

    and making this all possible; yvind Brandtsegg for technical and artistic insight,

    as well as tips from a users perspective; and Tor A. Ramstad for signal processing

    guidance.

    A special thank you goes to Rory Walsh for developing Cabbage and for being ex-

    tremely helpful through the forum at www.thecabbagefoundation.com. We would

    also like to thank the Csound community for developing Csound and for quick and

    crucial help through the Csound developers list.

    For wasting our time with mindless babble and keeping us sane, we thank our

    study hall companions Thomas Christiansen, Niklas Skyberg, Bendik Paulsrud,

    Jrund Kaarstad Dahl and Rune Svensrud.

    For their musical contributions, we thank Thomas Etholm-Kjeldsen, Jakob Eri

    Myhre and Olaf Mundal.

    Antoine would like to thank Lars for truly giving his all during this semester,

    contributing heavily every step of the way, coming up with important ideas, and

    remaining motivated, as well as motivating, until the very last minute.

    Lars would like to thank Antoine for a partnership which will not be forgotten.

    His knowledge in signal processing, programming and music has been infectious.

    His effort has been remarkable.

    iii

  • Contents

    Abstract i

    Sammendrag ii

    Acknowledgements iii

    List of Figures vii

    Abbreviations x

    Symbols xi

    1 Introduction 1

    1.1 Problem Description . . . . . . . . . . . . . . . . . . . . . . . . . . 2

    1.2 How to Read This Thesis . . . . . . . . . . . . . . . . . . . . . . . . 3

    2 Theory 5

    2.1 Convolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

    2.1.1 Time Domain . . . . . . . . . . . . . . . . . . . . . . . . . . 5

    2.1.2 The (Circular) Convolution Theorem . . . . . . . . . . . . . 6

    2.2 The Fast Fourier Transform and Frequency Domain Multiplication . 7

    2.3 Theoretical Foundation for Real-Time Blockwise Convolution . . . . 8

    2.4 Transients and Transient Detection . . . . . . . . . . . . . . . . . . 12

    2.5 Latency Tolerance for Musicans . . . . . . . . . . . . . . . . . . . . 13

    3 Development Tools 14

    3.1 Matlab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

    3.2 Csound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

    3.3 Cabbage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

    4 Algorithm 17

    4.1 Preliminary Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 17

    4.1.1 Short Description . . . . . . . . . . . . . . . . . . . . . . . . 18

    4.1.2 Buffer Up Signals . . . . . . . . . . . . . . . . . . . . . . . . 18

    iv

  • Contents v

    4.1.3 Convolution Computation . . . . . . . . . . . . . . . . . . . 20

    4.1.4 Put Convolution Result on Output . . . . . . . . . . . . . . 20

    4.2 Algorithm Version 1 . . . . . . . . . . . . . . . . . . . . . . . . . . 21

    4.2.1 Short Description . . . . . . . . . . . . . . . . . . . . . . . . 22

    4.2.2 Buffer Partitioning . . . . . . . . . . . . . . . . . . . . . . . 23

    4.2.3 Cross Convolution of a Segment . . . . . . . . . . . . . . . . 24

    4.2.4 Output Buffer . . . . . . . . . . . . . . . . . . . . . . . . . . 25

    4.3 Algorithm Version 2 (Transient Detection) . . . . . . . . . . . . . . 25

    4.4 Algorithm Version 3 (Parallel Processes) . . . . . . . . . . . . . . . 27

    4.4.1 Alternative 1: ThrowAll (Used in Final Implementation) . . 28

    4.4.2 Alternative 2: ThrowLast . . . . . . . . . . . . . . . . . . . 29

    4.4.3 Alternative 3: TwoProc . . . . . . . . . . . . . . . . . . . . . 30

    4.4.4 Normalization . . . . . . . . . . . . . . . . . . . . . . . . . . 31

    5 Results 34

    5.1 Preliminary Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 34

    5.2 Algorithm Version 1 . . . . . . . . . . . . . . . . . . . . . . . . . . 38

    5.3 Algorithm Version 2 . . . . . . . . . . . . . . . . . . . . . . . . . . 42

    5.4 Algorithm Version 3 . . . . . . . . . . . . . . . . . . . . . . . . . . 44

    5.4.1 ThrowAll (Final Algorithm) . . . . . . . . . . . . . . . . . . 44

    5.4.2 ThrowLast . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

    5.4.3 TwoProc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

    5.5 Graphical User Interface . . . . . . . . . . . . . . . . . . . . . . . . 50

    5.5.1 Gain Knobs . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

    5.5.2 Transient Detection Section . . . . . . . . . . . . . . . . . . 51

    5.5.3 Convolution Section . . . . . . . . . . . . . . . . . . . . . . 52

    6 Discussion 53

    6.1 Preliminary Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 54

    6.1.1 Why the Preliminary Algorithm Fails . . . . . . . . . . . . . 54

    6.1.2 Independent Buffer Sizes, Overlap on Output and Fading ofOverlap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

    6.2 Algorithm Version 1 . . . . . . . . . . . . . . . . . . . . . . . . . . 56

    6.2.1 Delayed Change . . . . . . . . . . . . . . . . . . . . . . . . . 57

    6.2.2 Indistinct Transients . . . . . . . . . . . . . . . . . . . . . . 58

    6.2.3 Destructive Interference . . . . . . . . . . . . . . . . . . . . 58