1
RESEARCH POSTER PRESENTATION DESIGN © 2015 www.PosterPresentations.com Our project was successful in generating chords in the form of midi notes. Below are a few transcriptions of our results. As one can see, our program delivers in creating a simple harmonic structure around the melody that makes musical sense. Department of Electrical and Computer Engineering, University of Rochester, Rochester, New York, 14627 Sheng Xu, Albert Peyton, Ryan Bhular MELODY INFORMED MUSICAL CHORD GENERATION USING HMM Abstract Music Theory Our project introduces a melody informed chord generation model using hidden markov models (HMM). We claim that by using pitch information from a melody the HMM should be able to output an acceptable chord progression based both on the note played at the particular instance of the melody and the preceding chord. We assume that one of the three notes in a triad must be the current note in the melody . We trained 2 transition matrices of chordal states, based on a dataset of about 1,300 pop songs and another of 32 Beethoven sonatas (classical). With this information we were able to use the viterbi algorithm in order to calculate the most likely chord progression for our input melody. This chord progression is realized in discrete time according to each observation, and is written to MIDI so to be used in a DAW. Music can be broken down into 2 main parts: melody and harmony. Melody can be thought of as the subject of a song, and harmony can be thought of as giving context to this subject. Melody typically is known to be a single sequence of pitches, while harmony is usually a group of pitches played simultaneously called chords. In basic music theory, chords can be thought of as a collection of 3 pitches called triads. With more advanced music, this is still typically true but with the potential of additional different pitches to give chords different colors or functions. A triad can be visualized below. In our project we look only to recognize chords as triads, with the exception of the V7 dominant chord. With that in mind, if we treat each observed pitch in the melody as a chordal tone, there are only 3 chords that include this chordal tone: either as the root, the third or the fifth. System Model The general Hierarchy of our model can be seen below: Input Audio File (.wav,.mp3,etc) Observed Note Sequence Trained HMM Components Hidden State Sequence (Chords) Pitch Detection/ BeatTracking Viterbi Algorithm Pitch/Key Estimation To determine the pitches (or notes) present in the audio we have used the YIN Algorithm [3]. Each musical key only has 7 available notes. With this prior knowledge a dictionary matrix can be derived with dimension keys x notes. Each row is then intuitively weighted based on what notes are available and how often the corresponding scale degree occurs in that particular key. Multiplying both matrices defined, we can find a matrix containing the probability of each frame belonging to a key. By taking the sum of each row we can find the key with the highest probability. We can determine if the key is major or it’s relative minor based on the distribution of pitches. Beat detection Hidden Markov Model Data Representation: Once the key has been determined we can normalize all pitches to numeric scale degree values (1-7) based on the major tonic or the tonic of the relative minor key. By doing this we can also treat chords in a numeric fashion by only considering their relative chord number for the scale. Transition Matrix (A): We used the McGill Billboard Dataset and the Beethoven sonata Dataset in training our initial transition matrix [1][2]. By counting the transitions of all the songs and normalizing the matrix we can generate our transition space. This transition matrix shows the probability of one chord number to another. This creates a transition space from defined by the probability of one chord number transitioning to another. Emission Matrix (B): With a dimensionality of Observed States x 7 (for each scale degree), the emission matrix aims to map the possible chord choices to accompany the melody based on the note at that time instance. This has been done by using the broad assumption that the note in the melody must be present in the accompanying chord, with probabilities ½, ⅓, and of the melody note being in the root, third, or fifth (respectively). Initial Probability (): Like the emission matrix, we pick an initial probability where this first observation is a chordal tone. However, we weight chords I, IV, and V more than their counterparts as typically a musical phrase will begin with on one of these harmonies. Viterbi Algorithm The Viterbi Algorithm can be used to determine the best possible path for the chord progression. We Define a hidden Markov chain by = (A,B,), where: 1. A is the initial transition matrix of chords 2. B is the initial emission matrix 3. is the initial probabilities for all the states (chords). The initial parameters A, B and are estimated by using the general parameters estimated from the datasets. In addition, we define Y as our observation sequence (chord progression). I II III IV V VI VII 1 2 3 4 5 6 7 Hidden State: Chords Observed State: Scale Degree START We use Dynamic Programming to detect the beats so that logical and evenly spaced chords can be generated. This algorithm uses a recursive approach to estimate the locations of onsets in the musical piece. The score D of each onset can be determined by: Where defines the tradeoff between onset strength and beat, b is the beat instance, Δ is the observed onset strength, and P is a penalty function defined by: With being the rough estimate of beat spacing. With the found beat times, we use the pitches present at these beats to be our observed melody sequence. Results Conclusion In summary, our model worked adequately in determining a chord progression for a sequence of melody notes. Both the classical and pop transition matrices yielded decent results, but the results created with the classical matrix were undoubtedly better. We were successful in achieving our goal, but still recognize that our program could be expanded and improved. Notes (12) Key (12) x Notes (12) Frames = Frames Key (12) W H V C Major: C Major: A minor: A minor: References [1] John Ashley Burgoyne, Jonathan Wild, and Ichiro Fujinaga, 'An Expert Ground Truth Set for Audio Chord Recognition and Music Analysis', in Proceedings of the 12th International Society for Music Information Retrieval Conference, ed. Anssi Klapuri and Colby Leider (Miami, FL, 2011), pp. 633–38 [2] Chen, T. and Su, L. (2018). Functional Harmony Recognition of Symbolic Music Data With Multi-Task Recurrent Neural Networks. Proceedings of the 19th International Society for Music Information Retrieval Conference, Paris, France. [3] de Cheveigné, A. and Kawahara, H. (2002). YIN, a fundamental frequency estimator for speech and music. The Journal of the Acoustical Society of America, 111(4), pp.1917-1930.

Pitch/Key Estimation MELODY INFORMED MUSICAL CHORD ...zduan/teaching/ece477/... · chord progression for our input melody. This chord progression is realized in discrete time according

  • Upload
    others

  • View
    14

  • Download
    0

Embed Size (px)

Citation preview

  • RESEARCH POSTER PRESENTATION DESIGN © 2015

    www.PosterPresentations.com

    (—THIS SIDEBAR DOES NOT PRINT—)DESIGN GUIDE

    This PowerPoint 2007 template produces a 36”x48” presentation poster. You can use it to create your research poster and save valuable time placing titles, subtitles, text, and graphics.

    We provide a series of online tutorials that will guide you through the poster design process and answer your poster production questions. To view our template tutorials, go online to PosterPresentations.com and click on HELP DESK.

    When you are ready to print your poster, go online to PosterPresentations.com

    Need assistance? Call us at 1.510.649.3001

    QUICK START

    Zoom in and outAs you work on your poster zoom in and out to the level that is more

    comfortable to you. Go to VIEW > ZOOM.

    Title, Authors, and AffiliationsStart designing your poster by adding the title, the names of the authors, and the affiliated institutions. You can type or paste text into the provided boxes. The template will automatically adjust the size of your text to fit the title box. You can manually override this feature and change the size of your text.

    TIP: The font size of your title should be bigger than your name(s) and institution name(s).

    Adding Logos / SealsMost often, logos are added on each side of the title. You can insert a logo by dragging and dropping it from your desktop, copy and paste or by going to INSERT > PICTURES. Logos taken from web sites are likely to be low quality when printed. Zoom it at 100% to see what the logo will look like on the final poster and make any necessary adjustments.

    TIP: See if your school’s logo is available on our free poster templates page.

    Photographs / GraphicsYou can add images by dragging and dropping from your desktop, copy and paste, or by going to INSERT > PICTURES. Resize images proportionally by holding down the SHIFT key and dragging one of the corner handles. For a professional-looking poster, do not distort your images by enlarging them disproportionally.

    Image Quality CheckZoom in and look at your images at 100% magnification. If they look good they will print well.

    QUICK START (cont.)

    How to change the template color themeYou can easily change the color theme of your poster by going to the DESIGN menu, click on COLORS, and choose the color theme of your choice. You can also create your own color theme.

    You can also manually change the color of your background by going to VIEW > SLIDE MASTER. After you finish working on the master be sure to go to VIEW > NORMAL to continue working on your poster.

    How to add TextThe template comes with a number of pre-formatted placeholders for headers and text blocks. You can add more blocks by copying and pasting the existing ones or by adding a text box from the HOME menu.

    Text sizeAdjust the size of your text based on how much content you have to present. The default template text offers a good starting point. Follow the conference requirements.

    How to add TablesTo add a table from scratch go to the INSERT menu and click on TABLE. A drop-down box will help you select rows and columns.

    You can also copy and a paste a table from Word or another PowerPoint document. A pasted table may need to be re-formatted by RIGHT-CLICK > FORMAT SHAPE, TEXT BOX, Margins.

    Graphs / ChartsYou can simply copy and paste charts and graphs from Excel or Word. Some reformatting may be required depending on how the original document has been created.

    How to change the column configurationRIGHT-CLICK on the poster background and select LAYOUT to see the column options available for this template. The poster columns can also be customized on the Master. VIEW > MASTER.

    How to remove the info barsIf you are working in PowerPoint for Windows and have finished your poster, save as PDF and the bars will not be included. You can also delete them by going to VIEW > MASTER. On the Mac adjust the Page-Setup to match the Page-Setup in PowerPoint before you create a PDF. You can also delete them from the Slide Master.

    Save your workSave your template as a PowerPoint document. For printing, save as PowerPoint or “Print-quality” PDF.

    Print your posterWhen you are ready to have your poster printed go online to PosterPresentations.com and click on the “Order Your Poster” button. Choose the poster type the best suits your needs and submit your order. If you submit a PowerPoint document you will be receiving a PDF proof for your approval prior to printing. If your order is placed and paid for before noon, Pacific, Monday through Friday, your order will ship out that same day. Next day, Second day, Third day, and Free Ground services are offered. Go to PosterPresentations.com for more information.

    Student discounts are available on our Facebook page.Go to PosterPresentations.com and click on the FB icon.

    Our project was successful in generating chords in the form of midi notes. Below are a few transcriptions of our results.

    As one can see, our program delivers in creating a simple harmonic structure around the melody that makes musical sense.

    Department of Electrical and Computer Engineering, University of Rochester, Rochester, New York, 14627 Sheng Xu, Albert Peyton, Ryan Bhular

    MELODY INFORMED MUSICAL CHORD GENERATION USING HMM

    Abstract

    Music Theory

    Our project introduces a melody informed chord generation model using hidden markov models (HMM). We claim that by using pitch information from a melody the HMM should be able to output an acceptable chord progression based both on the note played at the particular instance of the melody and the preceding chord. We assume that one of the three notes in a triad must be the current note in the melody . We trained 2 transition matrices of chordal states, based on a dataset of about 1,300 pop songs and another of 32 Beethoven sonatas (classical). With this information we were able to use the viterbi algorithm in order to calculate the most likely chord progression for our input melody. This chord progression is realized in discrete time according to each observation, and is written to MIDI so to be used in a DAW.

    Music can be broken down into 2 main parts: melody and harmony. Melody can be thought of as the subject of a song, and harmony can be thought of as giving context to this subject. Melody typically is known to be a single sequence of pitches, while harmony is usually a group of pitches played simultaneously called chords.

    In basic music theory, chords can be thought of as a collection of 3 pitches called triads. With more advanced music, this is still typically true but with the potential of additional different pitches to give chords different colors or functions. A triad can be visualized below.

    In our project we look only to recognize chords as triads, with the exception of the V7 dominant chord. With that in mind, if we treat each observed pitch in the melody as a chordal tone, there are only 3 chords that include this chordal tone: either as the root, the third or the fifth.

    System Model

    The general Hierarchy of our model can be seen below:

    Input Audio File(.wav,.mp3,etc)

    Observed Note Sequence

    Trained HMM Components

    Hidden State Sequence (Chords)

    Pitch Detection/BeatTracking

    Viterbi Algorithm

    Pitch/Key Estimation

    To determine the pitches (or notes) present in the audio we have used the YIN Algorithm [3]. Each musical key only has 7 available notes. With this prior knowledge a dictionary matrix can be derived with dimension keys x notes. Each row is then intuitively weighted based on what notes are available and how often the corresponding scale degree occurs in that particular key.Multiplying both matrices defined, we can find a matrix containing the probability of each frame belonging to a key. By taking the sum of each row we can find the key with the highest probability. We can determine if the key is major or it’s relative minor based on the distribution of pitches.

    Beat detection

    Hidden Markov Model

    Data Representation:Once the key has been determined we can normalize all pitches to numeric scale degree values (1-7) based on the major tonic or the tonic of the relative minor key. By doing this we can also treat chords in a numeric fashion by only considering their relative chord number for the scale.Transition Matrix (A):We used the McGill Billboard Dataset and the Beethoven sonata Dataset in training our initial transition matrix [1][2]. By counting the transitions of all the songs and normalizing the matrix we can generate our transition space. This transition matrix shows the probability of one chord number to another. This creates a transition space from defined by the probability of one chord number transitioning to another.Emission Matrix (B):With a dimensionality of Observed States x 7 (for each scale degree), the emission matrix aims to map the possible chord choices to accompany the melody based on the note at that time instance. This has been done by using the broad assumption that the note in the melody must be present in the accompanying chord, with probabilities ½, ⅓, and ⅙ of the melody note being in the root, third, or fifth (respectively).Initial Probability (𝛑):Like the emission matrix, we pick an initial probability where this first observation is a chordal tone. However, we weight chords I, IV, and V more than their counterparts as typically a musical phrase will begin with on one of these harmonies.

    Viterbi Algorithm

    The Viterbi Algorithm can be used to determine the best possible path for the chord progression. We Define a hidden Markov chain by 𝜃 = (A,B,𝛑), where:

    1. A is the initial transition matrix of chords 2. B is the initial emission matrix 3. 𝛑 is the initial probabilities for all the states (chords).

    The initial parameters A, B and 𝛑 are estimated by using the general parameters estimated from the datasets. In addition, we define Y as our observation sequence (chord progression).

    I II III IV V VI VII

    1 2 3 4 5 6 7

    Hidden State: Chords

    Observed State: Scale Degree

    START

    We use Dynamic Programming to detect the beats so that logical and evenly spaced chords can be generated. This algorithm uses a recursive approach to estimate the locations of onsets in the musical piece. The score D of each onset can be determined by:

    Where 𝝀 defines the tradeoff between onset strength and beat, b is the beat instance, Δ is the observed onset strength, and P is a penalty function defined by:

    With being the rough estimate of beat spacing.With the found beat times, we use the pitches present at these beats to be our observed melody sequence.

    Results

    Conclusion

    In summary, our model worked adequately in determining a chord progression for a sequence of melody notes. Both the classical and pop transition matrices yielded decent results, but the results created with the classical matrix were undoubtedly better. We were successful in achieving our goal, but still recognize that our program could be expanded and improved.

    Notes (12)

    Key

    (12)

    x

    Not

    es (1

    2)

    Frames

    =

    Frames

    Key

    (12)

    W H V

    C Major:

    C Major:

    A minor:

    A minor:

    References

    [1] John Ashley Burgoyne, Jonathan Wild, and Ichiro Fujinaga, 'An Expert Ground Truth Set for Audio Chord Recognition and Music Analysis', in Proceedings of the 12th International Society for Music Information Retrieval Conference, ed. Anssi Klapuri and Colby Leider (Miami, FL, 2011), pp. 633–38

    [2] Chen, T. and Su, L. (2018). Functional Harmony Recognition of Symbolic Music Data With Multi-Task Recurrent Neural Networks. Proceedings of the 19th International Society for Music Information Retrieval Conference, Paris, France.

    [3] de Cheveigné, A. and Kawahara, H. (2002). YIN, a fundamental frequency estimator for speech and music. The Journal of the Acoustical Society of America, 111(4), pp.1917-1930.

    http://www.facebook.com/pages/PosterPresentationscom/217914411419?v=app_4949752878&ref=ts