In this activity, we play notes on a score sheet using image processing and scilab (I know, tones from programming languages are usually pure tone, they do not emulate real world instruments. Anyway, still its cool). The score sheet that I used is a score sheet of a common song, “Twinkle Twinkle Little Star”.
Figure 1. Score sheet from mamalisa.com. One of the coolest tones ever!
We read the image of the score sheet in scilab and do the usual (imread, rgb2gray, etc). The next thing that we do is crop a part of the image shown in figure 2.
Figure 2. In case you’re wondering what this is, this is the oval in the end of a half note.
What we are trying to do here is that we want to obtain the coordinates of the notes, at least one coordinate of each note whose qualification is consistent. This is because the row of the coordinate will be used to obtain the tone and the column used for the duration and sequence of the notes.
So what we are going to do next is to edit figure 1 such that the notes are sequential. To drive this idea home, I show figure 3.
Figure 3. The notes are now sequential. This is to facilitate better sequencing of notes.
We then do template matching of figure 2 and figure 3. After thresholding, we obtain the relative position of the notes. This works because all the notes are quarter and half notes so figure 2 is found in all the notes that are present.
Figure 4. The white dots are the position in which the template matching has the highest values.
We obtain the position of the white dots in figure 4, the x being the tone and y being the sequence and the length of each note. From figure 1, the notes are in the key of C. Therefore Do is C, Re is D, Mi is E, etc. Using the table from http://www.phy.mtu.edu/~suits/notefreqs.html, we can convert the x-axis into sinusoids. Using the sound function, we can play the notes. I currently cannot upload the file.
The author gives himself a grade of 9/10 for this activity.
Wynn186 