SongTrellis
Music and Musical Know-how For You

Improvised solos are unnotated at creation but can be transcribed later

When a musician takes an instrumental solo in many forms of popular music, including jazz, rock, blues, and country music, most frequently the musician improvises his solo. The solo is not written out ahead of time. Instead he reacts to harmony and rhythm ideas contributed by the other members of his band, and uses his knowledge of music and his own musical life experience to create something new.

After a solo is recorded, it's possible for an experienced musician to listen to a solo repeatedly and write down music notation in a score to accurately describe the solo so that others can perform it. The act of accurately notating music that a musician has listened to is called transcribing. Scores produced by transcribing are called transcriptions.

Most jazz solos remain unnotated until someone with adequate transcription skill believes that a solo is of sufficient value to be worthy of study and transcribes it.

Transcriptions can themselves by entered into digital scores that can be interpreted by computer software

If a solo transcription is available, it can manually be entered note by note into a score window provided a music editor. This reproduces the notation in a score window so that it can be manipulated by software. An experienced SongTrellis Editor user can type in the notation recorded on a dense page of a transcription, like those used to notate a jazz solo, in 15 or 20 minutes time.

Music can be typed into a music editor nearly as quickly as text can be typed into a word processor

Entering the score into the Editor is the first step, and usually the most lengthy procedure in creating a score animation with SongTrellis. If a solo moves quickly and lasts for several minutes, complete score entry might take two or three hours of typing.

Music editors and sequencers perform music for users so they can hear what its notation means.

Music editors and sequencers can synthesize the music recorded for a score and perform it so the user can listen to it.

Many music editors make some attempt to indicate which notes are sounding during a performance. A common method is to move a vertical line used as a cursor across the music to indicate the performance's progress through the measures of the score. This method suffices to give a hint to trained musicians about a performance's progress but is hard for neophytes and non-musicians to interpret.

The SongTrellis Editor produces detailed score animations instantly whenever a user asks to Play a score, allowing viewers to also see what score notation means

SongTrellis Editor users can play the score they are entering at any point in the process to aurally and visually check their work, by pressing the Play button which appears on every score window.

When the SongTrellis Editor synthesizes and performs a score to Play it, it simultaneously animates the score, to visually demonstrate the individual notes that are sounding at any instant in a performance.

The SongTrellis Editor provides two different methods to animate scores

A user of the Editor can choose one of the two animation methods that the Editor provides at their pleasure and can switch between them an any moment.

The first method, which is easily understood by almost any viewer even if they know nothing about how to read music notation, draws blank music staves as a performance begins and adds newly sounding notes to the staves at exactly the instant they first start to sound in the score.

The second method starts a performance by showing the sheet music notation for the beginning of the score to be performed drawn in black. As notes sound in the score, they are drawn in red at the instant they begin to play. They are returned to black coloring at the instant they stop sounding, as the next notes turn red. This causes a vertical band of reddened notes to move across the score from left to right.

Many trained musicians may appreciate this animation method because it allows them to scan ahead and anticipate what will happen next in the performance.

Neophytes can learn to follow this style of animation, although at first they may have trouble locating the red-colored region of the score, and may have some trouble tracking where the animation continues when it reaches the end of one staff of music and travels to the next. They are frequently confused by this animation scheme at first exposure.

In both animation methods, when all of the staves in a window are filled with notation, which means that the performance is going to play notes that are unseen, the Editor quickly scrolls ahead to make new staves visible onscreen.

In the SongTrellis Editor scores can be saved as QuickTime animations with a MIDI soundtrack

Once a a user completes score entry, has formatted the score properly, and set an appropriate performance tempo, they can create a score animation that is playable by QuickTime by choosing an option to save the score "As a Quicktime Animation" in the Editor's "Save As" dialog.

The Editor renders the animation by drawing each frame of the animation onscreen in a continous stream without hesitation. It stores each frame at exactly the right moment in the QuickTime movie that's being built. It also stores a MIDI performance of the score in the movie. When the image stream for the movie and the MIDI soundtrack are started at the same instant by a QuickTime Player viewers get the impression that the animated score images are causing the MIDI soundtrack to be performed by the user's computer.

How students can profitably use animations with MIDI soundtracks. These considerations guide your choice of the kind of animation you'll choose to record.

Animations created in this manner can be very interesting and valuable to a person who is studying music. If they don't read notation well, they can start to form some idea of how the music is supposed to go by listening to and watching it. Because it is realized in software that provides scroll bars and other ways to set selections within the animation, a student can focus their attention on successive small sections of a score until they memorize or can perform the entire score.

Notes in the animation can be color coded to help a student analyze and understand the musical materials used in a performance. The SongTrellis Editor can melodically color notes in melodies to show the size of the interval jumps made from one melody note to the next. It can harmonically color melody notes and notes within chords to show the harmonic meaning of any note with respect to the root of its accompaniment. This coloring lets the knowledgeable see at a glance whether a melody note will sound consonant or dissonant when its chord accompaniment sounds.

Since the person creating the animation rendering can choose too perform the score at any tempo, they can produce slowed down versions of the animation. This allows the user to hear and see the music at an easier to grasp tempo than that used in the original performance.

Why MIDI soundtracks suffer in comparison with the original recording. Why an a transcribed score animation synchronized to its original performance might be valuable to students.

Almost any listener would recognize the resemblence between the MIDI synthesis of a score and the actual recorded performance that was transcribed to produce the score. But those same listeners will complain about all that is lacking in the synthesized version. The synthesized version will likely sound stiff or have a mechanical quality in comparison to the original performance. The synthesized version will likely not properly represent the dynamics, the usage of sound volume that the soloist employs. Almost none of the expressive devices the soloist might have used (phrasing, swing feeling, vibrato, tone color, accents, pitch manipulations, tiny adjustments of tempo, the soloists placement of notes in relation to the beat) .will be missing from the synthesized version.

Extremely detailed score may attempt to notate these features of the performance, but the resulting score synthesis will nearly always fall short, because scores are necessarily abstractions of the original performance and leave so many details unspecified, making them uninterpretible.

In performance, tempo always varies. It's never fixed. It can never be repesented by an integer measurement.

Even if a score attempts to specify the average tempo that soloist and his band used during a performance, this tempo varies instant by instant from this ideal as the band members slow down or speed up to accomodate their common conception of the flow of time in performance. Also, such tempo markings usually specify that the music should be played at a tempo measured as certain integer number of beats per minute. In truth, if you measure the average tempo of a performance from start to finish, nearly always you would have to calculate a real-valued tempo that is measured to the accuracy of thousandths of beats per minute to properly describe their time flow.

Given a fixed tempo score animation of a transcription how can you create one that synchronizes with the original performance that was transcribed?

Bands usually keep consistent time throughout segments of a performance that can be seconds to tens of seconds in length.

If music performance software can perform real-valued tempos like 128.72375 bpm, as the SongTrellis Editor can,and can switch to a different real-valued tempo at any instant during a performance, it's possible to tightly synchronize it's score animation to the original performance.

Measurements must be made which are used to label a sprinkling of notes in a solo. Performance tempos can be interpolated by the Editor between these notes whose start times are precisely known.

When the interpolated tempo of a section between measured notes matches closely enough with the tempo the band plays in that section, the score animation and MIDI synthesis produced by the Editor will tightly synchronize with the original performance there.

To achieve tight synchronization the animation producer must accurately measure the timings at which a sprinkling of notes in a solo begin with respect to the beginning of a recording. The producer can load the mp3 or wav format version of the original performance into a sound editor like the open source Audacity application. He or she can measure the elapsed time at which the first note of the solo sounds and the elapsed time at which the last note of the solo goes silent.

The producer can use the SongTrellis Editor to add this timing information to the score by labelling the first score note with its start time and labelling the last note with its end time. From this the editor can calculate an average tempo to use to perform the entire score.

Attach orginal performance to score so that score and performance begin to sound at the same instant.

The producer will also attach the original mp3 performance to the transcription score, so that when the Play button is pressed, the MIDI sysnthesized version of the music starts to play at the same instant the mp3 performance plays.

Initial synchronization is almost always very loose. Drifts out of synch quickly. By taking more timing measurements and recording them within the Editor's transcription score, the animation can be synchronized to any necessary accuracy.

When the producer plays the score and watches and listens to the animation that is produce, they will most lilely notice that the first note of the animations begins at the right instant but that the animation rapidly drifts slower or rushes faster than the original performance.

If this happens, the producer measures elapsed start times for more notes from a sprinkling of notes from the middle of the perforamance and labels the corresponding notes with those times. Then the next time the user plays the score, the Editor will calculate tempos to use when the score is performed between labelled notes. When the animation reaches a new labeled note in the performance stream, the editor adjusts tempo so that the next labelled note will be performed at the instant specified by its label.

Usually by measuring the start times of between 5 to 30 notes sprinkled throughout a solo that may contain hundreds or thousands of notes and labeling those notes with those start times in the Editor's score, the producer can achieve close synchrony between a transcription animation and the originally transcribed solo performance.

The time necessary to synchronize an animation of single chorus solo might be only 20 or 30 minutes and may require the recording 4 or 5 note start times. The time to synchronize an animation that lasts for three or four minutes might be as much as three or four hours and may require 20 or 30 start times to be measured and recorded within a score.

Once close enough synchronization is achieved by playing a synchronized score within a score editing window, a QuickTime rendition of the synchronized animation can be produced with the Editor by saving the score as a QuickTime movie.

The QuickTime movie that will be saved will show a flow of notes that will be drawn on their staves at the instant they begin to play, or else a flow of notes that turn red at the instant they sound, depending on the animatiion style chosen. A MIDI track will play which is perfectly synchronized with the animation.

Using the current edition of the SongTrellis Editon, there is still a brief manual processing process which needs to be performed to add the original mp3 performance to the QuickTime movie that is generated. Using Audacity, it's necessary to select the source music for the transcription from within it's larger mp3 peformance, and save that in .wav format. Next, read this .wav excerpt into iTunes or another appropriate translation application. Using iTunes, an mp3 translation of the .wav excerpt is saved.

Using QuickTime the animation produced by the SongTrellis Editor, a .mov file is opened, and the performance excerpt produced by iTunes is also opened and converted into a movie window. The content of the excerpt is selected via the Select All command and copied. Switching to the window which contains the animation and making an insertion point at the beginning of the animation movie, Add To Movie is executed which adds the mp3 solo excerpt to the animation movie file.

When you play this movie, you will see the notes changing in the animation, and you will hear the MIDI track generated by the SongTrellis Editor playing in close synchrony with the sound of the solo performance. The MIDI performance can be removed by executing the QuickTime Player's "Show Movie Properties" command. Three tracks will be displayed wthin this view, the Video Track, which stores the Video animation, the Sound track which records the original solo performance, and the Music track which records the MIDI synthesis. If you click on the Music track, you can press the 'Delete' key which will remove it from the animation movie file.

Once you remove the MIDI Music track, when you play the animation you see the animation playing in synchrony with the original solo performance.

I believe that continued development work will allow me to directly add the MP3 excerpt to the animation move, and remove the MIDI track when a score is saved as a QuickTime animation movie.

After the MP3 performance has been added and the MIDI track has been removed, a final save of the animation is made using the QuickTime Player.

If you need a version of the animation that can be streamed across the Internet, it's probably best to translate the animation QuickTime movie into Flash streaming format. Adobe provides translation utilities that ban make this translation.

Summary of the production process

To summarize, the production method for synchronized animations is:

1) obtain or produce for yourself a transcription of the performance you''d like to animate

2) type the transcription into a SongTrellis Editor score (20 or 30 minutes per page of dense notation)

3) attach original performance to score so that performance and score synthesis sound at the same instant

4) measure beginning elapsed time for first note in the score and the ending elapsed time of the last note played in the performance. Label notes in the score with these times.

5) Play the score and notice how the animation note performance times drift with respect to the actual times where score notes should be played.

6) Until the animation is synchronized enough, measure the elapsed start times of a sprinkling of notes in the solo and label the corresponding notes in the score with those time. As you record more interior note start times and label the coresponding notes in the score, the animation will become more tightly synced. Stop the process when the animation has become tight enough to create the impression that the animated transcription is causing the original recording to be performed. .The time to complete measurement and note labeling can vary between a half an hour for a small one chorus score to several hours for a score that lasts for several minutes.

7) Choose a frame size for your animation. Choose which voices of your score should appear in the video. Choose a coloring scheme if necessary. Choose the animation type. Then render you score as a QuickTime animation. 2 or 3 minutes for each rendering attempt.

8) Open the QuickTime animation movie using the QuickTime Player. Copy the solo excerpt from the original performance, save this as an mp3 file, open the mp3 with QuickTime and add this to the animation as a Sound track. Remove the MIDI music track stored within the animation and save a new copy of the animation which contains only the original solo track excerpt.

8) Convert that QuickTime animation to Flash streaming format