Modeling Rests in Composed Music

There are at least two types of rests in music. The first are the ones the composer wrote into the score. The second are the ones that naturally occur during playing. Musicians pause, extend, chop, attack, cheat, and move notes around within the measure in order to emote, interpret, or express.

There are many more of the second type of rests in human-performed music than the first. The first type are represented on the score, but the second type makes an enormous difference in how the music is perceived stylistically. Recognizing, categorizing, and modeling both types of these rests is a goal of Organ Donor, with the expectation that introducing proper amounts of "space" into algorithmically produced music will create music that sounds more like a human is playing it. Being able to create different models of resting based on desired style would be a very powerful and useful result.

Another area of investigation is the minimum return distance from root notes, or Mean First Passage Times. I suspect that there might be some utility from these statistics in terms of creating believable phrasing - or uncovering patterns that reveal other hidden structures in composed music. Examining the minimum return distance for both types of rests as well as notes will help improve the understanding of the role and effect rests play in composition and style.

This area of math (MFPT) is used in a wide variety of fields to answer pragmatic questions about physical systems. There's no guarantee that it will provide repeatable or useful results in music, but we have the tools (thanks to the python library pykov) to start the process of finding out.

Here's some results from parsing MIDI files of performed music. The rests captured don't exist in the score. Some notes are staccato, and you can see the additional "distance" the musician included to create the desired effect. You can also see additional distance punctuating the end of the phrase. This is the violin 2 part from Beethoven's 7th, second movement.

Note 64 had tick duration 409
a rest had tick duration 66
Note 64 had tick duration 110 (staccato note)
a rest had tick duration 138 (results in longer space between notes)
Note 64 had tick duration 116 (staccato note)
a rest had tick duration 121 (results in longer space between notes)
Note 64 had tick duration 410
a rest had tick duration 67
Note 64 had tick duration 392 (ended a bit early, end of a phrase)
a rest had tick duration 91

Modeling music requires obedience to aesthetics, and this is where the difficulty - possibly the impossibility - lies. However, I cannot think of anything more worthy of analysis, modeling, machine learning, and algorithmic design. More soon!

Organ Donor opus 1 at Borrego Days Desert Festival

Two Sol Diego Artists Collaborative projects that previously appeared together at YouTopia 2014, Sol Pyramid and Organ Donor, were successfully installed for the weekend at the Borrego Days Desert Festival in Borrego Springs, California.

Held 24 – 26 October 2014, the 49^th annual festival celebrated the people and culture of Borrego Springs.  Live entertainment, astronomy demonstrations, contests, shopping, food, and art were features of the event, with a very popular parade rolling down the center of town on Saturday morning.

Borrego Springs is located in the middle of Anza Borrego State Park. It’s an unincorporated area of San Diego County with a long and colorful history. First homesteaded in 1875, busy during World War II due to war efforts activities, favored by Hollywood stars as a getaway during the 1950s, Borrego Springs became California’s first Dark Sky Community in 2009.

Sol Pyramid lead David Timms was invited to bring the Pyramid to Borrego Days after organizers saw the video of the Pyramid from YouTopia. Organ Donor enthusiastically agreed to accompany the Pyramid as an installation.

The site was located on the main street a short walk from Christmas Circle in downtown Borrego Springs. The lot was cultivated three years ago as an urban garden, with plantings of Ocotillo and Pallas Verde trees. The trees are well established and provide an extremely pleasant site. Efforts were made during the installation to preserve and protect the trees and the site in keeping with common sense and the Burner ethic.

The Pyramid construction began at 6pm Friday evening and lasted until 2am Saturday morning. Organ Donor construction began at 8:00am Saturday and concluded at 2:00pm Saturday. Programmed LED lighting repurposed from a 2012 Sol Diego project was installed on the Pyramid structure before sunset on Saturday. This represents decreased Borrego Days construction time for both projects compared to YouTopia.

Hours of Construction Time

Project	YouTopia	Borrego Days	Percent Change
Sol Pyramid	16	8	-50%
Sol Pyramid Lighting	3	2.5	-17%
Organ Donor	10	6	-40%

The liaison for communications about Borrego Days was David Timms. David provided substantial assistance and support to Organ Donor in terms of housing and offered reimbursement for the unplanned travel expense of renting a UHaul truck when Organ Donor’s tow vehicle failed Friday morning. David reported excellent communications from Borrego Days organization.

The construction site was essentially identical to the site at YouTopia, being level packed humus-free earth and small stones, directly accessible from a paved road.

Organ Donor worked well throughout the event, aside from a few fuses blown when participants tried to play too many notes simultaneously (just as at YouTopia). One of the eight strands of LED lighting failed to light up after installation, and on-site debugging attempts were unsuccessful.

The relatively short notice prevented substantial or coordinated advance advertisement. The groups that potentially missed out, according to the members of the public that visited the installation, were school groups and bands that participated in the parade. With more advanced notice, collaboration could have potentially moved the encounter with the art from being entirely within the “delightful surprise” category to occupy a bit more of the “planned presentation” rubric. However, this distinction is not necessarily an improvement. The sheer surprise of seeing a large LED-illuminated Pyramid arise in the center of town, and then disappear only two days later, is in keeping with the spirit of the endeavor and was entirely successful in surprising and delighting the steady stream of visitors.

And, surprised and delighted they were. Reaction was overwhelmingly positive. Visitors were welcomed by installation staff during the entire time the installation was deployed, with extended hours on Saturday evening.

Serving as docents were Organ Donor Paul, Organ Donor Abraxas, and Organ Donor Richard. Education about organ pipes, blower boxes, windchests, software, embedded processor control, machine learning, and direct electric action was provided.

Visitors were present in small groups and couples for almost the entire time the installation was open. Foot traffic came by at such a rate that extended conversations were possible. Multiple contacts were made involving potential future involvements and opportunities for Sol Pyramid as well as Organ Donor. Sol Diego made a very successful outreach into a non-burner community by participating in this festival.

The most significant contacts in terms of future opportunities were with the chair of NWEAMO festival, a teacher involved in founding a new Maker-based school in Escondido, a burner named Wobbly who is interested in founding a burner event on private land in the desert near Borrego Springs, and the chair of the Daffodil festival in Julian.

The next planned adventure for Organ Donor is the STEAM Festival at Del Mar Fairgrounds on 6 December 2014.

Video: Organ Donor opus 1 debut at YouTopia 2014

link to recording

Organ Donor Opus 1 at YouTopia 2014

YouTopia is the official San Diego Burning Man Regional event. Held annually in October at the La Jolla Indian Campground, it draws several thousand people each year to a festival of music and art. A Center Camp, dozens of theme camps, individual camping, and RV camping are placed along the narrow lacework of unpaved roads within the campground. 

The part of San Diego where YouTopia is held is located in the foothills of Palomar Mountain. There are trees and seasonal streams, poison oak and bees. Nighttime skies are very dark despite being well within striking distance of San Diego. Overnight temperatures are lower than near the coast with daytime temperatues largely equivalent. Mist, fog, and rain commonly occurs during the festival from hill effect, but is usually light. 

Sol Diego artists collective (http://www.sol-diego.org) applied for and was placed as a theme camp. YouTopia city planning divided the event into areas based on, among other factors, how loud the sound camps were intended to be. We were located in Darwin, near the entrance. This was intended to be one of the quieter areas, and it worked out that way for us. 

Sol Diego Theme Camp provided the Sol Pyramid, Organ Donor,  and the colaser cutter workshop by Paul with demonstration of work products. Brandie Maddalena presented her program of You Are The Art at YouTalks. Brandie was also a lead for the YouTopia Temple project, which was constructed at colab and built for YouTopia.

Early entry was on Wednesday, 15 October 2014. We arrived with Organ Donor packed in the travel trailer and tow vehicle in the evening. Part of Sol Pyramid has already arrived. The remainder arrived Thursday, 16 October. The pyramid was completed late Friday afternoon, 17 October. Physcial construction of Organ Donor took from late Friday afternoon until 1:00am Saturday, 18 October. Configuration of the MTPs, the controllers that translate MIDI commands into signals for the solenoids, took about 90 minutes on Saturday morning, 18 October. This step is necessary in order to map the randomly assigned pipes to the holes in the windchest. 

This shortened schedule was not ideal, as it gave only a day and a night of demonstration. The idea of arriving early, on Wednesday, was to set up and get several days worth of experminatation, modification, and feedback. The intended schedule was not met due to the size of the pyramid and the dependence on a mechanical lift to construct it on site. 

Despite the limited time, several hundred people came to the pyramid and played with, listened to, and talked about the Organ Donor. The feedback was valuable, and the project received favorable reviews. 

Organ Donor opus 1 as deployed at YouTopia 2014 had an eight-foot and a four-foot rank. The upper keyboard, in the pair of keyboards, played the four-foot rank. The lower keyboard played the eight-foot rank. Organ Donor Paul created a console and a controller that allowed registration to be selected by the operator. By pressing LED-backlight buttons, the operator could select single notes, select notes plus an octave above, select notes plus an octave below, select notes plus an octave above and an octave below, and crosslinking to the other rank. The console case was cut out of quarter-inch plywood on the colab laser cutter.

The pipes were not tunable as deployed. They were cut to calculated length and their pitches measured. Results were recorded in a spreadsheet. Results showed that the pipes were all consistently flat. When played together, some intonation issues were heard, but the overall tuning effect was acceptable.  

A small number of pipes did not play (were silent) at the designed wind pressure. The pipes were correctly connected and the solenoids observed to be working. All of these pipes could be manually played, but were quiet or required substantially more pressure to sound. It was therefore not suprising that the regulated pressure from the windchest did not sound them. These pipes will probably have to be re-cut.  

The smallest pipes had the most fallout, with the 4-foot rank suffering the most attrition in the highest notes. All pipes could be manually played, but at 2 inches of wind pressure, several were unexpectedly silent.

The largest pipes were quieter than expected but all successfully sounded.

During windchest construction, ciphers (valves that don’t completely close) were observed and tracked in a spreadsheet. Efforts were made to improve the seal between the pallet pad and the hole of the windchest. This greatly reduced, but did not completely eliminate, the number and severity of the ciphers. 

During deployment, when ciphers were identified, larger pipes were assigned to the ciphers. Since the volume of air required to sound larger pipes is larger than the volume of air required to sound smaller pipes, putting larger pipes on the ciphers succeeds in reducing or eliminating the irritating sound that a constantly playing pipe can cause. Two holes were not successfully reassigned, so the pipes were dropped from the ranks. 

During MIDI file playback, where the MIDI commands come from a file rather than an operator playing the MIDI keyboards, problems were observed. We saw that there were notes that stayed on. After printing out messages to the console and comparing the messages sent from the file to the messages interpreted by the console controller, it was observed that, occaisionally, “note off” MIDI messages were not appearing. The root cause was not identified until after returning to San Diego. The missing messages were from what appeared to be a misconfiguration between the MIDI conversion cable and the MIDI driver, and not code in either the computer playing the files, or the Arduino at the heart of the controller that arbitrates which MIDI messages are passed through to the MTPs. 

Fuses blew several times when operators played large numbers of notes. The fuse was set to a conservative level in order to protect the equipment. While easy to change out, it did interrupt a recital of Phantom of the Opera and a wide-ranging version of Heart and Soul among other works. The number of simultaneous notes allowed to be played is controlled by the Arduino arbitration controller in the console. The fuse selection for the weekend was below this software limit. It was not expected that the current consumption would reach this level, and observations of the meter supported this assumption. It is quite possible that startup currents were the reason for fuse wear. A review of current consumption controls, power supply design, and protection circuitry is ongoing.

Operators greatly enjoyed playing the Organ Donor opus 1. The instrument was busy the entire time it was available, all day Saturday and late into Saturday night and early Sunday morning. Operators were enthusiastic and spread the word. The demogrpahics of the event resulted in a large number of trained keyboard players, music theoreticians, other makers, and interested programmers. Outreach from Sol Diego Organ Donor to the local Burning Man community was successful. 

It was decided that the next deployment would be Borrego Days, held the following weekend in Borrego Springs, CA.

Independent Parallel Tracks and Hidden Markov Models

OK, so now we have code that analyzes each track of a multi-track midi file, and creates transition tables. These transition tables are used to generate new music that has some influence from the analyzed track. 

However, each new track is used independently. While the behavior of each new track is based completely on the statistics of the analyzed track, they will not sound like they are "composed together” when they are recombined. 

Organ Donor Frank Brickle says, "To keep them together you actually need to model the interaction in some way. Looking ahead a bit, you can probably see why a hidden Markov model of all the activity is one of the best ways of coordinating the subordinate parts."

Ok so what does this mean? 

A hidden markov model is where the observation and the state are separated. The simplest example is a coin. Usually you see the coin, and can read whether it came up heads or tails. A hidden markov model has the coin hidden, like behind a screen, and the observations (heads or tails) are read out to an audience (or user, or participant, or contestant). 

One of the jobs of the contestant is to figure out how many states are required to best explain the observations. For example, for five minutes the coin flipping produced about half heads and half tails. Then it suddenly changed, and the observations were mostly tails for four minutes. Then mostly heads for three minutes. Then it went back to a fair distribution for the rest of the session.

One way to explain this is with three coins. A fair coin, a heads-heavy coin, and a tail-heavy coin. The person behind the screen switched from one coin type to another and read off the resulting observations. The number of states in this hidden markov model would be three. Each coin is a state. Each state has an alphabet of two possible values. Each state’s alphabet has a different distribution of likelihoods. The probability distribution for each state (each coin) is different.

I believe our job is to figure out how to keep the tracks working together when new music is created. Analyzing each track separately stays in the toolbox, but analyzing the entire piece, and using that analysis to coordinate the production of new tracks must be done as well. 

Organ Donor Opus 1 polyphonic test

First New Music from Bach Violin Solo - Quick Sample

Here's the first simple example of the sort of files we're trying to produce for Organ Donor.

This file was created by taking the statistics of a Bach Violin Solo (the Partita No.2 Gigue), and analyzing them in the following way. Each note was examined (with a software program) to find out what note followed. After all the notes were counted up, we then calculated the probabilities of which note would follow any particular note that had occurred in the piece.

We then picked a random note, and "rolled the loaded dice" to see which note we should go with next. Once we saw which note we came up with, we did it again. We collected a 100-note-long sample.

For the original song that was analyzed, here's Hillary Hahn playing it:
https://www.youtube.com/watch?v=7eXzlg2Xcgg

For our very simplified example song:
http://www.delmarnorth.com/audio/bach_nmo.mid


midi file

Organ Donor Opus 1 Sounding a Pipe with Upgraded Blower Box

video

MIDI Files Playing Very Slowly Using Python Library MIDO

video

Blower Box Modifications

Organ Donor Paul describes possible modifications necessary to incorporate the upgraded blower motor for Organ Donor Pipe Organ Project. Our first blower (the fan that provides the air pressure to sound the pipes) wasn't quite strong enough. Now we need to modify the box for the new fan. 

video

Organ Donor at YouTopia 2014 Temple

Organ Donor will be in the 2014 YouTopia Temple. YouTopia is the San Diego Burning Man Regional Event, and will be held 16-18 October 2014. Here's the website:

http://sdyoutopia.com

Organ Donor Software Meeting 2014 July 7 - Audio

Audio of meeting

Draft Plans for the Expansion Chest for Organ Donor

Draft plans for the expansion chest for Organ Donor. The diamonds are for hexagonal accordion folded expansion chest experiment, and the box plan follows traditional bellows-based expansion chamber.

Accordian Experiment Plans

Bellows Expansion Chest Plans

OONTZ demo from Organ Donor Paul

video

Valve inventory complete

65 of 40ohm and 26 of 90ohm of Peterson style organ valves have been entered into inventory.

Organ Donor Update 14 June 2014

video

The Organ Donor Project is an acoustic pipe organ, built for the Burning Man Community.