Komorebi Kit is designed to generate music from the moving shadows of trees. To experience your kit, take it with you to a park, a forest or place it in your garden. Search for a shadow that is moving in the wind. Place Komorebi where the shadow flutters and observe. 

Explore how the music changes with the weather, the time of day, or the shapes created by different plants. Komorebi can also be activated by other changing light sources, such as a single candle oscillating in a gentle draft or a parade of clouds intermittently eclipsing the sun.

To power on and off use the switch which is positioned next to the micro-USB socket. Sliding the switch towards the USB connector turns the kit on. 

To charge the battery, use a micro-USB cable to connect the kit to a USB power source. Charging is independent from the power switch and works when the kit is either on or off. When USB power is connected the charging LED lights up red. When the battery is fully charged the LED turns off.

-The kit includes a rechargeable Li-Po battery. The battery should not be left to charge unattended. Do not puncture the battery. A visibly damaged or swollen battery should be disconnected and discarded immediately. A damaged battery is a fire hazard.

-The kit is composed of many exposed unprotected electronic components. To prevent short circuits from damaging the electronics, keep the kit away from humidity, water, metal objects and other conductive materials.

The kit is created around the Espressif ESP32 micro-controller. Audio is output from a PT8211 16bit DAC and amplified by a PAM8302 IC. The light sensor is an HT7833 photodiode amplified by a MCP6001 op-amp with variable gain resistor MCP4151. USB-serial interface is the CH340.

The GPIO pins of the ESP32 that are not in use are broken out to through-hole connections for further hacking and experimentation, as well as the direct DAC output.

The DAC output should never be used to directly drive a speaker or headphones. It requires an amplification stage first to prevent damage to the DAC. 

The code for Komorebi Kit is composed of two main parts: an input processing section and a sound generation section.  

The input processing section reads the input from the light sensor and sets the gain on the op-amp using the digital pot, adjusting the sensitivity of the input to match as much as possible the lighting conditions. This adjustment is gradual, so when moving Komorebi between environments with significantly different lighting (for example moving from indoors to outdoors) it can take several seconds for the sensitivity to fully adjust to the new situation. 

The light input is then processed further in the sound generation section. 

The sound generation section of the code is loosely modelled on the aeolian harp. While the aeolian harp is a stringed instrument whose physical strings are mechanically excited by the wind, Komorebi Kit has virtual strings which are excited by the shadow of a tree moving in the wind.

Komorebi Kit has 5 sets of 3 virtual strings each, the sound of which is produced by additive synthesis. Each set of 3 strings is tuned in just intonation to a chord using as intervals the fundamental, major third, perfect 5th and major 6th, in various voicings. The 5 sets are arranged on a circle of fifths starting on A2 (110 Hertz) and moving clockwise. The most common situation is for only one of the 5 sets of strings to play at one time. Which set is sounding depends on the average light input in the past period. In some situations multiple sets will sound together, usually when the lighting is transitioning between two or more sets. 

Each string is composed of 12 partials, which become independently excited depending on the changes in the light input. Each partial can be excited by the changes in light in two ways: as a drone, with a loudness that increases and decreases gradually over time, and as a pluck, which is triggered each time the amount of change reaches a certain threshold. The plucks are generated using Karplus-Strong string synthesis. Other parameters evolve over time in response to the light input: a detuning factor applied to the pitch of the strings as well as the loudness and resonance factor of the plucks.

Early prototypes were developed using the Arduino IDE software and a library for microcontroller audio synthesis called Mozzi. We provide basic code examples for programming the kit with the Arduino IDE below. 

The release version is developed using the Visual Studio Code editor with PlatformIO IDE plug-in and the Mozzi library. We publish this code and guide as well.

Requisites:

-Download and install Visual Studio Code: https://code.visualstudio.com/Download

-Download and install the CH340 USB driver: http://www.wch-ic.com/downloads/CH341SER_ZIP.html

-Download Komorebi Kit release code from GitHub, unzip and put it in your working folder: https://github.com/dietervandoren/komorebi-kit

 

Setup:

-Open the Visual Studio Code application

-In the left vertical menu click the Extensions button (4 squares icon)

-In the search field enter: PlatformIO

-The option PlatformIO IDE will appear, now click Install

-When install is finished (this might take a while), close and reopen Visual Studio Code

-The left vertical menu now has a new icon for PlatformIO (alien head), click it

-In the Platformio: Quick Access pane, click Open 

-In the PIO Home tab click Open Project 

-navigate into the downloaded GitHub project folder (containing the file platformio.ini), click Open “komorebi-kit-main”

-The project folder tree now appears in the left column (the project folder includes the Mozzi library, with external PT8211 DAC already enabled)

 

Modifying the behavior:

-The core program files are in the “src” folder, with “main.cpp” being the primary framework

-Header files are in the “include” folder

-The “lib” folder contains the Mozzi library which is pre-set for using for the PT8211 DAC (this library generally doesn’t require any changes)

-The main parameter variables are set in “LarvaDefs.hpp”

 

Examples of parameters that are easy to adjust:

-“cDroneMasterGain” sets the loudness of the drones

-“cPulseMasterGainMin” sets the minimum loudness of the pulses

-“cPulseMasterGainMax” sets the maximum loudness of the pulses

 

Upload modified code to the hardware:

-Connect a USB cable to computer and kit

-Turn on the kit power switch

-In Visual Studio Code, on the bottom blue horizontal bar, press the arrow pointing right

-The compiler will post a stream of status messages in the console

-When the console displays a growing line of dots, on the kit, keep the Flash button pressed then press and release the reset button(*). Release flash when uploading starts.

-The code firmware will upload (flash) to the ESP32 micro-controller, when finished press the Reset button  

(*) Due to wide tolerances in the electronic components some kits do not require pressing the Flash button and uploading will be automatic.

Note: the kit can not be reverted to our original behavior code using the Arduino IDE. For that, please see the Visual Studio Code section.

Requisites:

-Download and install the Arduino IDE application: https://www.arduino.cc/en/software 

-Download and install the CH340 USB driver: http://www.wch-ic.com/downloads/CH341SER_ZIP.html

-Download code template: https://github.com/dietervandoren/komorebi-kit 

 

Preparing the Arduino IDE software for ESP32 boards:

– Start the Arduino IDE software and go to File> Preferences

– In the Additional Boards Manager URLs field, enter (or copy-paste): https://dl.espressif.com/dl/package_esp32_index.json and click OK.

– Go to Tools> Board> Boards Manager …

– Enter “ESP32” in the search field and click Install in the search result “ESP32 by Espressif Systems”

– The board definitions are now downloaded and installed. INSTALLED appears when it is ready.

The Arduino IDE is now prepared to program ESP32 microcontrollers. (See also online guide here.)

 

Install and configure Mozzi library: 

– In the Arduino IDE, go to File> Preferences

– Look in the “Sketchbook location” field to see the working directory of the Arduino IDE on your system.

– Go to this folder with your file browser (File Explorer / Finder). Inside it go into the folder “libraries”.

– From the downloaded code template, copy the folder “Mozzi” found inside the “lib” folder and paste into the Arduino “libraries” folder.

The Mozzi synthesis library is now ready to use with the kit hardware in the Arduino IDE.

Note: the Mozzi library that we included in our code template is already configured to output on the PT8211 external DAC instead of the regular PWM pin. If one were to download Mozzi afresh from https://github.com/sensorium/Mozzi then configurations have to be modified in the file “AudioConfigESP32.h”.  

 

Code template explained:

The Arduino sketch in the folder “arduino_ide_template” is a minimal example of how to program the hardware using the Arduino IDE. The installation steps above are required (except for the Mozzi library in case one wishes to use other methods for producing audio signals). Every step is commented in the sketch code. 

The sketch generates a sine wave with variable frequency dependent on the intensity of light input. The main elements are explained below.

 

Photodiode light sensor pre-amp gain and input:

The HT7833 photodiode is connected to an MCP6001 op-amp to convert its current output to a voltage received on pin 34 of the ESP32 microcontroller. The gain of the op-amp is variable to accommodate a wide range of lighting situations. The gain is set using a MCP4151 digital potentiometer via the SPI protocol. SPI slave select pin on the microcontroller is number 5. The SPI library must be included. 

The gain is set through an 8 bit value with reversed range, meaning 255 is minimum gain and 0 is maximum gain. 255 is applicable for direct sunlight and 0 for twilight or candle light, with a wide spectrum in between to adapt to other situations.

The digitalPotWrite() function takes care of the SPI messaging.

 

Audio output amplifier enabling:

The kit includes a PT8211 digital-analog converter (DAC) which is automatically configured by the Mozzi library. To output sound on the embedded amplifier and speaker the PAM8302 amp IC has to be enabled first. It is disabled by default. Setting microcontroller pin 17 high enables the amp. Low disables (which can be useful for power saving).

The amplifier has a fixed gain. Output volume is regulated by setting the gain on the digital output signal generated by Mozzi.

 

Mozzi audio signal synthesis:

We use the Mozzi library to generate audio signals on a microcontroller like the ESP32. It’s an advanced library that provides many familiar synthesis functionalities like wavetables, oscillators, modulators, envelopes, etc. It also deals with the low level technicalities of outputting optimally on the microcontroller hardware, which can be a very complex affair. 

Including MozziGuts.h is required. Other Mozzi header files are required depending on which functionalities are used. The template only uses a basic oscillator for which Oscil.h is included, as well as a sine wavetable. A sine object to reference in further code is created. 

In setup(), startMozzi() enables audio processing.

Mozzi only requires calling audioHook() inside the loop().

All other Mozzi operations are performed in the functions updateControl() and updateAudio(). updateAudio() runs at audio rate (very fast) and should only contain the necessary audio sample calculations. updateControl() runs at a slower pace set with the CONTROL_RATE constant. Reading sensors, calculating synthesis parameters, printing serial messages, etc should be done in here whenever possible to not slow down the processor with unnecessary operations.      

In updateControl(), we are first reading the light sensing input value, then mapping it to a sine frequency range. The frequency of the sine object is then set using setFreq().

In updateAudio() we are just retrieving the next sine wave sample value from the sine object with next(). Mozzi takes care of giving the right value given the frequency that we have set.

With the return command we pass the sample value to the audio output buffer. Before outputting the amplitude is raised by bit shifting left/up. That’s a computationally cheap method for multiplying by powers of 2. The output resolution of the DAC is 16 bit while the sine wavetable contains 8 bit values, meaning much of the available amplitude/resolution is not used. Therefore the amplitude is raised to get a good output volume.

The Mozzi library includes many examples to browse and explore. 

Note: the Mozzi examples assume the standard 8 bit PWM output method. Most will require raising the output amplitude to yield a usable signal on the 16 bit kit hardware. Any Mozzi example will also require adding the amplifier enable code before the hardware will output sound.

 

Uploading code from the Arduino IDE:

Connect the USB cable and turn on the kit power switch.

Select “LOLIN D32” board type in Tools > Board > ESP32 Ardiuno

Select the serial port in Tools > Port

Click the upload button. When the code is compiled the IDE will try to flash the firmware onto the ESP32. Due to production tolerances on the hardware components some kits will flash automatically while others require a button press combination.

If a growing line of dots and underscores appears it means the button combination has to be performed. On the kit, hold the flash button while pressing and releasing the reset button. When uploading begins, release the flash button.

When uploading is completed, press the reset button.