MuseGAN: an open source model for generating multi-track music/soundtracks, easily creating music clips for multiple instruments

General Introduction

MuseGAN is a music generation project based on Generative Adversarial Networks (GANs) designed to generate multi-track (multi-instrument) music. The project is capable of generating music from scratch or accompanied by user-supplied tracks.MuseGAN is trained using the Lakh Pianoroll Dataset to generate snippets of popular songs containing bass, drums, guitars, piano, and strings. The latest implementation is based on a network architecture of 3D convolutional layers with a smaller network size but reduced control.MuseGAN provides a variety of scripts to facilitate operations such as managing experiments, training new models, and inference and interpolation using pre-trained models.

Function List

• Generate multi-track music: Generate music clips containing multiple instruments from scratch.
• Accompaniment Generation: Generate accompaniments based on user-supplied tracks.
• Training new models: Scripts and configuration files are provided to facilitate users to train their own music generation models.
• Using pre-trained models: Download and use pre-trained models for music generation.
• Data Processing: download and process training data with support for Lakh Pianoroll Dataset.
• Experiment management: Provide a variety of scripts to facilitate users to set up and manage experiments.
• Output format: the generated music can be saved in numpy arrays, image files and multi-track pianoroll file formats.

Using Help

Installation process

1. Make sure pipenv (recommended) or pip is installed.
2. Use pipenv to install the dependencies:

bash
pipenv install
pipenv shell

or use pip to install dependencies:

bash
pip install -r requirements.txt

### Data preparation
1. Download the training data:

bash
. /scripts/download_data.sh

2. Processing of training data:

bash
. /scripts/process_data.sh

### Training a new model
1. Set up the new experiment:

bash
. /scripts/setupexp.sh ". /exp/myexperiment/" "Remarks on the experiment"

2. modify the configuration file and the model parameter file to set the experiment parameters.
3. Train the model:

bash
. /scripts/runtrain.sh ". /exp/myexperiment/" "0"

Or run the full experiment (training + inference + interpolation):

bash
. /scripts/runexp.sh ". /exp/myexperiment/" "0"

### Using the pre-training model
1. Download the pre-training model:

bash
. /scripts/download_models.sh

2. Inference using pre-trained models:

bash
. /scripts/run_inference.sh ". /exp/default/" "0"

or interpolation:

bash
. /scripts/run_interpolation.sh ". /exp/default/" "0"

### Output management
Generated music samples are generated during training by default, this behavior can be disabled by setting `save_samples_steps` to 0 in the configuration file. Generated music samples are saved in the following three formats:
- `.npy`: raw numpy array
- `.png`: image file
- `.npz`: multi-track pianoroll files, which can be loaded via the Pypianoroll package
Saving of specific formats can be disabled by setting `save_array_samples`, `save_image_samples` and `save_pianoroll_samples` to False in the configuration file. Generated pianoroll files are saved in `.npz` format to save space and processing time. This can be written to a MIDI file using the following code:

python
from pypianoroll import Multitrack
m = Multitrack('. /test.npz')
m.write('. /test.mid')

Example results

Some sample results can be found in the. /exp/found in the catalog. More examples can be downloaded from the following links:

• sample_results.tar.gz(54.7 MB): Examples of inference and interpolation results
• training_samples.tar.gz(18.7 MB): Examples of results generated by different steps