Creative Prediction with Neural Networks

A course in ML/AI for creative expression

Making Predictive NIMEs with Neural Networks

Charles Martin - The Australian National University


What kinds of neural networks are useful in NIMEs?

How do recurrent neural networks work?

How can we use them to generate creative data?

Is there a more convenient way to do this?


Focus on hands on experiments

Focus on small data

Use widely available tools

Everybody will train a neural network (?)

Rough Plan

Learning to Predict Sequences

Recurrent Neural Networks and LSTM Units

Mixture Density RNNs

Interacting with Musical Predictions

Starting a community


Musical Predictions

What are predictions?

Where can they fit into a NIME?

Where can they fit into a performance?

Example: Wekinator-style NIMEs

Source: FlowerPowa74 (YouTube Video)

Example: RoboJam

RoboJam Demo

Example: Neural iPad Ensemble

YouTube Video


what kinds of predictions would you like a NIME to make?


Python is powerful.

We can all be Python superheroes!

Python setup can be hard.


Colab to the Rescue?

extra tools


Pure Data

Or use whatever computer music environment you want (needs to send and receive OSC messages).

get your tools together

Now’s the time to look at the setup page on

This course works best with Python, Keras, and Jupyter on your computer, but you can use (online) Google Colab Notebooks as well for most of the examples.