# Making Predictive NIMEs with Neural Networks Charles Martin - University of Oslo / Australian National University  <a href="https://creativecommons.org/licenses/by-nc-sa/4.0/"> <img src="/assets/resources/by-nc-sa.svg" style="height:1em; margin-left:0;" /> </a>

### What kinds of neural networks are useful in NIMEs? ### How do recurrent neural networks work? <!-- .element: class="fragment" --> ### How can we use them to generate creative data? <!-- .element: class="fragment" --> ### Is there a more convenient way to do this? <!-- .element: class="fragment" -->

### Focus on hands on experiments ### Focus on small data <!-- .element: class="fragment" --> ### Use widely available tools <!-- .element: class="fragment" --> ### Everybody will train a neural network (?) <!-- .element: class="fragment" -->

<!-- .slide: data-background="#000000" --> # Rough Plan

- 9:00 Meeting, Intro, and Python Setup - 9:15 [Overview of Deep Learning and Creativity](/nime/#overview-dl-creativity) - 9:45 [Generating Text and Music with RNNs (practical)](/nime/#generating-sequences) - 10:30 (coffee break) - 10:45 Hack: inventing Star Trek episode titles - 11:30 [Using Mixture Density Networks (MDNs)](/nime/#mixture-density) - 13:00 Lunch (90 minutes) - 14:30 [Making Predictive Musical Interactions with the IMPS system](/nime/#making-predictive) - 16:00 (coffee break) - 16:15 [Future directions for creative neural networks at NIME and beyond](/nime/#future-directions)

## Learning to Predict Sequences 

## Recurrent Neural Networks and LSTM Units 

## Mixture Density RNNs 

## Interacting with Musical Predictions 

## Starting a community 

<!-- .slide: data-background="#000000" --> # Predictions

## Musical Predictions 

## What are predictions? 

## Where can they fit into a NIME? 

## Where can they fit into a performance? 

## Example: Wekinator-style NIMEs  [Source: FlowerPowa74 (YouTube Video)](https://youtu.be/tcQpnV4ajLY)

## Example: RoboJam  [RoboJam Demo](https://youtu.be/n2xSHoB2_uY)

## Example: Neural iPad Ensemble  [YouTube Video](https://youtu.be/FpQCAd0zKiU)

## talk what kinds of predictions would you like a NIME to make?

<!-- .slide: data-background="#000000" --> # Tools

## Python is powerful.  We can all be Python superheroes! <!-- .element: class="fragment" -->

## Python setup can be hard.  (Sorry) <!-- .element: class="fragment" -->

## Colab to the Rescue? 

## extra tools  <!-- .element: width="10%" --> [Processing](https://processing.org)  <!-- .element: width="10%" --> [Pure Data](https://puredata.info/downloads/pure-data) Or use whatever computer music environment you want (needs to send and receive OSC messages).

## get your tools together Now's the time to look at the [setup](/setup) page on [creativeprediction.xyz](https://creativeprediction.xyz). This course works best with Python, Keras, and Jupyter on your computer, but you can use (online) Google Colab Notebooks as well for most of the examples.