Recap from Day 026

In day 026, we continued with how machine learning in the arts is different. We saw that they are a few big data sets out there in the arts, for example, collections of audio or text related to classical or popular music. But it’s less likely that you’ll find big data sets of gestures made by musicians or dancers, and pretty certain you won’t find a big data set of exactly the type of gesture you want to use in a customized musical instrument or installation, and that’s fine.

Today, we’ll bring how machine learning in the arts is different to an end.

You Are The Expert On Model Quality

“Finally, for most artistic applications, you are an expert on what the model you’re building should ultimately do. You will know whether your gesturally-controlled instrument is making a sound you’re happy with. You will know whether your dance motion classifier is accurately labeling the different actions a dancer is taking. This means that you’re not restricted to conventional metrics like cross-validation when you want to understand whether the model you’ve built is any good.

Cross-Validation — Source

You might be better off taking your new musical instrument and playing it, or taking your new dance classifier to a rehearsal and observing it in action. Compared to cross-validation, this will tell you a lot more useful information about the types of mistakes your model makes, how you might fix them by adding new data, and whether it’s enough to use in a performance.

Cross-validation still might be useful to you if you’re confident you have a very good data set that communicates your intentions for the model and if you want to compare one algorithm against another. However, you may also find that cross-validation isn’t a very meaningful measure for you in some contexts because it’s a measure of how well your algorithm can generalize from the training data.

If you prefer a model with high training accuracy and complex decisions boundaries to one with high generalization accuracy and smoother decision boundaries, you may find that training accuracy is a better measure for you.”

so even though most machine learning materials will tell you that training accuracy is a bad measure of performance, you should only adopt that assumption for your project if it really makes sense for your project goals and your training data.

Awesome. You made it to the end of day 027. I hope you found this informative. Thank you for taking time out of your schedule and allowing me to be your guide on this journey.

Reference

https://www.kadenze.com/courses/machine-learning-for-musicians-and-artists-v/sessions/developing-a-practice-with-machine-learning-wrap-up