Machine Learning with ML5

 Progress Documentation

     Hello, this page is for my Machine Learning project for Paul Robert's Digital Interventions class MADT 201. 

Students have been tasked with creating interactive artwork and light Machine Learning research using the ML5.js data libraries and P5.js. We have been given the option to train AI and develop databases but due to time constraints i've decided to utilize existing libraries re-appropriated for my creative use. 

The ML5.js reference contains a healthy amount of pre-trained models from Image classifiers, body and facial rigging to generative text AI. I have decided to use the Sound Classification reference with my project but eventually I do want to record my own library for audio visualizing suited for performances and interactive installations. This model recognizes speech commands is currently trained to  discern words: numbers from 0-9,  up, down, left, right, go, stop, yes, no, and other options. My goal with this project is to use the recorded words as a trigger for animation and generation of objects existing in my sketch.  

Current project can prompt an animation see Link, if you say the words "Up" it would cause the ball to go up. 

Ball on Neutral Position.

Currently Im using the p5js sketch to prototype the project but Im still unsure where to lead it into. Originally I wanted to just create an audio visualizer using the Pitch Detection Library but it proved finicky to code. 

Overall concept:

Project Title "HoUs1NG MArK3T" 

To create a p5js sketch utilizing the sound classification library to trigger functions in the web-app. 

    "HoUs1NG MArK3T" is an interactive installation that deals with the current housing crisis in North

America. Projected on a wall is a grid and moving line graph that symbolizes the current value of houses, by using the words "up" and "down" viewers are able to modulate the shape of the line. Included in the grid are two coloured lines that symbolizes the hyper inflation cap(Red) and deflation of value(Blue). The goal of the installation is to give viewers as sense of control over the stock prices and allowing them to crash the values or increase them over the simulation. 

When graph crosses the red line : 

When graph crosses the blue line : 

A major consideration for the concept is our collective experience with the housing market and its steady inflation of prices. Unfortunately the pricing trends increase started in the 1980's and with the ongoing pandemic, the situation has been magnified it was recently highlighted by an article from Business Insider  dealing with the current crisis. The constant fluctuations in value and the anxieties it had caused pushed me in the direction of creating this game. I want viewers to be given some sense of control over the situation. Hence by allowing them control over the line graph, it could create a therapeutic effect and a sense of respite over the current crisis we are facing.