Why you’ll love GitHub Copilot X as a creative coder
If you don’t write software for a living but you enjoy piecing together bits of code to create beautiful art pieces or interactive experiences, you are a “creative coder”.
I consider myself a creative coder as well. I create hardware and software controllers for music, as collaborative applications, interactive art, etc., and to do so, I use whatever programming language is better suited for each piece. This means I have experience with a variety of languages, from Web (JavaScript, HTML, CSS) to microcontrollers (Arduino/C++, Python), but I am not an expert in any of them. The code I write is far from being ready for deployment to users at scale, in production scenarios.
Getting into The Zone
The frequent switch of context and the lack of daily practice keeps me out of shape with all of those languages. When I want to build a new thing, I may have a clear idea of what I want to make, but I need to check tutorials, Stackoverflow or even my own old code to get started.
Usually getting initial “boilerplate” code is the most frustrating part of building something. From that point on, I get in “the zone” and things flow in a much quick and pleasant way.
Let me give you an example.
Just a simple MIDI light
I fiddle around a lot with Web, MIDI, and RGB LEDs. I’ve built a couple of projects on top of these technologies. A couple of days ago, I wanted to start a new proof-of-concept that required a LED ring that responded to MIDI controls coming from a Web-based dashboard. I had already written and tested the code for the hardware. All I needed was to put together the dashboard app.
I knew I needed to incorporate some Web MIDI code into a web application, a Web form, and some glue code to put it all together. But I don’t know the Web MIDI API by heart, so I knew I would have to google related examples, check the docs, and probably pay a couple of visits to Stack Overflow when debugging my code. I also needed to write the basic HTML code to contain the app and the HTML form. Because of my lack of expertise, that was going to take me at least 1–2 hours of boring work.
Enter the Copilot
Just as I opened my editor to start typing code, I remembered I have access to the GitHub Copilot X (full disclosure: I work for Microsoft, so we get to play with cool software before it is released).
Copilot X is a chat-like assistant within the Visual Studio Code editor. GitHub promotes it as an “AI pair programmer”, and you can ask it to write code for you, to answer technical questions, to create unit tests, etc.
So I opened the Copilot X chat and typed the following prompt:
Create a simple web app that does the following:
- It lists the available MIDI output devices in the system in a dropdown
- It lets you select one device from the list as the default output
- It uses two buttons to send MIDI controller number 110 to the default output. One button sends value 0 and the other value 127
Here’s the code it produced for me:
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>MIDI Output Selector</title>
</head>
<body>
<h1>MIDI Output Selector</h1>
<label for="midi-output">Select MIDI output device:</label>
<select id="midi-output"></select>
<br>
<button id="cc-0">Send CC 110 value 0</button>
<button id="cc-127">Send CC 110 value 127</button>
<script>
// Get the MIDI output devices
const midiOutputs = [];
const midiOutputSelect = document.getElementById('midi-output');
navigator.requestMIDIAccess().then(midiAccess => {
midiOutputs.push(...midiAccess.outputs.values());
// Add each MIDI output device to the dropdown
midiOutputs.forEach(output => {
const option = document.createElement('option');
option.value = output.id;
option.text = output.name;
midiOutputSelect.add(option);
});
}, () => {
console.log('Could not access your MIDI devices.');
});
// Send MIDI controller number 110 to the default output with the specified value
function sendCC(value) {
const outputId = midiOutputSelect.value;
const output = midiOutputs.find(output => output.id === outputId);
if (output) {
output.send([0xB0, 0x6E, value]);
}
}
// Add event listeners to the buttons
const offButton = document.getElementById('cc-0');
const onButton = document.getElementById('cc-127');
offButton.addEventListener('click', () => {
sendCC(0);
});
onButton.addEventListener('click', () => {
sendCC(127);
});
</script>
</body>
</html>
It worked perfectly, right away, with no modifications! My jaw dropped as my LED ring turned on and off in response to clicking the buttons.
Thanks to that initial code, I didn’t have to go and check the API to remember how to check whether MIDI is available in the browser, how to list devices, hot to send data to them, etc. This is exactly what I needed, and it saved me at least an hour of non-creative, tedious work!
At that point, I started making changes to the code, adding other controls for adjusting brightness, modifying the MIDI functions to smoothly fade the light in and out, making the dashboard prettier with some CSS, etc. Because I already had all the fundamental bits in place, it was super easy to move on with the actually interesting stuff.
I can also see how this would be super useful for other folks with no Web MIDI API experience. As Copilot added comments, it is easy to follow what each code section does. And as the code was written according to the specs (prompt), it is easier to relate the code to the result than it would be with some sample code obtained from somewhere else.
I’m a creative coder, too. Why should you use it?
There is a huge debate on whether AI may be a threat to a number of jobs, including software development. However, I am not discussing professional programming here. Instead, I believe that, for creative coders, AI-assistance is going to be a positive game changer.
Even more than in other software development contexts, in creative coding, writing code is a means to an end. We deal with prototypes more often than with finished products. We need to come up with quick proofs-of-concepts to be able to build upon them, to connect disparate systems, and to create beautiful interactive experiences that stimulate not only the brain, but the heart.
Creative coders have the specs clear in mind, and technical details are often obstacles to that end. AI assistants like GitHub Copilot X can help bridging these obstacles, allowing non-expert programmers, creative coders and artists to focus on the fun part: turning their ideas into reality.
