Pianojacq, an easy way to learn to play the piano(jacquesmattheij.com)
jacquesmattheij.com
Pianojacq, an easy way to learn to play the piano
https://jacquesmattheij.com/2020-09-20-pianojacq-learn-to-play-piano/
181 comments
That's an interesting comment. So, the midi->score conversion is indeed one of the harder parts, I spent a lot of time on that and I'm not yet satisfied with the results. There are quite a few ways in which this can be improved further, there is a long list of 'todo's' where I basically have worked out how to make the improvement but I have not yet gotten around to actually implementing it.
Midi is 'lossy' in the sense that it flattens a bunch of knowledge about how a piece is to be rendered into note-on/note-off pairs. You then have to do all kinds of tricks to even begin to approach what the score looked originally. Lots of enharmonically (identically sounding) combinations may have looked completely different on the original score, and when you start adding the various chord combinations it gets even more complicated. Even so, I'm strongly convinced it can be solved and we have made considerable progress on this already.
The 'wiggle room' aspect of it has been addressed to some extent. For instance, we build up a map of all the notes that are sounding at a given time, and will ignore notes that are technically 'wrong' but that are still sounding. This allows for much smoother restarts after mistakes.
Your final point is a bit tricky, but thank you for the hint, I will definitely add it to the list of things to look at to see if we can find a way to make that work.
Midi is 'lossy' in the sense that it flattens a bunch of knowledge about how a piece is to be rendered into note-on/note-off pairs. You then have to do all kinds of tricks to even begin to approach what the score looked originally. Lots of enharmonically (identically sounding) combinations may have looked completely different on the original score, and when you start adding the various chord combinations it gets even more complicated. Even so, I'm strongly convinced it can be solved and we have made considerable progress on this already.
The 'wiggle room' aspect of it has been addressed to some extent. For instance, we build up a map of all the notes that are sounding at a given time, and will ignore notes that are technically 'wrong' but that are still sounding. This allows for much smoother restarts after mistakes.
Your final point is a bit tricky, but thank you for the hint, I will definitely add it to the list of things to look at to see if we can find a way to make that work.
Musescore does do a great job and is open source, so there's that. Another option is to have users manually upload midi with a proper score attached to some store or community repo with user reviews. It's less satisfying as a solution, but it means all it takes is one motivated user for everyone to enjoy the score.
For my final point, you can model this as a hidden markov process. You're observing notes, but not the user moving his mental "cursor". Classical filtering algorithms should do well here.
Ah, one more missing feature, good scores have fingering, but it gets lost in the midi. I'm pretty sure fingering can be infered though and this would be pretty powerful. Amusingly, it can also be seen as a filtering problem:
- At any given point, a hand can have a finite number of fingerings, and that number isn't that large. Give each of them a difficulty.
- Now, make that difficulty conditional on the previous hand position.
- Run Viterbi's algorithm on the score to minimize the difficulty.
The difficulty can be obtained from an expert or through some physiological model, but they could also be inferred from an existing body of music sheets.
For my final point, you can model this as a hidden markov process. You're observing notes, but not the user moving his mental "cursor". Classical filtering algorithms should do well here.
Ah, one more missing feature, good scores have fingering, but it gets lost in the midi. I'm pretty sure fingering can be infered though and this would be pretty powerful. Amusingly, it can also be seen as a filtering problem:
- At any given point, a hand can have a finite number of fingerings, and that number isn't that large. Give each of them a difficulty.
- Now, make that difficulty conditional on the previous hand position.
- Run Viterbi's algorithm on the score to minimize the difficulty.
The difficulty can be obtained from an expert or through some physiological model, but they could also be inferred from an existing body of music sheets.
Fingering we have a pretty good hold on already, so I'm not too worried there.
Importing alternative formats (xml for instance) is definitely an option worth looking into, but because the bulk of the material that is out there that people can freely access is in midi format we chose that one first.
Importing alternative formats (xml for instance) is definitely an option worth looking into, but because the bulk of the material that is out there that people can freely access is in midi format we chose that one first.
> have users manually upload midi with a proper score attached to some store or community repo with user reviews. It's less satisfying as a solution, but it means all it takes is one motivated user for everyone to enjoy the score.
Musescore already has such a collection of sheet music https://musescore.com/hub/piano so it'd also be possible to allow users to import from there, going from sheet music to midi instead of trying to reverse the process.
Musescore already has such a collection of sheet music https://musescore.com/hub/piano so it'd also be possible to allow users to import from there, going from sheet music to midi instead of trying to reverse the process.
Yes, but then you would still lose quite a bit of information. Besides that, musescore is pulling a bait-and-switch on the user generated content that makes gracenote look good.
Piano Marvel allows you to add your own pieces to the library, I did it with one I found online. What they recommend is using Musescore to convert the MIDI to a score, fixing it up in Musescore and exporting as MusicXML. Musescore did a creditable but not perfect job of the conversion, but fixing it up was easy. Then you import both files into Piano Marvel. I'm actually not sure why they need the MIDI at all once you have the MusicXML, given that Musescore can play the music from the score directly.
> Midi is 'lossy' in the sense that it flattens a bunch of knowledge about how a piece is to be rendered into note-on/note-off pairs.
As using MIDI will probably never be perfect, have you thought about using the score in a suitable format, like LilyPond, directly?
As using MIDI will probably never be perfect, have you thought about using the score in a suitable format, like LilyPond, directly?
Yes, absolutely, and we will add more import formats.
I don't really follow the idea that "midi is lossy" in the sense described. Because it uses note-on/note-off pairs? I don't see the problem there. Certainly the idea that a score somehow has more information in it than a midi recording is ridiculous. A score always requires interpretation. The midi is, in contrast, fully specified, very little room for interpretation.
A midi recording contains pretty precise velocity info for each note (0-127) that is either not present at all in a score, or is present only in the form of a limited number of dynamic markings[1], which govern the flow of dynamics imprecisely, and almost not at all on a per-note basis.
With tempo it is similar to dynamics: midi will precisely determine when notes turn on and off, down to the millisecond, in the process resulting in the midi performance having a precise tempo. In contrast, musical scores have limited means to indicate proper tempo[2], necessitating (as with dynamic markings in a musical score) much interpretation by the musician.
All of this is not to say that a midi performance is somehow "better" than a score. There are good midi recordings and bad ones. A good recording will generally be made by an accomplished musician who in making the recording records fairly precisely a particular (and good) interpretation of the score. Likewise, a bad recording would generally be the result of a poor interpretation of a score. But whatever the case, the midi recording is far more precise than the score itself.
[1] https://www.aboutmusictheory.com/music-dynamics.html
[2] https://en.wikipedia.org/wiki/Tempo#Basic_tempo_markings
A midi recording contains pretty precise velocity info for each note (0-127) that is either not present at all in a score, or is present only in the form of a limited number of dynamic markings[1], which govern the flow of dynamics imprecisely, and almost not at all on a per-note basis.
With tempo it is similar to dynamics: midi will precisely determine when notes turn on and off, down to the millisecond, in the process resulting in the midi performance having a precise tempo. In contrast, musical scores have limited means to indicate proper tempo[2], necessitating (as with dynamic markings in a musical score) much interpretation by the musician.
All of this is not to say that a midi performance is somehow "better" than a score. There are good midi recordings and bad ones. A good recording will generally be made by an accomplished musician who in making the recording records fairly precisely a particular (and good) interpretation of the score. Likewise, a bad recording would generally be the result of a poor interpretation of a score. But whatever the case, the midi recording is far more precise than the score itself.
[1] https://www.aboutmusictheory.com/music-dynamics.html
[2] https://en.wikipedia.org/wiki/Tempo#Basic_tempo_markings
> Certainly the idea that a score somehow has more information in it than a midi recording is ridiculous
A midi file is designed to contain different kinds of information than a score, so it can absolutely contain less information (of a certain kind) than a score.
Imagine a group of notes that are attacked very close together in time in a midi file - is it meant to be a chord played without arpeggiation? A chord arpeggiated as fast as possible? A chord arpeggiated to a specific tempo?
When a composer writes (say) a chord with an arpeggiation sign next to it, it specifies that the notes are to be played in a certain order, and quickly. There are multiple possible performances that comply with this "specification". MIDI cannot encode this information (the information being the set of acceptable performances).
A midi file is designed to contain different kinds of information than a score, so it can absolutely contain less information (of a certain kind) than a score.
Imagine a group of notes that are attacked very close together in time in a midi file - is it meant to be a chord played without arpeggiation? A chord arpeggiated as fast as possible? A chord arpeggiated to a specific tempo?
When a composer writes (say) a chord with an arpeggiation sign next to it, it specifies that the notes are to be played in a certain order, and quickly. There are multiple possible performances that comply with this "specification". MIDI cannot encode this information (the information being the set of acceptable performances).
[deleted]
Thanks, yes, I would agree that a midi recording and a score to a large extent encode different types of information. The score has less of some of the kinds of information that are in fact present in midi. And the midi, of course, doesn't specify anything about ranges of interpretation; it simply represents a specific interpretation of a score.
Midi note code 61: is that a C# or a Db, you pick...
I think what's missing on most "piano-games" is that it doesn't really mirror the way I learn a piece.
The way I learn is by bars, repeatedly re-playing bars say, 1 to 4, until I get them, them 3 to 7, until I get them, etc.. But most piano apps want you to start playing from start to end slow, then faster, etc.
So: piano apps should allow breaking a piece into bars, allow you to repeat those bars again and again until you exceed certain score. Maybe even keep track of which bars are harder to play, and repeating the difficult bars more often, etc.
The way I learn is by bars, repeatedly re-playing bars say, 1 to 4, until I get them, them 3 to 7, until I get them, etc.. But most piano apps want you to start playing from start to end slow, then faster, etc.
So: piano apps should allow breaking a piece into bars, allow you to repeat those bars again and again until you exceed certain score. Maybe even keep track of which bars are harder to play, and repeating the difficult bars more often, etc.
I’ve been using piano marvel which does exactly that, and find it pretty great. The only issues with it are that it’s built on somewhat old tech and can be a bit buggy, but for actually learning pieces it is genuinely useful (I’m around grade 7) and also has method/technique pathways you can go through. I think there’s a free trial for it too.
(Not affiliated, just something I personally use)
(Not affiliated, just something I personally use)
This one actually does exactly that, so please give it a try. It also has an auto mode where it measures how well you play and then sends you to practice those bars where you are still weak.
sg! will take a look!
> you're playing at a given pace, miss, stop and repeat a bar, the app should detect that automatically instead of just carrying on expecting you to keep playing.
I'm a piano noob but wouldn't you want to practice catching back up with the beat if you ever intend to perform?
I'm a piano noob but wouldn't you want to practice catching back up with the beat if you ever intend to perform?
This is a really good question, so I don't know why there are downvotes. There are 2 scenarios: 1) you're trying to learn the notes and 2) you're trying to get it in shape for a performance. It sounds like this software is geared towards scenario 1 where you don't yet know the piece and want to learn it. Generally when learning the piece it's best to fix mistakes as they happen, otherwise your brain sort of subconsciously assumes that was what you intended. Once you have the piece down, then when you make a mistake you want to keep going and pretend it never happened because otherwise the rest of the band will leave you behind. (And for soloists, nobody wants to hear you go back and correct a mistake.)
Yes, you really don't want to end up 'practicing your mistakes'.
Notes should directly align with the keys for usability. Example: https://www.youtube.com/watch?v=6ZJTdQ-xCBE
I'm not quite happy with Synthesia. There is no visual feedback on completing a note. This has been a staple feature in Doremi Mania and Keyboard Mania for twenty years, so they have zero excuse for not implementing it.
I'm not quite happy with Synthesia. There is no visual feedback on completing a note. This has been a staple feature in Doremi Mania and Keyboard Mania for twenty years, so they have zero excuse for not implementing it.
Aside from the relatively difficult problem of creating scores from MIDI; when practicing piano as a beginner it'd be interesting to have help in positioning/finger selection. I wonder how hard it would be to compute fingering plans (maybe even with a wingspan parameter).
We are working very hard on solving this in a very interesting and beautiful way. Stay tuned.
I’m super curious, if you don’t mind my prodding. I looked into doing this for my own version of a JavaScript midi teacher a while back. Any chance you’re doing it via the neural nets sequence to sequence translation algorithms that are super popular in NLP right now? I think that’s a practical way to do it, except for needing a training corpus. I was going to reverse engineer synthesia’s data to try it out, for what’s worth...
We do not think that it will need a neural net in order to be able to do this, but we need to prove that in order to make it stick and we're not quite ready to do that. Keep in mind that this is in every sense of the word a side project, we only work on this when we do not have technical due diligence work for customers and that still continues quite strongly. But on 'off' weeks we work on this and the project progresses quite rapidly when we do.
I think it could be done with a constraint solver, with a lot fewer cycles.
This is the route we are looking at, with some twists.
I can imagine this being really useful for improving sight-reading.
However, for me what made piano a whole lot more fun was ditching the sheet music and opting to learn about chords, all the different ways to voice them, and the various scales that fit well on top of them. I started taking lessons with a pro jazz pianist (instead of a more typical classically-focused piano teacher), I gave up on sheet music altogether and started working off lead sheets instead (just chord symbols and a melody line). I am so pleased I made that decision, it's so much more satisfying playing a tune in your own way rather than just aiming for a note-for-note reproduction of what is written on a sheet.
I still have loads more to learn, but I'm fairly pleased with where I've got to so far. I've got a few videos on youtube so you can judge for yourself if interested: https://www.youtube.com/channel/UC_NWuVoCn-Oc1KIiHEhe7Eg
However, although I think I can play reasonably well now, my sight-reading is awful, and it does hold me back because it makes it a lot harder for me to learn new techniques from sheet music (which I am actually interested in doing, as opposed to simply learning the sheet music note for note). I can do it just about, but it's painfully slow, so I am usually too lazy to bother.
Given the choice of either playing with full sheet-music or learning chords/voicings/scales and how to put them together, I'd pick the latter no question, and that is what I recommend to other adults who are learning piano, but I do think it's best to have both. So I can definitely see the value in this, even if it doesn't currently seem to teach the theory and improvisation that, in my opinion, is what really brings the joy into piano playing. I might give it a proper go myself some time soon, to see if it can help me improve my rubbish sight reading.
I can also imagine this being very useful for a beginner... Although I'm singing the praises of chord theory and improvisation I'm guessing most beginners might realistically do better starting with sheet music, for a while at least.
In summary: it might not teach everything, but it looks really useful nonetheless :)
However, for me what made piano a whole lot more fun was ditching the sheet music and opting to learn about chords, all the different ways to voice them, and the various scales that fit well on top of them. I started taking lessons with a pro jazz pianist (instead of a more typical classically-focused piano teacher), I gave up on sheet music altogether and started working off lead sheets instead (just chord symbols and a melody line). I am so pleased I made that decision, it's so much more satisfying playing a tune in your own way rather than just aiming for a note-for-note reproduction of what is written on a sheet.
I still have loads more to learn, but I'm fairly pleased with where I've got to so far. I've got a few videos on youtube so you can judge for yourself if interested: https://www.youtube.com/channel/UC_NWuVoCn-Oc1KIiHEhe7Eg
However, although I think I can play reasonably well now, my sight-reading is awful, and it does hold me back because it makes it a lot harder for me to learn new techniques from sheet music (which I am actually interested in doing, as opposed to simply learning the sheet music note for note). I can do it just about, but it's painfully slow, so I am usually too lazy to bother.
Given the choice of either playing with full sheet-music or learning chords/voicings/scales and how to put them together, I'd pick the latter no question, and that is what I recommend to other adults who are learning piano, but I do think it's best to have both. So I can definitely see the value in this, even if it doesn't currently seem to teach the theory and improvisation that, in my opinion, is what really brings the joy into piano playing. I might give it a proper go myself some time soon, to see if it can help me improve my rubbish sight reading.
I can also imagine this being very useful for a beginner... Although I'm singing the praises of chord theory and improvisation I'm guessing most beginners might realistically do better starting with sheet music, for a while at least.
In summary: it might not teach everything, but it looks really useful nonetheless :)
I'm in a similar boat, and agree completely that it's super fun to learn this way. Coming from playing guitar, I was already used to starting from chords and listening for the rest, so it wasn't a big change. I'm a big believer in proper timing and solid rhythms being more important than notes, and learning by ear makes those aspects more intuitive, in my experience at least.
One thing I did find was the very helpful, was putting those stickers with note names on each note on the keyboard. They help me keep track of what I'm playing, and when I compose something I rely on the stickers to help me transcribe.
One thing I did find was the very helpful, was putting those stickers with note names on each note on the keyboard. They help me keep track of what I'm playing, and when I compose something I rely on the stickers to help me transcribe.
I'm lucky that I was taught how to read music as a kid... I'm still rubbish at it, it's slow and painful for anything vaguely complex, but at least I can do it badly when I put mind to it.
I find complex rhythms particularly hard to read. Figuring out timing from a sheet without hearing it is like pulling teeth :D That said, listening to a recording of a great pianist and trying to work out exactly what they are doing can also be very difficult and frustrating when there are chords and complex patterns in two hands, so a good transcription can help a lot. I have a book of Bill Evans transcriptions that I would definitely do more with if I were better at reading music, or had software assistance!
I find complex rhythms particularly hard to read. Figuring out timing from a sheet without hearing it is like pulling teeth :D That said, listening to a recording of a great pianist and trying to work out exactly what they are doing can also be very difficult and frustrating when there are chords and complex patterns in two hands, so a good transcription can help a lot. I have a book of Bill Evans transcriptions that I would definitely do more with if I were better at reading music, or had software assistance!
> "I gave up on sheet music altogether and started working off lead sheets instead..."
I've not taken the plunge yet, but I was convinced this was the way to learn by Scott the Piano guy infomercial on TV. https://youtu.be/k50uk8elR0M
Some day...
I've not taken the plunge yet, but I was convinced this was the way to learn by Scott the Piano guy infomercial on TV. https://youtu.be/k50uk8elR0M
Some day...
I've not tried anything of his myself, but Facebook keeps showing me his ads and it looks good. Good luck with it!
Interesting track, there are a number of pianists (very good ones) that went that route. Keith Jarrett for instance, and Friedrich Gulda.
Keith Jarrett is one of my piano heroes :)
The conventional way of learning piano has a student learning sheet-music pieces that get increasingly difficult as the student improves. It's a fairly linear path, and the vast majority of students will never be able to play a difficult piece like, say, Chopin's Fantaisie Impromptu, as well as a top performer.
Similarly the vast majority of students going the chords/voicings/improvisation route will never be able to play Over The Rainbow as well as Keith Jarrett, but they can at least have the satisfaction and enjoyment of doing it in their own unique way.
Composing your own music takes it a step further again.
The conventional way of learning piano has a student learning sheet-music pieces that get increasingly difficult as the student improves. It's a fairly linear path, and the vast majority of students will never be able to play a difficult piece like, say, Chopin's Fantaisie Impromptu, as well as a top performer.
Similarly the vast majority of students going the chords/voicings/improvisation route will never be able to play Over The Rainbow as well as Keith Jarrett, but they can at least have the satisfaction and enjoyment of doing it in their own unique way.
Composing your own music takes it a step further again.
I think the grading compared to a 'top performer' is about as silly as grading people that jog for an hour after dinner versus Usain Bolt. Music is something you to do enjoy, not because it is top sport!
Obviously there are going to be incredible performers at the high end but there is plenty of fun to be had in the middle and even on the low end.
I have a few pieces that are so beautiful compared to evertyhing else out there that if that were the norm those pieces might as well never be performed again, but then again, it's all subjective and what I like you might not.
> Similarly the vast majority of students going the chords/voicings/improvisation route will never be able to play Over The Rainbow as well as Keith Jarrett, but they can at least have the satisfaction and enjoyment of doing it in their own unique way.
Yes, true. And that too is a lot of fun. I did some jam sessions with the sax (and I wasn't all that good), most fun I had making music to date.
> Composing your own music takes it a step further again.
That might actually at some level be more essential than just reproduction. The cave men had it easy: everything they did was original.
Obviously there are going to be incredible performers at the high end but there is plenty of fun to be had in the middle and even on the low end.
I have a few pieces that are so beautiful compared to evertyhing else out there that if that were the norm those pieces might as well never be performed again, but then again, it's all subjective and what I like you might not.
> Similarly the vast majority of students going the chords/voicings/improvisation route will never be able to play Over The Rainbow as well as Keith Jarrett, but they can at least have the satisfaction and enjoyment of doing it in their own unique way.
Yes, true. And that too is a lot of fun. I did some jam sessions with the sax (and I wasn't all that good), most fun I had making music to date.
> Composing your own music takes it a step further again.
That might actually at some level be more essential than just reproduction. The cave men had it easy: everything they did was original.
Good points!
The beauty of a piece doesn't depend on its technical difficulty. Back when I used to play classical music more I loved playing a few of Chopin's Nocturnes repeatedly - they were about the right level for me so it was fun, and I could play them pretty well, I think. But I also spent a lot of time learning a few harder pieces (like Fantaisie Impromptu), and I remember getting frustrated that it took me ages to learn them and, although in the end I could play them sortof OKish, my efforts just weren't a patch on what a better pianist could do.
Looking back I can see that I shouldn't have picked such hard pieces until I was really ready for them. It was probably a big part of why I ended up quitting for quite a few years before taking it up again more recently with an entirely different approach.
The beauty of a piece doesn't depend on its technical difficulty. Back when I used to play classical music more I loved playing a few of Chopin's Nocturnes repeatedly - they were about the right level for me so it was fun, and I could play them pretty well, I think. But I also spent a lot of time learning a few harder pieces (like Fantaisie Impromptu), and I remember getting frustrated that it took me ages to learn them and, although in the end I could play them sortof OKish, my efforts just weren't a patch on what a better pianist could do.
Looking back I can see that I shouldn't have picked such hard pieces until I was really ready for them. It was probably a big part of why I ended up quitting for quite a few years before taking it up again more recently with an entirely different approach.
So, you are a much better player than I will likely ever be. For me what mattered is that the software would guide me to improve on a piece as fast as possible. To do this it measures your performance very precisely and then judges where you are still weak, and then it starts to bring you up by focusing on the weakest parts first until they are no longer the weakest parts. Like that you improve quite rapidly.
Give it a try if you have an opportunity, I'm curious what you make of it given that you are roughly on the same path but clearly very much ahead of me.
Give it a try if you have an opportunity, I'm curious what you make of it given that you are roughly on the same path but clearly very much ahead of me.
It sounds great to me - sight reading is difficult, and frustratingly hard work too, so assistance in mastering that skill is welcome. I am without MIDI keyboard at present as I play an acoustic upright, but I will look out for an opportunity to give it a proper go.
And I might as well put in a feature idea (which I suspect would be rather tricky to implement but here goes anyway!): it would be awesome if it could take input from a mic and figure out the notes from that, so it could work with acoustic instruments too :)
And I might as well put in a feature idea (which I suspect would be rather tricky to implement but here goes anyway!): it would be awesome if it could take input from a mic and figure out the notes from that, so it could work with acoustic instruments too :)
Had nearly the same idea for the exact same reasons, but wanted it to work on an iPad. Safari has no WebMIDI, so I ended up using WebAudio paired with pitch detection by consensus instead. Here's the result -
https://github.com/apankrat/note-detector
It's not without issues still, but tends to do a reasonably good job at what it aims to do.
https://github.com/apankrat/note-detector
It's not without issues still, but tends to do a reasonably good job at what it aims to do.
Oh, neat! That's very nice, we were actually discussing doing this the other day using FFTs to make it work with real pianos that do not have a silent option installed.
Thanks, yours is very nice too :)
For what it's worth, the auto-correlation route is simpler and more reliable. FFT alone is not enough, because the spectrum tends to be rather ambiguous. There's still a pattern to it for every note played, so it should be possible to deduce a note with a neural net. But I just didn't have time for that, maybe later.
PS. Give my thingy a try if you have a moment. I'm curious if it actually works with any pianos other than mine.
For what it's worth, the auto-correlation route is simpler and more reliable. FFT alone is not enough, because the spectrum tends to be rather ambiguous. There's still a pattern to it for every note played, so it should be possible to deduce a note with a neural net. But I just didn't have time for that, maybe later.
PS. Give my thingy a try if you have a moment. I'm curious if it actually works with any pianos other than mine.
Yes, I know about the auto correlation trick, especially on the low end of the spectrum. I wrote an MP3->midi convertor that is very fast and does not use the GPU or any kind of deep learning, it also gives results that a year ago were SOTA but in the meantime have been passed by deep learning based techniques.
I will definitely use your software. What would be great is to have a 'virtual midi device'. If it works well enough are you open to licensing it? We will sooner or later hit this problem and such a component would make our lives a lot easier, it would also allow me to practice on my real piano using the software, which if I didn't want to play the piano I'd give my left arm for.
On another note: when you know what should be played it is actually easier: you only need to output a '1' or a '0', what is played is what is expected vs what is played is not what is expected.
This may be a simpler problem to solve, and in that context it might even be able to detect chords.
I will definitely use your software. What would be great is to have a 'virtual midi device'. If it works well enough are you open to licensing it? We will sooner or later hit this problem and such a component would make our lives a lot easier, it would also allow me to practice on my real piano using the software, which if I didn't want to play the piano I'd give my left arm for.
On another note: when you know what should be played it is actually easier: you only need to output a '1' or a '0', what is played is what is expected vs what is played is not what is expected.
This may be a simpler problem to solve, and in that context it might even be able to detect chords.
Re: a virtual midi device - I don't know the specifics of WebMIDI, but I'd think it should be rather trivial to code on top of any pitch detector, including mine. In terms of licensing - the code is already BSD-2.
Re: the another note - yes, indeed. I too realized this, because mis-detected notes are usually off by a full octave.
As a side note, after going reading through various papers on the subject, it would seem that an ultimate note detector should really be based on a Const-Q spectrum analyzer feeding into an NN. Training the net, even from scratch, can be as simple as sitting in front of a piano and then repeatedly playing each note in all possible ways. Then, throw this detector in a group with waveform-based detectors, get a weighted consensus and that should, in theory, be pretty damn accurate.
Re: the another note - yes, indeed. I too realized this, because mis-detected notes are usually off by a full octave.
As a side note, after going reading through various papers on the subject, it would seem that an ultimate note detector should really be based on a Const-Q spectrum analyzer feeding into an NN. Training the net, even from scratch, can be as simple as sitting in front of a piano and then repeatedly playing each note in all possible ways. Then, throw this detector in a group with waveform-based detectors, get a weighted consensus and that should, in theory, be pretty damn accurate.
I've done that, and this is roughly how magenta works, which is pretty good, on random input it has about 92.5% accuracy which I think is very impressive.
This seems a bit similar to Melodics[1], except open source and it actually uses the standard notation.
I know how to read music and started taking classical lessons as an adult. Some time last year a friend recommended me to check out Melodics. It has 3 instruments: keys, pads (finger drumming), drums. I've played pads for few mont;hs, that was good, I've felt it improved my timing and rhythm skills quite a bit. Then I switched to keys. I didn't really like that; it felt like I'm mindlessly repeating whatever is on the screen instead of internalising music. OTOH, when playing pads it felt that something stays in my head after playing. So I guess this may be useful to some people.
For me it'd be more interesting to have some sort of memory trainer - I play something on screen few times, then I replay from memory and app shows me a diff maybe? Or when I'm learning a piece, it would give me a challenge "play bars 21-25 both hands" or "play bars 19-23 left hand only".
[1] https://melodics.com/, I'm not affiliated in any way
I know how to read music and started taking classical lessons as an adult. Some time last year a friend recommended me to check out Melodics. It has 3 instruments: keys, pads (finger drumming), drums. I've played pads for few mont;hs, that was good, I've felt it improved my timing and rhythm skills quite a bit. Then I switched to keys. I didn't really like that; it felt like I'm mindlessly repeating whatever is on the screen instead of internalising music. OTOH, when playing pads it felt that something stays in my head after playing. So I guess this may be useful to some people.
For me it'd be more interesting to have some sort of memory trainer - I play something on screen few times, then I replay from memory and app shows me a diff maybe? Or when I'm learning a piece, it would give me a challenge "play bars 21-25 both hands" or "play bars 19-23 left hand only".
[1] https://melodics.com/, I'm not affiliated in any way
This is just what I needed. I just bought an electric keyboard last week and have practiced with Melodics every day since (the free version, so 5 minutes every day :P), but I have some issues with things like how it just keeps going if you mess up or are too slow and the fact that it doesn't use sheet music or let you use your own music (granted, I haven't looked into the subscription version's features yet).
I just opened this and spent 20 minutes trying to play 4 bars I wrote a year ago in MuseScore. I am definitely a noob at playing, hahaha. I always figured being able to play in real-time isn't of much value, but I'd like to be able to improvise and use a piano to enter notes when composing at least, and then I saw some of Lionel Yu's videos and bought a keyboard... (https://www.youtube.com/c/LionelYuPiano/videos)
I definitely appreciate the auto-replay, simultaneous note count limit, waiting until you hit a key (both at the start and mid-song if you're slow), and scoring.
I just opened this and spent 20 minutes trying to play 4 bars I wrote a year ago in MuseScore. I am definitely a noob at playing, hahaha. I always figured being able to play in real-time isn't of much value, but I'd like to be able to improvise and use a piano to enter notes when composing at least, and then I saw some of Lionel Yu's videos and bought a keyboard... (https://www.youtube.com/c/LionelYuPiano/videos)
I definitely appreciate the auto-replay, simultaneous note count limit, waiting until you hit a key (both at the start and mid-song if you're slow), and scoring.
> Or when I'm learning a piece, it would give me a challenge "play bars 21-25 both hands" or "play bars 19-23 left hand only".
It does exactly that, pick 'auto' mode.
It does exactly that, pick 'auto' mode.
from the last paragraph of the above link
>The software is not ‘open source’, we have not yet decided on how we plan to go forward with this project in the longer term, but the code is on GitLab and if you want to see what makes it tick or suggest improvements then of course you are more than welcome to do so.
>The software is not ‘open source’, we have not yet decided on how we plan to go forward with this project in the longer term, but the code is on GitLab and if you want to see what makes it tick or suggest improvements then of course you are more than welcome to do so.
Ha ha, good one, Jacques.
Another piano app, by me, this is as rudimentary as one can get, in Python. It turns your PC keyboard into a "piano":
Play the piano on your computer with Python:
https://jugad2.blogspot.com/2013/04/play-piano-on-your-compu...
The comments by people wih more music knowledge than me (zero) were interesting and informative.
Another piano app, by me, this is as rudimentary as one can get, in Python. It turns your PC keyboard into a "piano":
Play the piano on your computer with Python:
https://jugad2.blogspot.com/2013/04/play-piano-on-your-compu...
The comments by people wih more music knowledge than me (zero) were interesting and informative.
> Unfortunately, because Mozilla is too busy improving the world instead of their browser and WebMIDI is not supported on Firefox you have to use Chrome
Attention Mozilla
Attention Mozilla
I did a lot of tab in my youth, but got nowhere with it; I think because I didn’t understand theory like chords, scales etc, and when exposed, I never recognized them. Ear players seem to be able to chord along to just about anything and then add detail until it resembles the original. There must be some way to help the learner recognize the theory while teaching rote.
This is something that is very high on my wishlist, what I would like to do (and some of the code is already in the project) is to detect chords as they are played and annotate them. But because there are a lot of ways to express the same chord this is not so simple. Work in progress!
I once wanted to build something like a graph of chord transitions from thousands of different songs, with weights based on how often that transition occurs in a genre given the previous N chords. Maybe something like that would help with giving chords natural names for their context?
Yes, that is one avenue. I made a - hopefully exhaustive - list of chords the other day and I was surprised at how much I did not know. The next step is to analyze my midi file collection to see what kind of graph can be constructed and how the edges can be annotated. It's a very interesting problem. And in a way that is why I love this project: it combines both of the things I love.
Do you mean guitar tabs? (By "tab")? Seeing the music theory and logic behinds chords and scales is a lot easier with keyboard instruments.
> Note that it would have been trivial for us to force our users into a relationship with us by having to make an account and we made a very conscious choice not to do this.
Bravo! Well done and well said - I really wish more projects and systems were developed with this mindset, viewing users as people to be empowered rather than as resources to be captured and exploited.
Bravo! Well done and well said - I really wish more projects and systems were developed with this mindset, viewing users as people to be empowered rather than as resources to be captured and exploited.
That's an interesting tool. As a former professional pianist, I'd repeat what I feel like the limitation of such a program: it does teach you note reading, which is nice, but really basic piano technique is a necessary preamble. You mentioned that your boy spent a whole day on this; with a bit of guidance, a full beginner after a few hours should be able to play this very simple melody with one hand, and to add a couple of bass notes with the other, for a much more satisfying result (for the player and for the audience). I know it because I've done it with many children in the past.
So you should probably add something like some introduction material / tutorial... maybe a couple of videos.
So you should probably add something like some introduction material / tutorial... maybe a couple of videos.
Yes, this is an urgent need and recognized. We are in the process of hiring a piano teacher to help with this as well as with making the program more absolute beginner friendly.
This project is a lot of work as it is today, it was made mostly to satisfy a need that I felt myself and my colleagues picked it up and ran with it so now it has some proper power behind it. We fully intend to 'stay the course' and make this a best-in-class offer but there is a long way to go before we can claim that we have achieved that.
As for the satisfaction, I think you may be off the mark there, I've had enough piano education to know what that was like and Luca is having a lot more fun with this program than I ever did being taught the piano by teachers at roughly the same age.
This project is a lot of work as it is today, it was made mostly to satisfy a need that I felt myself and my colleagues picked it up and ran with it so now it has some proper power behind it. We fully intend to 'stay the course' and make this a best-in-class offer but there is a long way to go before we can claim that we have achieved that.
As for the satisfaction, I think you may be off the mark there, I've had enough piano education to know what that was like and Luca is having a lot more fun with this program than I ever did being taught the piano by teachers at roughly the same age.
I understand the issue with Firefox not supporting WebMIDI, but it would be nice if instead of redirecting to a separate page you could show the same message as a warning on the main page. Let me see the app and see that it's worth coming back in Chrome.
Good point, thank you, I will add this to the todo list.
Regrettably it's not working on Firefox[1], but hey at least I can report wrongthink YouTube videos.
[1] https://pianojacq.com/firefox.html
[1] https://pianojacq.com/firefox.html
Yes, very much regrettably, I now find myself using Chrome when I really would like to avoid that. But unfortunately there is no way around this right now that I've found to be reliable enough and easy enough to the point that I would advise users to go that route. Sorry.
Note this 8(!) year old bug:
https://bugzilla.mozilla.org/show_bug.cgi?id=836897
It's embarrassing.
Note this 8(!) year old bug:
https://bugzilla.mozilla.org/show_bug.cgi?id=836897
It's embarrassing.
> Mozilla is too busy improving the world instead of their browser and WebMIDI is not supported
That is misleading. Mozilla chose not to implement it for security concerns. More context: https://github.com/mozilla/standards-positions/issues/58#iss...
That is misleading. Mozilla chose not to implement it for security concerns. More context: https://github.com/mozilla/standards-positions/issues/58#iss...
Security concerns that Chrome seems to be able to deal with just fine. If that's the issue then block sysex but allow the rest, the midi (v1) protocol isn't that hard.
Meanwhile, there are 100's of WebMIDI capable applications now and Mozilla still claims there is no demand for a feature they do not support. That's backwards, of course a feature that you do not support is not in demand because people simply go elsewhere.
The cumulative effect of a lot of little crappy decisions like this is part of why FireFox' market share eroded (the other is Google's anti competitive behavior).
Meanwhile, there are 100's of WebMIDI capable applications now and Mozilla still claims there is no demand for a feature they do not support. That's backwards, of course a feature that you do not support is not in demand because people simply go elsewhere.
The cumulative effect of a lot of little crappy decisions like this is part of why FireFox' market share eroded (the other is Google's anti competitive behavior).
> Unfortunately, because Mozilla is too busy improving the world instead of their browser and WebMIDI is not supported on Firefox you have to use Chrome
This may teach you to "play" piano, but the real question is whether this sort of rote memorization will really teach you how music fundamentally works. Seems a bit like re-typing Javascript programs in order to "learn" programming?
Will you be able to improvise after enough of this? Play songs by ear?
There are a few fundamental musical concepts that I'm not sure you easily pick up by rote memorization:
* Most music is based around "chords", which have non-obvious mappings to physical keys. (They're linear intervals in frequency space, but that doesn't make it easy to realize that D/F#/A and Ab/C/Eb are two representations of essentially the same thing)
* Likewise, songs can be transposed up or down without affecting their fundamental attributes. (They just sound "higher" or "lower"). However, to do that, you need to be able to shift +/- N keys which is tricky given the difficult layout of a piano. To really do this well, you need to understand the concept of scales/modes.
Will you be able to improvise after enough of this? Play songs by ear?
There are a few fundamental musical concepts that I'm not sure you easily pick up by rote memorization:
* Most music is based around "chords", which have non-obvious mappings to physical keys. (They're linear intervals in frequency space, but that doesn't make it easy to realize that D/F#/A and Ab/C/Eb are two representations of essentially the same thing)
* Likewise, songs can be transposed up or down without affecting their fundamental attributes. (They just sound "higher" or "lower"). However, to do that, you need to be able to shift +/- N keys which is tricky given the difficult layout of a piano. To really do this well, you need to understand the concept of scales/modes.
> Will you be able to improvise after enough of this? Play songs by ear?
How many Herbie Hancock's have you come across after finishing a grade 1 piano book? Of course a single exercise app or book isn't going to create the perfectly rounded musician. For a child or beginner player, this looks like a great tool.
Advanced musical skill is entirely composable. Having solid improvisational skill or technical ability largely depends on how much you practice those various facets of your instrument. The reality is that probably 95%+ of people who pick up an instrument in their lives never make it to that point anyway.
You're trying to critique this tool with the expectation that a single learning resource should address all aspects of playing an instrument, when that's never been true of any musical learning resource ever.
How many Herbie Hancock's have you come across after finishing a grade 1 piano book? Of course a single exercise app or book isn't going to create the perfectly rounded musician. For a child or beginner player, this looks like a great tool.
Advanced musical skill is entirely composable. Having solid improvisational skill or technical ability largely depends on how much you practice those various facets of your instrument. The reality is that probably 95%+ of people who pick up an instrument in their lives never make it to that point anyway.
You're trying to critique this tool with the expectation that a single learning resource should address all aspects of playing an instrument, when that's never been true of any musical learning resource ever.
It's a nice tool, didn't intend to come off that negatively. More of a thought experience about the limitations of these types of tools.
I'm not sure it's quite beginner level, as I don't see anything that shows you where the note is on the keyboard? You'd have to start with a bit of reading knowledge, or be willing to just hit random keys until you figure out the mappings.
I'm not sure it's quite beginner level, as I don't see anything that shows you where the note is on the keyboard? You'd have to start with a bit of reading knowledge, or be willing to just hit random keys until you figure out the mappings.
If you notice the video linked (you really should not be downvoted for good input like this), then you'll notice that my son Luca is playing a piece which is fairly repetitive, but he did not have any instruction at all, the app helps you figure out what is where all by itself, 'random' keys will quickly guide you to the right keys. and then from there once you get the hang of it relative movement should come fairly naturally. The biggest hang-up is the piano keyboard itself with its unequal spacing between adjacent white keys.
HN can be weirdly sensitive to comments about music being hard; fundamentals being mandatory, etc. I bet if we criticized the code on Music News (not a real thing) it'd be the same way.
i think op may be hinting that learning basic chords and scales (which is not difficult) makes it much more fun to play things (and thus more productive) rather than trying to follow each note on a pentagram.
No, it teaches you how to read the scores pretty good actually. The 'rote memorization' can be pushed as well but that's not the goal here, the goal is to learn to read notes fast enough that you can actually play the music. (See attached video).
As for improvisation and playing songs by ear: improv requires some knowledge about how music is structured, I have some ideas on showing what the underlying structure is but these have not yet been implemented, we are for now concentrating on accuracy of the rendering of the score (which is still plenty of work) and fingering.
Playing by ear is mostly to have a good association between the keys and the sounds they make, this gets harder with chords. As with everything, practice makes perfect and doing this frequently seems to help me quite a bit by picking out songs by ear. So even if it isn't a direct goal of any of the modes of operation of the software it does seem to be a side effect. We will add an 'ear training' module at some point.
As for improvisation and playing songs by ear: improv requires some knowledge about how music is structured, I have some ideas on showing what the underlying structure is but these have not yet been implemented, we are for now concentrating on accuracy of the rendering of the score (which is still plenty of work) and fingering.
Playing by ear is mostly to have a good association between the keys and the sounds they make, this gets harder with chords. As with everything, practice makes perfect and doing this frequently seems to help me quite a bit by picking out songs by ear. So even if it isn't a direct goal of any of the modes of operation of the software it does seem to be a side effect. We will add an 'ear training' module at some point.
I do like that you are showing the score, instead of doing some sort of Guitar Hero style / abstract grid. That will definitely help people that want to translate this into reading sheet music. And you may be right in that doing this enough will give you a sense of what keys make what sounds, which then can clue you into to how songs work "under the hood".
Since you're rendering the score in realtime, you could let users transpose songs up and down which would really help them learn scales.
Very nice project, though.
Since you're rendering the score in realtime, you could let users transpose songs up and down which would really help them learn scales.
Very nice project, though.
> I do like that you are showing the score, instead of doing some sort of Guitar Hero style/abstract grid
This and the way it shows these little lines when you hit some wrong keys is really helpful if you are just learning sight reading. Credit for this should go to Piano Booster probably though.
This and the way it shows these little lines when you hit some wrong keys is really helpful if you are just learning sight reading. Credit for this should go to Piano Booster probably though.
Absolutely, Pianobooster has some rally neat concepts.
Good idea, it is already on the list but not in that context, I will increase its priority.
I'd wager it can't teach you abstract concepts but since abstract concepts are much easier to learn with the practical knowledge of using them, it would definitely be helpful. Also honing motoric skills is a key part in doing many things - arts, sports - so getting accustomed to that is a big plus too. The downside of doing only that is yes, you will be quite lost with the abstract and also reinforce some bad habits.
So no cynical HN commentary from me! If it works, it is definitely a big help for learning to play piano.
So no cynical HN commentary from me! If it works, it is definitely a big help for learning to play piano.
Not everyone wants to become a composer. A lot of people just want to be able to sit down at a piano and play music they enjoy. Learning by rote works very well for that.
This is great! My son, who normally loses interest in long songs that are a bit harder than his level, got through The well tempered clavier (one of the sample songs).
The timing/spacing of the notes is a bit off though, I wish it scrolled according to beats instead of speeding up and down depending on what type of note was being played. I understand it has to stop and wait for the note to be played, but some of the whole notes threw me off.
The timing/spacing of the notes is a bit off though, I wish it scrolled according to beats instead of speeding up and down depending on what type of note was being played. I understand it has to stop and wait for the note to be played, but some of the whole notes threw me off.
Yes, this is one of the harder problems, a lot of brain cycles got spent on getting it as accurate as it is now but there definitely is still room for improvement. Keep in mind that the browser is a super hostile environment for this kind of application and accurate (millisecond level) time tracking in the browser is next to impossible. The solution we picked was to reduce/increase the speed of the scroll based on the error term between where we are and where we should have been. This keeps the scolling reasonably smooth while giving the smallest error in time. If we'd prioritize the scrolling it would look better but the timing would be further off. Suggestions for improvements to the code are of course very welcome, the code is extensively commented. There are some really nasty gotcha's in the interaction between the setup of the original midi file on the score leading to some decisions that retrospectively were probably sub-optimal but I will definitely revisit those in the near future to improve on them. Fun fact: the score is drawn three times to get it as close to time correct and visually attractive as possible, first a rough plan to get everything in the right bar and in the right order, then as close as possible to the right point in time and finally any odd space is used to spread out the note heads so it looks visually attractive. The plan is to remove all that and to first create an intermediary representation and then to draw that in one go.
My dad is a music tutor, piano and guitar.
He does two different "tracks" according to what the student wants: (1) learn currently popular songs or (2) learn didactic music in the classical tradition (e.g. Czerny). In either track he's really teaching music theory: most popular music will be I-IV-V; jazzy music with turnarounds do ii-V-I tricks, etc. He doesn't teach an instrument, he teaches music.
He does two different "tracks" according to what the student wants: (1) learn currently popular songs or (2) learn didactic music in the classical tradition (e.g. Czerny). In either track he's really teaching music theory: most popular music will be I-IV-V; jazzy music with turnarounds do ii-V-I tricks, etc. He doesn't teach an instrument, he teaches music.
Very nice! Is your dad available for a consultation?
We would very much like to teach things properly but for that we need more understanding and someone who has been teaching music would be a huge asset to us.
If you could please ask that would be awesome.
We would very much like to teach things properly but for that we need more understanding and someone who has been teaching music would be a huge asset to us.
If you could please ask that would be awesome.
My mother-in-law used to teach piano. What's the best way to put her in touch with you?
[email protected] , thank you very much.
There’s a cottage industry of piano learning apps, and an equivalent amount of people getting frustrated at the lack of progress and quitting
That's why we spent quite a bit of time on making interesting metrics that show your progress over time, even if it is small per increment the overall effect is substantial.
I had good experience with Android app made by soft²cat, but it long gone. Unlike other guitar-hero like apps, it waits for the note to be pressed, so I cannot miss it, and the app discards the note when I press the note earlier. It allows me to play melodies at my own tempo: slower or faster than the original, which is much easier for me.
The rate at which piano teaching apps come and go was one of the factors in our decision to ensure that the app would also work 'stand alone', it's just straight javascript and will work from a zip file taken from the repository. Just unzip and play, no toolchain required.
Search piano on iOS! It's depressing.
We purposefully chose not to make 'an app' right now but to work in the browser, this gives us a different form factor to play with and some pretty powerful tools to get to the point where we really understand the space well enough to make a move. Think of the present project as a study, a way to have a zero-tooling set-up that allows us to work on this without having to worry about platforms and other fast moving targets. The idea is to first make it work very well and to address all of the valid concerns raised in this thread (of which we for the most part were already aware, but HN wouldn't be HN without its brutal reception of any piece of work). Then when it is bullet proof against such criticism it can be launched to a much larger audience.
The sadness of my piano life is not my fingers but my dumb brain. No matter how intensely I practice and memorize a piece, if I step away from it for a week I'll forget 30% of it.
Curiously, my audio memory is very strong. I can hear music in my head clearly and in detail, even songs and pieces I haven't heard in years. This does not translate to "finger-memory".
Curiously, my audio memory is very strong. I can hear music in my head clearly and in detail, even songs and pieces I haven't heard in years. This does not translate to "finger-memory".
I'm going to guess here and say that you are practicing a very limited repertoire, rather than to learn to play the instrument, in which case you are memorizing movements instead of music.
Is that correct?
Is that correct?
A fine guess but actually I cycle through what I think is a pretty big range. E.g., several Denes Agay classical+romantic compilations, the standard basic Bach + Beethoven, simple Chopin, John Field, Philip Glass (repetitive patterns help), etc.
Another frustrating thing is it's not just long-term finger memory that kills me, my real-time memory bus is slow when learning a piece -- the transfer of "what notes comes next" down to my fingers, just very slow and error-prone. I've tried a mental exercise where I try to only think about what to play "next" and just let what I'm playing "now" take care of itself. It kind of works and feels very different. Basically, to stay 2 beats ahead of the music at all times.
Ugh, but it's still a struggle. My finger dexterity is fine. My musical knowledge is actually pretty advanced and I can play expressively and convincingly -- but only when I can remember the damn piece. My memory is like a balloon that won't stay inflated.
https://www.amazon.com/Classics-Moderns-Intermediate-Grade-M...
Another frustrating thing is it's not just long-term finger memory that kills me, my real-time memory bus is slow when learning a piece -- the transfer of "what notes comes next" down to my fingers, just very slow and error-prone. I've tried a mental exercise where I try to only think about what to play "next" and just let what I'm playing "now" take care of itself. It kind of works and feels very different. Basically, to stay 2 beats ahead of the music at all times.
Ugh, but it's still a struggle. My finger dexterity is fine. My musical knowledge is actually pretty advanced and I can play expressively and convincingly -- but only when I can remember the damn piece. My memory is like a balloon that won't stay inflated.
https://www.amazon.com/Classics-Moderns-Intermediate-Grade-M...
That's interesting. So, personal story time: I used to have a pretty scarily good memory. But as I get older I find that I only think that I still have a good memory. Every day there is evidence that it isn't nearly as good as it used to be and if there is anything that scares me about getting older it's that.
Some parts of what you write remind me of stuff I've read about flying an airplane: never let the airplane get ahead of you. In music that would be 'never let the music get ahead of you', once it does the cognitive overload becomes overwhelming.
Then: I listen a lot to the pieces that I want to play, over and over again, until I have the music in my head and then playing piano becomes a lot more like singing, instead of memorizing the movements you memorize the music and playing then becomes actual playing, you now have time and braincycles to spend on making it sound nice rather than just what to play and when. I only have a few pieces in my very limited repertoire at that level but the difference is huge with the remainder, I can make them sound in whatever way I want, I can vary them when and where I want to play variations on the entire piece by changing style and so on. That is what playing piano to me is all about and I intend to spend a lot of time on reaching that kind of proficiency with more music to broaden the range a bit.
I wished I had some tip to help you, also, I'd love it if you gave the software a spin and let me know if there is a way that I can improve it so that it does help you, if that requires setting up a trial version then I'm all for it. Let me know if you're up for this. best regards, Jacques
Some parts of what you write remind me of stuff I've read about flying an airplane: never let the airplane get ahead of you. In music that would be 'never let the music get ahead of you', once it does the cognitive overload becomes overwhelming.
Then: I listen a lot to the pieces that I want to play, over and over again, until I have the music in my head and then playing piano becomes a lot more like singing, instead of memorizing the movements you memorize the music and playing then becomes actual playing, you now have time and braincycles to spend on making it sound nice rather than just what to play and when. I only have a few pieces in my very limited repertoire at that level but the difference is huge with the remainder, I can make them sound in whatever way I want, I can vary them when and where I want to play variations on the entire piece by changing style and so on. That is what playing piano to me is all about and I intend to spend a lot of time on reaching that kind of proficiency with more music to broaden the range a bit.
I wished I had some tip to help you, also, I'd love it if you gave the software a spin and let me know if there is a way that I can improve it so that it does help you, if that requires setting up a trial version then I'm all for it. Let me know if you're up for this. best regards, Jacques
More thinking about this: have you thought of breaking up the pieces you practice in to much shorter sections of maybe 20-30 bars and practice those as individual pieces of music and the stitch them together once you have mastered all of them? That might help you with the memorization, memorization of a very lengthy piece is hard if you do it with playthroughs but if the playthrough is a smaller part then it might work.
That's actually how I usually practice, by section...
I think I'm finally gonna try something I've been thinking for years: Photocopy a score, then take scissors to it and come up with a condensed version with only the parts that I blank on. I might discover that in a four-page piece, I only need a half-dozen nudges.
Thanks for the thoughts, Jacques!!
I think I'm finally gonna try something I've been thinking for years: Photocopy a score, then take scissors to it and come up with a condensed version with only the parts that I blank on. I might discover that in a four-page piece, I only need a half-dozen nudges.
Thanks for the thoughts, Jacques!!
When you practice and learn a piece, how exactly to you practice and memorize it?
Slowly and deliberately, focusing on the parts that elude me (i.e., I'll skip the intro and other sections that are 100% solid in muscle memory). Paying attention to fingerings. I'll usually focus on 1-2 pieces at any given time.
But I do go through spurts, where I practice every day but then for whatever reason step away from days or 1-2 weeks, which might be killing the neuron connections.
I also round-robin my nighttime between piano, guitar, and composition, which probably ain't helping.
But I do go through spurts, where I practice every day but then for whatever reason step away from days or 1-2 weeks, which might be killing the neuron connections.
I also round-robin my nighttime between piano, guitar, and composition, which probably ain't helping.
It definitely isn't helping, haha.
I also play piano, guitar, and compose, and I used to struggle with memorizing, but I've been reading a lot of piano practicing theory and have drastically upgraded my finger memory, mostly by using these resources:
https://www.pianostreet.com/smf/index.php?topic=12590.0 (look for Bernhard's post)
http://www.pianopractice.org/
Loosely, stop using sheet music except when you're initially memorizing the piece, learn music in much smaller chunks than initially expected, and play entirely from memory whenever possible, even when you're practicing the repetition of something.
The fact that you can remember music easily in your head means that if you spend 1-3 years really banging out the hand-brain connection, you might be able to rocket your playing forward. Raw improvisation where you think of the melody before you play it will help, too, if you don't already do that.
I also play piano, guitar, and compose, and I used to struggle with memorizing, but I've been reading a lot of piano practicing theory and have drastically upgraded my finger memory, mostly by using these resources:
https://www.pianostreet.com/smf/index.php?topic=12590.0 (look for Bernhard's post)
http://www.pianopractice.org/
Loosely, stop using sheet music except when you're initially memorizing the piece, learn music in much smaller chunks than initially expected, and play entirely from memory whenever possible, even when you're practicing the repetition of something.
The fact that you can remember music easily in your head means that if you spend 1-3 years really banging out the hand-brain connection, you might be able to rocket your playing forward. Raw improvisation where you think of the melody before you play it will help, too, if you don't already do that.
Thanks for the links!
It'd be amazing if it's not a memory problem (at least, not any more so than for most other people who learn piano) but just have to learn a new way of connecting them neurons.
It'd be amazing if it's not a memory problem (at least, not any more so than for most other people who learn piano) but just have to learn a new way of connecting them neurons.
I wouldn't be surprised if you find yourself with a much greater facility for memorization. The only caveat is that it takes longer to get to the point where you can "play through" a piece, which feels less efficient than learning by sheet music, but I find that my ability to interpret and make phrasing choices with memorized music is much greater than it is with music I'm reading.
Bernhard is awesome. I've learned more about piano playing and teaching from his posts than from my 'live teacher'.
I completely agree; Bernhard's posts made me realize that even professionals do not necessarily know the most efficient methods for teaching and learning, and that it may be necessary for students, even as amateurs, to challenge and seek advice beyond what their teachers are capable of, despite the existing gap in skills.
I had the opportunity to beta test it and while it still has rough edges my daughter and me had a lot of fun with it. Everyone learns differently but in my experience direct feedback is often such a boost in learning speed and efficiency that I wish there were more applications like this. Also the attitude towards user privacy is highly appreciated.
On this note,seeing as cheap qwerty keyboards easily come in $5 and good in $20, is there a "piano" keyboard that only has keys laid out in piano fashion, rest left to drivers a small software that plays appropriate tones ....
I'd prefer something small- only two octaves and a plus/minus button to shift octaves.
I'd prefer something small- only two octaves and a plus/minus button to shift octaves.
I will look at using the regular PC keyboard as a stand-in midi keyboard of ~ 2 octaves. This is not going to give a great experience (especially because many keyboards have limited key-rollover) but it should give a less-hardware-required onboarding experience, which is always a good thing.
Thanks for responding. I've tried using paino software before but the "key press" experience is suboptimal.
While you're right about onboarding effort, my guess/hope is that the required driver software shouldn't be hard for me or manufacturer to write.
While you're right about onboarding effort, my guess/hope is that the required driver software shouldn't be hard for me or manufacturer to write.
The code is up on gitlab, you can hack on it for private purposes to your hearts content.
You may want to check out https://synthesiagame.com/ which is a commercial product that does the same thing. You don’t have to use the guitar hero style of notes, you can turn on proper sheet music.
Synthesisa is awesome. I think it is not a direct competition to pianojacq because while similar in some ways it is different in many ways too. murbard summarized some differences in another comment better than I could. In my opinion the two apps and a piano teacher complement each other.
I find it really bad taste to promote something this way. So a developer comes here to tell his story and open his mind a little about a great tool he developed and this is the only thing you have to say?
I think you're being harsh, It was in no way a critique of PianoJacq and I personally appreciate getting an overview of what similar things are out there that I could also be interested in and how they relate
It is relevant to the discussion. I agree the tone and content could be a bit different, but then so can yours.
[deleted]
This is good timing. I was thinking about taking up the piano for fun. My plan is to rent a keyboard for a month (Roland FP10/FP30) and going the YouTube route to see if I liked it.
There is an `about:config` setting to enable webmidi in Firefox `dom.webmidi.enabled`. It's not a great experience to have to set that by hand. It does work though, at least in Linux.
It looks like your site isn't detecting whether the webmidi feature is present though, it looks like its just doing dumb UA filtering.
It looks like your site isn't detecting whether the webmidi feature is present though, it looks like its just doing dumb UA filtering.
No, it isn't doing dump UA filtering, it tests whether the WebMIDI interface works and if it does then it proceeds to work.
So that must be something else.
You can check the code if you don't believe me, it's plain JS.
So that must be something else.
You can check the code if you don't believe me, it's plain JS.
It’s worth noting that I don’t use Firefox by default, but Safari, and the messaging only talks about Firefox.
I thought I’d read that WebMIDI was declined to be implemented by Apple because of security issues (including the ability for random webpages to send firmware updates to connected MIDI peripherals), and going back about four months I found an article here on HN (https://news.ycombinator.com/item?id=23676109 — note the top comment https://news.ycombinator.com/item?id=23679063).
I seriously doubt that Firefox is going to implement this.
I thought I’d read that WebMIDI was declined to be implemented by Apple because of security issues (including the ability for random webpages to send firmware updates to connected MIDI peripherals), and going back about four months I found an article here on HN (https://news.ycombinator.com/item?id=23676109 — note the top comment https://news.ycombinator.com/item?id=23679063).
I seriously doubt that Firefox is going to implement this.
That's just too bad then. Also, it really is nonsense, yes, you could do that, but that's a matter of blocking those specific calls, a pretty simple implementation would already be extremely useful and nobody even wants the firmware update extensions to be implemented, you can fairly trivially filter those out and redflag any application that even tries to do so.
It would make a lot of sense for a browser team to implement just the input part of the API and leave the firmware flashing part out. Or have it implemented but disabled by default, just like the access to the microphone / camera is.
Absolutely, and also program change/note on/note off those would already enable a whole pile of applications.
It's typical Mozilla. We won't do 'X' because 'Y' but when you drill down they could easily do 'X' but they won't and 'Y' is just whatever they can throw up to get away from it without having to dig in and get it over with in the best possible way. If users want something and you start throwing up technical reasons why you won't instead of can't then you've lost that user if alternatives are available.
This is how you kill your market share.
An interesting bit from one of my favorite IT related book, the Soul of a new machine revolves around the 'mode' bit. The engineers all want a 'mode' bit because it will make their life that much easier, and they keep throwing up flak why it has to be that way and the CEO keeps his foot down and says 'no mode bit, period'. 'I don't care how you get it done but that's the constraint and you will have to deal with that constraint'. And so in the end they find a way.
With all the smarts at Mozilla if they really wanted to get it done, I'm sure they'd find a way. The existence proof is Chrome, and I've yet to hear of a random website corrupting a synth's firmware using Chrome.
It's typical Mozilla. We won't do 'X' because 'Y' but when you drill down they could easily do 'X' but they won't and 'Y' is just whatever they can throw up to get away from it without having to dig in and get it over with in the best possible way. If users want something and you start throwing up technical reasons why you won't instead of can't then you've lost that user if alternatives are available.
This is how you kill your market share.
An interesting bit from one of my favorite IT related book, the Soul of a new machine revolves around the 'mode' bit. The engineers all want a 'mode' bit because it will make their life that much easier, and they keep throwing up flak why it has to be that way and the CEO keeps his foot down and says 'no mode bit, period'. 'I don't care how you get it done but that's the constraint and you will have to deal with that constraint'. And so in the end they find a way.
With all the smarts at Mozilla if they really wanted to get it done, I'm sure they'd find a way. The existence proof is Chrome, and I've yet to hear of a random website corrupting a synth's firmware using Chrome.
I would.... but you also keep redirecting me to /firefox.html. Thus, I can't look at the site (on my phone) to be able _see_ the code.
Wow, thanks for creating this @jacquesm! I'm going to give this a spin tonight with my kids!
Have fun! And if I can help let me know.
Works like a charm with Linux/Chrome and my Roland HP504! Interesting that there's no hints in Chrome that anything is connected.
For the product, a great next step would be a link to a suitable midi collection for beginners. The included Frere Jacques sample is very short, and the other midi's that I can find using google are two-handed right away.
For the product, a great next step would be a link to a suitable midi collection for beginners. The included Frere Jacques sample is very short, and the other midi's that I can find using google are two-handed right away.
Yes, material is our big next challenge, we really need to work on that. But I'm so happy that we got the software written as fast as we did, obviously it is built on a lot of hard work by others (Tone.js, VexFlow, Jquery, to name a few) but it was still quite a bit of work to get to this stage. The browser is a pretty hostile environment to write real time applications in.
Note that you can make any piece playable with just the one hand and you can reduce the complexity of the chords to just about your level and then slowly increase it from there. That's the quickest way to get really nice result and to have fun doing it.
Note that you can make any piece playable with just the one hand and you can reduce the complexity of the chords to just about your level and then slowly increase it from there. That's the quickest way to get really nice result and to have fun doing it.
I enjoyed PianoMarvel
I used that too for ~6 months some years back and definitely got something out of it, though it was imperfect. There's too much lag in any such MIDI system for it to truly correct your timing and PianoMarvel doesn't even attempt to judge your volume or the duration of notes, if I remember correctly. But it gives you a reasonable progression of pieces from easy to harder and forces you to actually play them. Also some scores were poorly typeset, particularly things like trills. Overall, it helped me improve my ability to read sheet music fluently, which is what I primarily wanted out of it. I learned more with a teacher, of course.
Trills and decoration in general (rolled chords) for instance are another area where anything midi based will fare poorly, because the 1:1 correspondence between note-heads and keys played is lost. We intend to try to reverse engineer such decorations and to show them properly. With trills and such it can be quite hard because it is perfectly possible to write out the same sequence, a trill is essentially a macro that expands to some number of notes (and some trills can be quite complex, for instance the first few bars of Partita 1 by JS Bach will throw you for a loop if you listen to them and then look at the score).
This will not teach you to play piano. It will teach you how to play beginner piano badly, and then will create intense confusion if you try to learn harder pieces because you will not have learned basic finger techniques - which include counter-intuitive finger-over/under moves and unexpected hand position changes that anticipate what you need to play next.
Piano sheet music includes these finger position hints. Unfortunately they're not optional.
It's hard to include correct fingering in the model without some kind of camera and image recognition system, or special keys and finger sensors, or something equally obscure.
There's nothing wrong with the idea of gameifying piano, but it's a really hard problem to solve well. The usual result with piano teaching apps of all kinds - as someone else commented below - is kids get up to Grade 1 (maybe 2) and then hit a brick wall.
The reality is that playing any instrument with non-trivial competence is really fucking hard. It takes everyone a ridiculous amount of time and effort to get good at it - even those with talent.
So there's a trade off between trying to offer easy but limited success with some entertainment value, and going the whole way and doing it properly - which all but the most musical kids get very bored by, because it's such hard work.
Piano sheet music includes these finger position hints. Unfortunately they're not optional.
It's hard to include correct fingering in the model without some kind of camera and image recognition system, or special keys and finger sensors, or something equally obscure.
There's nothing wrong with the idea of gameifying piano, but it's a really hard problem to solve well. The usual result with piano teaching apps of all kinds - as someone else commented below - is kids get up to Grade 1 (maybe 2) and then hit a brick wall.
The reality is that playing any instrument with non-trivial competence is really fucking hard. It takes everyone a ridiculous amount of time and effort to get good at it - even those with talent.
So there's a trade off between trying to offer easy but limited success with some entertainment value, and going the whole way and doing it properly - which all but the most musical kids get very bored by, because it's such hard work.
The reality is that playing any instrument with non-trivial competence is really fucking hard.
You're not joking either! I played trumpet when I was in MS/HS. Sure, you can play simple songs at your first concert after 3 months of playing - but it takes 3-5 years to get competent, not an expert mind you, but competent.
I started playing guitar as an older adult. Same story. You can start playing songs starting with lesson 1. This is great! Look at all the progress I'm making! I'm on year 7 now and I'm just starting to get to the point where I'd say yeah, I'm good. Not great, but good.
To your point, it takes a long time to master an instrument and no one has discovered a short cut. It's hard work.
On another note, once you appreciate the work it takes to not just become good but become great at playing an instrument then support those who've dedicated their lives to becoming great. Otherwise it's a skill humanity is going to lose over time and that would be really sad.
You're not joking either! I played trumpet when I was in MS/HS. Sure, you can play simple songs at your first concert after 3 months of playing - but it takes 3-5 years to get competent, not an expert mind you, but competent.
I started playing guitar as an older adult. Same story. You can start playing songs starting with lesson 1. This is great! Look at all the progress I'm making! I'm on year 7 now and I'm just starting to get to the point where I'd say yeah, I'm good. Not great, but good.
To your point, it takes a long time to master an instrument and no one has discovered a short cut. It's hard work.
On another note, once you appreciate the work it takes to not just become good but become great at playing an instrument then support those who've dedicated their lives to becoming great. Otherwise it's a skill humanity is going to lose over time and that would be really sad.
> So there's a trade off between trying to offer easy but limited success with some entertainment value, and going the whole way and doing it properly - which all but the most musical kids get very bored by, because it's such hard work.
I think the value is in option #1.
My son is 5 years old and just started grade 1; his best friend at school has two parents who are quite musical, and so he's been taking violin lessons. Thanks to COVID, he was stuck in the house, bored, all summer, and decided to spend basically the whole time practicing (mind you, the kid is barely 6) and now he's way beyond the other kids his age and is extremely proud of what he's accomplished.
Conversely, my son hates fine motor skill activities; he's not good at them, so he doesn't want to do them, so he doesn't get good at them. We couldn't convince him to practice drawing, writing his letters, anything; he didn't want anything to do with it. Until, that is, we told him (not entirely falsely) that improving his ability to use a pen, pencil, marker, etc. with precision would also help his ability to use a Switch joy-con thumbstick with more precision.
Last year, in kindergarden, he would get extremely (disruptively) upset when he even thought it would soon be time to do art, drawing, letters, whatever. Now he starts every morning in class by drawing. There are other things he could be doing, apparently, but he draws, every morning, because the idea of getting better at video games got him over his initial resistance to try something new that he wasn't good at and to improve at it.
I doubt I could ever get him to really practice piano, or to even get him interested in it, but if I put something in front of him that showed him how to play the Super Mario Bros theme I think he'd be pretty excited about it. I think I'll try this afternoon.
Sometimes it's not about getting great; sometimes it's not even about getting good. Sometimes it's just about starting in the first place.
I think the value is in option #1.
My son is 5 years old and just started grade 1; his best friend at school has two parents who are quite musical, and so he's been taking violin lessons. Thanks to COVID, he was stuck in the house, bored, all summer, and decided to spend basically the whole time practicing (mind you, the kid is barely 6) and now he's way beyond the other kids his age and is extremely proud of what he's accomplished.
Conversely, my son hates fine motor skill activities; he's not good at them, so he doesn't want to do them, so he doesn't get good at them. We couldn't convince him to practice drawing, writing his letters, anything; he didn't want anything to do with it. Until, that is, we told him (not entirely falsely) that improving his ability to use a pen, pencil, marker, etc. with precision would also help his ability to use a Switch joy-con thumbstick with more precision.
Last year, in kindergarden, he would get extremely (disruptively) upset when he even thought it would soon be time to do art, drawing, letters, whatever. Now he starts every morning in class by drawing. There are other things he could be doing, apparently, but he draws, every morning, because the idea of getting better at video games got him over his initial resistance to try something new that he wasn't good at and to improve at it.
I doubt I could ever get him to really practice piano, or to even get him interested in it, but if I put something in front of him that showed him how to play the Super Mario Bros theme I think he'd be pretty excited about it. I think I'll try this afternoon.
Sometimes it's not about getting great; sometimes it's not even about getting good. Sometimes it's just about starting in the first place.
> Conversely, my son hates fine motor skill activities; he's not good at them
My motor skills are _fine_, and I had a lot of issues with that throughout schooling (especially cursive, grrr, but drawing/art was also hard), but I played a string instrument (bass, and guitar), and didn't feel the same kind of frustration with that. I mean, chords are hard and kind of frustrating, but I think that's true for everyone. Piano and fretted guitars have the bonus that you can't be off pitch, as long as you hit the right one. Moving a pencil on a page is a lot different, you have to push down enough, but not too much, while making shapes in the right sequence, and you have to fit in the lines, etc, etc. UGH. Hitting the keys with the right strength is important for piano, but at a beginner level, just doing it not too hard is enough.
My motor skills are _fine_, and I had a lot of issues with that throughout schooling (especially cursive, grrr, but drawing/art was also hard), but I played a string instrument (bass, and guitar), and didn't feel the same kind of frustration with that. I mean, chords are hard and kind of frustrating, but I think that's true for everyone. Piano and fretted guitars have the bonus that you can't be off pitch, as long as you hit the right one. Moving a pencil on a page is a lot different, you have to push down enough, but not too much, while making shapes in the right sequence, and you have to fit in the lines, etc, etc. UGH. Hitting the keys with the right strength is important for piano, but at a beginner level, just doing it not too hard is enough.
> There's nothing wrong with the idea of gameifying piano
I wonder whether that's actually true. Or maybe it depends on what level you're trying to achieve.
If you want to become a good piano player, the feedback mechanism that corrects your playing should actually come from listening yourself. A large part of the challenge in becoming a good player is figuring out what to listen for -- this is also why you need a teacher, because you simply do not hear your mistakes yourself in the beginning.
Of course here I'm mostly referring to musicality, and not just whether you're hitting the correct notes. When learning the piano one usually starts with the latter, but I wonder if you make piano more game-like it causes you to underdevelop your own feedback and reward systems, leading to more problems at a later stage.
I wonder whether that's actually true. Or maybe it depends on what level you're trying to achieve.
If you want to become a good piano player, the feedback mechanism that corrects your playing should actually come from listening yourself. A large part of the challenge in becoming a good player is figuring out what to listen for -- this is also why you need a teacher, because you simply do not hear your mistakes yourself in the beginning.
Of course here I'm mostly referring to musicality, and not just whether you're hitting the correct notes. When learning the piano one usually starts with the latter, but I wonder if you make piano more game-like it causes you to underdevelop your own feedback and reward systems, leading to more problems at a later stage.
There was a lot of work done to the the timing as accurate as the chosen environment (the browser) allows and the grading takes into account both what you play as well as when you play it. There are more dimensions to add still: note release point and the volume at which the note is played.
I think the software actually helps you to spot subtle errors quite quickly, but you'd have to try it to see for yourself.
I think the software actually helps you to spot subtle errors quite quickly, but you'd have to try it to see for yourself.
FWIW both release point and volume are available to the software (MIDI includes strength/velocity of keypresses, vibrato, and release events)
Yes, that's what MIDI is all about. on/off/volume those are the basic components of any midi file.
Thinking some more about your comment, would you be willing to criticize the application in detail to give us pointers about where we can improve it? We have some interesting ideas for the fingering problem which I won't share publicly here until we are ready to prove them.
I'm an adult beginning to learn piano using Flowkey. It's criticized by the experts but I feel like it does a relatively good job at teaching basic piano finger techniques. I have no wish to become a piano expert anyway, I play music as a hobby.
I'm learning as an adult. My piano teacher keeps stressing musicality, feeling the music, feeling the beat, everything like that is more important than technique.
They are indeed more important than technique, but technique is a given. Without solid technique, musicality does not matter. Technique is what gives you confidence, it allows you to experiment and fine tune your artistic skills. To work on your expression instead of grappling with what's on the sheet.
Acquiring (and maintaining) technique requires hard work and time, but it's relatively easy once you combine these two factors. Developing a good ear, a good sense of rhythm, when to play on tempo, slightly before the tempo or slighly after, really understanding the music, that is infinitely more difficult so it make sense for a teacher to stress that part.
Acquiring (and maintaining) technique requires hard work and time, but it's relatively easy once you combine these two factors. Developing a good ear, a good sense of rhythm, when to play on tempo, slightly before the tempo or slighly after, really understanding the music, that is infinitely more difficult so it make sense for a teacher to stress that part.
The interesting thing for me is that this software taught me more about chords, beats, timing and so on than any of the teachers that I've had over the years because it is endlessly patient.
I guess. The teacher is the guide, to give direction, not to sit through the exercises you need to do with you.
> The usual result with piano teaching apps of all kinds - as someone else commented below - is kids get up to Grade 1 (maybe 2) and then hit a brick wall.
How many kids or adults who start with traditional piano lessons will actually make it past this level? I'm sure that it is the case that most people will eventually plateau using an app, but I doubt that a control group of people taking lessons would perform much better, and apps at least solve a problem of access.
The vast majority of children and virtually all adults starting piano will never achieve a high level of musicianship, but their lives may well still be enriched by the musical knowledge they gain.
How many kids or adults who start with traditional piano lessons will actually make it past this level? I'm sure that it is the case that most people will eventually plateau using an app, but I doubt that a control group of people taking lessons would perform much better, and apps at least solve a problem of access.
The vast majority of children and virtually all adults starting piano will never achieve a high level of musicianship, but their lives may well still be enriched by the musical knowledge they gain.
As a beginner who wants to improve, I'd like to understanding your finger technique point better.
I saw a video recently [1] which claims to use AI to predict finger technique for an arbitrary MIDI file. This was always an issue for me in using Synthesia [2] - a piece of software that 'gamifies' leaning a piece. Synthesia is capable of displaying fingering information, but only if someone has manually input it - and without the fingering it's almost impossible for a beginner to learn difficult passages.
If MIDI files contained accurate fingering (by which I mean which finger of which hand...although maybe there's more to it?), and then could be input into a game - is this useful education tool?
[1] https://twitter.com/fayezsalka/status/1314613736511016961?s=... [2] https://synthesiagame.com/
I saw a video recently [1] which claims to use AI to predict finger technique for an arbitrary MIDI file. This was always an issue for me in using Synthesia [2] - a piece of software that 'gamifies' leaning a piece. Synthesia is capable of displaying fingering information, but only if someone has manually input it - and without the fingering it's almost impossible for a beginner to learn difficult passages.
If MIDI files contained accurate fingering (by which I mean which finger of which hand...although maybe there's more to it?), and then could be input into a game - is this useful education tool?
[1] https://twitter.com/fayezsalka/status/1314613736511016961?s=... [2] https://synthesiagame.com/
This is an excellent point- as mentioned below, MIDI files do not take into account stuff like key, time signature, etc. , all of which are quite important to understanding a piece ; Western music notation is both incredibly dense and concise, with no information wasted.
Part of the art of practicing is the mental work of abstraction and internalization of the music - breaking it into smaller sections, understanding the key changes, seeing the broader rhythmic structure, etc. - if you don't have this and (more importantly) don't have a teacher to help guide you through this process, you're definitely going to hit a brick wall once things start to get woolly.
Part of the art of practicing is the mental work of abstraction and internalization of the music - breaking it into smaller sections, understanding the key changes, seeing the broader rhythmic structure, etc. - if you don't have this and (more importantly) don't have a teacher to help guide you through this process, you're definitely going to hit a brick wall once things start to get woolly.
> MIDI files do not take into account stuff like key, time signature, etc.
They actually do.
What they don't take into account is whether a particular black key is a sharp or a flat, and this causes all kinds of headaches. https://en.wikipedia.org/wiki/Enharmonic
They actually do.
What they don't take into account is whether a particular black key is a sharp or a flat, and this causes all kinds of headaches. https://en.wikipedia.org/wiki/Enharmonic
They do not take "key" into account. In terms of the note-related data contained within a MIDI file, it's just a series of noteOn and noteOff event pairs, with a note number between 0 and 127 for each event. There's nothing in a MIDI file that specifies the key/scale/mode.
They can describe time signatures and tempo.
And yes, they don't handle enharmonics because in MIDI, notes are known only by number, not by any name given to them by any part of any given musical culture or theory.
And because of the MIDI Tuning Standard, you cannot actually say a priori which notes (i.e frequency) a given note number corresponds to, although in practice most things will take the note number and map it to a a 12TET 440Hz standard.
They can describe time signatures and tempo.
And yes, they don't handle enharmonics because in MIDI, notes are known only by number, not by any name given to them by any part of any given musical culture or theory.
And because of the MIDI Tuning Standard, you cannot actually say a priori which notes (i.e frequency) a given note number corresponds to, although in practice most things will take the note number and map it to a a 12TET 440Hz standard.
Yes, but MIDI does allow for the file to contain a KEY specification (as well as tempo, time signature and a bunch of other info).
I know the scale/key/mode as such as not encoded in the note on/off signals but once you know what key you are in (and this can be determined in a variety of ways, including from the statistical properties of the note on/note off signals) the problem becomes a bit more tractable. Still, you will never get it perfect and that is not the goal, the goal is to get it as good as possible (and we are not even there yet).
> And because of the MIDI Tuning Standard, you cannot actually say a priori which notes (i.e frequency) a given note number corresponds to, although in practice most things will take the note number and map it to a a 12TET 440Hz standard.
This is true, but for a teaching application like this one it is simply not relevant.
I know the scale/key/mode as such as not encoded in the note on/off signals but once you know what key you are in (and this can be determined in a variety of ways, including from the statistical properties of the note on/note off signals) the problem becomes a bit more tractable. Still, you will never get it perfect and that is not the goal, the goal is to get it as good as possible (and we are not even there yet).
> And because of the MIDI Tuning Standard, you cannot actually say a priori which notes (i.e frequency) a given note number corresponds to, although in practice most things will take the note number and map it to a a 12TET 440Hz standard.
This is true, but for a teaching application like this one it is simply not relevant.
The key signature meta-event is barely functional if you have a remotely expansive view of "music theory". Identifying a root note, major/minor and number of sharps OR flats doesn't really pin down a large number of actual scales.
For your purposes, it may be enough, but in more general terms, it's woefully underspecified.
A better way for MIDI to have done this would have been to use a "standard" scale numbering system combined with a tonicity count. An example or two is given here:
https://ianring.com/musictheory/scales/
I appreciate that your current project doesn't need this, but if someone wanted to somehow create a MIDI file that included the information "this uses Locrian with a flat second", the current SMF spec is inadequate.
For your purposes, it may be enough, but in more general terms, it's woefully underspecified.
A better way for MIDI to have done this would have been to use a "standard" scale numbering system combined with a tonicity count. An example or two is given here:
https://ianring.com/musictheory/scales/
I appreciate that your current project doesn't need this, but if someone wanted to somehow create a MIDI file that included the information "this uses Locrian with a flat second", the current SMF spec is inadequate.
Absolutely, midi is clearly a device centric protocol, not a musically correct protocol. But with the various XML music formats there are alternatives that can be converted into midi fairly easily. But right now we only support midi, we will add more formats though.
I stand corrected. Enharmonic stuff has to be a nightmare.
That's putting it mildly, but I'm slowly getting more grip on it, fortunately a lot of people far smarter than me have spent a lot of time on this problem and bit by bit I'm implementing their thoughts into software.
A fun example is this bar from Liebenstraum no 3,
https://imgur.com/a/ud1VlJF
It's the second bar on, the left hand goes A F# B# D# B# F# - Aflat F# C Eflat C F#
Only the first note has actually changed though.
Enharmonics in the same bar for the same pattern. I suppose there's some musicological explanation for this, maybe something along the lines of working towards a key change, but good luck intuiting this in software...
It's the second bar on, the left hand goes A F# B# D# B# F# - Aflat F# C Eflat C F#
Only the first note has actually changed though.
Enharmonics in the same bar for the same pattern. I suppose there's some musicological explanation for this, maybe something along the lines of working towards a key change, but good luck intuiting this in software...
I have no illusion that this will ever be perfect, but there is something that may be 'good enough' for the general purpose and then maybe we can very incrementally improve from there. One way in which to solve this is to use an algorithm for the bulk and a database of exceptions for cases where the algorithm would get it wrong.
This is my first comment on HN. I wrote some algorithms to do with this topic. I've been working on some code for a long time which looks at the c=2^12 combinations of notes, their properties and so on. I call each of them changes. I'm actually still working on material for a book which contains dynamically generated layouts based on these changes. So far I have addressed many music set theory properties, as well as some visualisation on different instruments like piano, fretboard (arbitrary).
One of the algorithms looks at finding the most efficient enharmonic spelling for a given change.. It may be helpful. If anything you could look at the Python for that function and derive something suitable.. Feel free to PM..
Let's say that keys = 12, and changes = 2^12.. so you'd have 48152 possible key changes.. for each of these there are basically three enharmonic roots.. The you could multiply that by the potential variation within the scale degrees. If you just brute force that you'd essentially just keep the one with fewest accidentals
Super interested in this, could you please contact me via [email protected]? (your profile is not filled out for me so I don't know your email address, note that HN has the option to add a public email address as well in your profile but for you that is still blank).
How would this method handle the bar I posted?
I think it really comes down to what your purpose is.
If you're happy hitting grade 1 and hitting that brick wall just to have some fun, the apps do have their place, because these are people who wouldn't otherwise shell out for a teacher. It may even help people gauge whether or not they like an instrument before they move forward or change instruments.
I do 100% agree with you though that for anyone who wants to be even remotely serious, the cost of a teacher is a very good investment.
If you're happy hitting grade 1 and hitting that brick wall just to have some fun, the apps do have their place, because these are people who wouldn't otherwise shell out for a teacher. It may even help people gauge whether or not they like an instrument before they move forward or change instruments.
I do 100% agree with you though that for anyone who wants to be even remotely serious, the cost of a teacher is a very good investment.
Agreed. It’s fine if it’s just a game to get someone interested in the idea of playing the piano but you need to switch to proper instruction soon especially if the person starts playing regularly, as incorrect posture and hand position can cause injury with enough practice. Even if it doesn’t come to that, undoing learned bad muscle memory takes a lot of time and effort.
Fingering hints are work in progress, it is one of the most important things we will add. So completely agreed that it is important. That said, I've taught myself some pretty complicated pieces with it and fingering tends to be something that works out over time because there really is only one comfortable way of playing a piece. 90% or so will be obvious, and the 10% that remains will require some hard thinking on how to move your hands and place your fingers so that it feels natural. Bach is pretty brutal: try a fingering that isn't optimal and you simply won't be able to play it (or even injure yourself).
> There's nothing wrong with the idea of gameifying piano, but it's a really hard problem to solve well.
Agreed. So, my previous 'side project' I babysat for about two decades, I'm in this for the long haul and fully intend for it to be 'best of breed' and standard setting. Which is one of the reasons why we are recruiting a piano teacher right now.
> The usual result with piano teaching apps of all kinds - as someone else commented below - is kids get up to Grade 1 (maybe 2) and then hit a brick wall.
But that's already better than nothing at all, and if we can find a way around that brick wall then I see no reason why it could not go much further than that.
I don't think a piece of software can be a substitute for a teacher, but I do know that if I'm practicing I'd rather do that by myself than to have a teacher look over my shoulder for hours on end.
> The reality is that playing any instrument with non-trivial competence is really fucking hard. It takes everyone a ridiculous amount of time and effort to get good at it - even those with talent.
That we fully agree on. But that doesn't mean people can't get from zero to fifty before deciding that they need better tools and better teachers. There is room for a multitude of tracks some of which will lead to success. The main objectives for me are: to help beginners achieve a level of immediate success that it increases their interest, to ensure that they can get to some level of proficiency before shelling out a lot of money, to overcome the usual frustration that causes people to abandon their chosen instrument, to ensure that they don't learn bad habits.
> So there's a trade off between trying to offer easy but limited success with some entertainment value, and going the whole way and doing it properly - which all but the most musical kids get very bored by, because it's such hard work.
This is also very true. But it may be possible to chart a path that includes both of those in one offering, I'm not sure that we are on that path right now but it definitely is the intention.
Think of this as the beginnings of an engine, which can be used in many different ways. I now first want to get that engine perfected and then to create the rest of the application around it.
So far, especially the 'auto' mode which creates small lessons based on your performance so far has been a game changer for me and for the rest of the people that have been playing around with it, it is conceptually simple, a bit tricky to implement but now that it is there I wonder how I did without it before. The ROI of practice time invested in a piece versus my ability to play it has literally jumped up because of that.
> There's nothing wrong with the idea of gameifying piano, but it's a really hard problem to solve well.
Agreed. So, my previous 'side project' I babysat for about two decades, I'm in this for the long haul and fully intend for it to be 'best of breed' and standard setting. Which is one of the reasons why we are recruiting a piano teacher right now.
> The usual result with piano teaching apps of all kinds - as someone else commented below - is kids get up to Grade 1 (maybe 2) and then hit a brick wall.
But that's already better than nothing at all, and if we can find a way around that brick wall then I see no reason why it could not go much further than that.
I don't think a piece of software can be a substitute for a teacher, but I do know that if I'm practicing I'd rather do that by myself than to have a teacher look over my shoulder for hours on end.
> The reality is that playing any instrument with non-trivial competence is really fucking hard. It takes everyone a ridiculous amount of time and effort to get good at it - even those with talent.
That we fully agree on. But that doesn't mean people can't get from zero to fifty before deciding that they need better tools and better teachers. There is room for a multitude of tracks some of which will lead to success. The main objectives for me are: to help beginners achieve a level of immediate success that it increases their interest, to ensure that they can get to some level of proficiency before shelling out a lot of money, to overcome the usual frustration that causes people to abandon their chosen instrument, to ensure that they don't learn bad habits.
> So there's a trade off between trying to offer easy but limited success with some entertainment value, and going the whole way and doing it properly - which all but the most musical kids get very bored by, because it's such hard work.
This is also very true. But it may be possible to chart a path that includes both of those in one offering, I'm not sure that we are on that path right now but it definitely is the intention.
Think of this as the beginnings of an engine, which can be used in many different ways. I now first want to get that engine perfected and then to create the rest of the application around it.
So far, especially the 'auto' mode which creates small lessons based on your performance so far has been a game changer for me and for the rest of the people that have been playing around with it, it is conceptually simple, a bit tricky to implement but now that it is there I wonder how I did without it before. The ROI of practice time invested in a piece versus my ability to play it has literally jumped up because of that.
> It's hard to include correct fingering in the model without some kind of camera and image recognition system
That seems like a splendid idea. High-res cameras are cheap, and computer vision is all the rage lately.
That seems like a splendid idea. High-res cameras are cheap, and computer vision is all the rage lately.
but this might teach to have some kind of fun
too often lessons are sadistic chores
ps: I also beg to differ about finger pattern, I've seen some seasoned piano players doing just whatever worked, the kinetics of playing allow for more than one pattern most of the time
too often lessons are sadistic chores
ps: I also beg to differ about finger pattern, I've seen some seasoned piano players doing just whatever worked, the kinetics of playing allow for more than one pattern most of the time
Check out https://www.youtube.com/watch?v=6PyYcnXQZJY&list=RDzWmz5-NoF... for some very interesting fingering techniques.
Are you gatekeeping music?
Hey, that is the usual answer to "it's really hard to learn how to do something" here: music, programming, occasionally medicine.
It's true, getting very good is a great deal of work. But, you don't have to be a virtuoso to get some benefits. An app won't teach you to play well enough to give a concert at Carnegie Hall. But, statistically speaking, you aren't going to play Carnegie Hall at all.
It's true, getting very good is a great deal of work. But, you don't have to be a virtuoso to get some benefits. An app won't teach you to play well enough to give a concert at Carnegie Hall. But, statistically speaking, you aren't going to play Carnegie Hall at all.
“ Unfortunately, because Mozilla is too busy improving the world instead of their browser...”
Burn, but understand the sentiment :)
Burn, but understand the sentiment :)
Hahaha, wow! This is awesome to see people working on different solutions to something I'm working on!
@jacquesm if you're looking for someone to chat about this kind of stuff with - I would love to sync up!
--Shameless Plug--
I've been working on a passion project for over 2 years now called Piano Gym! https://pianogym.com
The idea is very much the same - in that a USB midi piano (in chrome for now) can be connected to the computer and used to play sheet music in contenxt.
HOWEVER, Piano Gym's goal is much larger than Piano, and much larger than sheet music, despite that being the current state of the product.
Piano Gym's focus is to apply spaced repetition and flash card based review systems to get people playing Piano and learning music theory in context, while ALSO managing review systems and check points. Eventually we want to do this for all instruments too!
We want you in the gym to do your reps!
Additionally anyone can create content there. Whether you're a learner or a teacher you can make your own schools, courses, and lessons that others can enroll in to learn from!
It's incredibly exciting and super cool to see other people trying to solve these problems.
If anyone is interested please come and join us at the gym - Piano Gym!
https://pianogym.com
@jacquesm if you're looking for someone to chat about this kind of stuff with - I would love to sync up!
--Shameless Plug--
I've been working on a passion project for over 2 years now called Piano Gym! https://pianogym.com
The idea is very much the same - in that a USB midi piano (in chrome for now) can be connected to the computer and used to play sheet music in contenxt.
HOWEVER, Piano Gym's goal is much larger than Piano, and much larger than sheet music, despite that being the current state of the product.
Piano Gym's focus is to apply spaced repetition and flash card based review systems to get people playing Piano and learning music theory in context, while ALSO managing review systems and check points. Eventually we want to do this for all instruments too!
We want you in the gym to do your reps!
Additionally anyone can create content there. Whether you're a learner or a teacher you can make your own schools, courses, and lessons that others can enroll in to learn from!
It's incredibly exciting and super cool to see other people trying to solve these problems.
If anyone is interested please come and join us at the gym - Piano Gym!
https://pianogym.com
wow... Way to try to steal the show. I'm now 100% certain I'll never try piano gym.
Right on! We'll be waiting if you change your mind!
Nice free resource and thank you for posting. So many great discoveries in this thread.
https://melodics.com/
https://github.com/apankrat/note-detector
https://www.pianostreet.com/smf/index.php?topic=12590.0
https://m.youtube.com/watch?feature=youtu.be&v=k50uk8elR0M
http://pianobooster.sourceforge.net/
https://musescore.com/hub/piano
https://github.com/apankrat/note-detector
https://synthesiagame.com/
https://pianojacq.com/
https://melodics.com/
https://github.com/apankrat/note-detector
https://www.pianostreet.com/smf/index.php?topic=12590.0
https://m.youtube.com/watch?feature=youtu.be&v=k50uk8elR0M
http://pianobooster.sourceforge.net/
https://musescore.com/hub/piano
https://github.com/apankrat/note-detector
https://synthesiagame.com/
https://pianojacq.com/
On the off-chance that this will inspire the article's author for their development, or simply for the curious, this has been my experience with it:
The primary mode of display is "Guitar Hero" style (it's hard to describe just look at this video to get an idea[2]) but it's possible to display the sheet music. It lets you practice and grades you depending on how close you're sticking to the score. It's very nice to be able to practice one hand separately while having the app play the other hand. I do fairly well reading sheet music but the first time deciphering it can be a bit tedious, using the app speeds this up considerably for me.
Synthesia is almost great but it has a lot of small shortcomings which add up to a bad user experience, for example:
- The conversion from MIDI to a readable score is very poor compared to, say, Musescore. The sharps and flats are generally off (e.g. randomly a double flat D will appear instead of a C, even though the score has a C, Musescore interprets the midi as a C, etc).
- Not enough wiggle room when playing, e.g. if a chord is not arpeggiated in the midi file, you won't be able to move forward by arpeggiating.
- The app really depends on you following the score exactly and linearly. It should also let the score follow you. For example, if you're playing at a given pace, miss, stop and repeat a bar, the app should detect that automatically instead of just carrying on expecting you to keep playing.
[1] https://synthesiagame.com/
[2] https://www.youtube.com/watch?v=sleZ-hzrtRY