HackerTrans
TopNewTrendsCommentsPastAskShowJobs

bkitano19

260 karmajoined hace 8 años

Submissions

Don't Try to Be Original

briankitano.com
3 points·by bkitano19·hace 8 días·1 comments

Barrington's Theorem for Mortals

briankitano.com
2 points·by bkitano19·hace 3 meses·0 comments

Mad: Watch agents do research live

briankitano.com
2 points·by bkitano19·hace 5 meses·0 comments

On Culmination and Not yet Turning Thirty

briankitano.com
1 points·by bkitano19·hace 5 meses·0 comments

Making Sense of Memory and Attention

memory.briankitano.com
8 points·by bkitano19·hace 6 meses·0 comments

Free hosted Postgres, as simple as mkdir

mkdb.sh
1 points·by bkitano19·hace 2 años·0 comments

Silon.ai – real-time global public figure recognition

silon.ai
13 points·by bkitano19·hace 2 años·0 comments

Browserslist/browserslist: `not and_UC all`

github.com
1 points·by bkitano19·hace 2 años·1 comments

What else can we do with the Web Audio API?

media-codings.com
1 points·by bkitano19·hace 2 años·0 comments

Llama from scratch, or how to implement a paper without crying

blog.briankitano.com
513 points·by bkitano19·hace 3 años·52 comments

comments

bkitano19
·el mes pasado·discuss
Warning: the photos are nightmare fuel and not safe for bedtime
bkitano19
·hace 4 meses·discuss
notable omission of deepgram models in comparisons?
bkitano19
·hace 4 meses·discuss
+1 to running. If you run consistently, you'll learn to believe in your body as something that naturally improves if you train it well, and that belief will cross over to your mind and heart.
bkitano19
·hace 6 meses·discuss
You can use voice prompting; it's supported on ElevenLabs and Hume.
bkitano19
·hace 9 meses·discuss
Awesome post!
bkitano19
·hace 11 meses·discuss
you might like https://en.wikipedia.org/wiki/Noether%27s_theorem
bkitano19
·el año pasado·discuss
https://huggingface.co/spaces/nvidia/parakeet-tdt-0.6b-v2
bkitano19
·el año pasado·discuss
Related work:

Interpreting Modular Addition in MLPs https://www.lesswrong.com/posts/cbDEjnRheYn38Dpc5/interpreti...

Paper Replication Walkthrough: Reverse-Engineering Modular Addition https://www.neelnanda.io/mechanistic-interpretability/modula...
bkitano19
·hace 2 años·discuss
hume.ai specializes in expressive prosody for TTS (disclaimer - I work here)
bkitano19
·hace 2 años·discuss
Time to first token is as important to know for many use cases, rarely are people reporting it
bkitano19
·hace 2 años·discuss
this is nuts
bkitano19
·hace 2 años·discuss
+1, had the fortune to work with him at a previous startup and meetup in person. Our convo very much broadened my perspective on engineering as a career and a craft, always excited to see what he's working on. Good luck Simon!
bkitano19
·hace 2 años·discuss
https://transformer-circuits.pub/2022/in-context-learning-an...

there is a lot of evidence to suggest that they are performing induction
bkitano19
·hace 2 años·discuss
Had this exact problem (Heroku Postgres to RDS) at my old co. Data migration went as bad as it possibly could (dropped indices, foreign keys, everything but the data itself). This would have saved us months of pain.
bkitano19
·hace 2 años·discuss
https://hootdoogs.com/ - "find food between you guys"
bkitano19
·hace 2 años·discuss
High integrity maintainer, big respect.
bkitano19
·hace 3 años·discuss
Karpathy covers this in Makemore, but the tl;dr is that if you don’t normalize the batch (essentially center and scale your activations down to be normally distributed), then at gradient/backprop time, you may get values that are significantly smaller or greater than 1. This is a problem, because as you stack layers in sequence (passing outputs to inputs), the gradient compounds (because of the Chain Rule), and so what may have been a well behaved gradient at the end layers has either vanished (the upstream gradients were 0<x<1 at each layer) or exploded (the gradients were x>>1 upstream). Batch normalization helps control the vanishing/exploding gradient problem in deep neural nets by normalizing the values passed between layers.
bkitano19
·hace 3 años·discuss
Wow, great catch. I will update this in the morning!
bkitano19
·hace 3 años·discuss
edit: bearblog getting ddos'd, here's the repo https://github.com/bkitano/llama-from-scratch
bkitano19
·hace 3 años·discuss
working on it hehe