I would say that the "more secure way" is to just use ComfyUI without installing any obscure nodes from unknown developers. You can do pretty much anything using just the default nodes and the big node packs.
The person who created the custom node is the same person who "hacked" it. Whether or not the account is technically owned by some unrelated civilian is not important, because there is no other activity on the account.
Okay, sure. But if we have an account which has never had any legitimate activity on it ever - an account that has only ever been used to push malware - then I don't know if it matters much who is the "rightful owner" of the account. Things would be different if the GitHub account had some legitimate activity before the "hack".
I have not seen a statement from Nullbulge so it's not appropriate to say that they took over the repo.
The author of the repo is claiming that their repo is hacked, but this is an obvious lie, because their very first GitHub commit is the one where they push the malware. Nobody would hack an empty GitHub account.
I don't know if the author of the repo is lying when they say that Nullbulge is behind the attack (perhaps the author is part of Nullbulge, perhaps not).
It was an extension for ComfyUI, which has 37k stars on GitHub. The way ComfyUI is commonly used is that a person shares a "workflow" file, which utilizes various obscure extensions (called "custom nodes") and then the people who want to run the workflow on their own computer will install all these obscure custom nodes that have like 40 stars on GitHub or so.
This is not true at all. I'm active in multiple NSFW AI Discords and Subreddits, and looking at the type of material people engage with, almost all of it is very clearly targeted at heterosexual men. I'm not even aware of any online communities that would have NSFW AI stuff targeting mainly female audience.
My library solves the following problem: how to tokenize text in a way that is compatible with llama3.
If you don't have any particular constraint (as in "tokenize text in a way that is compatible to model X"), then you can just write your own tokenization that tokenizes the text however you want. It doesn't really make sense to use a complicated tokenization scheme from some LLM model if you don't need to be compatible with that model.
If you really want each word to be its own token, you can easily do that by just splitting on whitespace and punctuation (though that will lead to a huge vocabulary).
> "What " might be a different token than "What" but the total token count shouldn't increment, would just be a different token, right?
The input string "What" (without trailing space) tokenizes into 1 token. The input string "What " tokenizes into 2 tokens. In theory, one might have a tokenizer that would simply tokenize "What " into a single token, but the actual tokenizers we have will tokenize that into at least 2 tokens.
> Curious then why this is called "LLaMA 3 tokenizer" what does it have to do with llama3?
When you input text into any of the LLaMA 3 models, the first step in the process is tokenizing your input. This library is called "LLaMA 3 tokenizer", because it produces the same tokenization as the official LLaMA 3 repo.
When I said that different models use different tokenization schemes, I am talking in comparison to other models, such as LLaMA 1, or GPT-4. Different models use different tokenizers, so the same text is tokenized into different tokens depending on if you're using GPT-4 or LLaMA 3 or what not.
> I'm not sure it's working correctly, I entered the word "what" and it says "4 characters, 3 tokens", I type a space and it says "4 tokens" - shouldn't it just be 1 token? and the space shouldn't count in this case?
When you enter the word "what", the 3 tokens were: start-of-string token, the token "what", and end-of-string token. I made a change now to hide the special start-of-string and end-of-string tokens so that the visualization is a bit simplified.
Adding a space to input changes the tokenization of the input. Sometimes the resulting token count is the same (if the space is merged into some other text), sometimes the resulting token count increases by one (if the space does not get merged).
That part of the tokenizer is working correctly.
> Also occasionally a space appears as a capital G (in Chrome)
Fixed, thanks for reporting! This is a fork of my earlier tokenizer for LLaMA 1 and the demo visualizer had special handling for tokens 0-256 in LLaMA 1. This LLaMA 3 tokenizer doesn't have same special tokens, so some tokens would be visualized in a weird way (like that G thing you reported). I removed that special handling now and it fixed the visualization issue.
> Question: Is there a special ruleset that llama3 follows that other LMs don't as far as what qualifies as a token?
Different models use different tokenization schemes. Most models use some kind of variant of Byte Pair Encoding, trained with their data (the tokenizer itself is also trained, not only the language model).
Can you show me one (1!) artist who makes a living wage off of crypto payments without any kind of ponzi-like speculative aspect? An artist who is just using crypto as means of payment AND actually getting paid with it. Just one example is enough.
> If an artist can’t manage to set up a crypto account—or a bank account, which requires the exact same documents—they are probably not in a position to run their own art business.
The artist, sure. But once the artist goes through all that work to set it up, will they get access to food which they need in order to continue living? No, they won't. Because the consumers will prefer products that are easy to pay for.
> No—I am talking broadly about artists selling their work through crypto, and some of it does include $5 comics, like Sloth Zine:
aaand the link you gave is an NFT collectible...
The price point was not the main point here - sure an NFT doesn't have to be expensive it can also be cheap. That was not the point. My point was that the audience for ponzi-like NFT games is different from the audience for more "traditional" consumers of art. If you are, for example, an artist who currently makes comic books that people will pay for with Visa and Mastercard, you will not be able to provide food on the table by switching into crypto payments.