HackerTrans
TopNewTrendsCommentsPastAskShowJobs

dmsnell

no profile record

Submissions

How to safely escape JSON inside HTML SCRIPT elements

sirre.al
86 points·by dmsnell·11 bulan yang lalu·46 comments

comments

dmsnell
·10 bulan yang lalu·discuss
Vapor chambers and heat pipes are exceptional at moving heat from a relatively small part of them to distribute it around the whole.

This is because the heat dumps into a liquid which is concentrated at the heat source, but as soon as it evaporates it fills the volume in which it’s contained. Also it’s because the evaporation process itself sucks out heat at a rate that’s orders of magnitude faster than via conduction, convection, or radiation alone.

They do not cool though. They rely on the fact that the heat source is relatively small and that something else can pull heat out of them fast enough to re-condense the liquid (and since they distribute the heat that cooling can attach anywhere or everywhere — this is like an embarrassingly parallel problem in software).

In a battery pack there is a lot of surface area that gets relatively and evenly hot, and little room to extract it between the cells. This would likely result in even heating around the heat pipe, which would tend to evaporate all of the liquid inside and do nothing but raise its overall temperature after an initial delay.

What it potentially could be used for is to draw heat out of the battery pack and up to some place where better airflow were possible, or for some active cooling system to extract the heat, but there are problems with scaling up like that (in typical heat pipes they manufacture wicking inner layers to draw the water back even against gravity).

At the points they could be used it’s likely significantly cheaper and easier to control some air or liquid cooling loop through the batteries.

Phones are ideal because a tiny little chip is producing almost all of the heat. It’s not even a lot, it’s just in a small area. Temperature goes up when heat can’t escape, so in this case, spreading the heat around even a few square inches can be a major factor in keeping down the temperature of the CPU/GPU.

Contrast this to EV batteries which are huge and produce evenly distributed heat already; there just isn’t the same value when things are big enough to add cooling systems, when cooling systems are necessary anyway, and when the heat pipe or vapor chamber just adds another piece too the system.
dmsnell
·11 bulan yang lalu·discuss
At some point I think I read a more complete justification, but I can’t find it now. There is evidence that it came about as a byproduct of the interaction of the HTML parser and JS parsers in early browsers.

In this link we can see the expectation that the HTML comment surrounds a call to document.write() which inserts a new SCRIPT element. The tags are balanced.

https://stackoverflow.com/questions/236073/why-split-the-scr...

In this HTML 4.01 spec, it’s noted to use HTML comments to hide the script contents from render, which is where we start to get the notion of using these to hide markup from display.

https://www.w3.org/TR/html401/interact/scripts.html

Some drafts of the HTML standard attempted to escape differently and didn’t have the double escape state.

https://www.w3.org/TR/2016/WD-html52-20161206/semantics-scri...

My guess is that at some point the parsers looked for balanced tags, as evidenced in the note in the last link above, but then practical issues with improperly-generated scripts led to the idea that a single SCRIPT closing tag ends the escaping. Maybe people were attempting to concatenate script contents wrong and getting stacks of opening tags that were never closed. I don’t know, but I suppose it’s recorded somewhere.

Many things in today’s HTML arose because of widespread issues with how people generated the content. The same is true of XML and XHTML by the way. Early XML mailing lists were full of people parsing XML with naive PERL regular expressions and suggesting that when someone wants to “fix” broken markup, that they do it with string-based find-and-replace.

The main difference is that the HTML spec went in the direction of saying, _if we can agree how to handle these errors then in the face of some errors we can display some content_ and we can all do it in the same way. XML is worse in some regards: certain kinds of errors are still ambiguous and up to the parser to determine how to handle, whether they are non-recoverable or recoverable. For those non-recoverable, the presence of a single error destroys the entire document, like being refused a withdrawal at the bank because you didn’t cross a 7.

At least with HTML5, it’s agreed upon what to do when errors are present and all parsers can produce the same output document; XML parsers routinely handle malformed content and do so in different ways (though most at least provide or default to a strict mode). It’s better than the early web, but not that much better.
dmsnell
·11 bulan yang lalu·discuss
When a process crashes, its supervisor restarts it according to some policy. These specify whether to restart the sibling process in their startup order or to only restart the crashed process.

But a supervisor also sets limits, like “10 restarts in a timespan of 1 second.” Once the limits are reached, the supervisor crashes. Supervisors have supervisors.

In this scenario the fault cascades upward through the system, triggering more broad restarts and state-reinitializations until the top-level supervisor crashes and takes the entire system down with it.

An example might bee losing a connection to the database. It’s not an expected fault to fail while querying it, so you let it crash. That kills the web request, but then the web server ends up crashing too because too many requests failed, then a task runner fails for similar reasons. The logger is still reporting all this because it’s a separate process tree, and the top-level app supervisor ends up restarting the entire thing. It shuts everything off, tries to restart the database connection, and if that works everything will continue, but if not, the system crashes completely.

Expected faults are not part of “let it crash.” E.g. if a user supplies a bad file path or network resource. The distinction is subjective and based around the expectations of the given app. Failure to read some asset included in the distribution is both unlikely and unrecoverable, so “let it crash” allows the code to be simpler in the happy path without giving up fault handling or burying errors deeper into the app or data.
dmsnell
·11 bulan yang lalu·discuss
The other comment explains this, but I think it can also be viewed differently.

It’s helpful to recognize that the inner script tags are not actual script tags. Yes, once entering a script element, the browser switches parsers and wants to skip everything until a closing script tag appears. The STYLE element, TITLE, TEXTAREA, and a few others do this. Once they chop up the HTML like this they send the contents to the separate inner parser (in this case, the JS engine). SCRIPT is unique due to the legacy behavior^1.

HTML5 specifies these “inner” tags as transitions into escape modes. The entire goal is to allow JavaScript to contain the string “</script>” without it leaking to the outer parser. The early pattern of hiding inside an HTML comment is what determined the escaping mechanism rather than making some special syntax (which today does exist as noted in the post).

The opening script tag inside the comment is actually what triggers the escaping mode, and so it’s less an HTML tag and more some kind of pseudo JS syntax. The inner closing tag is therefore the escaped string value and simultaneously closes the escaped mode.

Consider the use of double quotes inside a string. We have to close the outer quote, but if the inner quote is escaped like `\”` then we don’t have to close it — it’s merely data and not syntax.

There is only one level of nesting, and eight opening tags would still be “closed” by the single closing tag.

^1: (edit) This is one reason HTML and XML (XHTML) are incompatible. The content of SCRIPT and STYLE elements are essentially just bytes. In XML they must be well-formed markup. XML parsers cannot parse HTML.
dmsnell
·11 bulan yang lalu·discuss
This was my first submission, and the above comment was what I added to the text box. It wasn’t clear to me what the purpose was, but it seemed like it would want an excerpt. I only discovered after submitting that it created this comment.

I guess people just generally don’t add those?

Still, to help me out, could someone clarify why this was down-voted? I don’t want to mess up again if I did, but I don't understand what that was.
dmsnell
·11 bulan yang lalu·discuss
Discussing why parsing HTML SCRIPT elements is so complicated, the history of why it became the way it is, and how to safely and securely embed JSON content inside of a SCRIPT element today.
dmsnell
·12 bulan yang lalu·discuss
You understand it correctly: it doesn’t remove any value from fixup; instead it just means you can fixup to whatever and move the commit visually to where you want it — doesn’t have to be the main/trunk branch, and you can do it whenever is convenient.

That’s where I made the comment about not actually running fixup. Instead of claiming to fix a SHA, I leave myself a note like “fix tests” so I can move it appropriately even if I get distracted for a few days
dmsnell
·12 bulan yang lalu·discuss
You can interactively rebase after the fact while preserving merges even. I usually work these days on stacked branches, and instead of using —-fixup I just commit with a dumb message.

    git rebase -i --rebase-merges --keep-base trunk
This lets me reorganize commits, edit commit messages, split work into new branches, etc… When I add --update-refs into the mix it lets me do what I read are the biggest workflow improvements from jj, except it’s just git.

This article from Andrew Lock on stacked branches in git was a great inspiration for me, and where I learned about --update-refs:

https://andrewlock.net/working-with-stacked-branches-in-git-...
dmsnell
·12 bulan yang lalu·discuss
No but I think I saw that it’s a thermal pad, one with comparable thermal transfer characteristics to the high end (non-conductive) pastes.
dmsnell
·12 bulan yang lalu·discuss
TECs are wonderful little devices with operating characteristics unlike comparable devices.

They can be designed to move a specific amount of heat or to cool at some delta-T below the hot side (and due to inefficiencies the hot side can climb above ambient temperatures too, raising the “cold side” above ambient!)

I ran through a design exercise with a high quality TEC and at 8°C delta-T for a wine cooler you could expect a COP of around 3.5–4 (theoretically). This is pretty good! But below the 2.5V max to do that you’re only able to exhaust up to around 40W. For a wine cooler this is not so bad. For a refrigerator it’s a harder challenge because the temperature drops when the door opens, and if someone sticks in a pot of hot soup, it’s important to eject that heat before it raises the temperature inside to levels where food safety becomes a problem. For a CPU it’s basically untenable under load because it’s too much heat entering the cold side thus temperatures will rise.

https://fluffyandflakey.blog/2019/08/29/cooling-a-cpu-with-t...

Things often overlooked:

- Most TECs are cheap and small and come without data sheets, so people tend to become disillusioned after running them too hot.

- You have to keep the hot side cool or else the delta-T doesn’t help you. For a wine cooler this is probably no big deal: you can add a sizable fan and heat sink. For CPU cooling it becomes a tighter problem. You basically can’t win by mounting on the CPU; they are best at mediating two independent water-cooling loops.

- Q ratings are useless without performance graphs. It’s meaningless to talk about a “100W” TEC other than to estimate that it has a higher capacity than a “20W” TEC.

- Ratings and data sheets are hypothetical best cases. Reality constrains the efficiency through a thousand cuts.

When I think about TECs I think more about heat transfer than temperature drops. If you open a well-insulated wine cooler once a week then once it cools it will only need to maintain its temperature, and that requires very little heat movement. Since nothing inside is generating heat you basically have zero watts as a first-order approximation. For the same device mentioned above, it stops working below 1V, and at 8° delta-T that’s a drop in COP to around zero but it’s also nearly zero waste. If you were to maintain a constant 2.5V, however, it would continue to try and pull 40W to the hot side. This would cause the internal temperature to drop and your COP would decrease even though the TEC is using constant power. The delta-T would in fact increase until the inefficiencies match the heat transfer and everything stabilizes. In this case that’s around a 20° drop from the hot side, assuming perfect insulation.

Unlike compressors, TECs have this convenient ability to scale up and down and maintain consistent temperatures; they just can’t respond quickly and dump a ton of heat in the same way.

edit: formatting of list
dmsnell
·tahun lalu·discuss
Several years ago I replaced the thermal paste in my MacBook Pro and I did it in two steps: first to high-end paste; and second to liquid metal.

The results were impressive, and I think it’s a bit veiled how paste degradation over time impacts perceived laptop speeds. I’ve been tempted to replace the paste on new devices but haven’t taken that plunge.

https://fluffyandflakey.blog/2019/04/13/increasing-thermal-h...
dmsnell
·tahun lalu·discuss
Yes, that’s correct. All of these measures, of course, stand as a courtesy and are trivial to bypass, as ema notes.

Finding cryptographic-strength measures to identify LLM-generated content is a few orders of magnitude harder than optimistically marking them. Besides, it also relies on the content producer adding those indicators so that can’t be ignored as a major source of missing metadata.

But sometimes lossy mechanisms are still helpful because people who aren’t out with malicious purposes might copy and paste without being aware that the content is generated, while an auditor (be it anyone who inspects one level deeper) can discover in some (most?) cases the source of the content.
dmsnell
·tahun lalu·discuss
Unicode has a range of Tag Characters, created for marking regions of text as coming from another language. These were deprecated for this purpose in favor of higher level marking (such as HTML tags), but the characters still exist.

They are special because they are invisible and sequences of them behave as a single character for cursor movement.

They mirror ASCII so you can encode arbitrary JSON or other data inside them. Quite suitable for marking LLM-generated spans, as long as you don’t mind annoying people with hidden data or deprecated usage.

https://en.m.wikipedia.org/wiki/Tags_(Unicode_block)
dmsnell
·tahun lalu·discuss
HyTime was Kimber’s work, and I found this reflections of his on “worse is better” to be quite refreshing, especially when contrasting the formality of SGML against the flexibility of HTML.

https://drmacros-xml-rants.blogspot.com/2010/08/worse-is-bet...

I’ll have to spend more time in the SGML Handbook. It’s available for borrowing at https://archive.org/details/sgmlhandbook0000gold. So far, I’ve been focused on the spec and trying to understand the intent behind the rules.

> a couple of auxiliary conventions introduced to make attribute capture and mapping in link rules more useful

Do you have any pointers on how to find these in the spec or in the handbook? Some keywords? What I’ve gathered is that notations point to some external non-SGML processor to handle the rendering and interpretation of content, such as images and image-viewing programs.

Cheers!
dmsnell
·tahun lalu·discuss
HTML didn’t make sense to me until I realized it’s built on a state machine and its rules are based on what’s on the stack of open elements. For example, a number of tags trigger a rule to close open P elements or list items, and many end tags trigger a rule saying something like “close open elements until you’ve closed one with the same name as this tag.”

This, IMO, is a bigger reason to avoid regex and XML parsers for HTML documents. The rules aren’t apparent when thinking linearly about what strings appear after or before each other; they become clearer when thinking of HTML as a shorthand syntax for certain kinds of push and pop operations.

XHTML is easier to parse, but for well-formed documents pushes the complexity of invalid markup into the rendering side. For example, it’s well-formed to include a button inside a button, so XHTML browsers render exactly this, but it makes no sense from a UI perspective and strange things happen when invalid markup is sent in well-formed XML.
dmsnell
·tahun lalu·discuss
The story of XHTML is instructive to the field of software design. There are plenty of good resources on the web if you search why did XHTML fail?

HTML parsing at least is deterministic and fully specified, whereas XHTML, as an XML, leaves a number of syntax errors up to the parser and undefined.

  Conforming software may detect and report an error and may recover from it.
While fatal errors should cause all parser to reject a document outright, this also leaves the end-user without any recovery of the information they care about. So XHTML leaves readers at a loss while failing to eliminating parsing ambiguity and undefined behavior.

Interestingly, it’s possible to encode an invalid DOM with XHTML while it’s impossible to do so in HTML. That means that XML/XHTML has given up the possibility of invalid syntax (by acting like it doesn’t exist) for the sake of inviting invalid semantics.
dmsnell
·tahun lalu·discuss
The memory test is missing some deprecated and non-conforming elements. The HTML spec doesn’t have a single comprehensive list either, so it can be a little tricky to define or name “all” of the elements.

For example, there are elements like nextid or isindex which don’t have element definitions but which appear in the parsing rules for legacy compatibility. These are necessary to avoid certain security issues, but the elements should not be used and in a sense don’t exist even though they are practically cemented into HTML forever.
dmsnell
·tahun lalu·discuss
Took me a while to process your response; thanks for providing to so much to think about. And yes, it is indeed nice meeting other SGML enthusiasts!

Can’t say I’m familiar with the use of notations the way you are describing, and it doesn’t help that ISO-8879-1986 is devoid of any reference to the italicized or quoted terms you’re using. Mind sharing where in the spec you see these data attributes?

The SGML spec is hands-down the most complicated spec I’ve ever worked with, feeling like it’s akin to how older math books were written before good symbolic notation was developed — in descriptive paragraphs. The entire lack of any examples in the “Link type definition” section is one way it’s been hard to understand the pieces.

> some HyTime concepts

I’m familiar with sgmljs but have you incorporated HyTime into it? That is, is sgmljs more like SGML+other things? I thought it already was in the way it handles UTF-8.

> Why do you think so and why should this be required by tag inference specifically?

Perhaps it’d be clearer to add chunkable to my description of streaming. Either the parser gets the document from the start and stores all of the entities for processing each successive chunk, or it will mis-parse. The same is true not only for entities for also for other forms of MINIMIZATION and structure.

Consider encountering an opening tag. Without knowing that structure and where it was previously, it’s not clear if that tag needs to close open elements. So I’m contrasting this to things that are self-synchronizing or which allow processing chunks in isolation. As with many things, this is an evaluation on the relative complexity of streaming SGML — obviously it’s easier than HTML because HTML nodes at the top of a document can appear at the end of the HTML text; and obviously harder than UTF-8 or JSON-LD where it’s possible to find the next boundary without context and start parsing again.

> Regarding namespaces, one of their creators (SGML demi-good James Clark himself) considers those a failure:

Definitely aware of the criticism, but I also find them quite useful and that they solve concrete problems I have and wish SGML supported. Even when using colons things can get out of hand to the point it’s not convenient to write by hand. This is where default namespacing can be such an aid, because I can switch into a different content domain and continue typing with relatively little syntax.

Of note, in the linked reference, James Clark isn’t arguing against the use of namespaces, but more of the contradiction between what’s written in the XML and the URL the namespace points to, also of the over-reliance on namespacing for meaning in a document.

What feels most valuable is being able to distinguish a flight of multiple segments and a flight of different beers, and to do so without typing this:

  <com.dmsnell.things.liquids.beer.flight>
    <com.dmsnell.things.liquids.beer.beer>

  <com.dmsnell.plans.travel.itineraries.flight>
    <com.dmsnell.plans.travel.flights.segment>
Put another way, I’m not concerned about the problem of universal inter-linking of my documents and their semantics across the web, but neither was SGML.

Cheers!
dmsnell
·tahun lalu·discuss
Fun fact: this is very close but slightly inaccurate. I used to think this is how it worked before scrutinizing a rule in the HTML tree-building specification.

The tag leads the parser to interpret everything following it as character data, but doesn’t impact rendering. In these cases, if there are active formatting elements that would normally be reconstructed, they will after the PLAINTEXT tag as well. It’s quite unexpected.

  <a href="https://news.ycombinator.com"><b><i><u><s><plaintext>hi
In this example “hi” will render with every one of the preceding formats applied.

https://software.hixie.ch/utilities/js/live-dom-viewer/?%3Ca...

After I discovered this the note in the spec was updated to make it clearer.

  https://html.spec.whatwg.org/multipage/parsing.html#:~:text=A start tag whose tag name is "plaintext"
dmsnell
·tahun lalu·discuss
I’ve become quite a fan of writing in SGML personally, because much of what you note is spot-on. Some of the points seem a bit of a stretch though.

Any type-checking inside of SGML is more akin to unused-variable checking. When you say that macros/entities may contain parameters, I think you are referring to recursive entity expansion, which does let you parameterize macros (but only once, and not dynamically within the text). For instance, you can set a `&currentYear` entity and refer to that in `copywrite "&currentYear/&currentDay`, but that can only happen in the DTD at the start of the document. It’s not the case that you could, for instance, create an entity to generate a Github repo link and use it like `&repoName = "diff-match-patch"; &githubLink`. This feature was used in limited form to conditionally include sections of markup since SGML contains an `IGNORE` “marked section”.

   <!ENTITY % private-render "IGNORE">
   ...
   <![%private-render[
   <side-note>
   I’m on the fence about including this bit.
   It’s not up to the editorial standards.
   </side-note>
   ]]>

SGML also fights hard against stream processing, even more so than XML (and XML pundits regret not deprecating certain SGML features like entities which obstruct stream processing). Because of things like this, it’s not possible to parse a document without having the entire thing from the start, and because of things like tag omission (which is part of its syntax “MINIMIZATION” features), it’s often not possible to parse a document without having _everything up to the end_.

Would love to hear what you are referring to with “safe” third-party transclusion and also what features are available for removal or rejection of undesired script in user content.

Apart from these I find it a pleasure to use because SGML makes it easy for _humans_ to write structured content (contrast with XML which makes it easy for software to parse). SGML is incredibly hard to parse because in order to accommodate human factors _and actually get people to write structured content_ it leans heavily on computers and software doing the hard work of parsing.

It’s missing some nice features such as namespacing. That is, it’s not possible to have two elements of the same name in the same document with different attributes, content, or meanings. If you want to have a flight record and also a list of beers in a flight, they have to be differentiated otherwise they will fail to parse.

   <flight-list>
   <flight-record><flight-meta pnr=XYZ123 AAL number=123>
   </flight-list>

   <beer-list>
   <beer-flight>
   <beer Pilsner amount=3oz>Ultra Pils 2023
   <beer IPA>Dual IPA
   <beer Porter>Chocolate milk stout
   </beer-list>

DSSSL was supposed to be the transforms into RSS, page views, and other styles or visualizations. With XML arose XSL/XSLT which seemed to gain much more traction than DSSSL ever did. My impression is that declarative transforms are best suited for simpler transforms, particularly those without complicated processing or rearranging of content. Since `osgmls` and the other few SGML parsers are happy to produce an equivalent XML document for the SGML input, it’s easy to transform an SGML document using XSL, and I do this in combination with a `Makefile` to create my own HTML pages (fair warning: HTML _is not XML_ and there are pitfalls in attempting to produce HTML from an XML tool like XSL).

For more complicated work I make quick transformers with WordPress’ HTML API to process the XML output (I know, XML also isn’t HTML, but it parses reliably for me since I don’t produce anything that an HTML parser couldn’t parse). Having an imperative-style processor feels more natural to me, and one written in a programming language that lets me use normal programming conveniences. I think getting the transformer right was never fully realized with the declarative languages, which are similar to Angular and other systems with complicated DSLs inside string attribute values.

I’d love to see the web pick up where SGML left off and get rid of some of the legacy concessions (SGML was written before UTF-8 and its flexibility with input encodings shows it — not in a good way either) as well as adopt some modern enhancements. I wrote about some of this on my personal blog, sorry for the plug.

https://fluffyandflakey.blog/2024/10/11/ugml-a-proposal-to-u...

Edit: formatting