Show HN: TeXMe Demo – Self-Rendering Markdown and LaTeX Documents(opendocs.github.io)
opendocs.github.io
Show HN: TeXMe Demo – Self-Rendering Markdown and LaTeX Documents
https://opendocs.github.io/texme/examples/demo.html
55 comments
Gitlab renders equations in .md files automatically.
There’s been some form of notebook support in VSCode for a while, but nothing that I liked enough to start using vscode over a notebook server. But I’d love to be able to double click a .ipynb file and start editing and exploring.
Can you link the the recent support you talk about so I can compare it to what I’ve already tried in vscode?
Can you link the the recent support you talk about so I can compare it to what I’ve already tried in vscode?
https://code.visualstudio.com/docs/python/jupyter-support
It is exactly as you mentioned, you click the notebook, it renders and opens a Kernel on the background.
The plots have a default dark theme, but can be changed in the same way as if it was inside a normal Jupyter instance.
It's pretty convenient, but a bit buggy still for larger notebooks (I have to constantly save or some actions cause the cells to rollback in time).
It is exactly as you mentioned, you click the notebook, it renders and opens a Kernel on the background.
The plots have a default dark theme, but can be changed in the same way as if it was inside a normal Jupyter instance.
It's pretty convenient, but a bit buggy still for larger notebooks (I have to constantly save or some actions cause the cells to rollback in time).
Yeah, I noticed that down arrow from the end of a markdown cell would leave the cell and lose typed content. Fortunately I use Emacs keybindings (C-f, C-b, C-p, C-n) to navigate for the most part.
Markdeep [1] does something very similar.
(Disclaimer: Not my project, but I've developed several Markdeep-adjacent tools [2][3].)
[1]: https://casual-effects.com/markdeep/
[2]: https://github.com/doersino/markdeep-slides
[3]: https://github.com/doersino/markdeep-diagram-drafting-board
(Disclaimer: Not my project, but I've developed several Markdeep-adjacent tools [2][3].)
[1]: https://casual-effects.com/markdeep/
[2]: https://github.com/doersino/markdeep-slides
[3]: https://github.com/doersino/markdeep-diagram-drafting-board
This looks cool, even cooler than TexMe! Was going to start writing a Project brief today, looks like I'm using Markdeep!
Markdeep is a great project and supports a lot of impressive features. TeXMe on the other hand takes a minimalist approach.
There are some fundamental differences between the two tools. For example, in Markdeep, the user input goes into the <body> element whereas in TeXMe the user input goes into the <textarea> element. The choice of <textarea> element was made in TeXMe to make Markdown rendering more robust. Copy-pasting a valid Markdown code into Markdeep may not always render correctly without further editing. By the way, TeXMe also supports content in <body> as an optional feature.
Further, Markdeep does not care about conforming to the CommonMark spec whereas TeXMe does. If you care about CommonMark spec, TeXMe is a good choice. However, if you need additional features like tables, diagrams, etc., Markdeep is a good choice.
For a detailed comparison of the two tools, please see this thread: https://news.ycombinator.com/item?id=18314175.
There are some fundamental differences between the two tools. For example, in Markdeep, the user input goes into the <body> element whereas in TeXMe the user input goes into the <textarea> element. The choice of <textarea> element was made in TeXMe to make Markdown rendering more robust. Copy-pasting a valid Markdown code into Markdeep may not always render correctly without further editing. By the way, TeXMe also supports content in <body> as an optional feature.
Further, Markdeep does not care about conforming to the CommonMark spec whereas TeXMe does. If you care about CommonMark spec, TeXMe is a good choice. However, if you need additional features like tables, diagrams, etc., Markdeep is a good choice.
For a detailed comparison of the two tools, please see this thread: https://news.ycombinator.com/item?id=18314175.
wow. thanks for that. It's _really_ impressive.
I prefer katex [0] over mathjax these days. Seems to render a lot faster. I wonder how this compares to Pandoc in terms of features. I really would like a complete solution to take .tex files (theorems, references, figures) and turn it into a html page, instead of a pdf. Pandoc is not quite there for me (theorem environments!).
The main issue is that things like LaTeX theorem environments don't translate exactly markdown (plus the theorem numbering/referencing), but there should be some choice of convention. Seems like there was a focus on features that map directly between different formats.
[0] https://katex.org/
The main issue is that things like LaTeX theorem environments don't translate exactly markdown (plus the theorem numbering/referencing), but there should be some choice of convention. Seems like there was a focus on features that map directly between different formats.
[0] https://katex.org/
Have you tried Jupyter notebooks? I do not know if they handle references, but it does the rest you want. The markdown cella support latex math and you can include figures or even videos.
Isn't the TeX support in Jupyter based on MathJax?
Have you tried RMarkdown? It is a lot more focused on producing a pdf or html document than what Jupyter is. It even has decent support for cross-references, and for citations via the knitcitations package.
I'm not sure I understand what "self-rendering" means in this context? You have some javascript that is taking markdown and spitting out HTML, right?
How is that different from any of the existing JS libs? The fact that it is one line of code vs. a couple extra?
How is that different from any of the existing JS libs? The fact that it is one line of code vs. a couple extra?
Probably because it's supposed to be "introspective", in the sense that you just add a single HTML line to your otherwise usual markdown/tex files, and the javascript recognizes this patterns ie. it ignores the <script> tag and renders everything else.
I thought view source tells the story quite well... The document looks very clean.
This project is awesome, but by relying on Commonmark, you can't use this together with code blocks containing $ (meaning it doesn't work for languages like PHP and R). It's unfortunate that the Commonmark creators ignored requests to handle cases like this.
> It's unfortunate that the Commonmark creators ignored requests to handle cases like this.
What’s your proposed solution?
FWIW the main goal of CommonMark is to fix the ambiguity of Markdown.pl and provide a common subset of features for all parsers to target, instead of trying to include everyone’s pet feature.
What’s your proposed solution?
FWIW the main goal of CommonMark is to fix the ambiguity of Markdown.pl and provide a common subset of features for all parsers to target, instead of trying to include everyone’s pet feature.
> provide a common subset of features for all parsers to target, instead of trying to include everyone’s pet feature
I'm aware. Equations aren't exactly a "pet feature" though, unless things like code blocks are also pet features.
There's an easy solution - have syntax that turns off parsing (a nowiki type syntax).
Commonmark accomplishes very little if it can't be used to produce the documents people are creating. If you want to add equations, you need an extension. And if you use two extensions, you need to make sure they interact correctly. And you have to be sure that all the extensions that might possibly be used interact nicely together. At that point, you have your own version of markdown, and you need to preprocess all your commonmark documents.
But yeah, it's nice that all commonmark implementations use the same syntax for bold and italics. Unfortunately writing involves a lot more than that.
I'm aware. Equations aren't exactly a "pet feature" though, unless things like code blocks are also pet features.
There's an easy solution - have syntax that turns off parsing (a nowiki type syntax).
Commonmark accomplishes very little if it can't be used to produce the documents people are creating. If you want to add equations, you need an extension. And if you use two extensions, you need to make sure they interact correctly. And you have to be sure that all the extensions that might possibly be used interact nicely together. At that point, you have your own version of markdown, and you need to preprocess all your commonmark documents.
But yeah, it's nice that all commonmark implementations use the same syntax for bold and italics. Unfortunately writing involves a lot more than that.
> There's an easy solution - have syntax that turns off parsing (a nowiki type syntax).
Okay, that's fair, but CommonMark standardized de facto standards -- it was born as Standard Markdown after all -- and AFAIK there's no de facto standard here, so it would be pretty dangerous for jgm to pull something out of thin air. I can see why there's reluctance here.
(IIRC fenced code blocks were pretty widespread before CommonMark, and even if not, being part of GFM would almost make it a de facto standard. Note that CommonMark was a joint effort among GitHub, Stack Exchange, Reddit, etc.; SE's <!-- lang --> solution was atrocious, so GFM fenced code block was an obvious win.)
Also, code blocks with language specification is apparently of way higher importance to programmers than typeset math, so comparing the two isn't that useful. (I say this as a physicist who breathe typeset math.)
Now, going back to
> you can't use this together with code blocks containing $ (meaning it doesn't work for languages like PHP and R).
Why do you need to typeset math within code blocks or code spans? Code blocks and code spans are verbatim by definition. Interpret $ inside code blocks and code spans as verbatim; interpret $ outside as LaTeX delimiters. Problem solved. Maybe I'm missing something.
Okay, that's fair, but CommonMark standardized de facto standards -- it was born as Standard Markdown after all -- and AFAIK there's no de facto standard here, so it would be pretty dangerous for jgm to pull something out of thin air. I can see why there's reluctance here.
(IIRC fenced code blocks were pretty widespread before CommonMark, and even if not, being part of GFM would almost make it a de facto standard. Note that CommonMark was a joint effort among GitHub, Stack Exchange, Reddit, etc.; SE's <!-- lang --> solution was atrocious, so GFM fenced code block was an obvious win.)
Also, code blocks with language specification is apparently of way higher importance to programmers than typeset math, so comparing the two isn't that useful. (I say this as a physicist who breathe typeset math.)
Now, going back to
> you can't use this together with code blocks containing $ (meaning it doesn't work for languages like PHP and R).
Why do you need to typeset math within code blocks or code spans? Code blocks and code spans are verbatim by definition. Interpret $ inside code blocks and code spans as verbatim; interpret $ outside as LaTeX delimiters. Problem solved. Maybe I'm missing something.
> Why do you need to typeset math within code blocks or code spans?
The problem is that commonmark doesn't recognize math at all. If you want to use texme, you can't have code blocks that containing $. For it to work properly, you'd need a preprocessor to parse the whole file, convert equations in the text as needed but ignore text in code, output to a format that doesn't cause problems with commonmark (for instance, protecting underscores outside code blocks). At that point you are way beyond just "using commonmark". It's simply not suited for even moderately complicated documents.
The problem is that commonmark doesn't recognize math at all. If you want to use texme, you can't have code blocks that containing $. For it to work properly, you'd need a preprocessor to parse the whole file, convert equations in the text as needed but ignore text in code, output to a format that doesn't cause problems with commonmark (for instance, protecting underscores outside code blocks). At that point you are way beyond just "using commonmark". It's simply not suited for even moderately complicated documents.
Commonmark wasn't trying to be a "useful" or "good" version of Markdown. Commonmark was trying to nail down the specification of a _particular existing version_ of Markdown because devs weren't handling the edge cases correctly and it was problematic for users.
From a look at the github, there is a sudden "surge" or commits, likely to prepare to present it to hackernews.
But overall this github doesn't seem to be maintained and users issues are sitting there unanswered.
https://github.com/susam/texme/commits/master
https://github.com/susam/texme/commits/master
You are right. When I was about to share the project demo page today, I realized that it should have a link to the project README, so that visitors can read how to use this project. That's when I pushed a commit today. So while I was at it, it made sense to do a few other minor cleanups.
The tool, as is, works for me and my friends. I don't intend to make any major changes to this tool except for occasional bug fixes, so the number of commits is going to be less in future.
I know there are a few outstanding bugs. I have a full-time job and I am contributing to a few other open source projects too, so I don't have a lot of spare time to investigate and fix these issues right now but I may get to it later.
If there are others who might want to help the project by investigating the bugs and/or fixing them, they are very welcome. For any fixes merged to the project, I will be grateful to the contributors and as a token of gratitude, I will mention their names in the README and thank them.
The tool, as is, works for me and my friends. I don't intend to make any major changes to this tool except for occasional bug fixes, so the number of commits is going to be less in future.
I know there are a few outstanding bugs. I have a full-time job and I am contributing to a few other open source projects too, so I don't have a lot of spare time to investigate and fix these issues right now but I may get to it later.
If there are others who might want to help the project by investigating the bugs and/or fixing them, they are very welcome. For any fixes merged to the project, I will be grateful to the contributors and as a token of gratitude, I will mention their names in the README and thank them.
> If there are others who might want to help the project by investigating the bugs and/or fixing them, they are very welcome. For any fixes merged to the project, I will be grateful to the contributors and as a token of gratitude, I will mention their names in the README and thank them.
Your claim doesn't match your intent really. Some of your repos have long(many months) old issues with zero interaction from you yet I see activity in commit history.
For small projects like yours, there might be first-time contributors who look up to an author. If you don't interact, they'll get demotivated. If you don't want to interact, leave a comment on your README that "hey am super busy, send a PR with proper tests and may be I'll look when am free, no promises!". :) A little kindness goes a long way.
Your claim doesn't match your intent really. Some of your repos have long(many months) old issues with zero interaction from you yet I see activity in commit history.
For small projects like yours, there might be first-time contributors who look up to an author. If you don't interact, they'll get demotivated. If you don't want to interact, leave a comment on your README that "hey am super busy, send a PR with proper tests and may be I'll look when am free, no promises!". :) A little kindness goes a long way.
[deleted]
Thank you for the feedback. I usually do not prefer commenting on an issue when I know I cannot add something new about the issue. At the same time, I understand your point about the need to interact. I will keep it in mind for future. I will also begin responding to the old issues in the next few days. :-)
Most repos are interesting but unmaintained, it appears that author brushes up a repo before posting by commit waves and then present on HN.
FINALLY!
Thank you for this!
Edit: If any @Amazon Kindle devs read this, can you please reskin your technical texts to adopt this?
Thank you for this!
Edit: If any @Amazon Kindle devs read this, can you please reskin your technical texts to adopt this?
Previous discussion (2018-10): https://news.ycombinator.com/item?id=18307747
This is brilliant.
All we need now is a way for it to fall back to pre-rendered static html for cases where there is no JS, and to be indexed.
All we need now is a way for it to fall back to pre-rendered static html for cases where there is no JS, and to be indexed.
Wouldn't that entirely defeat the purpose? If you're already pre-generating the HTML anyway, it's a waste not to just always serve it, making the self-rendering part redundant.
Bingo. But it is still brilliant.
(Shameless self promotion) I created something equivalent that has an editor too: http://mathdown.novidee.com/14db95afe202f4c1adfc3214aa57af01
Given that it’s parsing from a textarea, can TeXMe be used as an editor? I think some contenteditable editors have the rendering from markdown as well.
Do you mean something like this: https://opendocs.github.io/texme/examples/interactive-demo.h...? TeXMe can be used in this manner but not out of the box. If you see the source code of this interactive demo page, it uses the texme.render() function call from the TeXMe API to take input from the textarea element and render it in a div element.
Providing an interactive editor was not the goal of this project. The goal of this project was to write a text file using a mix of LaTeX and Markdown (while conforming to CommonMark specification as much as possible) and then share this file with my friends in such a way that they can render it easily by just opening it in a web browser. As a result, it completely replaces the textarea element with the rendered HTML.
If someone wants to take this project and create an interactive editor with it, please feel free to fork this project and do so. If you face any problems, please let me know by filing issues on the GitHub project: https://github.com/susam/texme.
Providing an interactive editor was not the goal of this project. The goal of this project was to write a text file using a mix of LaTeX and Markdown (while conforming to CommonMark specification as much as possible) and then share this file with my friends in such a way that they can render it easily by just opening it in a web browser. As a result, it completely replaces the textarea element with the rendered HTML.
If someone wants to take this project and create an interactive editor with it, please feel free to fork this project and do so. If you face any problems, please let me know by filing issues on the GitHub project: https://github.com/susam/texme.
Would this work offline, by <script> referring to your local copy of texme.min.js https://github.com/susam/texme/blob/master/texme.min.js ?
Or is there much more to it?
Or is there much more to it?
Apparently TeXMe needs to load MathJax which in turn needs to load a bunch of fonts, so no, that script alone won't work offline.
Consider pre-rendering formulas instead of doing it client-side (KateX supports this, dunno about MathJax). In my experience, rendering a lot of equations on screen via MathJax can be quite taxing.
As cool as this is, I just don't really see the use for this, when static site generators like Hugo and Jekyll exist, and produce pages that will load measurably faster...
I use org mode + pandoc to achieve the same thing. Jekyll than converts the markdown to html and with a mathjax header, makes a really nice blogging system.
What are the pros and cons of rendering content like this?
Debatable, but it might
+ save some network bandwidth in case of a sufficiently large/complex document
- burn some extra energy client-side in case of a sufficiently large/complex document
+ allow some kind of interactive manipulation/modification by the reader
+ save some network bandwidth in case of a sufficiently large/complex document
- burn some extra energy client-side in case of a sufficiently large/complex document
+ allow some kind of interactive manipulation/modification by the reader
I would like this for Asciidoc which is the better Markdown
is this just latex math or the full latex?
Rendering quality is really good. Looks just like PDF.
> Looks just like PDF.
It's better than PDF. Text in PDF is not reflowable. This demo is by virtue of being HTML under the hood. Looks good even after resizing the browser.
I wish PDFs had a reflowable mode. What would it take to add such a mode to the PDF spec without breaking backward compatibility?
PDF's fixed layout is a real problem when we try to convert PDF books containing math to EPUB or MOBI with Calibre. It really messes up the beautifully typeset math in the PDF during the conversion. What's a good solution for this?
It's better than PDF. Text in PDF is not reflowable. This demo is by virtue of being HTML under the hood. Looks good even after resizing the browser.
I wish PDFs had a reflowable mode. What would it take to add such a mode to the PDF spec without breaking backward compatibility?
PDF's fixed layout is a real problem when we try to convert PDF books containing math to EPUB or MOBI with Calibre. It really messes up the beautifully typeset math in the PDF during the conversion. What's a good solution for this?
That’s the point of PDFs and EPUBs. If you want resizable PDFs, use EPUB which is an organized and compressed version of HTML.
Edit.
I think the reason you lose quality during conversion is because without the original LaTeX code, you’re just importing copied images of math. If you have access to the original content, then you can maintain quality regardless of window size. One example is arxivvanity which uses the original TeX files on arxiv to rerender in HTML. What do you do without the source code? You can either try vectorizing the images to get scalable SVGs, which depends on heuristics and doesn’t recover typeset perfectly. Or perhaps reverse engineer the formulas with OpenCV or another ML method; this one is pretty hard to do and I’ve only seen mathpix, a paid service, which does this.
Either case it’s a strong argument that content should be separate from content rendering which is why I like TeXMe as a concept and why people should avoid PDFs.
Edit.
I think the reason you lose quality during conversion is because without the original LaTeX code, you’re just importing copied images of math. If you have access to the original content, then you can maintain quality regardless of window size. One example is arxivvanity which uses the original TeX files on arxiv to rerender in HTML. What do you do without the source code? You can either try vectorizing the images to get scalable SVGs, which depends on heuristics and doesn’t recover typeset perfectly. Or perhaps reverse engineer the formulas with OpenCV or another ML method; this one is pretty hard to do and I’ve only seen mathpix, a paid service, which does this.
Either case it’s a strong argument that content should be separate from content rendering which is why I like TeXMe as a concept and why people should avoid PDFs.
> without the original LaTeX code, you’re just importing copied images of math
Not necessarily. See MathML.
Not necessarily. See MathML.
PDF is optimized for printing paged media. When I read (instead of skim) physics and math papers I still prefer to print them out. Pagination support is still very primitive in the web stack, at least when I last tried https://www.pagedmedia.org/paged-js/. EPUBs are good until you have equation blocks, floats, etc., then they are pretty far TeX/LaTeX-level typesetting.
> I wish PDFs had a reflowable mode. What would it take to add such a mode to the PDF spec without breaking backward compatibility?
PDF has everything including the kitchen sink. There’s a fairly esoteric feature called “tagged PDFs” which contain extra markup commands within the page description that map it to a logical structure and alternative textual presentations (e.g. MathML). The problem is that nobody uses it as it’s extremely hard to generate (most software which does text layout discards high level semantic details fairly early on, making it hard to retrofit - pdfTeX for example only recently started inserting space characters between words!). Likewise PDF viewers don’t bother to support it.
Reflowing a tagged pdf is fairly easy... once you have the pdf, which of course you don’t.
It’s a hard problem.
PDF has everything including the kitchen sink. There’s a fairly esoteric feature called “tagged PDFs” which contain extra markup commands within the page description that map it to a logical structure and alternative textual presentations (e.g. MathML). The problem is that nobody uses it as it’s extremely hard to generate (most software which does text layout discards high level semantic details fairly early on, making it hard to retrofit - pdfTeX for example only recently started inserting space characters between words!). Likewise PDF viewers don’t bother to support it.
Reflowing a tagged pdf is fairly easy... once you have the pdf, which of course you don’t.
It’s a hard problem.
> It's better than PDF. Text in PDF is not reflowable.
This is relative. Many people dislike reflowable text, and often it is not very convenient. Book typesetting is hard, and once it is done, reflowing the text changes the book, usually for worse. Maybe not for plain text, but definitely for math and poetry, were the "flow" is a crucial part of the work, and it was carefully set up by the author. You do not ever want to "reflow" math or poetry. For these two subjects (my favorite ones, incidentally) pdf seems to be the best option.
This is relative. Many people dislike reflowable text, and often it is not very convenient. Book typesetting is hard, and once it is done, reflowing the text changes the book, usually for worse. Maybe not for plain text, but definitely for math and poetry, were the "flow" is a crucial part of the work, and it was carefully set up by the author. You do not ever want to "reflow" math or poetry. For these two subjects (my favorite ones, incidentally) pdf seems to be the best option.
Btw, VSCode added native support for Jupyter notebooks recently.