Serialize Django Data for JavaScript(epicserve.com)
epicserve.com
Serialize Django Data for JavaScript
https://epicserve.com/django/2022/08/27/serialize-django-data-for-js.html
16 comments
I've updated the article based on some of the feedback.
Or simply use Django's builtin mechanism for that instead of poorly reimplementing it: https://docs.djangoproject.com/en/4.1/ref/templates/builtins...
Thank you for pointing this out, I forgot about this template filter. This unfortunately still wouldn't properly all the data types mentioned in my article (e.g. QuerySets).
Check out the "values_list" method for flattening a queryset.
> get_data_from_data_attribute({{ data|to_json }});
This constitutes an injection vulnerability that can be demonstrated with string "</script><script>alert(1<2)</script><script>".
If you want to inject arbitrary strings into a script tag, you need to reach for some more exotic escapes, starting with things like </script> → <\/script> or \u003c/script>. But I’m not going to give you a complete solution because I don’t want you to think this is a good idea. (But if you really want to know: look at the script data state in the HTML spec, and follow the parser. All up, you could do it two different ways, one requiring three things and the other two, though one of the two/three is impossible after JSON encoding.)
People often treat auto-escaping template languages as safe to do anything in, but they’re just not if you use HTML syntax—there are quite a few hazards to be aware of. Here you disabled auto-escaping (mark_safe in the to_json body) because the auto-escaping broke things in one way, but it introduced another vulnerability. For sanity in this context, you need to use XML syntax (by serving with the MIME type application/xhtml+xml), though some JavaScript libraries might misbehave due to side-effects if they make bad assumptions, and third-party templates might not be designed for XML syntax. (All up, I don’t generally recommend using XML syntax, though it’s a close thing.)
Fun fact: you can actually use entity encoding in JavaScript in the HTML syntax… by using the SVG script tag instead of the HTML one:
This constitutes an injection vulnerability that can be demonstrated with string "</script><script>alert(1<2)</script><script>".
If you want to inject arbitrary strings into a script tag, you need to reach for some more exotic escapes, starting with things like </script> → <\/script> or \u003c/script>. But I’m not going to give you a complete solution because I don’t want you to think this is a good idea. (But if you really want to know: look at the script data state in the HTML spec, and follow the parser. All up, you could do it two different ways, one requiring three things and the other two, though one of the two/three is impossible after JSON encoding.)
People often treat auto-escaping template languages as safe to do anything in, but they’re just not if you use HTML syntax—there are quite a few hazards to be aware of. Here you disabled auto-escaping (mark_safe in the to_json body) because the auto-escaping broke things in one way, but it introduced another vulnerability. For sanity in this context, you need to use XML syntax (by serving with the MIME type application/xhtml+xml), though some JavaScript libraries might misbehave due to side-effects if they make bad assumptions, and third-party templates might not be designed for XML syntax. (All up, I don’t generally recommend using XML syntax, though it’s a close thing.)
Fun fact: you can actually use entity encoding in JavaScript in the HTML syntax… by using the SVG script tag instead of the HTML one:
<svg><script>alert(1<2)</script></svg>This is an excellent point; I should have addressed safety in my article. I'll point out that in my use case, I'm using `safe` on data I create and not any user-generated data.
You should never use `safe` on user data unless you use something like bleach (https://github.com/mozilla/bleach) to sanitize the data. Even then, you should use caution.
You should never use `safe` on user data unless you use something like bleach (https://github.com/mozilla/bleach) to sanitize the data. Even then, you should use caution.
Uhh, don't use the "safe" filter on user data that you're embedding in javascript, as that introduces arbitrary code injection attacks.
As for the default representation not being flat like the author needed, you can use the "values_list" method on your queryset.
I worry that articles like this lead to "the blind leading the blind". The arbitrary js injection attack enabled by their first example is concerning, and really should be accompanied by a big disclaimer saying "don't mess around with this filter unless you actually know what you're doing".
As for the default representation not being flat like the author needed, you can use the "values_list" method on your queryset.
I worry that articles like this lead to "the blind leading the blind". The arbitrary js injection attack enabled by their first example is concerning, and really should be accompanied by a big disclaimer saying "don't mess around with this filter unless you actually know what you're doing".
See my comment here: https://news.ycombinator.com/item?id=32750266
Right, but in another comment you talk about serializing querysets, and I'd be surprised if you can guarantee that no other developer will ever put dangerous data in any of the rows in your queryset. That approach would be building a pretty dangerous foot gun.
Just the whole approach gets dangerously close to a big security issue, even if you do it "right".
Just the whole approach gets dangerously close to a big security issue, even if you do it "right".
This is valid. I'll add an update to the post people should use caution and think about security.
Neat.
I guess I'm a little surprised that the Django REST Framework isn't mentioned, since I thought that's the go-to for pretty much everyone for this task. Certainly this post's code is lighter weight if all you need to do is send some data out of Django.
https://www.django-rest-framework.org/
I guess I'm a little surprised that the Django REST Framework isn't mentioned, since I thought that's the go-to for pretty much everyone for this task. Certainly this post's code is lighter weight if all you need to do is send some data out of Django.
https://www.django-rest-framework.org/
DRF is for building REST APIs, but OP wants to dump the JSON-representation of an arbitrary object into an HTML template. That's a very different use case for which simple JSON-encoding and the json_script filter should suffice.
Definitely use DRF for this kind of usecases.
The problem with his approach is that it'll always expose ALL the fields of a model to the frontend, like hashed passwords, and can come at a performance cost if the queryset was run with `.only("some", "fields")`. This can be tolerable for small projects but it doesn't scale too well on the long term...
The problem with his approach is that it'll always expose ALL the fields of a model to the frontend, like hashed passwords, and can come at a performance cost if the queryset was run with `.only("some", "fields")`. This can be tolerable for small projects but it doesn't scale too well on the long term...
Surely it will only expose the fields you've defined in your Serializer class?
https://www.django-rest-framework.org/api-guide/serializers/...
https://www.django-rest-framework.org/api-guide/serializers/...
In the article, you'll notice I'm not using DRF.
I would certainly make improvements to the script to optimize the serialization of the QuerySet to only output the needed fields. It's hard to write an article that would capture every use case and optimization required.