Repository navigation
HTML escaping in JSON+LD #110
Description
Activity
I’m not sure what the expected behavior is.
The standard that you link says:
- That only
<script>,</script>,<!--and-->need escaping. - That the escaping will remain after processing.
To me it sounds like extruct is currently acting according to the standard.
I am unsure, though. Also, even in that case, I wonder if such decoding should be an optional feature at least.
- That only
The "Note" seems to imply encoding the "usual suspects" of HTML entities:
The reasoning behind it is specifically to avoid having
<script>and/or<!--in the middle of thescriptblock without escaping. However, it seems they decided to cover all the characters commonly escaped when within an HTML attribute.The standard is about rendering the JSON+LD into an HTML document, but extruct is for extracting the data back into python data structures so it should be doing the opposite. Right now extruct is not unescaping back into regular text.
Here's an example:
from extruct.jsonld import JsonLdExtractor from extruct.xmldom import XmlDomHTMLParser from lxml.html import fromstring domparser = XmlDomHTMLParser(encoding='UTF8') content = """ <script type="application/ld+json"> { "@context": "http://schema.org/", "@type": "Thing", "name": "<script> <!-- &"'" } </script> """ tree = fromstring(content, parser=domparser) jsonld = JsonLdExtractor().extract_items(tree) print(jsonld[0]['name']) # expected output *should* be `<script> <!-- &"'`
I wonder what it means for us that it says there “the content will remain escaped after processing through the JSON-LD API”.
Excellent point. Yeah, I was wondering about that too...
Here's some documentation on the JSON-LD API. The gist seems to be an API that "restructuring data according to the defined transformations" using methods
compact,expand, andflatten. The end result is still JSON-LD, but reformatted structurally. extruct'sJsonLdExtractoris not implementing that API.I found an example of
expand(example 2 going to example 3). That API looks like it'd be really useful to have to transform JSON-LD into a more standardized format before processing, but that's a separate issue.expandwould make the output closer (the same, even?) to the output fromRDFaExtractor.I guess that note is there because after processing JSON-LD into other JSON-LD, the end result may or may not be in an HTML context. I don't think it applies to the output of extruct.
Reacted by Adrián ChavesIs this the same issue?
jslde = JsonLdExtractor() schema = jslde.extract(article.html) Expecting ',' delimiter: line 7 column 54 (char 281)@gk2go - The issue is about unescaping escaped HTML entities. The issue you're having is about malformed JSON because the person didn't quote the
"as either\"or"."headline": "Argentina designates Hezbollah "terrorist" organisation",
Please use https://search.google.com/structured-data/testing-tool/ to test structured data first before suspecting a bug with extruct.
I came across some JSON+LD on a site that contained a
&and I assumed that I accidentally escaped something somewhere. However, I found that that was what was actually in the content and also that the standard says that it should be there.For my application, I would like that
&to be a&, but I was wondering if extruct should be doing this already?