[HDR-3940] BE - Convert Ruby hash to json if SyntaxError#5
[HDR-3940] BE - Convert Ruby hash to json if SyntaxError#5
Conversation
| begin | ||
| ActiveRecord::Coders::YAMLColumn.new(Object).load(obj) | ||
| rescue Psych::SyntaxError | ||
| obj = obj.gsub("=>", ":") | ||
| ActiveRecord::Coders::YAMLColumn.new(Object).load(obj) | ||
| end |
There was a problem hiding this comment.
First, Can we do it this way? To avoid parsing it twice?
| begin | |
| ActiveRecord::Coders::YAMLColumn.new(Object).load(obj) | |
| rescue Psych::SyntaxError | |
| obj = obj.gsub("=>", ":") | |
| ActiveRecord::Coders::YAMLColumn.new(Object).load(obj) | |
| end | |
| begin | |
| obj = obj.gsub("=>", ":") | |
| ActiveRecord::Coders::YAMLColumn.new(Object).load(obj) | |
| end |
Second, what happens if we have a value like this `"{"a" => "some value include =>"}"? Could you test it?
There was a problem hiding this comment.
- I don't want to modify a correct JSON parseable string which contains => for eq "{"a": "sample=>text"}"
- I don't think "{"a" => "some value include =>"}" is possible as the incorrect data also has pattern which is hash.to_s
There was a problem hiding this comment.
- I don't want to modify a correct JSON parseable string which contains => for eq "{"a": "sample=>text"}"
- I don't think "{"a" => "some value include =>"}" is possible as the incorrect data also has pattern which is hash.to_s
why is "sample=>text" possible in the first point and not possible in the 2nd point? It should be possible in both, that's why we should find a safer way to replace "=>" with ":" in both cases.
Consider the following:
"{"a": "value of a has a =>"}"
"{"a" => "value of a has a =>"}"both should be parsed as
"{"a": "value of a has a =>"}"
"{"a": "value of a has a =>"}"but now, they are parsed as:
"{"a": "value of a has a =>"}"
"{"a": "value of a has a :"}" # this is wrongI don't want to modify a correct JSON
If a json is already correct, then the proper replace function should not modify it. If a json contains "=>" in its value, we should not replace it.
There was a problem hiding this comment.
I think a proper test coverage would be better
https://accredible.atlassian.net/browse/HDR-3940
We store
audited_changesas JSON parseable string: "{"a":1}" which can be parsed by Audit gem serializer: YAMLIfTextColumnTypeBut some data are incorrectly stored as non parseable string: {"a" => 1}" which gives raises Psych::SyntaxError https://app.bugsnag.com/accredible/acms-api/errors/68a4828815e70991bae806b3?filters[event.since]=30d&filters[error.status]=open&filters[search]=psych
It is unknown to me how such datas got stored in the very first place. I tried to find sources but couldn't find any. So my assumption is that such bad data got stored due to some temporary code changes which then later got corrected