Add scraper for knowledge base solutions data #49
Conversation
lpiwowar
left a comment
lpiwowar
left a comment
There was a problem hiding this comment.
Overall, looks good to me!:) 👍 Just three small things (one blocking) that need to be polished.
| "clientName": "cli", | ||
| "expression": query, | ||
| "q": "*", | ||
| "rows": max_results, |
There was a problem hiding this comment.
Is there no limit on how many rows you can fetch? Do you need to fetch them in batches?
There was a problem hiding this comment.
Yes, the API doesn't limit us. We can fetch all records at once
There was a problem hiding this comment.
I vaguely remember seeing some issues, but I see only 6823 solutions for https://access.redhat.com/search/?q=*&p=1&rows=10&documentKind=Solution&sort=lastModifiedDate+desc&product=Red+Hat+OpenStack+Platform, so we should be fine.
| "topic": raw_result.get('publishedTitle', ''), | ||
| "issue": ''.join(raw_result.get('issue', '')), | ||
| "diagnosticsteps": ''.join(raw_result.get('solution_diagnosticsteps', 'N/A')), | ||
| "text": ''.join(raw_result.get('solution_resolution', 'N/A')), |
There was a problem hiding this comment.
Isn't there root_cause and environment fields as well?
There was a problem hiding this comment.
I’m not sure how much insight it will give us, since only half of the solutions include this field, but it’s definitely room for experimentation
EmilienM
left a comment
EmilienM
left a comment
There was a problem hiding this comment.
LGTM 👍
thanks for this work
EmilienM
dismissed
lpiwowar’s stale review
I think Lukas's comments were addressed, we can iterate afterwards.
No description provided.