[FEATURE] issue-44: Add support for file_encoding in sources#46
[FEATURE] issue-44: Add support for file_encoding in sources#46ChillarAnand wants to merge 2 commits intoblackrock:mainfrom
Conversation
277d1d6 to
04e024f
Compare
shpiyu
left a comment
There was a problem hiding this comment.
Thanks for your contribution Anand. I tried running the code in my local but ran into an error. It'd be great if you could fix those.
Thanks, @shpiyu I have updated the code to resolve the issues. |
swarnadhakad
left a comment
There was a problem hiding this comment.
LGTM, @shpiyu Pls review and test.
| self._src = source | ||
| else: | ||
| self._src = self.format_file_path(source, params_map) | ||
| if not source.get('file_encoding'): |
There was a problem hiding this comment.
@ChillarAnand I was thinking to remove this default setting here and add defaults in pd.read_csv calls in file_reader and xml_reader. Like encoding=src.get('file_encoding', 'utf-8'). The benefit is, if we set default closer to where it is being used it'd be easier for the reader to understand that utf-8 is the default encoding. Let me know your thoughts?
Description
Allow users to define custom encoding for files.
Changes Made
Add file_encoding support in sources.
Definition of Done
Before submitting this pull request, please ensure that the following criteria have been met:
Additional Notes
closes #44