From 4aa3f3aa097a785ccb18bebc30af440a826e0a3e Mon Sep 17 00:00:00 2001 From: qi-zhang Date: Wed, 11 Sep 2013 22:45:57 -0700 Subject: [PATCH 1/5] For 1st week. --- 2013-09-07.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/2013-09-07.md b/2013-09-07.md index e69de29..e18a077 100644 --- a/2013-09-07.md +++ b/2013-09-07.md @@ -0,0 +1,11 @@ +Reflection for the week 9/1 to 9/7. + +It was pretty challenging to get ssh access to the Ubuntu server running on my Virtual box. + +By default there is only one network adaptor, known as “Adaptor 1” and it was attached to “NAT”. Such kind of configuration makes it impossible to access the box with its IP address, not to mention establishing ssh access. + +I tried to re-configure the network adaptor to attach to “bridged adaptor” and I’m able to establish ssh access to it. Soon I found that was not a good idea since the IP address of the box keep changing every time I restarted it. + +Finally I configured the “Adaptor 1” back to attached to “NAT” and enabled “Adaptor 2” and let it attached to “Host-Only Adapter”. With help from server guide I found on Ubuntu’s website (https://help.ubuntu.com/13.04/serverguide/network-configuration.html), I finally get the box to support “Adaptor 2” and now I can establish ssh access to the box. + +It was pretty challenging for me since this is the first time I learn some terms in networking. But I’m so glad I did it. From aba31eeafe43e1c3662b191234c42b4d2ea79fb5 Mon Sep 17 00:00:00 2001 From: qi-zhang Date: Wed, 11 Sep 2013 22:49:58 -0700 Subject: [PATCH 2/5] Update 2013-09-07.md --- 2013-09-07.md | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-) diff --git a/2013-09-07.md b/2013-09-07.md index e18a077..831a521 100644 --- a/2013-09-07.md +++ b/2013-09-07.md @@ -1,11 +1 @@ -Reflection for the week 9/1 to 9/7. - -It was pretty challenging to get ssh access to the Ubuntu server running on my Virtual box. - -By default there is only one network adaptor, known as “Adaptor 1” and it was attached to “NAT”. Such kind of configuration makes it impossible to access the box with its IP address, not to mention establishing ssh access. - -I tried to re-configure the network adaptor to attach to “bridged adaptor” and I’m able to establish ssh access to it. Soon I found that was not a good idea since the IP address of the box keep changing every time I restarted it. - -Finally I configured the “Adaptor 1” back to attached to “NAT” and enabled “Adaptor 2” and let it attached to “Host-Only Adapter”. With help from server guide I found on Ubuntu’s website (https://help.ubuntu.com/13.04/serverguide/network-configuration.html), I finally get the box to support “Adaptor 2” and now I can establish ssh access to the box. - -It was pretty challenging for me since this is the first time I learn some terms in networking. But I’m so glad I did it. +Empty Template From cfff69f2796dfd9d9dab896972b34abdbc4a7716 Mon Sep 17 00:00:00 2001 From: qi-zhang Date: Sat, 19 Oct 2013 19:43:03 -0700 Subject: [PATCH 3/5] Create week7 --- week7 | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 week7 diff --git a/week7 b/week7 new file mode 100644 index 0000000..635ad14 --- /dev/null +++ b/week7 @@ -0,0 +1,12 @@ +

Weekly Reflections for the week 10/13-10/19

+ +

Working with JSON in Python

+ +JSON is format of hierarchical data file. The format of JSON file is similar to a flexible combination of dict and list object in Python. Hierarchical data file means same field name is reusable across the database as long as these field names are nested under different hierarch. That's why it is not a good idea to search for certain information with regular expression in hierarchical data files. To find certain information of a hierarchical file, it is necessary to figure the structure of a JSON file. + +In terms of file format, JSON file is similar to a dict object in Python, but it's value could be flexible combination of dict object and list object. Naturally, we can leverage concept of dict object and list object to parse a JSON file. + +The first step is to create a dict object from a JSON file with json.loads which is a function defined in json module. json.loads() returns a dict object. For example, + + +detail=json.loads(urllib.urlopen("http://earthquake.usgs.gov/product/nearby-cities/ci11380834/us/1382197630296/nearby-cities.json").read()) From 001a5bf8e7a3784fd618115969394fa0f86bc0f7 Mon Sep 17 00:00:00 2001 From: qi-zhang Date: Sat, 19 Oct 2013 20:21:30 -0700 Subject: [PATCH 4/5] Update 2013-09-07.md --- 2013-09-07.md | 35 ++++++++++++++++++++++++++++++++++- 1 file changed, 34 insertions(+), 1 deletion(-) diff --git a/2013-09-07.md b/2013-09-07.md index 831a521..d7ccf13 100644 --- a/2013-09-07.md +++ b/2013-09-07.md @@ -1 +1,34 @@ -Empty Template +

Weekly Reflections for the week 10/13-10/19

+ +

Working with JSON in Python

+ +At first, I tried to find the information of interest with regular expression. However it turns out that there are many matches and some of them does not make sense at all. So I tried to do some research on the format of JSON file and realized that it is a hierarchical data file. Hierarchical means same field name is reusable across the data file as long as these field names are nested under different hierarch. That's why regular expression might not work with hierarchical data files. To find certain information of a hierarchical file, it is necessary to figure out how was information organized in a JSON file. + +Fortunately, JSON file is similar to a dict or list object in Python, but it's value or element could be flexible combination of other dict or list objects. Naturally, I tried to treat it as dict or list object and use divide-and-conquer method to study how was information organized. + +The first step is to create a object from a JSON file with json.loads which is a function defined in json module. json.loads() might return a dict object or a list object. For example: + +cities=json.loads(urllib.urlopen("http://earthquake.usgs.gov/product/nearby-cities/ci11380834/us/1382197630296/nearby-cities.json").read()) + +returns a list object cities. This is a simple case so we can review the contents of the first element. + +{u'direction': u'ESE', + u'distance': 3, + u'latitude': 33.91752, + u'longitude': -116.78724, + u'name': u'Cabazon, California', + u'population': 2535} + +Apparently the element of the list object is a dict object with unicode keys. For example value of the key which equals to unicode(name) is unicode version of city name. To to get the population information of this city, we can get it simply by cities[0][unicode("population")]. + +This is a very simple example, but it shows how to get information as long as you know the structure of the JSON file. + +Sometime the JSON file could be pretty complicated and there is no any background information about the structure of the file. In that case, we might do it manually like this: + +1.Create a object from json file. + +2.Apply type() to verify the data type of this object. + +3.If it is a list, apply len function to find number of elements in it. Otherwise apply keys method to get more details about its subfields. + +4.Choose a element or a subfield as new object and repeat step 2,3 until you reach the leaf node or find the field of interest. From 46b9fbd4688858ada8b941227acf1655f4785ec6 Mon Sep 17 00:00:00 2001 From: qi-zhang Date: Sat, 19 Oct 2013 20:24:08 -0700 Subject: [PATCH 5/5] Delete week7 --- week7 | 12 ------------ 1 file changed, 12 deletions(-) delete mode 100644 week7 diff --git a/week7 b/week7 deleted file mode 100644 index 635ad14..0000000 --- a/week7 +++ /dev/null @@ -1,12 +0,0 @@ -

Weekly Reflections for the week 10/13-10/19

- -

Working with JSON in Python

- -JSON is format of hierarchical data file. The format of JSON file is similar to a flexible combination of dict and list object in Python. Hierarchical data file means same field name is reusable across the database as long as these field names are nested under different hierarch. That's why it is not a good idea to search for certain information with regular expression in hierarchical data files. To find certain information of a hierarchical file, it is necessary to figure the structure of a JSON file. - -In terms of file format, JSON file is similar to a dict object in Python, but it's value could be flexible combination of dict object and list object. Naturally, we can leverage concept of dict object and list object to parse a JSON file. - -The first step is to create a dict object from a JSON file with json.loads which is a function defined in json module. json.loads() returns a dict object. For example, - - -detail=json.loads(urllib.urlopen("http://earthquake.usgs.gov/product/nearby-cities/ci11380834/us/1382197630296/nearby-cities.json").read())