diff --git a/02_activities/assignments/DC_Cohort/.Assignment1.md.swp b/02_activities/assignments/DC_Cohort/.Assignment1.md.swp new file mode 100644 index 000000000..d59cc9087 Binary files /dev/null and b/02_activities/assignments/DC_Cohort/.Assignment1.md.swp differ diff --git a/02_activities/assignments/DC_Cohort/Assignment1.md b/02_activities/assignments/DC_Cohort/Assignment1.md index f78778f5b..9e998605f 100644 --- a/02_activities/assignments/DC_Cohort/Assignment1.md +++ b/02_activities/assignments/DC_Cohort/Assignment1.md @@ -204,6 +204,18 @@ Link if you encounter a paywall: https://archive.is/srKHV or https://web.archive Consider, for example, concepts of fariness, inequality, social structures, marginalization, intersection of technology and society, etc. + ``` Your thoughts... + +The importance of data systems encountered in daily life is often overlooked, yet they inherently embed value systems reflecting social, economic, and political climates. As a researcher in drug design and pharmaceuticals, clinical trial databases show how this embedding of values influences how drugs are designed, tested, and approved. + +Clinical trial databases focus on standardization, which shapes the concept of the 'ideal' patient. The 'ideal' patient is often thought of as young, male, and from majority ethnic groups, while excluding those with complex health conditions. This reflects a value system favoring simplicity and control over the outliers of real-world biological diversity. Fixed categories, such as binary sex or broad racial classifications, simplify complex identities but complicate individual variations essential to personalized medicine. + +Economic advantages govern which diseases and populations receive focus, often privileging those with financial viability while underrepresenting rare diseases and marginalized communities. Additionally, risk management values lead to excluding participants who could complicate data clarity, potentially reducing the applicability of trial outcomes to broader patient groups. + +Moreover, clinical trial data systems define 'meaningful' evidence primarily through quantifiable endpoints like biomarker levels, while not valuing subjective patient experiences. These design choices reflect broader data systems that prioritize efficiency, standardization, and legibility, potentially marginalizing those who don't fit dominant norms. + +Overall, clinical trial databases do not simply record data; they also construct realities, shaping patient inclusion, scientific discoveries, and ultimately, who benefits from medical innovation. Recognizing these embedded value systems is vital for designing more inclusive and ethically responsible data infrastructures that serve all patients equitably. This aligns with the insights from Qadri’s article on the social shaping of databases, emphasizing the importance of acknowledging and addressing the values embedded in our data systems. + ``` diff --git a/02_activities/assignments/DC_Cohort/Assignment2.md b/02_activities/assignments/DC_Cohort/Assignment2.md index 9b804e9ee..4dcc24367 100644 --- a/02_activities/assignments/DC_Cohort/Assignment2.md +++ b/02_activities/assignments/DC_Cohort/Assignment2.md @@ -54,8 +54,33 @@ The store wants to keep customer addresses. Propose two architectures for the CU **HINT:** search type 1 vs type 2 slowly changing dimensions. ``` -Your answer... -``` +Type 1 SCD: overwrites existing address data when a customer's address changes and does not store historical records. When a customer relocates, the new address is updated and the old address is permanently lost. + +CUSTOMER_ADDRESS (Type 1): +- address_id +- customer_id +- street_address +- city +- province +- postal_code +- country +- last_updated + +Type 2 SCD: retains a complete history of address changes by creating new records. + +CUSTOMER_ADDRESS (Type 2): +- address_id (PK) +- customer_id (FK) +- street_address +- city +- province +- postal_code +- country +- effective_date +- end_date +- current (boolean flag) + +When a customer's address changes, the existing record's end_date is filled in and current is set to FALSE. The new address is inputted by having current = TRUE, and end_date = NULL. *** @@ -183,5 +208,8 @@ Consider, for example, concepts of labour, bias, LLM proliferation, moderating c ``` -Your thoughts... -``` +An ethical issue is the inherent bias embedded within machine learning systems. Since machine learning models learn from humans, their infratructure is influenced by the assumptions and values of the people who built and labeled that data. These biases can subconsciously guide how we perceive events, identities, and social issues. This can stereotypes, distort reality, and propogate throughout society without the public being cognizant of this affect. + +In order to ethically integrate machine learning into society, public education and transparency are critical. People need to learn that the information machine learning models output are not unbiased facts, but instead are perceptions shaped by human interpretations of data and personal opinions. As a society, we cannot become fully dependent on these systems or assume they are always correct. Instead, we must actively verify, question, and synthesize the information they output. + +As a result, we need to work in partnership with machine learning tools versus surrendering our judgment to them. Retaining our autonomy to research, explain, and strategize independently of machine learning models ensures that we are working in harmony with these tools versus being replaced by them.``` diff --git a/02_activities/assignments/DC_Cohort/assignment 1/Entity-Relationship Diagram.pptx b/02_activities/assignments/DC_Cohort/assignment 1/Entity-Relationship Diagram.pptx new file mode 100644 index 000000000..18db2a50b Binary files /dev/null and b/02_activities/assignments/DC_Cohort/assignment 1/Entity-Relationship Diagram.pptx differ diff --git a/02_activities/assignments/DC_Cohort/assignment 2/ERD.pptx b/02_activities/assignments/DC_Cohort/assignment 2/ERD.pptx new file mode 100644 index 000000000..360137b68 Binary files /dev/null and b/02_activities/assignments/DC_Cohort/assignment 2/ERD.pptx differ diff --git a/02_activities/assignments/DC_Cohort/assignment1.sql b/02_activities/assignments/DC_Cohort/assignment1.sql index c992e3205..786d10cb1 100644 --- a/02_activities/assignments/DC_Cohort/assignment1.sql +++ b/02_activities/assignments/DC_Cohort/assignment1.sql @@ -1,20 +1,23 @@ /* ASSIGNMENT 1 */ /* SECTION 2 */ - --SELECT /* 1. Write a query that returns everything in the customer table. */ - - +SELECT * +FROM customer; /* 2. Write a query that displays all of the columns and 10 rows from the cus- tomer table, sorted by customer_last_name, then customer_first_ name. */ - - +SELECT * +FROM customer +ORDER BY customer_last_name, customer_first_name +LIMIT 10; --WHERE /* 1. Write a query that returns all customer purchases of product IDs 4 and 9. */ - +SELECT * +FROM customer_purchases +WHERE product_id IN (4, 9); /*2. Write a query that returns all customer purchases and a new calculated column 'price' (quantity * cost_to_customer_per_qty), @@ -23,39 +26,63 @@ filtered by customer IDs between 8 and 10 (inclusive) using either: 2. one condition using BETWEEN */ -- option 1 - +SELECT *, + quantity * cost_to_customer_per_qty AS price +FROM customer_purchases +WHERE customer_id >= 8 AND customer_id <= 10; -- option 2 - - +SELECT *, + quantity * cost_to_customer_per_qty AS price +FROM customer_purchases +WHERE customer_id BETWEEN 8 AND 10; --CASE /* 1. Products can be sold by the individual unit or by bulk measures like lbs. or oz. Using the product table, write a query that outputs the product_id and product_name columns and add a column called prod_qty_type_condensed that displays the word “unit” if the product_qty_type is “unit,” and otherwise displays the word “bulk.” */ - - +SELECT product_id, + product_name, + CASE + WHEN product_qty_type = 'unit' THEN 'unit' + ELSE 'bulk' + END AS prod_qty_type_condensed +FROM product; /* 2. We want to flag all of the different types of pepper products that are sold at the market. add a column to the previous query called pepper_flag that outputs a 1 if the product_name contains the word “pepper” (regardless of capitalization), and otherwise outputs 0. */ - - +SELECT product_id, + product_name, + CASE + WHEN product_qty_type = 'unit' THEN 'unit' + ELSE 'bulk' + END AS prod_qty_type_condensed, + CASE + WHEN LOWER(product_name) LIKE '%pepper%' THEN 1 + ELSE 0 + END AS pepper_flag +FROM product; --JOIN /* 1. Write a query that INNER JOINs the vendor table to the vendor_booth_assignments table on the vendor_id field they both have in common, and sorts the result by vendor_name, then market_date. */ - - - +SELECT * +FROM vendor AS v +INNER JOIN vendor_booth_assignments AS vba + ON v.vendor_id = vba.vendor_id +ORDER BY v.vendor_name, vba.market_date; /* SECTION 3 */ -- AGGREGATE /* 1. Write a query that determines how many times each vendor has rented a booth at the farmer’s market by counting the vendor booth assignments per vendor_id. */ - +SELECT vendor_id, + COUNT(*) AS booth_rental_count +FROM vendor_booth_assignments +GROUP BY vendor_id; /* 2. The Farmer’s Market Customer Appreciation Committee wants to give a bumper @@ -63,8 +90,14 @@ sticker to everyone who has ever spent more than $2000 at the market. Write a qu of customers for them to give stickers to, sorted by last name, then first name. HINT: This query requires you to join two tables, use an aggregate function, and use the HAVING keyword. */ - - +SELECT c.customer_first_name, + c.customer_last_name +FROM customer AS c +INNER JOIN customer_purchases AS cp + ON c.customer_id = cp.customer_id +GROUP BY c.customer_id, c.customer_first_name, c.customer_last_name +HAVING SUM(cp.quantity * cp.cost_to_customer_per_qty) > 2000 +ORDER BY c.customer_last_name, c.customer_first_name; --Temp Table /* 1. Insert the original vendor table into a temp.new_vendor and then add a 10th vendor: @@ -77,20 +110,31 @@ When inserting the new vendor, you need to appropriately align the columns to be -> To insert the new row use VALUES, specifying the value you want for each column: VALUES(col1,col2,col3,col4,col5) */ +CREATE TEMP TABLE new_vendor AS +SELECT * FROM vendor; - +-- Then, insert the new 10th vendor +INSERT INTO temp.new_vendor +VALUES (10, 'Thomass Superfood Store', 'Fresh Focused', 'Thomas', 'Rosenthal'); -- Date /*1. Get the customer_id, month, and year (in separate columns) of every purchase in the customer_purchases table. HINT: you might need to search for strfrtime modifers sqlite on the web to know what the modifers for month and year are! */ - - +SELECT customer_id, + STRFTIME('%m', market_date) AS month, + STRFTIME('%Y', market_date) AS year +FROM customer_purchases; /* 2. Using the previous query as a base, determine how much money each customer spent in April 2022. Remember that money spent is quantity*cost_to_customer_per_qty. HINTS: you will need to AGGREGATE, GROUP BY, and filter... but remember, STRFTIME returns a STRING for your WHERE statement!! */ - +SELECT customer_id, + SUM(quantity * cost_to_customer_per_qty) AS total_spent +FROM customer_purchases +WHERE STRFTIME('%m', market_date) = '04' + AND STRFTIME('%Y', market_date) = '2022' +GROUP BY customer_id; diff --git a/02_activities/assignments/DC_Cohort/assignment1_ERD.pdf b/02_activities/assignments/DC_Cohort/assignment1_ERD.pdf new file mode 100644 index 000000000..1fa3fb982 Binary files /dev/null and b/02_activities/assignments/DC_Cohort/assignment1_ERD.pdf differ diff --git a/02_activities/assignments/DC_Cohort/assignment2.sql b/02_activities/assignments/DC_Cohort/assignment2.sql index d6a10dbe0..3b8c7063d 100644 --- a/02_activities/assignments/DC_Cohort/assignment2.sql +++ b/02_activities/assignments/DC_Cohort/assignment2.sql @@ -21,7 +21,19 @@ The `||` values concatenate the columns into strings. Edit the appropriate columns -- you're making two edits -- and the NULL rows will be fixed. All the other rows will remain the same. */ +-- Step 1: select the NULL values +SELECT + product_name, + product_size, + product_qty_type +FROM product +WHERE product_size IS NULL + OR product_qty_type IS NULL; +-- Step 2: Replace NULLs with COALESCE +SELECT + product_name || ', ' || COALESCE(product_size, '') || ' (' || COALESCE(product_qty_type, 'unit') || ')' AS product_description +FROM product; --Windowed Functions /* 1. Write a query that selects from the customer_purchases table and numbers each customer’s @@ -33,18 +45,57 @@ each new market date for each customer, or select only the unique market dates p (without purchase details) and number those visits. HINT: One of these approaches uses ROW_NUMBER() and one uses DENSE_RANK(). */ - +SELECT + customer_id, + market_date, + vendor_id, + product_id, + quantity, + ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY market_date) AS visit_number +FROM customer_purchases +ORDER BY customer_id, market_date; /* 2. Reverse the numbering of the query from a part so each customer’s most recent visit is labeled 1, then write another query that uses this one as a subquery (or temp table) and filters the results to only the customer’s most recent visit. */ +-- Step 1: Reverse numbering +SELECT + customer_id, + market_date, + vendor_id, + product_id, + quantity, + ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY market_date DESC) AS visit_number +FROM customer_purchases; + +-- Step 2: Filter to most recent visit +SELECT * +FROM ( + SELECT + customer_id, + market_date, + vendor_id, + product_id, + quantity, + ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY market_date DESC) AS visit_number + FROM customer_purchases +) AS numbered_visits +WHERE visit_number = 1 +ORDER BY customer_id; /* 3. Using a COUNT() window function, include a value along with each row of the customer_purchases table that indicates how many different times that customer has purchased that product_id. */ - +SELECT + customer_id, + market_date, + product_id, + quantity, + COUNT(*) OVER (PARTITION BY customer_id, product_id) AS times_purchased +FROM customer_purchases +ORDER BY customer_id, market_date, product_id ; -- String manipulations /* 1. Some product names in the product table have descriptions like "Jar" or "Organic". @@ -58,11 +109,27 @@ Remove any trailing or leading whitespaces. Don't just use a case statement for Hint: you might need to use INSTR(product_name,'-') to find the hyphens. INSTR will help split the column. */ - +SELECT + product_name, + CASE + WHEN INSTR(product_name, '-') > 0 + THEN TRIM(SUBSTR(product_name, INSTR(product_name, '-') + 1)) + ELSE NULL + END AS description +FROM product; /* 2. Filter the query to show any product_size value that contain a number with REGEXP. */ - +SELECT + product_name, + product_size, + CASE + WHEN INSTR(product_name, '-') > 0 + THEN TRIM(SUBSTR(product_name, INSTR(product_name, '-') + 1)) + ELSE NULL + END AS description +FROM product +WHERE product_size REGEXP '[0-9]'; -- UNION /* 1. Using a UNION, write a query that displays the market dates with the highest and lowest total sales. @@ -74,8 +141,37 @@ HINT: There are a possibly a few ways to do this query, but if you're struggling 3) Query the second temp table twice, once for the best day, once for the worst day, with a UNION binding them. */ +WITH daily_sales AS ( + SELECT + market_date, + SUM(quantity * cost_to_customer_per_qty) AS total_sales + FROM customer_purchases + GROUP BY market_date +), +ranked_sales AS ( + SELECT + market_date, + total_sales, + RANK() OVER (ORDER BY total_sales DESC) AS sales_rank_desc, + RANK() OVER (ORDER BY total_sales ASC) AS sales_rank_asc + FROM daily_sales +) +SELECT + market_date, + total_sales, + 'Highest Total Sales' AS sales_category +FROM ranked_sales +WHERE sales_rank_desc = 1 +UNION +SELECT + market_date, + total_sales, + 'Lowest Total Sales' AS sales_category +FROM ranked_sales +WHERE sales_rank_asc = 1 +ORDER BY total_sales DESC; /* SECTION 3 */ @@ -90,7 +186,16 @@ Think a bit about the row counts: how many distinct vendors, product names are t How many customers are there (y). Before your final group by you should have the product of those two queries (x*y). */ - +SELECT + v.vendor_name, + p.product_name, + COUNT(DISTINCT c.customer_id) * 5 * vi.original_price AS total_revenue +FROM vendor_inventory vi +JOIN vendor v ON vi.vendor_id = v.vendor_id +JOIN product p ON vi.product_id = p.product_id +CROSS JOIN customer c +GROUP BY v.vendor_name, p.product_name, vi.original_price +ORDER BY v.vendor_name, p.product_name; -- INSERT /*1. Create a new table "product_units". @@ -98,19 +203,52 @@ This table will contain only products where the `product_qty_type = 'unit'`. It should use all of the columns from the product table, as well as a new column for the `CURRENT_TIMESTAMP`. Name the timestamp column `snapshot_timestamp`. */ - +CREATE TABLE product_units AS +SELECT + product_id, + product_name, + product_size, + product_category_id, + product_qty_type, + CURRENT_TIMESTAMP AS snapshot_timestamp +FROM product +WHERE product_qty_type = 'unit'; /*2. Using `INSERT`, add a new row to the product_units table (with an updated timestamp). This can be any product you desire (e.g. add another record for Apple Pie). */ - +INSERT INTO product_units ( + product_id, + product_name, + product_size, + product_category_id, + product_qty_type, + snapshot_timestamp +) +SELECT + product_id, + product_name, + product_size, + product_category_id, + product_qty_type, + CURRENT_TIMESTAMP AS snapshot_timestamp +FROM product +WHERE product_name = 'Apple Pie' + AND product_qty_type = 'unit' +LIMIT 1; -- DELETE /* 1. Delete the older record for the whatever product you added. HINT: If you don't specify a WHERE clause, you are going to have a bad time.*/ - +DELETE FROM product_units +WHERE product_name = 'Apple Pie' + AND snapshot_timestamp = ( + SELECT MIN(snapshot_timestamp) + FROM product_units + WHERE product_name = 'Apple Pie' + ); -- UPDATE /* 1.We want to add the current_quantity to the product_units table. @@ -129,6 +267,27 @@ Finally, make sure you have a WHERE statement to update the right row, you'll need to use product_units.product_id to refer to the correct row within the product_units table. When you have all of these components, you can run the update statement. */ +-- Step 1: Add new column +ALTER TABLE product_units +ADD current_quantity INT; +-- Step 2: Update current quantity total to last quantity value +UPDATE product_units +SET current_quantity = ( + SELECT COALESCE(vi.quantity, 0) + FROM vendor_inventory vi + WHERE vi.product_id = product_units.product_id + ORDER BY vi.market_date DESC + LIMIT 1 +) +WHERE product_units.product_id IN ( + SELECT DISTINCT product_id + FROM vendor_inventory +); + +-- Step 3: Update NULL values to 0 +UPDATE product_units +SET current_quantity = 0 +WHERE current_quantity IS NULL; diff --git a/02_activities/assignments/DC_Cohort/assignment2_logical_model_1.pdf b/02_activities/assignments/DC_Cohort/assignment2_logical_model_1.pdf new file mode 100644 index 000000000..a745cd42e Binary files /dev/null and b/02_activities/assignments/DC_Cohort/assignment2_logical_model_1.pdf differ diff --git a/02_activities/assignments/DC_Cohort/assignment2_logical_model_2.pdf b/02_activities/assignments/DC_Cohort/assignment2_logical_model_2.pdf new file mode 100644 index 000000000..f7a54fdcb Binary files /dev/null and b/02_activities/assignments/DC_Cohort/assignment2_logical_model_2.pdf differ