From 749c983ed617e6e202a5ed0aaa1186f3a17f12c5 Mon Sep 17 00:00:00 2001 From: thangb44 <71145730+thangb44@users.noreply.github.com> Date: Fri, 11 Sep 2020 14:56:55 -0500 Subject: [PATCH 01/17] Add files via upload --- Labs/Lab.1/Lab-1-Solutions.ipynb | 1022 ++++++++++++++++++++++++++++++ 1 file changed, 1022 insertions(+) create mode 100644 Labs/Lab.1/Lab-1-Solutions.ipynb diff --git a/Labs/Lab.1/Lab-1-Solutions.ipynb b/Labs/Lab.1/Lab-1-Solutions.ipynb new file mode 100644 index 0000000..26f17d6 --- /dev/null +++ b/Labs/Lab.1/Lab-1-Solutions.ipynb @@ -0,0 +1,1022 @@ +{ + "nbformat": 4, + "nbformat_minor": 0, + "metadata": { + "colab": { + "name": "Lab-1.ipynb", + "provenance": [], + "collapsed_sections": [] + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.0" + } + }, + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "O5vg8KKRq0sy" + }, + "source": [ + "# Lab 1\n", + "\n", + "## Python Notebooks on Google Colab\n", + "\n", + "Data 1401's Labs, Homework, and Exams will be all in form of iPython notebooks. You may already be familiar with python notebooks if you have used Jupyter before. If so, you are welcome to use whatever means you have to run Jupyter notebooks for this course, though you may get limited support. Our primary means of running python notebooks will be through [Google Colab](https://colab.research.google.com). You may also choose to store your files on google drive.\n", + "\n", + "You will need a google account. If you do not have one or you wish to use a different account for this course, please follow [these instructions](https://edu.gcfglobal.org/en/googledriveanddocs/getting-started-with-google-drive/1/) to make an account.\n", + "\n", + "Once you are ready with your account, you can continue in Colab. Click on the following badge to open this notebook in Colab:\n", + "\n", + "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/UTA-DataScience/DATA1401.2020.Fall/blob/master/Labs/Lab.1/Lab.1.ipynb)\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "oJ16BYj1JF89", + "colab_type": "code", + "colab": {} + }, + "source": [ + "echo" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "FVt_1hPt1dAK" + }, + "source": [ + "## Notebooks in Colab\n", + "\n", + "You now are presumably in Colab. Word of caution, by default, Google Colab does not save your notebooks, so if you close your session, you will loose your work.\n", + "\n", + "So first thing: from the file menu above select \"Save a copy in Drive\"." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "x0JBL_RFrDDj" + }, + "source": [ + "## Storing Notebooks in Google Drive (Optional)\n", + "If you like, you may save your notebooks directly into Google Drive and upload directly to Git (where you will be downloading and uploading your homework). In order properly setup Git, we'll need to work more directly in your Google Drive.\n", + "\n", + "On the left sidebar, press the file icon to see a listing of files accessibile to this Notebook. Then press \"Mount Drive\" and follow the instructions to mount your Google Drive in this notebook. A new cell will be inserted into this notebook, which after you run by pressing the play button will instruct you to follow a link to log into your Google Account and enable access to your Drive in another tab. Finally you will copy a link from the new tab back into the cell in this notebook. Once you are done, press refresh under files in the left sidebar and you should have \"drive/My Drive\" appear." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "hwJ6wJk3tiLv" + }, + "source": [ + "## Github\n", + "All the class material will be stored on github. You will also submit your homework using github. To do so, you will need a github account.\n", + "\n", + "If you do not already have a github account or wish to create a new one for this course, create one:\n", + "* Browse to [github.com](https://github.com).\n", + "* Click the green “Sign up for GitHub”\tbutton.\n", + "* Follow instructions for creating an account.\n", + "* Make sure you remember your github username and password.\n", + "\n", + "Write an email to the course Instructor titled \"Data 1401: Github account\" with your github username (not your password) as the contents.\n", + "\n", + "## Google Groups\n", + "\n", + "Class annoucements will be made via google groups. If you did not already receive an invite to the class google group, had trouble with the invite, or wish to use a different email address, write an email to the course TA titled \"Data 1401: Google Group\" with your preferred email.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "TjfIzdQZqvzk" + }, + "source": [ + "## Introduction: Unix, Git, and Jupyter\n", + "\n", + "This lab aims to introduce you to basic Unix, familiarize you with iPython notebooks and get you setup to submit your homework.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "C_LmOgzFqvzp" + }, + "source": [ + "\n", + "\n", + "### Terminal, Shell, and ssh\n", + "\n", + "\n", + "The terminal is a simple program that generally runs another program, taking mostly keyboard input from you, passing it to this other program, and taking the output of the program and displaying on the screen for you.\n", + "\n", + "The terminal usually runs a program called a shell. Shells present a command prompt where you can type in commands, which are then executed when you press enter. In most shells, there are some special commands which the shell will execute. Everything else you type in, the shell will assume is a name of a program you want to run and arguments you want to pass that program. So if the shell doesn't recognize something you type in, it'll try to find a program with a name that is the same as the first word you gave it. \n", + "\n", + "### Shell in Colab\n", + "\n", + "Unfortunately, google Colab does not allow you to open a terminal window. Jupyter does, so if you are running in Jupyter (which most of you will not be), you may choose to open a terminal window by returning to the jupyter file list tab and selecting new terminal from the top right.\n", + "\n", + "For Colab, we will have to do something non-ideal, but functional. There are several ways to execute shell commands from within a python notebook. For example, you can use any shell command by putting \"!\" in front of the command:\n", + "\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "KJ5f-WO0wcAv", + "colab": {} + }, + "source": [ + "!ls\n", + "!echo \"----------\"\n", + "!ls sample_data" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "8f-n4AXFw-dD" + }, + "source": [ + "Unfortunately, every time you use \"!\" a new environment is created and the state reverted to the original state. Try to understand the difference between the following two sets of commands:\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "99nrBYTWxZJr", + "colab": {} + }, + "source": [ + "!echo \"Technique 1:\"\n", + "!ls\n", + "!cd sample_data\n", + "!ls" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "2-Znf97Lxl-Z", + "colab": {} + }, + "source": [ + "!echo \"Technique 2:\"\n", + "!ls ; cd sample_data ;ls" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "4x9n1rAkxyYl" + }, + "source": [ + "Notebooks allow a bit of \"magic\" (using \"%\") to avoid some of these limitations:\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "vLBPTX4rx3gd", + "colab": {} + }, + "source": [ + "!echo \"Technique 3:\"\n", + "!ls \n", + "%cd sample_data \n", + "!ls" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "U8XpvPjcyH0w" + }, + "source": [ + "For our purposes, we are just going to explicitly start a new shell and interact with it in the output cell. Execute the following cell. You will be able to type and execute commands. Look around a bit using \"ls\" and \"cd. You can stop the cell from running by typing \"exit\". You may also stop the shell by pressing play/stop button." + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "MIDFitLZyuZy", + "colab": {} + }, + "source": [ + "!/bin/bash --noediting" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "q-4hfZBywW25" + }, + "source": [ + "While in this instance your shell is running in a this notebook, you can also run terminals natively on your own computer. On Linux or MacOS, you just have to run a program called terminal. In Windows you can start a \"command prompt\". \n", + "\n", + "Type in \"ls\" into the terminal above and press enter. The shell will find a program called \"ls\", a standard tool in Unix, and run it. \"ls\" lists the contents (files and directories) of your current directory. \n", + "\n", + "For your reference, [Unix/Linux Command Reference](https://github.com/UTA-DataScience/VersionControlSystem/blob/master/VersionControlSystem/linuxRef.pdf) summarizes some of the common shell commands.\n", + "\n", + "A subtle point to realize here is that while the terminal is running in the browser that is running on the computer in front of you, the shell is actually running on a machine on google hardware. The shell prompt typically displays the name of the machine you are using. What you are not seeing is that there is an intermidate program between the terminal running on your computer and the shell running on google. This intermidary program is taking your input from the terminal sending it over the network to google and bringing back the responses for you terminal to display.\n", + "\n", + "A bit of extra information. If you start a terminal on your own computer, the shell runs locally. The \"ls\" command would then list contents of a directory on your computer. You can typically connect to Unix computers by evoking a shell running on that machine over the network. In this case, you would have to initiate this intermidiary program yourself. The program is called \"ssh\" (secure shell). You can \"ssh\" to another machine from your machine, by simply typing \"ssh\" followed by the machine name or IP address. Most likely you would be prompted for a password, after which you would dropped into the prompt of a shell running on the remote machine. \n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "51Eya4LBqvzs" + }, + "source": [ + "## Programs and Environment Variables\n", + "\n", + "You have a listing of your current directory, but you don't know where that directory resides. You can see what directory you are using the command \"pwd\" (print working directory). Issue the command and look at the response. You'll get a slash (\"/\") separated list, known as the path, of the directory hierarchy of your current working directory. On Colab, this will start with \"contents\"\n", + "\n", + "Now back to thinking about the command prompt. Since \"ls\" is a program, it most be stored somewhere. It is clearly not in your working directory, because you didn't see it when you executed \"ls\". We can ask the shell to tell us where it found \"ls\" using the \"which ls\" command. Note that \"which\" is also a program. \"which ls\" comes back with \"/bin/ls\", telling you the \"ls\" program is sitting in \"/bin\" directory of the system. \n", + "\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "RBKYfBowJF9t", + "colab_type": "code", + "colab": {} + }, + "source": [ + "!which ls" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "A57gH8W6JF9y", + "colab_type": "text" + }, + "source": [ + "Lets see what else is in there by issuing a \"ls /bin\" command. " + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "JtfbjvYmJF9z", + "colab_type": "code", + "colab": {} + }, + "source": [ + "!ls /bin" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "YnsTweeGJF94", + "colab_type": "text" + }, + "source": [ + "You will get a long list of programs. You can run any of these programs by just typing their names and pressing enter. You may be able to guess what some of these programs do, but if you want to know, most of them provide you help, using \"--help\" or \"-h\" flag. For example execute \"ls --help\". Quick note that MacOS is a bit different and so \"--help\" doesn't seem to work." + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "1ppZwN9gJF96", + "colab_type": "code", + "colab": {} + }, + "source": [ + "!ls --help" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "l6s9aSprJF9-", + "colab_type": "text" + }, + "source": [ + " For more information about a program or command, you can use Unix's manual pages using the \"man\" command. Try typing \"man ls\". Note that you will need to press space to scroll through lengthy manual pages and \"q\" to exit back to the shell prompt. " + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "qEuDCPvoJF9_", + "colab_type": "code", + "colab": {} + }, + "source": [ + "!man ls" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "VAJaFtqhJF-D", + "colab_type": "text" + }, + "source": [ + "Another command interesting is \"echo\". \"echo\" simply prints whatever you put after it to the screen. " + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "UfS0lC3IJF-D", + "colab_type": "code", + "colab": {} + }, + "source": [ + "!echo \"Hello World\"" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "1VugUXikJF-J", + "colab_type": "text" + }, + "source": [ + "At this point, you may wonder how was it that the shell knew to look for programs in \"/bin\"? The shell keeps a list of places to look for programs an environment variable with the name \"PATH\". The shell keeps a table that map string variable names to string expressions. When the shell starts, its configuration files set some environment variables that it uses. You can see the full list of defined environment variables using the command \"printenv\".\n", + "\n", + "You can use a environment variable in a shell by prepending name of the variable with a dollar sign character. So you can print out the PATH environment variable using the command \"echo $PATH\". What you will see is a colon (\":\") separated list of directories that the shell will search (in order) whenever you type in anything." + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "-hqGpF4UJF-K", + "colab_type": "code", + "colab": {} + }, + "source": [ + "!echo $PATH" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "0ClBrBR8JF-P", + "colab_type": "text" + }, + "source": [ + "You can set you own environment variables. Different shells have different syntax. Lets first figure out what shell we are running. \n", + "\n", + "*Exercise 1:* Use the \"echo\" command to print out the value of the \"SHELL\" environment variable:" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "YS7YFiPwqvzu" + }, + "source": [ + "!/bin/bash --noediting" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "zplxWxIbMNkG", + "colab_type": "code", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 34 + }, + "outputId": "7dc0a807-2cb5-4d9b-8cdd-50781fd382fb" + }, + "source": [ + "!echo SHELL" + ], + "execution_count": 2, + "outputs": [ + { + "output_type": "stream", + "text": [ + "SHELL\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "YoEgruUhqvzw" + }, + "source": [ + "## Navigating Directories\n", + "\n", + "You can change your current directory using the \"cd\" shell command. Note that \"cd\" is not a Unix program. Once in a directory, you can use the \"ls\" command to list the contents or \"pwd\" to remind yourself your current working directory. You can move back one level in your current directory hierarchy using \"cd ..\". In general \"..\" represents the path to a directory one level above your current directory, \"../..\" represents two levels up, and so on. \".\" represents the current directory. If you look at the PATH environment variable, you'll notice that the last item is \".\", telling the shell to look into your current directory for commands. Finally the \"~\" character always refers to your home directory.\n", + "\n", + "Some other file manipulation commands:\n", + "\n", + " - The \"mkdir\" command creates new directories. \n", + " - \"cp\" and \"mv\" allow you to copy and move (or rename) files, taking 2 arguments: the original path/filename and the target path/filename. \n", + " - The \"rm\" and \"rmdir\" commands remove (delete) files and directories.\n", + "\n", + "\n", + "*Exercise 2:* Create a new directory called \"Data-1441\", and another directory inside \"Data-1441\" called \"Lab-1-Solutions\". Perform the rest of the lab in this directory." + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "A16VzZ3G0J8x", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 238 + }, + "outputId": "4755c271-b0b5-4acc-dce9-390e4d57e135" + }, + "source": [ + "!/bin/bash --noediting" + ], + "execution_count": 7, + "outputs": [ + { + "output_type": "stream", + "text": [ + "bash: cannot set terminal process group (1091): Inappropriate ioctl for device\n", + "bash: no job control in this shell\n", + "\u001b]0;root@76e673cd4ac7: /content/sample_data\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/sample_data\u001b[00m# ls\n", + "\u001b[0m\u001b[01;32manscombe.json\u001b[0m \u001b[01;34mData-1441\u001b[0m \u001b[01;32mREADME.md\u001b[0m\n", + "california_housing_test.csv mnist_test.csv\n", + "california_housing_train.csv mnist_train_small.csv\n", + "\u001b]0;root@76e673cd4ac7: /content/sample_data\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/sample_data\u001b[00m# cd Data-1441\n", + "\u001b]0;root@76e673cd4ac7: /content/sample_data/Data-1441\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/sample_data/Data-1441\u001b[00m# ls\n", + "\u001b[0m\u001b[01;34mLab-1-Solutions\u001b[0m\n", + "\u001b]0;root@76e673cd4ac7: /content/sample_data/Data-1441\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/sample_data/Data-1441\u001b[00m# cd Lab-1-Solutions\n", + "\u001b]0;root@76e673cd4ac7: /content/sample_data/Data-1441/Lab-1-Solutions\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/sample_data/Data-1441/Lab-1-Solutions\u001b[00m# ls\n", + "\u001b]0;root@76e673cd4ac7: /content/sample_data/Data-1441/Lab-1-Solutions\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/sample_data/Data-1441/Lab-1-Solutions\u001b[00m# exit\n", + "exit\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "qKQynO67MaQv", + "colab_type": "text" + }, + "source": [ + "mkdir Data-1441\n", + "cd Data-1441\n", + "mkdir Lab-1-Solutions" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "o38c4lbsqvzy" + }, + "source": [ + "## Exploring Unix Filesystem\n", + "\n", + "You can look at the root directory of the system by issuing \"ls /\". As explained in lecture, Unix uses the file system to communicate with devices and between processes. \"/etc\" keeps the configuration files of the system. \"/bin\" and \"/sbin\" store most of the standard Unix programs. \"/usr\" stores installed programs and their associate files, with \"/usr/bin\" usually storing the commands you can run. \n", + "\n", + "*Exercise 3:* List the \"/dev\" directory. How many SSD storage devices do you see? How many partitions does each device have? (Answer in box below)" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "yNj2LXzP2ksl", + "colab": {} + }, + "source": [ + "!/bin/bash --noediting" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "UC1Hgrt3MxMf", + "colab_type": "text" + }, + "source": [ + "0" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "7P9EG0KOqvz2" + }, + "source": [ + "## Text File Manipulation\n", + "\n", + "As explained in lecture, Unix stores most information in text files. For example, the list of all users and their home directories are stored in \"/etc/passwd\". Let get some familiarity with the most commonly used commands to manipulate files.\n", + "\n", + " - You can see the contents contents a file using the \"cat\" (concatenate) command. Try executing \"cat /etc/passwd\". You'll get a huge list that will go by your screen quickly. \n", + " \n", + " - To go through the file page by page, you can use the \"less\" or \"more\" commands. \n", + " \n", + " - You can see the first or last N (N=10 by default) lines of a file using \"head\" or \"tail\" commands. For example \"tail -20 /etc/passwd\" will list the last 20 lines. \n", + " \n", + " - You can search a test file using the \"grep\" command, which takes a string keyword as the first argument and a filename as the second, and by default prints out every line in the file that contrains the string. So for example you can do \"grep \\$USER /etc/passwd\" to find the line corresponding to your account. Some useful flags: \n", + " \n", + " - \"-i\" ignores the case of the keyword\n", + " - \"-v\" display those lines that do NOT match \n", + " - \"-n\" precede each matching line with the line number \n", + " - \"-c\" print only the total count of matched lines \n", + " \n", + " For example \"grep -c \\$USER /etc/passwd\" should show that you are in the password file just once. \n", + " \n", + " - The \"wc\" (word count) command counts the number of lines, words, and characters in a file. By default \"wc\" gives you all three numbers, but \"-w\", \"-l\", or \"-c\" flags \n", + "\n", + "*Exercise 4:* Count how many lines in the password file contain the letter \"w\". " + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "UlsANMuf2qMs", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 901 + }, + "outputId": "30edb85c-3ba2-4603-92fb-dc60d3ba42ec" + }, + "source": [ + "!/bin/bash --noediting" + ], + "execution_count": 3, + "outputs": [ + { + "output_type": "stream", + "text": [ + "bash: cannot set terminal process group (1397): Inappropriate ioctl for device\n", + "bash: no job control in this shell\n", + "\u001b]0;root@76e673cd4ac7: /content/Data-1401-Repo\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/Data-1401-Repo\u001b[00m# cat /etc/passwd\n", + "root:x:0:0:root:/root:/bin/bash\n", + "daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin\n", + "bin:x:2:2:bin:/bin:/usr/sbin/nologin\n", + "sys:x:3:3:sys:/dev:/usr/sbin/nologin\n", + "sync:x:4:65534:sync:/bin:/bin/sync\n", + "games:x:5:60:games:/usr/games:/usr/sbin/nologin\n", + "man:x:6:12:man:/var/cache/man:/usr/sbin/nologin\n", + "lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin\n", + "mail:x:8:8:mail:/var/mail:/usr/sbin/nologin\n", + "news:x:9:9:news:/var/spool/news:/usr/sbin/nologin\n", + "uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin\n", + "proxy:x:13:13:proxy:/bin:/usr/sbin/nologin\n", + "www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin\n", + "backup:x:34:34:backup:/var/backups:/usr/sbin/nologin\n", + "list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin\n", + "irc:x:39:39:ircd:/var/run/ircd:/usr/sbin/nologin\n", + "gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin\n", + "nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin\n", + "_apt:x:100:65534::/nonexistent:/usr/sbin/nologin\n", + "systemd-network:x:101:104:systemd Network Management,,,:/run/systemd/netif:/usr/sbin/nologin\n", + "systemd-resolve:x:102:105:systemd Resolver,,,:/run/systemd/resolve:/usr/sbin/nologin\n", + "nvidia-persistenced:x:103:107:NVIDIA Persistence Daemon,,,:/nonexistent:/usr/sbin/nologin\n", + "messagebus:x:104:108::/nonexistent:/usr/sbin/nologin\n", + "\u001b]0;root@76e673cd4ac7: /content/Data-1401-Repo\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/Data-1401-Repo\u001b[00m# tail -20 /etc/passwd\n", + "sys:x:3:3:sys:/dev:/usr/sbin/nologin\n", + "sync:x:4:65534:sync:/bin:/bin/sync\n", + "games:x:5:60:games:/usr/games:/usr/sbin/nologin\n", + "man:x:6:12:man:/var/cache/man:/usr/sbin/nologin\n", + "lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin\n", + "mail:x:8:8:mail:/var/mail:/usr/sbin/nologin\n", + "news:x:9:9:news:/var/spool/news:/usr/sbin/nologin\n", + "uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin\n", + "proxy:x:13:13:proxy:/bin:/usr/sbin/nologin\n", + "www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin\n", + "backup:x:34:34:backup:/var/backups:/usr/sbin/nologin\n", + "list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin\n", + "irc:x:39:39:ircd:/var/run/ircd:/usr/sbin/nologin\n", + "gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin\n", + "nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin\n", + "_apt:x:100:65534::/nonexistent:/usr/sbin/nologin\n", + "systemd-network:x:101:104:systemd Network Management,,,:/run/systemd/netif:/usr/sbin/nologin\n", + "systemd-resolve:x:102:105:systemd Resolver,,,:/run/systemd/resolve:/usr/sbin/nologin\n", + "nvidia-persistenced:x:103:107:NVIDIA Persistence Daemon,,,:/nonexistent:/usr/sbin/nologin\n", + "messagebus:x:104:108::/nonexistent:/usr/sbin/nologin\n", + "\u001b]0;root@76e673cd4ac7: /content/Data-1401-Repo\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/Data-1401-Repo\u001b[00m# grep -c $USER /etc/passwd\n", + "exi\n", + "\n", + "\u001b]0;root@76e673cd4ac7: /content/Data-1401-Repo\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/Data-1401-Repo\u001b[00m# \n", + "\u001b]0;root@76e673cd4ac7: /content/Data-1401-Repo\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/Data-1401-Repo\u001b[00m# ^C\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "SZuhLbD8qvz5" + }, + "source": [ + "## Redirection\n", + "\n", + "Unix provides programs \"pipes\" for input and output. Most of what you see on the screen when you run a program was written to the \"stdout\" (standard output) pipe. Other pipes are \"stdin\" (standard input) and \"stderr\" (standard error), where error messages are written.\n", + "\n", + "As discussed in lecture, the basic commands of are simple, but you can chain them to do complicated things. Redirection is how you chain these commands, directing the output of one command to the input of the next.\n", + "\n", + "As an example, consider the \"cat\" command. Cat takes stdin and outputs it to stdout. Type \"cat\" and press enter and confirm. You can get back to the command prompt by pressing \"control-c\" (sends terminate singal) or \"control-d\" (end of file character). Note that from now on we will use the convention: \"control-d\" = \"^D\"\n", + "\n", + "*Exercise 5a:* Using \"cat\" and indirection you can write things into a file. The \">\" symbol directs stdout into a file. Try \"cat > favorite-colors-list.txt\" and then type in your 3 favorite colors, each on it's own line. Use \"^D\" to end your input." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "H5vxtcXnqvz6" + }, + "source": [ + "Use \"cat\", \"more\", or \"less\" to confirm that you file is as you expect it. \">>\" allows you to append to the file. \n", + "\n", + "*Exercise 5b:* Append 2 more colors to your file." + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "twRKNaGy3XGw", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 255 + }, + "outputId": "8eba1f9a-7a49-4321-eeed-20c6c4fffc89" + }, + "source": [ + "!/bin/bash --noediting" + ], + "execution_count": 1, + "outputs": [ + { + "output_type": "stream", + "text": [ + "cat > favorite-colors-list.txt\n", + "bash: cannot set terminal process group (1397): Inappropriate ioctl for device\n", + "bash: no job control in this shell\n", + "\u001b]0;root@76e673cd4ac7: /content\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content\u001b[00m# cat > favorite-colors-list.txt\n", + "blue \n", + "white \n", + "black\n", + "\n", + "\u001b]0;root@76e673cd4ac7: /content\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content\u001b[00m# \n", + "\u001b]0;root@76e673cd4ac7: /content\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content\u001b[00m# cat >> favorite-colors-list.txt\n", + "red \n", + "purple\n", + "\n", + "\u001b]0;root@76e673cd4ac7: /content\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content\u001b[00m# ^C\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "DZODNKiAqvz8" + }, + "source": [ + "The \"sort\" command sorts what it sees on stdin. Instead of taking input from the terminal, you can direct the shell to take stdin from a file using \"<\". Try \"sort < favorite-color-list.txt\" and \"sort < favorite-color-list.txt > sorted-favorite-color-list.txt\".\n", + "\n", + "Finally, instead of piping input / output into files, you can directly chain one program into another using \"|\". So for example, you can do \"cat /etc/passwd | grep -i \\$USER | wc -l\". \n", + "\n", + "*Exercise 5c:* Use indirection to count the number of users on TACC with your first name. Copy the command you used into box below." + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "oP9XlZl_3iZD", + "colab": {} + }, + "source": [ + "!/bin/bash --noediting" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "v5IaZXNyqvz_" + }, + "source": [ + "## Git\n", + "\n", + "`git` is a Version Control System (VCS), typically used to organize the source code of software project but also good source of documents or web-pages. An instance of `git` server stores repositories, each typically containing the code relevant to a specific project. Users create local `clones` of repositories, change and develop the local copies of the code, `commit` the changes to their local repository, `push` to the server as a contribution, \n", + "`pull` updates from the server, and `merge` changes between local and remote versions. \n", + "\n", + "Besides cloning, repositories can be branched or forked. A repository generally starts with a `master` branch that evolves as push requests are merged in. Creating a new branch from an existing branch creates a snapshot of the which can evolve independently or be merged in later. Branches are easy to make and delete, and can serve various purposes. They can represent a stable version of software package. Or a parallel development for different operating system. A fork of a repository is a standalone instance of the repository which can be stored and managed independently from the original, where you can work independently without constraints or interference. \n", + "\n", + "[GitHub](github.com) provides a massive publically accessible instance of a `git` system besides sharing code, projects can be developed by the open source community. It provides tools for managing your repository and a wiki for documentation. Contributions to public software on GitHub generally require making a merge request which would be judged by the managers of the repository. That's why most software packages enourage you to create a new fork, so you can work independently.\n", + "\n", + "Lets take a look at some repositories:\n", + "\n", + "* [This class](https://github.com/UTA-DataScience/DATA1401.2020.Fall)\n", + "\n", + "\n", + "Read through [Version Control System Lecture](https://github.com/UTA-DataScience/VersionControlSystem/blob/master/VersionControlSystem/README.md) to better understand `git`. Note that you do not need to do the exercises at the end of the lecture.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "J_R64sQDqv0A" + }, + "source": [ + "## Plan\n", + "\n", + "You made a clone of the class repository at start of this lab. We will create a new fork where you can keep track and submit your work, following [these instructions](https://help.github.com/articles/fork-a-repo/).\n", + "\n", + "Goto to github.com and log in.\n", + "\n", + "Next, lets create a fork of the [class repository](https://github.com/afarbin/DATA1401-Spring-2019). Click the link and press the \"Fork\" button on the top right. Select your repository as where you want to place the fork.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "edTvE6rOqv0C" + }, + "source": [ + "Now we will check out your fork in your Google Drive / Colab.\n", + "\n", + "Note: Jupyter allows you to run shell directly in a notebook. We will use `!` and `%` to call shell commands directly in this notebook. Follow along yourself. Either create a new notebook or open a terminal. \n", + "\n", + "Start by listing the contents of your current directory." + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "e5tXg0f8qv0D", + "colab": {} + }, + "source": [ + "!ls" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "WYsyYcg1qv0J" + }, + "source": [ + "Make a new directory:" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "Z7noY1hMqv0L", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 51 + }, + "outputId": "135a2b59-7862-44cf-cc98-f1661ee79f5c" + }, + "source": [ + "!mkdir Data-1401-Repo\n", + "%cd Data-1401-Repo" + ], + "execution_count": 2, + "outputs": [ + { + "output_type": "stream", + "text": [ + "mkdir: cannot create directory ‘Data-1401-Repo’: File exists\n", + "/content/Data-1401-Repo\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "fwsBdTnYqv0Q" + }, + "source": [ + "From the github page for your fork, press the green \"Clone or download\" button and copy the URL.\n", + "\n", + "Goto to your notebook and use the following command to clone the repository, pasting the URL you just copied:\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "8w42MH6Jqv0S", + "colab": {} + }, + "source": [ + "# What you past here should look like:\n", + "#!git clone https://github.com//DATA1401.2020.Fall.git" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "cOAuqTVUqv0V" + }, + "source": [ + "Go into the directory:" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "b1Ew4tEZqv0X", + "colab": {} + }, + "source": [ + "%cd DATA1401.2020.Fall\n", + "!ls" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "IrhWToc-qv0a" + }, + "source": [ + "We will now connect your fork to the original so you can pull changes from there. \n", + "\n", + "Check remote status:" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "JxtMYR-9qv0c", + "colab": {} + }, + "source": [ + "!git remote -v" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "9ud3X0fBqv0f" + }, + "source": [ + "Now use the original class URL to set your upstream:" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "pgJlKxBqqv0h", + "colab": {} + }, + "source": [ + "!git remote add upstream https://github.com/UTA-DataScience/DATA1401.2020.Fall" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "id2yUEt9qv0k", + "colab": {} + }, + "source": [ + "!git remote -v" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "sAkgeJ6Iqv0n" + }, + "source": [ + "From now on, you can get the newest version of class material by using:" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "AGDsfTFLqv0o", + "colab": {} + }, + "source": [ + "!git pull" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "u9RAhs5b4vXY" + }, + "source": [ + "We will submit your Lab 1 using git at the next Lab." + ] + } + ] +} \ No newline at end of file From 923c93e9b901b66efa3ee9b8126c0010be2a845c Mon Sep 17 00:00:00 2001 From: thangb44 <71145730+thangb44@users.noreply.github.com> Date: Fri, 11 Sep 2020 15:15:26 -0500 Subject: [PATCH 02/17] Delete Lab-1-Solutions.ipynb --- Labs/Lab.1/Lab-1-Solutions.ipynb | 1022 ------------------------------ 1 file changed, 1022 deletions(-) delete mode 100644 Labs/Lab.1/Lab-1-Solutions.ipynb diff --git a/Labs/Lab.1/Lab-1-Solutions.ipynb b/Labs/Lab.1/Lab-1-Solutions.ipynb deleted file mode 100644 index 26f17d6..0000000 --- a/Labs/Lab.1/Lab-1-Solutions.ipynb +++ /dev/null @@ -1,1022 +0,0 @@ -{ - "nbformat": 4, - "nbformat_minor": 0, - "metadata": { - "colab": { - "name": "Lab-1.ipynb", - "provenance": [], - "collapsed_sections": [] - }, - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.8.0" - } - }, - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "O5vg8KKRq0sy" - }, - "source": [ - "# Lab 1\n", - "\n", - "## Python Notebooks on Google Colab\n", - "\n", - "Data 1401's Labs, Homework, and Exams will be all in form of iPython notebooks. You may already be familiar with python notebooks if you have used Jupyter before. If so, you are welcome to use whatever means you have to run Jupyter notebooks for this course, though you may get limited support. Our primary means of running python notebooks will be through [Google Colab](https://colab.research.google.com). You may also choose to store your files on google drive.\n", - "\n", - "You will need a google account. If you do not have one or you wish to use a different account for this course, please follow [these instructions](https://edu.gcfglobal.org/en/googledriveanddocs/getting-started-with-google-drive/1/) to make an account.\n", - "\n", - "Once you are ready with your account, you can continue in Colab. Click on the following badge to open this notebook in Colab:\n", - "\n", - "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/UTA-DataScience/DATA1401.2020.Fall/blob/master/Labs/Lab.1/Lab.1.ipynb)\n" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "oJ16BYj1JF89", - "colab_type": "code", - "colab": {} - }, - "source": [ - "echo" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "FVt_1hPt1dAK" - }, - "source": [ - "## Notebooks in Colab\n", - "\n", - "You now are presumably in Colab. Word of caution, by default, Google Colab does not save your notebooks, so if you close your session, you will loose your work.\n", - "\n", - "So first thing: from the file menu above select \"Save a copy in Drive\"." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "x0JBL_RFrDDj" - }, - "source": [ - "## Storing Notebooks in Google Drive (Optional)\n", - "If you like, you may save your notebooks directly into Google Drive and upload directly to Git (where you will be downloading and uploading your homework). In order properly setup Git, we'll need to work more directly in your Google Drive.\n", - "\n", - "On the left sidebar, press the file icon to see a listing of files accessibile to this Notebook. Then press \"Mount Drive\" and follow the instructions to mount your Google Drive in this notebook. A new cell will be inserted into this notebook, which after you run by pressing the play button will instruct you to follow a link to log into your Google Account and enable access to your Drive in another tab. Finally you will copy a link from the new tab back into the cell in this notebook. Once you are done, press refresh under files in the left sidebar and you should have \"drive/My Drive\" appear." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "hwJ6wJk3tiLv" - }, - "source": [ - "## Github\n", - "All the class material will be stored on github. You will also submit your homework using github. To do so, you will need a github account.\n", - "\n", - "If you do not already have a github account or wish to create a new one for this course, create one:\n", - "* Browse to [github.com](https://github.com).\n", - "* Click the green “Sign up for GitHub”\tbutton.\n", - "* Follow instructions for creating an account.\n", - "* Make sure you remember your github username and password.\n", - "\n", - "Write an email to the course Instructor titled \"Data 1401: Github account\" with your github username (not your password) as the contents.\n", - "\n", - "## Google Groups\n", - "\n", - "Class annoucements will be made via google groups. If you did not already receive an invite to the class google group, had trouble with the invite, or wish to use a different email address, write an email to the course TA titled \"Data 1401: Google Group\" with your preferred email.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "TjfIzdQZqvzk" - }, - "source": [ - "## Introduction: Unix, Git, and Jupyter\n", - "\n", - "This lab aims to introduce you to basic Unix, familiarize you with iPython notebooks and get you setup to submit your homework.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "C_LmOgzFqvzp" - }, - "source": [ - "\n", - "\n", - "### Terminal, Shell, and ssh\n", - "\n", - "\n", - "The terminal is a simple program that generally runs another program, taking mostly keyboard input from you, passing it to this other program, and taking the output of the program and displaying on the screen for you.\n", - "\n", - "The terminal usually runs a program called a shell. Shells present a command prompt where you can type in commands, which are then executed when you press enter. In most shells, there are some special commands which the shell will execute. Everything else you type in, the shell will assume is a name of a program you want to run and arguments you want to pass that program. So if the shell doesn't recognize something you type in, it'll try to find a program with a name that is the same as the first word you gave it. \n", - "\n", - "### Shell in Colab\n", - "\n", - "Unfortunately, google Colab does not allow you to open a terminal window. Jupyter does, so if you are running in Jupyter (which most of you will not be), you may choose to open a terminal window by returning to the jupyter file list tab and selecting new terminal from the top right.\n", - "\n", - "For Colab, we will have to do something non-ideal, but functional. There are several ways to execute shell commands from within a python notebook. For example, you can use any shell command by putting \"!\" in front of the command:\n", - "\n", - "\n", - "\n" - ] - }, - { - "cell_type": "code", - "metadata": { - "colab_type": "code", - "id": "KJ5f-WO0wcAv", - "colab": {} - }, - "source": [ - "!ls\n", - "!echo \"----------\"\n", - "!ls sample_data" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "8f-n4AXFw-dD" - }, - "source": [ - "Unfortunately, every time you use \"!\" a new environment is created and the state reverted to the original state. Try to understand the difference between the following two sets of commands:\n" - ] - }, - { - "cell_type": "code", - "metadata": { - "colab_type": "code", - "id": "99nrBYTWxZJr", - "colab": {} - }, - "source": [ - "!echo \"Technique 1:\"\n", - "!ls\n", - "!cd sample_data\n", - "!ls" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "colab_type": "code", - "id": "2-Znf97Lxl-Z", - "colab": {} - }, - "source": [ - "!echo \"Technique 2:\"\n", - "!ls ; cd sample_data ;ls" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "4x9n1rAkxyYl" - }, - "source": [ - "Notebooks allow a bit of \"magic\" (using \"%\") to avoid some of these limitations:\n" - ] - }, - { - "cell_type": "code", - "metadata": { - "colab_type": "code", - "id": "vLBPTX4rx3gd", - "colab": {} - }, - "source": [ - "!echo \"Technique 3:\"\n", - "!ls \n", - "%cd sample_data \n", - "!ls" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "U8XpvPjcyH0w" - }, - "source": [ - "For our purposes, we are just going to explicitly start a new shell and interact with it in the output cell. Execute the following cell. You will be able to type and execute commands. Look around a bit using \"ls\" and \"cd. You can stop the cell from running by typing \"exit\". You may also stop the shell by pressing play/stop button." - ] - }, - { - "cell_type": "code", - "metadata": { - "colab_type": "code", - "id": "MIDFitLZyuZy", - "colab": {} - }, - "source": [ - "!/bin/bash --noediting" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "q-4hfZBywW25" - }, - "source": [ - "While in this instance your shell is running in a this notebook, you can also run terminals natively on your own computer. On Linux or MacOS, you just have to run a program called terminal. In Windows you can start a \"command prompt\". \n", - "\n", - "Type in \"ls\" into the terminal above and press enter. The shell will find a program called \"ls\", a standard tool in Unix, and run it. \"ls\" lists the contents (files and directories) of your current directory. \n", - "\n", - "For your reference, [Unix/Linux Command Reference](https://github.com/UTA-DataScience/VersionControlSystem/blob/master/VersionControlSystem/linuxRef.pdf) summarizes some of the common shell commands.\n", - "\n", - "A subtle point to realize here is that while the terminal is running in the browser that is running on the computer in front of you, the shell is actually running on a machine on google hardware. The shell prompt typically displays the name of the machine you are using. What you are not seeing is that there is an intermidate program between the terminal running on your computer and the shell running on google. This intermidary program is taking your input from the terminal sending it over the network to google and bringing back the responses for you terminal to display.\n", - "\n", - "A bit of extra information. If you start a terminal on your own computer, the shell runs locally. The \"ls\" command would then list contents of a directory on your computer. You can typically connect to Unix computers by evoking a shell running on that machine over the network. In this case, you would have to initiate this intermidiary program yourself. The program is called \"ssh\" (secure shell). You can \"ssh\" to another machine from your machine, by simply typing \"ssh\" followed by the machine name or IP address. Most likely you would be prompted for a password, after which you would dropped into the prompt of a shell running on the remote machine. \n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "51Eya4LBqvzs" - }, - "source": [ - "## Programs and Environment Variables\n", - "\n", - "You have a listing of your current directory, but you don't know where that directory resides. You can see what directory you are using the command \"pwd\" (print working directory). Issue the command and look at the response. You'll get a slash (\"/\") separated list, known as the path, of the directory hierarchy of your current working directory. On Colab, this will start with \"contents\"\n", - "\n", - "Now back to thinking about the command prompt. Since \"ls\" is a program, it most be stored somewhere. It is clearly not in your working directory, because you didn't see it when you executed \"ls\". We can ask the shell to tell us where it found \"ls\" using the \"which ls\" command. Note that \"which\" is also a program. \"which ls\" comes back with \"/bin/ls\", telling you the \"ls\" program is sitting in \"/bin\" directory of the system. \n", - "\n" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "RBKYfBowJF9t", - "colab_type": "code", - "colab": {} - }, - "source": [ - "!which ls" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "A57gH8W6JF9y", - "colab_type": "text" - }, - "source": [ - "Lets see what else is in there by issuing a \"ls /bin\" command. " - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "JtfbjvYmJF9z", - "colab_type": "code", - "colab": {} - }, - "source": [ - "!ls /bin" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "YnsTweeGJF94", - "colab_type": "text" - }, - "source": [ - "You will get a long list of programs. You can run any of these programs by just typing their names and pressing enter. You may be able to guess what some of these programs do, but if you want to know, most of them provide you help, using \"--help\" or \"-h\" flag. For example execute \"ls --help\". Quick note that MacOS is a bit different and so \"--help\" doesn't seem to work." - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "1ppZwN9gJF96", - "colab_type": "code", - "colab": {} - }, - "source": [ - "!ls --help" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "l6s9aSprJF9-", - "colab_type": "text" - }, - "source": [ - " For more information about a program or command, you can use Unix's manual pages using the \"man\" command. Try typing \"man ls\". Note that you will need to press space to scroll through lengthy manual pages and \"q\" to exit back to the shell prompt. " - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "qEuDCPvoJF9_", - "colab_type": "code", - "colab": {} - }, - "source": [ - "!man ls" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "VAJaFtqhJF-D", - "colab_type": "text" - }, - "source": [ - "Another command interesting is \"echo\". \"echo\" simply prints whatever you put after it to the screen. " - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "UfS0lC3IJF-D", - "colab_type": "code", - "colab": {} - }, - "source": [ - "!echo \"Hello World\"" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "1VugUXikJF-J", - "colab_type": "text" - }, - "source": [ - "At this point, you may wonder how was it that the shell knew to look for programs in \"/bin\"? The shell keeps a list of places to look for programs an environment variable with the name \"PATH\". The shell keeps a table that map string variable names to string expressions. When the shell starts, its configuration files set some environment variables that it uses. You can see the full list of defined environment variables using the command \"printenv\".\n", - "\n", - "You can use a environment variable in a shell by prepending name of the variable with a dollar sign character. So you can print out the PATH environment variable using the command \"echo $PATH\". What you will see is a colon (\":\") separated list of directories that the shell will search (in order) whenever you type in anything." - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "-hqGpF4UJF-K", - "colab_type": "code", - "colab": {} - }, - "source": [ - "!echo $PATH" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "0ClBrBR8JF-P", - "colab_type": "text" - }, - "source": [ - "You can set you own environment variables. Different shells have different syntax. Lets first figure out what shell we are running. \n", - "\n", - "*Exercise 1:* Use the \"echo\" command to print out the value of the \"SHELL\" environment variable:" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "YS7YFiPwqvzu" - }, - "source": [ - "!/bin/bash --noediting" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "zplxWxIbMNkG", - "colab_type": "code", - "colab": { - "base_uri": "https://localhost:8080/", - "height": 34 - }, - "outputId": "7dc0a807-2cb5-4d9b-8cdd-50781fd382fb" - }, - "source": [ - "!echo SHELL" - ], - "execution_count": 2, - "outputs": [ - { - "output_type": "stream", - "text": [ - "SHELL\n" - ], - "name": "stdout" - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "YoEgruUhqvzw" - }, - "source": [ - "## Navigating Directories\n", - "\n", - "You can change your current directory using the \"cd\" shell command. Note that \"cd\" is not a Unix program. Once in a directory, you can use the \"ls\" command to list the contents or \"pwd\" to remind yourself your current working directory. You can move back one level in your current directory hierarchy using \"cd ..\". In general \"..\" represents the path to a directory one level above your current directory, \"../..\" represents two levels up, and so on. \".\" represents the current directory. If you look at the PATH environment variable, you'll notice that the last item is \".\", telling the shell to look into your current directory for commands. Finally the \"~\" character always refers to your home directory.\n", - "\n", - "Some other file manipulation commands:\n", - "\n", - " - The \"mkdir\" command creates new directories. \n", - " - \"cp\" and \"mv\" allow you to copy and move (or rename) files, taking 2 arguments: the original path/filename and the target path/filename. \n", - " - The \"rm\" and \"rmdir\" commands remove (delete) files and directories.\n", - "\n", - "\n", - "*Exercise 2:* Create a new directory called \"Data-1441\", and another directory inside \"Data-1441\" called \"Lab-1-Solutions\". Perform the rest of the lab in this directory." - ] - }, - { - "cell_type": "code", - "metadata": { - "colab_type": "code", - "id": "A16VzZ3G0J8x", - "colab": { - "base_uri": "https://localhost:8080/", - "height": 238 - }, - "outputId": "4755c271-b0b5-4acc-dce9-390e4d57e135" - }, - "source": [ - "!/bin/bash --noediting" - ], - "execution_count": 7, - "outputs": [ - { - "output_type": "stream", - "text": [ - "bash: cannot set terminal process group (1091): Inappropriate ioctl for device\n", - "bash: no job control in this shell\n", - "\u001b]0;root@76e673cd4ac7: /content/sample_data\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/sample_data\u001b[00m# ls\n", - "\u001b[0m\u001b[01;32manscombe.json\u001b[0m \u001b[01;34mData-1441\u001b[0m \u001b[01;32mREADME.md\u001b[0m\n", - "california_housing_test.csv mnist_test.csv\n", - "california_housing_train.csv mnist_train_small.csv\n", - "\u001b]0;root@76e673cd4ac7: /content/sample_data\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/sample_data\u001b[00m# cd Data-1441\n", - "\u001b]0;root@76e673cd4ac7: /content/sample_data/Data-1441\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/sample_data/Data-1441\u001b[00m# ls\n", - "\u001b[0m\u001b[01;34mLab-1-Solutions\u001b[0m\n", - "\u001b]0;root@76e673cd4ac7: /content/sample_data/Data-1441\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/sample_data/Data-1441\u001b[00m# cd Lab-1-Solutions\n", - "\u001b]0;root@76e673cd4ac7: /content/sample_data/Data-1441/Lab-1-Solutions\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/sample_data/Data-1441/Lab-1-Solutions\u001b[00m# ls\n", - "\u001b]0;root@76e673cd4ac7: /content/sample_data/Data-1441/Lab-1-Solutions\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/sample_data/Data-1441/Lab-1-Solutions\u001b[00m# exit\n", - "exit\n" - ], - "name": "stdout" - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "qKQynO67MaQv", - "colab_type": "text" - }, - "source": [ - "mkdir Data-1441\n", - "cd Data-1441\n", - "mkdir Lab-1-Solutions" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "o38c4lbsqvzy" - }, - "source": [ - "## Exploring Unix Filesystem\n", - "\n", - "You can look at the root directory of the system by issuing \"ls /\". As explained in lecture, Unix uses the file system to communicate with devices and between processes. \"/etc\" keeps the configuration files of the system. \"/bin\" and \"/sbin\" store most of the standard Unix programs. \"/usr\" stores installed programs and their associate files, with \"/usr/bin\" usually storing the commands you can run. \n", - "\n", - "*Exercise 3:* List the \"/dev\" directory. How many SSD storage devices do you see? How many partitions does each device have? (Answer in box below)" - ] - }, - { - "cell_type": "code", - "metadata": { - "colab_type": "code", - "id": "yNj2LXzP2ksl", - "colab": {} - }, - "source": [ - "!/bin/bash --noediting" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "UC1Hgrt3MxMf", - "colab_type": "text" - }, - "source": [ - "0" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "7P9EG0KOqvz2" - }, - "source": [ - "## Text File Manipulation\n", - "\n", - "As explained in lecture, Unix stores most information in text files. For example, the list of all users and their home directories are stored in \"/etc/passwd\". Let get some familiarity with the most commonly used commands to manipulate files.\n", - "\n", - " - You can see the contents contents a file using the \"cat\" (concatenate) command. Try executing \"cat /etc/passwd\". You'll get a huge list that will go by your screen quickly. \n", - " \n", - " - To go through the file page by page, you can use the \"less\" or \"more\" commands. \n", - " \n", - " - You can see the first or last N (N=10 by default) lines of a file using \"head\" or \"tail\" commands. For example \"tail -20 /etc/passwd\" will list the last 20 lines. \n", - " \n", - " - You can search a test file using the \"grep\" command, which takes a string keyword as the first argument and a filename as the second, and by default prints out every line in the file that contrains the string. So for example you can do \"grep \\$USER /etc/passwd\" to find the line corresponding to your account. Some useful flags: \n", - " \n", - " - \"-i\" ignores the case of the keyword\n", - " - \"-v\" display those lines that do NOT match \n", - " - \"-n\" precede each matching line with the line number \n", - " - \"-c\" print only the total count of matched lines \n", - " \n", - " For example \"grep -c \\$USER /etc/passwd\" should show that you are in the password file just once. \n", - " \n", - " - The \"wc\" (word count) command counts the number of lines, words, and characters in a file. By default \"wc\" gives you all three numbers, but \"-w\", \"-l\", or \"-c\" flags \n", - "\n", - "*Exercise 4:* Count how many lines in the password file contain the letter \"w\". " - ] - }, - { - "cell_type": "code", - "metadata": { - "colab_type": "code", - "id": "UlsANMuf2qMs", - "colab": { - "base_uri": "https://localhost:8080/", - "height": 901 - }, - "outputId": "30edb85c-3ba2-4603-92fb-dc60d3ba42ec" - }, - "source": [ - "!/bin/bash --noediting" - ], - "execution_count": 3, - "outputs": [ - { - "output_type": "stream", - "text": [ - "bash: cannot set terminal process group (1397): Inappropriate ioctl for device\n", - "bash: no job control in this shell\n", - "\u001b]0;root@76e673cd4ac7: /content/Data-1401-Repo\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/Data-1401-Repo\u001b[00m# cat /etc/passwd\n", - "root:x:0:0:root:/root:/bin/bash\n", - "daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin\n", - "bin:x:2:2:bin:/bin:/usr/sbin/nologin\n", - "sys:x:3:3:sys:/dev:/usr/sbin/nologin\n", - "sync:x:4:65534:sync:/bin:/bin/sync\n", - "games:x:5:60:games:/usr/games:/usr/sbin/nologin\n", - "man:x:6:12:man:/var/cache/man:/usr/sbin/nologin\n", - "lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin\n", - "mail:x:8:8:mail:/var/mail:/usr/sbin/nologin\n", - "news:x:9:9:news:/var/spool/news:/usr/sbin/nologin\n", - "uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin\n", - "proxy:x:13:13:proxy:/bin:/usr/sbin/nologin\n", - "www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin\n", - "backup:x:34:34:backup:/var/backups:/usr/sbin/nologin\n", - "list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin\n", - "irc:x:39:39:ircd:/var/run/ircd:/usr/sbin/nologin\n", - "gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin\n", - "nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin\n", - "_apt:x:100:65534::/nonexistent:/usr/sbin/nologin\n", - "systemd-network:x:101:104:systemd Network Management,,,:/run/systemd/netif:/usr/sbin/nologin\n", - "systemd-resolve:x:102:105:systemd Resolver,,,:/run/systemd/resolve:/usr/sbin/nologin\n", - "nvidia-persistenced:x:103:107:NVIDIA Persistence Daemon,,,:/nonexistent:/usr/sbin/nologin\n", - "messagebus:x:104:108::/nonexistent:/usr/sbin/nologin\n", - "\u001b]0;root@76e673cd4ac7: /content/Data-1401-Repo\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/Data-1401-Repo\u001b[00m# tail -20 /etc/passwd\n", - "sys:x:3:3:sys:/dev:/usr/sbin/nologin\n", - "sync:x:4:65534:sync:/bin:/bin/sync\n", - "games:x:5:60:games:/usr/games:/usr/sbin/nologin\n", - "man:x:6:12:man:/var/cache/man:/usr/sbin/nologin\n", - "lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin\n", - "mail:x:8:8:mail:/var/mail:/usr/sbin/nologin\n", - "news:x:9:9:news:/var/spool/news:/usr/sbin/nologin\n", - "uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin\n", - "proxy:x:13:13:proxy:/bin:/usr/sbin/nologin\n", - "www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin\n", - "backup:x:34:34:backup:/var/backups:/usr/sbin/nologin\n", - "list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin\n", - "irc:x:39:39:ircd:/var/run/ircd:/usr/sbin/nologin\n", - "gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin\n", - "nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin\n", - "_apt:x:100:65534::/nonexistent:/usr/sbin/nologin\n", - "systemd-network:x:101:104:systemd Network Management,,,:/run/systemd/netif:/usr/sbin/nologin\n", - "systemd-resolve:x:102:105:systemd Resolver,,,:/run/systemd/resolve:/usr/sbin/nologin\n", - "nvidia-persistenced:x:103:107:NVIDIA Persistence Daemon,,,:/nonexistent:/usr/sbin/nologin\n", - "messagebus:x:104:108::/nonexistent:/usr/sbin/nologin\n", - "\u001b]0;root@76e673cd4ac7: /content/Data-1401-Repo\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/Data-1401-Repo\u001b[00m# grep -c $USER /etc/passwd\n", - "exi\n", - "\n", - "\u001b]0;root@76e673cd4ac7: /content/Data-1401-Repo\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/Data-1401-Repo\u001b[00m# \n", - "\u001b]0;root@76e673cd4ac7: /content/Data-1401-Repo\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/Data-1401-Repo\u001b[00m# ^C\n" - ], - "name": "stdout" - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "SZuhLbD8qvz5" - }, - "source": [ - "## Redirection\n", - "\n", - "Unix provides programs \"pipes\" for input and output. Most of what you see on the screen when you run a program was written to the \"stdout\" (standard output) pipe. Other pipes are \"stdin\" (standard input) and \"stderr\" (standard error), where error messages are written.\n", - "\n", - "As discussed in lecture, the basic commands of are simple, but you can chain them to do complicated things. Redirection is how you chain these commands, directing the output of one command to the input of the next.\n", - "\n", - "As an example, consider the \"cat\" command. Cat takes stdin and outputs it to stdout. Type \"cat\" and press enter and confirm. You can get back to the command prompt by pressing \"control-c\" (sends terminate singal) or \"control-d\" (end of file character). Note that from now on we will use the convention: \"control-d\" = \"^D\"\n", - "\n", - "*Exercise 5a:* Using \"cat\" and indirection you can write things into a file. The \">\" symbol directs stdout into a file. Try \"cat > favorite-colors-list.txt\" and then type in your 3 favorite colors, each on it's own line. Use \"^D\" to end your input." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "H5vxtcXnqvz6" - }, - "source": [ - "Use \"cat\", \"more\", or \"less\" to confirm that you file is as you expect it. \">>\" allows you to append to the file. \n", - "\n", - "*Exercise 5b:* Append 2 more colors to your file." - ] - }, - { - "cell_type": "code", - "metadata": { - "colab_type": "code", - "id": "twRKNaGy3XGw", - "colab": { - "base_uri": "https://localhost:8080/", - "height": 255 - }, - "outputId": "8eba1f9a-7a49-4321-eeed-20c6c4fffc89" - }, - "source": [ - "!/bin/bash --noediting" - ], - "execution_count": 1, - "outputs": [ - { - "output_type": "stream", - "text": [ - "cat > favorite-colors-list.txt\n", - "bash: cannot set terminal process group (1397): Inappropriate ioctl for device\n", - "bash: no job control in this shell\n", - "\u001b]0;root@76e673cd4ac7: /content\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content\u001b[00m# cat > favorite-colors-list.txt\n", - "blue \n", - "white \n", - "black\n", - "\n", - "\u001b]0;root@76e673cd4ac7: /content\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content\u001b[00m# \n", - "\u001b]0;root@76e673cd4ac7: /content\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content\u001b[00m# cat >> favorite-colors-list.txt\n", - "red \n", - "purple\n", - "\n", - "\u001b]0;root@76e673cd4ac7: /content\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content\u001b[00m# ^C\n" - ], - "name": "stdout" - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "DZODNKiAqvz8" - }, - "source": [ - "The \"sort\" command sorts what it sees on stdin. Instead of taking input from the terminal, you can direct the shell to take stdin from a file using \"<\". Try \"sort < favorite-color-list.txt\" and \"sort < favorite-color-list.txt > sorted-favorite-color-list.txt\".\n", - "\n", - "Finally, instead of piping input / output into files, you can directly chain one program into another using \"|\". So for example, you can do \"cat /etc/passwd | grep -i \\$USER | wc -l\". \n", - "\n", - "*Exercise 5c:* Use indirection to count the number of users on TACC with your first name. Copy the command you used into box below." - ] - }, - { - "cell_type": "code", - "metadata": { - "colab_type": "code", - "id": "oP9XlZl_3iZD", - "colab": {} - }, - "source": [ - "!/bin/bash --noediting" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "v5IaZXNyqvz_" - }, - "source": [ - "## Git\n", - "\n", - "`git` is a Version Control System (VCS), typically used to organize the source code of software project but also good source of documents or web-pages. An instance of `git` server stores repositories, each typically containing the code relevant to a specific project. Users create local `clones` of repositories, change and develop the local copies of the code, `commit` the changes to their local repository, `push` to the server as a contribution, \n", - "`pull` updates from the server, and `merge` changes between local and remote versions. \n", - "\n", - "Besides cloning, repositories can be branched or forked. A repository generally starts with a `master` branch that evolves as push requests are merged in. Creating a new branch from an existing branch creates a snapshot of the which can evolve independently or be merged in later. Branches are easy to make and delete, and can serve various purposes. They can represent a stable version of software package. Or a parallel development for different operating system. A fork of a repository is a standalone instance of the repository which can be stored and managed independently from the original, where you can work independently without constraints or interference. \n", - "\n", - "[GitHub](github.com) provides a massive publically accessible instance of a `git` system besides sharing code, projects can be developed by the open source community. It provides tools for managing your repository and a wiki for documentation. Contributions to public software on GitHub generally require making a merge request which would be judged by the managers of the repository. That's why most software packages enourage you to create a new fork, so you can work independently.\n", - "\n", - "Lets take a look at some repositories:\n", - "\n", - "* [This class](https://github.com/UTA-DataScience/DATA1401.2020.Fall)\n", - "\n", - "\n", - "Read through [Version Control System Lecture](https://github.com/UTA-DataScience/VersionControlSystem/blob/master/VersionControlSystem/README.md) to better understand `git`. Note that you do not need to do the exercises at the end of the lecture.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "J_R64sQDqv0A" - }, - "source": [ - "## Plan\n", - "\n", - "You made a clone of the class repository at start of this lab. We will create a new fork where you can keep track and submit your work, following [these instructions](https://help.github.com/articles/fork-a-repo/).\n", - "\n", - "Goto to github.com and log in.\n", - "\n", - "Next, lets create a fork of the [class repository](https://github.com/afarbin/DATA1401-Spring-2019). Click the link and press the \"Fork\" button on the top right. Select your repository as where you want to place the fork.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "edTvE6rOqv0C" - }, - "source": [ - "Now we will check out your fork in your Google Drive / Colab.\n", - "\n", - "Note: Jupyter allows you to run shell directly in a notebook. We will use `!` and `%` to call shell commands directly in this notebook. Follow along yourself. Either create a new notebook or open a terminal. \n", - "\n", - "Start by listing the contents of your current directory." - ] - }, - { - "cell_type": "code", - "metadata": { - "colab_type": "code", - "id": "e5tXg0f8qv0D", - "colab": {} - }, - "source": [ - "!ls" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "WYsyYcg1qv0J" - }, - "source": [ - "Make a new directory:" - ] - }, - { - "cell_type": "code", - "metadata": { - "colab_type": "code", - "id": "Z7noY1hMqv0L", - "colab": { - "base_uri": "https://localhost:8080/", - "height": 51 - }, - "outputId": "135a2b59-7862-44cf-cc98-f1661ee79f5c" - }, - "source": [ - "!mkdir Data-1401-Repo\n", - "%cd Data-1401-Repo" - ], - "execution_count": 2, - "outputs": [ - { - "output_type": "stream", - "text": [ - "mkdir: cannot create directory ‘Data-1401-Repo’: File exists\n", - "/content/Data-1401-Repo\n" - ], - "name": "stdout" - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "fwsBdTnYqv0Q" - }, - "source": [ - "From the github page for your fork, press the green \"Clone or download\" button and copy the URL.\n", - "\n", - "Goto to your notebook and use the following command to clone the repository, pasting the URL you just copied:\n" - ] - }, - { - "cell_type": "code", - "metadata": { - "colab_type": "code", - "id": "8w42MH6Jqv0S", - "colab": {} - }, - "source": [ - "# What you past here should look like:\n", - "#!git clone https://github.com//DATA1401.2020.Fall.git" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "cOAuqTVUqv0V" - }, - "source": [ - "Go into the directory:" - ] - }, - { - "cell_type": "code", - "metadata": { - "colab_type": "code", - "id": "b1Ew4tEZqv0X", - "colab": {} - }, - "source": [ - "%cd DATA1401.2020.Fall\n", - "!ls" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "IrhWToc-qv0a" - }, - "source": [ - "We will now connect your fork to the original so you can pull changes from there. \n", - "\n", - "Check remote status:" - ] - }, - { - "cell_type": "code", - "metadata": { - "colab_type": "code", - "id": "JxtMYR-9qv0c", - "colab": {} - }, - "source": [ - "!git remote -v" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "9ud3X0fBqv0f" - }, - "source": [ - "Now use the original class URL to set your upstream:" - ] - }, - { - "cell_type": "code", - "metadata": { - "colab_type": "code", - "id": "pgJlKxBqqv0h", - "colab": {} - }, - "source": [ - "!git remote add upstream https://github.com/UTA-DataScience/DATA1401.2020.Fall" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "colab_type": "code", - "id": "id2yUEt9qv0k", - "colab": {} - }, - "source": [ - "!git remote -v" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "sAkgeJ6Iqv0n" - }, - "source": [ - "From now on, you can get the newest version of class material by using:" - ] - }, - { - "cell_type": "code", - "metadata": { - "colab_type": "code", - "id": "AGDsfTFLqv0o", - "colab": {} - }, - "source": [ - "!git pull" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "u9RAhs5b4vXY" - }, - "source": [ - "We will submit your Lab 1 using git at the next Lab." - ] - } - ] -} \ No newline at end of file From a94e3f3520f31a75ffb31acb16fa7eeafd765931 Mon Sep 17 00:00:00 2001 From: thangb44 Date: Fri, 11 Sep 2020 15:21:25 -0500 Subject: [PATCH 03/17] Lab 1 Solutions --- Labs/Lab.1/Lab-1-Solutions.ipynb | 1022 ++++++++++++++++++++++++++++++ 1 file changed, 1022 insertions(+) create mode 100644 Labs/Lab.1/Lab-1-Solutions.ipynb diff --git a/Labs/Lab.1/Lab-1-Solutions.ipynb b/Labs/Lab.1/Lab-1-Solutions.ipynb new file mode 100644 index 0000000..26f17d6 --- /dev/null +++ b/Labs/Lab.1/Lab-1-Solutions.ipynb @@ -0,0 +1,1022 @@ +{ + "nbformat": 4, + "nbformat_minor": 0, + "metadata": { + "colab": { + "name": "Lab-1.ipynb", + "provenance": [], + "collapsed_sections": [] + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.0" + } + }, + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "O5vg8KKRq0sy" + }, + "source": [ + "# Lab 1\n", + "\n", + "## Python Notebooks on Google Colab\n", + "\n", + "Data 1401's Labs, Homework, and Exams will be all in form of iPython notebooks. You may already be familiar with python notebooks if you have used Jupyter before. If so, you are welcome to use whatever means you have to run Jupyter notebooks for this course, though you may get limited support. Our primary means of running python notebooks will be through [Google Colab](https://colab.research.google.com). You may also choose to store your files on google drive.\n", + "\n", + "You will need a google account. If you do not have one or you wish to use a different account for this course, please follow [these instructions](https://edu.gcfglobal.org/en/googledriveanddocs/getting-started-with-google-drive/1/) to make an account.\n", + "\n", + "Once you are ready with your account, you can continue in Colab. Click on the following badge to open this notebook in Colab:\n", + "\n", + "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/UTA-DataScience/DATA1401.2020.Fall/blob/master/Labs/Lab.1/Lab.1.ipynb)\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "oJ16BYj1JF89", + "colab_type": "code", + "colab": {} + }, + "source": [ + "echo" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "FVt_1hPt1dAK" + }, + "source": [ + "## Notebooks in Colab\n", + "\n", + "You now are presumably in Colab. Word of caution, by default, Google Colab does not save your notebooks, so if you close your session, you will loose your work.\n", + "\n", + "So first thing: from the file menu above select \"Save a copy in Drive\"." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "x0JBL_RFrDDj" + }, + "source": [ + "## Storing Notebooks in Google Drive (Optional)\n", + "If you like, you may save your notebooks directly into Google Drive and upload directly to Git (where you will be downloading and uploading your homework). In order properly setup Git, we'll need to work more directly in your Google Drive.\n", + "\n", + "On the left sidebar, press the file icon to see a listing of files accessibile to this Notebook. Then press \"Mount Drive\" and follow the instructions to mount your Google Drive in this notebook. A new cell will be inserted into this notebook, which after you run by pressing the play button will instruct you to follow a link to log into your Google Account and enable access to your Drive in another tab. Finally you will copy a link from the new tab back into the cell in this notebook. Once you are done, press refresh under files in the left sidebar and you should have \"drive/My Drive\" appear." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "hwJ6wJk3tiLv" + }, + "source": [ + "## Github\n", + "All the class material will be stored on github. You will also submit your homework using github. To do so, you will need a github account.\n", + "\n", + "If you do not already have a github account or wish to create a new one for this course, create one:\n", + "* Browse to [github.com](https://github.com).\n", + "* Click the green “Sign up for GitHub”\tbutton.\n", + "* Follow instructions for creating an account.\n", + "* Make sure you remember your github username and password.\n", + "\n", + "Write an email to the course Instructor titled \"Data 1401: Github account\" with your github username (not your password) as the contents.\n", + "\n", + "## Google Groups\n", + "\n", + "Class annoucements will be made via google groups. If you did not already receive an invite to the class google group, had trouble with the invite, or wish to use a different email address, write an email to the course TA titled \"Data 1401: Google Group\" with your preferred email.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "TjfIzdQZqvzk" + }, + "source": [ + "## Introduction: Unix, Git, and Jupyter\n", + "\n", + "This lab aims to introduce you to basic Unix, familiarize you with iPython notebooks and get you setup to submit your homework.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "C_LmOgzFqvzp" + }, + "source": [ + "\n", + "\n", + "### Terminal, Shell, and ssh\n", + "\n", + "\n", + "The terminal is a simple program that generally runs another program, taking mostly keyboard input from you, passing it to this other program, and taking the output of the program and displaying on the screen for you.\n", + "\n", + "The terminal usually runs a program called a shell. Shells present a command prompt where you can type in commands, which are then executed when you press enter. In most shells, there are some special commands which the shell will execute. Everything else you type in, the shell will assume is a name of a program you want to run and arguments you want to pass that program. So if the shell doesn't recognize something you type in, it'll try to find a program with a name that is the same as the first word you gave it. \n", + "\n", + "### Shell in Colab\n", + "\n", + "Unfortunately, google Colab does not allow you to open a terminal window. Jupyter does, so if you are running in Jupyter (which most of you will not be), you may choose to open a terminal window by returning to the jupyter file list tab and selecting new terminal from the top right.\n", + "\n", + "For Colab, we will have to do something non-ideal, but functional. There are several ways to execute shell commands from within a python notebook. For example, you can use any shell command by putting \"!\" in front of the command:\n", + "\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "KJ5f-WO0wcAv", + "colab": {} + }, + "source": [ + "!ls\n", + "!echo \"----------\"\n", + "!ls sample_data" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "8f-n4AXFw-dD" + }, + "source": [ + "Unfortunately, every time you use \"!\" a new environment is created and the state reverted to the original state. Try to understand the difference between the following two sets of commands:\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "99nrBYTWxZJr", + "colab": {} + }, + "source": [ + "!echo \"Technique 1:\"\n", + "!ls\n", + "!cd sample_data\n", + "!ls" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "2-Znf97Lxl-Z", + "colab": {} + }, + "source": [ + "!echo \"Technique 2:\"\n", + "!ls ; cd sample_data ;ls" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "4x9n1rAkxyYl" + }, + "source": [ + "Notebooks allow a bit of \"magic\" (using \"%\") to avoid some of these limitations:\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "vLBPTX4rx3gd", + "colab": {} + }, + "source": [ + "!echo \"Technique 3:\"\n", + "!ls \n", + "%cd sample_data \n", + "!ls" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "U8XpvPjcyH0w" + }, + "source": [ + "For our purposes, we are just going to explicitly start a new shell and interact with it in the output cell. Execute the following cell. You will be able to type and execute commands. Look around a bit using \"ls\" and \"cd. You can stop the cell from running by typing \"exit\". You may also stop the shell by pressing play/stop button." + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "MIDFitLZyuZy", + "colab": {} + }, + "source": [ + "!/bin/bash --noediting" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "q-4hfZBywW25" + }, + "source": [ + "While in this instance your shell is running in a this notebook, you can also run terminals natively on your own computer. On Linux or MacOS, you just have to run a program called terminal. In Windows you can start a \"command prompt\". \n", + "\n", + "Type in \"ls\" into the terminal above and press enter. The shell will find a program called \"ls\", a standard tool in Unix, and run it. \"ls\" lists the contents (files and directories) of your current directory. \n", + "\n", + "For your reference, [Unix/Linux Command Reference](https://github.com/UTA-DataScience/VersionControlSystem/blob/master/VersionControlSystem/linuxRef.pdf) summarizes some of the common shell commands.\n", + "\n", + "A subtle point to realize here is that while the terminal is running in the browser that is running on the computer in front of you, the shell is actually running on a machine on google hardware. The shell prompt typically displays the name of the machine you are using. What you are not seeing is that there is an intermidate program between the terminal running on your computer and the shell running on google. This intermidary program is taking your input from the terminal sending it over the network to google and bringing back the responses for you terminal to display.\n", + "\n", + "A bit of extra information. If you start a terminal on your own computer, the shell runs locally. The \"ls\" command would then list contents of a directory on your computer. You can typically connect to Unix computers by evoking a shell running on that machine over the network. In this case, you would have to initiate this intermidiary program yourself. The program is called \"ssh\" (secure shell). You can \"ssh\" to another machine from your machine, by simply typing \"ssh\" followed by the machine name or IP address. Most likely you would be prompted for a password, after which you would dropped into the prompt of a shell running on the remote machine. \n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "51Eya4LBqvzs" + }, + "source": [ + "## Programs and Environment Variables\n", + "\n", + "You have a listing of your current directory, but you don't know where that directory resides. You can see what directory you are using the command \"pwd\" (print working directory). Issue the command and look at the response. You'll get a slash (\"/\") separated list, known as the path, of the directory hierarchy of your current working directory. On Colab, this will start with \"contents\"\n", + "\n", + "Now back to thinking about the command prompt. Since \"ls\" is a program, it most be stored somewhere. It is clearly not in your working directory, because you didn't see it when you executed \"ls\". We can ask the shell to tell us where it found \"ls\" using the \"which ls\" command. Note that \"which\" is also a program. \"which ls\" comes back with \"/bin/ls\", telling you the \"ls\" program is sitting in \"/bin\" directory of the system. \n", + "\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "RBKYfBowJF9t", + "colab_type": "code", + "colab": {} + }, + "source": [ + "!which ls" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "A57gH8W6JF9y", + "colab_type": "text" + }, + "source": [ + "Lets see what else is in there by issuing a \"ls /bin\" command. " + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "JtfbjvYmJF9z", + "colab_type": "code", + "colab": {} + }, + "source": [ + "!ls /bin" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "YnsTweeGJF94", + "colab_type": "text" + }, + "source": [ + "You will get a long list of programs. You can run any of these programs by just typing their names and pressing enter. You may be able to guess what some of these programs do, but if you want to know, most of them provide you help, using \"--help\" or \"-h\" flag. For example execute \"ls --help\". Quick note that MacOS is a bit different and so \"--help\" doesn't seem to work." + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "1ppZwN9gJF96", + "colab_type": "code", + "colab": {} + }, + "source": [ + "!ls --help" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "l6s9aSprJF9-", + "colab_type": "text" + }, + "source": [ + " For more information about a program or command, you can use Unix's manual pages using the \"man\" command. Try typing \"man ls\". Note that you will need to press space to scroll through lengthy manual pages and \"q\" to exit back to the shell prompt. " + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "qEuDCPvoJF9_", + "colab_type": "code", + "colab": {} + }, + "source": [ + "!man ls" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "VAJaFtqhJF-D", + "colab_type": "text" + }, + "source": [ + "Another command interesting is \"echo\". \"echo\" simply prints whatever you put after it to the screen. " + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "UfS0lC3IJF-D", + "colab_type": "code", + "colab": {} + }, + "source": [ + "!echo \"Hello World\"" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "1VugUXikJF-J", + "colab_type": "text" + }, + "source": [ + "At this point, you may wonder how was it that the shell knew to look for programs in \"/bin\"? The shell keeps a list of places to look for programs an environment variable with the name \"PATH\". The shell keeps a table that map string variable names to string expressions. When the shell starts, its configuration files set some environment variables that it uses. You can see the full list of defined environment variables using the command \"printenv\".\n", + "\n", + "You can use a environment variable in a shell by prepending name of the variable with a dollar sign character. So you can print out the PATH environment variable using the command \"echo $PATH\". What you will see is a colon (\":\") separated list of directories that the shell will search (in order) whenever you type in anything." + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "-hqGpF4UJF-K", + "colab_type": "code", + "colab": {} + }, + "source": [ + "!echo $PATH" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "0ClBrBR8JF-P", + "colab_type": "text" + }, + "source": [ + "You can set you own environment variables. Different shells have different syntax. Lets first figure out what shell we are running. \n", + "\n", + "*Exercise 1:* Use the \"echo\" command to print out the value of the \"SHELL\" environment variable:" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "YS7YFiPwqvzu" + }, + "source": [ + "!/bin/bash --noediting" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "zplxWxIbMNkG", + "colab_type": "code", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 34 + }, + "outputId": "7dc0a807-2cb5-4d9b-8cdd-50781fd382fb" + }, + "source": [ + "!echo SHELL" + ], + "execution_count": 2, + "outputs": [ + { + "output_type": "stream", + "text": [ + "SHELL\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "YoEgruUhqvzw" + }, + "source": [ + "## Navigating Directories\n", + "\n", + "You can change your current directory using the \"cd\" shell command. Note that \"cd\" is not a Unix program. Once in a directory, you can use the \"ls\" command to list the contents or \"pwd\" to remind yourself your current working directory. You can move back one level in your current directory hierarchy using \"cd ..\". In general \"..\" represents the path to a directory one level above your current directory, \"../..\" represents two levels up, and so on. \".\" represents the current directory. If you look at the PATH environment variable, you'll notice that the last item is \".\", telling the shell to look into your current directory for commands. Finally the \"~\" character always refers to your home directory.\n", + "\n", + "Some other file manipulation commands:\n", + "\n", + " - The \"mkdir\" command creates new directories. \n", + " - \"cp\" and \"mv\" allow you to copy and move (or rename) files, taking 2 arguments: the original path/filename and the target path/filename. \n", + " - The \"rm\" and \"rmdir\" commands remove (delete) files and directories.\n", + "\n", + "\n", + "*Exercise 2:* Create a new directory called \"Data-1441\", and another directory inside \"Data-1441\" called \"Lab-1-Solutions\". Perform the rest of the lab in this directory." + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "A16VzZ3G0J8x", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 238 + }, + "outputId": "4755c271-b0b5-4acc-dce9-390e4d57e135" + }, + "source": [ + "!/bin/bash --noediting" + ], + "execution_count": 7, + "outputs": [ + { + "output_type": "stream", + "text": [ + "bash: cannot set terminal process group (1091): Inappropriate ioctl for device\n", + "bash: no job control in this shell\n", + "\u001b]0;root@76e673cd4ac7: /content/sample_data\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/sample_data\u001b[00m# ls\n", + "\u001b[0m\u001b[01;32manscombe.json\u001b[0m \u001b[01;34mData-1441\u001b[0m \u001b[01;32mREADME.md\u001b[0m\n", + "california_housing_test.csv mnist_test.csv\n", + "california_housing_train.csv mnist_train_small.csv\n", + "\u001b]0;root@76e673cd4ac7: /content/sample_data\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/sample_data\u001b[00m# cd Data-1441\n", + "\u001b]0;root@76e673cd4ac7: /content/sample_data/Data-1441\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/sample_data/Data-1441\u001b[00m# ls\n", + "\u001b[0m\u001b[01;34mLab-1-Solutions\u001b[0m\n", + "\u001b]0;root@76e673cd4ac7: /content/sample_data/Data-1441\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/sample_data/Data-1441\u001b[00m# cd Lab-1-Solutions\n", + "\u001b]0;root@76e673cd4ac7: /content/sample_data/Data-1441/Lab-1-Solutions\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/sample_data/Data-1441/Lab-1-Solutions\u001b[00m# ls\n", + "\u001b]0;root@76e673cd4ac7: /content/sample_data/Data-1441/Lab-1-Solutions\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/sample_data/Data-1441/Lab-1-Solutions\u001b[00m# exit\n", + "exit\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "qKQynO67MaQv", + "colab_type": "text" + }, + "source": [ + "mkdir Data-1441\n", + "cd Data-1441\n", + "mkdir Lab-1-Solutions" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "o38c4lbsqvzy" + }, + "source": [ + "## Exploring Unix Filesystem\n", + "\n", + "You can look at the root directory of the system by issuing \"ls /\". As explained in lecture, Unix uses the file system to communicate with devices and between processes. \"/etc\" keeps the configuration files of the system. \"/bin\" and \"/sbin\" store most of the standard Unix programs. \"/usr\" stores installed programs and their associate files, with \"/usr/bin\" usually storing the commands you can run. \n", + "\n", + "*Exercise 3:* List the \"/dev\" directory. How many SSD storage devices do you see? How many partitions does each device have? (Answer in box below)" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "yNj2LXzP2ksl", + "colab": {} + }, + "source": [ + "!/bin/bash --noediting" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "UC1Hgrt3MxMf", + "colab_type": "text" + }, + "source": [ + "0" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "7P9EG0KOqvz2" + }, + "source": [ + "## Text File Manipulation\n", + "\n", + "As explained in lecture, Unix stores most information in text files. For example, the list of all users and their home directories are stored in \"/etc/passwd\". Let get some familiarity with the most commonly used commands to manipulate files.\n", + "\n", + " - You can see the contents contents a file using the \"cat\" (concatenate) command. Try executing \"cat /etc/passwd\". You'll get a huge list that will go by your screen quickly. \n", + " \n", + " - To go through the file page by page, you can use the \"less\" or \"more\" commands. \n", + " \n", + " - You can see the first or last N (N=10 by default) lines of a file using \"head\" or \"tail\" commands. For example \"tail -20 /etc/passwd\" will list the last 20 lines. \n", + " \n", + " - You can search a test file using the \"grep\" command, which takes a string keyword as the first argument and a filename as the second, and by default prints out every line in the file that contrains the string. So for example you can do \"grep \\$USER /etc/passwd\" to find the line corresponding to your account. Some useful flags: \n", + " \n", + " - \"-i\" ignores the case of the keyword\n", + " - \"-v\" display those lines that do NOT match \n", + " - \"-n\" precede each matching line with the line number \n", + " - \"-c\" print only the total count of matched lines \n", + " \n", + " For example \"grep -c \\$USER /etc/passwd\" should show that you are in the password file just once. \n", + " \n", + " - The \"wc\" (word count) command counts the number of lines, words, and characters in a file. By default \"wc\" gives you all three numbers, but \"-w\", \"-l\", or \"-c\" flags \n", + "\n", + "*Exercise 4:* Count how many lines in the password file contain the letter \"w\". " + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "UlsANMuf2qMs", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 901 + }, + "outputId": "30edb85c-3ba2-4603-92fb-dc60d3ba42ec" + }, + "source": [ + "!/bin/bash --noediting" + ], + "execution_count": 3, + "outputs": [ + { + "output_type": "stream", + "text": [ + "bash: cannot set terminal process group (1397): Inappropriate ioctl for device\n", + "bash: no job control in this shell\n", + "\u001b]0;root@76e673cd4ac7: /content/Data-1401-Repo\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/Data-1401-Repo\u001b[00m# cat /etc/passwd\n", + "root:x:0:0:root:/root:/bin/bash\n", + "daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin\n", + "bin:x:2:2:bin:/bin:/usr/sbin/nologin\n", + "sys:x:3:3:sys:/dev:/usr/sbin/nologin\n", + "sync:x:4:65534:sync:/bin:/bin/sync\n", + "games:x:5:60:games:/usr/games:/usr/sbin/nologin\n", + "man:x:6:12:man:/var/cache/man:/usr/sbin/nologin\n", + "lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin\n", + "mail:x:8:8:mail:/var/mail:/usr/sbin/nologin\n", + "news:x:9:9:news:/var/spool/news:/usr/sbin/nologin\n", + "uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin\n", + "proxy:x:13:13:proxy:/bin:/usr/sbin/nologin\n", + "www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin\n", + "backup:x:34:34:backup:/var/backups:/usr/sbin/nologin\n", + "list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin\n", + "irc:x:39:39:ircd:/var/run/ircd:/usr/sbin/nologin\n", + "gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin\n", + "nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin\n", + "_apt:x:100:65534::/nonexistent:/usr/sbin/nologin\n", + "systemd-network:x:101:104:systemd Network Management,,,:/run/systemd/netif:/usr/sbin/nologin\n", + "systemd-resolve:x:102:105:systemd Resolver,,,:/run/systemd/resolve:/usr/sbin/nologin\n", + "nvidia-persistenced:x:103:107:NVIDIA Persistence Daemon,,,:/nonexistent:/usr/sbin/nologin\n", + "messagebus:x:104:108::/nonexistent:/usr/sbin/nologin\n", + "\u001b]0;root@76e673cd4ac7: /content/Data-1401-Repo\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/Data-1401-Repo\u001b[00m# tail -20 /etc/passwd\n", + "sys:x:3:3:sys:/dev:/usr/sbin/nologin\n", + "sync:x:4:65534:sync:/bin:/bin/sync\n", + "games:x:5:60:games:/usr/games:/usr/sbin/nologin\n", + "man:x:6:12:man:/var/cache/man:/usr/sbin/nologin\n", + "lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin\n", + "mail:x:8:8:mail:/var/mail:/usr/sbin/nologin\n", + "news:x:9:9:news:/var/spool/news:/usr/sbin/nologin\n", + "uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin\n", + "proxy:x:13:13:proxy:/bin:/usr/sbin/nologin\n", + "www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin\n", + "backup:x:34:34:backup:/var/backups:/usr/sbin/nologin\n", + "list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin\n", + "irc:x:39:39:ircd:/var/run/ircd:/usr/sbin/nologin\n", + "gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin\n", + "nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin\n", + "_apt:x:100:65534::/nonexistent:/usr/sbin/nologin\n", + "systemd-network:x:101:104:systemd Network Management,,,:/run/systemd/netif:/usr/sbin/nologin\n", + "systemd-resolve:x:102:105:systemd Resolver,,,:/run/systemd/resolve:/usr/sbin/nologin\n", + "nvidia-persistenced:x:103:107:NVIDIA Persistence Daemon,,,:/nonexistent:/usr/sbin/nologin\n", + "messagebus:x:104:108::/nonexistent:/usr/sbin/nologin\n", + "\u001b]0;root@76e673cd4ac7: /content/Data-1401-Repo\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/Data-1401-Repo\u001b[00m# grep -c $USER /etc/passwd\n", + "exi\n", + "\n", + "\u001b]0;root@76e673cd4ac7: /content/Data-1401-Repo\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/Data-1401-Repo\u001b[00m# \n", + "\u001b]0;root@76e673cd4ac7: /content/Data-1401-Repo\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content/Data-1401-Repo\u001b[00m# ^C\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "SZuhLbD8qvz5" + }, + "source": [ + "## Redirection\n", + "\n", + "Unix provides programs \"pipes\" for input and output. Most of what you see on the screen when you run a program was written to the \"stdout\" (standard output) pipe. Other pipes are \"stdin\" (standard input) and \"stderr\" (standard error), where error messages are written.\n", + "\n", + "As discussed in lecture, the basic commands of are simple, but you can chain them to do complicated things. Redirection is how you chain these commands, directing the output of one command to the input of the next.\n", + "\n", + "As an example, consider the \"cat\" command. Cat takes stdin and outputs it to stdout. Type \"cat\" and press enter and confirm. You can get back to the command prompt by pressing \"control-c\" (sends terminate singal) or \"control-d\" (end of file character). Note that from now on we will use the convention: \"control-d\" = \"^D\"\n", + "\n", + "*Exercise 5a:* Using \"cat\" and indirection you can write things into a file. The \">\" symbol directs stdout into a file. Try \"cat > favorite-colors-list.txt\" and then type in your 3 favorite colors, each on it's own line. Use \"^D\" to end your input." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "H5vxtcXnqvz6" + }, + "source": [ + "Use \"cat\", \"more\", or \"less\" to confirm that you file is as you expect it. \">>\" allows you to append to the file. \n", + "\n", + "*Exercise 5b:* Append 2 more colors to your file." + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "twRKNaGy3XGw", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 255 + }, + "outputId": "8eba1f9a-7a49-4321-eeed-20c6c4fffc89" + }, + "source": [ + "!/bin/bash --noediting" + ], + "execution_count": 1, + "outputs": [ + { + "output_type": "stream", + "text": [ + "cat > favorite-colors-list.txt\n", + "bash: cannot set terminal process group (1397): Inappropriate ioctl for device\n", + "bash: no job control in this shell\n", + "\u001b]0;root@76e673cd4ac7: /content\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content\u001b[00m# cat > favorite-colors-list.txt\n", + "blue \n", + "white \n", + "black\n", + "\n", + "\u001b]0;root@76e673cd4ac7: /content\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content\u001b[00m# \n", + "\u001b]0;root@76e673cd4ac7: /content\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content\u001b[00m# cat >> favorite-colors-list.txt\n", + "red \n", + "purple\n", + "\n", + "\u001b]0;root@76e673cd4ac7: /content\u0007\u001b[01;32mroot@76e673cd4ac7\u001b[00m:\u001b[01;34m/content\u001b[00m# ^C\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "DZODNKiAqvz8" + }, + "source": [ + "The \"sort\" command sorts what it sees on stdin. Instead of taking input from the terminal, you can direct the shell to take stdin from a file using \"<\". Try \"sort < favorite-color-list.txt\" and \"sort < favorite-color-list.txt > sorted-favorite-color-list.txt\".\n", + "\n", + "Finally, instead of piping input / output into files, you can directly chain one program into another using \"|\". So for example, you can do \"cat /etc/passwd | grep -i \\$USER | wc -l\". \n", + "\n", + "*Exercise 5c:* Use indirection to count the number of users on TACC with your first name. Copy the command you used into box below." + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "oP9XlZl_3iZD", + "colab": {} + }, + "source": [ + "!/bin/bash --noediting" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "v5IaZXNyqvz_" + }, + "source": [ + "## Git\n", + "\n", + "`git` is a Version Control System (VCS), typically used to organize the source code of software project but also good source of documents or web-pages. An instance of `git` server stores repositories, each typically containing the code relevant to a specific project. Users create local `clones` of repositories, change and develop the local copies of the code, `commit` the changes to their local repository, `push` to the server as a contribution, \n", + "`pull` updates from the server, and `merge` changes between local and remote versions. \n", + "\n", + "Besides cloning, repositories can be branched or forked. A repository generally starts with a `master` branch that evolves as push requests are merged in. Creating a new branch from an existing branch creates a snapshot of the which can evolve independently or be merged in later. Branches are easy to make and delete, and can serve various purposes. They can represent a stable version of software package. Or a parallel development for different operating system. A fork of a repository is a standalone instance of the repository which can be stored and managed independently from the original, where you can work independently without constraints or interference. \n", + "\n", + "[GitHub](github.com) provides a massive publically accessible instance of a `git` system besides sharing code, projects can be developed by the open source community. It provides tools for managing your repository and a wiki for documentation. Contributions to public software on GitHub generally require making a merge request which would be judged by the managers of the repository. That's why most software packages enourage you to create a new fork, so you can work independently.\n", + "\n", + "Lets take a look at some repositories:\n", + "\n", + "* [This class](https://github.com/UTA-DataScience/DATA1401.2020.Fall)\n", + "\n", + "\n", + "Read through [Version Control System Lecture](https://github.com/UTA-DataScience/VersionControlSystem/blob/master/VersionControlSystem/README.md) to better understand `git`. Note that you do not need to do the exercises at the end of the lecture.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "J_R64sQDqv0A" + }, + "source": [ + "## Plan\n", + "\n", + "You made a clone of the class repository at start of this lab. We will create a new fork where you can keep track and submit your work, following [these instructions](https://help.github.com/articles/fork-a-repo/).\n", + "\n", + "Goto to github.com and log in.\n", + "\n", + "Next, lets create a fork of the [class repository](https://github.com/afarbin/DATA1401-Spring-2019). Click the link and press the \"Fork\" button on the top right. Select your repository as where you want to place the fork.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "edTvE6rOqv0C" + }, + "source": [ + "Now we will check out your fork in your Google Drive / Colab.\n", + "\n", + "Note: Jupyter allows you to run shell directly in a notebook. We will use `!` and `%` to call shell commands directly in this notebook. Follow along yourself. Either create a new notebook or open a terminal. \n", + "\n", + "Start by listing the contents of your current directory." + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "e5tXg0f8qv0D", + "colab": {} + }, + "source": [ + "!ls" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "WYsyYcg1qv0J" + }, + "source": [ + "Make a new directory:" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "Z7noY1hMqv0L", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 51 + }, + "outputId": "135a2b59-7862-44cf-cc98-f1661ee79f5c" + }, + "source": [ + "!mkdir Data-1401-Repo\n", + "%cd Data-1401-Repo" + ], + "execution_count": 2, + "outputs": [ + { + "output_type": "stream", + "text": [ + "mkdir: cannot create directory ‘Data-1401-Repo’: File exists\n", + "/content/Data-1401-Repo\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "fwsBdTnYqv0Q" + }, + "source": [ + "From the github page for your fork, press the green \"Clone or download\" button and copy the URL.\n", + "\n", + "Goto to your notebook and use the following command to clone the repository, pasting the URL you just copied:\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "8w42MH6Jqv0S", + "colab": {} + }, + "source": [ + "# What you past here should look like:\n", + "#!git clone https://github.com//DATA1401.2020.Fall.git" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "cOAuqTVUqv0V" + }, + "source": [ + "Go into the directory:" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "b1Ew4tEZqv0X", + "colab": {} + }, + "source": [ + "%cd DATA1401.2020.Fall\n", + "!ls" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "IrhWToc-qv0a" + }, + "source": [ + "We will now connect your fork to the original so you can pull changes from there. \n", + "\n", + "Check remote status:" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "JxtMYR-9qv0c", + "colab": {} + }, + "source": [ + "!git remote -v" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "9ud3X0fBqv0f" + }, + "source": [ + "Now use the original class URL to set your upstream:" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "pgJlKxBqqv0h", + "colab": {} + }, + "source": [ + "!git remote add upstream https://github.com/UTA-DataScience/DATA1401.2020.Fall" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "id2yUEt9qv0k", + "colab": {} + }, + "source": [ + "!git remote -v" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "sAkgeJ6Iqv0n" + }, + "source": [ + "From now on, you can get the newest version of class material by using:" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "AGDsfTFLqv0o", + "colab": {} + }, + "source": [ + "!git pull" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "u9RAhs5b4vXY" + }, + "source": [ + "We will submit your Lab 1 using git at the next Lab." + ] + } + ] +} \ No newline at end of file From 5fc2cb5cd2a51524694545a84891187ace4c823b Mon Sep 17 00:00:00 2001 From: Bawi Tha Thang <71145730+thangb44@users.noreply.github.com> Date: Wed, 23 Sep 2020 14:16:12 -0500 Subject: [PATCH 04/17] Create README.md --- Labs/Lab.2/Lab 2 Solutions/README.md | 1 + 1 file changed, 1 insertion(+) create mode 100644 Labs/Lab.2/Lab 2 Solutions/README.md diff --git a/Labs/Lab.2/Lab 2 Solutions/README.md b/Labs/Lab.2/Lab 2 Solutions/README.md new file mode 100644 index 0000000..8b13789 --- /dev/null +++ b/Labs/Lab.2/Lab 2 Solutions/README.md @@ -0,0 +1 @@ + From 6d40ce632e3033afffa03a099081006b8f52f20b Mon Sep 17 00:00:00 2001 From: Bawi Tha Thang <71145730+thangb44@users.noreply.github.com> Date: Wed, 23 Sep 2020 14:19:03 -0500 Subject: [PATCH 05/17] Add files via upload --- Labs/Lab.2/Lab 2 Solutions/Lab21401.ipynb | 729 ++++++++++++++++++++++ 1 file changed, 729 insertions(+) create mode 100644 Labs/Lab.2/Lab 2 Solutions/Lab21401.ipynb diff --git a/Labs/Lab.2/Lab 2 Solutions/Lab21401.ipynb b/Labs/Lab.2/Lab 2 Solutions/Lab21401.ipynb new file mode 100644 index 0000000..6d8c1a8 --- /dev/null +++ b/Labs/Lab.2/Lab 2 Solutions/Lab21401.ipynb @@ -0,0 +1,729 @@ +{ + "nbformat": 4, + "nbformat_minor": 0, + "metadata": { + "colab": { + "name": "Lab21401.ipynb", + "provenance": [], + "collapsed_sections": [] + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.0" + } + }, + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "uk7yc0nadBGa" + }, + "source": [ + "# Lab 2\n", + "\n", + "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/UTA-DataScience/DATA1401.2020.Fall/blob/master/Labs/Lab.2/Lab.2.ipynb)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "u9RAhs5b4vXY" + }, + "source": [ + "\n", + "## Make your fork private\n", + "\n", + "Navigate your fork of the course repository on GitHub and click the \"gear\" icon to change the settings. Select \"Options\" on the left and scroll all the way down. Then click on \"Make Private\" to make your repository private. \n", + "\n", + "Next select the collaborators on the left and add your Professor as a collaborator. " + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "q-gMWiE4dBG_" + }, + "source": [ + "## Python Programming\n", + "\n", + "In the remainder of this lab you will practice python by solving some simple exercises. \n", + "\n", + "*Exercise 1:* Write 2 functions `even(x)` and `odd(x)` that take an integer and returns True if the input is even or odd, otherwise returns False. Use cell below for your solution. Use the subsequent cell to demonstrate that your solution works. Feel free to add additional cell as needed using the \"+\" button on the button bar above. " + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "jwN5jff1dBG_", + "colab": {} + }, + "source": [ + "# Write you solution here\n", + "def even(num):\n", + " if num%2==0:\n", + " return True\n", + " else:\n", + " return False\n", + "\n", + "def odd(num):\n", + " if num%2==1: \n", + " return True\n", + " else:\n", + " return False" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "xNJAcodhdBHB", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 85 + }, + "outputId": "41b91589-2027-4825-ad79-875201085073" + }, + "source": [ + "# Test your solution here\n", + "print(\"Even test, even input: \", even(2))\n", + "print(\"Even test, odd input: \",even(3))\n", + "\n", + "print(\"Odd test, odd input: \",odd(3))\n", + "print(\"Odd test, even input: \",odd(4))" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "stream", + "text": [ + "Even test, even input: True\n", + "Even test, odd input: False\n", + "Odd test, odd input: True\n", + "Odd test, even input: False\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "KL_pyzG8dBHD" + }, + "source": [ + "*Exercise 2:* Write a function that takes a list of numbers as input and returns a list of the subset of elements that are less that 10. Test your solution." + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "g8nt0wnldBHE", + "colab": {} + }, + "source": [ + "# Write you solution here\n", + "def less_than_ten(lst):\n", + " out_list=list()\n", + " for element in lst: \n", + " if element<10:\n", + " out_list.append(element)\n", + "\n", + " return out_list" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "__HTUWA1dBHH", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 34 + }, + "outputId": "5d6e70f2-9e8e-4d4f-bd4a-60633dfd8691" + }, + "source": [ + "# Test your solution here\n", + "less_than_ten([1,3,6,11,40,50,100,7])" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "[1, 3, 6, 7]" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 19 + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "T0cx91JudBHK" + }, + "source": [ + "*Exercise 3:* Write a function that takes a number `x_max` as input and returns a function that performs the same task as exercise 2, but for `x_max` instead of 10." + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "PqummMcmdBHK", + "colab": {} + }, + "source": [ + "# Write you solution here\n", + "def less_than_max(x_max):\n", + " out_list=list()\n", + "\n", + " for num in range(x_max):\n", + " if num%2==1:\n", + " out_list.append(num)\n", + "\n", + " return out_list\n", + " " + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "evRYemjXdBHN", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 34 + }, + "outputId": "af490de9-bfbd-443c-f5a3-bf29b6c60afd" + }, + "source": [ + "# Test your solution here\n", + "less_than_max(10)" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "[1, 3, 5, 7, 9]" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 3 + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "h0I8M27LdBHP" + }, + "source": [ + "*Exercise 4:* Write a function that takes an interger as input and returns a list of all divisors of that number." + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "k6GUpDyrdBHP", + "colab": {} + }, + "source": [ + "# Write you solution here\n", + "def divisible_by(num):\n", + " divisible_list=list()\n", + " i=1\n", + " while i<=num:\n", + " if num%i==0:\n", + " divisible_list.append(i)\n", + " i=i+1\n", + "\n", + " return divisible_list\n" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "YDvRmft-dBHR", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 34 + }, + "outputId": "0b869d39-1c2f-4d37-a237-607d851bcf6e" + }, + "source": [ + "# Test your solution here\n", + "divisible_by(17)" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "[1, 17]" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 5 + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "IOWnLMvxdBHT" + }, + "source": [ + "*Exercise 5:* Write a function that takes 2 lists as input and returns a list that contains only the elements that are common between the lists (without duplicates). Make sure your program works on two lists of different sizes." + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "tR31bnTDdBHT", + "colab": {} + }, + "source": [ + "# Write you solution here\n", + "def common_numbers(flst,slst):\n", + " common_list=list()\n", + "\n", + " for num1 in flst:\n", + " for num2 in slst:\n", + " if num2==num1:\n", + " common_list.append(num2)\n", + " \n", + " return common_list\n" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "IJDf6ebYdBHV", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 34 + }, + "outputId": "f7dc12f8-c477-4c05-92c9-1e34a091562d" + }, + "source": [ + "# Test your solution here\n", + "common_numbers([1,2,5,8,10],[1,60,8,5])" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "[1, 5, 8]" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 12 + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "3vA7M9pldBHX" + }, + "source": [ + "*Exercise 6:* Write a function that reads takes a string and returns `True` if the string is a palindrome. (A palindrome is a string that reads the same forwards and backwards.)" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "ncyMDzp6dBHX", + "colab": {} + }, + "source": [ + "# Write you solution here\n", + "def palindrome(Word): \n", + " backward = \"\"\n", + " for char in Word[ : : -1]:\n", + " backward+=char\n", + " \n", + " if backward==Word:\n", + " return True\n", + " else:\n", + " return False\n" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "v5cmVQ6MdBHZ", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 34 + }, + "outputId": "375ab626-1a38-4273-bed5-8e7be351ac2e" + }, + "source": [ + "# Test your solution here\n", + "palindrome('aaeeieeaa')" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "True" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 27 + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "fT35xwandBHc" + }, + "source": [ + "*Exercise 7:* Write a Rock-Paper-Scissors game function, that takes 2 strings, the inputs of player 1 and player 2, and output 1 or 2 corresponding to which player wins, or 0 if draw.\n", + "\n", + "Implement a Rock-Paper-Scissors game by soliciting input from 2 players, testing with this function, and repeating if there is a draw.\n", + "\n", + "Remember the rules:\n", + "\n", + "* Rock beats scissors\n", + "* Scissors beats paper\n", + "* Paper beats rock" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "U1_HxxaWdBHd", + "colab": {} + }, + "source": [ + "# Write you solution here\n", + "def RPS(player1,player2):\n", + " exit = \"Please input valid entries\"\n", + " player1 = player1.lower()\n", + " player2 = player2.lower()\n", + " if ((player1 == 'rock' or player1 == 'scissors' or player1 == 'paper') or (player2 == 'rock' or player2 == 'scissors' or player2 == 'paper')):\n", + " if player1 == 'rock' and player2 == 'scissors':\n", + " return 1\n", + " elif player1 == 'scissors' and player2 == 'paper':\n", + " return 1\n", + " elif player1 == 'paper' and player2 == 'rock':\n", + " return 1\n", + " elif player2 == 'rock' and player1 == 'scissors':\n", + " return 2\n", + " elif player2 == 'scissors' and player1 == 'paper':\n", + " return 2\n", + " elif player2 == 'paper' and player1 == 'rock':\n", + " return 2\n", + " elif player1 == player2:\n", + " return 0\n", + " else:\n", + " return exit" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "ooR2ldZBdBHf", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 34 + }, + "outputId": "731ee703-f8e9-48a9-aa78-f37b76b15203" + }, + "source": [ + "# Test your solution here\n", + "RPS('rock','paper')\n" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "2" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 56 + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "dJQzXNKzdBHh" + }, + "source": [ + "*Exercise 8:* Write a function that takes a integer `n` as input and \n", + "outputs a list of the first `n` Fibonnaci numbers.\n", + "\n", + "The Fibonnaci seqence is a sequence of numbers where the next number in the sequence is the sum of the previous two numbers in the sequence. The sequence looks like this: 1, 1, 2, 3, 5, 8, 13, …)" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "G_4ooRXTdBHh", + "colab": {} + }, + "source": [ + "# Write you solution here\n", + "def Fibonnaci(n):\n", + " fibonnaci_sequence = list()\n", + " fibonnaci_sequence.append(1)\n", + " fibonnaci_sequence.append(1)\n", + " i = 1 \n", + " for i in range(2,n):\n", + " fibonnaci_sequence.append(fibonnaci_sequence[i-1]+fibonnaci_sequence[i-2])\n", + " i+=1\n", + "\n", + " return fibonnaci_sequence\n", + " \n", + " " + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "rzK5FskJdBHj", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 34 + }, + "outputId": "98d87cde-b294-4185-e921-482a47c8947f" + }, + "source": [ + "# Test your solution here\n", + "Fibonnaci(10)" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "[1, 1, 2, 3, 5, 8, 13, 21, 34, 55]" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 18 + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "q6c_AskadBHl" + }, + "source": [ + "*Exercise 9:* Write a function that takes a string of consisting of several words and returns a string that reverses the order of the words.\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "aJdXX6FHdBHl", + "colab": {} + }, + "source": [ + "# Write you solution here\n", + "def reverse(Word):\n", + " back = \"\"\n", + " for char in Word[ : : -1]:\n", + " back+=char\n", + " return back" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "nQyhnLZ_dBHn", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 35 + }, + "outputId": "993d4c1c-95b4-4d5c-9728-c069803e3d98" + }, + "source": [ + "# Test your solution here\n", + "reverse('hello')" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + }, + "text/plain": [ + "'olleh'" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 30 + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "NFSmRaSydBHq" + }, + "source": [ + "*Exercise 10:* Write a guessing game program that will repeatedly guess a number that the users picks, with the user indicating higher or lower, until it correctly guesses the number." + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "Ie2E1JzCdBHr", + "colab": {} + }, + "source": [ + "# Write you solution here\n", + "def guessinggame():\n", + " num = 17\n", + " usernum=input('Please guess the number between 1 - 20: ')\n", + " usernum=int(usernum)\n", + " while (usernum!=num):\n", + " if (usernum Date: Fri, 9 Oct 2020 11:48:49 -0500 Subject: [PATCH 06/17] Create Lab 3 solutions --- Labs/Lab 3/Lab 3 solutions | 1 + 1 file changed, 1 insertion(+) create mode 100644 Labs/Lab 3/Lab 3 solutions diff --git a/Labs/Lab 3/Lab 3 solutions b/Labs/Lab 3/Lab 3 solutions new file mode 100644 index 0000000..c1e3b52 --- /dev/null +++ b/Labs/Lab 3/Lab 3 solutions @@ -0,0 +1 @@ +HI From fe9fa020dcb44b0fca00f7e4ac6404849ef35f27 Mon Sep 17 00:00:00 2001 From: Bawi Tha Thang <71145730+thangb44@users.noreply.github.com> Date: Fri, 9 Oct 2020 11:49:02 -0500 Subject: [PATCH 07/17] Delete Lab 3 solutions --- Labs/Lab 3/Lab 3 solutions | 1 - 1 file changed, 1 deletion(-) delete mode 100644 Labs/Lab 3/Lab 3 solutions diff --git a/Labs/Lab 3/Lab 3 solutions b/Labs/Lab 3/Lab 3 solutions deleted file mode 100644 index c1e3b52..0000000 --- a/Labs/Lab 3/Lab 3 solutions +++ /dev/null @@ -1 +0,0 @@ -HI From 40b21f04979eff349cfd0c53b3e76fe0c1038fc9 Mon Sep 17 00:00:00 2001 From: Bawi Tha Thang <71145730+thangb44@users.noreply.github.com> Date: Fri, 9 Oct 2020 11:49:37 -0500 Subject: [PATCH 08/17] Create Lab --- Labs/Lab 3/Lab 3 Solutions/Lab | 1 + 1 file changed, 1 insertion(+) create mode 100644 Labs/Lab 3/Lab 3 Solutions/Lab diff --git a/Labs/Lab 3/Lab 3 Solutions/Lab b/Labs/Lab 3/Lab 3 Solutions/Lab new file mode 100644 index 0000000..781e137 --- /dev/null +++ b/Labs/Lab 3/Lab 3 Solutions/Lab @@ -0,0 +1 @@ +df From 4470317ed46906bee6b4184b63045f75645b8150 Mon Sep 17 00:00:00 2001 From: Bawi Tha Thang <71145730+thangb44@users.noreply.github.com> Date: Fri, 9 Oct 2020 11:50:16 -0500 Subject: [PATCH 09/17] lab 3 --- Labs/Lab 3/Lab 3 Solutions/Lab3.ipynb | 577 ++++++++++++++++++++++++++ 1 file changed, 577 insertions(+) create mode 100644 Labs/Lab 3/Lab 3 Solutions/Lab3.ipynb diff --git a/Labs/Lab 3/Lab 3 Solutions/Lab3.ipynb b/Labs/Lab 3/Lab 3 Solutions/Lab3.ipynb new file mode 100644 index 0000000..65d16a4 --- /dev/null +++ b/Labs/Lab 3/Lab 3 Solutions/Lab3.ipynb @@ -0,0 +1,577 @@ +{ + "nbformat": 4, + "nbformat_minor": 0, + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.0" + }, + "colab": { + "name": "Lab3.ipynb", + "provenance": [], + "collapsed_sections": [] + } + }, + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "HOgOcBfDxlfL" + }, + "source": [ + "# Lab 3- Tic Tac Toe\n", + "\n", + "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/UTA-DataScience/DATA1401.2020.Fall/blob/master/Labs/Lab.3/Lab.3.ipynb)\n", + "\n", + "In this lab your will build a n x n Tic Tac Toe game. As you do the exercises, make sure your solutions work for any size Tic Tac Toe game. " + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "HS0FKa9qxlfP" + }, + "source": [ + "*Exercise 1:* Write a function that creates an n by n matrix (of list of lists) which will represent the state of a Tie Tac Toe game. Let 0, 1, and 2 represent empty, \"X\", or \"O\".\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "InTdgUVjxlfQ" + }, + "source": [ + "# Write you solution here\n", + "def create_2D_array(n):\n", + " player1 = \"X\"\n", + " player2 = \"O\"\n", + " empty = 0\n", + " game_board = list()\n", + " for i in range (n):\n", + " game_board.append([empty]*n)\n", + " # game_board = [[empty]*n for i in range(n)]\n", + " return game_board" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "cQj2x8n_xlfc", + "outputId": "0613bab9-5459-481d-f4e1-75f6d195b446", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 34 + } + }, + "source": [ + "# Test your solution here\n", + "board = create_2D_array(3)\n", + "board" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "[[0, 0, 0], [0, 0, 0], [0, 0, 0]]" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 15 + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "E8oRp_SGxlfk" + }, + "source": [ + "*Exercise 2:* Write a function that takes a `n` by `n` matrix representing a tic-tac-toe game, and returns -1, 0, 1, or 2 indicating the game is incomplete, the game is a draw, player 1 has won, or player 2 has one, respectively. Here are some example inputs you can use to test your code:" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "AjoGs3ohxlfl" + }, + "source": [ + "winner_is_2 = [[2, 2, 0],\n", + "\t[2, 1, 0],\n", + "\t[2, 1, 1]]\n", + "\n", + "winner_is_1 = [[1, 2, 0],\n", + "\t[2, 1, 0],\n", + "\t[2, 1, 1]]\n", + "\n", + "winner_is_also_1 = [[0, 1, 0],\n", + "\t[2, 1, 0],\n", + "\t[2, 1, 1]]\n", + "\t\n", + "no_winner = [[1, 2, 0],\n", + "\t[2, 1, 0],\n", + "\t[2, 1, 2]]\n", + "\n", + "also_no_winner = [[1, 2, 0],\n", + "\t[2, 1, 0],\n", + "\t[2, 1, 0]]\n", + "\n", + "winner1 = [\n", + "\t [0,0,1],\n", + "\t\t\t\t\t\t[0,1,0],\n", + "\t\t\t\t\t\t[1,0,0]\n", + "]\n", + "winner2 = [\n", + "\t [0,0,2],\n", + "\t\t\t\t\t\t[0,2,0],\n", + "\t\t\t\t\t\t[2,0,0]\n", + "]\n", + "\n", + "winner_2 = [\n", + "\t [2,0,0],\n", + "\t\t\t\t\t\t[0,2,0],\n", + "\t\t\t\t\t\t[0,0,2]\n", + "]" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "M5OvNo-C3fPu" + }, + "source": [ + "# Write you solution here\n", + "def check_matrix(matrix):\n", + " for i in range (len(matrix)):\n", + " for j in range (len(matrix)):\n", + " if (matrix[i][j]==0):\n", + " return False\n", + " else:\n", + " return True\n", + "\n", + "def game_outcome(matrix):\n", + " incomplete = -1\n", + " player1_won = 1\n", + " player2_won = 2\n", + " draw = 0\n", + " size = len(matrix)\n", + "\n", + " column = list()\n", + " row = list()\n", + " for j in range(size):\n", + " csum = 0\n", + " rsum = 0\n", + " for i in range(size):\n", + " csum = csum + matrix[i][j]\n", + " rsum = rsum + matrix[j][i]\n", + " column.append(csum)\n", + " row.append(rsum)\n", + "\n", + " i = 0\n", + " j = 0\n", + "\n", + " for i in range(size):\n", + " if ((column[i]==size) or (row[i]==size)):\n", + " return player1_won\n", + " elif ((column[i]==(2*size)) or (row[i]==(2*size))):\n", + " return player2_won\n", + "\n", + " j = 0\n", + " i = 0\n", + " sum1 = 0 \n", + " sum2 = 0\n", + "\n", + " for i in range(size): \n", + " sum1 += matrix[i][i]\n", + " sum2 += matrix[i][size-1-i]\n", + "\n", + " for j in range (size):\n", + " for k in range (size):\n", + " if ((matrix[j][k]==matrix[k][j]) and ((sum1 == size) or (sum2 == size))):\n", + " return player1_won\n", + " elif ((matrix[j][k]==matrix[k][j]) and ((sum1 == (2*size)) or (sum2 == (size*2)))):\n", + " return player2_won\n", + " \n", + " \n", + " \n", + " \n", + "\n", + "\n", + " \n" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "7TOkOwsUxlfy", + "outputId": "9d8b6299-d73d-4632-daba-b9664571ca3f", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 34 + } + }, + "source": [ + "# Test your solution here\n", + "game_outcome(winner_2)" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "2" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 198 + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "5W2tCH-Uxlf4" + }, + "source": [ + "*Exercise 3:* Write a function that takes 2 integers `n` and `m` as input and draws a `n` by `m` game board. For example the following is a 3x3 board:\n", + "```\n", + " --- --- --- \n", + " | | | | \n", + " --- --- --- \n", + " | | | | \n", + " --- --- --- \n", + " | | | | \n", + " --- --- --- \n", + " ```" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "C23isHrk3Xk_" + }, + "source": [ + "# Write your solution here\n", + "def draw(n,m):\n", + " rows = 0\n", + " i = 0\n", + "\n", + " for i in range (50):\n", + " if ((3+(2*i))==n):\n", + " rows = 3 + 4 + (2*i) + (n - 3)\n", + "\n", + " board = \"\"\n", + " vertical = '| '\n", + " horizontal = ' ---'\n", + " j = 0\n", + "\n", + " extravert = m + 1\n", + "\n", + " for j in range (rows):\n", + " if (j%2==0):\n", + " board += horizontal * n\n", + " else:\n", + " board += vertical * extravert\n", + " board += \"\\n\"\n", + " \n", + " print (board)\n" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "VOuhz59CxlgB", + "outputId": "e47c65f2-4ccf-4ec5-c9b7-21602b4be8d5", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 151 + } + }, + "source": [ + "# Test your solution here\n", + "draw (3,3)" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "stream", + "text": [ + " --- --- ---\n", + "| | | | \n", + " --- --- ---\n", + "| | | | \n", + " --- --- ---\n", + "| | | | \n", + " --- --- ---\n", + "\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "vboJk1LSxlgJ" + }, + "source": [ + "*Exercise 4:* Modify exercise 3, so that it takes a matrix of the form from exercise 2 and draws a tic-tac-tie board with \"X\"s and \"O\"s. " + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "y82Kt0iIxlgK" + }, + "source": [ + "# Write you solution here\n" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "VrJ6D2_yxlgQ" + }, + "source": [ + "# Test your solution here" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "ZwIpi1yNxlgX" + }, + "source": [ + "*Exercise 5:* Write a function that takes a game board, player number, and `(x,y)` coordinates and places \"X\" or \"O\" in the correct location of the game board. Make sure that you only allow filling previously empty locations. Return `True` or `False` to indicate successful placement of \"X\" or \"O\"." + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "JJN3459bxlgX" + }, + "source": [ + "# Write you solution here" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "OLN6n_dLxlgc" + }, + "source": [ + "# Test your solution here" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "cnpOjnUlxlgh" + }, + "source": [ + "*Exercise 6:* Modify Exercise 4 to show column and row labels so that players can specify location using \"A2\" or \"C1\"." + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "dezy5Iw9xlgk" + }, + "source": [ + "# Write you solution here" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "Qy3imbquxlgs" + }, + "source": [ + "# Test your solution here" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "9n3S0jrTxlgz" + }, + "source": [ + "*Exercise 7:* Write a function that takes a board, player number, and location specified as in exercise 6 and then calls exercise 5 to correctly modify the board. " + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "uEzi5fT2xlg0" + }, + "source": [ + "# Write you solution here" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "AEk_ClcJxlg7" + }, + "source": [ + "# Test your solution here" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "YoSTZ5MCxlhD" + }, + "source": [ + "*Exercise 8:* Write a function is called with a board and player number, takes input from the player using python's `input`, and modifies the board using your function from exercise 7. Note that you should keep asking for input until you have gotten a valid input that results in a valid move." + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "lU_JIV1ixlhE" + }, + "source": [ + "# Write you solution here" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "30T6KFbzxlhL" + }, + "source": [ + "# Test your solution here" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "8oEIL_fnxlhR" + }, + "source": [ + "*Exercise 9:* Use all of the previous exercises to implement a full tic-tac-toe game, where an appropriate board is drawn, 2 players are repeatedly asked for a location coordinates of where they wish to place a mark, and the game status is checked until a player wins or a draw occurs." + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "o8V1Up59xlhT" + }, + "source": [ + "# Write you solution here" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "NeD2dA-kxlhX" + }, + "source": [ + "# Test your solution here" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "AxgbIFLpxlhe" + }, + "source": [ + "*Exercise 10:* Test that your game works for 5x5 Tic Tac Toe. " + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "e0S1caT5xlhf" + }, + "source": [ + "# Test your solution here" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "-T-P8mujxlhl" + }, + "source": [ + "*Exercise 11: (Extra Credit)* Develop a version of the game where one player is the computer. Note that you don't need to do an extensive seach for the best move. You can have the computer simply protect against loosing and otherwise try to win with straight or diagonal patterns." + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "WHsjs3Hfxlhm" + }, + "source": [ + "# Write you solution here" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "mXuQz49Txlhw" + }, + "source": [ + "# Test your solution here" + ], + "execution_count": null, + "outputs": [] + } + ] +} \ No newline at end of file From db552f584cf5e009efbd1014c1f07c5f017c7350 Mon Sep 17 00:00:00 2001 From: Bawi Tha Thang <71145730+thangb44@users.noreply.github.com> Date: Fri, 9 Oct 2020 13:59:48 -0500 Subject: [PATCH 10/17] Create hi --- Quiz/Quiz1/hi | 1 + 1 file changed, 1 insertion(+) create mode 100644 Quiz/Quiz1/hi diff --git a/Quiz/Quiz1/hi b/Quiz/Quiz1/hi new file mode 100644 index 0000000..8b13789 --- /dev/null +++ b/Quiz/Quiz1/hi @@ -0,0 +1 @@ + From 170d3a0b3554b8e7d771df8b85e234ba7db00552 Mon Sep 17 00:00:00 2001 From: Bawi Tha Thang <71145730+thangb44@users.noreply.github.com> Date: Fri, 9 Oct 2020 14:00:02 -0500 Subject: [PATCH 11/17] Add files via upload --- Quiz/Quiz1/Quiz_1.ipynb | 208 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 208 insertions(+) create mode 100644 Quiz/Quiz1/Quiz_1.ipynb diff --git a/Quiz/Quiz1/Quiz_1.ipynb b/Quiz/Quiz1/Quiz_1.ipynb new file mode 100644 index 0000000..e5f07de --- /dev/null +++ b/Quiz/Quiz1/Quiz_1.ipynb @@ -0,0 +1,208 @@ +{ + "nbformat": 4, + "nbformat_minor": 0, + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.0" + }, + "colab": { + "name": "Quiz.1.ipynb", + "provenance": [] + } + }, + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "rmVKdH1Tjpi9" + }, + "source": [ + "# Quiz 1\n", + "\n", + "1. Using the lists below, create a function (`make_deck`) that returns a list of the 52 cards in a usual playing card deck. Represent each card as a tuple (e.g. `('Clubs', 3)`). " + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "2UyBHnS1jpi-", + "outputId": "236ff328-a6d9-4f25-ea52-877b53950b75", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 50 + } + }, + "source": [ + "suits = [\"Clubs\", \"Diamonds\", \"Hearts\", \"Spades\"]\n", + "values = list(range(2,11)) + [ \"Jack\", \"Queen\", \"King\", \"Ace\"]\n", + "print(suits)\n", + "print(values)" + ], + "execution_count": 30, + "outputs": [ + { + "output_type": "stream", + "text": [ + "['Clubs', 'Diamonds', 'Hearts', 'Spades']\n", + "[2, 3, 4, 5, 6, 7, 8, 9, 10, 'Jack', 'Queen', 'King', 'Ace']\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "J7EAeCspjpjB" + }, + "source": [ + "# Your solution here\n", + "def make_deck(suits,values):\n", + " deck = list()\n", + " for i in range(len(suits)):\n", + " for j in range(len(values)):\n", + " deck.append(str(suits[i])+str(values[j]))\n", + " return deck\n", + " " + ], + "execution_count": 44, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "mkPqVkPujpjD", + "outputId": "81d57397-fc05-45d4-ba6d-2dc2c7af8404", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 890 + } + }, + "source": [ + "# Test your solution here\n", + "make_deck(suits,values)" + ], + "execution_count": 45, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "['Clubs2',\n", + " 'Clubs3',\n", + " 'Clubs4',\n", + " 'Clubs5',\n", + " 'Clubs6',\n", + " 'Clubs7',\n", + " 'Clubs8',\n", + " 'Clubs9',\n", + " 'Clubs10',\n", + " 'ClubsJack',\n", + " 'ClubsQueen',\n", + " 'ClubsKing',\n", + " 'ClubsAce',\n", + " 'Diamonds2',\n", + " 'Diamonds3',\n", + " 'Diamonds4',\n", + " 'Diamonds5',\n", + " 'Diamonds6',\n", + " 'Diamonds7',\n", + " 'Diamonds8',\n", + " 'Diamonds9',\n", + " 'Diamonds10',\n", + " 'DiamondsJack',\n", + " 'DiamondsQueen',\n", + " 'DiamondsKing',\n", + " 'DiamondsAce',\n", + " 'Hearts2',\n", + " 'Hearts3',\n", + " 'Hearts4',\n", + " 'Hearts5',\n", + " 'Hearts6',\n", + " 'Hearts7',\n", + " 'Hearts8',\n", + " 'Hearts9',\n", + " 'Hearts10',\n", + " 'HeartsJack',\n", + " 'HeartsQueen',\n", + " 'HeartsKing',\n", + " 'HeartsAce',\n", + " 'Spades2',\n", + " 'Spades3',\n", + " 'Spades4',\n", + " 'Spades5',\n", + " 'Spades6',\n", + " 'Spades7',\n", + " 'Spades8',\n", + " 'Spades9',\n", + " 'Spades10',\n", + " 'SpadesJack',\n", + " 'SpadesQueen',\n", + " 'SpadesKing',\n", + " 'SpadesAce']" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 45 + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "6RKcBUC5jpjF" + }, + "source": [ + "2. Create the following functions that take in a list of [usually 5] cards (`hand`) and return True/False:\n", + " * `flush(hand)`: Checks if the cards in the hand are of the same suit.\n", + " * `straight(hand)`: Checks if the cards in the hand are in a order.\n", + " * You can start with just checking numerical cards.\n", + " * Extra Credit: Include all cards.\n", + " * `straight_flush(hand)`: Combination of both." + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "nCvl9dHmjpjG" + }, + "source": [ + "# Your solution here\n", + "def check(cards):\n", + " for i in range (len(cards)):\n", + " if (all(cards[i])==True):\n", + " return True\n", + " elif ()" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "akwkO9wYjpjI" + }, + "source": [ + "# Test your solution here" + ], + "execution_count": null, + "outputs": [] + } + ] +} \ No newline at end of file From dc398efc62414742c133e1ed120fb7b8588af704 Mon Sep 17 00:00:00 2001 From: Bawi Tha Thang <71145730+thangb44@users.noreply.github.com> Date: Fri, 9 Oct 2020 14:00:17 -0500 Subject: [PATCH 12/17] Delete hi --- Quiz/Quiz1/hi | 1 - 1 file changed, 1 deletion(-) delete mode 100644 Quiz/Quiz1/hi diff --git a/Quiz/Quiz1/hi b/Quiz/Quiz1/hi deleted file mode 100644 index 8b13789..0000000 --- a/Quiz/Quiz1/hi +++ /dev/null @@ -1 +0,0 @@ - From 9d3b38b6b0e997339f7197e47f9e163ff7fc1b64 Mon Sep 17 00:00:00 2001 From: Bawi Tha Thang <71145730+thangb44@users.noreply.github.com> Date: Mon, 19 Oct 2020 13:21:20 -0500 Subject: [PATCH 13/17] Add files via upload --- Quiz/Quiz_2.ipynb | 208 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 208 insertions(+) create mode 100644 Quiz/Quiz_2.ipynb diff --git a/Quiz/Quiz_2.ipynb b/Quiz/Quiz_2.ipynb new file mode 100644 index 0000000..8ba57d5 --- /dev/null +++ b/Quiz/Quiz_2.ipynb @@ -0,0 +1,208 @@ +{ + "nbformat": 4, + "nbformat_minor": 0, + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.0" + }, + "colab": { + "name": "Quiz.2.ipynb", + "provenance": [] + } + }, + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "VAsRc8SWDM5M" + }, + "source": [ + "# Quiz 2\n", + "\n", + "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/UTA-DataScience/DATA1401.2020.Fall/blob/master/Quizzes/Quiz.2/Quiz.2.ipynb)\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "EbgKsK2VDM5N" + }, + "source": [ + "1. Using the python math and random libraries, implement the Box-Muller transform. This transform generates 2 random variables $Z_0$ and $Z_1$ that are Normal distributed from two uniformly distributed random variables $U_1$ and $U_2$ using the relation:\n", + "\n", + "\\begin{align}\n", + "Z_0&=&\\sqrt{-2 \\log{U_1}} \\cos{(2\\pi U_2)}\\\\\n", + "Z_1&=&\\sqrt{-2 \\log{U_1}} \\sin{(2\\pi U_2)}\\\\\n", + "\\end{align}\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "JfsjfCfWDM5O" + }, + "source": [ + "Recall that you can import the math and random libraries using:\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "l42kjWBFDM5O" + }, + "source": [ + "import math\n", + "import random" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "t-hqofH5DM5R" + }, + "source": [ + "The functions and variables you will need are:" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "rjt5EjBvDM5S", + "outputId": "b49fc99f-fd55-45c5-ce2c-61d428a501ed" + }, + "source": [ + "x = .1\n", + "\n", + "# Math\n", + "math.pi\n", + "math.sin(x)\n", + "math.cos(x)\n", + "math.log(x)\n", + "math.sqrt(x)\n", + "\n", + "# Random\n", + "random.random()" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "0.9766680279361165" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 3 + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "DOUHUWlODM5V" + }, + "source": [ + "Note that math.log is the natural logarithm. Your solution should look like:" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "waf_CA6BDM5V" + }, + "source": [ + "def generate_normal(N):\n", + " out = list() \n", + " # Write your solution here\n", + " # …\n", + " U1 = random.random()\n", + " U2 = random.random()\n", + " Z0 = math.sqrt((-2)*math.log(U1))*math.cos(2 * math.pi * U2)\n", + " Z1 = math.sqrt((-2)*math.log(U1))*math.sin(2 * math.pi * U2)\n", + " for i in range(N):\n", + " out.append(Z0)\n", + "\n", + " for i in range(N):\n", + " out.append(Z1)\n", + " \n", + " return out\n" + ], + "execution_count": 2, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "xaF5tdjxDM5X" + }, + "source": [ + "Where N is the number of values of the random variable to generate, and the return value out is a list containing N normal distributed values. \n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "TniVB1MzDM5Y" + }, + "source": [ + "2. Test your code by generating 1000 normal distributed values and showing that the mean is 0 and variance is 1. You can use the following 2 functions:\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "_0cbBSX7DM5Y" + }, + "source": [ + "def mean(x):\n", + " return sum(x)/float(len(x))\n", + "\n", + "def variance(x):\n", + " return sum(map(lambda y: y**2,x))/float(len(x))-mean(x)**2\n" + ], + "execution_count": 3, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "pEXuP_F8DM5a" + }, + "source": [ + "3. Submit your solutions to GitHub in same way as your labs/homework." + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "5Mp1FZMkDM5b" + }, + "source": [ + "" + ], + "execution_count": 4, + "outputs": [] + } + ] +} \ No newline at end of file From 79d48057c2abe91b9cd169449cf2d9b831d48e44 Mon Sep 17 00:00:00 2001 From: Bawi Tha Thang <71145730+thangb44@users.noreply.github.com> Date: Fri, 23 Oct 2020 12:10:55 -0500 Subject: [PATCH 14/17] Create Lab 4 --- Labs/Lab 4/Lab 4 Solutions/Lab 4 | 1 + 1 file changed, 1 insertion(+) create mode 100644 Labs/Lab 4/Lab 4 Solutions/Lab 4 diff --git a/Labs/Lab 4/Lab 4 Solutions/Lab 4 b/Labs/Lab 4/Lab 4 Solutions/Lab 4 new file mode 100644 index 0000000..18fb245 --- /dev/null +++ b/Labs/Lab 4/Lab 4 Solutions/Lab 4 @@ -0,0 +1 @@ +fd From 48879e65eb41ce813c6cff3970b3501580f85abb Mon Sep 17 00:00:00 2001 From: Bawi Tha Thang <71145730+thangb44@users.noreply.github.com> Date: Fri, 23 Oct 2020 12:11:18 -0500 Subject: [PATCH 15/17] Add files via upload --- Labs/Lab 4/Lab 4 Solutions/Lab 4.ipynb | 1369 ++++++++++++++++++++++++ 1 file changed, 1369 insertions(+) create mode 100644 Labs/Lab 4/Lab 4 Solutions/Lab 4.ipynb diff --git a/Labs/Lab 4/Lab 4 Solutions/Lab 4.ipynb b/Labs/Lab 4/Lab 4 Solutions/Lab 4.ipynb new file mode 100644 index 0000000..685b980 --- /dev/null +++ b/Labs/Lab 4/Lab 4 Solutions/Lab 4.ipynb @@ -0,0 +1,1369 @@ +{ + "nbformat": 4, + "nbformat_minor": 0, + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.1" + }, + "colab": { + "name": "Copy of Lab.4.ipynb", + "provenance": [], + "collapsed_sections": [] + } + }, + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "6-vpSjjf6Set" + }, + "source": [ + "## Lab 4\n", + "\n", + "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github//afarbin/DATA1401-Spring-2020/blob/master/Labs/Lab-4/Lab-4.ipynb)\n", + "\n", + "In this lab we will become familiar with distributions, histograms, and functional programming. \n", + "\n", + "\n", + "### Uniform Distribution\n", + "Lets start with generating some fake random data. You can get a random number between 0 and 1 using the python random module as follow:" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "jcv1BEJ46Seu", + "outputId": "3b83b672-2f5f-470b-f46f-c6beee7827df", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 34 + } + }, + "source": [ + "import random\n", + "x=random.random()\n", + "print(\"The Value of x is\", x)" + ], + "execution_count": 122, + "outputs": [ + { + "output_type": "stream", + "text": [ + "The Value of x is 0.029034337513467845\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "L42ss5lV6Sey" + }, + "source": [ + "Everytime you call random, you will get a new number.\n", + "\n", + "*Exercise 1:* Using random, write a function `generate_uniform(N, mymin, mymax)`, that returns a python list containing N random numbers between specified minimum and maximum value. Note that you may want to quickly work out on paper how to turn numbers between 0 and 1 to between other values. " + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "0wDtJQf16Sey" + }, + "source": [ + "# Skeleton\n", + "def generate_uniform(N,x_min,x_max):\n", + " out = []\n", + " ### BEGIN SOLUTION\n", + "\n", + " # Fill in your solution here \n", + " min = x_min\n", + " max = x_max\n", + "\n", + " while (len(out)!=N):\n", + " ran = random.uniform(min,max)\n", + " out.append(ran)\n", + "\n", + "\n", + " \n", + " \n", + "\n", + " \n", + "\n", + "\n", + " \n", + " \n", + " ### END SOLUTION\n", + " return out" + ], + "execution_count": 123, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "n_lqhCa86Se0", + "outputId": "e1556143-a101-465d-98f6-7585d9607623", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 101 + } + }, + "source": [ + "# Test your solution here\n", + "data=generate_uniform(1000,-10,10)\n", + "print (\"Data Type:\", type(data))\n", + "print (\"Data Length:\", len(data))\n", + "if len(data)>0: \n", + " print (\"Type of Data Contents:\", type(data[0]))\n", + " print (\"Data Minimum:\", min(data))\n", + " print (\"Data Maximum:\", max(data))" + ], + "execution_count": 124, + "outputs": [ + { + "output_type": "stream", + "text": [ + "Data Type: \n", + "Data Length: 1000\n", + "Type of Data Contents: \n", + "Data Minimum: -9.982577040895737\n", + "Data Maximum: 9.937914063835578\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "pO9iyDFI6Se2" + }, + "source": [ + "*Exercise 2a:* \n", + "Write a function that computes the mean of values in a list." + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "YtVAvdg56Se3" + }, + "source": [ + "# Skeleton\n", + "def mean(Data):\n", + " m=0.\n", + " \n", + " ### BEGIN SOLUTION\n", + "\n", + " # Fill in your solution here \n", + " length = len(data)\n", + " sum = 0\n", + " for elements in Data: \n", + " sum += elements\n", + " m = sum/length\n", + " ### END SOLUTION\n", + " \n", + " return m" + ], + "execution_count": 125, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "yTPeQ72P6Se5", + "outputId": "c21496c3-391d-4cb3-89e3-eb03723e98f7", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 34 + } + }, + "source": [ + "# Test your solution here\n", + "print (\"Mean of Data:\", mean(data))" + ], + "execution_count": 126, + "outputs": [ + { + "output_type": "stream", + "text": [ + "Mean of Data: 0.015390163258806157\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "ofbYz_Hr6Se7" + }, + "source": [ + "*Exercise 2b:* \n", + "Write a function that computes the variance of values in a list." + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "-QoM52oy6Se7" + }, + "source": [ + "# Skeleton\n", + "import math\n", + "def variance(Data):\n", + " m=0.\n", + " \n", + " ### BEGIN SOLUTION\n", + "\n", + " # Fill in your solution here \n", + " length = len(Data)\n", + " Sum = 0\n", + " for elements in Data: \n", + " Sum += elements\n", + " mean = Sum/length\n", + " sum=0\n", + " for numbers in Data: \n", + "\n", + " sum += math.pow(numbers,2)\n", + " m = sum/length\n", + " ### END SOLUTION\n", + " \n", + " return m" + ], + "execution_count": 127, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "JTUW6zok6Se-", + "outputId": "61b13d40-4b50-4308-99f8-831e74a0606a", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 34 + } + }, + "source": [ + "# Test your solution here\n", + "print (\"Variance of Data:\", variance(data))" + ], + "execution_count": 128, + "outputs": [ + { + "output_type": "stream", + "text": [ + "Variance of Data: 32.908681828203015\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "-vKM0r7X6SfA" + }, + "source": [ + "## Histogramming" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "3akQSzOz6SfB" + }, + "source": [ + "*Exercise 3:* Write a function that bins the data so that you can create a histogram. An example of how to implement histogramming is the following logic:\n", + "\n", + "* User inputs a list of values `x` and optionally `n_bins` which defaults to 10.\n", + "* If not supplied, find the minimum and maximum (`x_min`,`x_max`) of the values in x.\n", + "* Determine the bin size (`bin_size`) by dividing the range of the function by the number of bins.\n", + "* Create an empty list of zeros of size `n_bins`, call it `hist`.\n", + "* Loop over the values in `x`\n", + " * Loop over the values in `hist` with index `i`:\n", + " * If x is between `x_min+i*bin_size` and `x_min+(i+1)*bin_size`, increment `hist[i].` \n", + " * For efficiency, try to use continue to goto the next bin and data point.\n", + "* Return `hist` and the list corresponding of the bin edges (i.e. of `x_min+i*bin_size`). " + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "fnMK1VGP6SfB" + }, + "source": [ + "# Solution\n", + "def setup(min,max,steps=10):\n", + " step = (max-min)/steps\n", + " out = list()\n", + " for i in range(steps):\n", + " out.append(min)\n", + " min += step\n", + " return out\n", + "def histogram(x,n_bins=10,x_min=None,x_max=None):\n", + " ### BEGIN SOLUTION\n", + "\n", + " # Fill in your solution here \n", + " if x_min==None:\n", + " x_min = min(x)\n", + " if x_max==None:\n", + " x_max = max(x)\n", + " \n", + " bin_size = x_max/n_bins\n", + " bin_edges = setup(x_min,x_max,n_bins)\n", + " hist =[0]*n_bins\n", + "\n", + " for xvalues in x: \n", + " for i in hist: \n", + " if xvalues>=(x_min+i*bin_size) and xvalues<=(x_max+i*bin_size):\n", + " hist[i] = hist[i]+1\n", + " continue\n", + " \n", + "\n", + " ### END SOLUTION\n", + "\n", + " return hist,bin_edges" + ], + "execution_count": 132, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "0TnHc9j36SfD", + "outputId": "fcf7d5e8-0a85-4aec-cbaa-74a0a868f759", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 313 + } + }, + "source": [ + "# Test your solution here\n", + "h,b=histogram(data,100)\n", + "print(h)" + ], + "execution_count": 133, + "outputs": [ + { + "output_type": "error", + "ename": "IndexError", + "evalue": "ignored", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mIndexError\u001b[0m Traceback (most recent call last)", + "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0;31m# Test your solution here\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mh\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mb\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mhistogram\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m100\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 3\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mh\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;32m\u001b[0m in \u001b[0;36mhistogram\u001b[0;34m(x, n_bins, x_min, x_max)\u001b[0m\n\u001b[1;32m 23\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mhist\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 24\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mxvalues\u001b[0m\u001b[0;34m>=\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx_min\u001b[0m\u001b[0;34m+\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mbin_size\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mxvalues\u001b[0m\u001b[0;34m<=\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx_max\u001b[0m\u001b[0;34m+\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mbin_size\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 25\u001b[0;31m \u001b[0mhist\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mhist\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m+\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 26\u001b[0m \u001b[0;32mcontinue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 27\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;31mIndexError\u001b[0m: list index out of range" + ] + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "lGxPnaus6SfF" + }, + "source": [ + "*Exercise 4:* Write a function that uses the histogram function in the previous exercise to create a text-based \"graph\". For example the output could look like the following:\n", + "```\n", + "[ 0, 1] : ######\n", + "[ 1, 2] : #####\n", + "[ 2, 3] : ######\n", + "[ 3, 4] : ####\n", + "[ 4, 5] : ####\n", + "[ 5, 6] : ######\n", + "[ 6, 7] : #####\n", + "[ 7, 8] : ######\n", + "[ 8, 9] : ####\n", + "[ 9, 10] : #####\n", + "```\n", + "\n", + "Where each line corresponds to a bin and the number of `#`'s are proportional to the value of the data in the bin. " + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "6qxQQIzW6SfF" + }, + "source": [ + "# Solution\n", + "def draw_histogram(x,n_bins,x_min=None,x_max=None,character=\"#\",max_character_per_line=20):\n", + " ### BEGIN SOLUTION\n", + "\n", + " # Fill in your solution here \n", + " \n", + " ### END SOLUTION\n", + "\n", + " return hist,bin_edges" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "p4IdpjDd6SfH", + "outputId": "c7104290-9140-429e-f840-4ed4f6ac4dc7", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 296 + } + }, + "source": [ + "# Test your solution here\n", + "h,b=histogram(data,20)" + ], + "execution_count": 116, + "outputs": [ + { + "output_type": "error", + "ename": "IndexError", + "evalue": "ignored", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mIndexError\u001b[0m Traceback (most recent call last)", + "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0;31m# Test your solution here\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mh\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mb\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mhistogram\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m20\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m", + "\u001b[0;32m\u001b[0m in \u001b[0;36mhistogram\u001b[0;34m(x, n_bins, x_min, x_max)\u001b[0m\n\u001b[1;32m 24\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mhist\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 25\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mxvalues\u001b[0m\u001b[0;34m>=\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx_min\u001b[0m\u001b[0;34m+\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mbin_size\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mxvalues\u001b[0m\u001b[0;34m<=\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx_max\u001b[0m\u001b[0;34m+\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mbin_size\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 26\u001b[0;31m \u001b[0mhist\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 27\u001b[0m \u001b[0;32mbreak\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 28\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;31mIndexError\u001b[0m: list index out of range" + ] + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "uop3HWw96SfJ" + }, + "source": [ + "## Functional Programming\n", + "\n", + "*Exercise 5:* Write a function the applies a booling function (that returns true/false) to every element in data, and return a list of indices of elements where the result was true. Use this function to find the indices of entries greater than 0.5. " + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "HznZlz1G6SfJ" + }, + "source": [ + "\n", + "def where(mylist,myfunc):\n", + " out= []\n", + " \n", + " ### BEGIN SOLUTION\n", + "\n", + " # Fill in your solution here \n", + "\n", + " for i,element in enumerate(mylist):\n", + " if myfunc(element)==True and element>0.5:\n", + " out.append(i)\n", + " ### END SOLUTION\n", + " \n", + " return out" + ], + "execution_count": 134, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "yxOnX71e6SfL", + "outputId": "0f6b2bd7-9d74-498a-deda-a6522229ea86", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 1000 + } + }, + "source": [ + "# Test your solution here\n", + "where(data,bool)" + ], + "execution_count": 135, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "[1,\n", + " 5,\n", + " 9,\n", + " 10,\n", + " 11,\n", + " 15,\n", + " 19,\n", + " 20,\n", + " 21,\n", + " 22,\n", + " 23,\n", + " 24,\n", + " 25,\n", + " 26,\n", + " 27,\n", + " 29,\n", + " 30,\n", + " 32,\n", + " 33,\n", + " 36,\n", + " 38,\n", + " 40,\n", + " 41,\n", + " 42,\n", + " 43,\n", + " 51,\n", + " 52,\n", + " 60,\n", + " 61,\n", + " 62,\n", + " 66,\n", + " 67,\n", + " 70,\n", + " 74,\n", + " 75,\n", + " 76,\n", + " 77,\n", + " 78,\n", + " 79,\n", + " 81,\n", + " 82,\n", + " 84,\n", + " 86,\n", + " 89,\n", + " 90,\n", + " 93,\n", + " 95,\n", + " 96,\n", + " 97,\n", + " 98,\n", + " 99,\n", + " 103,\n", + " 107,\n", + " 108,\n", + " 111,\n", + " 113,\n", + " 114,\n", + " 117,\n", + " 119,\n", + " 122,\n", + " 123,\n", + " 126,\n", + " 127,\n", + " 129,\n", + " 131,\n", + " 136,\n", + " 138,\n", + " 139,\n", + " 141,\n", + " 144,\n", + " 148,\n", + " 150,\n", + " 151,\n", + " 152,\n", + " 154,\n", + " 157,\n", + " 165,\n", + " 166,\n", + " 171,\n", + " 177,\n", + " 178,\n", + " 179,\n", + " 180,\n", + " 187,\n", + " 188,\n", + " 190,\n", + " 191,\n", + " 194,\n", + " 198,\n", + " 204,\n", + " 206,\n", + " 208,\n", + " 209,\n", + " 212,\n", + " 214,\n", + " 215,\n", + " 223,\n", + " 225,\n", + " 226,\n", + " 227,\n", + " 228,\n", + " 230,\n", + " 231,\n", + " 232,\n", + " 233,\n", + " 235,\n", + " 240,\n", + " 241,\n", + " 242,\n", + " 248,\n", + " 249,\n", + " 250,\n", + " 251,\n", + " 255,\n", + " 257,\n", + " 258,\n", + " 259,\n", + " 260,\n", + " 261,\n", + " 262,\n", + " 266,\n", + " 269,\n", + " 270,\n", + " 272,\n", + " 273,\n", + " 274,\n", + " 277,\n", + " 280,\n", + " 282,\n", + " 284,\n", + " 287,\n", + " 289,\n", + " 290,\n", + " 291,\n", + " 292,\n", + " 294,\n", + " 295,\n", + " 296,\n", + " 298,\n", + " 299,\n", + " 300,\n", + " 301,\n", + " 302,\n", + " 304,\n", + " 307,\n", + " 308,\n", + " 309,\n", + " 311,\n", + " 312,\n", + " 313,\n", + " 316,\n", + " 317,\n", + " 319,\n", + " 321,\n", + " 322,\n", + " 324,\n", + " 326,\n", + " 328,\n", + " 329,\n", + " 331,\n", + " 332,\n", + " 337,\n", + " 339,\n", + " 340,\n", + " 342,\n", + " 343,\n", + " 344,\n", + " 345,\n", + " 346,\n", + " 347,\n", + " 348,\n", + " 349,\n", + " 350,\n", + " 351,\n", + " 352,\n", + " 353,\n", + " 356,\n", + " 358,\n", + " 361,\n", + " 363,\n", + " 366,\n", + " 368,\n", + " 370,\n", + " 372,\n", + " 373,\n", + " 376,\n", + " 378,\n", + " 379,\n", + " 381,\n", + " 383,\n", + " 384,\n", + " 385,\n", + " 387,\n", + " 388,\n", + " 393,\n", + " 394,\n", + " 395,\n", + " 402,\n", + " 403,\n", + " 406,\n", + " 409,\n", + " 411,\n", + " 414,\n", + " 415,\n", + " 416,\n", + " 418,\n", + " 422,\n", + " 424,\n", + " 425,\n", + " 428,\n", + " 430,\n", + " 432,\n", + " 433,\n", + " 434,\n", + " 436,\n", + " 438,\n", + " 443,\n", + " 445,\n", + " 446,\n", + " 449,\n", + " 452,\n", + " 455,\n", + " 457,\n", + " 459,\n", + " 460,\n", + " 461,\n", + " 462,\n", + " 463,\n", + " 465,\n", + " 472,\n", + " 473,\n", + " 474,\n", + " 476,\n", + " 477,\n", + " 478,\n", + " 479,\n", + " 483,\n", + " 484,\n", + " 487,\n", + " 489,\n", + " 490,\n", + " 494,\n", + " 495,\n", + " 496,\n", + " 498,\n", + " 499,\n", + " 500,\n", + " 504,\n", + " 505,\n", + " 510,\n", + " 512,\n", + " 513,\n", + " 516,\n", + " 517,\n", + " 519,\n", + " 520,\n", + " 522,\n", + " 523,\n", + " 524,\n", + " 526,\n", + " 527,\n", + " 528,\n", + " 530,\n", + " 535,\n", + " 544,\n", + " 545,\n", + " 546,\n", + " 547,\n", + " 551,\n", + " 555,\n", + " 556,\n", + " 557,\n", + " 559,\n", + " 561,\n", + " 564,\n", + " 566,\n", + " 567,\n", + " 570,\n", + " 571,\n", + " 572,\n", + " 573,\n", + " 574,\n", + " 575,\n", + " 579,\n", + " 580,\n", + " 583,\n", + " 584,\n", + " 587,\n", + " 588,\n", + " 590,\n", + " 591,\n", + " 592,\n", + " 593,\n", + " 594,\n", + " 596,\n", + " 597,\n", + " 601,\n", + " 602,\n", + " 609,\n", + " 611,\n", + " 612,\n", + " 614,\n", + " 616,\n", + " 617,\n", + " 622,\n", + " 626,\n", + " 632,\n", + " 633,\n", + " 634,\n", + " 635,\n", + " 637,\n", + " 638,\n", + " 645,\n", + " 646,\n", + " 648,\n", + " 651,\n", + " 652,\n", + " 654,\n", + " 655,\n", + " 661,\n", + " 662,\n", + " 664,\n", + " 667,\n", + " 673,\n", + " 674,\n", + " 681,\n", + " 683,\n", + " 684,\n", + " 685,\n", + " 687,\n", + " 690,\n", + " 691,\n", + " 697,\n", + " 699,\n", + " 701,\n", + " 703,\n", + " 704,\n", + " 711,\n", + " 714,\n", + " 716,\n", + " 721,\n", + " 724,\n", + " 725,\n", + " 730,\n", + " 731,\n", + " 732,\n", + " 733,\n", + " 734,\n", + " 735,\n", + " 737,\n", + " 739,\n", + " 740,\n", + " 742,\n", + " 743,\n", + " 749,\n", + " 750,\n", + " 755,\n", + " 759,\n", + " 761,\n", + " 763,\n", + " 765,\n", + " 766,\n", + " 769,\n", + " 773,\n", + " 776,\n", + " 779,\n", + " 781,\n", + " 783,\n", + " 784,\n", + " 785,\n", + " 787,\n", + " 788,\n", + " 789,\n", + " 790,\n", + " 791,\n", + " 793,\n", + " 795,\n", + " 797,\n", + " 798,\n", + " 807,\n", + " 808,\n", + " 809,\n", + " 811,\n", + " 812,\n", + " 813,\n", + " 815,\n", + " 817,\n", + " 818,\n", + " 819,\n", + " 822,\n", + " 827,\n", + " 829,\n", + " 831,\n", + " 833,\n", + " 834,\n", + " 838,\n", + " 839,\n", + " 840,\n", + " 841,\n", + " 842,\n", + " 843,\n", + " 845,\n", + " 846,\n", + " 848,\n", + " 849,\n", + " 850,\n", + " 852,\n", + " 853,\n", + " 854,\n", + " 855,\n", + " 857,\n", + " 858,\n", + " 863,\n", + " 869,\n", + " 870,\n", + " 873,\n", + " 875,\n", + " 879,\n", + " 880,\n", + " 882,\n", + " 883,\n", + " 885,\n", + " 888,\n", + " 890,\n", + " 891,\n", + " 897,\n", + " 898,\n", + " 901,\n", + " 903,\n", + " 906,\n", + " 908,\n", + " 913,\n", + " 914,\n", + " 917,\n", + " 919,\n", + " 920,\n", + " 923,\n", + " 924,\n", + " 925,\n", + " 926,\n", + " 927,\n", + " 928,\n", + " 934,\n", + " 935,\n", + " 936,\n", + " 938,\n", + " 939,\n", + " 940,\n", + " 941,\n", + " 943,\n", + " 944,\n", + " 946,\n", + " 947,\n", + " 950,\n", + " 953,\n", + " 954,\n", + " 955,\n", + " 957,\n", + " 960,\n", + " 961,\n", + " 965,\n", + " 968,\n", + " 969,\n", + " 970,\n", + " 973,\n", + " 974,\n", + " 975,\n", + " 976,\n", + " 979,\n", + " 980,\n", + " 983,\n", + " 984,\n", + " 986,\n", + " 987,\n", + " 988,\n", + " 990,\n", + " 991,\n", + " 993,\n", + " 997,\n", + " 998,\n", + " 999]" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 135 + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "PwJdP0qh6SfN" + }, + "source": [ + "*Exercise 6:* The inrange(mymin,mymax) function below returns a function that tests if it's input is between the specified values. Write corresponding functions that test:\n", + "* Even\n", + "* Odd\n", + "* Greater than\n", + "* Less than\n", + "* Equal\n", + "* Divisible by" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "Ep0eh6FT6SfO", + "outputId": "09b7129e-4b86-48fe-89af-db58af639f16", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 84 + } + }, + "source": [ + "def inrange(mymin,mymax):\n", + " def testrange(x):\n", + " return x=mymin\n", + " return testrange\n", + "\n", + "# Examples:\n", + "F1=inrange(0,10)\n", + "F2=inrange(10,20)\n", + "\n", + "# Test of in_range\n", + "print (F1(0), F1(1), F1(10), F1(15), F1(20))\n", + "print (F2(0), F2(1), F2(10), F2(15), F2(20))\n", + "\n", + "print (\"Number of Entries passing F1:\", len(where(data,F1)))\n", + "print (\"Number of Entries passing F2:\", len(where(data,F2)))" + ], + "execution_count": 136, + "outputs": [ + { + "output_type": "stream", + "text": [ + "True True False False False\n", + "False False True True False\n", + "Number of Entries passing F1: 481\n", + "Number of Entries passing F2: 0\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "y83oMS1n6SfQ" + }, + "source": [ + "### BEGIN SOLUTION\n", + "\n", + " # Fill in your solution here \n", + "def Even(num):\n", + " if (num%2==0):\n", + " return True\n", + " else:\n", + " return False\n", + "\n", + "def Odd(num):\n", + " if (num%2==1):\n", + " return True\n", + " else:\n", + " return False\n", + "\n", + "def GreaterThan(numtobetested,constant):\n", + " if (numtobetested>constant):\n", + " return True\n", + " else:\n", + " return False\n", + "\n", + "def LessThan(numtobetested,constant):\n", + " if (numtobetestedconstant) else False\n", + "Less = lambda numtobetest,constant: True if (numtobetest 0. \n", + "\n", + "Use the test function below and your histogramming functions above to demonstrate that your generator is working properly.\n", + "\n", + "Hint: A simple, but slow, solution is to a draw random number test_x within the specified range and another number p between the min and max of the function (which you will have to determine). If p<=function(test_x), then place test_x on the output. If not, repeat the process, drawing two new numbers. Repeat until you have the specified number of generated numbers, N. For this problem, it's OK to determine the min and max by numerically sampling the function. " + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "ZIXFbxUY6SfX" + }, + "source": [ + "def generate_function(func,x_min,x_max,N=1000):\n", + " out = list()\n", + " ### BEGIN SOLUTION\n", + "\n", + " # Fill in your solution here \n", + " while(len(out)!=N):\n", + " test_x = random.randint(x_min,x_max) #let's just use only integers even though we could use random.uniform for decimals for cleanliness\n", + " min_func = min(func)\n", + " max_func = max(func)\n", + " p = random.randint(min_func,max_func)\n", + " if (p<=func(test_x)):\n", + " out.append(test_x)\n", + " else: \n", + " continue\n", + " ### END SOLUTION\n", + " \n", + " return out" + ], + "execution_count": 104, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "5kegMXVI6SfZ" + }, + "source": [ + "# A test function\n", + "def test_func(x,a=1,b=1):\n", + " return abs(a*x+b)\n" + ], + "execution_count": 114, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "jGov63UX6Sfb" + }, + "source": [ + "*Exercise 8:* Use your function to generate 1000 numbers that are normal distributed, using the `gaussian` function below. Confirm the mean and variance of the data is close to the mean and variance you specify when building the Gaussian. Histogram the data. " + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "XDXFCcbl6Sfb" + }, + "source": [ + "import math\n", + "\n", + "def gaussian(mean, sigma):\n", + " def f(x):\n", + " return math.exp(-((x-mean)**2)/(2*sigma**2))/math.sqrt(math.pi*sigma)\n", + " return f\n", + "\n", + "# Example Instantiation\n", + "g1=gaussian(0,1)\n", + "g2=gaussian(10,3)" + ], + "execution_count": 144, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "x3MtNngobkk8", + "outputId": "c27d0dd5-b533-440b-b4ab-ac9ff1e67bab", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 34 + } + }, + "source": [ + "gaussian(0,1)" + ], + "execution_count": 146, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + ".f>" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 146 + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "goMKFptQ6Sfd" + }, + "source": [ + "*Exercise 9:* Combine your `generate_function`, `where`, and `in_range` functions above to create an integrate function. Use your integrate function to show that approximately 68% of Normal distribution is within one variance." + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "AcZhvjM36Sfd" + }, + "source": [ + "def integrate(func, x_min, x_max, n_points=1000):\n", + " \n", + " return integral" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "JXI3efa96Sff" + }, + "source": [ + "" + ], + "execution_count": null, + "outputs": [] + } + ] +} \ No newline at end of file From 8597aeb3b77408e83b84bb898a6093212c8fe27b Mon Sep 17 00:00:00 2001 From: Bawi Tha Thang <71145730+thangb44@users.noreply.github.com> Date: Fri, 23 Oct 2020 12:11:36 -0500 Subject: [PATCH 16/17] Delete Lab 4 --- Labs/Lab 4/Lab 4 Solutions/Lab 4 | 1 - 1 file changed, 1 deletion(-) delete mode 100644 Labs/Lab 4/Lab 4 Solutions/Lab 4 diff --git a/Labs/Lab 4/Lab 4 Solutions/Lab 4 b/Labs/Lab 4/Lab 4 Solutions/Lab 4 deleted file mode 100644 index 18fb245..0000000 --- a/Labs/Lab 4/Lab 4 Solutions/Lab 4 +++ /dev/null @@ -1 +0,0 @@ -fd From f5c07b4082d0b455d02e8362b29385770376699b Mon Sep 17 00:00:00 2001 From: Bawi Tha Thang <71145730+thangb44@users.noreply.github.com> Date: Fri, 30 Oct 2020 12:31:15 -0500 Subject: [PATCH 17/17] Mid-term --- MidTerm_Exam.ipynb | 770 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 770 insertions(+) create mode 100644 MidTerm_Exam.ipynb diff --git a/MidTerm_Exam.ipynb b/MidTerm_Exam.ipynb new file mode 100644 index 0000000..fdd9062 --- /dev/null +++ b/MidTerm_Exam.ipynb @@ -0,0 +1,770 @@ +{ + "nbformat": 4, + "nbformat_minor": 0, + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.0" + }, + "colab": { + "name": "MidTerm-Exam.ipynb", + "provenance": [], + "collapsed_sections": [] + } + }, + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "ISjrL2rPYx4L" + }, + "source": [ + "# Mid-term Exam\n", + "\n", + "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/UTA-DataScience/DATA1401.2020.Fall/blob/master/Exams/Mid-term/Exam.ipynb)\n", + "\n", + "Add cells to this notebook as you need for your solutions and your test of your solutions." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "e2nVpulpYx4M" + }, + "source": [ + "1a. Write a function `first_alphabetically(lst)` that takes a list `lst` of strings and returns the string that is alphabetically first. For example, calling your function with the list of states:" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "ozazANLnYx4N" + }, + "source": [ + "states=['Mississippi', 'Maryland', 'Delaware', 'Connecticut', 'Virginia', 'Utah', 'Kansas',\n", + " 'Wyoming', 'Indiana', 'Louisiana', 'Missouri', 'Illinois', 'Minnesota', 'Vermont', \n", + " 'New Mexico', 'North Dakota', 'Wisconsin', 'Tennessee', 'New York', 'Oklahoma', \n", + " 'Colorado', 'Pennsylvania', 'West Virginia', 'Alabama', 'Montana', 'Texas', \n", + " 'Washington', 'Michigan', 'New Hampshire', 'Arkansas', 'Hawaii', 'Iowa', \n", + " 'Idaho', 'Kentucky', 'Ohio', 'Nebraska', 'Alaska', 'Oregon', 'South Dakota', \n", + " 'New Jersey', 'Florida', 'Georgia', 'Rhode Island', 'Arizona', 'Maine', \n", + " 'South Carolina', 'California', 'Nevada', 'Massachusetts', 'North Carolina']" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "6as7JB7zYx4Q" + }, + "source": [ + "should return the string `\"Alabama\"`. Note that you can compare strings:" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "l62nW7LkYx4Q", + "outputId": "b7c246ad-02a8-4cb4-f841-65e90a2bb02e", + "colab": { + "base_uri": "https://localhost:8080/" + } + }, + "source": [ + "print(\"A\">\"B\")\n", + "print(\"B\">\"A\")\n", + "print(\"A\">\"a\")\n", + "print(\"bca\">\"bbc\")" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "stream", + "text": [ + "False\n", + "True\n", + "False\n", + "True\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "GGigiwvicWHx" + }, + "source": [ + "def first_alphabetically(lst):\n", + " min = lst[0]\n", + " for i in range(len(lst)):\n", + " if lst[i]=bin_edges[i] and d\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mintegrate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnormalgraph\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m90\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m", + "\u001b[0;32m\u001b[0m in \u001b[0;36mintegrate\u001b[0;34m(func, x_min, x_max, steps)\u001b[0m\n\u001b[1;32m 2\u001b[0m \u001b[0mout\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0mx_scan\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0marange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx_min\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mx_max\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m90\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m \u001b[0my_scan\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmap\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfunc\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mx_scan\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 5\u001b[0m \u001b[0my_min\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0my_scan\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 6\u001b[0m \u001b[0my_max\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0my_scan\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;31mTypeError\u001b[0m: 'list' object is not callable" + ] + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "FVtO4hrtYx4Z" + }, + "source": [ + "4a. Recall in quiz 1 we implemented parts of a poker game in python. The solution to quiz 1 is below. Modify the `flush`, `straight`, and `straight_flush` functions to return `None` instead of `False` and the highest card (value-wise) instead of \"True\"." + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "d1LWYkFgYx4Z" + }, + "source": [ + "suits = [\"Clubs\", \"Diamonds\", \"Hearts\", \"Spades\"]\n", + "values = list(range(2,11)) + [ \"Jack\", \"Queen\", \"King\", \"Ace\"]\n", + "\n", + "def make_deck():\n", + " deck=list()\n", + " for suit in suits:\n", + " for value in values:\n", + " deck.append((suit,value))\n", + " return deck\n", + "\n", + "def flush(hand):\n", + " suit=None\n", + " for card in hand:\n", + " if suit:\n", + " if not card[0]==suit:\n", + " return None\n", + " else:\n", + " suit=card[0]\n", + " max_card=0\n", + " for index in range(len(hand)): \n", + " if hand[index] == \"Jack\":\n", + " max_card = \"Jack\"\n", + " elif (hand[index] == \"Queen\"):\n", + " max_card = \"Queen\"\n", + " elif (hand[index] == \"King\"):\n", + " max_card = \"King\"\n", + " elif (hand[index] == \"Ace\"):\n", + " max_card = \"Ace\"\n", + " else:\n", + " max_card = max(hand[1])\n", + " return max_card\n", + "\n", + "def consecutive(lst): \n", + " return sorted(lst) == list(range(min(lst), max(lst)+1)) \n", + "\n", + "def straight(hand): \n", + " return consecutive([values.index(card[1]) for card in hand])\n", + "\n", + "def straight_flush(hand):\n", + " return straight(hand) and flush(hand)" + ], + "execution_count": 38, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "WB9wk2pXGewH", + "outputId": "8f166ea8-6415-4d46-c2ab-56f5d41f3969", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 35 + } + }, + "source": [ + "cards = [('Diamonds', 10),\n", + " ('Diamonds', 'Jack'),\n", + " ('Diamonds', 'Queen'),\n", + " ('Diamonds', 'King'),\n", + " ('Diamonds', 'Ace')]\n", + "\n", + "flush(cards)" + ], + "execution_count": 39, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + }, + "text/plain": [ + "'Jack'" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 39 + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "JEx0TuD3Yx4c" + }, + "source": [ + "4b. Add two additional functions `three_of_a_kind` and `pair` that work in an analogous manner." + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "GAKvkgniYx4c" + }, + "source": [ + "def three_of_a_kiind(hand):\n", + " for index in range(len(hand)): \n", + " if hand[index].count(hand[1])>= 3:\n", + " return True\n", + " return False\n", + " " + ], + "execution_count": 45, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "VhGdNoOGfcno" + }, + "source": [ + "def pair(hand):\n", + " for index in range(len(hand)): \n", + " if hand[index].count(hand[1]) ==2:\n", + " return True\n", + " return False" + ], + "execution_count": 46, + "outputs": [] + } + ] +} \ No newline at end of file