diff --git a/LinearRegression/Blog/1.png b/LinearRegression/Blog/1.png new file mode 100644 index 0000000..a594fe7 Binary files /dev/null and b/LinearRegression/Blog/1.png differ diff --git a/LinearRegression/Blog/10.png b/LinearRegression/Blog/10.png new file mode 100644 index 0000000..56392e3 Binary files /dev/null and b/LinearRegression/Blog/10.png differ diff --git a/LinearRegression/Blog/11.png b/LinearRegression/Blog/11.png new file mode 100644 index 0000000..be0d7c0 Binary files /dev/null and b/LinearRegression/Blog/11.png differ diff --git a/LinearRegression/Blog/12.png b/LinearRegression/Blog/12.png new file mode 100644 index 0000000..c8e588f Binary files /dev/null and b/LinearRegression/Blog/12.png differ diff --git a/LinearRegression/Blog/13.png b/LinearRegression/Blog/13.png new file mode 100644 index 0000000..33d7506 Binary files /dev/null and b/LinearRegression/Blog/13.png differ diff --git a/LinearRegression/Blog/14.png b/LinearRegression/Blog/14.png new file mode 100644 index 0000000..dbdfbfa Binary files /dev/null and b/LinearRegression/Blog/14.png differ diff --git a/LinearRegression/Blog/15.png b/LinearRegression/Blog/15.png new file mode 100644 index 0000000..589f14a Binary files /dev/null and b/LinearRegression/Blog/15.png differ diff --git a/LinearRegression/Blog/16.png b/LinearRegression/Blog/16.png new file mode 100644 index 0000000..b642045 Binary files /dev/null and b/LinearRegression/Blog/16.png differ diff --git a/LinearRegression/Blog/17.png b/LinearRegression/Blog/17.png new file mode 100644 index 0000000..60a9b5f Binary files /dev/null and b/LinearRegression/Blog/17.png differ diff --git a/LinearRegression/Blog/18.png b/LinearRegression/Blog/18.png new file mode 100644 index 0000000..4efa76a Binary files /dev/null and b/LinearRegression/Blog/18.png differ diff --git a/LinearRegression/Blog/19.png b/LinearRegression/Blog/19.png new file mode 100644 index 0000000..2b338cd Binary files /dev/null and b/LinearRegression/Blog/19.png differ diff --git a/LinearRegression/Blog/2.png b/LinearRegression/Blog/2.png new file mode 100644 index 0000000..b6dd554 Binary files /dev/null and b/LinearRegression/Blog/2.png differ diff --git a/LinearRegression/Blog/20.png b/LinearRegression/Blog/20.png new file mode 100644 index 0000000..f3e037b Binary files /dev/null and b/LinearRegression/Blog/20.png differ diff --git a/LinearRegression/Blog/21.png b/LinearRegression/Blog/21.png new file mode 100644 index 0000000..3634981 Binary files /dev/null and b/LinearRegression/Blog/21.png differ diff --git a/LinearRegression/Blog/22.png b/LinearRegression/Blog/22.png new file mode 100644 index 0000000..12ca8ab Binary files /dev/null and b/LinearRegression/Blog/22.png differ diff --git a/LinearRegression/Blog/23.png b/LinearRegression/Blog/23.png new file mode 100644 index 0000000..3786329 Binary files /dev/null and b/LinearRegression/Blog/23.png differ diff --git a/LinearRegression/Blog/24.png b/LinearRegression/Blog/24.png new file mode 100644 index 0000000..f206d6c Binary files /dev/null and b/LinearRegression/Blog/24.png differ diff --git a/LinearRegression/Blog/25+.png b/LinearRegression/Blog/25+.png new file mode 100644 index 0000000..e348618 Binary files /dev/null and b/LinearRegression/Blog/25+.png differ diff --git a/LinearRegression/Blog/25.png b/LinearRegression/Blog/25.png new file mode 100644 index 0000000..c8c5585 Binary files /dev/null and b/LinearRegression/Blog/25.png differ diff --git a/LinearRegression/Blog/26.png b/LinearRegression/Blog/26.png new file mode 100644 index 0000000..c492e06 Binary files /dev/null and b/LinearRegression/Blog/26.png differ diff --git a/LinearRegression/Blog/27.png b/LinearRegression/Blog/27.png new file mode 100644 index 0000000..51c5c0d Binary files /dev/null and b/LinearRegression/Blog/27.png differ diff --git a/LinearRegression/Blog/28.png b/LinearRegression/Blog/28.png new file mode 100644 index 0000000..0f0411b Binary files /dev/null and b/LinearRegression/Blog/28.png differ diff --git a/LinearRegression/Blog/29.1.png b/LinearRegression/Blog/29.1.png new file mode 100644 index 0000000..0c079cc Binary files /dev/null and b/LinearRegression/Blog/29.1.png differ diff --git a/LinearRegression/Blog/29.2.png b/LinearRegression/Blog/29.2.png new file mode 100644 index 0000000..c39c8df Binary files /dev/null and b/LinearRegression/Blog/29.2.png differ diff --git a/LinearRegression/Blog/3.png b/LinearRegression/Blog/3.png new file mode 100644 index 0000000..072ff9f Binary files /dev/null and b/LinearRegression/Blog/3.png differ diff --git a/LinearRegression/Blog/30.png b/LinearRegression/Blog/30.png new file mode 100644 index 0000000..99c4715 Binary files /dev/null and b/LinearRegression/Blog/30.png differ diff --git a/LinearRegression/Blog/31.png b/LinearRegression/Blog/31.png new file mode 100644 index 0000000..af3ce4d Binary files /dev/null and b/LinearRegression/Blog/31.png differ diff --git a/LinearRegression/Blog/32.png b/LinearRegression/Blog/32.png new file mode 100644 index 0000000..4afa498 Binary files /dev/null and b/LinearRegression/Blog/32.png differ diff --git a/LinearRegression/Blog/33.png b/LinearRegression/Blog/33.png new file mode 100644 index 0000000..5abda3a Binary files /dev/null and b/LinearRegression/Blog/33.png differ diff --git a/LinearRegression/Blog/34.png b/LinearRegression/Blog/34.png new file mode 100644 index 0000000..bdbf381 Binary files /dev/null and b/LinearRegression/Blog/34.png differ diff --git a/LinearRegression/Blog/35+.png b/LinearRegression/Blog/35+.png new file mode 100644 index 0000000..36a9930 Binary files /dev/null and b/LinearRegression/Blog/35+.png differ diff --git a/LinearRegression/Blog/35.png b/LinearRegression/Blog/35.png new file mode 100644 index 0000000..83388d1 Binary files /dev/null and b/LinearRegression/Blog/35.png differ diff --git a/LinearRegression/Blog/36.png b/LinearRegression/Blog/36.png new file mode 100644 index 0000000..68cfe8e Binary files /dev/null and b/LinearRegression/Blog/36.png differ diff --git a/LinearRegression/Blog/37.png b/LinearRegression/Blog/37.png new file mode 100644 index 0000000..fe30925 Binary files /dev/null and b/LinearRegression/Blog/37.png differ diff --git a/LinearRegression/Blog/38.png b/LinearRegression/Blog/38.png new file mode 100644 index 0000000..a4e7396 Binary files /dev/null and b/LinearRegression/Blog/38.png differ diff --git a/LinearRegression/Blog/39.png b/LinearRegression/Blog/39.png new file mode 100644 index 0000000..2ed0e5d Binary files /dev/null and b/LinearRegression/Blog/39.png differ diff --git a/LinearRegression/Blog/4.png b/LinearRegression/Blog/4.png new file mode 100644 index 0000000..30e780a Binary files /dev/null and b/LinearRegression/Blog/4.png differ diff --git a/LinearRegression/Blog/40.png b/LinearRegression/Blog/40.png new file mode 100644 index 0000000..71025ae Binary files /dev/null and b/LinearRegression/Blog/40.png differ diff --git a/LinearRegression/Blog/41.png b/LinearRegression/Blog/41.png new file mode 100644 index 0000000..cab9c8f Binary files /dev/null and b/LinearRegression/Blog/41.png differ diff --git a/LinearRegression/Blog/42.png b/LinearRegression/Blog/42.png new file mode 100644 index 0000000..23ff2ef Binary files /dev/null and b/LinearRegression/Blog/42.png differ diff --git a/LinearRegression/Blog/43.png b/LinearRegression/Blog/43.png new file mode 100644 index 0000000..16c8455 Binary files /dev/null and b/LinearRegression/Blog/43.png differ diff --git a/LinearRegression/Blog/44.1.png b/LinearRegression/Blog/44.1.png new file mode 100644 index 0000000..9c424f9 Binary files /dev/null and b/LinearRegression/Blog/44.1.png differ diff --git a/LinearRegression/Blog/44.2.png b/LinearRegression/Blog/44.2.png new file mode 100644 index 0000000..0bffdfe Binary files /dev/null and b/LinearRegression/Blog/44.2.png differ diff --git a/LinearRegression/Blog/45.png b/LinearRegression/Blog/45.png new file mode 100644 index 0000000..e7bb707 Binary files /dev/null and b/LinearRegression/Blog/45.png differ diff --git a/LinearRegression/Blog/46.1.png b/LinearRegression/Blog/46.1.png new file mode 100644 index 0000000..5914437 Binary files /dev/null and b/LinearRegression/Blog/46.1.png differ diff --git a/LinearRegression/Blog/46.2.png b/LinearRegression/Blog/46.2.png new file mode 100644 index 0000000..45162b9 Binary files /dev/null and b/LinearRegression/Blog/46.2.png differ diff --git a/LinearRegression/Blog/47.png b/LinearRegression/Blog/47.png new file mode 100644 index 0000000..39e71ae Binary files /dev/null and b/LinearRegression/Blog/47.png differ diff --git a/LinearRegression/Blog/48.png b/LinearRegression/Blog/48.png new file mode 100644 index 0000000..0fc88b8 Binary files /dev/null and b/LinearRegression/Blog/48.png differ diff --git a/LinearRegression/Blog/49.png b/LinearRegression/Blog/49.png new file mode 100644 index 0000000..f207a9d Binary files /dev/null and b/LinearRegression/Blog/49.png differ diff --git a/LinearRegression/Blog/5.png b/LinearRegression/Blog/5.png new file mode 100644 index 0000000..42c7661 Binary files /dev/null and b/LinearRegression/Blog/5.png differ diff --git a/LinearRegression/Blog/6.png b/LinearRegression/Blog/6.png new file mode 100644 index 0000000..3642c66 Binary files /dev/null and b/LinearRegression/Blog/6.png differ diff --git a/LinearRegression/Blog/7.png b/LinearRegression/Blog/7.png new file mode 100644 index 0000000..713bdf0 Binary files /dev/null and b/LinearRegression/Blog/7.png differ diff --git a/LinearRegression/Blog/8.png b/LinearRegression/Blog/8.png new file mode 100644 index 0000000..ac668b7 Binary files /dev/null and b/LinearRegression/Blog/8.png differ diff --git a/LinearRegression/Blog/9.png b/LinearRegression/Blog/9.png new file mode 100644 index 0000000..143ed5b Binary files /dev/null and b/LinearRegression/Blog/9.png differ diff --git a/LinearRegression/Blog/main.tex b/LinearRegression/Blog/main.tex new file mode 100644 index 0000000..7de1f59 --- /dev/null +++ b/LinearRegression/Blog/main.tex @@ -0,0 +1,235 @@ +\documentclass{article} +\usepackage[utf8]{inputenc} + +\title{Intro to Multiple Linear Regression Analysis in Python} +\author{Bowen Zhu} +\date{May 2020} + +\usepackage{bm} +\usepackage{esvect} +\usepackage{commath} +\usepackage{amsmath,amsfonts,amssymb} +\usepackage{physics} +\usepackage{natbib} +\usepackage{graphicx} +\usepackage[breaklinks=true]{hyperref} +\usepackage[driver=pdftex]{geometry} +\usepackage{subcaption} +\usepackage{float} +\usepackage{wrapfig} +\usepackage[strict]{changepage} +\hypersetup{ + colorlinks=true, + linkcolor=blue, + urlcolor=blue, +} + +\begin{document} +\newgeometry{top=3.5cm} +\maketitle + +\textbf{Linear Regression models} allow us to summarize and study relationships between variables by fitting a straight line to the observed data. You may have heard of \textbf{simple linear regression}, which establishes the relationship between two quantitative variables--one predictor variable and one target variable. + +However, in reality, it is rare that a target variable is explained by only one predictor. So what if we want to estimate a continuous variable using more than one predictors? If two or more predictor variables have a linear relationship with the target variable, a \textbf{multiple linear regression analysis} allows us to quantify the strength of the relationship and estimate the target variable at a certain value of the predictor variables. + +This tutorial will show you step-by-step how to perform a multiple linear regression analysis using Python, from checking assumptions to evaluating performance. +\section*{Getting Started} +\subsection*{Import Libraries} +First, let's important all necessary libraries in the beginning to keep our code organized: +\begin{figure}[H]\includegraphics[width=\linewidth]{1}\end{figure} +\restoregeometry +\subsection*{Read Dataset} +For this analysis, I will use a real estate dataset that can be downloaded from \href{https://www.kaggle.com/quantbruce/real-estate-price-prediction}{here}. I choose to upload it to Google Colab from local drive. Of course, you can also replace the first argument in read\_csv() with the filepath or URL to the dataset that you'd like to use instead. The delimiter parameter is set to comma by default, but you can change it depending on the delimiter of your csv file. +\begin{figure}[H]\includegraphics[width=\linewidth]{2}\end{figure} +We can take a look at what the data looks like: +\begin{figure}[H]\includegraphics[width=\linewidth]{3}\end{figure} +\begin{figure}[H]\includegraphics[width=\linewidth]{4}\end{figure} +As you can see, the first column in the real estate data set is redundant, so we should probably remove it. Otherwise, this data set seems directly usable. However, it's likely that your own data set will require more preprocessing--dealing with missing/invalid values, restructuring the data, and encoding dummy variables, etc. +\begin{figure}[H]\includegraphics[width=\linewidth]{5}\end{figure} +After data preprocessing, we can use the describe() function to obtain some summary statistics of our target variable, and the displot() function from the seaborn library to plot its distribution. +\begin{figure}[H]\includegraphics[width=1.1\linewidth]{6}\end{figure} +\begin{figure}[H]\centering\includegraphics[width=0.5\linewidth]{7}\end{figure} +We see that the values of house price of unit area are normally distributed with perhaps a few outliers on the right. +\section*{Checking Multicollinearity} +Since a multiple regression involves more than one predictor variables, multicollinearity occurs when some of them are highly correlated with each other, which means each of these predictors will account for similar variance in the target variable. Though the presence of multicollinearity will not affect the predictive power of our model, it will make it more difficult for us to assess the individual influence of a predictor on our target variable. Therefore, it is important to check the ”no multicollinearity” condition before our analysis. We can detect multicollinearity using either a correlation matrix or VIF factors. +\subsection*{Correlation Matrix} +The correlation coefficient is a statistic that measures linear correlation between two variables. Its value lies between -1 and +1. The absolute value gives the strength of the relationship between the two variables. The larger its magnitude, the stronger their relationship. In general, a correlation coefficient above +0.8 or below -0.8 indicates a strong correlation, and we should take some measures to reduce multicollinearity before we continue. + +The pandas corr() function calculates the correlations among all variables in the dataframe. We can visualize the resulting correlation matrix using a heatmap from the seaborn library. The diagonal values should always equal 1 since they are the correlations of a variable with itself. +\begin{figure}[H]\includegraphics[width=1.1\linewidth]{8}\end{figure} +\begin{figure}[H]\includegraphics[width=0.9\linewidth]{9}\end{figure} +\subsection*{VIF Factors} +VIF factor is an alternative measure for multicollinearity, which ranges from 1 upwards. A rule of thumb used in practice is if a VIF value is greater than 4, then multicollinearity is a problem. We can check VIF factors using the variance\_inflation\_factor() function from the statsmodels package. +\begin{figure}[H]\includegraphics[width=\linewidth]{10}\end{figure} +\begin{figure}[H]\centering\includegraphics[width=0.5\linewidth]{11}\end{figure} +You should see in the correlation matrix that the correlation coefficient between X3 and X6 is $-0.81 < -0.8$, and the VIF factor for X3 is also slightly higher than 4, which is unfavorable. + +A simple solution is to remove one of the correlated variables from our model. After removing X3 from the list of column names and rerunning the previous code, the new correlation matrix should show no significant multicollinearity, and the VIF factors of all the remaining variables are around 1, so we are in good shape, and can proceed with our regression. +\begin{figure}[H]\includegraphics[width=0.8\linewidth]{12}\end{figure} +\begin{figure}[H]\centering\includegraphics[width=0.5\linewidth]{13}\end{figure} +But wait, you may ask, why did I choose to delete X3 rather than X6? Indeed, my choice was kind of arbitrary here. To remove the redundant variables more carefully, we can consider using stepwise regression to only keep the predictors that lead to the best performance; I will describe backwards stepwise regression later. + +Another way to deal with multicollinearity without the need to drop predictors before the analysis is to perform regression with regularization techniques, such as Lasso and Ridge. Regularization can help you handle multicollinearity, so if you don't want to delete any variables, you may choose to skip the OLS model below and directly jump to the Lasso/Ridge/Elastic Net regression presented in the end. + +\section*{Performing the Regression} +\subsection*{A Bit of Math} +Now it's finally time to introduce our regression model. Under the linear model, the actual value of our target variable should be a linear combination of all predictor variables plus a constant (intercept) as well as some errors. Thus it can be written in the form of an equation $y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \dots + \epsilon_i$, and our goal is to estimate $\beta_0$, $\beta_1$, $\beta_2$, ... + +Assuming you’re not scared of matrices, let's first understand some mathematical background for linear regression in linear algebra. More generally speaking, suppose our data consisits of $n$ observations, each of which contains the values of $k$ predictor variables (not including the intercept) as well as the corresponding value of the target variable. In matrix notation, we can write the regressor matrix $\text{X}$ as +$\begin{bmatrix} +1 & X_{11} & \dots & X_{1k} \\ +1 & X_{21} & \dots & X_{2k} \\ +\vdots & \vdots & \ddots & \vdots \\ +1 & X_{n1} & \dots & X_{nk} \\ +\end{bmatrix}$, the target vector $\mathbf{y}$ as +$\begin{bmatrix} +y_1 \\ +y_2 \\ +\vdots \\ +y_n \\ +\end{bmatrix}$, the coefficient vector $\bm{\beta}$ as +$\begin{bmatrix} +\beta_0 \\ +\beta_1 \\ +\vdots \\ +\beta_k \\ +\end{bmatrix}$, and the residual vector $\bm{\epsilon}$ as +$\begin{bmatrix} +\epsilon_1 \\ +\epsilon_2 \\ +\vdots \\ +\epsilon_n \\ +\end{bmatrix}$. Then, our linear model can be rewritten as the following: +$\begin{bmatrix} +y_1 \\ +y_2 \\ +\vdots \\ +y_n \\ +\end{bmatrix} = +\begin{bmatrix} +1 & X_{11} & \dots & X_{1k} \\ +1 & X_{21} & \dots & X_{2k} \\ +\vdots & \vdots & \ddots & \vdots \\ +1 & X_{n1} & \dots & X_{nk} \\ +\end{bmatrix} +\begin{bmatrix} +\beta_0 \\ +\beta_1 \\ +\vdots \\ +\beta_k \\ +\end{bmatrix} + +\begin{bmatrix} +\epsilon_1 \\ +\epsilon_2 \\ +\vdots \\ +\epsilon_n \\ +\end{bmatrix}$. Or, more simply, as $\mathbf{y} = \text{X} \bm{\beta} + \bm{\epsilon}$, where for each $i \in [1, n]$, $y_i = \beta_0 + \beta_1 X_{i1} + \beta_2 X_{i2} + \dots + \beta_k X_{ik} + \epsilon_i$. + +The simplest model that we're going to start with is the ordinary least squares (OLS) regression, which aims to minimize the sum of the squared residuals $\norm{\epsilon}^2$, which is equivalent to $\norm{\mathbf{y} - \text{X} \bm{\beta}}^2$. It can be shown via matrix calculus that the least squares estimator for our coefficient vector $\bm{\beta}$ is given by $\bm{\hat{\beta}} = (\text{X}^\intercal \text{X})^{-1} \text{X}^\intercal \mathbf{y}$. +\subsection*{Getting the Coefficients} +The above calculation has already been implemented in the scikit-learn library so we don't need to calculate $\bm{\hat{\beta}}$ manually. After instantiating the LinearRegression class, we can easily obtain the coefficients along with the intercept by calling the fit() method along with our data. +\begin{figure}[H]\includegraphics[width=\linewidth]{14}\end{figure} +\begin{figure}[H]\centering\includegraphics[width=0.6\linewidth]{15}\end{figure} +This means, for example, that for every one unit of change in X1 transaction date, the change in the house price of unit area is approximately 4.21. + +We can compare and visualize some predicted values of the house price with the corresponding actual values in a bar graph: +\begin{figure}[H]\includegraphics[width=\linewidth]{16}\end{figure} +\vspace*{-2cm} +\begin{figure}[H]\includegraphics[width=0.25\linewidth]{17}\end{figure} +\begin{figure}[H]\includegraphics[width=\linewidth]{18}\end{figure} + +\section*{Checking the Residuals} +Congrats! You have successfully obtained the regression line. Now we can get the residuals of the regression by calculating the difference between each data point and the corresponding predicted value. In order to ensure the validity of our analysis, we need to use the residuals to verify a few more conditions apart from multicollinearity. + +First, we can use the scatter() function from the matplotlib library to plot the residuals (between the actual and predicted values) against the predicted values. We expect to see that the points are randomly distributed in the scatter plot. More specifically, they should show no curvature (which ensures linearity), and constant variation along the predicted value (which ensures homoscedasticity). +\begin{figure}[H]\centering\includegraphics[width=0.9\linewidth]{19}\end{figure} +\begin{figure}[H]\centering\includegraphics[width=0.6\linewidth]{20}\end{figure} +Finally, let's generate a normal probability plot using probplot() to test the normality of the residuals. If the residuals are normally distributed, the points in the normal probability plot should stick to the red diagonal line as closely as possible. As shown in the plot below, the residuals of our real estate model show a slight deviation from the ideal line, but the deviation or bend is not very serious. +\begin{figure}[H]\centering\includegraphics[width=\linewidth]{21}\end{figure} +\begin{figure}[H]\centering\includegraphics[width=0.6\linewidth]{22}\end{figure} +The above procedure indicates that our residuals satisfy the regression conditions. If, however, your residuals do show some strong patterns, you should consider transforming your data using negative reciprocal, logarithm, square root, square, or exponentiation and then redo the regression. If none of the above transformation options can solve the problem, it's possible that your data set does not fit the linear model, and you may resort to a nonlinear model. + +\section*{Making Predictions} +After all the previous conditions have been verified, you now have a trustworthy regression equation which enables you to make some predictions if given an extra piece of data. While we can use predict() in scikit-learn to get the mean of our prediction, confidence intervals are available via get\_prediction() in the statsmodels package. However, unlike the scikit-learn library, statsmodels does not add a column of ones to the regressor matrix by default, so before calling get\_prediction() we need to add the ones using the add\_constant() function. +\begin{figure}[H]\centering\includegraphics[width=\linewidth]{23}\end{figure} +\begin{figure}[H]\centering\includegraphics[width=1.05\linewidth]{24}\end{figure} +Note that apart from the mean and standard error, summary\_frame() spits out two intervals at 95\% confidence level (we have set alpha = 0.05). The first pair of lower bound and upper bound forms a confidence interval, which is an estimate of the mean price of all the houses that match our randomly generated data. The second pair, on the other hand, forms a prediction interval, which estimate the price of a particular house with this set of data. Because individual values vary more than means, we can expect the prediction interval to be wider than the confidence interval at the same confidence level. + +\section*{Evaluations} +While it is relatively straightforward to obtain regression lines and predictions, there are numerous evaluation metrics to measure the accuracy of our model. In fact, the summary() function from statsmodels provides you with an overwhelming amount of data: +\begin{figure}[H]\includegraphics[width=\linewidth]{25}\end{figure} +Don't panic, let's interpret some important values one by one. +\subsubsection*{R-squared and Adjusted R-squared} +\begin{figure}[H]\includegraphics[width=0.5\linewidth]{26}\end{figure} +R-squared, a.k.a. coefficient of determination, measures the proportion of the variance of the target variable that can be explained by the predictors in our model. $R^2$ ranges from 0 to 100\%. The higher the value, the better the model. +Our model has $R^2 = 0.542$, which means approximately 54\% of the variation in the house price can be accounted for by our OLS model. + +Nevertheless, $R^2$ only works as intended in a simple linear regression model with only one predictor. In a multiple regression, when a new predictor is added to the model, $R^2$ can only increase but never decrease. This implies that a model may have a higher $R^2$ simply because it has more predictors. On the other hand, the adjusted $R^2$ takes into account the number of predictors in the model. As a result, it only increases if the new predictor improves the model more than expected by chance and decreases if it fails to do so. +\subsubsection*{F-statistic} +\begin{figure}[H]\includegraphics[width=0.5\linewidth]{27}\end{figure} +Below $R^2$ and the adjusted $R^2$ is the F-statistic which is the statistic of an F-test checking whether our model fits the data well. The null hypothesis of the F-test is that the model with no predictors but only an intercept can fit the data as well as the proposed one. To put it simply, it's testing whether $R^2$ is significantly greater than zero. The F-statistic is the ratio of the explained variance to the unexplained variance, so a higher value of F-statistic is evidence against the null hypothesis. + +The fourth row on the right gives the p-value corresponding to the F-statistic. Our model yields a very small p-value of 5.12e-67, so there is sufficient evidence to reject the null hypothesis and conclude that our model does have some predictive power. +\subsubsection*{Confidence Intervals and t-tests} +\begin{figure}[H]\includegraphics[width=\linewidth]{28}\end{figure} +Perhaps the most important information is located in the central part of the summary--the estimated values of each coefficient in the first column, followed by an estimate of the standard deviation of the coefficient. The last two columns provide 95\% confidence intervals constructed from the coefficients and their standard errors in the first two columns. + +The two columns in the middle are the results of t-tests checking for the relevancy of each coefficient. The null hypothesis of each t-test is that the corresponding coefficient equals zero, meaning no linear relationship between this predictor and the target variable. The t-statistic is the value in the first column (mean) divided by the second column (standard error); the p-values corresponding to each t-statistic are presented in the next column. + +In our case, you can see that the p-values of all the coefficients except X1 are smaller than .01, so we can safely reject the null hypotheses and say that they are indeed relevant to predicting our target variable. However, X1 may be a redundant variable at .01 significance level. + +\section*{Stepwise Regression} +The results of the OLS regression seem significant but the real estate data set involves only 6 predictors. When we have a substantial number of predictors in the model, it's unlikely that all of them will have significant p-values. Oftentimes, there exist predictors that give redundant information. As the model increases in complexity, the variance of our estimator can get very large. Thus we may consider getting rid of some of the predictors in order to simplify the model and prevent overfitting. + +In theory, we could try all possible combinations of our predictors and choose the best combination (for example, the one that yields the highest adjusted $R^2$). However, imagine your data set contains have 20 predictors, then you would need to compare $2^{20} > 1,000,000$ potential models! + +Backwards stepwise regression provides an easier procedure to select reliable predictors. Starting with all predictors in the model, in each iteration we find out which predictor to delete, if any, will lead to the best model. We repeat this step until we can't improve our model by removing any variables. + +The question then arises: how do we determine quantitatively which model is the best? One method to measure the performance of models is the leave-one-out cross-validation--in each iteration, we pick a data point (a row of $X_{i\cdot}$ and $y_i$) from our data set, and refit an OLS model on the remaining data points. Based on this model, we make a prediction $\hat{y}_{i}$ for the data point $y_i$ that we picked in the current iteration. We sum up the squared differences between each data point and its predicted value, i.e. $\sum_{i=1}^{n} (y_i - \hat{y}_i)^2$ as our evaluation metric, called the risk estimator. A good model should yield small difference between each pair of predicted and actual values, so smaller risk estimators are preferred. + +Using the leave-one-out cross-validation, we are able to quantitatively compare the models during each iteration of backwards stepwise regression. This procedure is not supported in Python libraries, but the process is not hard to implement: +\begin{figure}[H]\centering\includegraphics[width=1.1\linewidth]{29.1}\end{figure} +\begin{figure}[H]\centering\includegraphics[width=1.1\linewidth]{29.2}\end{figure} +We can apply the above function to our real estate data, though it is too small in size to let the above algorithm run for many rounds and make significant improvement. Still, backwards stepwise regression reveals that X6 is the only predictor among X1-X6 that should be dropped so that the risk estimator can be improved from 33075 to 32952 (obviously, very little improvement here). +\begin{figure}[H]\includegraphics[width=0.8\linewidth]{30}\end{figure} +We can try to rerun the OLS regression with X6 removed from the model. The adjusted $R^2$ now increases to 0.577. Thus to reduce multicollinearity, deleting X6 is perhaps a better option than X3. +\begin{figure}[H]\includegraphics[width=1.1\linewidth]{31}\end{figure} +\begin{figure}[H]\includegraphics[width=\linewidth]{32}\end{figure} + +\section*{Regularization} +However, stepwise regression has many potential drawbacks. For example, as a greedy search algorithm, it is not guaranteed to find the best model, especially when predictors are correlated. + +Another approach that performs predictor selection and also works in the presence of multicollinearity is regularization, a technique for reducing the complexity of the model by adding an extra penalty term to the errors. More specifically, our objective here is still to minimize $\norm{\mathbf{y} - \text{X} \bm{\beta}}^2$ (same as the OLS model), but we now impose an additional constraint that the coefficients should not exceed some threshold. The specific constraint depends on the type of the regularization. Three popular regularization techniques are mentioned below--Lasso, Ridge, and Elastic Net. + +Regardless of the type of regularization, the new model will always penalize large coefficients, so we need to standardize our predictor variables to ensure that the penalty is applied fairly across all predictors. Standardization means converting the values in each column of the regressor matrix into z-scores, so the standard score of $x$ is calculated as $z = \frac{x - \mu}{\sigma}$, where $\mu$ is the mean of all values in the same column and $\sigma$ is their standard deviation. +\begin{figure}[H]\includegraphics[width=0.6\linewidth]{33}\end{figure} +After the standardization, we are ready to choose a specific model to perform. Let's start with Lasso regression. Lasso regression requires that the sum of the absolute values of the coefficients $\norm{\beta}_1$ is no larger than some prespecified value. Using Lagrange Multiplier, we can write our objective function to minimize as $\norm{\mathbf{y} - \text{X} \bm{\beta}}^2_2 + \lambda \norm{\beta}_1$. $\lambda \norm{\beta}_1$ is the extra penalty term here and can be controlled by changing the value of $\lambda$. When $\lambda$ is zero, the model reduces to OLS; when $\lambda$ tends to infinity, all coefficients approach zero. As you gradually increase $\lambda$, the magnitude of the coefficients has to decrease towards zero. This can be illustrated by the code below: +\begin{figure}[H]\includegraphics[width=0.8\linewidth]{34}\end{figure} +\begin{figure}[H]\centering\includegraphics[width=0.5\linewidth]{35}\end{figure} +To improve the predictive power of our model, we can again resort to cross-validation and choose the $\lambda$ that optimizes some evaluation metrics, such as the cross-validated sum of squared residuals. This can be done by the GridSearchCV() function in scikit-learn. It searches over some range of $\lambda$ to find the optimal evaluation score. Note that the scoring metric is negative mean squared error since the scoring API is designed to maximize rather than minimize the score. Of course, we can easily convert the score back to positive using the negative() function in numpy. +\begin{figure}[H]\centering\includegraphics[width=\linewidth]{36}\end{figure} +\begin{figure}[H]\includegraphics[width=0.5\linewidth]{37}\end{figure} +We can visualize where the optimal value of $\lambda$ is located in the interval: +\begin{figure}[H]\includegraphics[width=\linewidth]{38}\end{figure} +\begin{figure}[H]\centering\includegraphics[width=0.6\linewidth]{39}\end{figure} +Putting the optimal value of $\lambda$ in the parameter of Lasso(), we can obtain the coefficients under the Lasso model: +\begin{figure}[H]\includegraphics[width=\linewidth]{40}\end{figure} +\begin{figure}[H]\includegraphics[width=0.75\linewidth]{41}\end{figure} +Note that X6 is again reduced to zero. The significant changes in the magnitude of other coefficients is due to the fact that we have standardized our predictors before the regression. To recover the interpretability of our results, we may choose to back-convert the coefficients (by dividing by the standard deviation) and the intercept (by subtracting all the coefficients times the means of the corresponding predictors and divided by the corresponding standard deviations): +\begin{figure}[H]\includegraphics[width=\linewidth]{42}\end{figure} +\begin{figure}[H]\includegraphics[width=0.75\linewidth]{43}\end{figure} +We can also try Ridge and Elastic Net regression. Their procedures are exactly the same except that we need to call different functions for different types of regressions. + +For Ridge regression, the only difference is that the objective now becomes $\min\limits_{\beta} \norm{Y - X\beta}^2_2 + \lambda \norm{\beta}^2_2$. (The constraint is now on the sum of squared coefficients.) While Lasso promotes predictor selection, Ridge will decrease the values of coefficients but is unable to force a coefficient to exactly 0. +\begin{figure}[H]\includegraphics[width=\linewidth]{44.1}\end{figure} +\begin{figure}[H]\includegraphics[width=\linewidth]{44.2}\end{figure} +\begin{figure}[H]\includegraphics[width=0.63\linewidth]{45}\end{figure} +Lastly, Elastic Net regression is a combination of Lasso and Ridge--the objective is $\min\limits_{\beta} \norm{Y - X\beta}^2_2 + \lambda \left(\alpha \norm{\beta}_1 + \frac{1 - \alpha}{2} \norm{\beta}^2_2\right)$, and $\alpha = 0.5$ by default. +\begin{figure}[H]\includegraphics[width=1.1\linewidth]{46.1}\end{figure} +\begin{figure}[H]\includegraphics[width=1.1\linewidth]{46.2}\end{figure} +\begin{figure}[H]\includegraphics[width=0.6\linewidth]{47}\end{figure} +Again, we can calculate the adjusted $R^2$ of each of these regressions. You can see that their values are all around 0.576, which is better than the adjusted $R^2$ of our initial OLS model. +\begin{figure}[H]\includegraphics[width=1\linewidth]{48}\end{figure} +\begin{figure}[H]\includegraphics[width=0.6\linewidth]{49}\end{figure} +That's it for a complete procedure to perform linear regression analysis in Python! We have checked the necessary conditions, calculated the coefficients under different models, and compared their performance. If you want to find an even better regression model, perhaps you could go on to try some nonlinear models in the future! +\end{document} diff --git a/LinearRegression/Multiple_Linear_Regression.ipynb b/LinearRegression/Multiple_Linear_Regression.ipynb new file mode 100644 index 0000000..469c337 --- /dev/null +++ b/LinearRegression/Multiple_Linear_Regression.ipynb @@ -0,0 +1,1581 @@ +{ + "nbformat": 4, + "nbformat_minor": 0, + "metadata": { + "colab": { + "name": "Multiple_Linear_Regression", + "provenance": [], + "collapsed_sections": [] + }, + "kernelspec": { + "name": "python3", + "display_name": "Python 3" + } + }, + "cells": [ + { + "cell_type": "code", + "metadata": { + "id": "sQ9c41eD2hZ6", + "colab_type": "code", + "outputId": "0a5416d9-eb79-41b2-9855-34507664c7cd", + "colab": { + "resources": { + "http://localhost:8080/nbextensions/google.colab/files.js": { + "data": "Ly8gQ29weXJpZ2h0IDIwMTcgR29vZ2xlIExMQwovLwovLyBMaWNlbnNlZCB1bmRlciB0aGUgQXBhY2hlIExpY2Vuc2UsIFZlcnNpb24gMi4wICh0aGUgIkxpY2Vuc2UiKTsKLy8geW91IG1heSBub3QgdXNlIHRoaXMgZmlsZSBleGNlcHQgaW4gY29tcGxpYW5jZSB3aXRoIHRoZSBMaWNlbnNlLgovLyBZb3UgbWF5IG9idGFpbiBhIGNvcHkgb2YgdGhlIExpY2Vuc2UgYXQKLy8KLy8gICAgICBodHRwOi8vd3d3LmFwYWNoZS5vcmcvbGljZW5zZXMvTElDRU5TRS0yLjAKLy8KLy8gVW5sZXNzIHJlcXVpcmVkIGJ5IGFwcGxpY2FibGUgbGF3IG9yIGFncmVlZCB0byBpbiB3cml0aW5nLCBzb2Z0d2FyZQovLyBkaXN0cmlidXRlZCB1bmRlciB0aGUgTGljZW5zZSBpcyBkaXN0cmlidXRlZCBvbiBhbiAiQVMgSVMiIEJBU0lTLAovLyBXSVRIT1VUIFdBUlJBTlRJRVMgT1IgQ09ORElUSU9OUyBPRiBBTlkgS0lORCwgZWl0aGVyIGV4cHJlc3Mgb3IgaW1wbGllZC4KLy8gU2VlIHRoZSBMaWNlbnNlIGZvciB0aGUgc3BlY2lmaWMgbGFuZ3VhZ2UgZ292ZXJuaW5nIHBlcm1pc3Npb25zIGFuZAovLyBsaW1pdGF0aW9ucyB1bmRlciB0aGUgTGljZW5zZS4KCi8qKgogKiBAZmlsZW92ZXJ2aWV3IEhlbHBlcnMgZm9yIGdvb2dsZS5jb2xhYiBQeXRob24gbW9kdWxlLgogKi8KKGZ1bmN0aW9uKHNjb3BlKSB7CmZ1bmN0aW9uIHNwYW4odGV4dCwgc3R5bGVBdHRyaWJ1dGVzID0ge30pIHsKICBjb25zdCBlbGVtZW50ID0gZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgnc3BhbicpOwogIGVsZW1lbnQudGV4dENvbnRlbnQgPSB0ZXh0OwogIGZvciAoY29uc3Qga2V5IG9mIE9iamVjdC5rZXlzKHN0eWxlQXR0cmlidXRlcykpIHsKICAgIGVsZW1lbnQuc3R5bGVba2V5XSA9IHN0eWxlQXR0cmlidXRlc1trZXldOwogIH0KICByZXR1cm4gZWxlbWVudDsKfQoKLy8gTWF4IG51bWJlciBvZiBieXRlcyB3aGljaCB3aWxsIGJlIHVwbG9hZGVkIGF0IGEgdGltZS4KY29uc3QgTUFYX1BBWUxPQURfU0laRSA9IDEwMCAqIDEwMjQ7Ci8vIE1heCBhbW91bnQgb2YgdGltZSB0byBibG9jayB3YWl0aW5nIGZvciB0aGUgdXNlci4KY29uc3QgRklMRV9DSEFOR0VfVElNRU9VVF9NUyA9IDMwICogMTAwMDsKCmZ1bmN0aW9uIF91cGxvYWRGaWxlcyhpbnB1dElkLCBvdXRwdXRJZCkgewogIGNvbnN0IHN0ZXBzID0gdXBsb2FkRmlsZXNTdGVwKGlucHV0SWQsIG91dHB1dElkKTsKICBjb25zdCBvdXRwdXRFbGVtZW50ID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQob3V0cHV0SWQpOwogIC8vIENhY2hlIHN0ZXBzIG9uIHRoZSBvdXRwdXRFbGVtZW50IHRvIG1ha2UgaXQgYXZhaWxhYmxlIGZvciB0aGUgbmV4dCBjYWxsCiAgLy8gdG8gdXBsb2FkRmlsZXNDb250aW51ZSBmcm9tIFB5dGhvbi4KICBvdXRwdXRFbGVtZW50LnN0ZXBzID0gc3RlcHM7CgogIHJldHVybiBfdXBsb2FkRmlsZXNDb250aW51ZShvdXRwdXRJZCk7Cn0KCi8vIFRoaXMgaXMgcm91Z2hseSBhbiBhc3luYyBnZW5lcmF0b3IgKG5vdCBzdXBwb3J0ZWQgaW4gdGhlIGJyb3dzZXIgeWV0KSwKLy8gd2hlcmUgdGhlcmUgYXJlIG11bHRpcGxlIGFzeW5jaHJvbm91cyBzdGVwcyBhbmQgdGhlIFB5dGhvbiBzaWRlIGlzIGdvaW5nCi8vIHRvIHBvbGwgZm9yIGNvbXBsZXRpb24gb2YgZWFjaCBzdGVwLgovLyBUaGlzIHVzZXMgYSBQcm9taXNlIHRvIGJsb2NrIHRoZSBweXRob24gc2lkZSBvbiBjb21wbGV0aW9uIG9mIGVhY2ggc3RlcCwKLy8gdGhlbiBwYXNzZXMgdGhlIHJlc3VsdCBvZiB0aGUgcHJldmlvdXMgc3RlcCBhcyB0aGUgaW5wdXQgdG8gdGhlIG5leHQgc3RlcC4KZnVuY3Rpb24gX3VwbG9hZEZpbGVzQ29udGludWUob3V0cHV0SWQpIHsKICBjb25zdCBvdXRwdXRFbGVtZW50ID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQob3V0cHV0SWQpOwogIGNvbnN0IHN0ZXBzID0gb3V0cHV0RWxlbWVudC5zdGVwczsKCiAgY29uc3QgbmV4dCA9IHN0ZXBzLm5leHQob3V0cHV0RWxlbWVudC5sYXN0UHJvbWlzZVZhbHVlKTsKICByZXR1cm4gUHJvbWlzZS5yZXNvbHZlKG5leHQudmFsdWUucHJvbWlzZSkudGhlbigodmFsdWUpID0+IHsKICAgIC8vIENhY2hlIHRoZSBsYXN0IHByb21pc2UgdmFsdWUgdG8gbWFrZSBpdCBhdmFpbGFibGUgdG8gdGhlIG5leHQKICAgIC8vIHN0ZXAgb2YgdGhlIGdlbmVyYXRvci4KICAgIG91dHB1dEVsZW1lbnQubGFzdFByb21pc2VWYWx1ZSA9IHZhbHVlOwogICAgcmV0dXJuIG5leHQudmFsdWUucmVzcG9uc2U7CiAgfSk7Cn0KCi8qKgogKiBHZW5lcmF0b3IgZnVuY3Rpb24gd2hpY2ggaXMgY2FsbGVkIGJldHdlZW4gZWFjaCBhc3luYyBzdGVwIG9mIHRoZSB1cGxvYWQKICogcHJvY2Vzcy4KICogQHBhcmFtIHtzdHJpbmd9IGlucHV0SWQgRWxlbWVudCBJRCBvZiB0aGUgaW5wdXQgZmlsZSBwaWNrZXIgZWxlbWVudC4KICogQHBhcmFtIHtzdHJpbmd9IG91dHB1dElkIEVsZW1lbnQgSUQgb2YgdGhlIG91dHB1dCBkaXNwbGF5LgogKiBAcmV0dXJuIHshSXRlcmFibGU8IU9iamVjdD59IEl0ZXJhYmxlIG9mIG5leHQgc3RlcHMuCiAqLwpmdW5jdGlvbiogdXBsb2FkRmlsZXNTdGVwKGlucHV0SWQsIG91dHB1dElkKSB7CiAgY29uc3QgaW5wdXRFbGVtZW50ID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQoaW5wdXRJZCk7CiAgaW5wdXRFbGVtZW50LmRpc2FibGVkID0gZmFsc2U7CgogIGNvbnN0IG91dHB1dEVsZW1lbnQgPSBkb2N1bWVudC5nZXRFbGVtZW50QnlJZChvdXRwdXRJZCk7CiAgb3V0cHV0RWxlbWVudC5pbm5lckhUTUwgPSAnJzsKCiAgY29uc3QgcGlja2VkUHJvbWlzZSA9IG5ldyBQcm9taXNlKChyZXNvbHZlKSA9PiB7CiAgICBpbnB1dEVsZW1lbnQuYWRkRXZlbnRMaXN0ZW5lcignY2hhbmdlJywgKGUpID0+IHsKICAgICAgcmVzb2x2ZShlLnRhcmdldC5maWxlcyk7CiAgICB9KTsKICB9KTsKCiAgY29uc3QgY2FuY2VsID0gZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgnYnV0dG9uJyk7CiAgaW5wdXRFbGVtZW50LnBhcmVudEVsZW1lbnQuYXBwZW5kQ2hpbGQoY2FuY2VsKTsKICBjYW5jZWwudGV4dENvbnRlbnQgPSAnQ2FuY2VsIHVwbG9hZCc7CiAgY29uc3QgY2FuY2VsUHJvbWlzZSA9IG5ldyBQcm9taXNlKChyZXNvbHZlKSA9PiB7CiAgICBjYW5jZWwub25jbGljayA9ICgpID0+IHsKICAgICAgcmVzb2x2ZShudWxsKTsKICAgIH07CiAgfSk7CgogIC8vIENhbmNlbCB1cGxvYWQgaWYgdXNlciBoYXNuJ3QgcGlja2VkIGFueXRoaW5nIGluIHRpbWVvdXQuCiAgY29uc3QgdGltZW91dFByb21pc2UgPSBuZXcgUHJvbWlzZSgocmVzb2x2ZSkgPT4gewogICAgc2V0VGltZW91dCgoKSA9PiB7CiAgICAgIHJlc29sdmUobnVsbCk7CiAgICB9LCBGSUxFX0NIQU5HRV9USU1FT1VUX01TKTsKICB9KTsKCiAgLy8gV2FpdCBmb3IgdGhlIHVzZXIgdG8gcGljayB0aGUgZmlsZXMuCiAgY29uc3QgZmlsZXMgPSB5aWVsZCB7CiAgICBwcm9taXNlOiBQcm9taXNlLnJhY2UoW3BpY2tlZFByb21pc2UsIHRpbWVvdXRQcm9taXNlLCBjYW5jZWxQcm9taXNlXSksCiAgICByZXNwb25zZTogewogICAgICBhY3Rpb246ICdzdGFydGluZycsCiAgICB9CiAgfTsKCiAgaWYgKCFmaWxlcykgewogICAgcmV0dXJuIHsKICAgICAgcmVzcG9uc2U6IHsKICAgICAgICBhY3Rpb246ICdjb21wbGV0ZScsCiAgICAgIH0KICAgIH07CiAgfQoKICBjYW5jZWwucmVtb3ZlKCk7CgogIC8vIERpc2FibGUgdGhlIGlucHV0IGVsZW1lbnQgc2luY2UgZnVydGhlciBwaWNrcyBhcmUgbm90IGFsbG93ZWQuCiAgaW5wdXRFbGVtZW50LmRpc2FibGVkID0gdHJ1ZTsKCiAgZm9yIChjb25zdCBmaWxlIG9mIGZpbGVzKSB7CiAgICBjb25zdCBsaSA9IGRvY3VtZW50LmNyZWF0ZUVsZW1lbnQoJ2xpJyk7CiAgICBsaS5hcHBlbmQoc3BhbihmaWxlLm5hbWUsIHtmb250V2VpZ2h0OiAnYm9sZCd9KSk7CiAgICBsaS5hcHBlbmQoc3BhbigKICAgICAgICBgKCR7ZmlsZS50eXBlIHx8ICduL2EnfSkgLSAke2ZpbGUuc2l6ZX0gYnl0ZXMsIGAgKwogICAgICAgIGBsYXN0IG1vZGlmaWVkOiAkewogICAgICAgICAgICBmaWxlLmxhc3RNb2RpZmllZERhdGUgPyBmaWxlLmxhc3RNb2RpZmllZERhdGUudG9Mb2NhbGVEYXRlU3RyaW5nKCkgOgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAnbi9hJ30gLSBgKSk7CiAgICBjb25zdCBwZXJjZW50ID0gc3BhbignMCUgZG9uZScpOwogICAgbGkuYXBwZW5kQ2hpbGQocGVyY2VudCk7CgogICAgb3V0cHV0RWxlbWVudC5hcHBlbmRDaGlsZChsaSk7CgogICAgY29uc3QgZmlsZURhdGFQcm9taXNlID0gbmV3IFByb21pc2UoKHJlc29sdmUpID0+IHsKICAgICAgY29uc3QgcmVhZGVyID0gbmV3IEZpbGVSZWFkZXIoKTsKICAgICAgcmVhZGVyLm9ubG9hZCA9IChlKSA9PiB7CiAgICAgICAgcmVzb2x2ZShlLnRhcmdldC5yZXN1bHQpOwogICAgICB9OwogICAgICByZWFkZXIucmVhZEFzQXJyYXlCdWZmZXIoZmlsZSk7CiAgICB9KTsKICAgIC8vIFdhaXQgZm9yIHRoZSBkYXRhIHRvIGJlIHJlYWR5LgogICAgbGV0IGZpbGVEYXRhID0geWllbGQgewogICAgICBwcm9taXNlOiBmaWxlRGF0YVByb21pc2UsCiAgICAgIHJlc3BvbnNlOiB7CiAgICAgICAgYWN0aW9uOiAnY29udGludWUnLAogICAgICB9CiAgICB9OwoKICAgIC8vIFVzZSBhIGNodW5rZWQgc2VuZGluZyB0byBhdm9pZCBtZXNzYWdlIHNpemUgbGltaXRzLiBTZWUgYi82MjExNTY2MC4KICAgIGxldCBwb3NpdGlvbiA9IDA7CiAgICB3aGlsZSAocG9zaXRpb24gPCBmaWxlRGF0YS5ieXRlTGVuZ3RoKSB7CiAgICAgIGNvbnN0IGxlbmd0aCA9IE1hdGgubWluKGZpbGVEYXRhLmJ5dGVMZW5ndGggLSBwb3NpdGlvbiwgTUFYX1BBWUxPQURfU0laRSk7CiAgICAgIGNvbnN0IGNodW5rID0gbmV3IFVpbnQ4QXJyYXkoZmlsZURhdGEsIHBvc2l0aW9uLCBsZW5ndGgpOwogICAgICBwb3NpdGlvbiArPSBsZW5ndGg7CgogICAgICBjb25zdCBiYXNlNjQgPSBidG9hKFN0cmluZy5mcm9tQ2hhckNvZGUuYXBwbHkobnVsbCwgY2h1bmspKTsKICAgICAgeWllbGQgewogICAgICAgIHJlc3BvbnNlOiB7CiAgICAgICAgICBhY3Rpb246ICdhcHBlbmQnLAogICAgICAgICAgZmlsZTogZmlsZS5uYW1lLAogICAgICAgICAgZGF0YTogYmFzZTY0LAogICAgICAgIH0sCiAgICAgIH07CiAgICAgIHBlcmNlbnQudGV4dENvbnRlbnQgPQogICAgICAgICAgYCR7TWF0aC5yb3VuZCgocG9zaXRpb24gLyBmaWxlRGF0YS5ieXRlTGVuZ3RoKSAqIDEwMCl9JSBkb25lYDsKICAgIH0KICB9CgogIC8vIEFsbCBkb25lLgogIHlpZWxkIHsKICAgIHJlc3BvbnNlOiB7CiAgICAgIGFjdGlvbjogJ2NvbXBsZXRlJywKICAgIH0KICB9Owp9CgpzY29wZS5nb29nbGUgPSBzY29wZS5nb29nbGUgfHwge307CnNjb3BlLmdvb2dsZS5jb2xhYiA9IHNjb3BlLmdvb2dsZS5jb2xhYiB8fCB7fTsKc2NvcGUuZ29vZ2xlLmNvbGFiLl9maWxlcyA9IHsKICBfdXBsb2FkRmlsZXMsCiAgX3VwbG9hZEZpbGVzQ29udGludWUsCn07Cn0pKHNlbGYpOwo=", + "ok": true, + "headers": [ + [ + "content-type", + "application/javascript" + ] + ], + "status": 200, + "status_text": "" + } + }, + "base_uri": "https://localhost:8080/", + "height": 493 + } + }, + "source": [ + "import pandas as pd\n", + "import numpy as np\n", + "from scipy import stats\n", + "import statsmodels.api as sm\n", + "from statsmodels.stats.outliers_influence import variance_inflation_factor\n", + "from sklearn import datasets\n", + "from sklearn import linear_model\n", + "from sklearn import metrics\n", + "from sklearn.preprocessing import StandardScaler\n", + "from sklearn.model_selection import GridSearchCV\n", + "import matplotlib.pyplot as plt\n", + "import seaborn as sn\n", + "from random import uniform\n", + "from google.colab import files\n", + "import io\n", + "\n", + "uploaded = files.upload()\n", + "dataset = pd.read_csv(io.BytesIO(uploaded['Real estate.csv']), delimiter=',')\n", + "\n", + "dataset.head()\n", + "\n", + "dataset.drop(['No'], axis=1)" + ], + "execution_count": 67, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " Upload widget is only available when the cell has been executed in the\n", + " current browser session. Please rerun this cell to enable.\n", + " \n", + " " + ], + "text/plain": [ + "" + ] + }, + "metadata": { + "tags": [] + } + }, + { + "output_type": "stream", + "text": [ + "Saving Real estate.csv to Real estate (3).csv\n" + ], + "name": "stdout" + }, + { + "output_type": "execute_result", + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
X1 transaction dateX2 house ageX3 distance to the nearest MRT stationX4 number of convenience storesX5 latitudeX6 longitudeY house price of unit area
02012.91732.084.878821024.98298121.5402437.9
12012.91719.5306.59470924.98034121.5395142.2
22013.58313.3561.98450524.98746121.5439147.3
32013.50013.3561.98450524.98746121.5439154.8
42012.8335.0390.56840524.97937121.5424543.1
........................
4092013.00013.74082.01500024.94155121.5038115.4
4102012.6675.690.45606924.97433121.5431050.0
4112013.25018.8390.96960724.97923121.5398640.6
4122013.0008.1104.81010524.96674121.5406752.5
4132013.5006.590.45606924.97433121.5431063.9
\n", + "

414 rows × 7 columns

\n", + "
" + ], + "text/plain": [ + " X1 transaction date X2 house age \\\n", + "0 2012.917 32.0 \n", + "1 2012.917 19.5 \n", + "2 2013.583 13.3 \n", + "3 2013.500 13.3 \n", + "4 2012.833 5.0 \n", + ".. ... ... \n", + "409 2013.000 13.7 \n", + "410 2012.667 5.6 \n", + "411 2013.250 18.8 \n", + "412 2013.000 8.1 \n", + "413 2013.500 6.5 \n", + "\n", + " X3 distance to the nearest MRT station X4 number of convenience stores \\\n", + "0 84.87882 10 \n", + "1 306.59470 9 \n", + "2 561.98450 5 \n", + "3 561.98450 5 \n", + "4 390.56840 5 \n", + ".. ... ... \n", + "409 4082.01500 0 \n", + "410 90.45606 9 \n", + "411 390.96960 7 \n", + "412 104.81010 5 \n", + "413 90.45606 9 \n", + "\n", + " X5 latitude X6 longitude Y house price of unit area \n", + "0 24.98298 121.54024 37.9 \n", + "1 24.98034 121.53951 42.2 \n", + "2 24.98746 121.54391 47.3 \n", + "3 24.98746 121.54391 54.8 \n", + "4 24.97937 121.54245 43.1 \n", + ".. ... ... ... \n", + "409 24.94155 121.50381 15.4 \n", + "410 24.97433 121.54310 50.0 \n", + "411 24.97923 121.53986 40.6 \n", + "412 24.96674 121.54067 52.5 \n", + "413 24.97433 121.54310 63.9 \n", + "\n", + "[414 rows x 7 columns]" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 67 + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "ODM2nIVEbwWY", + "colab_type": "code", + "outputId": "8fa9d34a-20df-4137-f852-e3b96e28be19", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 418 + } + }, + "source": [ + "y = dataset['Y house price of unit area'].values # target vector\n", + "print(dataset['Y house price of unit area'].describe())\n", + "sn.distplot(y, bins = 13)\n", + "plt.show()" + ], + "execution_count": 68, + "outputs": [ + { + "output_type": "stream", + "text": [ + "count 414.000000\n", + "mean 37.980193\n", + "std 13.606488\n", + "min 7.600000\n", + "25% 27.700000\n", + "50% 38.450000\n", + "75% 46.600000\n", + "max 117.500000\n", + "Name: Y house price of unit area, dtype: float64\n" + ], + "name": "stdout" + }, + { + "output_type": "display_data", + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD4CAYAAADlwTGnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXhc9X3v8fd3ZrTvqy1rsWRb3sCswgaCSYAsJgGcPoEbCA3khpabJrS5Tfv0kqc39DY3fdrcLmnSEFoakpI0hC1J4zYQEzCEXVgmBm+yLcu2LMmy9t1aZuZ7/5gjIoRkja0ZnVm+r+eZRzPn/M7Rd47l+cz5nXN+R1QVY4wxycfjdgHGGGPcYQFgjDFJygLAGGOSlAWAMcYkKQsAY4xJUj63CzgbxcXFWl1d7XYZxhgTV3bt2tWtqiUzp8dVAFRXV9PQ0OB2GcYYE1dE5Phs060LyBhjkpQFgDHGJKmwAkBEtojIQRFpEpF7Z5mfJiKPOfPrRaTamb5RRHY7j7dE5HfCXacxxpjomjcARMQL3A9cD6wHbhOR9TOa3QX0qeoq4BvA153pe4E6Vb0I2AL8i4j4wlynMcaYKApnD2Aj0KSqzao6ATwKbJ3RZivwsPP8SeA6ERFVHVVVvzM9HZgaeCicdRpjjImicAKgHDgx7XWrM23WNs4H/gBQBCAim0RkH7AH+JwzP5x14ix/t4g0iEhDV1dXGOUaY4wJR9QPAqtqvaqeB1wGfFlE0s9y+QdVtU5V60pK3nMaqzHGmHMUTgC0AZXTXlc402ZtIyI+IA/omd5AVQ8Aw8D5Ya7TJAFVxYYkN8Yd4QTATqBWRGpEJBW4Fdg2o8024E7n+c3ADlVVZxkfgIgsB9YCx8Jcp0lwLT2jXP/Nl7ju73/NT3a14g8E3S7JmKQybwA4ffb3ANuBA8DjqrpPRL4qIjc5zR4CikSkCfgSMHVa51XAWyKyG/gZ8HlV7Z5rnZF8Yya2vdrUzU33v8zJgTFSfR7+5Im32PLNl+gfnXC7NGOShsTT7nddXZ3aUBDxb3/7IDd9+2VqirP41zvqqCrM5Om9Hfzhj9/kM1fWcN+NdkawMZEkIrtUtW7mdLsS2Cy6f3z2EBmpXp743BVUF2fh8Qgfu6CMT15WyQ9eO8bR7hG3SzQmKVgAmEW1t22AZ/af4q6rasjPTH3XvD/+0GrSfB7+5ukDLlVnTHKxADCL6pvPHSY33cdnr6p5z7zSnHT+4AMr2b7vFPXNPbMsbYyJJAsAs2j2tg3wq/2n+L3NK8hNT5m1ze9tXkFxdio/eG3W0WuNMREUV/cDMLHnkfqWsNs+3nCC9BQP2Wm+WZf71KYq0lO8fGxDGY81nGB43E92mv2JGhMttgdgFsWEP8j+9kE2lOeRnuI9Y9sbL1zG2GSQZ/efWqTqjElOFgBmUTR2DDIRCHJBRf68bS+pKmBZXjr/+Vb7IlRmTPKyADCL4q3WAXLSfdQUZ83b1uMRbrhwGS8e7rILw4yJIgsAE3WnJwIcOjXEBeV5eETCWubGC5YxGVC27+uIcnXGJC8LABN1+9oHCASVCyvn7/6Zcn55LtVFmWyzbiBjosYCwETdW639FGalUp6fEfYyIsL1G8p4vbmXobHJKFZnTPKyADBRNTrup7lrhAsq8pAwu3+mbF5VTCCovHG0N0rVGZPcLABMVB3qHEKBdUtzz3rZS5YXkObz8NLh7sgXZoyxADDR1dgxRFaaj/KC8Lt/pqSneNlYU8grTRYAxkSDBYCJmkBQOXRqiLVLcsI++2emq1YVc7hzmFODYxGuzhhjAWCi5njvCGOTQdYszTnndbxvVTGA7QUYEwUWACZqDnYM4RWhtjT7nNexviyXwqxUXrYAMCbiLABM1DR2DFFTnEXaPGP/nInHI1y5soiXD3fbzeONiTALABMVPcPjdA2NL6j7Z8pVq4rpHBrncOdwBCozxkyxADBRcfDUEABrIxAAV64MHQewm8QYE1kWACYqDnYMUZydRlF22oLXVVmYQWlOGg3H+yJQmTFmigWAibjxyQDN3SMR+fYPoWEhLqsupOGYBYAxkWQBYCKuqWuYQFAjFgAAddUFtPWfpr3/dMTWaUyyswAwEdfYMUR6ioflRfOP/R+uuuWFANYNZEwEWQCYiAqqcqhjiNrSHLyec7v6dzbrynLITPXScMwGhjMmUiwATES1959maNwf0e4fAJ/XwyVVBXYcwJgICisARGSLiBwUkSYRuXeW+Wki8pgzv15Eqp3pHxKRXSKyx/l57bRlXnDWudt5lEbqTRn3NHYMIcDqJZENAAgdB2jsGGTQ7g9gTET45msgIl7gfuBDQCuwU0S2qer+ac3uAvpUdZWI3Ap8Hfgk0A3cqKrtInI+sB0on7bc7araEKH3YmJAY8cglYWZZKXN+6f1Ho/Ut5xx/uBpP0GFf3jm0LwB86lNVWf9+41JNuHsAWwEmlS1WVUngEeBrTPabAUedp4/CVwnIqKqv1HVqXv67QMyRGThJ4abmNQ3MkF7/xjnLTv7sf/DUVmQgUfgeM9oVNZvTLIJJwDKgRPTXrfy7m/x72qjqn5gACia0eYTwJuqOj5t2ved7p+vyNneLsrEnH3tA0BoALdoSEvxsjQvneO9I1FZvzHJZlEOAovIeYS6hf7HtMm3q+oGYLPz+PQcy94tIg0i0tDV1RX9Ys0529c+SFleekSu/p1LVWEmrX2nCdrAcMYsWDgB0AZUTntd4UybtY2I+IA8oMd5XQH8DLhDVY9MLaCqbc7PIeARQl1N76GqD6pqnarWlZSUhPOejAsGxyZp6R2NWvfPlKrCTCb8QbtBjDEREE4A7ARqRaRGRFKBW4FtM9psA+50nt8M7FBVFZF84BfAvar6ylRjEfGJSLHzPAW4Adi7sLdi3LS/fRAFzluWF9XfU1UYurispdeOAxizUPMGgNOnfw+hM3gOAI+r6j4R+aqI3OQ0ewgoEpEm4EvA1Kmi9wCrgPtmnO6ZBmwXkbeB3YT2IP41km/MLK797YMUZ6dSmhPdY/wFmSlkpflosQPBxixYWOfqqepTwFMzpt037fkYcMssy30N+Nocq700/DJNLBsd99PcPczm2hKifSxfRKgqzLQ9AGMiwK4ENgu2u7WfoMIFFdHt/plSVZhJz8gEI+P+Rfl9xiQqCwCzIKpKw7E+yvMzKMvLWJTfWVWYCcAJ2wswZkEsAMyCtPWfpmNwjLrqgkX7neX5oQvCWvosAIxZCAsAsyA7j/WR4hUurMhftN+Z6vOwNC/djgMYs0AWAOacjYz7eau1nw3l+aSneBf1d9sFYcYsnAWAOWe/ePskE/4gly1i988UuyDMmIWzADDnZDIQ5DsvNLE0N/2dg7KLyS4IM2bhLADMOXmioZVjPaN8aP2SqJ/7Pxu7IMyYhbMAMGdtbDLAN587xCVV+RG/81e47IIwYxbOAsCctR+8doxTg+P82Za1rnz7n1JVkGEXhBmzABYA5qyc6B3l2zuauHp1CZevmHnLh8VVWWQXhBmzEBYAJmxjkwH+4Ee7UOCrN53ndjlU5GfaBWHGLMDZ37jVJK2/+Pk+9rYN8t076qguznK7HLsgzJgFsj0AM69AUPnrpw7wWMMJ/vDaVXxw/RK3S3qHXRBmzLmzPYAk9Eh9S9htT08EeKyhhUOnhtlUU8iS3PSzWj7aqgozeb25l1ODY4s2GJ0xicICwMwqqMpbJ/rZvq+DkfEAv3NROZfVFLpd1ntUFoQOBLf0jloAGHOWLADMe7T0jvKLt9s50Xea8vwMbt+0nEoXrvYNR2FWKlmpXlp6RtlU4+5ZScbEGwsA846hsUme3tvB7hP95KT7uPnSCi6qzMfj4rn+87ELwow5dxYABoC2vtP88PVjjE4E+MCaEt6/uoQ03+KO8HmuqgozOdAxxMi4n6w0+5M2Jlz2v8Wwt22AJ3adICvVxx98YGXc9aVPvyBsbVmuy9UYEz/sNNAkd7xnhEd3trA0Nz0uP/zBLggz5lzZHkASGx738+M3WsjPTOUzV9aQkRofXT4z2QVhxpwb2wNIUkFVHtvZwuhEgNs3VcXth/+UygK7IMyYs2UBkKTqj/ZypGuErRcti8tun5nsDmHGnD0LgCQ04Q/yfGMnNcVZXFK1+LdzjIapu5JZN5Ax4bMASEKvHulmeNzPh126m1c0TL8gzBgTnrACQES2iMhBEWkSkXtnmZ8mIo858+tFpNqZ/iER2SUie5yf105b5lJnepOIfEsS5ZMoxg2MTvLi4S7WLs1heZH7I3pGil0QZszZmzcARMQL3A9cD6wHbhOR9TOa3QX0qeoq4BvA153p3cCNqroBuBP44bRlHgB+H6h1HlsW8D5MmB586Qhjk0E+FEMjekbK8qIsekYmGBqbdLsUY+JCOHsAG4EmVW1W1QngUWDrjDZbgYed508C14mIqOpvVLXdmb4PyHD2FsqAXFV9XVUV+AHw8QW/G3NGY5MBflTfwnnLchPiwO9M1c4FYcetG8iYsIQTAOXAiWmvW51ps7ZRVT8wAMwcmesTwJuqOu60b51nnSbCntl/iv7RSTbG4KiekbCsIIMUr3CsZ8TtUoyJC4tyIZiInEeoW+jD57Ds3cDdAFVVVRGuLLk8trOFioIMVpZku11KVPg8HioKMi0AjAlTOHsAbUDltNcVzrRZ24iID8gDepzXFcDPgDtU9ci09hXzrBMAVX1QVetUta6kpCSMcs1sWnpGeaWph0/WVcb06J4LVV2Uxcn+MTsOYEwYwgmAnUCtiNSISCpwK7BtRptthA7yAtwM7FBVFZF84BfAvar6ylRjVT0JDIrI5c7ZP3cAP1/gezFn8FhDCx6Bm+sq5m8cx6qLM1HgzZZ+t0sxJubNGwBOn/49wHbgAPC4qu4Tka+KyE1Os4eAIhFpAr4ETJ0qeg+wCrhPRHY7j1Jn3ueB7wJNwBHg6Ui9KfNu/kCQJxpa+cCa0oQ8+DtdVWFoYLidR3vdLsWYmBfWMQBVfQp4asa0+6Y9HwNumWW5rwFfm2OdDcD5Z1OsOTf1R3vpHBrnlksT+9s/QJrPS1leBm8cswAwZj52JXAS2L6vg/QUDx9YUzp/4wRQXZTJ7hP9jPsDbpdiTEyzAEhwqsoz+05xdW1J3I/4Ga7q4iwm/EHebh1wuxRjYpoFQIJ7u3WAjsExPnLeUrdLWTQ1zhAXrx/pcbkSY2KbBUCC276vA69HuG5dcnT/AGSm+VhXlsurFgDGnJEFQILbvq+DTTWF5Gemul3KorpyZRG7WvoYm7TjAMbMxQIggTV1DnOkaySpun+mXLmyiAl/kDdb+twuxZiYZQGQwH61/xRAQo78OZ+NNYV4PcJr1g1kzJwsABLYrw91snZpDsvyE/vir9nkpKdwfnmeBYAxZ2ABkKBGJ/zsOt7H+1cn7/hJV64sYveJfkbG/W6XYkxMsgBIUPXNvUwGlKtqi90uxTVXrizCH1R22lXBxszKAiBBvXi4izSfh8uqE3Ps/3DULS8kxWvHAYyZiwVAgnr5cDcbawpJT0mOq39nk5Hq5eLKAl5rtgAwZjYWAAno5MBpDncOszmJu3+mXLGyiL1tAwyM2v0BjJnJAiABvXS4G4DNtcl7AHjKlSuLCCrUH7W9AGNmsgBIQC8f7qY4O421S3PcLsV1F1Xlk57isW4gY2ZhAZBgVJVXmrrZXFuMJPCtH8OV5vNyWXWhHQg2ZhYWAAmmqXOYnpEJrlhR5HYpMePyFUU0dgzRPTzudinGxBQLgART79wKcWNN8p7+OdOVK0Nh+Lp1AxnzLhYACeaNo70syU1jeVGm26XEjA3leWSn+awbyJgZLAASiKpSf7SHjTVF1v8/jc/rYVONHQcwZiYLgATS0jvKqcFx6/6ZxRUri2juHqG9/7TbpRgTMywAEkh9c6j//3ILgPeYuibiZecaCWOMBUBCqT/aS2FWKqtKs90uJeasXpJNaU4aLx7ucrsUY2KGBUACeeNYDxurC63/fxYiwubaEl5u6iYQVLfLMSYmWAAkiPb+05zoPW39/2dw9epi+kcn2dc+4HYpxsQEC4AE8Yad/z+v960KDY734iHrBjIGwgwAEdkiIgdFpElE7p1lfpqIPObMrxeRamd6kYg8LyLDIvLtGcu84Kxzt/MojcQbSlb1R3vISfexrizX7VJiVnF2GueX5/KiHQg2BggjAETEC9wPXA+sB24TkfUzmt0F9KnqKuAbwNed6WPAV4A/nWP1t6vqRc6j81zegAmpP9rLZdWhG6GbuW2uLeHN430M220ijQlrD2Aj0KSqzao6ATwKbJ3RZivwsPP8SeA6ERFVHVHVlwkFgYmSrqFxmrtGrPsnDJtri/EHldftojBj8IXRphw4Me11K7Bprjaq6heRAaAImG9f+/siEgB+AnxNVe30jDk8Ut8y57w9baGDmgOjk2dsZ+DS5QVkpHh56XAXH1y/xO1yjHGVmweBb1fVDcBm5/Hp2RqJyN0i0iAiDV1ddvBuNke7R0j1eliWn+F2KTEvzefl8hWFdhzAGMILgDagctrrCmfarG1ExAfkAWfcx1bVNufnEPAIoa6m2do9qKp1qlpXUmJ3uJrNse4Rqooyrf8/TFevLuFo9wgnekfdLsUYV4UTADuBWhGpEZFU4FZg24w224A7nec3AzvO1J0jIj4RKXaepwA3AHvPtngDoxN+OgbHqC7KcruUuDE1LMRLthdgkty8xwCcPv17gO2AF/iequ4Tka8CDaq6DXgI+KGINAG9hEICABE5BuQCqSLyceDDwHFgu/Ph7wWeBf41ou8sSRzvCX2LrSm2AJjuTMdCVJW8jBR+VH88rHV9alNVpMoyJqaEcxAYVX0KeGrGtPumPR8Dbplj2eo5VntpeCWaMznaPYLPI1QUWP9/uESE2tJs9rYPEAiqdZ2ZpGVXAse5o90jVBRkkuK1f8qzsao0m7HJIG19dhzAJC/71Ihj45MB2vtPW/fPOVhVmo0AhzuH3S7FGNdYAMSx472jKNb/fy4yU32UF2RYAJikZgEQx452j+ARqCq0+/+ei9rSbFr7Rjk9EXC7FGNcYQEQx452j1Cen0Gqz/4Zz8Wq0hyCCs3dthdgkpN9csSpCX+Qtj7r/1+IqsJM0nweDp+yADDJyQIgTp3oGyWgSrUFwDnzeoQVJdkc7hzChqEyycgCIE4d7R5BwK4AXqDa0mz6RifpHZlwuxRjFp0FQJw61j1CWV466Slet0uJa7Wl2YCdDmqSkwVAHPIHg7T0jlr/fwQUZqVSkJnC4VNDbpdizKKzAIhDJ3pP4w9a/38khIaFyOFI9wiBoB0HMMnFAiAONXcPI9gFYJFSuySbCX9or8qYZGIBEIeau0L9/5mpYY3lZ+axsiQbj2DdQCbpWADEmclAkBO9o6woyXa7lISRnuKlqjCTQ50WACa5WADEmZbeUfxBZYV1/0TU6iU5tPePMTQ26XYpxiwaC4A409wV6v+3A8CRVbskB4AmOx3UJBELgDjT3DVCeUGGnf8fYWV56WSl+ThoxwFMErEAiCMT/iCtfadZUWz9/5HmEWF1aTZNncMEbVgIkyQsAOLI8Z4RAqqsKLHun2hYvSSH0YkAbX2n3S7FmEVhARBHmp3x/5cX2fj/0TB1lzA7G8gkCwuAONLcNUxFQSZpPuv/j4asNOcuYTY8tEkSFgBxYmwyQFv/aev+ibLVS3I40TvK6ITf7VKMiToLgDhxrGeEoIauWjXRs7o0G8VOBzXJwQIgTjR3jeD1iN3/N8oqCjPJSPFyyLqBTBKwAIgTzd3DVBVmkuK1f7Jo8oiwqjSbw6fsLmEm8dmnSRwYGJ3kZP+YDf+wSFYvyWFo3E/H4JjbpRgTVRYAcaD+aA8KNgDcIqldEtrO1g1kEl1YASAiW0TkoIg0ici9s8xPE5HHnPn1IlLtTC8SkedFZFhEvj1jmUtFZI+zzLdERCLxhhLRa809pHiFyoIMt0tJCrnpKZTlpXPIhoUwCW7eABARL3A/cD2wHrhNRNbPaHYX0Keqq4BvAF93po8BXwH+dJZVPwD8PlDrPLacyxtIBq8d6WF5YRY+6/9fNLWlORzvGWFsMuB2KcZETTifKBuBJlVtVtUJ4FFg64w2W4GHnedPAteJiKjqiKq+TCgI3iEiZUCuqr6uoSNtPwA+vpA3kqg6h8Zo7Biy8/8X2ZqlOQTVbhZvEls4AVAOnJj2utWZNmsbVfUDA0DRPOtsnWedAIjI3SLSICINXV1dYZSbWF5p6gZ+O1yxWRxVzumgjScH3S7FmKiJ+T4FVX1QVetUta6kpMTtchbdS4e6KcxKpSwv3e1SkorXI6xZmsPBU0N2s3iTsMIJgDagctrrCmfarG1ExAfkAT3zrLNinnUmPVXlxcPdXLWqGI8dI190a5eGRgd9s6XP7VKMiYpwAmAnUCsiNSKSCtwKbJvRZhtwp/P8ZmCHnuEqGlU9CQyKyOXO2T93AD8/6+oT3IGTQ3QPj7O5ttjtUpLS6iU5eASePXDK7VKMiYp5A8Dp078H2A4cAB5X1X0i8lURuclp9hBQJCJNwJeAd04VFZFjwD8AnxGR1mlnEH0e+C7QBBwBno7MW0ocLx0OHfPYXJt8XV+xID3FS01xFs8d6HS7FGOiwhdOI1V9CnhqxrT7pj0fA26ZY9nqOaY3AOeHW2gyeulwN6uXZLPU+v9ds3ZpLr/Yc5LjPSMsL7IzsUxiifmDwMnq9ESAN4712rd/l60rywXgWdsLMAnIAiBG1R/tYcIftP5/lxVmpVJbms1zdhzAJCALgBj1wsEu0lM8XL7iTJdTmMVw3bolvHG0l8GxSbdLMSaiLABikKqyo7GTK1cWk55it3902wfXleIPKr8+mHwXIprEZgEQg450jdDSO8o1a0vdLsUAF1cVUJiVat1AJuFYAMSg5xtDBxyvtQCICV6P8IE1JTx/sAt/IOh2OcZEjAVADNrR2MmaJTmU59vwz7Hig+uWMHB6kl3H7apgkzgsAGLM4NgkO4/1WvdPjNlcW0yKV3iu0U4HNYnDAiDGvHy4G39QuWaNnf8fS3LSU7h8RRHP7j9l9wo2CcMCIMbsaOwkN93HpcsL3C7FzPCR85bS3D1it4o0CcMCIIYEg8oLBzu5enWJ3f0rBn3kvKWIwFN7TrpdijERYZ8yMWRP2wDdwxN29k+MKslJY2N1IU/vtQAwicECIIbsaOxEBN6/2vr/Y9VHN5Rx6NQwTZ12w3gT/ywAYsjzBzu5qDKfouw0t0sxc9hy/lIAnt7T4XIlxiycBUCM6Boa5+3WAa5dY90/sWxJbjp1ywt4aq8FgIl/FgAx4oWDofPL7fz/2Hf9hjIOnBzkaPeI26UYsyAWADHi+YOdLMlN47xluW6XYubx0Q2hs4G27W53uxRjFsQCIAZMBoK8dKiba9aUInbz95hXlpfBpppCfr67zS4KM3HNAiAG1Df3MjTut+6fOPLxi8pp7h5hT9uA26UYc84sAGLA9n0dpKd4uNpu/xg3rt9QRqrXw89+0+Z2KcacMwsAlwWDyq/2n+L9q0vISLWbv8SLvIwUrllbwn++ddKGiDZxywLAZXvaBugYHOPD65e6XYo5Sx+/qJzu4XFePdLjdinGnBMLAJdt39eB1yNct876/+PNNWtLyUn3WTeQiVsWAC57Zv8pNtUUkp+Z6nYp5iylp3i56cJlPL33JAOn7YbxJv743C4gkT1S33LG+V1D4zR1DrNuac68bU1suvWyKn5U38K23W18+opqt8sx5qzYHoCL9p8cBGBdmV38Fa/OL89lfVkuj+484XYpxpy1sAJARLaIyEERaRKRe2eZnyYijznz60Wketq8LzvTD4rIR6ZNPyYie0Rkt4g0ROLNxJu9bQNUFmRY908cExFu21jJvvZB9to1ASbOzBsAIuIF7geuB9YDt4nI+hnN7gL6VHUV8A3g686y64FbgfOALcB3nPVNuUZVL1LVugW/kzjTMzxOW/9pzi/Pc7sUs0A3XVROms/Dj9+wbjwTX8LZA9gINKlqs6pOAI8CW2e02Qo87Dx/ErhOQmMabAUeVdVxVT0KNDnrS3pT3xYtAOJfXkYKH9tQxrbd7YyM+90ux5iwhRMA5cD0Ds5WZ9qsbVTVDwwARfMsq8AzIrJLRO6e65eLyN0i0iAiDV1dXWGUGx/2tIe6fwqs+ych3H75cobG/fzkzVa3SzEmbG4eBL5KVS8h1LX0BRG5erZGqvqgqtapal1JSWIMldAzPE57/xgb7Nt/wrh0eQEXVebz/VeOEQzaAHEmPoQTAG1A5bTXFc60WduIiA/IA3rOtKyqTv3sBH5GEnUN7bHun4T02atqONo9wo7GTrdLMSYs4QTATqBWRGpEJJXQQd1tM9psA+50nt8M7NDQOLnbgFuds4RqgFrgDRHJEpEcABHJAj4M7F3424kPe9oGqCrMtLN/Esz15y+lLC+d771y1O1SjAnLvAHg9OnfA2wHDgCPq+o+EfmqiNzkNHsIKBKRJuBLwL3OsvuAx4H9wC+BL6hqAFgCvCwibwFvAL9Q1V9G9q3Fpo7BMU4OjHFBhX37TzQpXg93XFHNq0d62N8+6HY5xswrrCuBVfUp4KkZ0+6b9nwMuGWOZf8K+KsZ05qBC8+22ESwu6UPj8AFFflul2LCdDZXaad6PaT6PPyvn7zNbRur3jP/U5veO80Yt9iVwIsoqMruE/2sWZJDdpqNwpGIMlK9XLGiiL3OKK/GxDILgEV0pGuYwTE/F1cVuF2KiaLNq4pJ9XnYceCU26UYc0YWAIvoNy39pKd4WLs0x+1STBRlpvm4cmURe9sHOTlw2u1yjJmTBcAiGZ8MsK99gAvK8/F5bbMnuqtWlZDm8/DsATsl1MQu+yRaJG+3DTAZUC6usoO/ySAj1cvm2hIOnBykqXPY7XKMmZUFwCJQVV5v7mFpbjpVhZlul2MWyebaYgqzUvnPt9rxB+2+wSb2WAAsghO9o5wcGGPTikJCY+SZZJDi9XDDhjK6hsd5tcnuG2xijwXAInj9aC9pPg8XVVr3T7JZW5bL2qU57GjspH90wu1yjHkXC4AoGx73s6dtgEuqCkjzeedfwCYE9NQAAAu4SURBVCScGy5YBgKPN5zAH7CuIBM7LACirOFYL4GgsmlFodulGJcUZqWy9cJlHOsZ5Z92NLldjjHvsACIogl/kFeP9LCqJJvSnHS3yzEuuriqgIsr8/mnHYd5vdmOB5jYYAEQRTuP9TI87ueataVul2JiwE0XLqO6KIvP/fsumjqH3C7HGAuAaBmbDPDi4S5qirOoKc5yuxwTA9JSvHz/v1+Gz+PhjofeoL3frhI27rIAiJInGk4wNObnWvv2b6ZZXpTFw5+9jKExP59+qJ42CwHjIguAKBibDPDAC0dYXpjJCvv2b2Y4b1keD33mMjoHx/n4/a+wp3XA7ZJMkrIAiIIHXjhC+8AYH1y/xC78MrPaWFPITz5/JaleD//tX17jyV2thG6iZ8zisQCIsOM9Izzw6yPceOEyVpZku12OiWGrl+TwH194Hxsq8vjTJ97irocbOGX3EDCLyAIgglSV+36+j1Svh//9sXVul2PiQElOGo/+/uV85Yb1vHqkm2v+7gX+/pmDDI5Nul2aSQIWABH0X2+f5NeHuvjjD61mSa6d92/C4/EId11Vwy+/eDXXrC3ln3Y0cfX/e56/3d5Ix4DtEZjosfsSRkhT5zBf/ukeLqrM584rlrtdjolR891f+H0ri6kuzOL5g5185/kjPPDCEdYszeXCijzWLs0l1ffb72x2f2GzUBYAETA87udz/76LNJ+HB373Ervhi1mQ8oIMfvfy5fSOTFDf3MPu1n4OnBwk1ethXVkOF1Tks6rUji+ZhbMAWKDJQJAvPbab5q5h/v2uTZTlZbhdkkkQhVmpXL+hjI+cv5Sj3SO83TrA3rYB3modINXr4ZWmbq5dW8q1a0sptS5Hcw4sABZgbDLAPY+8ybMHOvmLG9dz5apit0syCcgjwsqSbFaWZHPjhWUc6RyhsWOQvW0DPLM/dOP5DeV5XLu2lOvWlXL+sjw8Hjv92MzPAuAcDYxO8vlHdvFKUw//d+t5fPqKardLMknA5/GwZmkOa5bmcNvGSho7htjR2MmOxk6+teMw33zuMCU5aVy7ppRr15Vy1apistLsv7mZnf1lnIMdjaf48k/30D08wd/fciGfuLTC7ZJMEhIR1pXlsq4sly9cs4qe4XF+faiL5xo7eWrvSR5rOEGq18OmFYVct7aU69YtodJuSWqmkXi6+rCurk4bGhpc+/1vnejnn399hKf3drBmSQ5/d8uFbKjIm7P9fGd8GBMtgaByvGeExo4hGjuG6B4eB6A0J421S3NYvTSHyoJMUrweO5soCYjILlWtmzk9rD0AEdkCfBPwAt9V1b+ZMT8N+AFwKdADfFJVjznzvgzcBQSAP1LV7eGsM1a09Izy7IFTPL33JDuP9ZGT7uOL19Xy+WtW2h2+TMzyeoQVJdmsKMnmoxvK6B4e52DHEAc7hnilqYcXD3fj9Qjl+Rkc6xnhkqoCLq7KpzQnLamHLwkGlcGxSbqHJ+gZHqdnZIKRcT/j/iBjkwHG/UEmA0FSfR4yU7xkpHpJT/GSneajICuVwsxUCrJSyU33xcV2nHcPQES8wCHgQ0ArsBO4TVX3T2vzeeACVf2ciNwK/I6qflJE1gM/BjYCy4BngdXOYmdc52wWsgegqviDij+gTASC+ANBJgPKZCDI6ckA/aOT9I1O0DEwRlv/aQ6dGmJf+yBdQ6FvTqtKs/lkXSW3bqwkJz0lrN9pewAmFo1NBmjuGuF47wgtPaOcHBhjwrlVZWaql+VFWdQUZ7K8KItleenkZaaSl5FCfkYKeRkppKd4SfEKPq8n9NMT+rnYH3iqSlBDeztBVQJBJaBKMPjb5xP+IKMTAX72Zhvj/iAT/gCjEwGGx/2MjPudn87ridC0YAQ6RXweoSArlaKsVAoyUynMDoVDYda7HwWZqWSkhrZnqtdDitdDii+0PVM8nogdzF/IHsBGoElVm50VPQpsBaZ/WG8F/o/z/Eng2xL6a9gKPKqq48BREWly1kcY64yYLf/4Io0d4d+AI9XnoaYoi82rirmgIo9r1payvMhG9TSJIT3Fy/pluaxflgvAJy4tZ2/bAHvbBjnWM8Kx7hEaTw7xzL5T+M/i09DrEbwiMMdn1myT58oMmWMlQf3th/1CP6jTfB6y0nzvfHuvKMggO833zrSsNB9ZaV7SfN53Bd3vXr6ciUAoWE5PBjg94Wd4PEDfyAS9U4/RCfpGJugZCf08cHKQ3pEJ+kfPbogPkdBZYALs/cuPkJ4S2V6HcAKgHDgx7XUrsGmuNqrqF5EBoMiZ/vqMZcud5/OtEwARuRu423k5LCIHw6h5wQ5D8TPQvRi/K4KKsZqjLd7qhXlqvn0RCzkLMbud75x7VlRrzvjrBS0+6/AEMX8WkKo+CDy42L9XRBpm22WKZVZz9MVbvWA1L5Z4rDmcMQvagMppryucabO2EREfkEfoYPBcy4azTmOMMVEUTgDsBGpFpEZEUoFbgW0z2mzjt3tGNwM7NHR0eRtwq4ikiUgNUAu8EeY6jTHGRNG8XUBOn/49wHZCp2x+T1X3ichXgQZV3QY8BPzQOcjbS+gDHafd44QO7vqBL6hqAGC2dUb+7S3Ionc7RYDVHH3xVi9YzYsl7mqOqwvBjDHGRI6NW2yMMUnKAsAYY5KUBcAMIrJFRA6KSJOI3Ot2PbMRkUoReV5E9ovIPhH5ojO9UER+JSKHnZ8Fbtc6k4h4ReQ3IvJfzusaEal3tvdjzkkBMUNE8kXkSRFpFJEDInJFrG9nEflj5+9ir4j8WETSY207i8j3RKRTRPZOmzbrdpWQbzm1vy0il8RIvX/r/F28LSI/E5H8afO+7NR7UEQ+stj1hssCYBpn2Iv7geuB9cBtznAWscYP/ImqrgcuB77g1Hkv8Jyq1gLPOa9jzReBA9Nefx34hqquAvoIjRsVS74J/FJV1wIXEqo9ZreziJQDfwTUqer5hE6yuJXY287/BmyZMW2u7Xo9oTMIawldFPrAItU43b/x3np/BZyvqhcQGtrmywDO/8VbgfOcZb7jfLbEHAuAd3tn2AtVnQCmhqiIKap6UlXfdJ4PEfpQKidU68NOs4eBj7tT4exEpAL4GPBd57UA1xIaPgRirGYRyQOuJnSWG6o6oar9xPh2JnR2X4ZzTU4mcJIY286q+iKhMwanm2u7bgV+oCGvA/kiUrY4lYbMVq+qPqOqfufl64SuZ4JpQ+Co6lFg+hA4McUC4N1mG/aifI62MUFEqoGLgXpgiaqedGZ1AEtcKmsu/wj8GRB0XhcB/dP+E8Xa9q4BuoDvO91W3xWRLGJ4O6tqG/B3QAuhD/4BYBexvZ2nzLVd4+H/5WeBp53n8VAvYAEQ10QkG/gJ8D9VdXD6POdCvJg5x1dEbgA6VXWX27WcBR9wCfCAql4MjDCjuycGt3MBoW+gNYRG4M3ivV0XMS/WtuuZiMifE+qW/ZHbtZwtC4B3i5shKkQkhdCH/49U9afO5FNTu8bOz0636pvF+4CbROQYoa61awn1r+c7XRUQe9u7FWhV1Xrn9ZOEAiGWt/MHgaOq2qWqk8BPCW37WN7OU+barjH7/1JEPgPcANyuv72oKmbrnckC4N3iYogKp+/8IeCAqv7DtFnTh+S4E/j5Ytc2F1X9sqpWqGo1oe26Q1VvB54nNHwIxF7NHcAJEVnjTLqO0FXtMbudCXX9XC4imc7fyVTNMbudp5lru24D7nDOBrocGJjWVeQaCd3U6s+Am1R1dNqsuYbAiT2qao9pD+CjhI7oHwH+3O165qjxKkK7x28Du53HRwn1qT8HHCZ0851Ct2udo/4PAP/lPF9B6D9HE/AEkOZ2fTNqvQhocLb1fwAFsb6dgb8EGoG9wA+BtFjbzoRuFHUSmCS0p3XXXNuV0K0E7nf+T+4hdIZTLNTbRKivf+r/4D9Pa//nTr0Hgevd/puY62FDQRhjTJKyLiBjjElSFgDGGJOkLACMMSZJWQAYY0ySsgAwxpgkZQFgjDFJygLAGGOS1P8HouLTC1K8ER0AAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "tags": [], + "needs_background": "light" + } + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "WPiAKFpzcf4U", + "colab_type": "code", + "outputId": "256cb956-e362-40dd-aae5-65cb89f4b406", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 470 + } + }, + "source": [ + "df = pd.DataFrame(dataset,columns=['X1 transaction date', 'X2 house age', 'X3 distance to the nearest MRT station', 'X4 number of convenience stores', 'X5 latitude', 'X6 longitude']) # regressor matrix\n", + "corrMatrix = df.corr()\n", + "sn.heatmap(corrMatrix, annot=True) # visualize the correlation matrix using a heatmap" + ], + "execution_count": 69, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 69 + }, + { + "output_type": "display_data", + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAG0CAYAAADgoSfXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdd3hUVfrA8e+bnkCAhJbQu0iNEJAiShEB14KriKIull3RVVmwg6goKtgL2PuuiIqFHxYEpKxKUULoCIggLQkthJqe9/fHvYkzIQkJSRjGfT/Pc5/k3nvuue89mcy8c865M6KqGGOMMcb4SoCvAzDGGGPM/zZLRowxxhjjU5aMGGOMMcanLBkxxhhjjE9ZMmKMMcYYn7JkxBhjjDE+ZcmIMcYY8z9GRN4RkT0israY/SIiL4nIZhFZLSKdPPYNF5Ff3WV4RcRjyYgxxhjzv+c9YGAJ+wcBLd3lZuBVABGJBh4Gzga6Ag+LSFR5g7FkxBhjjPkfo6rfA6klFLkU+Lc6lgI1RCQWGADMVdVUVT0AzKXkpKZULBkxxhhjTGH1gR0e6zvdbcVtL5eg8lZgzP+y7H1b/Or7FBa0HevrEMokXHJ8HUKZHVX/e1oNIc/XIZRJakCwr0M4KVckT5XyHF+W55uQ2s1H4Ayv5HtDVd8oz/krk//91xhjjDH/i/JyS13UTTzKk3zsAhp6rDdwt+0CehfavrAc5wFsmMYYY4zxD5pX+qX8ZgJ/c++q6QYcVNVkYDZwgYhEuRNXL3C3lYv1jBhjjDH+IK/ihtNEZBpOD0ctEdmJc4dMMICqvgZ8A1wIbAaOATe4+1JFZAKwzK3qUVUtaSJsqVgyYowxxvgBrZgeD7cuvfoE+xW4rZh97wDvVFgwWDJijDHG+Idc/5vQXVqWjBhjjDH+oAwTWP2NJSPGGGOMP6jAYZrTjSUjxhhjjD+owAmspxtLRowxxhg/UJETWE83lowYY4wx/sB6RowxxhjjU7nZvo6g0lgyYowxxvgDG6YxxpwK4554ju8X/Ux0VA1mfPCaz+Ko2acjrR8bjgQGsHPqfH6fPNNrv4QE0X7KbVTr0JTsA0dYdfOLZOzYS3BUVTq+PZpqcc1J+ui/bBj7bsExMZf1oOm/BoMqmSkHWHPby2SnHq6wmKP6xNFswg1IYAApU+exc8qM42I+Y/IdVO3QjOwDR9gw4jkyd+wFIOLMxrR8+mYCIyMgL48VA+9HM7NpfP/V1B1yHkE1qrC4+XXljrFWn46c+dhwcNt1axHt2qFQu6a7MTYbeSn1h/WB3Dx+eeA99i1cDUDjfwyiwbV9Adg5dT7b3phVUF+jmwbQ6IYL0Nw89n63gk0TPixX/NF9OtLyMaeNk6fOY9vk/zsu/jZTbieyQzOyDxxm3c0vkLFjL5FnNaf1MyPcQrD16ensm+V8gGf3ZVPIPZqB5uahObkkDBhTrhiL0nHC34jt15Gc9CwSRr1O2prfjytTo0MTurxwC4FhwSTPW8WqB//ttb/liAvpOP4aZrYdQVbqESJbxBL//AhqtG/CukmfsOm1byo87uP8iYdp7LtpTlMi0lBEtopItLse5a43cde/FZE0EfmqhDquF5F6pybi0hGROBG50GP9EhG5vxLOM15E7j5BmcEi0qaiz10egy/sz2vPPebbIAKEMyfdSOKwSSzqdRexl/WkSivvbwhvMKwP2WlH+LHbKLa9/jWtHhwGQF5mNpsnfcKm8R94lZfAAFo/NpyEv05gSZ/7OLx+O41uHFCBMQfQfOLfWTfscZafO5ral51DRKsGXkVihvUjJ+0oCd3vIOn1r2g67lpnR2AArV8eyeZ73yDxvNGs/uvDaLbzeQ6pcxJYMaiCHp4BQptJN5IwbBI/nqBdf+g2it892rVKq/rEDO7Bj+feTcLVE2nz5E0QIFRt3YAG1/ZlycAHWNz3Pmr370REk7oARPdsQ52B8Szqex+LzruH318t9qmi1PGfMekmVg17gp96jabOZT2JKBR/vWF9yUk7ytJuI9nx+tc0f/AaAI5u2EHCBfezrN+9rLrqCVo/czMS+MfLz4q/PsKyfvdWSiIS07cjkc1i+LbHXSTe8zadJt1QZLlOk25k+d1v8W2Pu4hsFkNM344F+8LrRVO3d3uO7txXsC3rwFFWjvs3m177usJjLtap/W6aU8qSkdOUqu4AXgUmuZsm4XwF9O/u+tPAid6qXQ8UmYyISGD5ozwpcTjfdwCAqs5U1UkllK9Mg4HTKhmJj2tP9WqRPo2heqcWHNuaQvq2PWh2LikzFlNnYLxXmdoD40n65HsAdn/5E9HntAUg91gmaT9vJC+z0Ni2CCAERoQCEBQZTsbuAxUWc+RZLcjYmkLG9j1odg57ZywiekAXrzI1B3Rh9ycLAdj71RJqnNMegKjeHTm6fhtH128DIOfAkYJ3oIcTfyV7T1qFxFijiHatW6hd6xZq15puu9YdGE/KjMVoVg7p2/dybGsKNTq1oErL+hxM3Exeehaam8eBxb9Q9y9dAWg4vD9bJ/8fmuV8amfWvkPlir+aG3+GG/+eGYupPdC7jWsNjCc5v42/XErUOe0ACuIDCAgLBtVyxVIW9QZ2Ztv0HwBITdxMcLUIwurU8CoTVqcGQZHhpCZuBmDb9B+oN7Bzwf6Oj1zHmgnTvOLO3H+IA6u2FCSup0ReXukXP2PJyOnteaCbiIwCzgGeyd+hqvOAYvu4ReQKIB6YKiIrRSRcRH4XkSdFJBEYIiL/EJFlIrJKRD4TkQj32PdE5CURWSwiW9y6EJFYEfnerW+tiPRyt78qIgkisk5EHvGIoYtbxyoR+VlEqgOPAkPdOoa6vTdT3PJNRGS+iKwWkXki0qikeIq45gdEZJOI/Aic4bH9uOsUkR7AJcDTbizN3eVbEVkuIj+ISOuy/8n8X1hMNBlJ+wvWM5JSCY2J9i4TG03GLqeM5uaRczid4OjikyjNyeWX+96mx8KnOG/1q1Rt1YBdU+dXWMyhsdFkJnm8a03eT2isd8whnmVy88g5fIyg6EjCm9VDVWk3bRxnzXmKBrddWmFxecUYE036Cdo1NDaa9CLaNTTmj+0AGcnOsUc27CDq7NYER1UlIDyE2ufHEVa/JgBVmscSdXZrus16jK5fPES1uGbljj/TI/7MpP1Fxp/pEX/u4WMFj4tqnVrQ9b/P0nXhs2y4582C5AQg7uMHiJ8ziXrX9StXjEUJj4nmmEfc6cmphMdGeZeJjSI9KdW7jHttsQM6k56SysH12ys8trLSvOxSL/7G5oycxlQ1W0TuAb4FLlDVUj/CVPVTEbkduFtVEwBEBGC/qnZy12uq6pvu748BNwGT3SpicRKg1jhfJf0pMAyYraqPuz0rEW7ZB9xvcgwE5olIB2AD8DEwVFWXiUg1nG9+fAiIV9Xb3fNe7xH2ZOB9VX1fRG4EXsLpvSgungIi0hm4CqfnJQhIBJa7uz8vfJ2qOllEZgJfqeqn7r55wC2q+quInA28AvQtbZub4klQIA2u78+SfmNI37ab1k/cQNN/DWbr81/4OjQkKJDqZ7dmxcD7yUvPpP30hzmyagtpP67xdWgndPTXJLZMmUn8x2PJPZbJobXbCl7kJSiQ4KiqLB00jupnNSfuzVF832Wkz2I9lLiZn8+7i4iW9Wkz+TZS568kLzOb5Rc/SFbKAYJrVSPuk3Ec+zWJtKW/+CxOT4HhIZw58hK+v8pXnbeF+GGPR2lZz8jpbxCQDLSroPo+9vi9ndsDsAa4BmjrsW+Gquap6nqgrrttGXCDiIwH2qtqfs/MlW5vywq3jjY4PRPJqroMQFUPqeqJvuWpO5A/w+4/OMlHSfF46gV8oarHVPUQTsJSmusEQESqAj2A6SKyEngdJwE6jojc7PYEJbz172knuCT/k5GSSli9mgXrYfWiyUzx/obwjOTUgnfgEhhAUGR4iZNRI9s1BiB9224Ads9cQo34VhUWc2ZyKqH1ahWsh8TWJDPZO+YszzKBAQRFRpCTepispP0cXPoLOamHyUvPInXeCqp0aFphsRXEmJJK+AnaNTM5lfAi2jUz5Y/t4PRM5R+768MFLLlgLD8PfoTsg0c5+lsyABlJ+9n99c8AHFzxG+QpwTVPfggwMyWVUI/4Q+vVLDL+UI/4AyMjjntcHPt1F7lHM6jSuiEAWSnOcF32vkPs+2YZkWe1OOkY8zW/vj/nz32C8+c+QfqeNCI84g6PjSY92XuIMD35AOH1or3LpKRSpXFdIhrVpv+8iQz6+QXCY6M5f87jhNauXu4YT4rNGTG+ICJxQH+gGzBaRIp8cSyjox6/vwfcrqrtgUeAMI99mZ6hAKjq98C5wC7gPRH5m4g0Be4G+qlqB+DrQvVUlOPiKYP3KP468wUAaaoa57GcWVRlqvqGqsaravzf/1bit3D7pUMrfiOiWQzhjWojwYHEDO7BntnLvcrsnb2celeeC0Ddi88m9cd1JdaZmXyAqq3qF7wYRp/XgaO/7qqwmA+v3ExYs1hCG9VBgoOoPbgnqXOWeZXZPyeBulf2BqD2Rd1JW7QWgAMLV1KldSMCwkMgMIDq3dtwbNPOCost38FStOueQu26323XPbOXEzO4BxISRHij2kQ0iyHNnd8QUqsaAGH1a1L3wi4kf77IOWZWAtE9nbw7olksEhxE9v6Tv3vp8IrfiGgWS5gbf53BPdg3O8GrzL7Zy4nNb+OLu3HAjT+sUe2CCathDWoR0aIeGTv2EhARSmAV598xICKU6N4dOLqh/MMhv703l+/6j+W7/mNJmpVA4yG9AIju1ILsw+lkFJoHlLEnjZzD6UR3chKhxkN6kfTtcg5t2MFX7f/JrK6jmNV1FOnJqXx3wQNk7j1Y7hhPSl5u6Rc/Y8M0pylxxlReBUap6nYReRpnzsg1ZajmMFDSW6FIIFlEgt16S3x1EJHGwE5VfVNEQoFOwCqcBOegiNTF6clZCGwEYkWkiztMEwmknyCmxThDLf9x4/mhVFfp+B4nQZqI87i+GKd3o6TrLIhFVQ+5dysNUdXpbvt3UNVVZYih3O55eBLLVqwmLe0Q/QZfyz9vuo7LL67Au05KQXPz2DDmXTp9NBYJDGDXtAUc3biT5vcO4dCqLeydvZxdHy6g3ZTbOGfpC2SnHWH1iJcKju+1bDJBkeFISBB1BsWzfOgTHN20i9+e+YwuM8ajOTlk7NzH2pGvVlzQuXn8NvYt2k0bhwQGsHvafI5t3Enje4dyeOVvpM5JIOXDeZwxZSTxSyaTk3aEDSOeByDn4FF2vv4lcd8+CaqkzkvkwHeJADR58FrqXNaLgPBQuia+TsqH89j+zCcnFaLm5rF+zLvEu+26c9oCjmzcSYt7h3DQbdedHy6gw5Tb6OW26yq3XY9s3EnKzCX0+uFZNCeX9fe/C3nOZMq4t+8kJKoqeTm5rB/zLjmHjgGwc9oC2r9wCz3/+zR5WTmsGflKuZpYc/PYNOYd4j56AAkMIMl9XDS990oOr/qNfbOXk/zhfNpMuZ1uS18iJ+0Ia0e8AECNrq1pdMdgNCcX8vLYeP/bZKceJqxxHdq/69z0JoGB7P7iR1IXVOy/XMq8lcT0i2PgkufITc8iYfTrBfvOn/sE3/UfC8CKMe8S/8IIAsNCSJm/ipT5JccRWrs6/b59jODIcDQvjxb/GMSc8+4l50h6hcbvxQ97PEpL9BTOajalJyI34/Q2DHXXA3GGSUar6n9F5Aec+RNVgf048yBmF6rjcuAJnCSgO/ALznyNfe7+W4F7gb3AT0Ckql4vIu/hPZfiiKpWFZHhwD1ANnAE+JuqbnXL9wB2AAeBmar6noh0wZkHEu7GcD4QAswGgoGJ7r54Vb3dTXbeBWq5Md3gJmJFxlNEmz0ADAf2ANuBRFV9poTr7Am8idPrcgWQh5MAxrrxfaSqj5b0d8ret8Wv/oEWtB3r6xDKJFxONLJ3+jmq/vceLwT/epFLDQj2dQgn5YrkqWXt1fWSsfTjUj/fhHUbWq5znWqWjBhTDpaMVC5LRk4NS0ZOjXInI4umlj4Z6XmNXyUj/vdfY4wxxvwv+hPfTWPJiDHGGOMHVP1vYmppWTJijDHG+IM/cc+I3dprjDHG+IMK/pwRERkoIhtFZLMU8R1hIvK8+wnVK91Pt07z2JfrsW9m4WPLynpGjDHGGH9QgT0j7h2aL+N8ltVOYJmIzHQ/WBIAVR3tUf4O4CyPKtJVNa6i4rGeEWOMMcYf5OaUfjmxrsBmVd2iqlnAR0BJX8x0NVBpHzltyYgxxhjjDyp2mKY+zmdD5dvpbjuO+xlQTQHPb7cMc78WY6mIDC7quLKwYRpjjDHGH5RhmMb94MybPTa9oapvnOSZrwI+Ve/beRqr6i4RaQbMF5E1qvrbSdZvyYgxxhjjF8qQjLiJR0nJxy6gocd6A4r/SpCrgNsK1b/L/blFRBbizCc56WTEhmmMMcYYf1CxwzTLgJYi0lREQnASjuPuihGR1kAUsMRjW5T7/WSISC2gJ7C+8LFlYT0jxhhjjD8o3cTUUlHVHBG5Hee7wgKBd1R1nYg8CiSoan5ichXO93R5fhT9mcDrIpKH06kxyfMunJNhyYgxxhjjDyr4Q89U9Rvgm0LbHiq0Pr6I4xYD7SsyFktGjDHGGH9Qyg8z80eWjBhTDv72Lbh91j3h6xDKZGHbMb4Oocwi5M/7/SGnC7/6quyK9Cf+OHhLRowxxhh/YMmIMcYYY3xK/7x9QpaMGGOMMf4gp+LupjndWDJijDHG+AObwGqMMcYYn7I5I8YYY4zxKZszYowxxhifsp4RY4wxxviUJSPGGGOM8SXN/fN+oJ4lI8YYY4w/sJ4RY4wxxviU3dprjDHGGJ/Ks7tpjDHGGONLNkxjjDlZNft0pPVjw5HAAHZOnc/vk2d67ZeQINpPuY1qHZqSfeAIq25+kYwdewmOqkrHt0dTLa45SR/9lw1j3y04JuayHjT912BQJTPlAGtue5ns1MOn+tIAGPfEc3y/6Geio2ow44PXfBIDOO18xmPXI4EB7Jo6n98n/5/XfgkJot2U26jWoRnZBw6z2m3n6HPb03LcMCQkCM3KYdOjH3Dgx3XOMcGBtJ54I1E92kCesnniR+z5+udKiT+qTxzNJ9yABAaQMnUeO6bMOC7+MybfQaQb/y8jnidzx15CG9Ym/vsXSP8tCYBDyzex+b43/S7Gtu/fR1jjOizvfVeFxx034W/E9utITnoWy0a9Ttqa348rU6NDE7q+cAuBYcEkz1vFygf/7bW/1YgL6Tj+Gv6v7QiyUo8QXD2CLs/fTJXGdcnLzGbZ6Dc4tHFnhcfu5U88gTXA1wGY04uINBSRrSIS7a5HuetNRCRORJaIyDoRWS0iQ4upY6GIxJ/ayE9TAcKZk24kcdgkFvW6i9jLelKlVX2vIg2G9SE77Qg/dhvFtte/ptWDwwDIy8xm86RP2DT+A6/yEhhA68eGk/DXCSzpcx+H12+n0Y0DTtklFTb4wv689txjPjs/AAFC60k3smLYRBb3upOYItq5/rC+5KQdZVG3f7Ht9W9o6bZzduphVl73FEt738Paka/QbsrtBcc0HfVXsvYdYnGP0SzudRcHlvxSSfEH0GLiTawd9jgJ546m9mU9iWjVwKtIzLC+5KQdYVn3O9j1+lc0HXdtwb6MbSkknn8PieffU2mJSGXGWPPCruQezaiUsGP6dqRqsxhm9biL5fe8TadJNxRZrvOkG0m4+y1m9biLqs1iiOnbsWBfeL1o6vZuz9Gd+wq2nTnyUtLWbmduvzH8PPJV4iZcVynxe8nLK/3iZywZMV5UdQfwKjDJ3TQJeENVfweOAX9T1bbAQOAFEanhk0D9RPVOLTi2NYX0bXvQ7FxSZiymzkDvPK32wHiSPvkegN1f/kT0OW0ByD2WSdrPG8nLzPauVAQQAiNCAQiKDCdj94FKv5bixMe1p3q1SJ+dH/LbebdXO9ce2MWrjNPO/wVgz5dLiT6nHQCH1/5Optt+RzfsIDAsBAlxOo3rX92brS+57/5VK633KfKsFqRvTSFj+x40O4e9MxZRc4D346TmgC7sduPf+9VSotz4T5XKijEgIowGIy5m+wufVUrc9QZ2Ztv0HwBITdxMSLUIwup4P22F1alBUGQ4qYmbAdg2/QfqDexcsD/uketYPWGa1yegVmtVnz2LnB60w5uTqdKwNqG1qlXKNRTI09IvfsaSEVOU54FuIjIKOAd4BkBVN6nqr+7vScAeoHYxdQwRkZ9FZJOI9AIQkTAReVdE1ojIChHp426/XkSm5B8oIl+JSG8RCRSR90RkrXvMaHd/cxH5VkSWi8gPItK68MlFpKvbi7NCRBaLyBnu9ggR+URE1ovIFyLyU34vjohc4B6TKCLTRaRqeRsyLCaajKT9BesZSamExkR7l4mNJmOXU0Zz88g5nE5wdPEv7pqTyy/3vU2PhU9x3upXqdqqAbumzi9vqH4tNCaaTI92zkzaT2hMlFeZ49v52HHtXOeiszm0ZiualUNQtQgAWtx3JWfPnUSHN0cTUrt65cQfWyj+5FRCYmsWUcZ9Z+7GH+TGH9aoDp3mPkWHLx6h2tnH/Tuc1jE2uW8oO1/7ktz0zEqJOzwmmmMecR9LTiU81vuxER4bRXpSasF6enIq4e7/ab0BnUlPSeXg+u1ex6St3079C52ENyquGRENahFez/t/u8JpXukXP2PJiDmOqmYD9+AkJaPcdS8i0hUIAX4rppogVe0KjAIedrfd5lSv7YGrgfdFJKyEUOKA+qrazj0mf9LEG8AdqtoZuBt4pYhjNwC9VPUs4CHgCXf7P4EDqtoGeBDo7F5PLWAccL6qdgISgDuLCkpEbhaRBBFJ+Ca9uMuvPBIUSIPr+7Ok3xj+2+FWDq/f7swfMeVS5YwGtHxwGL/c7QwhSFAgYfVrkbZsEz/1v5+0hE20fPjaE9Ry6mXtPsBPnW8lsf+9bHn4fc585V8EVg33dVheiouxStsmhDeJYf+sypmHU16B4SG0HnkJa5/69Lh9GyZ/SXC1CPrPfYKWNw0gbe3vaG4l90j8iXtGbAKrKc4gIBloB8z13CEiscB/gOGqxabgn7s/lwNN3N/PASYDqOoGEdkGtCohhi1AMxGZDHwNzHF7K3oA00Ukv1xoEcdWx0l2WgIKBHvE8KIbw1oRWe1u7wa0ARa59YYAS4oKSlXfwEmImFP3qhL/6zNSUgmr98e7x7B60WSmpHqXSU4lrH5NMpNTkcAAgiLDSxwOiGzXGID0bbsB2D1zCU3uuLSkMP70MlNSCfVo59B6NclM8R66Or6dIwraOTQ2mo7v3sXa218paNfs1MPkHssomLC6+8ul1B/Wp3LiTy4Uf2w0Wcn7iyhTi6zkVHDjz3Hjz8k6AsCR1VtI37ab8OaxHFm15bSPMTKuBVU7NqPrspeRwECCa1Wnw+fjWf3X8eWKtfn1/Wl2jfO3Sl21hYh6NcmPNCI2mvRk78dGevIBr16N8Nho0lNSqdK4LlUa1eaCeRMLtvef8zjfDXqIzL0HSRj9RsExF/78Ake37SlX3CeifjgXpLSsZ8QcR0TigP44L9Cj3eQjf181nMTgAVVdWkI1+X2uuZw46c3B+7EYBqCqB4COwELgFuAtt1yaqsZ5LGcWUecEYIGqtgMuzq+zBALM9aizjaredIJjTujQit+IaBZDeKPaSHAgMYN7sGf2cq8ye2cvp96V5wJQ9+KzSXXv5ChOZvIBqraqT3BNp/s7+rwOHP11V3lD9Wv57Rzm0c57Zyd4ldk7O4F6V54HQJ2LuxW0c1C1CM6aej+bH5vGwWUbvY+Zk0hUzzYARPdqx9FNldPOh1duJrxZLGGN6iDBQdQe3JP9c7zj3z8ngbpu/LUv6kbaorUABNesBgHOv09YozqEN40loxJeFCsjxuT35/BT3Ah+7nIbKy99kPQtSeVORAB+e28uc/uPZW7/seyalUDjIb0AiO7UguzD6WTsSfMqn7EnjZzD6UR3agFA4yG9SPp2OYc27ODL9v/km66j+KbrKNKTU5l7wQNk7j1IcLUIJDgQgKbX9GHv0g3kHEkvd+wlys0t/VIKIjJQRDaKyGYRub+I/deLyF4RWekuf/fYN1xEfnWX4eW9NOsZMV7E6RZ4FWd4ZruIPI0zZ+QaEQkBvgD+rarH91ue2A/ANcB8EWkFNAI2AtWAf4pIAFAf6OrGUgvIUtXPRGQj8IGqHnLv7hmiqtPdeDuo6qpC56oO5L9yXO+xfRFwJbBARNoA7d3tS4GXRaSFqm4WkSo4Q0SbTuI6C2huHhvGvEunj8Y6t5xOW8DRjTtpfu8QDq3awt7Zy9n14QLaTbmNc5a+QHbaEVaPeKng+F7LJhMUGY6EBFFnUDzLhz7B0U27+O2Zz+gyYzyak0PGzn2sHflqecIsl3sensSyFatJSztEv8HX8s+bruPyi0/t3T2am8fGMe8UtHPStIXHtXPShwtoN+V2ei59key0I6wZ8SIADW8aSETTujS763Ka3XU5AMuHPk72vkP8OmEq7abcTtCE4WTtP8T6f1VSO+fmsXns27Sb9oBz2+y0BRzbuJPG9w7l8MrfSJ2TQMqH82k95Q66LJlMdtoRNox4HoDq3c6k8b1D0excNC+PX+99g5y0I/+bMRYhZd5KYvvFMWjJc+SmZ7Fs9OsF+/rPfYK5/ccCkDjmXbq8MILAsBBS5q8iZX7hpxRvkS3r0fXFW0CVg5t2kXDnGyWWrxAVOPwiIoHAyzhvPHcCy0RkpqquL1T0Y1W9vdCx0TjD7/E4Pc/L3WNPeia9qPrf2JKpPCJyM9BPVYe664HAMmA00BBn3obnW/frVXVloToWAneraoKbUCSoahN3fsirOA/gHOBOVV3gJhQf4Mzf+AWIAsYDB9zz5feajFHVWSLS1K0nFmf45SNVfbRQDN2B94GjOD0517oxVHG3t8GZV9IMGKKqv4pIX+BJ/hj2Gaeq3h8KUsiJhmlON33WPXHiQqeRhW3H+DqEMguVP29X+ulit4T4OoSTMiR5qpy4VPGOjr+61M83VcZPK/Fc7nPkeFUd4K6PAVDViR5lrgfii0hGrgZ6q+oId1UjR94AACAASURBVP11YKGqTittfIVZz4jx4jkfwl3PBTp5FPnguIOOr6O3x+/7cOeMqGoGcNxN/upkxNcUU12nwhtUdSvOrcUlxbAE7/ko49yfGTiJSYaINAe+A7a5x8wHvO8HNcaY00XFTkytD+zwWN8JnF1EuctF5FxgEzDa/fiHoo6tX8SxpWZzRsz/mgjgRxFZhTPk9E9VzfJxTMYYc2JluLXX864/d7n5JM74JdBEVTvg3MjwfsVe0B+sZ8T8T1HVwzjDRMYY41/K0DNSuJe7CLtwht7zNeCPeXb5dXjeLvUW8JTHsb0LHbuw1MEVwXpGjDHGGD+gObmlXkphGdBSRJq6NydcBXjNkfO8kxK4BGdOH8Bs4AJxvi4kCrjA3XbSrGfEGGOM8QcVOGdEVXNE5HacJCIQeEdV14nIozg3HcwERorIJTg3HKTi3pmoqqkiMgEnoQF4VFVTjztJGVgyYowxxviDCv6Yd1X9Bvim0LaHPH4fAxR5S5uqvgO8U1GxWDJijDHG+AM//Jj30rJkxBhjjPEDasmIMcYYY3yqdBNT/ZIlI8YYY4w/sJ4RY4wxxviUJSPGGGOM8aU/83fJWTJijDHG+APrGTHGFCVccnwdQpn427fg9l438cSFTjNxba/2dQhlFij+9WHc6/dv93UIJ2VIeSuwZMQYY4wxvqQ5FfuhZ6cTS0aMMcYYf/DnzUUsGTHGGGP8gX3omTHGGGN8y5IRY4wxxviUDdMYY4wxxpdsmMYYY4wxPqU5lowYY4wxxpdsmMYYY4wxvqSWjBhjjDHGpywZMcYYY4wvWc+IMcYYY3xK/eursMrEkhFjjDHGD/yZe0b866sajfFDUX3i6Pzji8QvmUyD2wcft19Cgmj9+mjil0ym4zcTCW1Yu2BfxJmN6fjV43T67/N0WvAsEhoMQOP7r6br8tfo8dt/KiXmmn060mPR8/Rc+iJN7ri0yJjbv/Evei59ka6zHiPMjTn63PacPWci3RY+zdlzJhJ1Tts/jgkO5Mxn/kGPxc/T48fnqPOXrpUS+4mMe+I5zv3LVQy+9hafnL+0xjx+J7OWfsrnCz7gzPZnFFkmODiI8c+M4evF0/nyx4/p/5c+pzTG+x4bzVdLpvPp/P9wZvtWRZZ5+/OXmfnjR3zy3ft88t37RNeKAiC2QQxvTp/Mp/P/w9ufv0zd2NpFHl/Rnn/uUTas/5HE5XM5K65dkWWGDr2UFYnfkbh8Ll9/+QE1azoxX375RaxaOZ+sjB107tThlMTrSfNKv5SGiAwUkY0isllE7i9i/50isl5EVovIPBFp7LEvV0RWusvM8l5bicmIiDQUka0iEu2uR7nrTUSksYgkuoGsE5ET/meLSG8R+cr9/ZKiLt6jbJyIXFjWC6oI7vUNK88xInK9iEyp+OhOLREZJSIRxexbKCLbRUQ8ts0QkSPu701EJN19jKwXkX+LSF2PB3CKiOzyWA8pRTzXi0i9spYTkbdEpE3prroCBQTQfOLfWTfscZafO5ral51DRKsGXkVihvUjJ+0oCd3vIOn1r2g67lpnR2AArV8eyeZ73yDxvNGs/uvDaHYuAKlzElgxqNh/n3LGLLSedCMrhk1kca87ibmsJ1Va1fcqUn9YX3LSjrKo27/Y9vo3tHzQeehnpx5m5XVPsbT3Pawd+QrtptxecEzTUX8la98hFvcYzeJed3FgyS+VE/8JDL6wP68995hPzl1avfr1oHHThgzqdgXj757EQ0/dW2S5m0fdQOq+VP7SYwiX9LqKZUsST1mM5/TrTuNmDbmo+xAevXsS454sOkaA+28bz5XnD+fK84eTuu8AAHc9fAdfTp/FFX2v4/Vn32Hk2FsrPeZBA/vSskVTWrc5h1tvvY+Xp0w8rkxgYCDPP/so5/cfQqfO/Vmz9hdu++cNAKxbt4EhV/6DH35YWumxFkml9MsJiEgg8DIwCGgDXF3Ec+QKIF5VOwCfAk957EtX1Th3uaS8l1ZiMqKqO4BXgUnupknAG6r6O5AMdFfVOOBs4P7SvEh41D1TVSeVUCQO8EkyAjQBypSMnOQxp5Q4ytobNgooMhlxpQE93fprALGF9v/mPkbaAw2A8/MfwMBrwPMeD+isUsRzPVCax5lXOVX9u6quL8VxFSryrBZkbE0hY/seNDuHvTMWET2gi1eZmgO6sPuThQDs/WoJNc5pD0BU744cXb+No+u3AZBz4AjkOW95Dif+SvaetEqJuXqnFhzbupv0bXvQ7FxSZiym9kDvmGsPjCfpk/8CsOfLpUSf47zDPLz2dzJ3Oy82RzfsIDAsBAlxRoPrX92brS/NcCpQJTv1cKXEfyLxce2pXi3SJ+curb4Dz2Xm9FkArF6+lshqkdSqU/O4cpddfTFvvvQ+AKpKWurBUxZjnwHn8uUnboyJ64isVrXIGIvTrFUTfvoxAYCfFy2nz8BzKyVOTxdfPID/TP0UgJ9+TqR6jerExNTxKiMiiAhVqjhPe5GRkSQl7QZgw4bNbNr0W6XHWZwK7hnpCmxW1S3uc+9HgFc3qKouUNVj7upSnOfwSlGaF6bngW4iMgo4B3gGQFWzVDXTLRNaXF1uN9AGEUkE/uqxvaDnQESGiMhaEVklIt+775AfBYa675iHikhXEVkiIitEZLGInOFRz+ci8q2I/CoiTxU6d6Jb7zx3WxUReUdEfnbrOr4P2km6ernnHi0iYSLyroiscY8pqi/U6xh3W71i4rrAvZZEEZkuIlWLaLeFIvKkG+cmEenlbg8UkadFZJnbdTbC3V7V7UZLdOO81N3eRJxuuH8Da4GGInKPx/GPeLTL125brXXbfCTOC/oCEVlQ1N8X5wF8lfv7X4HPiyqkqrnAz0D9ovYXcf2BIvKeG8sa9+9wBRAPTHXbOVxEHnKvZa2IvOEmXEWVWygi8W7dV7t1rhWRJz3OeUREHnfbYKmI1C1NrCUJjY0mM2lfwXpW8n5CY6O9yoR4lsnNI+fwMYKiIwlvVg9Vpd20cZw15yka3FbUQ7XihcZEk5m0v2A9M2k/oTFRXmXCYqPJ2OWUUTfm4GjvF/g6F53NoTVb0awcgqo5T+wt7ruSs+dOosObowmpXb2Sr8R/1YmtTcqu3QXru5P3HDeMEVnNedq4474RTJ/7Ps+9+QQ1a3s/tio9xiTPGPdSp5ihlgkvjOOT797n5tE3FGzbtG4z51/YG4B+F55H1cgqVI+qVqkx168Xw84dSQXru3YmU79ejFeZnJwcbrtjDCsT57FjWyJtzmzJO+9Oq9S4SkvzpNRLKdQHdnis76Tk5+ebgFke62EikuA+Vx4//lxGJ0xGVDUbuAcnKRnlrgMFwzircS7oSVVN8jxWRMKAN4GLgc6A91/9Dw8BA1S1I3CJm6U9BHzsvmP+GNgA9FLVs9x9T3gcHwcMxXn3PdSNq7Z77svdeoe4ZR8A5qtqV6AP8LSIVCkUz/3AD+65nwduc5pC2wNXA++711bSMcXFVQsYh9ND0AlIAO4spl2C3DhHAQ+7224CDqpqF6AL8A8RaQpkAJe5dfYBnhUpGD5pCbyiqm2BM9z1rm58nUXkXGAgkKSqHVW1HfCtqr4EJAF9VLW4weh5wLnidPldBXxcVCG3vc4Gvi2mnsLigPqq2s5t93dV9VOc9rrGbed0YIqqdnFjDgcuKqZcfhz1gCeBvu45unj8I1UBlrqPl++BfxRzLTe7/4QJM49tKeXllJ0EBVL97NZsuO1FVl06jpqDuhb0mpzuqpzRgJYPDuOXu98EnGsJq1+LtGWb+Kn//aQlbKLlw9f6OEr/FhgUSGz9uqxctoYh/YezKmENdz880tdhHWfMP8dzeZ9ruf7SW+l0dkcuHjIIgGcfmUzn7mfx8dz3ie9+FruT9pCX6/sZmkFBQdxy89+I7zqAho07sXrNL9x/3x2+DguAvFwp9eL5POUuN5/seUXkWpw3eE97bG6sqvE4IwIviEjz8lxbabvsB+EMy3jN9lHVHe5YUgtgeBHvJFsDW1X1V1VV4INi6l8EvCci/wACiylTHZguImtxEqO2HvvmqepBVc0A1gONgW7A96q61Y011S17Ac6Q0kpgIRAGNCrx6p0eoQ/cejYA24CiZ2t5Ky6uNsAiN4bh7vai5PcyLMcZBsqP/2/usT8BNXGSCwGecJPD73Ay3Py/xzZVXepx/AU4Y4GJOH+jlsAaoL/bG9NLVUvb35sL/IiTiIS7Q3iemrux7gaSVXV1KevdAjQTkckiMhA4VEy5PiLyk4iswUkw2hZTLl8XYKGq7lXVHGAqkN8/nAV85f7u2eZeVPUNVY1X1fhLIpqVeLLM5FRC69UqWA+JrUlmcqpXmSzPMoEBBEVGkJN6mKyk/Rxc+gs5qYfJS88idd4KqnRoeoLLK7/MlFRC6/3R3R5aryaZKQe8ymQkpxJW3ykjbsz5wy6hsdF0fPcu1t7+CunbnHfO2amHyT2WwZ6vfwZg95dLqda+8q/Fn1x9wxV8Nu8/fDbvP+zbvY+Y+n88ndaNrcPu5L1e5dNSD3LsWDpzv3Y6LWd/OY82xUx0rShDb7i8YCLqvt37iKnnGWNt9hSKEWBPirPt2NFjfPPFHNqd5UxL2Lt7H3feNIah/Yfz0sTXATh86EiFx3zrLcNJWDaHhGVzSE7ZTYOGf4zy1m8Qy66kFK/ycR2dp5AtW5zh0U8//ZLu3TpXeFwnoyzDNJ7PU+7yRqHqdgENPdYbuNu8iMj5OG/iL/EYDUFVd7k/t+C8lp5Vnms7YTIiInFAf5wX0dEiUnhOAG6PyFqg18kEoaq34PQWNASWi0hRA48TgAXuO+CLcZKIfJkev+dS8i3LgtNbkj9PoZGqVtZMuqLiEmCux/nbqOpNJzje85oEuMPj+KaqOge4BqgNdHbnY+zmjzY66lGnABM9jm+hqm+r6iagE05S8piIPFSG6/wIeAn4pIh9+XNGmuP0wpRqopOqHgA64jzIbwHeKlzG7W15BbjC7T15E+/HRVllu0kznPhxVCqHV24mrFksoY3qIMFB1B7ck9Q5y7zK7J+TQN0rewNQ+6LupC1aC8CBhSup0roRAeEhEBhA9e5tOLZpZ3lDOqFDK34jolkMYY1qI8GBxAzuwd7ZCV5l9s5OoN6V5wFQ5+JupP64DoCgahGcNfV+Nj82jYPLNnofMyeRqJ7OC1F0r3Yc3XTc897/tGnvfsrl/a7j8n7XMW/W91zi9iB06NyOI4ePsG/P/uOOWTjnR7r27ARAt15d+G3T1kqN8eN3PyuYiDr/2++5+Eo3xk5tOXz46HExBgYGUiPaGY4LCgrkvP492bzB6U2sEV2d/M7bv4/8G1989BWV4dXX3ie+ywXEd7mAmTNnc901VwBwdtdOHDp4iJSUPV7ldyWlcOaZLalVyxnyOv/8c9mwYXOlxFZWFTxMswxoKSJNxZkacRXgdVeMiJwFvI6TiOzx2B4lIqHu77Vw5g2Wa07eie6mEZwJrKNUdTtOF80z7r4GIhKeHxhO78HGQlVsAJp4dN9cXcx5mqvqT6r6ELAXJyk5DHgOQlfnj6zt+lJc21Kc4YOm7jnyB1NnA3fkD2G4jV1Y4XP/gPNij4i0wulJKXythY8pKa6eItLCra+KW2dpzQZuFZHg/HjcYabqwB5VzRZnTktxvS2zgRvFnaciIvVFpI47fHFMVT/A+Tt3KsN1/QBMBIodWFXVfThDWWNKc5HuAzxAVT/DSVSLiic/8djnXs8VHlUUF/fPwHkiUssdWroa+G9pYjopuXn8NvYt2k0bR+cfXmDfzMUc27iTxvcOJfqCeABSPpxHUHQk8UsmU/+Wi/j9MacDMefgUXa+/iVx3z5Jp3nPcGTNFg5859wt0eTBa+ma+DoB4aF0TXydRndfWWEha24eG8e8Q6ePxtLjx+fYPXMJRzfupPm9Q6g9wHmHmPThAoKjIum59EUa3/IXNj/+IQANbxpIRNO6NLvrcrrNe5Ju854kuJYzD+DXCVNpfvcQui14itghvdg0/t8VFnNZ3PPwJK4ZMZrft++k3+Br+ezL2T6JoyTff7eInduSmPXTZzzy7Bgm3PfHTQyfzfvjdu7nJkzhn3f/g88XfMDFQwbx9PgXT1mMP3y3mJ3bkvh66XQefnYMj9//Rw/+J985k2pDQoN5bdoLfDr/P3wy79/sSd7LZx/8HwBdenRi5qKPmbnoY2rWjubNF96r9Ji/mTWPLVu3s/GXRbz22lPcfsfYgn0Jy+YAkJy8mwmPPc+C+Z+TuHwuHTu2ZeKTkwG49NKB/L4lgW7dOjPz//7NN19NrfSYPamWfjlxXZoD3I7zmvAL8ImqrhORRz3eND4NVMUZlfC8hfdMIEFEVgELgEnlvUFAtISo3TGmfqo61F0PxMmmRgMhwLOA4rzbnlJENxBuF/sLwDGcF63mqnqRiFyPc8vQ7SLyOX8MNczDmSMRhdNIwTgvdNuB93He5X8NXKuqTTzrcc/3FfCMqi4UkUE4c0sCcF6o+7sJ1AtAD3f7VlW9qFDMwe65awLv4SRkr+KMmeUAd6rqghMcc6CEuPrizFsIdQ8fp6qFM9KFwN2qmuC+MCe41xsAPIbTOyQ4ydtgt52+xHngJOD0ZA1yq/vK7VHKr/tfwN/d1SPAtThDbU/jfPtBNnCre+47cB6wSYXnjXjGWGj7EVWtKiJNPM/tJoArgdtV9QcRGQ8cUdVnKEREOgLv8kfCPEZVZ4nI5Th/03SgO0734dVACrAJZ0hqfBHlZnm059XAWLf9vlbV+zzjdn+/Amf+yfWFY/P0Q8wVfvWd3hla3Cjo6an3uuNvvTzdxbUt8j3XaS2wzDfZ+db61O2+DuGk5GTtKlWXRXG2dTq/1M83jRO/K9e5TrUSkxFjTMksGalcloycGpaMnBrlTUa2duxf6uebpqvm+lUyYh8Hb4wxxviBUs4F8UuWjBhjjDF+QEvxyar+ypIRY4wxxg/8mb8oz5IRY4wxxg/kWc+IMcYYY3zJhmmMMcYY41N5uZaMGGOMMcaH7G4aY4wxxviUzRkxxhhjjE/ZnBFjjDHG+NSf+QPTLRkxxhhj/IAN0xhjjDHGp/JsAqsxpihH1b/+hSIk19chlIk/funcynXTfB1CmWU8OtLXIZRRTV8H4BPWM2KMMcYYn7IJrMYYY4zxKesZMcYYY4xP/YlvprFkxBhjjPEHuXkBvg6h0lgyYowxxviBPF8HUIksGTHGGGP8gGJzRowxxhjjQ3l/4kkjlowYY4wxfiDvT9wz8uedDWOMMcb8iShS6qU0RGSgiGwUkc0icn8R+0NF5GN3/08i0sRj3xh3+0YRGVDea7NkxBhjjPEDuUiplxMRkUDgZWAQ0Aa4WkTaFCp2E3BAVVsAzwNPuse2Aa4C2gIDgVfc+k6aJSPGGGOMH8grw1IKXYHNqrpFVbOAj4BLC5W5FHjf/f1ToJ+IiLv9I1XNVNWtwGa3vpNmyYgxxhjjB8qSjIjIzSKS4LHcXKi6+sAOj/Wd7rYiy6hqDnAQ54uBSnNsmdgEVmOMMcYPlOXWXlV9A3ij8qKpWJaMGFMJavXpyJmPDYfAAHZOnc/WyTO99ktIEB2m3Ea1Dk3JPnCEVTe/SPqOvQA0G3kp9Yf1gdw8fnngPfYtXA1A438MosG1fQHYOXU+296YVVBfo5sG0OiGC9DcPPZ+t4JNEz6ssGuJ6hNH8wk3IIEBpEydx44pM467ljMm30Fkh2ZkHzjMLyOeJ3PHXkIb1ib++xdI/y0JgEPLN7H5vjcrLK6yGPP4nZzbrwfp6Rk8MHICv6zZeFyZ4OAgHph4D116dCIvL4+XJr7G3K8X+CDa44174jm+X/Qz0VE1mPHBa74OB4DAMzsR9tebISCA7CVzyPru0yLLBXXsQfhNYzn69Cjydmwu2C5Rtaky9hUyZ31I9vwvTuuYJboOVca+St6eXQDk/r6RzE9ePiUxe8qr2JtpdgENPdYbuNuKKrNTRIKA6sD+Uh5bJqdFMiIiDYHvgc6qmioiUUAi0EdVf3fLVAPWAzNU9fZTGNt44IiqPlPJ5+kFvAZkA91VNb0yz1ceIvIW8Jyqrj8NYukNZKnqYl/HUiBAaDPpRpZd+TgZSfvpPvsJ9sxeztFNf/yvNhjWh+y0I/zQbRQxg7vT6sFhrLr5Raq0qk/M4B78eO7dhMVE0WX6OL7vPoqqrerT4Nq+LBn4AJqVQ+ePxrB3TiLHft9NdM821BkYz6K+96FZOYTUqlaB1xJAi4k3sebKCWQmp3LWtxPZPyeBY5t2FhSJGdaXnLQjLOt+B7Uv7UHTcdeyYcTzAGRsSyHx/HsqLp6T0KtfDxo3bcigblfQoXM7HnrqXq4edNNx5W4edQOp+1L5S48hiAjVoyqwHctp8IX9GXb5JYydUKlPQ6UnAYQNuZVjL49D0/YTcffz5Kz9ibyUHd7lQsMJPu8Scn/fcFwVoZf9nZz1y09RwJQ75rx9KRx7auSpi7cIFXxr7zKgpYg0xUkkrgKGFSozExgOLAGuAOarqorITOBDEXkOqAe0BH4uTzCnxZwRVd0BvApMcjdNAt7IT0RcE3ASFr8hjtK28TXARFWNO50TEQBV/fvpkIi4egM9ynKAm+FXmhqdWnBsawrp2/ag2bmkzFhM3YHxXmXqDown6RPn4bz7y5+oeU7bgu0pMxajWTmkb9/Lsa0p1OjUgiot63MwcTN56Vlobh4HFv9C3b8488UaDu/P1sn/h2blAJC171CFXUvkWS1I35pCxvY9aHYOe2csouYA72upOaALuz/5LwB7v1pK1DntKuz8FaHvwHOZOd3pRVq9fC2R1SKpVafmceUuu/pi3nzJmaunqqSlHjylcZYkPq491atF+jqMAgGNW5G3Nxndvxtyc8hJ/J6g9t2OKxf6l2vJ+u5TNDvba3tQ+27o/hTyUrafqpDLHfPpILcMy4m4c0BuB2YDvwCfqOo6EXlURC5xi70N1BSRzcCdwP3useuAT3A6CL4FblPV0py2WKdFMuJ6HugmIqOAc4CCtwAi0hmoC8wp7mAR+V1EHhGRRBFZIyKt3e3jReRuj3JrRaSJu2wQkfdEZJOITBWR80VkkYj8KiKeM4M7isgSd/s/POq6R0SWichqEXnE3dbEve/638BavLuyEJF+IrLCjfEd9z7uvwNXAhNEZGoR1/Y39xyrROQ/HueZ726fJyKN3O3vichLIrJYRLaIyBXu9o9E5C8edb4nIleISKCIPO1xHSPc/b1FZKGIfOq201R3FjXu9nj39wvctkkUkekiUvUEf4+qIvKuu221iFxeUj2F2mGkiKx3j/tInHvebwFGi8hKEel1gnZ5TUR+Ap4SkeYi8q2ILBeRHzziG+I+RlaJyEklv6Ex0aQn7S9Yz0hKJTQm2rtMbDTpu5wymptHzuF0gqMjnWN3eRyb7Bx7ZMMOos5uTXBUVQLCQ6h9fhxh9Z0X1CrNY4k6uzXdZj1G1y8eolpcs5MJu+hriY0m0+NaMpNTCYmtWUSZfc5Kbh45h48RFO28cIY1qkOnuU/R4YtHqHZ26wqLqyzqxNYmZdfugvXdyXuoG1vbq0xkNefhdsd9I5g+932ee/MJatb2/puZPwTUqEle2t6C9by0fUh178dFQIPmSI1a5K5P8D44JIyQ868gc9a0UxHqH/GUJ2YgoGZdIu59kfCREwls1rbS4y1Knkipl9JQ1W9UtZWqNlfVx91tD6nqTPf3DFUdoqotVLWrqm7xOPZx97gzVHVWcecordMmGVHVbOAenKRklLuO27PwLHB3CYfn26eqnXB6WUpTvoVbd2t3GYaTCN0NjPUo1wHoC3QHHhKReiJyAU7XVFcgDugsIue65VsCr6hqW1Xdll+JiIQB7wFDVbU9zjDZrar6Fk532D2qeo1ngCLSFhgH9FXVjsC/3F2TgfdVtQMwFXjJ47BY9zou4o/epo9xEh5EJAToB3yNcx/5QVXtAnQB/iFOtx3AWcAonHvQmwE9C8VWy43tfLfdE3Cy53xF/T0edM/X3o19finqyXc/cJZ73C1uz9lrwPNuj9IPJ2iXBkAPVb0TZ2LXHara2Y3tFbfMQ8AAt60v4TRx9NcktkyZSfzHY4mfNoZDa7ehuc4NfBIUSHBUVZYOGsfGR6cS9+YoH0fryNp9gJ8630pi/3vZ8vD7nPnKvwisGu7rsIoUGBRIbP26rFy2hiH9h7MqYQ13P+zbLnm/JkLoZX8nc8bbx+0KHTSMrIUzICvDB4GVoISY9VAqRx6+gWNP/YvML94ibPjdEHbqH8tahsXfnBZzRjwMApKBdsBcd9s/gW9UdaecONv73P25HPhrKc63VVXXAIjIOmCeOx62BmjiUe7/3KGTdBFZgJOAnANcAKxwy1TFSUK2A9tUdWkR5zvDPecmd/194DbghRJi7AtMV9V9AKqa6m7v7nGN/wGe8jhmhqrmAetFpK67bRbwooiE4nxIzfeqmu4mVR3ye1BwJii1BLKAn1V1p9s+K902+dHjPN1wEpVF7t8mBGdsMV9Rf4/zccYmca/ngIhcdIJ68q0GporIDGBGEftP1C7TVTXX7XXpAfw/e/cdH1WZ/XH8800oSei9KSDYCyJiwy5gYS3YcBV7QdHFtjbsq+7C6s+2uquLuoi6uva6q4LYCwoiWFABUUAISgs9Icmc3x/3JkxCqiS5d8J5+5rXzH3unZmTGDInTznPs0k/U43D+4+ARyU9kxR/CQqWyA0DGNGsL4Mye5Y4n7doGZmdN/zFldG5NXmLlpW8JnsZmV3akJe9DKWn0aBZJvnLVgXP7ZL03E4bnrvgyXdY8GQwoXKba39PbthjkbtwKb/8NxiuXfHFD5AwGrZpRv7SVeV8i6ouL3sZjZO+lsadWrM+e2kZ17RlffYySE+jQbMsCpYF712wfjUAq7+cw7q5v5DZsxOrp8+htp181gmccGpQkl5OrQAAIABJREFUMuHraTPo2KVD8bkOndrzS/biEtfnLFvB2rXriiesvvnqRI47JTa5aOwkcpbSsOWG3qW0lm2xFUk/F40zSevUlawRowBQ81ZkDruBdWNuJa37djTovS+Njz4LZTYBM8jPJ/+D12Ibc2L+bCgIfqYT838gsWQRae26lJiQWxd81946IKk3MJDgA+5DSf8xs2yCD5f9JV1I8IHfSNJqM9uodC2QF94XsuFrK6BkD1BGGddD8P85L+lx8vemdKJpgAjmePyz1NfRHVhTzpdZV5K/LkHQ3SbpXeAw4CSCAjdF50eY2ZvJL6BgYmjy6yR/T5Nfe4KZnVxJHGU9tzqvU+R3wAHAUcB1knap5PrSiv6/pAE5Zta79AVmdoGkvcL3+lzS7ma2tNQ1xUvm3ujw+43+CFnxxQ9k9ehIZtd25GYvo+Pgfnw5/L4S1/z65ud0HnIAOVNm0eGovVj64TfF7b0eGMGPD/6XjI6tyOrRkZypwS+8Rm2bs37JSjK6tKHDoD2YNOiG4DmvT6H1vjux7KMZZPXohBo2qJFEBGDVtNlk9uhERtf25GUvo93gffnuwntLXLN0/BQ6DDmQVZ/PpN2Re5Pz0dcANGzTnPzlqyGRIKNrezK36kTu3F9rJK7KPDX2OZ4aG6yUOGDAvpxy9gn878Xx9Np9Z1avWs2SX5du9Jx3x3/Invv24dMPP2fv/ffgh5k/1kmsqSgxbyZp7Tqj1h2wFUtp0OcAcsfdseGC3LWsuXZDJ2/miFHkvfQIifmzWXfv1cXtjY44BctbV+uJyKbGrKbNsTWrwRKoTQfS2nUmsXRRrce80ddQf7emiUcyEs5FeIBgeGaepDsI5owMTR62kHQm0LecRKQ8PxEMVyCpD7BVhVeX7RhJo4AmBBMmrwHWEc7xMLPVkroQrISpyPdAd0lbm9ls4DTgvUqe8zbwoqS7zGyppNZh78jHBD0MjxNMfv2gCl/H08C5QF/gzLDtTWC4pLfNLF/StlR9idYk4O9FX4+kJkCXpJ6fskwg6A26FEDByqlKXyccrtvSzN6R9GH4tTcFVgHJyx4q/b6Y2UpJP0o60cyeDX/+epnZdEk9zexT4FNJRxDM+dn4k6sCVphgxsix9P3PtSg9jZ+feofV3//M1ledyIrpc1j85uf8/OQ79Lr/IvafdA/5OauZfn4wmrT6+59Z9Mon7P/BnVhBITOuGVu8VWfvRy6nUaumJAoKmTFyLAUr1wLw81PvsMs9F7Dve3eQWF/AVxf/o9zYqq0wwexrH2Hnp64LlvY+9Q5rv/+ZbledxKppP7Bs/BQWPfk2298/gj0+uY/8nNXFK2la7L0D3a46CcsvxBIJZl01hoKc1TUXWxW9/9ZHHNC/H69/+jy563K5/pJbi889P/Fxju9/GgB33Xo/o++/matvvYzlS3NKXBe1K28azeQvviQnZyX9B5/KheecxvFHbfJ2IL9dIkHucw+SdeEtwTLZSRNILJpHo0FDKZw3i8KvN2lhRe3YhJjTe+5Mo0FDobAQLEHuM3+HtXX/s1yVMu+pSmbRjy6F3d79zeyk8DidYNnRZWb2XtJ1ZxIkIxst7ZX0U3huSTi58v/M7CBJmcDLBNXhPiXoaTkifNprZrZz+PxHw+Pnwt6N18xsZwVLe3sQDF20BW43s4fC51xC8OEOsBo4laAXoPh1y4izP0Gi1SD8GoebWV7y+5fxnDMI5tMUAl+Y2ZmSugFjw5gWA2eFiVyJ1wl7kYomlTYEfiEYdjorbEsDbiPobVD4WoMJ5otcYWZFidz9wBQzezTsYbnCzKZIOoRgv4KiYY7rzeyVCv5/NCXYD2H38Ov5k5m9UN7rJH0PGgLvEAwjCXjCzEaHydNzBL1ZIwiGyaryfdmKIAHuBDQkKG18i6QXwv/XAiYSJMjl/iMpq2ckzrK0SRPe69xwq7vVFjVl2jd1OzGzJuTe4vNj6kKzv722SdnEY11OrfLvm9MXPJFSmUsskhHnUpUnI7XLk5G64clI3djUZOTRaiQjZ6ZYMhKLYRrnnHPOVSyl/vKpJk9GnHPOuRTgE1idc845Fylf2uucc865SBV6z4hzzjnnouQ9I84555yLlCcjzjnnnIuUr6ZxzjnnXKR8NY1zzjnnIuXDNM4555yLVGrVT64eT0acc865FODDNM4555yLlA/TOOfK1Khe/3qI3gPqyh/4OeowqiUVN53LuPFvUYdQLetGXhB1CJHw1TTOOReBVEtEnKtNiXqcjngy4pxzzqWA+jyBNS3qAJxzzjlXuUQ1bptCUmtJEyTNCu9blXFNb0mfSPpG0peSTko696ikHyVNC2+9K3tPT0acc865FJBQ1W+b6BpgopltA0wMj0tbC5xuZjsBhwP3SGqZdP5KM+sd3qZV9oaejDjnnHMpIIFV+baJjgHGhY/HAYNLX2BmM81sVvh4IfAr0O63vqEnI84551wKsGrcJA2TNCXpNqwab9XBzLLDx4uADhVdLGlPoBHwQ1Lzn8Phm7slNa7sDX0Cq3POOZcCqjMXxMzGAGPKOy/pLaBjGaeuK/U6JqncrhZJnYDHgTPMrCjEkQRJTKMwhquBWyqK15MR55xzLgUU1uDSXjMbUN45Sb9I6mRm2WGy8Ws51zUH/gtcZ2aTkl67qFclT9JY4IrK4vFhGueccy4F1NVqGuAV4Izw8RnAy6UvkNQIeBF4zMyeK3WuU3gvgvkmX1f2hp6MOOeccymgDiewjgYGSpoFDAiPkdRX0sPhNUOAA4Azy1jC+29JXwFfAW2B2yp7Qx+mcc4551JAXdVfNbOlQP8y2qcA54aPnwCeKOf5h1T3PT0Zcc4551JAfd4Jy5MR55xzLgXU5ATWuPFkxLla1vrgXdnmtrNQehrZ/57I3PtKzgVTowbseP8faNarB/nLV/HNsHvInb+YZrv1ZPv/Oz+8CH6841mWvD4ZgH0m30/hmlysMIEVFDLlsJE1GnOrg3vT89Yg5kX/nsj8+1/aKObt7htRHPO3599N3vzFNN6yHX3fv4d1PywEYOXnM5l99UMlnrvTuKvJ6Naezw/6Y43GXNrVt13G/v37kbsulxsuuZVvv5q50TWPvPB32rVvQ25uHgAX/P5Sli1ZTqctOnLL3dfRqk1LVuSs5NqLbuaX7MW1Gm/6Dn3IOG4YpKWR/8l41r/1XJnXNdi1H5nnXMuaOy4lMX92cbtataPJtf8g7/UnyX/7xVqNtSqu/8tdvP/RZ7Ru1ZKXnngw6nAASN+pLxm/H47S0lj/wRusf+PpMq9r0Gc/sobfyOrbLiIxdxZq0ozMC24gvft25H88ntyn/l7HkQd8ozznKiFpS+B9YHczWxbuZTAVONjMfpJUSDCZCWCemR1dxms8CrxWemZ2qWvOBMaHFf8IJ1PdZWYzJF1rZn+pZtzdw/fcuTrPq7I0sd3oc/hiyG3kLVxK3zdHsfjNKayduaD4ks6nHEJBzhom7X0x7Qf3o+cNQ/lm2D2s+W4+Uw69BitM0Kh9S/Z85w4+Gv85Vhh01n5x3J/IX7aqFmJOY+tR5/DVkFvJy17Gbm+MYun4KayduWEH3Y6nHEJBzmom7zOCdsf0Y6vrT+W78+8GIHfuIqYOuLLMl24zaE8K1+TWfMyl7Nd/H7r12JIj9zmRXn124vq/XsXQQeeWee01F93MjOnflWj7400jePXZ13nlmf+x5767c/G1w7luRIVlEjaN0sg4cThr/349lrOUrCvupuDrT0ksml/yusaZNDzwaAp/+m6jl2h87LkUzPi89mKspsGDBnLK8Udz7a3/F3UoAaWRecofWHP3NdjyJTS57j4Kpn9CInteyesaZ9Ko/7EUzPm2uMny88l7eRxpXbqT3rl73cadpP6mIr6axtUQM5sPPEA46zq8H2NmP4XH65L2KdgoEamGM4HOSe97rpnNCA+v3YTXrRXN+2zN2h8XkTv3Vyy/kF9f+ph2h+9R4pq2h/cl+5l3AVj86iRa7RfkRYl164sTj7SMhmB186uo2W5bs+7HReTO+xXLL2DxSx/R5rC+Ja5pc9ge/PLMe0HMr22IuSJpWRlscf5RzLvn+VqJO9nBhx3Aq8+8DsCXU7+hWfOmtG3fpsrP77Ftdz79cAoAn330OQcffkCtxFkkrdu2JBZnY0t/gcICCqa+T4Nd9t7ousa/O5X1bz2H5eeXaG+wy97Y0kUkFs3b6DlR6dt7F1o0bxZ1GMXSt9qOxOKF2JJFUFhA/uT3aNC730bXNR58RtBjkr9+Q+P6XApnf1OyLQJ1uJqmznky4mrS3cDeki4F9gN+859Ekm6UNFnS15LGKHAC0Jdg2dg0SZmS3g2Xm40GMsP2f0vqLunrpNe7QtLN4ePdJU2XNB24KOmadEl3hO/7paTzf2v8RRp3bE3ewqXFx3kLl9K4Y+uS13RqTd6C4BorTFC4ai0NWwe/xJv32Zo937uTPd+9k++ufKg4OQHo/fR19B0/ms6nbTTpfdNi7lQq5uxlNOrUpoxrlgQHhQkKVq2lQRhzRtf29JlwO71e/BPN99q++Dndrz6Jnx98lcJ1eTUab1nad2rHooW/FB//kr2Y9p3K3jbj1nuu55m3xjHssrOK22Z+M5sBgw4CoP+gA2narAktWjWvtXjTWrYhkbNhGCiRswS1KPk9T9uiJ2rZlsIZU0o+uVEGjQacQN7rT9VafPWBWrYlsWzD99iWLyatZanvcdetSWvVjoKvPqvr8KqkDuuM1DlPRlyNMbN84EqCpOTS8LhIRrg/wiRJG226VIb7zWyPcPgkEzgyHL6ZAgwNe1jWJb33NWzofRlayWuPBUaY2a6l2s8BVpjZHsAewHmStqpCrLVm5dTZfHbgH5ly2Ei6X3IsaY0bAvD5UTcweeA1TD/lL3Q56zBa7r1DlGEWW//Lcj7dfThTB17FnJvGscM/LiG9aSZNdupOZveOLH09Xr/kR154M8cffCpnHjOcPnvtylEnHgHAnX+6j9332Y2nJ4yj7z678cvCX0kURvgrXqLxseeS99IjG51qfMQprH/3JVhf+8Nf9ZpExpDzyX223ArqkbNq/JdqfM6Iq2lHANnAzsCEpPZuZrZAUg/gbUlfmdkPZb5C4GBJVwFZQGvgG+DVTQ0u3OK6pZm9HzY9HsYMcCjQK+yBAWgBbAP8WOo1hgHDAC5rtjtHZvYo9/3yFi2jcecNf3017tyGvEXLSl6TvYzGXdqQl70MpaeR3ixro7kga2ctoHBNLk2235JV0+ewftFyAPKXrGTJ/ybTbLetyZn0LTUhL7tUzJ1asz57aRnXtGV99jJIT6NBsywKwpgL1q8GYPWXc1g39xcye3aiWe+tabprD/ac/HeUnk7Dti3o9cLNfHnczTUSM8BJZx3P8UODEcBvpn1Lx84b9vbq0Kkdv5YxAfXXRUHb2jVr+d+L49l5tx159dnXWfzLEi4/J5gUnJmVyYDfHcyqlatrLNbSEjlLadhyQ89NWsu22Iqk73njTNI6dSVrxCgA1LwVmcNuYN2YW0nrvh0Neu9L46PPQplNguG8/HzyP3it1uJNRZazhLTWG77HatWORE7S9zgjk7TO3WlyxR3B+RatyfrDLay9/0YSc2fVdbhl8tU0zlVBWH1vILA38KGk/xTtUWBmC8L7OZLeBXaj5A6Pya+TAfwD6Gtm88PhlYxqhlNAyZ6/qjxfBD0mb1Z0UfIGVG93GFLhb4dVX/xAVo9OZHRtR172MtoP7seM4X8rcc2SNz+n05CDWDllFu2O2pvlH34TBNy1HXkLlmKFCTK2aEvW1p3Jnb+YtKzGSKJwTS5pWY1pfVAvfryz3Dm/1bZq2mwye3Qio2t78rKX0W7wvnx34b0lrlk6fgodhhzIqs9n0u7Ivcn5KBgRa9imOfnLV0MiQUbX9mRu1Yncub+yevocsseNB6Dxlu3Y+fFrajQRAXh67PM8PTaYj7L/gH6cfPYJvP7SBHr12YlVq9aw5NeSCVV6ejrNWjQlZ9kKGjRI58CB+zLp/WAIpGXrFqxYvhIz49yLT+fF/9TuB3ti3kzS2nVGrTtgK5bSoM8B5I67Y8MFuWtZc+2GDr/MEaPIe+kREvNns+7eq4vbGx1xCpa3zhORMhT+9D1p7bugth2x5UtouMeBrHt49IYL1q1l9eUnFh9mXXEHuc+OiU0iAqk5/FJVnoy4GhHuQfAAwfDMPEl3EMwZGRqurFlrZnmS2gL7ArdX8HJFicMSSU2BE4CiT9tVQHmz4vIlNQyHh34B2ktqA6wGjgTeMLMcSTmS9jOzD4HkIZ03geGS3jazfEnbAgvMbE01vx3FrDDBzJH/ovd/rkPpaSx86h3WfP8zW101hFXTf2DJm5+T/eTb7Hj/H9h70t8oyFnN1+ffA0DLPben64jBWEEhJBJ8f80j5C9bRUa39uwyNth3Sunp/PLihyx7Z/pvDXFjhQlmX/sIOz8VxLzoqXdY+/3PdLvqJFZN+4Fl46ew6Mm32f7+EezxyX3k56wuXknTYu8d6HbVSVh+IZZIMOuqMRTk1F6PQnk+eOtj9u/fj/9OepbcdXnccOmGatTPvDWOIQPOoFHjhjz41D00aNiAtPQ0Pn1/Ms8/ESy73qNfHy6+djhmxtRJ0/jzyFpeEZJIkPvcg2RdeEuwtHfSBBKL5tFo0FAK582i8Ot4DW9VxZU3jWbyF1+Sk7OS/oNP5cJzTuP4ow6LLqBEgtwn7yfr0r8gpbH+ozdJLJxL46NPp3DuTAqmT6rw6U1HPYYysyC9IQ1268fau0duvBKnliXqaBJ7FGT1+ItzdSccuuhvZieFx+nAZOAyIB/4J0FinwbcY2YbDX4nL+2VdBtwMsE21DOBuWZ2s6Tjgb8A64B9gNeBK8xsiqS/AkcDU81sqKSLgUuABcAc4KfwNXYH/kWwUm48MMjMdpaURrCHwlEEvSSLgcFmtqK8r7uynpG4aVD+TuCx9Ad+rvyimPloSNVX7cRFxo1/q/yiGFk38oKoQ/hNmj80Xpvy/FO7HVflf8BPzH1hk96rrnky4twm8GSkdnkyUjc8Gakbm5qMnNLt2Cr/A35y7osplYz4MI1zzjmXAlJxlUxVeTLinHPOpYACT0acc845FyXvGXHOOedcpHxpr3POOeciVZ8XnHgy4pxzzqWAVNwAr6o8GXHOOedSgJeDd84551ykvGfEOeecc5Gqz3NG0iq/xDnnnHNRS1TjtikktZY0QdKs8L5VOdcVSpoW3l5Jat9K0qeSZkt6WlKjyt7TkxHnnHMuBVg1/ttE1wATzWwbYGJ4XJZ1ZtY7vB2d1P5X4G4z2xpYDpxT2Rt6MuKcc86lgARW5dsmOgYYFz4eBwyu6hPDHdwPYcNO61V6vicjzjnnXAootESVb5KGSZqSdBtWjbfqYGbZ4eNFQIdyrssIX3uSpKKEow2QY2YF4fHPQJfK3tAnsDq3CZalNYw6hGpJtelvM5bOizqE3yD1du1NtV1wM0c9GHUIkajO8IuZjQHGlHde0ltAxzJOXVfqdUwqd7vvbma2QFIP4G1JXwErqhxkEk9GnHPOuRSQqMHVNGY2oLxzkn6R1MnMsiV1An4t5zUWhPdzJL0L7AY8D7SU1CDsHdkCWFBZPD5M45xzzqUAq8ZtE70CnBE+PgN4ufQFklpJahw+bgvsC8ywYP3xO8AJFT2/NE9GnHPOuRRQhxNYRwMDJc0CBoTHSOor6eHwmh2AKZKmEyQfo81sRnjuauBySbMJxi0fqewNfZjGOeecSwF1VYHVzJYC/ctonwKcGz7+GNilnOfPAfasznt6MuKcc86lgELb1HJm8eXJiHPOOZcCaqCYWWx5MuKcc86lgPq8N40nI84551wK8F17nXPOORcp7xlxzjnnXKQKN3k/3vjyZMQ555xLATVZgTVuPBlxzjnnUoCvpnHOOedcpOpzz4iXg3eujux66+kc/vGdDJg4ipa7dC/zmpa9ujPw7dEc/vGd7Hrr6Rud3+b8QZyQ/W8atW4KQLOtO3Hwqzdz7E+Psu0Fg2o03t63ns4RH9/JwEriPfTt0Rzx8Z30LiPebc8fxIlJ8TZskUW/f13KwImj6P+/W2i+3RY1GnOyu++6he9mfMjUzyewW++dy7zmpJOO4YupbzH18wn899UnaNOmFQDHH38k06e9zfrc+ezep1etxZgsfYc+NLnuQZrcMIZGA04o97oGu/aj2d9eI23LrQFQ6/Y0/b/nybrqb2Rd9TcaD7mobuLdqS9Nbn2Epn8eS6PDTyo/3j770fyh8aR12yaIt0kzsv54O83ue5mMk+sm1qq4/i93ccDvfs/gU+O7g7FV479U48mIq5SkLSX9KKl1eNwqPO4eHneVNF7St5JmFLWXeo1HJZX/G/a3x/ZxeN9d0im/4flnSrq/puMqreMhu9KsR0fe6PdHpl75CH1Gn1XmdX1Gn83nVzzMG/3+SLMeHel4yK7F5zI7t6bDQbuw5uclxW3rl69h2vWPMfPB/9Z4vE17dOT1fn/k8wri3X302Uy54mFe7/dHmlYh3h0uPoacr+cxof9IPrv4AXrfelqNxl3kiMMPYZutt2L7Hfdj+PCr+fv9oza6Jj09nbvvvIUBA0+kz+4D+errb7nowuDr/Oab7zhxyHl88MGkWolvI0oj48ThrH3wJtb85UIa7H4gaR233Pi6xpk0PPBoCn/6rkRzYski1t5+MWtvv5i8Z/5eJ/FmnvIH1t57HatvPI+Gex5EWqeuZcbbqP+xFMz5trjJ8vPJe3kcuc+Vu7t9JAYPGsiDd90WdRgVSphV+ZZqPBlxlTKz+cADhJslhfdjzOyn8Pgx4A4z24FgP4Iyt5uupdj6hQ+7A9VORupK58N3Z+6zHwCwbOpsGjbPIqN9yxLXZLRvSYNmmSybOhuAuc9+QOfDdy8+v+ufTuOrW5+CpF80eUtXsnz6HCy/sFbjbfQb4u39p9P4slS8zbftwq8ffQPAqtnZNNmyHY3bNq/R2AGOOuowHv/3cwB8+tlUWrRsQceO7UtcIwlJNGmSBUCzZs1YuPAXAL77bjYzZ/5Q43GVJ63btiQWZ2NLf4HCAgqmvk+DXfbe6LrGvzuV9W89h+Xn11lsZUnfajsSixdiSxZBYQH5k9+jQe9+G13XePAZrH/jachfv6FxfS6Fs78p2RYDfXvvQovmzaIOo0KFlqjyLdV4MuKq6m5gb0mXAvsB/wcgaUeggZlNADCz1Wa2tqIXktRf0heSvpL0r6RtqH+S9CdJU8Nz24ft7SRNkPSNpIclzQ23rEbS6vBlRwP7S5om6bLSPR6SXpN0UPj4LEkzJX1GsO01Se/zvKTJ4a343KbK7NiatQuXFh+vy15GZqdWJa/p1Ip1C5eVvKZjawA6HbY76xYtY8WMeTUVUrXiXVvNeDuXE2/OjHl0GbQHAK169yBri7Zkdm5d4/F36dyRn+cvLD5e8HM2XTp3LHFNQUEBF40YybSpE5k/dyo77rAN/xr7VI3HUhVpLduQyFlcfJzIWYJatCl5zRY9Ucu2FM6YsvHz23Qg66p7ybx4FOk9dqr1eNWyLYllG+K15YtJa1kq3q5bk9aqHQVffVbr8WwufJjGbfbMLB+4kiApuTQ8BtgWyJH0Qphg3CEpvbzXkZQBPAqcZGa7EEyiHp50yRIz60PQE3NF2HYT8LaZ7QQ8B5TRH8w1wAdm1tvM7q7g/TsBfyJIQvYDdkw6fS9wt5ntARwPPLzxK4CkYZKmSJoyYe3s8t6qxqRnNmKHi4/mm9ufq/X3qgnpmY3Y/uKj+bqMeL+771UaNs9i4IS/sM05h5Hz9U9YYTS/OBs0aMAFw06n756HsWW3Pnz51bdcc/WISGKplETjY88l76WNd2K3lctYfdNZrL39EvJefJiMM66AjMwIgkwikTHkfHKfjddQTKozS1T5lmp8NY2rjiOAbGBnYELY1gDYH9gNmAc8DZwJbPxbM7Ad8KOZzQyPxwEXAfeExy+E958Dx4WP9wOOBTCzNyQt34SvYS/gXTNbDCDpaYKECmAAsKOkomubS2pqZquTX8DMxgBjAJ7rNLTcT9KeZw5kq6EHA7Bs+hyyOrehqK8hs1Nr1mWX/DLWZS8v0UuQ2ak16xYto0m3DmR1bcfAiaOK2weM/zMTj7iRvMUrqv8dqCDeHuXEm1XNeJt0bcehSfEOHP9n3grjnXLZhg+oQZ/dw5q5NTOqN/yCMzjnnKEATJkyjS227Fx8rssWnViwcFGJ63vvGvQgzJkzF4DnnnuVq66MZkJlImcpDVu2Kz5Oa9kWW7GhZ4rGmaR16krWiOB7quatyBx2A+vG3Epi/mwoWBW8zvwfSCxZRFq7LkF7LbGcJaS13hCvWrUjkZMUb0YmaZ270+SKO4LzLVqT9YdbWHv/jSTmzqq1uOo7LwfvNnuSegMDgb2BDyX9x8yygZ+BaWY2J7zupfCa8pKRyuSF94Vs2s9nASV7/jKq8Jw0YG8zy92E9y32w6MT+OHRIGfr2L83W599KPNf+oTWfbYmf9U6cn/NKXF97q85FKxaR+s+W7Ns6my6nbg/sx95k5Xfzee1XS4svu6Iz+5h4uHXs35ZiRwpVvG+mhTvoM/u4a0w3obNsyhYl4flF7LV0INZPOk7Clavq5H4H3hwHA88OC54zyP6c+HwM3n66ZfZa88+rFyxkkWLSiY9CxYuYocdtqFt29YsWbKMAQMO4Lvvar+nqyyJeTNJa9cZte6ArVhKgz4HkDvujg0X5K5lzbVDiw8zR4wi76VHSMyfjZo2x9asBkugNh1Ia9eZxNJFZbxLzSn86XvS2ndBbTtiy5fQcI8DWffw6A0XrFvL6stPLD7MuuIOcp8d44nIJvJy8G6zpqCr4AGC4Zl5ku4gmDMyFJgMtJTULuxtOATYeFB7g++B7pK2NrPZwGnAe5WE8BEwBPirpEOBVmV5ZoEGAAAgAElEQVRcswpInn32E3ChpDSgC8HEWoBPgXsltQFWAicC08Nz44ERwB3h193bzKZVEluVLJo4jY79e3P4J3dRuG49Uy77Z/G5ARP+wlsDrwXgi5Fj6XvP+aRnNGLR29NZ9Pb08l4SgMbtWtD/jdto2CwTSyTY+rwjGH/gVZv8Ab9o4jQ69e/NEWG8k5PiHTjhL0wI4506cix7VCPeZtt0Zs97LwAzVsxcwJTLa6cb/3+vT+Twww/h+28/Yu26dZx77uXF56ZMHk/fPQ4lO/sXbr3tbt55+wXy8/OZN28BZ59zGQDHHHM49959G+3ateaVlx9j+vRvGHTk0PLebtMlEuQ+9yBZF94CaWnkT5pAYtE8Gg0aSuG8WRR+Xf68i/SeO9No0FAoLARLkPvM32FtzSaqZcb75P1kXfoXpDTWf/QmiYVzaXz06RTOnUnB9IpXITUd9RjKzIL0hjTYrR9r7x5JIrtu5kOV58qbRjP5iy/JyVlJ/8GncuE5p3H8UYdFGlNp9blnRPU503I1Q9IwoL+ZnRQepxMkIZeZ2XuSBgJ3AiIYXhlmZutLvcajwGtm9pyk/gTJTIPwdYabWZ6kn4C+ZrZEUl/g/8zsIEntgaeADsAnwJFA9/A5q82sqaSGwJtAG4I5KfcATwC7A98SJDA3m9m7ks4CRgI5wDRgvZn9IZwU+3dghzC2982swqIDFQ3TxFFKBQucvPTdqEOotuUX7BZ1CNVm6+K1sqUymaMejDqE36Rh2x6q/KrydWq5Y5X/CWfnzNik96prnoy42AtX2xSaWYGkfYAHzKx31HGBJyO1zZORuuHJSN3Y1GSkY8sdqvxPeFHOtymVjPgwjUsFXYFnwiGX9cB5EcfjnHN1rj53Hngy4mLPzGYRrNZxzrnNVn2eM+J1RpxzzrkUYGZVvm0KSa3DQpOzwvuNFg1IOjgsMll0y5U0ODz3aLhlSNG5SofVPRlxzjnnUkBhIlHl2ya6BphoZtsAE8PjEszsnbDIZG+CVZRrCVYkFrmy6HxVViV6MuKcc86lgARW5dsmOoagICXh/eBKrj8BeL2yrUAq4smIc845lwLqapgG6BAWtQRYRFBWoSK/Jyi/kOzPkr6UdHfR/mMV8QmszjnnXApIVCPJCOtDDUtqGhNuZVF0/i2g40ZPhOuSD8zMJJX7xuF+X7sQ1HkqMpIgiWlEsHXG1cAtFcXryYhzzjmXAqqzG2/yHlrlnB9Q3jlJv0jqZGbZYbJR0QZSQ4AXkzZPJalXJU/SWDZselouH6ZxzjnnUkDCrMq3TfQKcEb4+Azg5QquPZlSQzRhAlO0lchg4OvK3tCTEeeccy4FJCxR5dsmGg0MlDSLYDfz0QCS+kp6uOgiSd2BLdl4f7F/S/oK+ApoC9xW2Rv6MI1zzjmXAuqqAquZLQX6l9E+BTg36fgngo1IS193SHXf05MR55xzLgXU53LwvlGeczEkaVjyzPdUkGoxp1q84DHXhVSLt77wOSPOxdOwyi+JnVSLOdXiBY+5LqRavPWCJyPOOeeci5QnI84555yLlCcjzsVTKo5Zp1rMqRYveMx1IdXirRd8AqtzzjnnIuU9I84555yLlCcjzjnnnIuUJyPOOZcCJLWS1CvqOOorSVlRx7A582TEuRiQtK2kiZK+Do97Sbo+6rgqIilL0g2SHgqPt5F0ZNRxlUdSO0nXShoj6V9Ft6jjqoikdyU1l9QamAo8JOmuqOOqjKRukgaEjzMlNYs6pvJI6idpBvBdeLyrpH9EHNZmx5MR5+LhIWAkkA9gZl8Cv480osqNBfKAfcLjBVRhQ6wIvQy0AN4C/pt0i7MWZrYSOA54zMz2Iti4LLYknQc8B/wzbNoCeCm6iCp1N3AYsBTAzKYDB0Qa0WbI96ZxLh6yzOyzYMftYgVRBVNFPc3sJEknA5jZWpX6AmImy8yujjqIamoQbsc+BLgu6mCq6CJgT+BTADObJal9tCFVzMzml/rRLYwqls2V94w4Fw9LJPUEDEDSCUB2tCFVar2kTDbE3JOgpySuXpM0KOogqukW4E3gBzObLKkHMCvimCqTZ2briw4kNSD8GYmp+ZL6ASapoaQrgG+jDmpz43VGnIuB8ENmDNAPWA78CAw1s7mRBlYBSQOB64EdgfHAvsCZZvZulHGVR9IqoAmwnnA4DDAzax5dVPWPpNuBHOB0YARwITDDzGLZsyOpLXAvwfCXCH6WLzGzpZEGtpnxZMS5GJC0lZn9KKkJkGZmq4raoo6tIpLaAHsT/BKfZGZLIg6pXpG0LfAA0MHMdg5X0xxtZrGdmyMpDTgHOJTg5+JN4GHzDxtXAU9GnIsBSVPNrE+pts/NbPeoYqqMpD5lNK8A5ppZLOe7SDqaDZMT3zWz16KMpzKS3gOuBP5pZruFbV+b2c7RRpb6JN1HBcNHZnZxHYaz2fMJrM5FSNL2wE5AC0nHJZ1qDmREE1WV/QPoA3xJ8BfwzsA3BF/LcDMbH2VwpUkaDewB/DtsukTSvmY2MsKwKpMyE5slfUXFH+5xq5EyJbzfl2Co8enw+ERgRiQRbcY8GXEuWtsBRwItgaOS2lcB50USUdUtBM4xs28AJO1IMOHyKuAFgrH3OBkE9DazBICkccAXBEuq4yqVJjYX1Zi5KLx/PLw/lRhOYDWzcQCShgP7FfXmSXoQ+CDK2DZHnow4FyEzexl4WdI+ZvZJ1PFU07ZFiQiAmc2QtL2ZzYnxCt+WwLLwcYsoA6miiwgmNm8vaQHhxOZoQypb0WRrSQOLhpRCV0uaClwTTWSVakXQE1n0c9E0bHN1yJMR5+LhC0kXEQzZFA/PmNnZ0YVUqW8kPQD8Jzw+CZghqTEbVqvEySiC7/M7BMNKBxDfD0gkpQMXmtmA5InNUcdVBQqHvz4KD/oR7zISo9n45+LmSCPaDPkEVudiQNKzBOWoTyEY6hgKfGtml0QaWAXCGiMXAvuFTR8RzCPJJZjrsDqq2MoTFhDbIzz8zMwWRRlPZSRNMrO9o46jOiTtDvyLoOdJBEvVzzazqZEGVgFJHYG9wsNP4/5zUR95MuJcDEj6wsx2k/SlmfWS1BD4INU+iOIoHDr6rpzVP8T8Q/IBoAvwLLCmqN3MXogsqCqS1ALAzFZEHUtFJJVZ+t3M3q/rWDZnPkzjXDwUDWvkSNoZWATEuoS2pG0Ihj52pOTQUo/Igirb5cAw4M4yzhlwSN2GUy0ZBHumJMdoBBOEY0nSjaWOATCzWyIJqHJXJj3OIChl/znx/rmodzwZcS4exkhqBdwAvEIwie7Gip8SubHATQQbjR0MnEUM5waY2bDw4RFmlpt8TlKsl0+b2VlRx/AbrEl6nEGwyia25dXNLHkVG5K2BO6JKJzNlg/TOOd+k6KibJK+MrNdktuijq0s5RSW26gtTiRtAdxHUAsDgiWnl5jZz9FFVT3hhOY3zeygqGOpinCzx2/MbMeoY9mceM+IcxGSdHlF583srrqK5TfIC0t/z5L0B2ABQY9OrISTE7sAmZJ2I5hUCcFyzqzIAquascCTBIW4IKjZMRYYGFlE1ZcFbBF1EOUpVYk1DegNxHYeUX3lyYhz0WoW3m9HsMrjlfD4KOCzSCKquksIPmguBm4lGKo5I9KIynYYcCbBB2JycrcKuDaKgKqhnZmNTTp+VNKlkUVTBaUqsaYD7Qh+PuJqStLjAuCpomXJru74MI1zMSDpfeB3RXUkJDUD/mtmZc70d9Un6Xgzez7qOKpD0kSCnpCnwqaTgbPMrH90UVVMUrekwwLgl7juVQQg6RIzu7eyNle7PBlxLgYkfQ/0MrO88Lgx8KWZbRdtZPWLpN+xcWG5uK7yKPpgvw/Yh6C34WNghJnNjzSwCkh63MxOq6wtLsqZS/RFqSqyrpb5MI1z8fAY8JmkF8PjwcCj0YVT/4R7jmQRDCc9DJxA/IfCtjCzo5MbJO0LxDYZIUj2iklqAMRuUrOkkwmKDG4l6ZWkU83YUBre1RHvGXEuJsKiXPuHh++b2RdRxlPfJBWUK7pvCrxuZvtX+uSIpNIKIEkjCebgZAJri5qB9cCYuO2OHPY6bUVQKyd5W4BVBL2SsR1aqo+8Z8S5mAgrgabMLH5J2wIPAB3MbGdJvYCjzey2iEMrz7rwfq2kzgTFxDpFGE+5JO0D9APalVpx1ZxgUmjsmNkoYJSkUXFLPMoSbuw3l2AIzEUsdgWKnHMp4yFgJGH1WDP7Evh9pBFV7DVJLYE7CJK+n9gwMTRuGhEsk25AMGxQdFtJMLwUO5K2Dx8+K6lP6VukwZVB0ofh/SpJK5NuqyStjDq+zY0P0zjnfhNJk81sj+TJfpKmmVnvqGMri6TGpSYIZwC5RW1xJKlb+Bc8YU2XpmYWyw9KSWPMbFi4+21pZmZeXt2Vy4dpnHO/1RJJPQlrSkg6AciONqQKfQL0AQgTkDxJU4vaYmqUpAuAQmAy0FzSvWZ2R8RxbSRVy+5Lal1G8yozyy+j3dUSH6ZxLgYkHSdplqQVKdRVfBHwT2B7SQuAS4Hh0Ya0MUkdw23tMyXtljR0cBDxr8C6Y9gTMhh4nWDCZSyXyCb5uIptcTEVWAzMBGaFj3+SNDX8uXF1wHtGnIuH24GjzCy2G4qVZmZzgAGSmgBpRQXbYii5AuudbCgHnwoVWBtKakiQjNxvZvmSYjm2nsJl9ycAz5nZmwCSDgWOJyg29w9grwhj22z4nBHnYkDSR2a2b+VXxoekSwh+Ya8imMzaB7jGzMZHGlg5UrQC68XA1cB04HdAV+CJOC5HlnQGQdLXl5Il1lcBj5rZC1HEVZnkjR6T2oqWf8d2DlR948mIczEg6V6gI/ASUDyhMq6/wAEkTTezXSUdBlwAXA88HscaGJB6yVNZwh1l0+NcAyPVkj5J44GJwH/CppMINiI8HJgc15/n+saHaZyLh+YEhaIOTWozILbJCBu64QcBj5nZN+GHZVydbWb3hslTG4K5F48DKZOMWPDXY2wTEQAzez7Fyu6fAtxE8IcAwEdhWzowJKqgNjeejDgXA2Z2VtQx/Aafh39VbgWMDDf3S0QcU0VSLXlKSalWdt/MlgAjyjk9uy5j2Zz5MI1zMSBpC4IN0YrmjXwAXGJmP0cXVcXCuhe9gTlmliOpDdAlLH4WO5LGEkyw3ArYleAv33fNzFdM1KBUK7sfVhK+AuhO0h/oXhelbnnPiHPxMBZ4EjgxPD41bBsYWUSV2y+875UiHQznsCF5WhsmT7HukZKUBfwR6Gpm50naBtjOzF6LOLSKpEzZ/dCzwIMEvTiFEcey2fJkxLl4aGdmY5OOH5V0aWTRVM2VSY8zgD2Bz4FY/kVpZgmS9v4xs6UEH5RxNpbge1q0f8oCgg/POCcjpcvuG8EHfVwVmNkDUQexufNhGudiQNJEgg+eor1STgbOMrP+0UVVPZK2BO4xs+OjjqW+kDTFzPqWKrk/3cx2jTq2qigqu29mK6KOpTySbgZ+BV6k5Eq2ZVHFtDnynhHn4uFsgjkjdxP8JfkxMR9CKMPPwA5RB1GapAZxXgpbifWSMtlQcr8nSR+YcSLpuArOxXmZ+hnhfXJPnwE9Iohls+XJiHMxEG6GdnTUcVSHpPsIPyQJtpboTdIwSIx8Rrz3n6nITcAbwJaS/k0wwfnMSCMq31EVnIvtMnUz2yrqGJwP0zgXKUlXmdntpT7Yi5nZxRGEVSVhxc0iBcBPZvZRVPGUJ3mIIxWFE233JliaPClciupqSFhufzhwQNj0LvBP3yivbnky4lyEJB1lZq+W+mAvZmbj6jqm6pDUCNg2PPw+jr/AJf0M3FXeeTMr91zUJB0LvF005yKcGHqQmb1U8TNdVUl6GGgIFP1bOw0oNLNzo4tq8+PDNM5FyMxeDR+uNbNnk89JOrGMp8RGuOvtOOAngr/at5R0hpm9H2VcZUgHmrKh6FkqucnMXiw6COu5JFcLdZtuj1ITgt+WND2yaDZTnow4Fw8jCZZsVtYWJ3cCh5rZ91BcPOopIG5FxLJjXIq8MmlltPnv7ZpVKKmnmf0AIKkHXm+kzvkPtXMRknQEQXnyLpL+lnSqOTHfgwRoWJSIAJjZzHD8PW7K7RGRtG8c57kkmSLpLuDv4fFFBHVHYk/SVsBuwAwz+y7qeCpwJfCOpDkEPyvdSL2VbCnP54w4FyFJuxKsQrkFuDHp1CrgHTNbHklgVSDpXwR70TwRNg0l2FH27Oii2pik9kB/glLwb5jZ15KOBK4FMuM8uVVSE+AGYEDYNAG4zczWRBdV2SS9ZGaDw8fHAPcQTAbtB4wys0eji65iYT2U7cLD780slsun6zNPRpyLAUnNgTVmVhgepwONzWxttJGVL/wFfhEbysJ/APwjbr/IJT0KbEmwxHcvYCHQF7jGJ4LWnFKF2T4GhprZj5LaAhPjVqitorooQJzrotRLPkzjXDyMJ/jrd3V4nBm29YssokqEScddVLBSJSb2AHYxs4SkDGAR0DMsBx9rKbaJW/Jftg3M7EcIdsWVFMfdnFOyLkp95cmIc/GQYWZFiQhmtjrcJC22JO0L3Ewwxp78QRm3ypV54b40mFmupDmpkIiEUmkTt10lrSSYd9FYUiczyw6Xf6dHHNtGzMznhcSIJyPOxcMaSX3MbCqApN3ZsPtpXD0CXEYwoTLOH5TbS/oyfCygZ3gswMysV3ShVSplNnEzs/ISjizg/LqMxaUeT0aci4dLgWclLST4kOwInBRtSJVaYWavRx1EFcRuv5xqeFXShaTAJm6SuprZvNLtYW0U/6xxFfIJrM7FRLgsNnlGf+yqmQJIKtrnZQhB9/sLlPygjOP+NClJ0o9lNFsMh8IIl8Y+CNyZNBG7A0E9mu3NrG+U8bl482TEuZiQtDOwI5BR1GZmj0UXUdkkvVPBaYvb5EpJqyg5uVLhcdEwTfNIAqtnJLUCRhNMur4E2AW4HLgdeKBo3k5cSOoK/BrOIxLBBoR9gBnAQym803NK8mTEuRgIS3wfRJCM/A84AvjQzE6IMq76QNJLBMNeLwD/KWsoIa7CScyXA13NbJikbYDtzOy1iEMrl6RLgLsJllDvbWY/RxxSmSR9DexpZmsl/RXoSVBm/xCAuNXLqe/KKjXsnKt7JxAU5loUzvLfFWgRbUj1Q1iI6zBgMfCQpPckXSipdcShVcVYYD0blngvAG6LLpzySWop6Z8E1UsPB54DXpcUq56yJGlJdXwGAEPM7IkwCYnblgb1nicjzsXDurAbuyAsgPYrQaEuVwPMbIWZjSXocfonQcXbMyMNqmp6mtntQD5A+OEZ1w3/pgKzgL5mNt7MLiXYAfc2SU9FG1qZ5iclSj8R/nuT1CayiDZjPsPZuXiYEm4P/xDBUtnVwCfRhlR/SOoHnAzsD3wIHGtmH0QbVZWsl5RJOOdFUk+SJgvHzAGlh2TMbBrQT9J5EcVUkXOBxyTdDKwApkmaBrQE/hhlYJsjnzPiXMxI6g40N7MvK7k0MmHvTbuinU6T2nvFLW5JPwE5wH+Atym1AWGcV/9IGghcTzCXaDywL3Cmmb0bZVz1iaQdgG0J/jj/GZgct8m2mwNPRpyLgbCa6TQzWyPpVIJZ/fea2dyIQ9uIpCEEm6D9CjQk+HCcHJ6bamZ9Knp+XZP0LhtW0xStoikSu9U/pYXDBnsTxD3JzJZEHFK9UF5dlPDc/inSc1ZveDLiXAyEFUF3BXoBjxKU/x5iZgdGGVdZwq7sI8JS33sCjwEjzezF5M3S3G8naXsz+y6ppksJce7NSRVeFyVefM6Ic/FQYGYWbr1+v5k9IumcqIMqR7qZZQOY2WeSDgZek7QlJet5uN/ucmAYwQdjaUa4/NRtkt0J6qJMC5cjJ9dFOT3KwDZH3jPiXAxIeg94g2BZ5AEEQyDTzWyXSAMrQ7g9/GnJ80UkNSOo0bCfmTWOLDjnqilV6qLUd94z4lw8nAScApxjZovC6pB3RBxTeYZTqiyAma2SdDhBiXhXg8KVQN0puTNy7Crzpppw9dpfgb0I6qIMIqiLcomZvR1pcJsh7xlxzlVLqk78kzTRzPpX1hYnkh4nqAw6jQ07I5uZXRxdVPVDOGfkH8A9RaXfJfUO2+aa2clRxre58Z4R52JA0nEEf6W1J1g1Eed9U96VVO7EPyBWE/8kZRBsY9823D+laDVNc6BLZIFVTV9gR/O/GmtDqtVFqde8Aqtz8XA7cLSZtTCz5mbWLKaJCAQT/3oSTPw7JBxz/4ygSNuekUZWtvMJCsltH94X3V4G7o8wrqr4mmBfHVfDKpobYmYP1WUszodpnIsFSR+Z2b5Rx1EdqTbxT9IIM7sv6jiqI9whuTdBsldcedXMjo4sKOdqgQ/TOBcPUyQ9TbAiJflD54XoQipbCk/8WySpWTjZ9nqCwnK3xbxmx81RB+BcXfCeEediQNLYMpotjtuYp+rEP0lfmlkvSfsR7Hx7B3Cjme0VcWgVktQN2MbM3pKURVDnZVXUcTlXkzwZcc5Vi6QtyhuSkXReXMfbi6rDShoFfGVmT8a9Ymw4kXIY0NrMekraBngwziuAnPstPBlxLgbCFR/nADsBGUXtcewZSVWSXgMWAAMJhmjWAZ+Z2a6RBlaBsPT+nsCnRUmTpK/iWAzPuU3hq2mci4fHCVZNHAa8B2wBeFd8zRoCvAkcZmY5QGvgymhDqlSema0vOpDUAC+57+ohT0aci4etzewGYI2ZjQN+RzBB1NUQM1tLUGZ/v7CpAJgVXURV8p6ka4FMSQOBZ4FXI47JuRrnyYhz8ZAf3udI2hloQVAAzdUQSTcBVwMjw6aGwBPRRVQl1wCLga8I6qX8D7g+0oicqwW+tNe5eBgTVge9HngFaArcEG1I9c6xwG7AVAAzWxhu8Bdng4HH4jop2Lma4smIcxGTlAasNLPlwPtAj4hDqq/Wm5lJMgBJTaIOqAqOAu6W9D7wNPBG0XJq5+oTH6ZxLmJmlgCuijqOzcAzkv4JtAyXzL4FxLrHwczOArYmmCtyMvCDpIejjcq5mudLe52LAUmjgSUEf/2uKWo3s2WRBVWPSBLBCqXtgUMJNst708wmRBpYFUlqSFDt9iyCDd7aRhySczXKkxHnYkDSj2U0m5n5kE0NScX6HJKOAE4CDgLeBZ4BxvtQjatvfM6Ic/Gwg5nlJjeEhdBczZkqaQ8zmxx1INVwOkFv2flmllfZxc6lKu8ZcS4GJE01sz6VtbnfTtJ3BPMv5hIMhYmg96lXpIE557xnxLkoSeoIdCEoarUbwQckQHMgK7LA6qfDog6guiQdR7BDcnuCn42iBKp5pIE5V8O8Z8S5CEk6AzgT6AtMZkMyshIYZ2YvRBRavSWpPSX3/5kXYTgVkjQbOMrMvo06FudqkycjzsWApOPN7Pmo46jPJB0N3Al0JigL3w341sx2ijSwCkj6yMz2jToO52qbJyPOuc2CpOnAIcBbZrabpIOBU83snIhDK5ekewk2UHwJKJ7A6j1mrr7xOSPOuc1FvpktlZQmKc3M3pF0T9RBVaI5sJagNkoRAzwZcfWKJyPOuc1FjqSmwAfAvyX9SlKBuTgKK7A6V+95OXjnYircMt7VnGMIehkuBd4AfiDY+yW2JG0h6UVJv4a35yVtEXVcztU0nzPiXExJmmdmXaOOoz6R1A3YxszekpQFpJvZqqjjKo+kCcCTwONh06nAUDPzRNXVK56MOBchSa+Udwo4xMxSYWfZlBBujjcMaG1mPSVtAzxoZv0jDq1ckqaZWe/K2pxLdT5nxLlo7U/w1+7qUu0C9qz7cOq1iwi+p58CmNmssOZInC2VdCrwVHh8MrA0wnicqxWejDgXrUnAWjN7r/QJSd9HEE99lmdm64MNfEFSA4KVKXF2NnAfcDdBrB8T7NzrXL3iwzTOuc2CpNuBHILN50YAFwIzzOy6SANzzvlqGueiJKncCaqS9q/LWDYD1wCLga+A84H/AddHGlElJI2T1DLpuJWkf0UZk3O1wXtGnIuQpDnAg8CdZlYYtnUgKFu+vZn1jTI+Fy1JX5jZbpW1OZfqvGfEuWjtDvQEpkk6RNIlwGfAJ/gE1holaV9JEyTNlDRH0o9hMhhnaZJaFR1Iao3P9XP1kP9QOxchM1sOnB8mIW8BC4G9zeznaCOrlx4BLgM+BwojjqWq7gQ+kfRseHwi8OcI43GuVvgwjXMRCucD/BXYC7gKGAT0By4xs7ejjK2+kfSpme0VdRzVJWlHgg3+AN42sxlRxuNcbfBkxLkIhcME/wDuMbOCsK132DbXzE6OMr76RNJoIJ1gk7nkHXCnRhaUcw7wZMS5SEnaorwhGUnnmdlDdR1TfSXpnTKazcwOKaPdOVeHPBlxzrmYkdTYzPIqv9K5+sFX0zjnXPx8AiDp8coudK4+8NU0zjkXP40knQL0k3Rc6ZNm9kIEMTlXazwZcc65+LkAGAq0BI4qdc4IJuE6V2/4nBHn3GZBUhbwR6CrmZ0naRtgO7P/b+/uY/QqyzyOf38gIiJuiagYFiogqGix8iKERVdLVHaNmCXxBcyqRNHoGlDikmxMdFfNqkTDqhFEV4mmxGiMRhaDoCgqqay0tdIQbIzVGu2q8R2XFYpc+8c5j07rTKeaeeZ+5pzvJ5nMnHv44/cH6VzPfV3nvuu6xtEWlOTlVfXh1jmkabMYkTQKST5Bd+DZS6rqiX1xsqGq1jaOtqAkD6TbJXlav/QV4ANVtatdKmnpOcAqaSyOrarLgF0AVXU3kLaRFnUF3ZUBV/RfJwFXNk0kTYEzI5LG4t4kB9HNXJDkWOYcfjajTq2qJ815/lKSbzVLI02JOyOSxuLNwOeBI5NcA9xEdwT/LPt9XzQBkOQYVs69OtI+c2ZE0mgkeRhwOl175taq+lnjSHuV5CzgamA7XebVwAVVNd9pstKKZTEiaTSSHEH3B/0PLeqq+mq7RItLciDw2P5xmwIP67UAAAjBSURBVCezaogsRiSNQpJ3Ai8E7gDu75erqs5pl0oSWIxIGokk24AT3VmQZo8DrJLGYjtwQOsQ+yqdI1vnkJaDOyOSBi3J++he5z0CeBLdWzR/2B2pqosaRVtUkq1VtaZ1DmnaPGdE0tBt7L9vAq7d43ez/mlsc5JTq+q21kGkaXJnRNIoJLm4qt6z2NosSfJt4DHADuB/6V7vrao6sWkwaYlZjEgahSSbq+qkPda+WVVPbpVpMUlWz7deVTuWO4s0TbZpJA1akvOA84Gjk8xt0xwC/KJNqn1TVTuSnAkcV1VXJ3k48JDWuaSlZjEiaeg2AP8DHAa8e876XcDtTRLtoyRvBk6hO/Tsarq3gdYDf9Myl7TUbNNI0oxKsgV4MrB50k5KcrszIxoazxmRpNl1b3WfGCc3DR/cOI80FRYjkjS7PpnkKmBVkguBLwIfapxJWnK2aSSNRpIHAsf3j9uqalfLPPsiyTOBZ/WPN1bVF1rmkabBAVZJo5Dk6cBHge/TnddxZJKXzvqtvcBW4CC6Vs3WxlmkqXBnRNIoJNkEnF9V2/rn44GPV9XJbZMtLMkrgDcBX6IroP4WeEtVfaRpMGmJWYxIGoX53kKZ9TdT+puGz6iqn/fPDwM2VNVj2yaTlpZtGkljsTHJf9Kd0wHwYv54b82s+jndeSgTd/Vr0qC4MyJpFJIcCPwTcGa/9DXg/VV1b7tU80tySf/jWmAN8Fm6mZHnAbdX1csaRZOmwmJE0iispIvy+pNXF1RV/7ZcWaTlYDEiaRRW4kV50lg4MyJp0FbyRXlJTgHeCKxmzr/Xszx0K/0lLEYkDd2KvSgPuAb4Z7rzRe5vnEWaGts0kjSjktxSVWcu/l9KK5vFiCTNqCRnAecBNwH3TNar6tPNQklTYJtGkmbXBcDjgAP4Y5umAIsRDYo7I5JGI8lBwFGTI+FnXZJtnraqMdivdQBJWg5JngtsAT7fP6/d4+2aWbQhyQmtQ0jT5s6IpFHoL8pbB9w8OVskydaqWtM22cKS3AkcC3yPbmYkQPlqr4bGmRFJY7Grqn6dZO7arH8aO7t1AGk5WIxIGos7kpwP7J/kOOAiujNIZtmsF0vSkrBNI2kUkjyY7jTTZ/VLNwBvq6rftUu1d0m20hUkAR4EHA1sq6onNA0mLTGLEUlaIZKcBLymql7ROou0lHybRtIoJPlCklVzng9NckPLTH+uqtoMnNY6h7TUnBmRNBaHVdWvJg9V9cskj2gZaDFJLpnzuB9wErCzURxpatwZkTQW9yc5avKQZDWzPyB6yJyvA4HPAc9rmkiaAmdGJI1CkrOBDwJfoRsIfSrwyqpaUa0aaYgsRiSNRpLDgNP7x1ur6mct8ywmyfHAG4BHM6etXlXrWmWSpsFiRNJoJDkCWM3uf9i/2i7R3iX5FvABYBPw+8l6VW1qFkqaAgdYJY1CkncCLwTuYPcbcGe2GAHuq6orW4eQps2dEUmjkGQbcGJV3dM6y75K8q/AT4HP0N1NA0BV/aJVJmkaLEYkjUKS64HnV9VvW2fZV0m+N89yVdUxyx5GmiLbNJLG4m5gS5Kb2H2X4aJ2kfauqo5unUFaDhYjksbi2v5L0oyxTSNJkppyZ0TSKCQ5Dng7cALdDbgAOH8htedx8JLG4mrgSuA+4BnAx4D1TRP9GZL8e+sM0rTYppE0Ckk2VdXJSbZW1Zq5a62z7SnJe/dcAv6RroCa6aFb6S9hm0bSWNyTZD/gO0leC/wIeEjjTAv5B7o7dG6kK0QAXkR3Eqs0OO6MSBqFJKcCdwKrgLcCDwUuq6r/bhpsHkkOocv4COANVbUzyXbnWzRU7oxIGotHV9VtwG+BCwCSPB+YuWKkqu4CXpfkZOCaJJ/DGT8NmP9zSxqLf9nHtZnRX4i3Dvg/4JbGcaSpsU0jadCS/B3w98ALgE/M+dVDgROq6ilNgu1FkqOq6gcL/O6pVfW15c4kTZM7I5KGbiewEfgd3QDo5Ota4NkNc+3NzUkuTbL/ZCHJI5OsBy5vmEuaCndGJI1CkgOqalf/86HAkVV1e+NY8+rzvQM4A7gYWANcAlwGXFlV9zeMJy05ixFJo5DkZuAcusH9TcBPgQ1V9fqWufYmycV0OyE7gdOr6oeNI0lTYZtG0lj8VVX9BjgX+FhVnQac1TjTvJKsSnIV3Vs/ZwOfAq5Psq5tMmk6LEYkjcUDkjyKbpD1utZhFrEZ+A5wSlXdWFWvozuB9W1JPt42mrT0PGdE0li8BbgBuKWqbktyDN0f/Fn0tD1bMlW1BTgjyYWNMklT48yIJElqyp0RSYOW5NKquizJ+4A/+fTlpXNSexYjkobuzv77xqYpJC3INo0kSWrKnRFJg5bkv5inPTNRVecsYxxJ87AYkTR07+q/nwscDqzvn88DftIkkaTd2KaRNApJNlbVKYutSVp+HnomaSwO7s8WASDJ0cDBDfNI6tmmkTQWr6e7DXc7EGA18Mq2kSSBbRpJI5LkQOBx/eO3q+qelnkkdSxGJElSU86MSJKkpixGJElSUxYjkgYvyeFJDu9/fniSc5M8oXUuSR2LEUmDluRVwNeBW5O8GrgOeA7w6SQvbxpOEuAAq6SBS7IVOA04CNgBPKaqfpzkUODLVbW2aUBJnjMiafB2VdXdwN1JvltVPwaoql8m8dOYNANs00gaukpyQP/zcyaLSR6E/wZKM8E2jaRBS3IUsLOq7ttj/Qjg8VX1xTbJJE34qUDS4O1ZiPRrPwI8gVWaARYjkobu5iSXJtl/spDkkUnWA5c3zCWpZzEiaehOBo4FtiRZl+Ri4Bt0r/s+pWkySYAzI5JGoi9CLgd2AqdX1Q8bR5LUc2dE0qAlWZXkKuAC4GzgU8D1Sda1TSZpwp0RSYOWZDtwBfAfk0HWJGv7tR1VdV7LfJIsRiQNXJK/Xqglk+TCqvrQcmeStDuLEUmS1JQzI5IkqSmLEUmS1JTFiCRJaspiRJIkNWUxIkmSmvp/+JnuMj1s2mUAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "tags": [], + "needs_background": "light" + } + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "GgUlNT1tFPYD", + "colab_type": "code", + "outputId": "59c8ba46-8d74-48e2-c32b-fc5ae30e425d", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 235 + } + }, + "source": [ + "X = dataset[['X1 transaction date', 'X2 house age', 'X3 distance to the nearest MRT station', 'X4 number of convenience stores', 'X5 latitude', 'X6 longitude']]\n", + "X2 = sm.add_constant(X) # augmented regressor matrix\n", + "vif = pd.DataFrame()\n", + "vif['VIF Factor'] = [variance_inflation_factor(X2.values, i) for i in range(X2.shape[1])]\n", + "vif['Predictor'] = X2.columns\n", + "vif = vif.drop(0)\n", + "vif" + ], + "execution_count": 70, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
VIF FactorPredictor
11.014674X1 transaction date
21.014287X2 house age
34.323019X3 distance to the nearest MRT station
41.617038X4 number of convenience stores
51.610234X5 latitude
62.926302X6 longitude
\n", + "
" + ], + "text/plain": [ + " VIF Factor Predictor\n", + "1 1.014674 X1 transaction date\n", + "2 1.014287 X2 house age\n", + "3 4.323019 X3 distance to the nearest MRT station\n", + "4 1.617038 X4 number of convenience stores\n", + "5 1.610234 X5 latitude\n", + "6 2.926302 X6 longitude" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 70 + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "6eOByNtRqzG1", + "colab_type": "code", + "outputId": "9aaa25e1-3691-4a1c-e094-72ea66044369", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 444 + } + }, + "source": [ + "df = pd.DataFrame(dataset,columns=['X1 transaction date', 'X2 house age', 'X4 number of convenience stores', 'X5 latitude', 'X6 longitude']) # regressor matrix\n", + "corrMatrix = df.corr()\n", + "sn.heatmap(corrMatrix, annot=True) # visualize the correlation matrix using a heatmap" + ], + "execution_count": 71, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 71 + }, + { + "output_type": "display_data", + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfsAAAGaCAYAAAAII+otAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdd3wUVdfA8d9JCEXpIqTSmwpKLyK99ygoICoqiuJjAV8Q7ICPig187GJDEUHpAYGINAFBOggIiNRUVAhFWsp5/9ghbEhbSNkQz9fPfszM3Ll77m7ImVt2R1QVY4wxxuRfPt4OwBhjjDE5y5K9McYYk89ZsjfGGGPyOUv2xhhjTD5nyd4YY4zJ5yzZG2OMMfmcJXtjjDEmDxGRz0XksIhsS+e4iMg7IrJHRLaKSL3M6rRkb4wxxuQtE4FOGRzvDFRzHoOADzOr0JK9McYYk4eo6k/AkQyK9AS+Upc1QEkRCcioTkv2xhhjzJUlCDjkth3h7EtXgRwNxxgvif9rb777HujyVbt5O4QckaRJ3g4h2xUuUNDbIeSIqBN/ezuEHBF/LlKyXIeHf3MKXlvlIVxD7+dNUNUJWX3+zFiyN8YYY7IqKdGjYk5iz2pyjwRC3LaDnX3psmF8Y4wxJqs0ybNH9ggD7nFW5TcBjqlqdEYnWM/eGGOMyaqk7JuOEpEpQCugjIhEAC8CfgCq+hEwH+gC7AFOAfdlVqcle2OMMSaLNDEh++pS7ZfJcQX+cyl1WrI3xhhjsiqPLzS1ZG+MMcZklYcL9LzFkr0xxhiTVdazN8YYY/K5bFyglxMs2RtjjDFZlJ0L9HKCJXtjjDEmq2wY3xhjjMnnbIGeMcYYk89Zz94YY4zJ52yBnjHGGJPPWc/eGGOMyd80Md7bIWTIkr0xxhiTVXm8Z2+3uDXmMj33yjhadO1L6F0PezuUTLVuewsr1n3PzxsX8uiQB1IdL1jQj48+f4ufNy7k+x+nElw+EIBSpUowfe4X7IlYz8uvP5vinNBeXViyajaLV83im+kfU7p0yVxpx6r1C1izKZzHhj6YZjsmfDGONZvCWbD4W0LKByUfe/zJQazZFM6q9Qto1faW5P0PPnw3y1eHsXzNXAYNvidFfQMH3cXKdfNZvmYuz48ZlnMNc9OyTTOW/BLG8nXzGPzE/amOFyzox3ufvs7ydfOY/cNkgkNc79VN9Woxf9l3zF/2HQuWT6Nj1zbJ56zctIDwFTOYv+w75i6ekivtuNj4cWP4bcdKNm5YRN06tdIsU69ubTZt/JHfdqxk/LgxqY4PGfIQ8eciueaaUgDUqFGFFT+FcfLEXoYOfShH489UUpJnDy+xZH+JRCRERPaJSGlnu5SzXdHZXigicSIyL4M67hWRwNyJ2DMiUkdEurht9xCRkTnwPKNEJMO/miISKiLXZ/dzZ7fQLu35aNx/vR1Gpnx8fHjlzefo3/shWjbuTmjvLlSvUSVFmX539+JY3HFurteJCR98yXOj/g+AM2fP8frL7zLm+TdSlPf19eWlsU/Tu/u9tG12Kzu27+a+Qf1zvB1j33qBO3s/SPNG3bi1V9dU7bjznt7ExR2nSd2OfPzBlzw/2tWO6jWqEHpbF1o07ka/Xg/w2lsv4OPjQ83rqnHXgNvp1OYO2jQLpX2nVlSsXB6AZs0b06lrG9o060nLJt358J3Pc7R959v40uvPMOCOwbS7OZQet3WmWo3KKcr0ues2jsUdp2XDbnz24SRGvjgEgF2/7aF72350aXUHA+4YzCtvvYCvr2/yeX17DqRLqzvo3jbDG6rliE6d2lC1aiWuu/4WBg8ewXvvvZpmuffee5WHH36K666/hapVK9GxY+vkY8HBgbRv14IDByKS9x05EsfQoc8zbvzHOd6GTOXu/ewvmSX7S6Sqh4APgbHOrrHABFXd72y/AdydSTX3AmkmexHxTWt/LqiD6/7IAKhqmKqOzaB8TgoF8nyyb1CnNiWKF/N2GJmqW782+/ce5OCBCOLj45kzYwEdu7RJUaZTlzZ8N2U2APPm/EDzlk0AOH3qNGvXbOTM2bMpyosIIsJVV18FQLFiRYmNPpyj7ahX/0b27T3Igf2udsyeOZ9OXdte1I62fPeNqx1zZ4dzS8umrv1d2zJ75nzOnYvn4IFI9u09SL36N1KtRmU2btjK6dNnSExM5OeV6+javT0AAwb25d3xn3DunGsu9q+/juRo+wDq1KvF/n0HOXQgkvj4BObOWkj7zq1TlGnfuRUzpoYBMD9sEc1aNAbgjNMGgEKFCuG6C2re0KN7R76ePB2AX9ZupETJEvj7l01Rxt+/LMWKF+OXtRsB+HrydHr26JR8/M03R/H0My+naNeff/7N+g1biI/PA/PlSYmePbzEkv3lGQ80EZEhwC3Am+cPqOpi4ER6J4pIb6ABMFlENotIERHZLyKvichG4HYReVBE1onIFhGZISJXOedOFJF3RORnEdnr1IWIBIjIT05920SkubP/QxFZLyLbRWS0WwwNnTq2iMhaESkBjAH6OHX0cUYf3nPKVxSRJSKyVUQWi0j5jOJJo83PishuEVkJ1HDbn6qdInIz0AN4w4mlivNYKCIbRGSFiNS89Lfs38s/oByRkTHJ29FRMfgHlE1VJsopk5iYyPHjJzIclk9ISGDEk2NYsmo2m3cup3rNKnwzaUbONOB8jIHliIqMTt6OiozBP6BcijIBAWWJdMokJiZywmmHf0A5IiMunBsdFYN/YDl27vidxk0bUKpUSYoUKUy7Di0JCgoAoEqVijRu2oAFi79l1veTqFMv7aHnbG1jQDmiI2Pd4oxN+72KinVr40lKOe9Vnfq1WbRqJuErZvDssJeSkz8KX0//mHmLp9Lvnl453o6LBQb6E3EoKnk7MiKaoED/FGWCAv1TvEcREdEEOmW6d+9AVGQ0W7fuyJ2AL0digmcPL7FkfxlUNR4YjivpD3G2PT13OrAe6K+qdVT1tHPob1Wtp6pTgZmq2lBVbwJ+Awa6VRGA6wKjGxdGF+4EwlW1DnATsNnZ/6yqNgBuBFqKyI0iUhD4FnjCqb8d8A/wAvCtE9O3F4X9LvClqt4ITAbeySSeZCJSH+jLhZGDhm6HU7VTVX8GwoDhTix/ABOAx1S1PjAM+CCt11ZEBjkXN+s//co785L/FgUKFGDAwL60b9GLOjVbsmPbLh5/MvUcel73++69vPf2J3w7+zOmzPiEbb/+lpwgCxTwpVSpEnRu24cxz7/OJxPf9nK0mdu84VfaN7uNHu378ciQgRQqVBCAXl0H0LVNHwb0eYR7BvalUdP6Xo7Uc0WKFGbkiMcYNfrNzAt7kw3j51udgWgguy733RNsLacH+yvQH7jB7dhsVU1S1R3A+W7NOuA+ERkF1FbV8yMLdzijBZucOq7H1bOOVtV1AKp6XFUzu9xsCnzj/DwJV3LPKB53zYFZqnpKVY/jSuSetBMAESkK3AxME5HNwMe4LjBSUdUJqtpAVRs8cE/uz0vmVTHRsQQFXehFBQT6E3PRkHtMdCyBThlfX1+KFy/GkSNx6dZ5Q23X4MqB/YcAmDt7IQ0a1c3u0FPGGBVLYNCFtz4wyJ+Y6NgUZaKjDyf3zH19fSnmtCMmOpag4AvnBgT6E+P0jr+ZNIMOLXsR2uVujsUd548/9gMQFRXL93MXAbBp468kJSUlLwzLKTHRsQQEXfhnFBBYLu33KtBVxtXGohy96L3as3sfp/45TfXrqgIkT7H8/dcRwr9fkiujFIMfHsD6dT+wft0PxMTEJi8kBAgKDiAyKiZF+ciomBTvUXBwAFFRMVSpUpGKFcuzYf0ift+9huDgANb+Ek65ctfmeBsuiS3Qy39EpA7QHmgCDBWRNJPPJfrH7eeJwKOqWhsYDRR2O+Y+eSoAqvoT0AKIBCaKyD0iUglXL7it0yP//qJ6skuqeC7BRNJv53k+QJzTyz//uO7yQv132rxxG5WqVCCkQhB+fn707NWZ8AVLU5QJX7CUO/qFAtCtZwdW/vRLhnXGRMdSvUaV5OTXovXN/L57b840wLFp469UrlKB8k47Qm/rQvj8JSnKhM9fwh13utrRPbQjK39ak7w/9LYuFCzoR/kKQVSuUoGNG7YCUKZMacCVgLp0b8/Maa61tQu+/5FmzRsBULlKRfz8/Pj776M52sYtm7ZTqXIFQsoH4edXgO63dmLRgmUpyvy4cBm9+vYAoEuP9vy8Yi0AIeWDkhfkBQUHUKVaRSIORlHkqiJcXdS1tqLIVUVo0bopu37bk6PtAPjwoy9p0LADDRp2YE5YOHf1d83yNW5Uj+PHjhMTc9FFTMxhThw/QeNG9QC4q39vwuaGs23bToKCb6Ja9SZUq96EiIhoGjXuSGzsnznehkuSx5O9fc7+EomI4FqgN0RVD4rIG7jm7C9lKfIJIKOVXcWAaBHxc+qNzCSmCkCEqn4iIoWAesAWXBcQx0SkHK6RiGXALiBARBqq6joRKQacziSmn3ENxU9y4lnhUStdfsJ1AfIqrt+37rh65xm1MzkWVT3ufNrhdlWd5rz+N6rqlkuIIUcMf3Es6zZtJS7uOG1D7+KRgXfTq3tHb4eVSmJiIs8Mf5kpMz7B19eHqV/PYvfOPQx/5lG2bNrODwuWMmXSDN79+DV+3riQuKNxPHz/hQ9MrN26iKLFilLQz49OXdvS77YH2b3rD8a99gGz5n9FfEICEYeiGDL4mRxvx9PDXmLqzM/w9fVhytcz2LVzD0898xhbNm0jfMFSvpk0nfcmvM6aTeHEHT3GQ/c/CcCunXsIm72AFWu/JyEhkZH/N4Yk5w/vZ5PeoVTpkiTEJ/D0sDEcP+YaGJsyaSZvv/8yy1eHcS4+nscHZ/uHU9Js4wsjXuGraR/i6+vLd9/M5vddf/DkyEfYunkHPy5cxrdfz2L8h6+wfN084uKO8egDTwHQoEldHnnifuLjE9Ak5bnhL3P0SBwhFYKY8JVrCqJAAV/mzFjA8iWrcrwt7hYsWEznTm3Y+dsqTp8+zQMPPJl8bP26H2jQsAMAjz32DJ9+Np4ihQsTHr6UhQuXpFclAOXKXcua1QsoXrwoSUlJPP7Yg9x4UytOnDiZo+1Ji2revhGO5KUVm1cCERmEq7fcx9n2xTWMPlRVl4vICqAmUBT4G9c8dPhFdfQCXsGVZJvimq9uoKp/OccHA08BfwK/AMVU9V4RmQjMc+b9EZGTqlpURAbgWkMQD5wE7lHVfU75m4FDwDEgTFUnikhDXPPwRZwY2gEFgXDAD3jVOdZAVR91Lia+AMo4Md3nXOikGU8ar9mzwADgMHAQ2Kiqb2bQzmbAJ7hGDXoDSbgusAKc+KaqauoP4bqJ/2tvvvvFLl+1m7dDyBFJefzLSC5H4QIFvR1Cjog68be3Q8gR8eciL3VUMpXTyz736G9OkVb3Z/m5Locle5MvWbK/cliyv3JYsk/f6cUTPEv2bQd5JdnbML4xxhiTVXn8otWSvTHGGJNVdotbY4wxJp+znr0xxhiTz1nP3hhjjMnnLNkbY4wx+ZwXv/feE5bsjTHGmKyyOXtjjDEmn7NhfGOMMSafs569McYYk89Zz94YY4zJ5xLz9o1w7Ba3xhhjTFZl4y1uRaSTiOwSkT0ikup2iyJSXkSWisgmEdkqIl0yq9OSvTHGGJNV2ZTsnTupvo/rtuTXA/1E5PqLij0HfKeqdXHdfvyDzOq1ZG+MMcZklSZ59shcI2CPqu5V1XPAVKDnxc8GFHd+LgFEZVapzdkbY4wxWeX5EP0gYJDbrgmqOsFtOwg45LYdATS+qJpRwA8i8hhwNdAus+e1ZG+MMcZklXp0O3ucxD4h04IZ6wdMVNW3RKQpMElEaqmmP3Rgyd7kS+WrdvN2CNnu4J553g4hR9Ss2dvbIWQ7X8mfM6SepbN/qYRs+7rcSCDEbTvY2eduINAJQFVXi0hhoAxwOL1K8+dvpDHGGJObsm/Ofh1QTUQqiUhBXAvwwi4qcxBoCyAi1wGFgT8zqtR69sYYY0wWaVL2jHuoaoKIPAqEA77A56q6XUTGAOtVNQz4P+ATERmKa8DlXtWM5xEs2RtjjDFZlY3foKeq84H5F+17we3nHUCzS6nTkr0xxhiTVfbd+MYYY0w+l03D+DnFkr0xxhiTVdm3Gj9HWLI3xhhjssrDz9l7iyV7Y4wxJqvsFrfGGGNMPmdz9sYYY0w+Z6vxjTHGmPxNExK9HUKGLNkbY4wxWWXD+MYYY0w+Z8P4xhhjTD5nPXtjjDEmn7OP3hljjDH5XB7v2dv97I1x07rtLaxY9z0/b1zIo0MeSHW8YEE/Pvr8LX7euJDvf5xKcPlAAEqVKsH0uV+wJ2I9L7/+bIpzQnt1Ycmq2SxeNYtvpn9M6dIlc6Utl+O5V8bRomtfQu962NuhXJIWbW5m0ZqZLFk7h4cevzfV8YIF/Xjn07EsWTuHGeFfEhQSAEBQSADbD/3M3KVTmLt0Ci+9+UwuR55a8zZNWbh6BovWzmLQ4wNSHfcr6Mfbn7zCorWzmLZwYoq2bD24kjlLJzNn6WRGv/F0qnM/nDSOeT99m+NtSMv4cWPYuWMlGzcsom6dWmmWqVe3Nps2/sjOHSsZP25MquNDhzxEwrlIrrmmFAAlS5Zg+rRP2bhhEatXzeOGG2rkaBsylJjo2cNLLNlfQUQkRET2iUhpZ7uUs11RROqIyGoR2S4iW0WkTzp1LBORBrkb+ZXBx8eHV958jv69H6Jl4+6E9u5C9RpVUpTpd3cvjsUd5+Z6nZjwwZc8N+r/ADhz9hyvv/wuY55/I0V5X19fXhr7NL2730vbZreyY/tu7hvUP9fadKlCu7Tno3H/9XYYl8THx4dRr43g/j6P0bFZL7rf1omq1SulKHN7/1COxR2nTaOefPHRZEa8+ETysYP7I+jeuh/dW/fj+WGv5Hb4Kfj4+PDi2BE82PdxujS7nW63dqRKqrb05FjcCdo3upWJH33D8BceSz52cH8kPVv3p2fr/rw4/NUU53Xo2ppT/5zKlXZcrHOnNlSrWoma19/C4MEjeP+9V9Ms9/57r/Lww09R8/pbqFa1Ep06tk4+FhwcSPt2LThwICJ539MjHmPLlu3Uq9+ee+9/gvFvpb5AyC2alOTRw1ss2V9BVPUQ8CEw1tk1FpigqvuBU8A9qnoD0Al4W0TybhcyD6pbvzb79x7k4IEI4uPjmTNjAR27tElRplOXNnw3ZTYA8+b8QPOWTQA4feo0a9ds5MzZsynKiwgiwlVXXwVAsWJFiY0+nAutuTwN6tSmRPFi3g7jktxUrxYH9kVw6EAk8fEJzJsVTrvOrVKUade5FTOnzgNgQdhimjZv6IVIM3djvRs4sP9Qclu+n/0D7Tq3TFGmbeeWzPrW1ZaFcxfTtHmjTOu96uoi3De4Px+M+yxH4s5M9+4dmTR5OgC/rN1IiZIl8Pcvm6KMv39ZihUvxi9rNwIwafJ0evTolHz8rTdHMfKZl1G376C/7rrqLF26CoBdu/6gQoVgypYtk9PNSVuSevbwEkv2V57xQBMRGQLcArwJoKq7VfV35+co4DBwbTp13C4ia0Vkt4g0BxCRwiLyhYj8KiKbRKS1s/9eEXnv/IkiMk9EWomIr4hMFJFtzjlDneNVRGShiGwQkRUiUvPiJxeRRs4oxCYR+VlEajj7rxKR70Rkh4jMEpFfzo9CiEgH55yNIjJNRIpmz8t5gX9AOSIjY5K3o6Ni8A8om6pMlFMmMTGR48dPZDgsn5CQwIgnx7Bk1Ww271xO9ZpV+GbSjOwO/V+tXMC1REddeN9iog5TLtX7di3Rbu/bieMnKeW8b8Hlgwhb8g3fhH1CgyZ1cy/wNJQLKEtMZGzydlptKedflminzIW2lAAguHwgs5dM5us5H9OgSZ3kc54YOZjPP/iaM6fP5EIrUgsK9CfiUFTydmRENEGB/qnKREZEp1mme/cOREZGs3XrjhTnbP11B7eGdgGgYYM6VKgQTHBQQE41I2OW7E12UtV4YDiupD/E2U5BRBoBBYE/0qmmgKo2AoYALzr7/uOqXmsD/YAvRaRwBqHUAYJUtZZzzhfO/gnAY6paHxgGfJDGuTuB5qpaF3gBOD92+ghwVFWvB54H6jvtKQM8B7RT1XrAeuDJDGLLMwoUKMCAgX1p36IXdWq2ZMe2XTz+5IPeDss4/oz9i+Z1utCjzZ288vw43v74ZYoWvdrbYV2Ww7F/0apuN0Lb9OfV58fz1kf/5eqiV3NdreqUrxjMovnLvB3iZSlSpDBPj3iMUaPfTHXstdffo0TJ4qxf9wP/+c/9bNq8jURvDZVrkmcPL7HV+FemzkA0UAtY5H5ARAKAScAA1XR/s2Y6/98AVHR+vgV4F0BVd4rIAaB6BjHsBSqLyLvA98APTm/7ZmCaiJwvVyiNc0vgupioBijg5xbD/5wYtonIVmd/E+B6YJVTb0Fg9cWVisggYBBA8SL+XFWwVAbhpxYTHUtQ0IXeRkCgPzEXDbnHRMcSGORPdFQsvr6+FC9ejCNH4tKt84baroGNA/sPATB39kIeHWLJPjvFRv9JgFsv0T+wbKqpkpjoPwkIcr2fvr6+FCtelKPO+3bu3DEAtm35jQP7I6hUtTy/bv4t9xrgJjb6MP5B5ZK302pLbMxhAoLKEZuiLa42xDlt2b51Jwf3R1KpSnlq172eWnWuY8mGMAoU8KV0mdJMmv0xd4c+lKNtGfzwAAYOdK1PWb9+M8EhgcnHgoIDiHQbjQGIjIohKDggVZkqVSpSsWJ5Nq53/akLDg5g3S/hNG3WldjYP3ngwQvX/Xt2r2Hv3gM52az02Wp8k51EpA7QHlcCHOok9/PHiuNKvM+q6poMqjk/sZxI5hd8CaT8PSkMoKpHgZuAZcDDwKdOuThVreP2uC6NOl8ClqpqLaD7+TozIMAitzqvV9WBFxdS1Qmq2kBVG1xqogfYvHEblapUIKRCEH5+fvTs1ZnwBUtTlAlfsJQ7+oUC0K1nB1b+9EuGdcZEx1K9RpXk1cMtWt/M77v3XnJsJn1bN22nYuUQgssH4udXgG63dmTxwuUpyixeuJzb+nYDoHOPtqxesQ6A0teUxMfH9esdUiGIipXLc3B/ZO42wM2vm3ZQsdKFtnQN7cDihT+lKLNk4U/c2sfVlk7d27J6pastpVK1JYRDByKZMnEGzWt3pk39HvTr9gD7/ziY44ke4MOPvqRBww40aNiBsLBw7u7fG4DGjepx/NhxYmIuuiCLOcyJ4ydo3KgeAHf3783cueFs27aTwOCbqFq9CVWrNyEiIpqGjTsSG/snJUoUx8/P1VcYeP+drFj5CydOnMzxtqVFE5I8eniL9eyvIOLq1n6Ia/j+oIi8gWvOvr+IFARmAV+p6vTLqH4F0B9YIiLVgfLALqA48IiI+ABBQCMnljLAOVWdISK7gK9V9bjz6YDbVXWaE++NqrrloucqAZz/i3qv2/5VwB3AUhG5Hqjt7F8DvC8iVVV1j4hcjWsKYfdltDNdiYmJPDP8ZabM+ARfXx+mfj2L3Tv3MPyZR9myaTs/LFjKlEkzePfj1/h540Lijsbx8P3Dks9fu3URRYsVpaCfH526tqXfbQ+ye9cfjHvtA2bN/4r4hAQiDkUxZLD3P96VnuEvjmXdpq3ExR2nbehdPDLwbnp17+jtsDKUmJjI6JGvMXHa+/j4+DD9mzB+37WXISMf5tfNO1i88Ce+mzybtz54iSVr5xAXd4wnHnR9LK1h03oMGTmYhPgEkjSJ54e9wrG4415ty5in3+Cz797F18eX6VPC2LNrL4+PeIhtm39jSfhPTJs8hzc+GMOitbM4dvQ4Qwc9k9yWJ0Y8REJCAklJygvDXvVqW9zNX7CYTp3asOu3VZw6fZoHHrjQG1+/7gcaNOwAwKOPPcNnn42nSOHCLAxfyoKFSzKs97qa1fj887dRVXbs2MWDg4ZlWD5H5fEv1RH3lY0mb3OGqduqah9n2xdYBwwFQnDNm293O+VeVd18UR3LgGGqut5J2OtVtaIzP/8h0ABXb/5JVV3qJOyvcc2f/waUAkYBR53nO9/rf1pVF4hIJaeeAFzD81NVNcXnYUSkKfAl8A+ukYi7nBiudvZfj2tevzJwu6r+LiJtgNe4MC3wnKqGpfdaBZS8Pt/9Yh/cM8/bIeSImjV7ezuEbOcr+XPQdO+x6MwLXYESzkVK5qUyduKRzh79zSn2wYIsP9flsGRv8gzn4sVPVc+ISBXgR6CGqp671Los2V85LNlfOSzZp+/Ew508S/YfLfRKsrdhfJOXXIVrCN8P1zz9I5eT6I0xJrfl9Y6zJXuTZ6jqCVzTCMYYc2Xx4uI7T1iyN8YYY7JI8/hH7yzZG2OMMVllyd4YY4zJ5/L2KL4le2OMMSarbBjfGGOMye8s2RtjjDH5myZYsjfGGGPytzw+Z58/v+bJGGOMyUWapB49PCEinURkl4jsEZGR6ZS5Q0R2iMh2EfkmszqtZ2+MMcZkVTb17J2vDX8f191NI4B1IhKmqjvcylQDngaaqepRESmbWb3WszfGGGOySJM8e3igEbBHVfc6Xxc+Feh5UZkHgfedW42jqofJhCV7Y4wxJos0wbOHB4KAQ27bEc4+d9WB6iKySkTWiEinzCq1YXxjjDEmqzwcxnduVT7IbdcEVZ1wic9WAKgGtAKCgZ9EpLaqxmV0gjHGGGOywMMhepzEnlFyjwRC3LaDnX3uIoBfVDUe2Cciu3El/3XpVWrD+MYYY0wWZeOc/TqgmohUEpGCQF8g7KIys3H16hGRMriG9fdmVKn17E2+lOTpZfYVpGbN3t4OIUfs3Dnd2yFku1U3jPB2CDnCp3Qlb4eQZ2XXnxxVTRCRR4FwwBf4XFW3i8gYYL2qhjnHOojIDiARGK6qf2dUryV7Y4wxJqtUsq8q1fnA/Iv2veD2swJPOg+PWLI3xhhjsigpIfuSfU6wZG+MMcZkUV6fObRkb4wxxmSRZuMwfk6wZG+MMcZkkfXsjTHGmHxOk6xnb4wxxuRrmrdvZ2/J3hhjjMmqpIS8/R11luyNMcaYLLKevTHGGJPP2Zy9McYYk8/ZR++MMcaYfM4+emeMMcbkc4lJtqKb71gAACAASURBVEDPGGOMyddszt4YY4zJ52w1vjHGGJPPWc/eGGOMyeeS8vhq/Ly9osCYXNC67S2sWr+ANZvCeWzog6mOFyzox4QvxrFmUzgLFn9LSPmg5GOPPzmINZvCWbV+Aa3a3pK8/8GH72b56jCWr5nLoMH3pKhv4KC7WLluPsvXzOX5McNyrmHpaNHmZhatmcmStXN46PF7Ux0vWNCPdz4dy5K1c5gR/iVBIQEABIUEsP3Qz8xdOoW5S6fw0pvP5HLkl++5V8bRomtfQu962NuhXJLSrevQaNX/aLzmXco/FppuuTJdG9MqdhrFbqqcYn+hoDI03zuJkMHdczrUS1KqdR0arPwfDVe/S8ijGberRcw0ijrtKhRyLc32Tabej29Q78c3qPpa6n+v3qIqHj28JcOevYiEAD8B9VX1iIiUAjYCrVV1v1OmOLADmK2qj+ZwvO6xjQJOquqbOfw8zYGPgHigqaqezsnnywoR+RQYp6o78kAsrYBzqvqzt2PJiI+PD2PfeoE7Qu8nKjKW8KXTCJ+/hN27/kguc+c9vYmLO06Tuh0J7dWF50f/H4Pue5LqNaoQelsXWjTuhn9AWabN+YKm9TpRvUYV7hpwO53a3MG5c/FMnfkJP4QvY//egzRr3phOXdvQpllPzp2Lp0yZ0rne3lGvjWBA70eIiYpl1qKvWbxwOXt270suc3v/UI7FHadNo550u7UDI158gscfGAnAwf0RdG/dL1djzg6hXdpzZ68ePPNSjv65yF4+PlQbO5Atd7zE2agj1A9/lb/C13Nqd0SKYr5XFyb4wS4c37A7VRVVRw/g78Wbcitiz/j4UPXVgfx6x0ucjT5C3YWv8vcPabcr6IHU7TpzIIaN7YbnZsQeSczjw/gZ9uxV9RDwITDW2TUWmHA+0TtewnVBcMUQF09HNfoDr6pqnbyc6AFU9YG8kOgdrYCbL+UEEcn1aaV69W9k396DHNgfQXx8PLNnzqdT17YpynTq0pbvvpkNwNzZ4dzSsqlrf9e2zJ45n3Pn4jl4IJJ9ew9Sr/6NVKtRmY0btnL69BkSExP5eeU6unZvD8CAgX15d/wnnDsXD8Bffx3JxdbCTfVqcWBfBIcORBIfn8C8WeG069wqRZl2nVsxc+o8ABaELaZp84a5GmNOaFCnNiWKF/N2GJekeL2qnN4Xw5kDh9H4BA7PXkWZTg1Slas0si8H35tD0pn4FPvLdG7I6YOHObXrUG6F7JFidZ12HXS168/Zq7imY+p2VRjRl0PvzyHpbHwateQ9eb1n70nCGw80EZEhwC1A8qWxiNQHygE/pHeyiOwXkdEislFEfhWRms7+USIyzK3cNhGp6Dx2ishEEdktIpNFpJ2IrBKR30WkkVv1N4nIamf/g251DReRdSKyVURGO/sqisguEfkK2AaEXBRnWxHZ5MT4uYgUEpEHgDuAl0Rkchptu8d5ji0iMsnteZY4+xeLSHln/0QReUdEfhaRvSLS29k/VUS6utU5UUR6i4iviLzh1o6HnOOtRGSZiEx3XqfJIiLOsWUi0sD5uYPz2mwUkWkiUjST96OoiHzh7NsqIr0yquei1+FxEdnhnDdVRCoCDwNDRWSziDTP5HX5SER+AV4XkSoislBENojICrf4bnd+R7aISLZdXPoHliMqMjp5OyoyBv+AcinKBASUJdIpk5iYyInjJyhduiT+AeWIjLhwbnRUDP6B5di543caN21AqVIlKVKkMO06tCQoyDUUXqVKRRo3bcCCxd8y6/tJ1KlXK7ua4pFyAdcSHRWTvB0TdZhyAWVTlPEPuJboSFcZV3tPUqp0SQCCywcRtuQbvgn7hAZN6uZe4P9ChfxLczbq7+Tts1FHKOR/TYoyRWtXolDgNRz5cWOK/b5XFSbk0VAOvDktV2K9FIUCLmpX9BEKBnjWLoDC5ctSb9Hr3DhrNMUb18zxeD2l6tnDWzLtSalqvIgMBxYCHVQ1HsDpGb8F3AW0y6Sav1S1nog8AgwDHsikfFXgduB+YB1wJ64LjR7AM8D5SZ4bgSbA1cAmEfkeqAVUAxoBAoSJSAvgoLN/gKqucX8yESkMTATaqupu54JgsKq+LSK3APNUdfpF59wAPAfcrKp/icj58dh3gS9V9UsRuR94xy3eAKcdNYEwYDrwLa4Liu9FpCDQFhgMDASOqWpDESkErBKR8xdVdYEbgChgFdAMWOkWWxkntnaq+o+IjACeBMY4RdJ6P553nq+2U0cpD+o5byRQSVXPikhJVY0TkY9wm2YRkbkZvC7BzuuYKCKLgYdV9XcRaQx8ALQBXgA6qmqkiJQkDSIyCBgEUKxwOYoUTLNYjvt9917ee/sTvp39Gaf+OcW2X38jMTERgAIFfClVqgSd2/ahbr3afDLxbRremNk/n7zhz9i/aF6nC3FHj1Hrpuv46Ku36NTsdk6e/Mfbof07iVB19AB2PvF+qkMVh99OxMfzSDx1xguBZZEIlUcPYFca7ToXe5Rf6g8m4ehJit5YmRu+GM76lk+SeNL7g675ZYFeZyAaVyI97xFgvqpGpH1KCjOd/28AKnpQfp+q/qqqScB2YLGqKvDrRefPUdXTqvoXsBRXgu/gPDbhWl9QE1eSBzhwcaJ31HCe8/zk0JdAi0xibANMc54bVT0/HtsU+Mb5eRKu5H7ebFVNcobaz3cfFwCtnYTeGfjJmS7oANwjIpuBX4Br3NqxVlUjnNdnM6lf0ybA9bguEDYDA4AKbsfTej/aAcn/ulT1qAf1nLcVmCwidwEJaRzP7HWZ5iT6oriG/qc5z/cxrgskcF3UTBTXCI5vWk+gqhNUtYGqNvA00cdExRIYFJC8HRjkT0x0bIoy0dGHk3vmvr6+FCtejCNH4oiJjiUo+MK5AYH+xES5zv1m0gw6tOxFaJe7ORZ3nD/+2A9AVFQs389dBMCmjb+SlJTENdeU8ijW7BAb/ScBgf7J2/6BZYmNPpyiTEz0nwQEucq42luUo0fiOHcunrijxwDYtuU3DuyPoFLV8rkW+7/N2ZgjFAq80OMtFFiaszEXesS+RYtwdc0Q6swcRZN171O8fjVqfTWCYjdVpli9alR5/i6arHuf4EFdKf/EbQTd38kbzUjlbPRF7Qoozbnoi9pVI4SbZo6i0br3KV6vGjd8OYKiN1VGzyWQcPQkACe37uX0gViKVAlI9RzekNeH8TPt2YtIHaA9rj/8K0VkqqpG4/rj3dzpHRYFCorISVUdmUY1Z53/J7o9ZwIpLzYKp1EeIMltO+mimC8eFFFcvflXVfXji9pREfB2F8S9XQKgqmdEZBnQEegDTHU7/piqhrtXIK6Fb+71uL+m7nUvUtX0VlKl9X6kJbN6zuuK6+KoO/CsiNTOpPzFzr8vPkCcqta5uICqPuz09LsCG0Skvqr+fXG5S7Vp469UrlKB8hWCiI46TOhtXRj8QMoV8uHzl3DHnaGsX7eZ7qEdWfnTmuT9H376Jh+99wX+AWWpXKUCGzdsBaBMmdL89dcRgoID6NK9PV3a9QFgwfc/0qx5I1at+IXKVSri5+fH338fzWozPLZ103YqVg4huHwgsdGH6XZrR4Y+lHJV/eKFy7mtbzc2rd9K5x5tWb1iHQClrylJ3NHjJCUlEVIhiIqVy3Nwf2Suxf5vc2LTHopUDqBw+bKcjT5C2dBm7Bj8v+TjiSdOser6gcnbdWaO4o/RX3Fiy14293wheX/FYbeT+M8ZIj9fmKvxp+fE5pTtuja0GTsfSdmu1TdcaNeNM0exd/RXnNyyF79rihN/9CQkJVG4fFmKVArgzIHDaT1NrsvrPfvMVuMLrgV6Q1T1oIi8gWvOvr+q9ncrdy/QIJ1En579QDfn/HpApUsLHYCeIvIqrmH8VriGk0/jzLGr6kkRCcK1kj4ju4CKIlJVVfcAdwPLMzlnCTBLRMap6t8iUtrp3f8M9MXVe+0PrPCgHd/iGkpvANzr7AsHBovIEmcqpTrg6V/WNcD759sjIlcDQW4jF2lZBPwHGAKuYXxP6nGmc0JUdamIrHTaXhQ4ARR3qz/T10VVj4vIPhG5XVWnOb9/N6rqFhGpoqq/AL+ISGdcay6ynOwTExN5ethLTJ35Gb6+Pkz5ega7du7hqWceY8umbYQvWMo3k6bz3oTXWbMpnLijx3jo/icB2LVzD2GzF7Bi7fckJCQy8v/GkJTkuhvGZ5PeoVTpkiTEJ/D0sDEcP3YCgCmTZvL2+y+zfHUY5+LjeXzwpfyTybrExERGj3yNidPex8fHh+nfhPH7rr0MGfkwv27eweKFP/Hd5Nm89cFLLFk7h7i4Yzzx4NMANGxajyEjB5MQn0CSJvH8sFc4Fnc8V+O/XMNfHMu6TVuJiztO29C7eGTg3fTq3tHbYWVIE5P4/enPuHHqs4ivD9FTlnJqVwQVn+rDiS1/8Hf4em+HeHkSk9jzzGfUmuJqV4zTrgpP9eHE5j848kP67SrR5DoqPNUHjU9Ek5L4/akJJMSdzMXg05eYx5O9aAYrBpw50Laq2sfZ9sU1hz5UVZe7lbsXV7JP9dE7EdnvHPvLWTz2pqq2EpEiwBwgCNcwdVNcw9jgmiOv5Zw/0dme7vTO56lqLXF99K4yrqHtMsDrqvqJc84TXFgXcBLXuoJE93rTiLMtrguZAk4bBztz0MnPn8Y5A4DhTt2bVPVeEakAfOHE9Cdwn3OhlKIeZxTk/KI5PyAW17TEfc4+H+C/uHrL4tQVimu+fpiqnr9Qeg9Yr6oTnRGCYaq6XkTaAK8BhZxwn1PVsAzej6K4hvHrO+0Zraoz06vH7TXwwzWFUsKJ82tVHetcnEzHNRrzGK41E568LpVwXWAGAH7AVFUdIyIznfdagMW4LkDT/eUtV6JmHv/yyktX1O8qb4eQI3buTPVP64q36oYR3g4hR/jk7Xx22VrETMtyy1b59/bob06zmOleeRUzTPbGXKks2V85LNlfOSzZp2+Fh8m+uZeSvX1drjHGGJNFSt6+ErJkb4wxxmRRUh4fS7Rkb4wxxmRRYh6/1Ywle2OMMSaLkrwdQCYs2RtjjDFZZHP2xhhjTD5nPXtjjDEmn8vryT5vrygwxhhjrgCKePTwhIh0EtddWveISLpfsykivUREnS9Iy5D17I0xxpgsSpDsmbN3vqn2fVz3pIkA1olImHMDNfdyxYAncH0DbaasZ2+MMcZkkXr48EAjYI+q7lXVc7hujtYzjXIv4foqc4/uY2zJ3hhjjMmiJA8fIjJIRNa7PQZdVFUQcMhtO8LZl8y5eVyIqn7vaXw2jG+MMcZkUZKHw/iqOgGYcLnP49wkbRwX7pDqEevZG2OMMVmUjcP4kbhu4X1eMClvb14MqAUsc+5i2gQIy2yRnvXsjTHGmCzKxo/erQOqObf7jgT6AneeP6iqx3DdKhwA91ubZ1SpJXtjjDEmi7JrNb6qJojIo0A44At8rqrbRWQMsF5Vwy6nXkv2xhhjTBZl503vVHU+MP+ifS+kU7aVJ3Vasjf5UuECBb0dQrbzlfy5xGbVDSO8HUK2a7b9NW+HkCMW1nrW2yHkWUl5+6vxLdkbY4wxWZXXvy7Xkr0xxhiTRdk5jJ8TLNkbY4wxWZRgw/jGGGNM/mbD+MYYY0w+p9azN8YYY/I369kbY4wx+Zwle2OMMSafs9X4xhhjTD5nq/GNMcaYfM6G8Y0xxph8zobxjTHGmHzOvhvfGGOMyedsGN8YY4zJ52wY3xhjjMnnEvJ4urdkb4wxxmRR3k714OPtAIzJS1q2acaSX8JYvm4eg5+4P9XxggX9eO/T11m+bh6zf5hMcEggADfVq8X8Zd8xf9l3LFg+jY5d2ySfs3LTAsJXzGD+su+Yu3hKrrXFXfM2TVm4egaL1s5i0OMDUh33K+jH25+8wqK1s5i2cCJBIQEABIUEsPXgSuYsncycpZMZ/cbTqc79cNI45v30bY63ISOlW9eh0ar/0XjNu5R/LDTdcmW6NqZV7DSK3VQ5xf5CQWVovncSIYO753So2ea5V8bRomtfQu962NuhXLJrW99E65Vv0Wb1eKo+2iPdcgFdG9E9ZgolnPfLr1RRms54js5/fEGtV+7NpWg9k+Thw1ss2edzIhIiIvtEpLSzXcrZruhsJ4rIZucRlk4dE0WkdybPc6+IBLptfyoi1zs/P3MZcVcUkW2Xel5W+Pj48NLrzzDgjsG0uzmUHrd1plqNlEmhz123cSzuOC0bduOzDycx8sUhAOz6bQ/d2/ajS6s7GHDHYF556wV8fX2Tz+vbcyBdWt1B97b9crNJgKtdL44dwYN9H6dLs9vpdmtHqlSvlKLM7f17cizuBO0b3crEj75h+AuPJR87uD+Snq3707N1f14c/mqK8zp0bc2pf07lSjvS5eNDtbED2Xrny6xtPpSytzbjqurBqYr5Xl2Y4Ae7cHzD7lTHqo4ewN+LN+VGtNkmtEt7Phr3X2+Hcel8hNqv3scvd77G0hbDCLz1ZopWD0pVzPfqwlR6oBNHN/yevC/pbDy7XpvGjtGTczNijySJZw9vsWSfz6nqIeBDYKyzaywwQVX3O9unVbWO80j/Ejtz9wLJyV5VH1DVHc7mJSd7b6hTrxb79x3k0IFI4uMTmDtrIe07t05Rpn3nVsyY6rommh+2iGYtGgNw5vQZEhMTAShUqBCqeWdQ78Z6N3Bg/6Hkdn0/+wfadW6Zokzbzi2Z9e08ABbOXUzT5o0yrfeqq4tw3+D+fDDusxyJ21PF61Xl9L4Yzhw4jMYncHj2Ksp0apCqXKWRfTn43hySzsSn2F+mc0NOHzzMqV2HcivkbNGgTm1KFC/m7TAuWam6VflnXwynDh5G4xOJmr0a/46p36+aI+5gz/tzSTx74f1KPHWWI2t3kXj2XG6G7JEk1KOHt1iy/3cYDzQRkSHALcCbl1uRiLwgIutEZJuITBCX3kADYLIzQlBERJaJSAMRGQsUcfZPvrjHLiLDRGSU83N9EdkiIluA/7iV8RWRN5zn3SoiD11u/BnxDyhHdGRs8nZ0VCz+AWVTlYmKcpVJTEzkxPGTlCpdEoA69WuzaNVMwlfM4NlhLyUnfxS+nv4x8xZPpd89vXIi9AyVCyhLjFu7YqIOU+6idpXzL5vc9gvtKgFAcPlAZi+ZzNdzPqZBkzrJ5zwxcjCff/A1Z06fyYVWpK+Qf2nORv2dvH026giF/K9JUaZo7UoUCryGIz9uTLHf96rChDwayoE3p+VKrAYKB5TitNv7dSb6bwoHlEpRpkTtihQJLM3hH6+c0ZZEDx/eYsn+X0BV44HhuJL+EGf7vMIisl5E1ohI+pOdF7ynqg1VtRZQBOimqtOB9UB/Z4TgtNtzj+TC6EH/TOr+AnhMVW+6aP9A4JiqNgQaAg+KSKVUZ3vZ5g2/0r7ZbfRo349HhgykUKGCAPTqOoCubfowoM8j3DOwL42a1vdypJ47HPsXrep2I7RNf159fjxvffRfri56NdfVqk75isEsmr/M2yFmToSqowfwx6ivUh2qOPx2Ij6eR+Ip716wGDciXD/6braP/trbkVySvN6zt9X4/x6dgWigFrDIbX8FVY0UkcrAEhH5VVX/yKCe1iLyFHAVUBrYDszNanAiUhIoqao/ObsmOTEDdABudFs3UAKoBuy7qI5BwCCA0lcFUbRw6UuKISY6loCgcsnbAYHliIk+nKpMYGA5YqJi8fX1pVjxohw9EpeizJ7d+zj1z2mqX1eVXzfvINap4++/jhD+/RLq1KvF2tUbLim2rIiNPoy/W7v8A8smx5RcJuYwAUHliI0+7NauYwDEnXP9f/vWnRzcH0mlKuWpXfd6atW5jiUbwihQwJfSZUozafbH3B2aI4MuGTobc4RCgRd68oUCS3M25kLP0bdoEa6uGUKdmaMAKFi2JLW+GsG2e16jWL1qXNutCVWev4sCJa5Gk5Sks/FEfr4wt5vxr3Em+ihF3N6vwgHXcCb6aPJ2gaKFKV4jhJtnvgBAoWtL0OjLYawd8CbHtuzN9Xg9lXcm7tJmyf5fQETqAO2BJsBKEZmqqtEAqhrp/H+viCwD6gJpJnsRKQx8ADRQ1UPO8HvhSwwngZQjSp6cL7h6/OEZFVLVCcAEgArX3HjJ//a2bNpOpcoVCCkfREx0LN1v7cTjg0amKPPjwmX06tuDjeu30qVHe35esRaAkPJBREXGkJiYSFBwAFWqVSTiYBRFriqCj4/wz8lTFLmqCC1aN+V/b3x8qaFlya+bdlCxUgjB5QOJjT5M19AOPPnwcynKLFn4E7f26cbm9b/SqXtbVq9cB0Cpa0py7OhxkpKSCKkQRMXKIRw6EMm2Lb8xZeIMwLVi/+PJb3sl0QOc2LSHIpUDKFy+LGejj1A2tBk7Bv8v+XjiiVOsun5g8nadmaP4Y/RXnNiyl809X0jeX3HY7ST+c8YSfQ6L2/wHV1f2p0j5azkTfYTA0KZsfOS95OMJJ04TfsOg5O2mM59nx+jJeTrRg32DnvEyERFcC/SGqOpBEXkD15x9fxEpBZxS1bMiUgZoBryeQXXnE/NfIlIU6A1Md/adANJbLRQvIn7O9EEsUFZErgFOAt2AhaoaJyJxInKLqq4E3If8w4HBIrJEVeNFpDoQqar/XOLLkaHExEReGPEKX037EF9fX777Zja/7/qDJ0c+wtbNO/hx4TK+/XoW4z98heXr5hEXd4xHH3gKgAZN6vLIE/cTH5+AJinPDX+Zo0fiCKkQxISv3gagQAFf5sxYwPIlq7IzbI/aNebpN/jsu3fx9fFl+pQw9uzay+MjHmLb5t9YEv4T0ybP4Y0PxrBo7SyOHT3O0EGuNZUNm9bjiREPkZCQQFKS8sKwVzkWdzxX48+MJibx+9OfcePUZxFfH6KnLOXUrggqPtWHE1v+4O/w9d4OMUcMf3Es6zZtJS7uOG1D7+KRgXfTq3tHb4eVKU1MYtszE2ky5WnE14dDU5ZxclcENZ7qTdzmfcT+kPGoV9t171CgaBF8ChbAv1MD1vR9lZO7I3Mp+vR5c4jeE5KXVg2b7OcMbbdV1T7Oti+wDhgKxAMf47oo9QHeVtVUS6tFZCIwT1Wni8h/gX5ADLAbOKCqo0SkF/AKcBpoCiwAhqnqehF5DegBbFTV/iLyOPAEEAnsBfY7ddQHPsc1IvYD0EVVa4mID/BfoDuuXv6fQKiqHkuv3ZfTs8/rCvkU9HYIOWKCT55bfpFlzba/5u0QcsTCWs96O4Qc0T1mSpY/FDe0Yl+P/uaM3z/VKx/As2Rv8iVL9lcOS/ZXDkv26Xu8Yh+P/ua8s/9bryR7G8Y3xhhjssjm7I0xxph8Lq/P2VuyN8YYY7Iob6d6S/bGGGNMllnP3hhjjMnnEvN4srevyzXGGGOyKDtvcSsinURkl4jsEZGRaRx/UkR2OPcKWSwiFTKr05K9McYYk0Xq4X+Zcb4L5X1cXxd+PdDv/O3C3WzC9U2mN+L6YrOMvgwNsGRvjDHGZFk29uwbAXtUda+qngOmAj3dC6jqUlU95WyuAYIzq9Tm7I0xxpgsSsq+L6gLAg65bUcAjTMoPxDXN5ZmyJK9McYYk0Wepnr3u3M6Jjg38bpkInIX0ABomVlZS/bGGGNMFiV6OEjvfnfOdEQCIW7bwc6+FESkHfAs0FJVz2b2vDZnb4wxxmRRNs7ZrwOqiUglESkI9AXC3AuISF1cNzHroaqHPanUevbGGGNMFmXXl+qoaoKIPIrr1t6+wOequl1ExgDrVTUMeAMoCkxz3cWcg6raI6N6LdkbY4wxWeTJx+o8rkt1PjD/on0vuP3c7lLrtGRvjDHGZJHd9c4YY4zJ5zT7PnqXIyzZm3wp6sTf3g4h2+XtPyWXz6d0JW+HkO0W1nrW2yHkiE7bXvZ2CHlWQh7/F2rJ3hhjjMmi7JyzzwmW7I0xxpgsslvcGmOMMfmczdkbY4wx+ZytxjfGGGPyOU+/LtdbLNkbY4wxWWTD+MYYY0w+Zwv0jDHGmHzOPnpnjDHG5HNJNoxvjDHG5G95O9VbsjfGGGOyLMFW4xtjjDH5m63GN8YYY/I5W41vjDHG5HO2Gt8YY4zJ52wY3xhjjMnnbBjfGGOMyecSNW+vxvfxdgDG5DXjx43htx0r2bhhEXXr1EqzTL26tdm08Ud+27GS8ePGpDo+ZMhDxJ+L5JprSgFQo0YVVvwUxskTexk69KEcjT8948eNYaeH7dqZTruGDnmIBLd2lSxZgunTPmXjhkWsXjWPG26okaNtSE+p1nVosPJ/NFz9LiGPhqZbrkzXxrSImUbRmyoDUCjkWprtm0y9H9+g3o9vUPW1B3MrZI9c2/omWq98izarx1P10R7plgvo2ojuMVMo4bTLr1RRms54js5/fEGtV+7NpWizx3OvjKNF176E3vWwt0O5JOrhf95iyf4KJiIhIrJPREo726Wc7YrOdnkR+UH+v707j7OiutM//nlYBBTBXVkUEdxwQ0BFXH4GN0wGo8Zo3MbdRI2ijproJFEz5ucedeK+bxknozFGnagY97ghIqJiiCsRAXcFRUHgO39UNVzapunb99Knb/Xzfr361bdO3W6fEuhvn1OnzpFekzSxrr3e97hJ0t5LIdvT+ee1Je3fjK8/RNJl1c61JCNGDKd//75sOGBbjj76Z1x22TkNvu+yy87hJz85lQ0HbEv//n3ZddfvLDjXu3dPdt5peyZPnrKg7ZNPPuPEE3/Jby++eqlfQ0N2GzGcdfv3ZYP8ui5fzHVdnl/XBgO2Zd3+fRmxhOs67WfH8dJLrzJo8M4cctgoLr7o278gLHXt2tH/nMN5Zf/fMHb7E1l1z21Ydr3e33pb++U60+uI7zLjhX8s0v715OmM2+kUxu10Cm/87NqWSr1k7cQm5xzKc/ufx6Pbn0zPPYfRdb1e33pb++U60/eIEXz6wusL2ubP/oZJ593BxLN+35KJq2KPUX6hhgAAIABJREFU7+7MVb89O3WMss2PaNJHKi72NSwi3gWuBM7Nm84FromId/LjW4ALImJDYEvggxbMNix/uTZQdrFPZfeRu3Lb7+8E4Lkx4+i+QnfWWGO1Rd6zxhqrsXy35XluzDgAbvv9nXx/9xELzl944ZmcdvpvFpmw8+GHHzP2hZf45ptvWuAqvm3kyF25tczruvX3d7J7yXVddOGZ/LzedW244Xo8+uhTAEya9CZ9+vRmtdVWWdqXs4jlN+/PV29P5+t/fkB8M5cP736KlXcd8q339fnZj3j38j8zf3aaP4Nyrbh5f758ezqz/vkB8c08pt79DGs0cF0b/Gwf3rj8XuaVXNe8WbP5ZMwk5s2e05KRq2LIwE3o3m351DHK5p69LW0XA0MlnQBsC1wIIGkA0CEiHgKIiC8iYlZj30jSjpJelPSypBskdcrb35F0lqRx+bkN8vZVJT0k6VVJ10maLGmV/NwX+bc9F9hO0nhJJ9bvsUu6T9IO+etDJf1D0hhgm5L3rCrpj5Kezz8WnKu2nj3XYMq7UxccvzdlGr16rrHIe3r1XIP3pkxbcDxlyjR65u8ZOXIXpr43jQkTJi6tiM3SqxnXVfqekSN34b0GrmvCyxPZc4/vArDFkIH06dOb3r16LK3LaFCnHisxe+rHC45nT/uEZXqsvMh7um7Sl049V+aTv4771td3Xms1Bj10Ppv+6Sy6bbXBUs/bVJ17rMhXJdf19bSP6dxjxUXe032TtenScyU++OuLLR3P6nHP3paqiPgGOIWs6J+QHwOsB3wm6a68gF8gqf3ivo+kzsBNwL4RsQnZ5M2jS97yUUQMIhtJODlvOwN4JCI2Au4E1mrgW/8ceDIiBkbExY3893sAZ5EV+W2BASWnLwUujogtgB8A1y3mexwlaayksfPnf7m4/9RS06VLZ37+s+M486wLW/y/vTR16dKZ0xZzXeedfxndV+jG2OdHc+yxh/Hi+FeYN7+VTVSSWOesg3nrrFu+dWrO+5/y3OCjGbfzqbx1xs1seMUo2nftkiBkM0gMOOsgXj3rttRJjGyCXlM+UvFs/GLYDZgGbAw8lLd1ALYDNgf+CfwBOAS4fjHfY33g7Yiou6F5M3AscEl+fFf++QVgr/z1tsCeABHxgKRPK7iGrYDHIuJDAEl/IPuFBWAnYICkuvd2k9Q1Ir4o/QYRcQ1wDUDHZXo1+Vfoo39yMIcffgAAY8eOp/eaPRec69W7B+9Nnb7I+9+bOp1evRf2Xnv37sHUqdPp129t1l57LV4Y+9CC9jHPPciwbb7H++9/2NQ4VVPpddW9p+66xpVc1/PPPcjW+XUdceRJC77mjX88y1tvTV6al/Uts6d9QqeeC3vynXqsxJxpC3vE7bt2Ybn112Szu84EYJlVV2Cjm3/GqwefxxcvvcXcOdlfoy8mvMVXk9+nS78efPHSWy16DQ35etqndCm5rs49VubraQv/iXXo2plu66/JsLt+BUCnVbuz5c0nM+bgC/m8FeRva1r7ojru2dc4SQOBnYGhwIl5DxlgCjA+It6KiLnA3cCgCv5Ts/PP86jsl8S5LPr3rnMTvqYdMDQfHRgYEb3qF/pKXHnVzQzZYheGbLELf77nQQ48IJuvuNWWg5jx+QymT190qsP06R8wc8ZMttoy+9954AF7c8+9D/LKK3+nV+/NWHe9oay73lCmTJnGllvtmqTQw6LXdc89D3JQmdd10AF7c29+XT17b0b/9YbSP7+uLfLr6t69Gx07dgTg8MP258m/PcfMmVX7o2mSmePfoMs6Pei81mqoYwdW3WMbPh49dsH5eTNn8cxGhzNmi2MZs8WxzBj3+oJC33HlbtAu++vYea3V6NK3B19PbrGpLY36bPybLLfOGnRZa1XUsT0999ia6aNfWHB+7syveHCjo3h4i+N5eIvj+XTcGy70CbX2YXz37GuYsq7ulWTD9/+UdAHZPfsDgOeBFSStmveWhwNjF//dmASsLal/RLwBHAQ8voQITwH7AOdJ2gVYsYH3zARKZ9u8AxwjqR3Qi2ziIMBzwKWSVgZmAD8EXsrPjQaOAy7Ir3tgRIxfQrZmuf/+h9ltxHD+/tpTfPXVVxxxxMJe69jnRzNki10AOO6407nu+ovp0rkzDz74KA888Eij33f11Vfl2Wfup1u3rsyfP5/jjzuSTTfbocUK41/uf5gRI4Yz6bWnmNXIdf30uNO5Pr+uBx58lPuXcF0bbrAuN9xwCRHBxImTOPKokxt9/1Ixbz5vnH49G9/+76h9O6bf/iizJk2hz6n7MnP8m3wyevF/7bsP3ZA+p+5LfDOPmD+f10+9hrmftewvK4sT8+bzyuk3MfT201D7drx7+2N8MWkK65+6N5+Nf5v3Swp/Q3Z8/j/p0LUL7ZbpwBojhvDsj87hi3+810Lpm++UM87l+Rcn8NlnM9hxjwM55vCD+MHIXVPHWqLW3rNXa1/izxZP0lHAjhGxb37cnqzInxgRj0vaGbgIENnw+1ERMafe97gJuC8i7pS0I9kvCx3y73N0RMyW9A4wJCI+kjQEuDAidpC0GnA7sDrwDPAvwNr513wREV0ldQQeBFYmmxNwCXAbMBh4jewXhDMj4jFJhwKnAZ8B44E5EfHTfNLf5cCGebYnIqLRh3DLGcavFYW7oNwjKw1b8ptqzOcF7UeNeOU3qSMsFR1XWUdLflfj+q68WZP+ib798UsV/7eaw8Xemi2frT8vIuZK2hq4MiIGps4FLva1xMW+drjYL16flTdt0j/RyR9PSFLsi/k30lrKWsD/5EPyc4DWtfyYmVkLae3L5brYW7NFxOtks/3NzNq01j5K7mJvZmZWoZQz7ZvCj96ZmZlVqJrL5UoaIWmSpDck/byB850k/SE//1xD+57U52JvZmZWoYho0seS5E9VXU62WNoAYL98+fNShwOfRkR/stVTz1vS93WxNzMzq9B8okkfTbAl8Ea+INoc4L+B79d7z/fJVjmFbKnyHVWyxGhDXOzNzMwqNG/+/CZ9lO7hkX8cVe9b9QLeLTmekrc1+J58hdTPydYyWSxP0DMzM6tQU2fjl+7h0ZLcszczM6tQFYfx3wPWLDnunbc1+B5JHYDuwMc0wsXezMysQtWaoEe2VPm6kvpKWgb4EXBPvffcAxycv96bbKvxRr+5h/HNzMwqVK3n7PPlx39KtqdIe+CGiHhV0q+BsRFxD9lW5bdKegP4hOwXgka52JuZmVWomsvlRsRfgL/Ua/tVyeuvyXYGbTIXezMzswp5uVwzM7OCa+372bvYm5mZVcg9ezMzs4Jr7cVerT2gWWsn6ah8oYzCKOI1QTGvq4jXBMW9rlT8nL1Z5eovd1kERbwmKOZ1FfGaoLjXlYSLvZmZWcG52JuZmRWci71Z5Yp4X7GI1wTFvK4iXhMU97qS8AQ9MzOzgnPP3szMrOBc7M3MzArOxd7M2gRJK0raNHUOa5ykZVNnKCIXe7MySVpP0sOSXsmPN5X0i9S5KiFpWUm/lHRtfryupH9JnatSkh6T1E3SSsA44FpJv02dq1KS+kjaKX/dRdLyqTNVStIwSROBv+fHm0m6InGswnCxNyvftcBpwDcAETGBJuwn3crdCMwGts6P3wPOThenarpHxAxgL+CWiNgK2ClxpopIOhK4E7g6b+oN3J0uUdVcDOwKfAwQES8B2ydNVCAu9mblWzYixtRrm5skSfX0i4jzWfgLzCxAaSNVRQdJPYB9gPtSh6mSY4FtgBkAEfE6sFrSRFUSEe/Wa5qXJEgBudible8jSf0g29NS0t7AtLSRKjZHUhcWXlM/sp5+rfs18CDwZkQ8L2kd4PXEmSo1OyLm1B1I6gCtfH/VpnlX0jAgJHWUdDLwWupQReHn7M3KlBeMa4BhwKfA28ABETE5abAKSNoZ+AUwABhN1nM8JCIeS5nLvk3S+cBnwL8CxwHHABMj4t+TBquQpFWAS8lus4js7+GoiPg4abCCcLE3K5OkvhHxtqTlgHYRMbOuLXW2SkhaGRhK9oP22Yj4KHGkiklaD7gSWD0iNs5n4+8eETU7H0FSO+BwYBeyP6sHgevCP8ytES72ZmWSNC4iBtVreyEiBqfKVClJgxpo/hyYHBE1Ox9B0uPAKcDVEbF53vZKRGycNpnVkfQ7GrkNERHHt2CcwuqQOoBZrZC0AbAR0F3SXiWnugGd06SqmiuAQcAEst7ixsCrZNd6dESMThmuAstGxBhpkbmGNfnLi6SXabwo1uoaAmPzz9uQ3Ub6Q378Q2BikkQF5GJv1nTrA/8CrACMLGmfCRyZJFH1TAUOj4hXASQNIJvcdipwF9n901pUpMmUdeseHJt/vjX/fCA1PEEvIm4GkHQ0sG3dSJKkq4AnU2YrEg/jm5VJ0tYR8UzqHNXU0NB2XZuk8RExMFW2ShR0MuWLdbckStq+dWup1kiaBGwdEZ/kxyuSzR1ZP22yYnDP3qx8L0o6lmxIf8HwfUQcli5SxV6VdCXw3/nxvsBESZ3In72vNZLaA8dExE6lkylT56oCSdomIp7KD4ZRjMeozyX7t/Uo2a2k7YEzkyYqEPfszcok6Q6yJT33JxvqPgB4LSJGJQ1WgfwZ+2OAbfOmp8ju439Ndt/7i1TZKiHp2YgYmjpHNUkaDNwAdCcrip8Ch0XEuKTBqkDSGsBW+eFzETE9ZZ4icbE3K1PdMKqkCRGxqaSOwJNFKypFkI9W9ALuAL6sa4+Iu5KFqhJJ3QEi4vPUWapBUoNL40bEEy2dpYg8jG9Wvrph7c8kbQxMp8aXK5W0LnAO2Wzo0lsT6yQLVR2dydZaH17SFmSTDmuSpF/VOwYgIn6dJFD1nFLyujOwJfACi/7ZWTO52JuV75p88tAvgXuArsCvGv+SVu9G4AyyzUi+AxxKAe4DR8ShqTMsBV+WvO5MNku/5peVjYjSJ1yQtCZwSaI4heNhfDNbsCiQpJcjYpPSttTZKiGpN/A7sme4IXuUa1RETEmXqrrySZQPRsQOqbNUk7Ihi1cjYkDqLEXgnr1ZE0k6qbHzEVHL+6TPzpdhfV3ST8m2uO2aOFM13Aj8F9kCLZA9k34jsHOyRNW3LNk2tzWt3kp67YCBQM1POmwtXOzNmm75/PP6wBZkQ/iQLbBTf8vbWjOKrGgcD/wH2VD+wUkTVceqEXFjyfFNkk5IlqYK6q2k1x5YlezPrNaNLXk9F7i97vFCq5yH8c3KJOkJ4Ht1z2xLWh7434hocDaxpSPpYbKe/O15037AoRGxY7pUlZHUp+RwLvB+Le9fUEfSqIi4dElt1jw1PwHHLIHVgTklx3PyNmt9DgP2IXtiYhqwN3BIykBVcHZETM4/3ouIuZJuXfKXtXoNjSQd0tIhisrD+GbluwUYI+lP+fEewE3p4lgjekfE7qUNkrYB3k2Upxo2Kj2Q1AGo2YmUkvYjW6Cqr6R7Sk4tD3ySJlXxeBjfrBnyLWG3yw+fiIgXU+axhi1mO+KaXEde0mnA6UAXYFZdM9nI0jURcVqqbJXIb0v0JVvn4eclp2YCE4pwi6I1cLE3MyStB1wJrJ5vfrMpsHtEnJ04WrNI2pps85sTyNYOqNMN2DMiNksSrAoknVOrhd3S8T17MwO4FjiNfHXAiJgA/ChpososQ/boYAey4eC6jxlk9+1rjqQN8pd3SBpU/yNpuApI+lv+eaakGSUfMyXNSJ2vKNyzNzMkPR8RW5Run1rLW9vWkdSnbjvbfB2BrhFRkwVE0jURcVS+K1x9ERFeVtYWyxP0zAzgI0n9yJ/flrQ32ez1WneOpJ8A84DngW6SLo2ICxLnKltEHJW/3C0ivi49J6lzA19SUySt1EDzzIioyS2WWxsP45uVSdJekl6X9HmBhhuPBa4GNpD0Htm97qPTRqqKAXlPfg/gfrKJYAeljVSxp5vYVmvGAR8C/wBez1+/I2lcvq2vVcA9e7PynQ+MjIia33ykTkS8BewkaTmgXd2CQQXQMd+CeA/gsoj4RlJN3rvM93rvBXSRtDnZTHzIJh0umyxY9TwE3BkRDwJI2gX4AdmiSFewcJ97awb37M3K936RCj1kK5VJ6kb2SNfFeW9ql9S5quBq4B1gOeCJ/DGvWh2F2RW4kGwd/N8CF+UfJ5E9klfrhtYVeoCIGA1sHRHPAp3SxSoGT9AzK5OkS4E1gLuB2XXtEVHLe6S/FBGbSdoV+AnwC+DWWnwevTH5Tmrta/nZbUk/iIg/ps5RbZJGAw8D/5037Uu2YdEI4Pmi/V1saR7GNytfXQ+4tOcbQM0WexYOCX8XuCUiXs0LY6FE1rup2UIPEBF/lPQ9spX0Ope0/zpdqqrYHziD7JdogKfytvZkSx5bBdyzNzMk3Uh2P7gvsBnZD9jHan0/+yKSdBXZPfrvANeRrRswJiIOTxrMWjUXe7MySeoN/A7YJm96EhgVEVPSpapM/gz6QOCtiPhM0spAr3xxHWtFJE2IiE1LPncF7o+I7Zb4xa1YvorjycDalIw6e/2A6vAwvln5bgT+C/hhfnxg3rZzskSV2zb/vGmRRu8lLQv8G7BWRBwpaV1g/Yi4L3G0SnyVf54lqSfwMdAjYZ5quQO4imy0Yl7iLIXjYm9WvlUj4saS45sknZAsTXWcUvK6M7Al8AJQ672qG8muY+v8+D2yolLLxf4+SSsAF5A9mx5kBbLWzY2IK1OHKCoP45uVSdLDZEXk9rxpP+DQiNgxXarqkrQmcElE/CB1lkpIGhsRQ+otA/xSLW+EU0pSJ6BzRHyeOkulJJ0JfAD8iUWfcvE2t1Xgnr1Z+Q4ju2d/MVmv6mng0KSJqm8KsGHqEFUwR1IXFi4D3I+SQlJLJO3VyLmafvQzd3D+uXSUKYB1EmQpHBd7szLlG6vsnjpHNUn6HXlBJFtsayDZEHGtOwN4AFhT0u/JJlUekjRR841s5FytP/pJRPRNnaHIPIxv1kSSTo2I8+sVxgUi4vgEsapC0sElh3OBdyLiqVR5qil/smAo2VoCz0bER4kjWQPyZY2PBrbPmx4DrvZGONXhYm/WRJJGRsS99QrjAhFxc0tnqiZJywDr5YeTivBDVtKewCN197TziW07RMTdjX+ltTRJ1wEdgbp/RwcB8yLiiHSpisPF3qxMkn4YEXcsqa2WSNqB7IfsO2Q94DWBgyPiiYSxKiZpfEQMrNe2YLKetR4NTZws0mTK1LwRjln5TmtiWy25CNglIv5fRGxPtunKxYkzVUNDP+M8V6l1mpdPoARA0jr4efuq8V96syaStBvZ2vG9JP1nyalu1Ph660DHiJhUdxAR/8jvoda6sZJ+C1yeHx9L9tx9zZPUF9gcmBgRf0+dpwpOAR6V9BbZ6FIfiveUSzIexjdrIkmbkc1S/zXwq5JTM4FHI+LTJMGqQNINwHzgtrzpALLd4Q5Ll6pykpYDfgnslDc9BJwdEV+mS9U8ku6OiD3y198HLiGbxDYMOCcibkqXrjrydQPWzw8nRURNPibZGrnYm5Up3/f9y4iYlx+3BzpFxKy0yZov/yF7LAuXzX0SuMI/bFuPegsDPQ0cEBFvS1oFeLhW7203tn4A1PbW0a2Jh/HNyjearKf4RX7cJW8blixRhfKi/tv8ozAKtrlKac+sQ0S8DRARH0manyhTNRR6/YDWwsXerHydI6Ku0BMRX+QbrtQsSdsAZ5LdJy0tirW+elmRNlfZTNIMsvvZnST1iIhp+SOT7RNna7aI8H35FuBib1a+LyUNiohxAJIGs3Anslp1PXAi2eS1Wi+KpQqzuUpELK6gLwv8uCWzWO1xsTcr3wnAHZKmkvWy1gD2TRupYp9HxP2pQywF90o6hgJsriJprYj4Z/32iPhMkn+WW6M8Qc+sGfLH0kpnDdfkanOSBuUv9yEbCr6LRYtiTa+PL+ntBpqjFm9P5I+kXQVcVDI5dHWyNRI2iIghKfNZ6+Zib9YMkjYGBpDt/Q5ARNySLlHzSHq0kdNRoxPZCknSisC5ZBNBRwGbACcB5wNXRkRNTtKTtBbwQUR8LUlkGxUNAiYC10ZEra9h0Sq42JuVSdIZwA5kxf4vwG7A3yJi75S57NvyiZMnAWtFxFGS1gXWj4j7EkdrNkmjyFY3nAoMjYgpiSNVRNIrwJYRMUvSeUA/4G5gOECtr/XQWni5XLPy7Q3sCEzPZxJvBnRPG8kW40ZgDgsfi3wPODtdnOaTtIKkq8lWlRsB3AncL6nWR1/alaxRsROwT0Tclhf5wQlzFYqLvVn5vsqHTOfmC+x8QLZxjLU+/SLifOAbgLyoKG2kZhsHvA4MiYjREXEC2c5wZ0u6PW20irxb8gvLO+T/lvKtia1KPIPTrHxj861SryV7VO0L4Jm0kWwx5kjqQr4gTb7RSq2uCrh9/SH7iBgPDJN0ZKJM1XAEcIukM4HPgfGSxgMrAP+WMliR+J69WQUkrQ10i4gJiaM0Wz46sWpEvFmvfdNavi4ASTsDvyCbXzEa2AY4JCIeS5nLvk3ShsB6ZJ3QKcDztTrpsDVysTcrU77a3PiI+FLSgWQzhy+NiMmJo5VN0j5kG6p8AHQkK4TP5+fGRcSgxr6+FuTDwUPJhu+fjYiPEkeyEotbPyA/t11EPNnSmYrI9+zNynclMCvfBe/fgDeBmnvsLnc6MDgiBpJN/LpV0p75uVq9t42kDfLPg8iWAJ5GNnt9rZK1Bax1eEzSqfmGUkC2foCk28ieOrAq8D17s/LNjYjItxm9LCKul3R46lDN1D4ipgFExBhJ3wHuk7Qmi268UmtOAo4iW3CmviB/rMtahcFk6weMzx8rLF0/4F9TBisSD+OblUnS48ADZD3h7cmGwF+KiE2SBmuGfKvUg0rv10tanuw5520jolOycNamFG39gNbGPXuz8u0L7A8cHhHT8xXALkicqbmOpt7tvIiYKWkE2RK6NU/SML69xW2t3nYpnPzJlvOArcjWD/gu2foBoyLikaThCsQ9e7M2rOiToyTdSrYi23gW7uYXEXF8ulRWKl/z/wrgkrqlcSUNzNsmR8R+KfMVhYu9WZkk7UXWE1mNbBKbyApIt6TBmqHom6tIeg0YEP5B12pJ6r24IXtJR0bEtS2dqYg8G9+sfOcDu0dE94joFhHL12Khzw0m7/lKGp7fNx1DtkjQlkmTVccrZFsQWyvV2L15F/rq8T17s/K9HxGvpQ5RDRHxKfDjvMj/leJNjloFmChpDItu3bt7ukhmLc/F3qx8YyX9gWzGemkBuStdpOZpA5OjzkwdwKw18D17szJJurGB5qjFrTjbwuQoSX2AdSPir/mWt+0jYmbqXGYtycXerA0r+uSofIOYo4CVIqJfvp/9VRGxY+JoZi3Kxd6sTJI6A4cDGwGd69prsWdfdPnuaVsCz0XE5nnby7W4AJJZJTwb36x8t5LN8N4VeBzoDXhYuHWaHRFz6g4kdaC2lwE2axYXe7Py9Y+IXwJfRsTNwPfIJrhZ6/O4pNOBLvl2t3cA9ybOZNbiXOzNyvdN/vkzSRsD3ckW2LHW5+fAh8DLwI+Bv5Dtb2/WpvjRO7PyXSNpRbKicQ/QFfhl2ki2GHsAt9T6REOzSrnYm5VBUjtgRr4YzRPAOokjWeNGAhdLegL4A/BA3SOGZm2JZ+OblUnS2FpfM74tkdQR2I1st8JtgYci4oi0qcxalou9WZkknQt8RNZT/LKuPSI+SRbKGpUX/BHAocD2EbFK4khmLcrF3qxMkt5uoDkiwkP6rYykuh79DsBjwP8Aoz2Ub22Ni71ZmSR1joivl9Rm6Um6nWwE5v6ImL2k95sVlYu9WZkkjYuIQUtqMzNrLTwb36yJJK0B9CJboGVzQPmpbsCyyYLZYknai2xXv9XI/rxEdsulW9JgZi3MPXuzJpJ0MHAIMAR4noXFfgZwcy1ucVt0kt4ARkbEa6mzmKXkYm9WJkk/iIg/ps5hSybpqYjYJnUOs9Rc7M2ssCRdSrZp0d3Aggl6HoWxtsb37M2syLoBs4BdStoCcLG3NsU9ezMzs4LzrndmVZBvn2qtjKTekv4k6YP844+SeqfOZdbSXOzNquP61AGsQTeS7UzYM/+4N28za1M8jG/WRJLuWdwpYHhELNeSeWzJJI2PiIFLajMrOk/QM2u67YADgS/qtQvYsuXjWBN8LOlA4Pb8eD/g44R5zJJwsTdrumeBWRHxeP0TkiYlyGNLdhjwO+Bisln4T5PtfGfWpngY38zMrOA8Qc+siSSt1ci57VoyizWNpJslrVByvKKkG1JmMkvBxd6s6R6TdKqk9nUNklaXdBvZMLG1PptGxGd1BxHxKbB5wjxmSbjYmzXdYKAfMF7ScEmjgDHAM3iCXmvVTtKKdQeSVsJzlawN8l96sybKe4U/zov8X4GpwNCImJI2mTXiIuAZSXfkxz8EfpMwj1kSnqBn1kT5vd/zgK2AU4HvAjsCoyLikZTZbPEkDQCG54ePRMTElHnMUnCxN2siSW8BVwCXRMTcvG1g3jY5IvZLmc/MbHFc7M2aSFLvxQ3ZSzoyIq5t6UxmZk3hYm9mhSOpU0TMXvI7zdoGz8Y3syJ6BkDSramDmLUGno1vZkW0jKT9gWGS9qp/MiLuSpDJLBkXezMrop8ABwArACPrnQvAxd7aFN+zN7PCknR4RFyfOodZai72ZlZYkpYh6+Vvnzc9DlwVEd+kS2XW8lzszaywJF0HdARuzpsOAuZFxBHpUpm1PBd7MyssSS9FxGZLajMrOj96Z2ZFNk9Sv7oDSesA8xLmMUvCs/HNrMhOAR7NlzoW0Ac4NG0ks5bnYXwzKzRJnYD188NJXlnP2iIXezMzs4LzPXszM7OCc7E3s0JSZs3UOcxaAxd7MyukyO5R/iV1DrPWwMXezIpsnKQtUocwS80T9MyssCT9HegPTAa+JHv8LiJi06TBzFqYi72ZFZakPg21R8Tkls5ilpKH8c2ssPLjSc5uAAADzklEQVSiviYwPH89C//cszbIPXszKyxJZwBDgPUjYj1JPYE7ImKbxNHMWpR/wzWzItsT2J3sfj0RMRVYPmkiswRc7M2syObkj+AFgKTlEucxS8LF3syK7H8kXQ2sIOlI4K/AtYkzmbU437M3s0KTtDOwS344OiIeSpnHLAVvcWtmRfcy0IVsKP/lxFnMkvAwvpkVlqQjgDHAXsDewLOSDkubyqzleRjfzApL0iRgWER8nB+vDDwdEes3/pVmxeKevZkV2cfAzJLjmXmbWZvie/ZmVjiSTspfvgE8J+nPZPfsvw9MSBbMLBEXezMrorqFc97MP+r8OUEWs+R8z97MzKzg3LM3s8KSNAT4d6APJT/vvMWttTXu2ZtZYeWz8U8he75+fl27t7i1tsY9ezMrsg8j4p7UIcxSc8/ezApL0o7AfsDDwOy69oi4K1koswTcszezIjsU2ADoyMJh/ABc7K1Ncc/ezApL0iSvlmfmFfTMrNieljQgdQiz1NyzN7PCkvQa0A94m+yevYDwo3fW1rjYm1lhSerTULsfvbO2xhP0zKzI3Jsxwz17MyswSS+TFXwBnYG+wKSI2ChpMLMW5p69mRVWRGxSeixpEHBMojhmybhnb2ZtiqSX6/8SYFZ07tmbWWGV7GsP2aPGg4CpieKYJeNib2ZFtnzJ67nA/wJ/TJTFLBkP45uZmRWce/ZmVliS1gNOBtZm0f3sh6fKZJaCe/ZmVliSXgKuAl4A5tW1R8QLyUKZJeBib2aFJemFiBicOodZai72ZlZYks4EPgD+xKL72X+SKpNZCi72ZlZYkt5uoDkiYp0WD2OWkIu9mZlZwXk/ezMzs4JzsTczMys4F3szM7OCc7E3szZB0v9PncEsFa+gZ2aFI+k/6zcBB0nqChARx7d8KrN0XOzNrIj2BB4HRpMVeoAfka2kZ9bm+NE7MyscScsD/wGsBpwcEVMlveXn662tcs/ezAonImYCJ0gaDPxe0v/iOUrWhvkvv5kVVr7hzXDgK+BvieOYJeNhfDMrHElrRcQ/F3Nuu4h4sqUzmaXknr2ZFdFjkk6V1L6uQdLqkm4DLk6YyywJF3szK6LBQD9gvKThkkYBY4BngC2TJjNLwMP4ZlZYeZG/GJgKDI2IKYkjmSXhnr2ZFY6kFSRdDRwKjADuBO6XNDxtMrM03LM3s8KR9BZwBXBJRMzN2wbmbZMjYr+U+cxamou9mRWOpN6LG7KXdGREXNvSmcxScrE3MzMrON+zNzMzKzgXezMzs4JzsTczMys4F3szM7OCc7E3MzMruP8DEqTaX6bb4j4AAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "tags": [], + "needs_background": "light" + } + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "RUSAN1_lFVqX", + "colab_type": "code", + "outputId": "dedb63f1-52c9-434c-d177-edfd7ad203b3", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 204 + } + }, + "source": [ + "X = dataset[['X1 transaction date', 'X2 house age', 'X4 number of convenience stores', 'X5 latitude', 'X6 longitude']]\n", + "X_aug = sm.add_constant(X) # augmented regressor matrix with a column of ones inserted at the front\n", + "vif = pd.DataFrame()\n", + "vif['VIF Factor'] = [variance_inflation_factor(X_aug.values, i) for i in range(X_aug.shape[1])]\n", + "vif['Predictor'] = X_aug.columns\n", + "vif = vif.drop(0)\n", + "vif" + ], + "execution_count": 72, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
VIF FactorPredictor
11.005277X1 transaction date
21.012527X2 house age
31.398292X4 number of convenience stores
41.349140X5 latitude
51.365232X6 longitude
\n", + "
" + ], + "text/plain": [ + " VIF Factor Predictor\n", + "1 1.005277 X1 transaction date\n", + "2 1.012527 X2 house age\n", + "3 1.398292 X4 number of convenience stores\n", + "4 1.349140 X5 latitude\n", + "5 1.365232 X6 longitude" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 72 + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "mtuKLp1upZob", + "colab_type": "code", + "outputId": "90478839-ac46-49a0-b657-1a951487429b", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 136 + } + }, + "source": [ + "regressor = linear_model.LinearRegression()\n", + "regressor.fit(X, y)\n", + "coeff_df = pd.DataFrame(regressor.coef_, X.columns, columns=['Coefficient'])\n", + "# print the calculated coefficients as well as the intercept\n", + "print('Intercept %.6f' %(regressor.intercept_))\n", + "print(coeff_df)" + ], + "execution_count": 73, + "outputs": [ + { + "output_type": "stream", + "text": [ + "Intercept -42310.207230\n", + " Coefficient\n", + "X1 transaction date 4.209764\n", + "X2 house age -0.279726\n", + "X4 number of convenience stores 1.565796\n", + "X5 latitude 337.629436\n", + "X6 longitude 209.338154\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "Jv9fR3k4FrEf", + "colab_type": "code", + "outputId": "99dc9c80-e038-425a-a878-d18b021e7a4e", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 1000 + } + }, + "source": [ + "y_hat = regressor.predict(X) # predicted values\n", + "df_comparison = pd.DataFrame({'Actual': y, 'Predicted': y_hat}) # put the actual and predicted values together\n", + "df_comparison = df_comparison.head(50) # show top 50 rows of this data frame\n", + "print(df_comparison)\n", + "df_comparison.plot(kind = 'bar') # visualize the comparison\n", + "plt.grid(which = 'major', linestyle = '-', linewidth = '0.5', color = 'green')\n", + "plt.grid(which = 'minor', linestyle = ':', linewidth = '0.5', color = 'black')" + ], + "execution_count": 74, + "outputs": [ + { + "output_type": "stream", + "text": [ + " Actual Predicted\n", + "0 37.9 48.404279\n", + "1 42.2 49.290901\n", + "2 47.3 50.890731\n", + "3 54.8 50.541321\n", + "4 43.1 47.018079\n", + "5 32.1 30.828825\n", + "6 40.3 39.923115\n", + "7 46.7 47.081491\n", + "8 18.8 14.384292\n", + "9 22.1 32.889071\n", + "10 41.4 29.658713\n", + "11 58.1 53.456919\n", + "12 39.3 39.269362\n", + "13 23.8 27.573710\n", + "14 34.3 48.325797\n", + "15 50.5 38.696363\n", + "16 70.1 51.532243\n", + "17 37.4 33.168931\n", + "18 42.3 47.268804\n", + "19 47.7 46.197155\n", + "20 29.3 34.528204\n", + "21 51.6 50.308416\n", + "22 24.6 30.417507\n", + "23 47.9 49.014246\n", + "24 38.8 33.751416\n", + "25 27.0 29.968922\n", + "26 56.2 47.656063\n", + "27 33.6 38.653993\n", + "28 47.0 41.601329\n", + "29 57.1 47.160289\n", + "30 22.1 15.896112\n", + "31 25.0 42.328283\n", + "32 34.2 27.555088\n", + "33 49.3 46.873734\n", + "34 55.1 48.514296\n", + "35 27.3 46.313934\n", + "36 22.9 28.890080\n", + "37 25.3 32.225209\n", + "38 47.7 46.930124\n", + "39 46.2 46.404920\n", + "40 15.9 16.628513\n", + "41 18.2 18.236063\n", + "42 34.7 34.247509\n", + "43 34.1 42.865920\n", + "44 53.9 48.680398\n", + "45 38.3 41.336871\n", + "46 42.0 48.451930\n", + "47 61.5 36.028046\n", + "48 13.4 15.523962\n", + "49 13.2 13.216184\n" + ], + "name": "stdout" + }, + { + "output_type": "display_data", + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD7CAYAAAB37B+tAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3de5gU1bnv8e8SMKggymgISGTYATSgQhxAo2BGQSAxERWIt20gj4R4TkCNSTYk2QnDPomSnBPvSdR4I4m6UYLCAxpRpDdGgxEU0YBcgoMiiDBcFVEG3/NHrR5qmr5PT0/XzO/zPP1M11pVb61eXf3W6uqqGmdmiIhIdB3W1A0QEZGGUSIXEYk4JXIRkYhTIhcRiTglchGRiFMiFxGJuNbFXNlxxx1n5eXlANTsraHsyLKk8+VT19LiRbntpR4vym1XX0QnXi7rWrZs2TYzOz5pEAAzK9qjoqLC4qYummqp5FPX0uIVc10tLV4x11Xq8Yq5rpYWL5d1AUstTW7VoRURkYhTIhcRiTglchGRiCvqj53J7N+/n40bN7Jv3766suEdhrNq1aqk86eqy2eZKMdLrGvbti1du3alTZs2SecVkearyRP5xo0bad++PeXl5TjnANi0ZxNd2ndJOn+qunyWiXK8cJ2ZUVNTw8aNG+nevXvSeUWk+cp4aMU5d5Jzbnnosds5d71zrqNz7hnn3Fr/99h8GrBv3z7KysrqkrjkzjlHWVlZvW81ItJyZEzkZrbazPqZWT+gAtgLPA5MARaaWU9goZ/Oi5J4w6kPRVquXH/sHAL8y8w2ACOBGb58BnBRIRtWbE888QTOOd58882089166618tPejvNfz4IMPMnHixLyXFxFJ5CyHfyzhnLsfeMXM7nTO7TSzY3y5A3bEpxOWmQBMACjrXFYx8eEgicWqY1SWVzK8w3C69ehWN/9Zv3y1Ia/nEC/+9Evs+XgP7T/TPml9vO6acdewZfMWzj7nbH740x+mXOaMU87gsQWPcWKXE9PGS1U+86GZrHhlBb/8zS8zLpNr3YZ1G3h619N1fZtMrnW3PruGfj02FSxevssUM14x11Xq8Yq5rpYWL5d1TTt32jIz6580CGR/ZSdwOLAN6OSndybU78gUI9mVnStXrqx3BVO3yfMK+jAze3f3uymvrHp397u2Z88e69Kli61evdp69eplZmZv73jbfvCDH1ifPn3s1FNPtdtvv91uu+02a9OmjZ3c+2SrrKw0M7OjjjqqLtZjjz1mY64YY2Zmc+fOtYEDB1q/fv1syJAhtnzdcjMze+CBB+x73/veIW1I175s6+J9WcirzLpNnlfyV8jpasbGi1fMdbW0eLmsiwxXduZy1spXCUbjW/z0FudcZzPb7JzrDLyfQ6ySMmfOHEaMGEGvXr0oKytj2bJlLFi8gOrqapYvX07r1q3Zvn07HTt25Oabb+ax+Y9xSvkpaWMOGjSIJUuW4Jzj3nvv5Xe3/o6777i7SK9IRFqSXBL55cAjoem5wFhguv87p4DtKqpHHnmE6667DoDLLruMRx55hFVrV3H9xOtp3Trooo4dO+YUc+PGjVx66aVs3ryZTz75hC6fT34KoYhIQ2WVyJ1zRwHnA98NFU8HHnXOXQ1sAL5Z+OY1vh3bd/Dcc8/x+uuv45zjwIEDOOc49UunZrV8+GyR8Ol/kyZN4oYbbuDCCy8kFovxk5/9pOBtFxGBLM9aMbMPzazMzHaFymrMbIiZ9TSzoWa2vfGa2Xjmz5nPVVddxYYNG6iuruadd96he/fu9D6lN3fffTe1tbUAbN8evLz27dvzwZ4P6pbv1KkTq1at4tNPP+Xxxx+vK9+1axcnnHACADNmzEBEpLG0+HutPDHrCS6++OJ6ZaNGjWLLli2ceOKJnHbaafTt25eHH34YgAkTJnDlJVdy7rnnAjB9+nS+/vWvc9ZZZ9G5c+e6GFVVVYwZM4aKigqOO+644r0gEWlxmvwS/UTV0y8o+CXw6cyaP+uQ5a699tq6eDfffHO9ukmTJjFq3Ki6ZUaPHs3o0aPrtQNg5MiRjBw58pDycePGMW7cuJzbKSKSSosfkYuIRJ0SuYhIxCmRi4hEnBK5iEjEKZGLiEScErmISMQpkQOtWrWiX79+nHLKKYwZM4a9e/fmHev6a65n1qxZAIwfP56VK1emnDcWi/Hiiy/mvI7y8nK2bduWdxtFpHkpufPIqepAurPBU9WlXKZqV6qaOkcccQTLly8H4Morr+Suu+7isu9cVldfW1tbd8+VXNx7771p62OxGO3ataP81PKcY4uIxGlEnmDw4MGsW7eOF59/kcGDB3PhhRfSu3dvDhw4wI9+9CMGDBjA0C8P5e67gzsZmhkTJ07kpJNOYujQodRsramLVVlZydKlSwFY9MwiTj/9dPr27cuQIUOorq7mrrvu4pZbbuH8s8/n+eefZ+vWrYwaNYoBAwYwYMAAXnjhBQBqamoYNmwYffr0Yfz48fHbBouIAKU4Im9CtbW1PPXUU4wYMQKAV155hTfeeIPu3btzzz330KFDB15++WXe2vYWY0aMYdiwYbz66qusXr2alStXsmXLFr7Y+4v+32gctHXrVn507Y944fkX6N69e90tca+55hratWvHFd+9gi7tu3DFFVfw/e9/n0GDBvH2228zfPhwFv5jIdOmTWPQoEH8/Oc/Z/78+dx3331N0DsiUqqUyIGPPvqIfv36AcGI/Oqrr2bus3MZOHBg3X+lX7BgAStWrGDWrFns/3Q/e/fsZe3atSxevJjLL7+cVq1a0aVLF84+5+xD4i9ZsoQzzzqzLlaqW+I+++yz9Y6p7969mw8/+JDFixcze/ZsAC644AKOPTav/3MtIs2UEjn1j5GHHXXUUXXPzYw77riD4cOH17uvy5NPPlmwdnz66acsWbKEtm3b1pXF79EiIpKKjpFnafjw4fz+979n//79AKxZs4YPP/yQc845h5kzZ3LgwAE2b97Mi88fehbKmWeeyZIXl/DWW28B9W+Ju2fPnrr5hg0bxh133FE3Hd+5nHPOOXV3X3zqqafYsWNH47xIEYkkJfIsjR8/nt69e3P66adz3hnn8d3vfpfa2louvvhievbsSe/evfnWt75FxYCKQ5Y9/vjj+fVtv+aSSy6hb9++XHrppQB84xvf4PHHH6/7sfP2229n6dKlnHbaafTu3Zu77roLgKlTp7J48WL69OnD7NmzOfHE5P/4WURaptI7tFK1q6i3sQX44IMPDik7a/BZjP7awdvTHnbYYdx4443ceOONh6zrzjvvTNqOWCxWV37esPP491H/Xm8dvXr1YsWKFfWWmTlz5iGvq6ysjAULFhzSRh12ERHQiFxEJPKUyEVEIq70Dq2IiDQD5VPmAzBuROOvK6sRuXPuGOfcLOfcm865Vc65LzvnOjrnnnHOrfV/8z65WVcqNpz6UKTlyvbQym3AX83sZKAvsAqYAiw0s57AQj+ds7Zt21JTU6NE1ABmRk1NTb3zz0Wk5ch4aMU51wE4BxgHYGafAJ8450YClX62GUAMmJxrA7p27crGjRvZunVrXdnOfTvZ1Tb5za5S1eWzTJTjJda1bduWrl27Jp1PRJq3bI6Rdwe2Ag845/oCy4DrgE5mttnP8x7QKZ8GtGnTpu7S9biqWBVVX6pKOn+qunyWiXK8THUi0nK4TIc0nHP9gSXA2Wb2knPuNmA3MMnMjgnNt8PMDjlO7pybgL+NVFnnsoqJD08EIFYdo7K8Muk686lrafGKta5bn11Dvx6bSrZ9jRGvmOsq9XjFXFdzi3frs2sA8vr8JJZPO3faMjPrnzQIBMdX0z2AzwHVoenBwHxgNdDZl3UGVmeKVVFRYXFTF021VPKpa2nxirWubpPnlXT7GiNeMddV6vGKua7mFq/b5Hl5f34Sy4Gllia3Zvyx08zeA95xzp3ki4YAK4G5wFhfNhaYkymWiIgUXrbnkU8CHnLOHQ6sB75NcMbLo865q4ENwDcbp4kiIpJOVonczJYDyY7PDClsc0REJFe6srNEFfOqMBGJNt1rRUQk4pTIRUQiTolcpInED5+JNJQSuYhIxCmRi4hEnBK5SJZ0KERKlRK5iEjEKZGLiEScLggSacF04Vnz0CxG5Dp2KSItWbNI5CIiLZkSuYhIxOkYuYgI0f69QCPyUhe7Cao6NHUrRKSEKZGLiEScErmISMQpkYuIRJwSuYhIxCmRi4hEnBK5ZNbEZ86UT5mvq3dF0sjqPHLnXDWwBzgA1JpZf+dcR2AmUA5UA980sx2N00wREUkllxH5uWbWz8z6++kpwEIz6wks9NONL3ZTUVYjIhIVDTm0MhKY4Z/PAC5qeHMaQBfO5C0yhy60ExdJyplZ5pmcewvYARhwt5nd45zbaWbH+HoH7IhPJyw7AZgAUNa5rGLiwxMBiFXHqCyvTLq+tHWx/0Nl5c/qld367Br6tZ5JJa2h8se5xUtRl3f7ChTv1mfXAOT1uvJeV49NSeuaum/r9UXCe59vG/JtR7I+KpV4+bYBkr/3hV5XqcfL1BdN2bfTzp22LHQ05FBmlvEBnOD/fhZ4DTgH2Jkwz45McSoqKixu6qKplkrauqmHH1LWbfK8oHzq0bnHS1GXd/sKFK/b5Hl5v66815Wirqn7tl5fFKgN+bajlOPl24Z829HUbS90vEx90ZR9Cyy1NLk1q0MrZvau//s+8DgwENjinOsM4P++n02srOgwiUjBRObQmeQtYyJ3zh3lnGsffw4MA94A5gJj/WxjgTmN1UiRvGlQIC1ANiPyTsDfnHOvAf8A5pvZX4HpwPnOubXAUD8tIhJtEfxRPWMiN7P1ZtbXP/qY2S99eY2ZDTGznmY21My257RmjZSahvpdktF20TBN3H+6slNEJOKUyKWoivmjW6OsSyPXvOlH18YTmUSujSCCInisUSSKIpPIG6TQoyiNyiRLGQcgLXFb0g6+4FpGIi8QfSuQSGuJO40WQolcRNIr1ghaO5q8KZFL8ekDKwn0TbdhlMhFRCJOiVxEJOKUyEVEIk6JXEQk4pTIRUQiTolcRCTilMhFRCJOiVxEJOKadSLXJfUiklQzuyitWSdyEZGWoHVTNyAsPnoe17aJGyINovexCGI3QWVVU7dCSoRG5CIiDVAKh2+VyEWKQffglkaUdSJ3zrVyzr3qnJvnp7s7515yzq1zzs10zh3eeM1snvRjrEiElPAPpLmMyK8DVoWmfwXcYmY9gB3A1YVsmIiIZCerRO6c6wpcANzrpx1wHjDLzzIDuKgxGigiIullOyK/FfgP4FM/XQbsNLNaP70ROKHAbWsyOtwhIlGS8fRD59zXgffNbJlzrjLXFTjnJgATAMo6l1EVqwIgRi1V7AM/DbCz9Zrs6kLl8bpc48XFqmPJ48VmUhW7CSp/3OB46dYT1G1KXVegddXVpeunFO3Ip//q1tXI8VItk++6MrWjqfuiri6Hvsj6c5WqHTn2e1btKGRfFPpzkOr1pmhfxvyT5HPVkLYnZWZpH8BNBCPuauA9YC/wELANaO3n+TLwdKZYFRUVFjd16uFmU4+2sG6T51m3yfMy1yXIapmEurp2LJra6PHSrafb5Hnp6wq0rrq6dK8rh75It6583qt846VaJt91ZWpHU/dFXV0OfZH15ypVO3Ls96zakaSukO9VxmVyzDGplokvl+vnKte2A0stTW7NeGjFzH5sZl3NrBy4DHjOzK4EFgGj/WxjgTnZ7z6kIerOdtEpbSJCw84jnwzc4JxbR3DM/L7CNEmkcem0T2luckrkZhYzs6/75+vNbKCZ9TCzMWb2ceM0UYpGI/yGKeHzjKV505WdIiIZlPo3uKLfNEs3VBIRKSyNyEVEIq6kbmMr0VH3zWpEEzdERDQiFxGJOiVyEZGIUyIXEYk4JXIRkcZUhOsLlMhFRCJOiVxEJOKUyEVEIk6JXEqL7lciBVDql9QXmhJ5C6E7/jV/en8TJBkUNNfPgRK5tFwa/UszoUv0RRpIN4KTpqYRuTSMRrUiTU6JvFiU8ESkkSiRi4hEXItN5M3112uRkqBvoEXVYhO5iEhzoUQuIhJxGRO5c66tc+4fzrnXnHP/dM5N8+XdnXMvOefWOedmOucOb/zmiohIomxG5B8D55lZX6AfMMI5dybwK+AWM+sB7ACubrxmNqLYTU3dAhGRBsmYyC3wgZ9s4x8GnAfM8uUzgIsapYUimeiHNWnhsrqy0znXClgG9AB+C/wL2GlmtX6WjcAJKZadAEwAKOtcRm3rhwCIUUsV+yBWVTfvztZrsqsLlcfrmjpeXKw6dki8jG2o3pS0Dfm0PVUbSr1v69qez+tNEa/Q72PR+yLFtpQ2XkO2pXTvST7tK1ZfNPF71Sj5Is3rTcrMsn4AxwCLgEHAulD554E3Mi1fUVFh3SbPs26T59nUqYebTT3awrKuS1AK8eKmLpqavLwh7cuh7anaUOp9W9f2fF5vingNeh9LoS9SbEtp29eQ9z6HvsiqfcXqixSvuVjvVaPki4RyYKmlya053WvFzHY65xYBXwaOcc61tmBU3hV4N5dYImG6X4lI/rI5a+V459wx/vkRwPnAKoKR+Wg/21hgTmM1UkREUsvmrJXOwCLn3ArgZeAZM5sHTAZucM6tA8qA+xqvmSJScvQjc8nIeGjFzFYAX0pSvh4Y2BiNEhGR7OnKThGRiFMiFxGJOCXy5kjHLluuAr73ukNodOhfvYmE6DRIiSKNyEVEIk6JXEQk4pTIRUQiTolcRCTi9GOniBSMfixuGkrkIhGiRCnJ6NCKRJrOcxZRIhcRiTwlchGRiFMib2Z0qEGk5VEiFxGJOCVyEcmZvvmVFiVyEZGIUyIXEYk4JfIi0NdQyYXuAy65UiIXEYk4JXIRkYjLmMidc593zi1yzq10zv3TOXedL+/onHvGObfW/z228ZsrIiKJshmR1wI/MLPewJnA95xzvYEpwEIz6wks9NMiIlJkGRO5mW02s1f88z3AKuAEYCQww882A7iosRopIiKp5XQbW+dcOfAl4CWgk5lt9lXvAZ1SLDMBmABQ1rmM2tYPARCjlir2Qayqbt6drddkVxcqj9c1dby4WHWs8O2LWF+UQrwot71umepNSZdptPaVYF/EJftcpasr9e0s0+tK93qTcWaW3YzOtQP+B/ilmc12zu00s2NC9TvMLO1x8v79+9u2odMAGNf2EqpoC1W76uoP3ms5Q13Vx/Xilk+Z3+Tx4qpiVVRVVhW2fRHri1KIF+W21y0z4uWk21Kjta8E+yIu2ecqXV2pb2eZXldiuXNumZn1T9oBZHnWinOuDfAX4CEzm+2LtzjnOvv6zsD72cQSEZHCyuasFQfcB6wys5tDVXOBsf75WGBO4ZsnIiKZZHOM/GzgKuB159xyX/YTYDrwqHPuamAD8M3GaWI0HPw63MQNEZEWJ2MiN7O/AS5F9ZDCNkdEJDsHj0HfBCmOn7cUurJTRCTilMhFRCJOiVxEJOKUyEVEIk6JXERKX+wmqOrQ1K0oWUrkIiIRp0QuUqo0CtV/S8qSErmISFOJ3VSQMErkIiIRp0QuIhJxSuQiIhGnRF5oBTrmJSKSLSVyEZGIy+Y2tiIiUkAH79xYmHgakYuIlJDyKfNzvoZAiVxEJOKUyEVEIk6JXEQk4pTIRUQiTolcRCTiMiZy59z9zrn3nXNvhMo6Oueecc6t9X+PbdxmiohIKtmMyB8ERiSUTQEWmllPYKGfFhGRJpAxkZvZYmB7QvFIYIZ/PgO4qMDtEhGRLOV7jLyTmW32z98DOhWoPSIikqMGX6JvZuacs1T1zrkJwASAss5l1LZ+CIAYtVSxD2JVdfPubL0mu7pQebyuWceLctubKF6U267tLDrxir2uVJxZyhx8cCbnyoF5ZnaKn14NVJrZZudcZyBmZidlitO/f3/bNnQaAOPaXkIVbaFqV139wfsPZKir+rhe3PIp85t3vCi3vYniRbnt2s6iE69Y63LOLTOz/qSQ76GVucBY/3wsMCfPOCIi0kDZnH74CPB34CTn3Ebn3NXAdOB859xaYKifFhGRJpDxGLmZXZ6iakiB2yIiInnQlZ0iIhGnRC4iEnFK5CIiEadELiIScUrkIiIRp0QuIhJxSuQiIhGnRC4iEnFK5CIiEadELiIScUrkIiIRp0QuIhJxSuQiIhGnRC4iEnFK5CIiEadELiIScUrkIiIRp0QuIhJxSuQiIhGnRC4iEnFK5CIiEdegRO6cG+GcW+2cW+ecm1KoRomISPbyTuTOuVbAb4GvAr2By51zvQvVMBERyU5DRuQDgXVmtt7MPgH+GxhZmGaJiEi2nJnlt6Bzo4ERZjbeT18FnGFmExPmmwBM8JMnAav98+OAbSnC51PX0uIVc10tLV4x11Xq8Yq5rpYWL5d1dTOz41PEADPL6wGMBu4NTV8F3JnD8ksLWdfS4kW57aUeL8ptV19EJ16+60r2aMihlXeBz4emu/oyEREpooYk8peBns657s65w4HLgLmFaZaIiGSrdb4Lmlmtc24i8DTQCrjfzP6ZQ4h7ClzX0uIVc10tLV4x11Xq8Yq5rpYWL991HSLvHztFRKQ06MpOEZGIUyIXEYk4JXIRkYjL+8fOXDnnTia48vMEX/QuMNfMVmVY5gTgJTP7IFQ+AtgOmJm97G8NMAJ408yeTIjxRzP7VpLYgwiuTn0D2AWsMrPdzrkjgCnA6cBKoAZ4xMzeSVg+fqbOJjN71jl3BXAWsIrgh4rPA5f4vweANcDDZrY7c29JMTjnPmtm7+e4TJmZ1TRWm0TyUZQfO51zk4HLCS7j3+iLuxIkwv82s+lJlrkW+DHwEtAPuM7M5vi6TcDbBDuiZ4AzgEXA9cD7wNp4GOBc4DlgsJkd65f/DvA94HFgGNADOMGfiXMPsBeYBQwB/hPYCvwLeAR4zMy2Ouce8us/EtgJtANm+2W+SLBzWAx8DXjVz3Mx8L/NLJZfTzZcKSUv51wHgvf4IuCzgBG8f3OA6Wa2M8kyRxP054vAU2b2cKjuAWAf8Cnwc2ASMIpg5zoV2BIOBSwDfgg8Z2bbfXtuBgYQ7OC3Af9lZtucc/2BR33sNsDHwB8JdvL/CrWhP/B/CQYqPwbuJxgwrAGuIxhwjCLY/j8h2K7uAv4MXE2wjXTx4d71fXGfme1P0hd/AJb6WH81sxdCddOAD32f3kHwWbsEeNO/pg8SYq0BRpvZCj/dBpjMwcHOVuBPvi96+Nd1GsGV2nuAB4EnEgZc/0bw+dkETAduAb7s34//IPhsxvsiPtj5A8HnMet+KGRfOOfWmFkv59xpefTFp8CvCS6MTOzffPrirqxzRS5XD+X78I1qk6T8cGBtimVeB97xz8v9m3Sdn/6I4JTHI4HdwNG+/FVgB1AJfMX/3eyfrw3Ffhk43j8/CtgXqnsloR17CQ5BDQPu82/iX4F3gPYEyXwL0MrP7+Lt89NHAjH//ETgNf9GvknwraLGv5nTgWPS9OEC4CbgT8AVofLPEWxcvwXKgCrfd48S7FA6hh5lQDVwLMGHNh6jg39tK4B/Al/05f2B9cA6YIN/H/8T+EKS9vUn2Jn+meBbyDMEO7OXCb6p/JePvcv34RLfzsnA5xJez2Tg7wTfihIfC4EPCJL/XOAvwGf8srsIkvcU/1om+7ZMIvgQv5Xw2E+QkNf75e8FfgF0A74P7Aq1axEwwD/v5Zf7fwQDin/4+bv4518lGLi8E+9ngh38dmAcwYf1BuBnQE9ghu+b3wNn+vqu/vn9BAOOjhz6Xn4IPEwwgFkG3Bxq7w7gN8DvfJ/dCQwm2MnsJ/jc7CZIwnsIkscBYLdf/jcEyfkrBElnZyj2fOBi/7zS98Us//oeJUjChxMMZP6Xfz/eAH7g34+rCRJaFTAIuJVg+zgfeA94Pkk//D5FPzSkLz7xj8R+2AMcCC2fbV9sJdj51+sHX5dPXzwLTMoqxxYpkb9JcK+AxPJuBCOoFUke+4CPQ/O2I0igNwN7Q+Wvhp4fRrD3fgbo58viH9LXCBJYGQmXv/o3+tv++QNA/9AH9sOEedsAFxKMsLf6mHuAjr6+rW97PLkcG16fnzfX5FVBsHOYTkIC4+BOJZfk9VZC34YT2CaCkRVkmbx8fT4JbBdwY4ptxgi+SS1KeOwBPgrN91PgBf++hreLtxPivev76tRQ2VuEdtzA8oRl9gGt/fMlCXXhNgwmSBLv+fZNSNGGjxKmXw5tt5+k6IcDBMkm/B6u938/Dc3XmuCQ3my/Xez15c63y4WmtxF8m+iU0Bfhz9Jy/ODLL7Mvsd2Jrws4muBWHU8SfDZqgGFZ9sUS/3ctwWHOVNtE/LUXoi9uJ/js1+uHJHklq74gGEiuSNIPDxDcYDDVtpmqLz6Tqi8O6ZtsZmrog+Dr5DrgKd/B9xB8qNb5juxHkETCjxeB9xPitPYboAFHxj8EofoOwCsECeMxgj3v276uOvSmrwc6+/J2vvMfJPia+xJBslsP/E+ajer7PtYG4FqCPf0fCEaZT/qYfyDYicV3EscTSjYpNtRkyWtReEP188YT2Ap8MkqygSRNXv5v0gRG8O1geXiDSrbBUT95LQqvO4sNNZ7AFhAcSgl/kDoR7Iw+AHom6aNV+G9qobJxBCPaT0Jlv0iY5/XQdnEzwbep9QSH+m4gGCGtx3/IQ/23ADiPYMR0G8GobBpQk6RtrXz7ngbG+G3jIl/3Ff+aBvnpC4Gnw33klwlvz4cRfKN8NXFdvn5/krKpfrsI76jvT5jnNYLBwXME2+5h/rWvJzjkMIqE7Z7gW+eDwL8BPyEY+XYDvk3om0to/jL/+v9OcEhiGwcHSD0Ivul+wU+fDiyOb3MEA4HEfriUYMd6YoH7Yk1iP/jy9QQj6lz6ohqYl6QfriEY9fciOGyXVV/46ZWp8kW99WQzUyEevpPO9B0zyj9vRfCVflCS+bsCs1PEqkxRfhz1k9YFpBjxheY5Eujunx8N9PUbeSdf1ivNsl04OCI9huBGYgP9dB8/fXLCMgsIjollnbziG2p44/Zl4/zGvcFPZ5W8fF3SBEYwkt9DbslrBEFCHkYOCYzg20oNwc5uB8HIfRXwK3cAGKUAAAORSURBVP/aTkqyvl8DP09SPsLHapekrgcwKzR9IUHCeI/gwx5+xA+5fY5g0FAJzCQYbcV30hOAmSnep74Eifwp4GTffzsJdjTfIvjmsgP4W/z1Eezgq/x63idILmv882XAV1Os62WCO5Amlo8nOF6brC++APwt9Jm8luBQxiaCkWP40SnUFwv9e/ISQSLaQ3AywI3ACynaN4TgGPoqgsMGfyEYcb9PcIjubT/9FsGdUyFIZKsIRrJrQvPPJPjdo2+h+yKxH3zdgxn64ttJ+uJtoEMeffEzv+w63xdnhraLX6fLX3Xxs5lJj8I9CJLXrzh4jDycvMaSJHn55WYDQ5OUP0Toa1uoPGXy8tPpEtjT5JC8/HLpEthV1E9gvUIb6i+AoYkfNILEfLL/ACTWjU9R/tU0y9SLBxwBnJIhXro2pKv7Tpplvpjm9Z5BMHotA84m+CH2a74sfoirN8EO+Gt+Ote6CwgOD4TrBhMkya/5NmQTrw/BICBtOxLi9Ym/Jj/95VTr8mVl/vHnNNvdHxtSR/1vYJ1JMljJMt6f8lhmHn5w5t+T47JZLtlDl+iXEOfct83sgULVJZb7Uyu/YGZvFCJeAdqX7sykdwi+cq4K1znnJhH8YPfXbJcpcrxrCb41pIr3IcFOPNOZWAOBGMGPYYcRjNzCZ2idT3DsvB2Hnr2Va12qdcXL0y2Tbl35xEs88wyCb4fP+Xn+ESoPn5WWT138zJJwXap1ZVOXGC9dG7JdF2Z2IZnkkvX1aNwHCceWG1oXgXiZzkxql1jnl3ktl2UiFC/ZmVhvEPwOklh+RJpl8q0r9LryiZf2zDOCs6IKVbeG4FBnrvHWpKhbm2e8dHVfySp3NHXyamkPkp+hs4IgAXyaR91HKepKJV66unRnJiX+QBqv20b9H2ezWSYK8VKdifUqB398fjVh+aTL5FtX6HXlE4/0Z54dRnCSQUHqSj1eTnmlKZNaS3wQ/OKd7CydcoJTzXKt20rwVbRU46Wry3RmUr8kdZsJneOb5TJRiZfsTKyl+ETHoWdofZhimXzrCr2ufOMlPfMsNE9B60o9XlZ5pakTW0t7kOIsHV/3r1zrfLxnSjVehnWlOzPpIkLn2ics841clolIvMoU5V0InYkVKj8OOD3FMvnWFXpd+cbL6syzQteVerx0D/3YKSIScbr7oYhIxCmRi4hEnBK5iEjEKZGLiEScErmISMT9fx6kG5/WidSRAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "tags": [], + "needs_background": "light" + } + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "91uqmsQ6l210", + "colab_type": "code", + "outputId": "53e48100-7327-4ce3-f539-a9eb6f351615", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 282 + } + }, + "source": [ + "epsilon_hat = y - y_hat # residuals\n", + "plt.grid(color = 'b', linestyle= '--' , linewidth = 0.5)\n", + "plt.scatter(y_hat, epsilon_hat) # scatter plot" + ], + "execution_count": 75, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 75 + }, + { + "output_type": "display_data", + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD4CAYAAAAJmJb0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO2df3RU13Xvv1ujQYwkBwkZgxDIwo7BsaOAihJDyXvFdovzGkhUk5iA22XavKW+1TZNHJcWmqxnua/+kUUTJyttVstK+vB6NRjbwrKF04f8YtTEDriRIhQF24AdY8xINlgwNoKRGEnn/XHnSldX9+fMvXNGs89nLS1J8+Oe/Z1zZ99z99lnHxJCQKFQKBSFSZFsAxQKhUIRHsrJKxQKRQGjnLxCoVAUMMrJKxQKRQGjnLxCoVAUMMWyDTBy9dVXi7q6ulCOffEicNVVoRw671HaZVshD876OWnv7u5+Xwgxz+q5vHLydXV16OrqCuXY/f3AwoWhHDrvUdplWyEPzvo5aSeit+2eYxOu2bVLtgXyUNr5wlk/Z+1G2Dj5qirZFshDaecLZ/2ctRth4+TXrpVtgTyUdr5w1s9ZuxE2Tr61VbYF8lDa+cJZP2ftRtg4ec5XdaWdL5z1c9ZuJK+ya8Kkv1+2BfJQ2mcGbT1x7Dx4HP2JJBZWxLDtjmVoaqjJ6pgzSX/QcNZuJJCRPBHdS0THiOjXRLSXiGYT0RIieoWI3iCifUQ0K4i2MuXECZmty0Vpz3/aeuLYsb8P8UQSAkA8kcSO/X1o64lnddyZoj8MOGs3krWTJ6IaAH8JoFEI8XEAEQBfAvAtAI8KIT4K4AKAL2fbVjY0N8tsXS5Ke/6z8+BxJFNjUx5Lpsaw8+DxrI47U/SHAWftRoKKyRcDiBFRMYBSAAMAbgPwdPr5xwA0BdRWRnDOmVXa85/+RNLX416ZKfrDgLN2I1nH5IUQcSL6BwCnASQBdADoBpAQQoymX3YGgGVwkYiaATQDwPz5tWhpATZuBDo7gcFB7Wq8axdQXw+UlwOHDwObNwMHDgAjI8CWLcDu3cDKldrxuruBrVuBPXuAkhJg/Xpg717g/Hlttr2vb/KYVVXa5Exrq/a7v1+7xdOfr64GGhuB9nZg3TrtuVOnJp+vqwOWLgU6OoANG4CuLmBgYPL5pUu1FXedneFoWr0aGBpy19TVpf0uJE1e+6mrS2s33zVFT9Xh7JkoypefxlBvLSLlI5i1IIHiM4vw859n3k9dXVq7+d5PYZx7v/418NBDhaXJrp+coGx3hiKiSgCtADYBSAB4CtoIviUdqgERLQbw7+lwji2NjY0irLIG3d3ePpBCRGmXbYU7ekzeGLKJRSN4+M76rCZfZ4r+MOCknYi6hRCNVs8FEa75XQBvCSHOCSFSAPYDWAOgIh2+AYBFALKbQcqS9naZrctFac9/mhpq8PCd9aipiIEA1FTEsnbwwMzRHwactRsJIoXyNIBVRFQKLVxzO4AuAIcAfAHAEwDuAfBsAG1lzLp1MluXi9I+M2hqqMnaqZuZSfqDhrN2I1mP5IUQr0ALz/wSQF/6mLsA/A2ArxPRGwCqAPwo27aygXM6ldLOF876OWs3EshiKCHE/QDuNz38GwCfCuL4QXDqlGwL5KG084Wzfs7ajWQ98RokYU68cqotbUZpl22FPDjr56Q97InXGQHnnFmlnS+c9XPWboSNkw9pV8EZgdLOF876OWs3wsbJL10q2wJ5KO184ayfs3YjbJx8R4dsC+ShtPOFs37O2o2wcfIbNsi2QB5KO1846+es3QgbJx9S0s6MQGnnC2f9nLUbYePkBwZkWyAPpZ0vnPVz1m5E5ckzQGmXbYU8OOvnpF3lyYN3zqzSzhfO+jlrN8LGyXNOp1La+cJZP2ftRtg4eS63bVYo7XzhrJ+zdiNsnHxnp2wL5KG084Wzfs7ajbBx8hs3yrZAHko7Xzjr56zdCBsnz/mqrrTzhbN+ztqNsHHyg4OyLZCH0s4Xzvo5azei8uQZoLTLtkIenPVz0q7y5ME7Z1Zp5wtn/Zy1G2Hj5OvrZVsgD6WdL5z1c9ZuhI2TLy+XbYE8lHa+cNbPWbsRNk7+8GHZFshDaecLZ/2ctRth4+Q3b5ZtgTyUdr5w1s9ZuxE2Tv7AAdkWyENp5wtn/Zy1G2Hj5EdGZFsgD6WdL5z1c9ZuhI2T37JFtgXyUNr5wlk/Z+1GAnHyRFRBRE8T0etE9BoRrSaiuUT0AhGdTP+uDKKtTNm9W2brclHa+cJZP2ftRoIayX8PwP8VQtwIYDmA1wBsB/ATIcQNAH6S/l8aK1fKbF0uSjtfOOvnrN1I1k6eiOYA+K8AfgQAQogrQogEgM8DeCz9sscANGXblkKhUCj8EcRIfgmAcwD+NxH1ENEPiagMwHwhhL6V7rsA5gfQVsZ0d8tsXS5KO1846+es3UhxQMf4LQBfEUK8QkTfgyk0I4QQRGRZCY2ImgE0A8D8+bVoadHqQHd2alXkmpu1GhT19doKtsOHtfzXAwe02fMtW7TYm35r1t0NbN0K7NkDlJQA69cDe/cCS5YAra1AX9/kMauqgLVrtcfXrtUKGp04Mfl8dTXQ2Ai0twPr1mnPnTo1+XxdnbbFWEcHsGED0NWl7RCvP790qVYgqbMzHE2rVwNDQ+6aEgntdyFp8tpPiYTWbiFp8tNPiYTWbiFp8tpPqRTw0EOFpcmunxwRQmT1A2ABgFOG//8LgOcBHAdQnX6sGsBxt2OtXLlShMWDD4Z26LxHaecLZ/2ctAPoEjZ+NetwjRDiXQDvENGy9EO3A3gVwHMA7kk/dg+AZ7NtKxtKSmS2LhelnS+c9XPWbiSIcA0AfAXA40Q0C8BvAPwxtHj/k0T0ZQBvA7groLYyYv16ma3LRWnnC2f9nLUbCSSFUghxVAjRKIT4hBCiSQhxQQgxKIS4XQhxgxDid4UQ54NoK1P27pXZulyUdr5w1s9ZuxE2K15Xr5ZtgTyUdr5w1s9ZuxE2Tn5oSLYF8lDa+cJZP2ftRtg4+b4+2RbIQ2nnC2f9nLUbURt5M0Bpl22FPDjr56RdbeQN3pv6Ku184ayfs3YjbJx8VZVsC+ShtPOFs37O2o2wcfJr18q2QB5KO1846+es3QgbJ9/aKtsCeSjtfOGsn7N2I2ycPOerutLOF876OWs3wsbJ9/fLtkAeSjtfOOvnrN0IGyd/4oRsC+ShtPOFs37O2o2oPHkGKO2yrZAHZ/2ctKs8efDOmVXa+cJZP2ftRtg4+epq2RbIQ2nnC2f9nLUbYePkGy1vZHigtPOFs37O2o2wcfLt7bItkIfSzhfO+jlrN8LGya9bJ9sCeSjtfOGsn7N2I2ycPOd0KqWdL5z1c9ZuhI2TP3VKtgXyUNr5wlk/Z+1GVJ48A5R22VbIg7N+TtpVnjx458wq7XzhrJ+zdiNsnHxdnWwL5KG084Wzfs7ajbBx8kuXyrZAHko7Xzjr56zdCBsn39Eh2wJ5KO184ayfs3YjbJz8hg2yLZCH0s4Xzvo5azfCxsmHlLQzI1Da+cJZP2ftRgJz8kQUIaIeIjqQ/n8JEb1CRG8Q0T4imhVUW5kwMCCzdbko7XzhrJ+zdiNBjuS/CuA1w//fAvCoEOKjAC4A+HKAbfmmuVlm63JR2vnCWT9n7UYCcfJEtAjAZwH8MP0/AbgNwNPplzwGoCmItjKFc86s0s4Xzvo5azdSHNBxvgvgrwFclf6/CkBCCDGa/v8MgBqrNxJRM4BmAJg/vxYtLcDGjUBnJzA4qF2Nd+0C6uuB8nLg8GFg82bgwAFgZATYsgXYvRtYuVI7Xnc3sHUrsGcPUFICrF8P7N0LDA9ru7f39U0es6pK2+y3tVX73d+v1bvQn6+u1sqVtrdrxY5OnNCWSuvP19VpaVodHdokT1eXdouoP790qbbirrMzHE2rVwNDQ+6a+vq034WkyWs/9fVp7RaSJj/91Nen/RSSJq/9dPIk8NBDhaXJrp+cyLqsARGtB/D7Qog/I6K1AP4KwFYAR9KhGhDRYgD/LoT4uNOxwixr0NnJd/d2pV22FfLgrJ+T9rDLGqwB8DkiOgXgCWhhmu8BqCAi/U5hEYB4AG1lTGenzNblorTzhbN+ztqNZO3khRA7hBCLhBB1AL4E4EUhxN0ADgH4Qvpl9wB4Ntu2smHjRpmty0Vp5wtn/Zy1GwkzT/5vAHydiN6AFqP/UYhtucL5qq6084Wzfs7ajQQ18QoAEEJ0AuhM//0bAJ8K8vjZMDgo2wJ5KO184ayfs3Yjqp48A5R22VbIg7N+TtpVPXnwzplV2vnCWT9n7UbYOPn6etkWyENp5wtn/Zy1Gwk0Jp/PlJfLtkAeSjtfcqm/rSeOnQePoz+RxMKKGLbdsQxNDZZrIHMC977XYTOSP3xYtgXyUNr5kiv9bT1x7Njfh3giCQEgnkhix/4+tPXE0dYTx5pHXsSS7c9jzSMvoq0nN0tmuPe9DpuR/ObNsi2Qh9LOl1zp33nwOJKpsSmPJVNjeKD9GIZT4xPP6c4fQOijfO59r8NmJH/ggGwL5KG08yVX+vsTScvHL1xOWTr/nQePh24T977XYePkR0ZkWyAPpZ0vudK/sCLm6/V2F4Ug4d73Omyc/JYtsi2Qh9LOl1zp33bHMsSikSmPxaIRVMSilq/3e1HIBO59r8PGye/eLdsCeSjtfMmV/qaGGjx8Zz1qKmIgADUVMTx8Zz1aPnezpfPfdsey0G3i3vc6bCZevdRdLlSUdr7kUn9TQ43tZKqM1Erufa/DxskrFAo5ODl/RfiwCdd0d8u2wJkwc4nzXXuYcNFud/5w0W8FZ+1G2Izkt26VbYE9+kKSsHKJ81l72MjUnqsVoE7nz9atfEfQnM97I2xG8nv2yLbAHruFJEHlEuez9rCRpd1pBWjQOJ0/qu8VbJx8SYlsC+yxyxkOKpc4n7WHjSztYV+4jTidP6rvFWyc/Pr1si2wxy5nOKhc4nzWHjYytLf1xBEP+cJtxOn8UX2vYOPk9+6VbYE9dgtJgsolzmftYZNr7XqYxo6K0mjgE+xO54/qewWbidfVq2VbYI8+GRfWJF0+aw+bXGu3CtPoRCOEoeFRXLicAhDcBLvT+RM7m/FhZzycz3sjbJz80JBsC5wJM5c437WHSa61O4VjymYVI5FMTXlMj9Nn2/d254/qewWbcE2f/R10waO05w67+HhNRQwfmBy8TpjFulTfK9g4+eZm2RbIQ2nPHU7x8bAn2M209cTx/JWXcr5ZR77A+bw3wsbJc97UV2nPHXaFupoaakKfYDeiTwCf/Ok1oefp5yucz3sjbGLyVVWyLZCH0p5b7OLjYU+wG9EngItikyGioOL/MwXO570RNk5+7VrZFshDac8fclWsS4/zz64dtHy8EDGXkbjrupsBzM/4/bI3Ig8KNuGa1lbZFshDaeeHHue/fHyB5eOFhlUZif/1jxc9h6dyWYYi12Tt5IloMREdIqJXiegYEX01/fhcInqBiE6mf1dmb27m5NuILpco7XIJs8KoHXr83ziSz9VmHTKwWp9QvOic5zISuSxDkWuCGMmPArhPCHETgFUA/pyIbgKwHcBPhBA3APhJ+n9p9PfLbF0uSrs82nriuO+p3ikjxPue6g3d0esTwB8Z/8i0CeBCxCoMNTY023N4Kuz6UTLJOiYvhBgAMJD++yIRvQagBsDnAaxNv+wxAJ0A/ibb9jLlxAlZLctHaZfHN57pw9i4mPLY2LjAN57pC93hNjXU4OiNNWhpuTnUdvKBhRWxafWCUufLPIenrN6vPz7TCXTilYjqADQAeAXA/PQFAADehc0MCBE1A2gGgPnza9HSAmzcCHR2AoODWq7rrl1AfT1QXg4cPgxs3gwcOKDtxr5li7aXo77VV3e3Vkd6zx6tCt369VoNi6VLtfhsX9/kMauqtNv51lbtd3+/5hT056urgcZGoL0dWLdOe+7Uqcnn6+q043Z0ABs2AF1dwMDA5PNLlwILF2pawtC0erW2qs9N08WL2u9C0uS1ny5e1NqVpSn+4nUoWXgB46lipM5dhfLlpzHUW4sPYyn09YV/7l28qH2W+d5P2Z57i99qxKnBNzFadAUj/ZUo+1g/iouKsOTMSpw6Za3pm//wAV566yxGF8YxevJaFFUPIjUcmein4b463LT66pz0U7bnnhMkhHB/lQeIqBzAfwB4UAixn4gSQogKw/MXhBCOcfnGxkbR1dUViD1mWlq0H44o7fLar9v+vO1zpx75bOjty9afS8zZMYvfasQT//IR29caN1rRKSJgXGjhrZmUXUNE3UKIRqvnAhnJE1EUQCuAx4UQ+9MPv0dE1UKIASKqBiC1VFJ1tczW5aK0y8Ep7k45siGX+mWnIJrTU//lX+xfa1dIblxMTlDPFAfvRhDZNQTgRwBeE0J8x/DUcwDuSf99D4Bns20rGxotr3E8UNpzj1vJ4btX1ebEjlzpz8cURCftThOqhZJVoxNEds0aAH8E4DYiOpr++X0AjwD4PSI6CeB30/9Lo71dZutyUdpzj1PJ4T9cVYu/b6rPiR250p+PKYhO2t0mVAshq0YniOyal2B/93l7tscPinXrZFsgD6U999g5CQJy5uCB3OnPxxREJ+3b7lhmGZPXKYSsGh02K15lp9LJRGnPPbmuOGlHrvTni14jTtr1dQQVsei05wpt0RgbJ3/qlLy2Zax4NCJTu2xkac9lxUkncqU/X/QacdPe1FCDo/evw3c3rbCsGlooBJZCGQRhplD292v5qLnGKlUrFo3k9ESSpT0XuGV0yNTuZlsuslFyoV/XEU8kESHCmBB5kYJYyOe9GacUSjYjeVm1pfNhQqpQ62p7yeiQqb2poQYvb78Nbz3yWby8/bZpDj6IbBS3u8Sw9Rt1AMCYEHmTglio571f2Dj5ujo57ebDhJQs7WHj5QKar9qDuPh7uVD40Z9JWPGB9mPSBzF26Nplh0tlw6ae/NKlctrNpCZG0LfxsrSHjd2FMp5IYsn257GwIoYNC+oBzMutYR7I5uJvDI+YMW8M4rXvzWFF/YIBwPbca+uJ48LlcPatDeI7sHRpZroKDTYj+Y4OOe36nZAKY1GJLO1h43Sh1D+7f/jXD/Jy5JZpNoo5PGKF0cF67ftM7iycnssmqyao70BHR36ES2XDxslv2CCnXac9P60I46SUpT1srC6gZqLXDeTlFzrTbBSnRVY6Rgfrte+d7orsnKvThcZpEOMWOsnmO2A8/jODv7C10UpvoYZ12IRrurq8VWwLAz9bvoURw5epXSeoEJT5OBtX1uD5Xw3Yhg2uvFuB/gWnszU/KxuttLrt92p3DLfzwHyhcOp7YxtF6awYK+zCGxGb9xBZh0K8hk4y/Q6Yj3/m5Gx8ZIV2V2fGfKdRyGEdNk5+YMD9NflAGHWtZWv3+gXyknJoPs6+X7xj/S1OMzZUkpMFOcY4OWHSJCdnYXfxd/q87M4PwLpyol3fm9uwc/CA/Qbgdu+xO5TTCL2poWbiM7SzxKofnS5UY0MlEMCU/gCs75jcbLNCdkE2r7AJ1zQ3y7bAG2EsKpGt3cvtt5c4rNVxUmMCqXF7B3V1Yzz0BTnmOLnZGr/hNqfPy+r8iBYRKkuj6E8ksfPg8SmfmV3fewn7GLG6sNTYXDwrS6evIgWcR+hucw2xaAS33jhvSjjlm219U84Z80WnfLl2ByfStjqFS/3ePeRjQTY72Dj5mZIz6zeG7wXZ2r18gbxcCPyGrGoqYmhMrrQdLTvFX/3EZ704TKf4tRmnz8t8flTEogABFy6nLJ2NXd/7/SwJ00snb7tjGaKR6WWrhoZHLT9PuwpXFaVRx88wQoTfqp2D1u74FKf6+JHTjp/7UK9W6bOmIma7XkHHz0R4W08c9z3ZO2MmdNk4+ZmURui0iCYTZGv38gXyciHwE3bRv9if+53pm0a4jcL8jtK8OEwrJ2mH2+dlPD/KSoqRGps6gjU6G7u+9xvCEpieTdPUUIOyWdMjvqlxYXmXZhfGEcL5MxwTAj9/8/w0p+q2Vj8695Lnu2Cvd9C6FrtQVT5Wr2Tj5Gf68uZsZv5la/fyBfJyIbAMVUQI0aKpQ0Tjsa20u901+M3u8OIwBYCW54556kO7rKF4Iom67c9jxQMdE+91uzja9b2XzCS7Yxr5IOmcJ2836jUfw+0z9FN8JUIEAjB/gfB8F+z1Dtrtri0fq1eymXjt7NT2aJyJWE3E3bvvKLrePu+pbK1s7W6ZJIB16ddohHBpZHRiYdO2O5bh4Tvrpx3H6dhW2t0co9vzxgm3itIohj3GthPJFBJpp+g2IatrsgrzJJIpbHuqF4D7RL1T35cUF/mKy1s5MKf23Ua9xte6lf61w2pSVXfOLS1AU4P3Y3nJgnMaqcsuyGYHGye/caNsCyYxz8rfeuM8HHr9nK0DtBo9CACPHzmNxmvnup6Y+aDd7QtkvhBUlEYxNDw6zSk+fGc9Xt5+m+37zVhpd3OMXhyX3h92qZteSKbG8LV9RycmVI0a9M9rzSMvWtqih0SsnKPR2Vjpt9vfNBYtwnBqHHNiUVy6MjolDGTnwJza9zJXYa5zY3dhs6O4CCifHUXicmradyeM897u3IgQ5W31Sjbhms5O2RZoWMV7/+3Iacf4r93owSpOakW+aHfDGGsunVU8LWsmk4ktK+1u4SOn5/1mpXhBvzOrswjjOI0crSZizWGGzs7pob6/3f8rSw3J1DjuXlWLo/evw84vLPc0+e/Uvlt8mgjYuHLy4t/UUINtdyyzzc6xIjUODKfG8eimFdPmrzI5793ColaTzdEI4dt3Lc9LBw8wGskPDsq2QMOLkzDn5zrlRnuZ6MkX7X4IalGYlXa38JHT8/fuO+qrfa/Y5dU79b1xItbOwbz86w/Re3ZqqM8J492hV6dl91on2wFtwrW1Oz7Rnt0dhhtWufb9iSRGu27C4k/P8qzD84Ioc/Qpy2rtYefbq3ryAeKls5Zsf97TOUEA3nrksxPHvXffUcv36VkkTszEutp2YQoveo0Epd2pKJgdRPYLg9zQdbb1xLHt6d5pGTSAlo9+4XLKsYb7J//2JZwb/8BX2xEKZmT6zbY+/NsR99XGESKMC+G46tYNAvDophVTnPToxRLEKq6gbFbxxOSuU01/u/aNfXHfk72Wr9E1+HXSQe03oerJI7d1tfWwy9f2HUXD33VMueWbY7HdmBXGSa6mhhrcvap2Wpqx14ke2XnymRDUorAgtHspCmYmFo3g7ltqp2mw2wzZjH7H0tRQg02fXGz5Gn0+QHc6VqG+kz+9xrPNOmNCBLKw59Dr5zy3Z7WYyQ8LK2LT7pKHemuRGhNIJK3XEJi/s05pkW6TyLqGeCKJbU/1ev7sclFAbcaHa7ze6tRnsHey12M7XeEvXE5NueW7Mup+K2rlzP6+qR6N187N6LYuE+2y8ZKR44UgtPuNw1fEoiDSQh8VpVGUFBdNGUl+zUPIxzgY8OosgemhvgV1Ixj2/G7749jhlESQyxjBtjuWTQulReddnPY6oy6v/Wp1AXEiNS7Q8twxT+dqLvabmNFO3k9RofLycI7tJU3MeGW+nBp3bLciFkXL524GoIUsrByc/qXSj+l2MvnVni/4iQvbEYR2v1+4hCF3/MLlFGLRCB7dtMJXBsmlK9qq0aaGGl93EIB2rl6/48fYfMtiNFxfjsMjvt4+gZvub7b14fEjp6fMJ3gJzwRNLFo04bSNn1VRdNTy9XplTa+L2Px+/sDUc8CJMGpVmZnR4Ro/tzqHD4dzbK9X+Hgiifue7HV93cjoOLrePm+54tJcq8NrvQy/2mcSbtkQQWjP9gtnPm+8LERKjU2uGo2Q1yDPJGNC4N+OnMYLh6wdnRfcNrYxOvgwcVM/Oi7Q1hPHrTfOm/Lakf5K2/fs2N9nGzrVP29zDn4Y5GID9Bnt5P3c6mzeHM6xvY7yCN5ijsnUGPa+8o7lBcbucbf4nV/tfpFVh9tL+YF5y9/L2rZMVoeaMZ4n5rRDt/dkE6suWZZZXxDsa8IDcKwWaXe8Co/zUWbc2kmNCTzQfgyt3fEpry37WL+tg0umxkAESwf77bu09FG3dmPRCMpmWZ8XXtNAw6hVZWZGh2v83OocOAAs83Fx9HpstzQxwP+IwGlyxwq3C41f7X6QWYfbS+naB35wEaUrkxO23bvvKL6276hlJooTs6P+VoeaMZ83XkI3+ntqPJxjdiTfvAbRqrd8vYcA3L2q1rK2fUVpFEJ4D0cA2sj4zYd/H2seedHX+/xgtSgt+eY1KLn6Ldsvn/4eq+wkp1RZwuQqXQDTsp+iEcL9G272bHsQYUknZvRI3s+tzojPuKTXYzuVftWvzH7HYX5vz93CCX61+0Hm9mpud1s7Dx5H6srU58z56G4je/0ils3KVrtCV9ue7rV13tEIOS7O8ooY8/YV1+u91FTE8OimFRPlMsx3Sxcup3w7an1wkslkov9A1SRirAgOVagnGBNi2spbu+9UTUVsSuHApoaaaQvHdn4hvxZGhT6SJ6LPAPgegAiAHwohHgnq2H4yMLZsCebYwPQJUat6KsZRkF2OexFpXy7jys5YNIKNK2vQ2h33PHK8NDI5SWeFX+1+yEV2gB1ud1v9iSTKbuq3fb+XDJJsV7ja3TE80H7MMvd9AsNT+nu//uRRS6dF0Jb3W83pO+nXiRYRdn7R2jEFscJXrzvv5a7XiNc7YIKWjWS++HjRrmM+F9zKRRgJeySeLaGO5IkoAuCfAPw3ADcB2ExENwXZhteyvLt3Z39sAJYxYAC2NjjFLrfcUoudX1w+8SWIECGZGsOB3gEU+RjCJJIpx1FpJtq9kumG1EHgdrdVURrFUN8ix2PEE0nHWH02FyunkJDbnYG5XG9TQw0+Mts6zitg7eABWOo3n1p6W1afQbYXa3O5CD8jc693wALA+uXV0x5363szTvMmfmLl+bZXbNgj+U8BeEMI8RsAIKInAHwewKshtzuNIPY4zWSLMKcvyd7/fAeN187FrTfOw+NHTk/c1mYSt3SyI8z9Xf2MeNzwu7y7qaEGXW+fx95X3sGYEIgQTdRCaeuJY2h4FLMWuK/21BeufX8sCzMAABfbSURBVOOZPkQjWk67HnvOJrvCPD9h1OcF8+vsyvo6YaXfSpP+GZjnK7yOvqNFNK3WEGF6bZp/OnQSJ89e8q3Djb2vvDPtMS99b8Rq3sTvCD0f94oN28nXADB++mcA3GJ8ARE1A2gGgPnza9HSolWP6+zU6o40N2urFuvrtZznw4e1bJEDB7RY85Yt2khVd2Td3cDWrcCePUBJCbB+PbB3r7bE/MoVoK9v8phVVVoZ1tZW7Xd/P3DixOTz1dVAYyPQ3g6sWwcc/1kVRj+IoXz5aQz11qJ4ThLFlZdw7KWr0f172qbJAwOT71+6FCg7X40zr5ajdNm7GD5dhfFkdOL90XkX8Zc7+/Hh6Tko+1gZkm9eAzFWhLKb+jHUt2jiJL3y7hyU15/BpVcXgiLjiF1/FpdeW4iShRcwnipG6txVKF9+GsdeWoTvl03X1N6ufT67dgHvpS7g5x++gbd75mBx/Ye4dcH1mCsqJ2yuq9Ps7ugANmyw1rRwodY/GzcC77xUg+Vnr8KbVb/CyZ9egwV1I7hrdTWOPns1PlbqvZ9mX/ce7n80iciCMoyn5uDYuavwV++ewBP/fBXWfPwjlv1U+9sD+OfvxCBKF2PWggSSb8zH/xk4jw97L+CZn12BWDYLl15diPHkLBRXXsLwqasR++h7uPJuBcaGSib7Ye4lRMqHkThdle6nGlyY6KcbEJ13EUXRUYz0V6LsY/2++inx2kL86fEzuLfoHD6Il6H0E+MY6r0BRbEUZtcO4vLxBZhdO4ixodlInS+bsClSPoJFNwyjpUU7906cAEa7boJY9uaUc89N06VXF6J4TtLy3LPTdOylRfiz7gF8fdZZnH97kadz74NfXYuSshSKat6foukHL89G20v/gTNHqnFV5ShGqyJIvnEDZte9j9ELZZbfJzdNwxP9NKkpYaHp8on5SJ27yvP3afXtUbS2Zucjtj36Ac4NXDdF03DlJXxl2xUs/pb79ylTv+dEqLVriOgLAD4jhPjv6f//CMAtQoi/sHp9mLVrWlq0n2zIpJ5KW0/c0wrHILCzQ9ceVJ2MMLD7bCtLoyidVWw5unfqD33FZeKlG1Dx6ZNhmx8K+sI44/yO3wJeudQfdl55NEL4VF0lXn7zvKfX+9Xuty6SFXa1qYy1qMLAqXZN2CP5OABj4Y1F6cdyztat2R8jyNBEJlSWRjGcGrf8khs3OjY7RF17JuGmXGEXwrhwOTURvzbf+tq9J55ITsR+y+vPBG5rrtDnWnRanjvmexI0l/rDXjg0Nibwy9MJz6/3qz2IZIFcrGD1S9gplL8AcAMRLSGiWQC+BOC5kNu0ZM+e7I+RyWRMkKmEicupifaByVTLmorYREaO1cIgXbuXTBh90qhu+/O4fsePLWuch4HXL4ExPdPpPbrDufRqfpffrCyNTpxPVqmzydQYHmg/hm1P9WY0V5Pv+nVuuKZsQn+ECFELzzQOrea9E8ZUUL/ag3DEuVjB6pdQR/JCiFEi+gsAB6GlUP6rEOJYmG3aUVISzHH8TsZkuojFioUVMdv21zzyou0o/c4S57r0+sltDgeYqxsC4U0e+dn+Tb8o3XrjPNdaKRRxdgoyiUUjuH/DZDhmyfbnLV+XTY5+PusHJhdeGYvvLZgzO6PvjTH02NYTx590eJ94DcoRB1VYL0hCz5MXQvwYwI/DbseN9evltBvxWCNbjyPrlfzMefJuJ6HTKH39H2t/u4WbnHKiww7r6JkyXuqh6BclL9UZY9efDcC64DHH2wH/eeRe8KvfLuc8LHQHb85I8QsRpjj4Hfv7ELt+tuN7Mq0B70a+5c3P6LIGfti7N/uJ10zw4uAjRNMmfPyUFW7ridtueLCwIjah3W2U4RaTDHuB06HXz3mK6+oXJS/2XHptYV5OvI6MaiNsc8kAcypiLBpBSXGRq9O1G0z40a+XH2jrieMbz/Th0pVgtzm0orU7jgO9A57nGkqjRUiNi2llBIyrTPXBipv2cSFCnQzNF9g4+dWr5bTrpe7I5lumbwrhdTTgVOo4WqQtjY8ZBnNOx3UbSYY9eeTFaVfEohP2V6R3RnKiZOGFQGwLmmRqDC3PHcPI6OREuq5Fz1LR89UBYNtTvdPy0I2suq4Svzz9wTRn6Uf/5lsWZ7wFX6YkU2O+2rpz5aIpAyB9PcO9hg3R9fPITbvd+Rz2dny5ZkbXrvHD0JCcdp3qjkSI8IeraifqhGSC47Lz9DyeV+1OtuZi8sjtIhKLRiZq7euLndwYT9mPYzKo4BsoiWTKsu8EMKWWSlNDDXZ+cbljFcdTg8kpk/I6VvpjpllNAibOwwfa/Wfw2KFPggb5OT9+5PRESvLdq2oxnBqftvNTRboCpFPf253PXiqbzjTYjOT7+rTFBrkm7IkYp9Fvakzgvid7MfizS/jOyTOeVpDqtsYTSce9Q8PAas7APKrVY6529YB0dNtT564Clr1r+ZqKmJaDH3QcPAjMcyC6s7fLw+5PJKfcpTX8XQcuXE5N019ZGkXP/1xn2WZbTzyrSV4z+rkT5OdrLDBnNX+TTI2hpLgIsWgECYP2aIQc93oF7Hd4C2s+Kld3DGycfHOzvLbDmIjRTxC3GPaYEChffhrxxIinDJkgbc2kTAFgfUHUj+VlYZmeZQEAf3HxNdvXJS6ncP+Gm3ManvCD1QXcblLUvAGGrmt0+WT2kZ7NY0fQlUMz3VXJK3bnfiKZwnc3rcCDY2/h/XF4OvfcdngLej4ql+UP2Dj5XbvkTLyGgd+46VBvLSo+fTKnC58yPYmtLjJ+9Or1a/QLxVCv/apHPSUV0BYa5SqjxCvm8FVbTxyXrliHqIzbBQKTn/FXtl0BNb7qydEF6chysauSU9tNDTU4+myN5++820KzTOej7AY6uVyYyMbJV1XJtiA4/JZ/LYpNOq9clAAGgl1d60fvqusqp6SfGrWb2XbHMm1jkfbwHHwsGsnoLsEqZrzz4HHb8sT6doHmzzZamprmbO0cT7YpnMaURJkhMF2v1+98W0/csf/NfeH1DtVpoJPLEt1snPzatbItCA6nE0HPtzemVM6uHZx4PlfLq4M8if2858hvLky55TZqN6JPYoYRqimNFiGZGp9wAF427jZingPRnYrbMfSyyXrWydDwKIbnlWMWJh1M19vnp1wEjY7Hz4I080jdXANJnxPwSixahLllJYFdHJZsfx5zhq/G4p5FroMKpzBVhGiKLj93qE4DnVyWP2CTXdPaKtuC4HDatUava//tu5ZPZMpcPr4AQG6XVwdZZ97Pe8wxVV27ET1LJ5MNMSIeMkVGRgUe3bRiYm8Bu93DrPYM0HeEMjsVL85Pj4HrOzilxsUU/W77BJvLdthJ1VepmvdBMNak91v3cDg1jpe334bvblqBqMWHHC3SMtGMJUXs9lcFtAvQqa5KT5kxToOIb981dTMVPzuhOQ10cln+gI2TL6SRvJcTxPiFnV07GMoGwdnamM2x7DDXf9FH8saaJvrnkMldhdNmTpOvEdixvw/fbOvDmkdexL37jmJ2tAgVscktIXd+cfm0yVJgMuyi4/VCZBcDN9/JuE0sGjfKsZMqAPx9U/1Ev5jLX7T1xH3Xvi8iwpLtz2PnwePY9MnFUzbCrohFsfOLy9F47dwp7/mD36qxvCDozK4dRDI1hvue7HV09HaDiMrS6LTvi5/6T3afnz4XFPYG3jpswjX93ncCy3u8pmXqk5h79oS7BWA2NmZ6LLvSD+atE8eGZtuWUw4zdpxMjU1J8btwOTXNESdswhlGh+F0ITKmuNrpGBtyXtqvY+Xo7FbR6hfSTMIRdhgvFK3d8Wn9ZRUmae2OY9MnF+PQ6+cmSktPOWZau37RBawn/e3KfVhlIvmt/2SGoNVc0m3JxaCLjZM/cUK2BcHi5wSRpT3Ik9jqWHalH4yPz75cgYfvvMbzlztIzE7HvIm43Ypdo8O1cyrm2ud2tfVT58umPWaeDLa7w7Ib9bttzN2fSOLRTSssP9siAsbF5AXE6kJiNUFvd0E50DuAo/dref/mz8Co3WnS38+AJJv6T4B2DrR2x9F47dyc3VWzcfIy8+RlU6ja7S4ixsf7+7Wdd+zeD8ByAUzYGBftODlcr3sY2F2wypdPrdKpT+p6cWh2dwhuG3M7paaOi6mTtHaVN80XELsLSiKZmkgdNX8GZu1Od0VeByTZ1n8Ccr+HA5uY/K5dsi2Qh9JuT1NDDcZz7OB1EskUSoonv4KVpdGJHH99E2gAnmK35hhvZbrY2VBv7cRrjKUS7DaeN+I2r+Jl3uWiRekJL3sCmB93mnzXj6V/Bno4yajd7Rh+cPr8vLaRq1RmgNFIvnr6Zu5sUNqdcYofF0HbrMIPBOCj15S5blhNmLpp+9DwKPb94p2JXHg9rPPwnfWetqUzj0bbeuK477WLGIe3VZ9WxwPsR61uK5S9rCD1c6dit9rZ6DB1m3bs78PF8hHHY4aB1xBgLneKYuPkGy13P+SB0u6M3RdzzfVz8cXGWk9fWnNtci8lAszuz6rKZDa39k0NNVj8TWDlyht9v9d4jEzKYLjFpnUn5ycersfz7Y5ltAkA7k/040NkdoHLFLMefb2CuXx0LneKYuPk29u97WxeiCjtzq/x4mj05+wCO+ba5PcGuHl7Nrf2svreyWardF8vdWWsHLydw2xqqMHRKu9lDYLE6o5KZuliNk5+nXXhPRYo7e44ORrjc3ZZLFYx5KDSM7O5tZfV93b6zStIvWB3V+B2rHw573OVKmkHm4nXQkuh9IPSHhxeF3m53Y5XlkY9LfDSV8Bmiqy+t/uczCtI3WjridteLMeFcDwW5/PeCJuR/KlTsi2Qh9IeHH4WotlVtiRgYqGNfhy77RvLZhVnNQqU1ffZLoZr64m7VgZ1u8PhfN4bISEpfcyKxsZG0dXVFcqxnfKlCx2lXU7bVqsf9bov5t3A7DYDISCrfUhnYt97KS1tt4rZyEzUnilE1C2EsEwzYBOuUbniPJGp3ao+yaObVlhu9xhkQTcjfvXrdVf0PH0Z2955qdfjJa7P+bw3wiZcU1cn2wJ5KO3y8Drp5jVf3C9+9OdytyIn3LKJagwrap2Q3ff5ApuR/NKlsi2Qh9Ke/4RVldCPfj9ldMPE6e7Fz4VvpvR92LAZyXd0AL/927KtkIPSLtsKb4SRaudHf1i7FVnliQP2k7J2i9MqS6O4f8PNnj+jmdT3YZKVkyeinQA2ALgC4E0AfyyESKSf2wHgywDGAPylEOJglrZmxYYNMluXi9LOFz/6w9ityCoEtO3pXkBMrvA1h4WCKlPNve91sg3XvADg40KITwA4AWAHABDRTQC+BOBmAJ8B8AMi8rbrQ0iElLQzI1Da+eJHv17n3OvjXrAKAaXGxLQSDuawkNciak5w73udrJy8EKJDCKGXmTsCYFH6788DeEIIMSKEeAvAGwA+lU1b2TIwILN1uSjtfPGj/9Dr53w97gU/oZ6gKzNy73udICde/wTAv6f/rgHwjuG5M+nHpFGoNdW9oLTzxY/+MGLyfkI9QVdm5N73Oq4xeSL6fwCm74YMfEMI8Wz6Nd8AMArgcb8GEFEzgGYAmD+/Fi0twMaNQGcnMDioddSuXUB9PVBeDhw+DGzeDBw4AIyMaNva7d49WYSpuxvYuhXYswcoKQHWrwf27gXeeAP4gz8A+vomj1lVpe392tqq/e7v15ZC689XV2tVDNvbtToYJ05oq+j05+vqtBn8jg4t/tfVpY0e9OeXLtUWY3R2hqNp9WpgaMhd065d2nsLSZPXftqyRWunkDT56aeWFuD73/emabznRoxd9zaG+hZh1oIPAABX3p2D69e8j4ceykzT4rca8V6sG+931SBSPoJZCxIYeXMBSpecw/BgGUY/iKF8+WkM99Vh5e1V+PnPg+unr3xFs2Mm9FO2556jj812xSsRbQXwpwBuF0JcTj+2AwCEEA+n/z8IoEUIcdjpWGGueJWxz2m+oLTLtkIefvRbrTT1srLUy3H9ZNcEBae+d1rxmm12zWcA/DWA39EdfJrnAOwhou8AWAjgBgD/mU1b2cJlebMVSjtf/OgPcvN183Gd9lYNC+59r5Ntnvw/AigB8AJpW24dEUL8DyHEMSJ6EsCr0MI4fy6ECGe3ZI90dmq3WxxR2mVbIQ+/+mWXxQ0S7n2vk5WTF0J81OG5BwE8mM3xg2TjRtkWyENp5wtn/Zy1G2FT1qCzU7YF8lDa+cJZP2ftRtg4+cFB2RbIQ2nnC2f9nLUbUfXkGaC0y7ZCHpz1c9Ku6smDd21ppZ0vnPVz1m6EjZOvn75PAxuUdr5w1s9ZuxE2Tr68XLYF8lDa+cJZP2ftRtg4+cOOa20LG6WdL5z1c9ZuhI2T37xZtgXyUNr5wlk/Z+1G2Dj5AwdkWyAPpZ0vnPVz1m6EjZMfGZFtgTyUdr5w1s9ZuxE2Tp5LNTorlHa+cNbPWbsRNk5+927ZFshDaecLZ/2ctRth4+S9FNcvVJR2vnDWz1m7ETZOXqFQKDjCxsl3d8u2QB5KO1846+es3Ui2m4bMGLZulW2BPJR2vpj1W23FVyibhJjh3vc6bEbye/bItkAeSjtfjPr1PVzjiSQEgHgiiR37+9DWE5dmX5hw73sdNk6+pES2BfJQ2vli1L/z4PEpm3QDQDI1hp0Hj+fYqtzAve912Dj59etlWyAPpZ0vRv39iaTla+wen+lw73sdNk5+717ZFshDaeeLUf/Cipjla+wen+lw73sdNk5+9WrZFshDaeeLUf+2O5YhFo1MeT4WjWDbHctybFVu4N73Omyya4aGZFsgD6WdL0b9ehYNl+wa7n2vw2Yk39cn2wJ5KO18MetvaqjBy9tvw1uPfBYvb7+tYB08oPpeR23kzQClXbYV8uCsn5N2tZE3eG/qq7TzhbN+ztqNsHHyVVWyLZCH0s4Xzvo5azfCxsmvXSvbAnko7XzhrJ+zdiNsnHxrq2wL5KG084Wzfs7ajeTVxCsRnQPwdkiHvxrA+yEdO99R2vnCWT8n7dcKIeZZPZFXTj5MiKjLbva50FHaeWoHeOvnrN0Im3CNQqFQcEQ5eYVCoShgODl5zlmzSjtfOOvnrH0CNjF5hUKh4AinkbxCoVCwQzl5hUKhKGAKzskT0b8S0Vki+rXhsblE9AIRnUz/rpRpY1gQ0WIiOkRErxLRMSL6avpxLvpnE9F/ElFvWv8D6ceXENErRPQGEe0jolmybQ0LIooQUQ8RHUj/z0I7EZ0ioj4iOkpEXenHWJz3bhSckwewG8BnTI9tB/ATIcQNAH6S/r8QGQVwnxDiJgCrAPw5Ed0EPvpHANwmhFgOYAWAzxDRKgDfAvCoEOKjAC4A+LJEG8PmqwBeM/zPSfutQogVhtx4Lue9IwXn5IUQPwVw3vTw5wE8lv77MQBNOTUqRwghBoQQv0z/fRHal70GfPQLIYS+VUQ0/SMA3Abg6fTjBaufiBYB+CyAH6b/JzDRbgOL896NgnPyNswXQgyk/34XwHyZxuQCIqoD0ADgFTDSnw5XHAVwFsALAN4EkBBCjKZfcgbaha8Q+S6AvwYwnv6/Cny0CwAdRNRNRM3px9ic906w2f5PRwghiKig80aJqBxAK4CvCSE+1AZ0GoWuXwgxBmAFEVUAeAbAjZJNyglEtB7AWSFENxGtlW2PBD4thIgT0TUAXiCi141PFvp57wSXkfx7RFQNAOnfZyXbExpEFIXm4B8XQuxPP8xGv44QIgHgEIDVACqISB/QLAIQl2ZYeKwB8DkiOgXgCWhhmu+Bh3YIIeLp32ehXdw/BYbnvRVcnPxzAO5J/30PgGcl2hIa6RjsjwC8JoT4juEpLvrnpUfwIKIYgN+DNi9xCMAX0i8rSP1CiB1CiEVCiDoAXwLwohDibjDQTkRlRHSV/jeAdQB+DSbnvRsFt+KViPYCWAutzOh7AO4H0AbgSQC10EoZ3yWEME/OzniI6NMAfgagD5Nx2b+FFpfnoP8T0CbYItAGME8KIf6OiK6DNrqdC6AHwB8KIUbkWRou6XDNXwkh1nPQntb4TPrfYgB7hBAPElEVGJz3bhSck1coFArFJFzCNQqFQsES5eQVCoWigFFOXqFQKAoY5eQVCoWigFFOXqFQKAoY5eQVCoWigFFOXqFQKAqY/w8rVeMi+sOQYgAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "tags": [], + "needs_background": "light" + } + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "iBqjnpxOFwrO", + "colab_type": "code", + "outputId": "3e45301f-00a1-49bf-986f-4a2b6f161c23", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 549 + } + }, + "source": [ + "pp = sm.ProbPlot(epsilon_hat, fit=True)\n", + "pp.ppplot(line='45') # normal probability plot" + ], + "execution_count": 76, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEKCAYAAAAMzhLIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3dd5xU5fXH8c8BRBQUdbFFARtGwC52Y0zAWIOJ+oslWBBBdkWxd8WOXSwYQDAoTGxoFBUVxRoV6SBiCWIBKwRUQFTK+f3x3IVh2Z29M7szOzP7fb9e+9op9945e8U9+7TzmLsjIiJSlQZ1HYCIiOQ3JQoREUlJiUJERFJSohARkZSUKEREJCUlChERSSlricLMHjCz78xsehXvm5ndbWYzzWyame2erVhERCRz2WxRDAUOTfH+YUCb6KsH8I8sxiIiIhnKWqJw9zeA+SkOOQp4yIOxwAZmtnm24hERkcw0qsPP3gKYnfR8TvTa1xUPNLMehFYHTZs23WOHHXbISYAiIvls/nz49NOq32/EMloym42Yz0RKcJ9nmXxOXSaK2Nx9EDAIoEOHDj5hwoQ6jkhEpO6UlcE/UnbWO8fzCHdzNs1ZyA1czUSeyfjz6nLW05dAy6TnW0aviYhIFTp1Sp0ktmAOI+nMw5zILLZhdyZxLX1q9Jl1mShGAidHs5/2AX5w9zW6nUREJCgrgzFjKn/PWEF3BvE+7enIGM7lDvbjbd5nx+iIFSsy/dxsTo99GHgH+K2ZzTGzbmbW08x6RoeMAmYBM4H7gbJsxSIiUsgSCWjWrOqWxLbMZAwdGcQZTGQPduI9+nEuK2hISQkMHw4weXKmn5+1MQp3P6Ga9x04M1ufLyJSDBIJ6NoVli5d872GLKM3d3EdV7KUtTid+xlCN0pLjfvuW/3YLl0yj6EgBrNFROqryy+vPEnsyHsMoRt7MZ6n6UwZ9/Fjsy0YPgD+/vfajUElPERE8tgXX6z+vDG/cDV9mMTubMVnHMcjHL/2U9wyfAsWLqz9JAFKFCIiea1Vq1WP9+JdJrIHfbiWRziedsygpPQ4lvxsWUkQ5ZQoRETy2A03QPNGi7md83iHfWnODxzBs5zMMP5W2mKNsYhs0BiFiEge+/vmr9B5o+6s990s7qOUS7iJxiXrM/yu7HQzVUYtChGRfPT999C9O3TsiDdoyHGbvkYvu4+NWq/PXTlMEqAWhYhI/nn6aSgtZcU339JvrYu4/Jur+Zl1APj8c+jRIxymFoWISH3z3Xdw/PHwl78w59eN2cvf5fylN69MEuV++ilMm80VJQoRkbrmHpZPt20L//43I/e6jq3/N4GJdKjylIrTZrNJiUJEpC7Nng1HHgknnQTbb88z103mL+OvYBlrpTwtedpstmmMQkSkLqxYAQMHwsUXw/Ll0K8f9OpF100b4p76VLMwbTZXlChERHLt44/DjKY33gh1wwcNgq23pqwM/ve/6k/v2TO3s57U9SQikivLlsEtt8Auu8C0afDAAzB69MokkXozolBBdvhwcrLILplaFCIiuTB1Kpx2GkyaBH/9K/TvD5tvHitBlFu4MLshVkUtChGRbPrlF7jySujQAebMgccfhyeeSDtJtG6d3TBTUYtCRCRb3nkHunWDDz6Ak0+GO+6AkpKVbw8cGO8yjRvndvC6IrUoRERq26JFcM45sP/+sHgxPP88PPjgyiRRvmNdnM1JmzULQxm5HLyuSC0KEZHa9NJLocbGZ59Br15w442w3nor3061Y11FpaW5H7iujFoUIiK1YcGCMFj9pz/B2mvDm2/CPfesliQgTG0tpCQBShQiIjX3739Du3bw0ENw6aUwZQoccMDKtxMJaNEiLJRbtCj1pczqZgpsKup6EhHJ1DffwFlnwYgRsOuu8NxzsPvuqx2SzswmMxg2rG7HIyqjFoWISLrcQ+uhXTt45pkwDjFu3GpJonzAOm6SgNyvuI5LLQoRkXR8/jmccQa8+CLstx8MGQI77LDy7XRaEMmGD8/PJAFKFCIi8axYEQYOLrkkPL/nnpAVGqzqmOnUCcaMSf/SpaX5myRAXU8iItX76CM48MAwHnHAAfD++2Hqa5QkyruZMkkSHTvm18B1ZZQoRESqsnQp9O0bivjNmAFDh4bFc0n1NMrKoEuXsK4uHSUlobvp5ZdrN+RsUNeTiEhlJk8O5TcmT4Zjjw1dTZttttohiQQMGJDeZRs2DIu087mrqSK1KEREkv38M1x2Gey5J3z1VSjg9/jjqyWJ8nURXbpQ7SZDyUpKCi9JgFoUIiKr/Oc/cPrpYUyia1e4/XbYcMPVDkmnBAdAkyYweHDhJYdkalGIiCxcGAanf/e7UBZ89OhQia9CkgDo3Tt+kigthSVLCjtJgBKFiNR3L74IO+4Yph6dfTa89x4cfPDKt5PLb5jF26oU8q8MR02o60lE6qf58+Hcc8MK6x12CN1O++232iHpdjOVa9268FsRydSiEJH6Z8QIaNsW/vUvuPzyMLOpQpKA8Fa6SaKuNxnKBrUoRKT++PrrMBbx5JOhLtOLL4ZiflX4/PP0Ll9SAnfdVVytCVCiEJH6wD0sljvvvDD99eabw+NGVf8KTCTCmESc6a8lJTBvXu2Fm2+y2vVkZoea2UdmNtPMLqnk/VZm9qqZTTazaWZ2eDbjEZF66NNPw2ZCp50GO+0EU6fCRRelTBIQup3iJImGDUMrophlLVGYWUOgP3AY0A44wczaVTjsCuAxd98NOB4okjkCIlLnli+Hu+8OM5rGjg1TkF57DbbfPtbpcbqdCnUBXbqy2fW0FzDT3WcBmNkjwFHAjKRjHFg/etwc+CqL8YhIffHBB6H8xjvvwGGHhTobrVrFPr2srOr3ir2bqTLZ7HraApid9HxO9Fqyq4EuZjYHGAWcVdmFzKyHmU0wswlz587NRqwiUgyWLg1TjnbdNayuHjYs7DoXM0lkstlQfVDX02NPAIa6+5bA4cAwM1sjJncf5O4d3L3DxhtvnPMgRaQATJwIHTrAFVfAX/4SWhVduoQR6RgSCTjllOqrwM6fXwuxFphsJoovgZZJz7eMXkvWDXgMwN3fAZoALbIYk4gUmyVL4OKLYe+9Ye5c+Pe/4dFHYZNN0rpM795hWKM6afRgFY1sJorxQBsz29rMGhMGq0dWOOYLoCOAmbUlJAr1LYlIPG+8EfaKuOWWsIR6xozQmkhDeYmOOKU5inExXRxZSxTuvgzoBbwIfECY3fS+mV1rZp2jw84HupvZVOBh4FT3dIr2iki99OOPYcT597+HZcvC7j/33w8bbJDWZTp1Cr1TcZJEs2ahTmCxz3CqTFYX3Ln7KMIgdfJrVyU9ngHsn80YRKTIjBoFPXvCnDmhVtN110HTpmldIpEIyyp+/TXe8aWlxVPgLxN1PZgtIhLPvHlw0klwxBGw3nrw9ttwxx1pJYnyWU1duihJpEOJQkTymzs89hi0awePPAJXXQWTJsE++6R1mbizmpKVlChJgGo9iUg+++qrMBbx9NNh6uvLL8POO6d9mUQiNEbSGQFt3Lj4S3PEpRaFiOQf97B/aLt2ocLrbbeFVdYZJomuXdPf27q+DlxXRi0KEckvs2ZB9+7wyithVtPgwbDddhlfLp09JTp2DI0WWZ1aFCKSH5YvhzvvDEX8xo+HgQNDsqhBkigri1fcr1mzsHWpkkTl1KIQkbr3/vuhiN+774ZZTQMGwJZbpnWJRCKsro67p3U5zWqqnloUIlJ3fv0Vrr0WdtsNPvkkbE36zDMZJYlTTlGSyBa1KESkbowfH1a9TZ8OJ54I/fpBhkU/e/aMV6cp2fDhGqyOSy0KEcmtn36CCy4I6yAWLICRI0OTIIMkUb6AbtGi9M5r3VpJIh1qUYhI7rz2Gpx+euhmOuOMsHd18+YZXapTJxgzJv3zGjWqn4X9aqLaFoWZ3WJm65vZWmY2xszmmlmXXAQnIkXihx9CYvjDH8LzV14JA9YZJIlEAtZeO7Mk0aQJDB2q1kS64nQ9/cndfwSOBD4DtgMuzGZQIlJEnn0W2rcP6yEuuACmTVuVMNJUVpZenaZyJSVhTGLJEiWJTMRJFOXdU0cAj7v7D1mMR0SKxdy5YZD6z3+GDTcMK6tvvRXWXTejy5WVpbdFaWlpWI3tHuoJKkFkLk6ieNbMPgT2AMaY2cbAz9kNS0QKlnuY5tq2LYwYAddcE7Yp3WuvjC6XyT7WmvZau6odzHb3S8zsFuAHd19uZj8BR2U/NBEpOHPmhN/Szz4btiYdMiR0O2Ug3RYEhDGIwYPVeqhtcQaz1wXKgPL/ZL8BOmQzKBEpMCtWhJIb7dqFUeY77oC33sooSZQPVqebJDp21BhEtsTpevon8CuwX/T8S+D6rEUkIoVl5szwW7pnT9hzz7CA7txzoWHDtC5TVgZm6Q9Wq05T9sVZR7Gtux9nZicAuPtPZmZZjktE8t2yZWE19ZVXhs0b7r8/1GtK89dDutuSJmvaFBYuTP88SU+cRPGrma0DOICZbQv8ktWoRCS/vfdeSArjx0PnzmHkeIst0r5M+V4RccuAJ2vYMPR2SfbF6XrqA7wAtDSzBDAGuCirUYlIfvrlF+jTB3bfHT77DB59FJ56KqMkAaHaayZJoqQEHnxQ4xG5EmfW00tmNgnYBzCgt7vPy3pkIpJfxo4NrYgZM8JAQr9+4Td2hsrK0qv2qhlNdafKFoWZ7RB93x1oDXwNfAW0il4Tkfpg8WI47zzYbz/48Ud47jkYNqxGSSKRiD+rqXywWjOa6k6qFsV5QA/g9krec+CPWYlIRPLHmDFhW9JPPw3rI266CdZfv0aXTCTgpJOqP04tiPxRZaJw9x7Rw8PcfbWV2GbWJKtRiUjd+v57uPDC8Ju6TRt4/XU48MCML5fO7nMNGsBDDylB5JM4g9lvx3xNRIrB00+HhXMPPAAXXQRTp9YoSZQX8os7HqEkkX+qbFGY2WbAFsA6ZrYbYSAbYH0gs6peIpK/vv0Wzj4bHnsMdt45bCjUIbMiDGVloYq4e3rnNW2qJJGPUo1RHAKcCmwJ3JH0+kLgsizGJCK55L6qb2jRIrj++tCSWGutjC6XSY2mcloXkZ9SjVE8CDxoZse4+xM5jElEcuWLL0Lpjeefh333DUX82rZN6xKJRNiTaPHimoVSWqrWRL5K1fXUxd2HA1uZ2XkV33f3Oyo5TUQKQXkRv4suCo/vugvOPDOj+kyZth7KNWsWuqmUJPJXqq6nptH3ZrkIRERy5OOPw77Vb74ZNp4eNAi23jqtSyhB1C+pup4GRt+vyV04IpI1y5aF8t99+oRFCg88AKeeGruIXzpTXFNRgig8qbqe7k51orufXfvhiEhWTJ0aSrROmgR//Sv07w+bbx7r1JpUd01WUhJ6uJQgCk+qrqeJOYtCRLLj55/DLKabbw6/qUeMgGOOSXlKbQ1Ol2vcODRelCAKV3WznmrEzA4F7gIaAoPd/aZKjvkbcDWhLMhUdz+xpp8rIsDbb4cifh9+CKecErqdNtoo5SmJBJx8chjfrg1qRRSHVF1P/dz9HDN7hmgvimTu3jnVhc2sIdAfOBiYA4w3s5HuPiPpmDbApcD+7r7AzDbJ8OcQkXKLFsHll8M990DLlvDCC3DIIbFO7d27ZkmitDRsTSHFJVXX07Do+20ZXnsvYKa7zwIws0eAo4AZScd0B/q7+wIAd/8uw88SEYDRo6FHD/j8c+jVC268EdZbL9apiUTmA9Uq4Ffcqqz15O4To++vA+8AC4D5wDvRa9XZApid9HxO9Fqy7YHtzewtMxsbdVWtwcx6mNkEM5swd+7cGB8tUs8sWBC2ijvkkPBb+803Q4siZpKA0JpIV0mJSoDXB9UWBTSzI4BPgLuBe4GZZnZYLX1+I6ANcBBwAnC/mW1Q8SB3H+TuHdy9w8Ybb1xLHy1SJJ58MhTxGzYMLr0UpkyBAw6IfXoiEaasxm1NmIUuJneYN08Joj6Is2f27cAf3H0mrNwz+zng+WrO+xJomfR8y+i1ZHOAd919KfCpmX1MSBzjY8QlUr99803oXnriCdh1Vxg1CnbbLa1LJBJhnHv58uqP1cB0/RWnzPjC8iQRmUUoDFid8UAbM9vazBoDxwMjKxzzFKE1gZm1IHRFzYpxbZH6yz1sGN2uHTz7bBiHGDcu7SQBobupuiQxfLhaD/VdqllPR0cPJ5jZKOAxwuyn/yPGX/zuvszMegEvEqbHPuDu75vZtcAEdx8ZvfcnM5sBLAcudPcarvsUKWKffx4WObz4Iuy/fxhB3mGHjC4VZ/C6pETJQVJ3Pf056fG3wO+jx3OBdeJc3N1HAaMqvHZV0mMnbLm6RtFBEUmyYkWYd3rJJeH5PfeEgksN4nQKrC7ugjqz0NUkkmrBXddcBiIiVfjww1DE7623wqymgQOhdeuMLpVIhMlRS5dWf2zPnmpNSFDtYHa0P3Y3oD2wcq9sdz8ti3GJyNKlcOutcM01Yeu3Bx+Ek06KXcQvWboF/bRwTpLFabcOAzYj7Hj3OmH2UpzBbBHJ1OTJsNdeYYV1587wwQehtkYGSSLdPatLSpQkZHVxEsV27n4lsDiq/3QEsHd2wxKpp37+OayF2HPPMP31iSfg8cdh000zuly6+0Y0bqxxCVlTnHUU5b2Z35vZjsA3gGoyidS2//wnFPH7+OMwkHD77bDhhhlfLpFIL0lonwipSpxEMcjMNgSuJKyDaBY9FpHasHBhaEX07w9bbRXqNR18cI0vm05JjpKSsE5CpDLVdj25+2B3X+Dur7v7Nu6+SfnudyJSQy+8ADvuGAYFeveG996rcZJItySHupukOnFqPZWY2T1mNsnMJppZPzMryUVwIkXrf/8LtTMOOyzMaHrrLejXL/yGr4Hygeu4mw6VlGhTIalenMHsR4DvgGOAY4F5wKPZDEqkaLmHXebatYN//QuuuCLMcNp33xpfOu7AdcOGKssh6YkzRrG5u1+X9Px6MzsuWwGJFK2vv4Yzz4R//xv22COMReyyS40vm87MpgYNwnIMJQdJR5wWxWgzO97MGkRffyPUaBKRONzhn/8MrYjnnw/7V48dW+MkkUjA2mvHTxJm8NBDShKSvlRFARcSigAacA4wPHqrAbAIuCDr0YkUuk8/DTvOvfwy/O53oYjf9tvX+LLpro8AleSQzKWq9RR/aywRWd3y5XDvvXDZZWFQ4L77QiW+DIr4VdSpE4wZk945KskhNRFnjAIz6wwcGD19zd2fzV5IIgVuxoxQxO+dd8KspoEDoWXL6s+rQrp1mpJpEZ3UhjjTY28CegMzoq/eZtY324GJFJylS+H668MGQh9/HKYWPfdcjZNE166ZJYnS0rCWT0lCaipOi+JwYFd3XwFgZg8Ck4FLsxmYSEGZOBFOOw2mTYPjjoO774ZNal7ppmfPeCXBkzVpEoZClCCktsTtMN0g6XHzbAQiUpCWLIGLLw6VXufOhaeegkceqZUk0akTLFoU//hmzUIjZskSJQmpXXFaFDcCk83sVcIMqAOBS7IalUgheP116N4d/vvfMCZx662wwQbVn5dCpuMRHTuGiVUi2ZAyUZhZA2AFsA+wZ/Tyxe7+TbYDE8lbP/4YWhEDBsA224Tf0B071viymUx5BSUJyb6UXU/RuMRF7v61u4+MvpQkpP4aNQrat4dBg+C888KYRIZJorx4n1n4SjdJlJSEriYlCcm2OGMUL5vZBWbW0sw2Kv/KemQi+WTevFBt74gjYP314e23w34RTZvGOj2RgBYtViUFs/SK9yUrLVWdJsmtOGMU5XWdzkx6zYFtaj8ckTzjDo89BmedBQsWQJ8+Ye+ItdeOfYlMu5Qq07GjFs5J7lWbKNx961wEIpJ3vvwy/JYfORI6dAjLoXfaKa1LZLKKuipaXS11pcquJzPb28ymmtkiM3vHzNrmMjCROuMO998fiviNHg233RZWWadIEhXHG8q/aiNJlE97VZKQupJqjKI/ofBfCXAH0C8nEYnUpU8+Cf07PXrA7ruHHefOPx8arWp81+Z4QyrlCUKrq6WupUoUDdz9JXf/xd0fBzbOVVAiObd8OdxxR2g1TJwY6jONGQPbbbfykLKyVUkhk5IacZXPZlKCkHyRaoxiAzM7uqrn7v5k9sISyaHp06FbNxg3Do48Mow8b7nlaofU5lhDRSUlYc9qJQXJV6kSxevAn6t47oAShRS2X3+Fvn3hhhugefOwNenxx4dmQ5KystpNElogJ4Um1X4UXXMZiEhOjRsXWhHTp8OJJ0K/frDx6r2rtTmttZyShBSimu+iIlJIfvoJLrgA9t03rIsYOTKMTicliXS3GI1Dq6ilkMXauEikKLz6aijeN2tW2G3u5ptDl1OkNloQGm+QYqREIcXvhx/gootCfaZttw0J46CDVjskk8Fq7fsg9UWcHe7WNbMrzez+6HkbMzsy+6GJ1IJnngkL5wYPDl1O06atkSQyGawuLdW+D1J/xBmj+CfwC7Bv9PxL4PqsRSRSG+bODYPUnTuH/qCxY8N+Eeuuu/KQ8tXU6XY3qZSG1DdxEsW27n4LsBTA3X8ibGAkkn/cwzTXtm1hxAi45hqYMAH23HO1wxIJOOWU9FZTN2miUhpSP8VJFL+a2TqEtROY2baEFka1zOxQM/vIzGaaWZW74pnZMWbmZtYhVtQilZk9G/7859AftN12MHkyXHUVNG68RtmNLl3CYuw4tMWo1HdxEkUf4AWgpZklgDHARdWdZGYNCfWiDgPaASeYWbtKjlsP6A28m0bcIqusWBFKbrRvHwaq77wT3norPCeMQWRSdqO8BaFSGlLfxSkz/pKZTSJsh2pAb3efF+PaewEz3X0WgJk9AhwFzKhw3HXAzcCF6QQuAoT9qrt3D/tXd+wYZjZts03Ge0+X0ziEyCpVJgoz273CS19H31uZWSt3n1TNtbcAZic9nwPsXclntHT358ysykRhZj2AHgCtWrWq5mOlXli2LKymvvLKsDpu8GASa59G772sxgX7lCREVpeqRXF7ivcc+GNNPtjMGhDKl59a3bHuPggYBNChQwevyedKEZg2LZTfmDABjjoK7ruPsut/UysrqZUkRNaUqtbTH2p47S+BlknPt4xeK7cesCPwmoUibJsBI82ss7tPqOFnSzH65ZdQwK9vX9hwQ3j0Ufi//6PsTFOSEMmiascozKwJUAYcQGhJvAkMcPefqzl1PNDGzLYmJIjjgRPL33T3H4AWSZ/zGnCBkoRUauxYvj+2Gxt8OYNhdOGcuf2Yf1zJqh3da0BlN0RSizPr6SGgPXAPcG/0eFh1J7n7MqAX8CLwAfCYu79vZteaWefMQ5Z6ZfFiOPdcVuy7Hz9+uZDDeY6TGcZ8SjK+ZPl0V/fwNW+ekoRIKnFqPe3o7snTWl81s4ozlyrl7qOAURVeu6qKYw+Kc00pfolEqNm39+Ix3E93tuFT/kEZl9KXhayf8XWbNYMBA5QURNIVp0Uxycz2KX9iZnsD6h6SWlNxMdyZXb6n3+LTGUMnltGIA3mdXvTPOEloa1GRmonTotgDeNvMvoietwI+MrP3AHf3nbMWnRS98lIa5aukj+Ip7qOMTfiOm7iYa+jDz6yT9nUbNoQHH1RiEKkNcRLFoVmPQuqdigviNuFb7uEs/sbjTGEX/swzTGKPjK6tLiaR2hVnZfbnZrYhYapro6TXq1twJ7KGRAJOOy1sVx04XRhOP86hGYu4nOu5hYtYxlppX1uzl0SyI8702OsIi+I+ISoMSC0suJP6o6pyGi35ggH05HCe5232pRtD+JC2sa6priWR3InT9fQ3QqnxX6s9UiSSqtaSsYKeDOBmLsZwzuJu7qOMFTSMdW21HERyK06imA5sAHyX5VikCKzZtbS6NnzMYE7nQN5kNAfTg0F8zlaVHquEIJIf4iSKvsBkM5tO0j4U7q5Fc7KaRAJOPjlU/a6oIcs4n9u5hj4sYR1O5Z88yClUtgeWSmmI5Jc4ieJBQhnw94BKfgWIBL17V54kdmEKQ+jGHkziSf7KmfTnGzZf4zi1IETyU5xE8ZO73531SKQgla+irmxL0bX5mSu5jou5mXm04BhG8CTHrHaMGfTsqRaESD6LkyjeNLO+wEhW73rS9Nh6rqyMKqu27svbDKEbbfmQoZzCedzBAjZa+b7WOogUjjiJYrfo+z5Jr2l6bD2WqhXRlEXcyGX04l5m05JDeIHRHLLyfXUviRSeOAvuarovhRSBOFuLHsxoBtGDVnxBf87kMm5kEetpcFqkwMVpUWBmRxDKizcpf83dr81WUJJfUnUxAWzIfG7nfLoylA/5LQfyBm9xAGYwfJhaDyKFLs7K7AHAusAfgMHAscC4LMcleaJTJxgzpur3j+YJ+nMmLZjHjVzKtVzFL9HfEz17KkmIFIM4Zcb3c/eTgQXufg2wL7B9dsOSfJAqSWzKNzzOsTzBsXzN5uzJeC7nRn6hycqNgdTdJFIc4iSKJdH3n8zsN8BSqGQSvBSF5L0hKk8SzikMZQbtOJJnuYS+7MU4prAbpaVhxzjt+yBSXOKMUTxrZhsAtwKTCDOe7s9qVFInKu4NUVFrPmMgZ3AIo3mTAzidwXzMb2nSBIYOVnIQKVZxZj1dFz18wsyeBZq4+w/ZDUtyLZGAk04KLYKKjBWcSX/6cimOcSb38g9KsQYNKD1DXUwixa7KRGFmewKz3f2b6PnJwDHA52Z2tbvPz1GMkiWp1kOU+y0fMpjTOYC3eIFDOIOBtOnYmhUv5y5OEalbqcYoBgK/ApjZgcBNwEPAD8Cg7Icm2VRWBl26VJ0kGrGUS7mRqexCO2ZwMg9yGM/TpmNrXlaSEKlXUnU9NUxqNRwHDHL3JwhdUFOyH5pkQ3VrIgB2YxJD6MZuTOEx/o+zuIfv2FQL50TqqVQtioZmVp5IOgKvJL0Xa6Ge1L2yMmjQIMxiMkudJJqwhBu5lHHsxWZ8w195kuN4jJ+abarpriL1WKpf+A8Dr5vZPMIU2TcBzGw7QveT5Kk45TYq2p//MIRu/JaPGcJpXMBtfM+GakWISNWJwt1vMLMxhDUTo91XzodpAJyVi+AkfXG6lpI1YyF9uZRe9OdTtqITLzGGTshoPtEAABFoSURBVGHRnKq7igjVdCG5+9hKXvs4e+FIpqrbgrQyh/I8AzmDLZlDP3pzBddjzZopQYjIajTWUODiTHGtaCP+x52cy8kMYwZt2Z+32K10Xxapi0lEKqFEUcCqK9i3JudYRnAvvdiI+VzLldy+9uXcN2RttSBEpEpxaj1JHqm+FlPlNuNrnuRoHudvzKYlnZpPYNvh1/LDz0oSIpKaWhQFIJNZTKs4Xfknd9p5NG30C9xwCx3OPZfXG+k/vYjEo98WeSqTsYeKtmYWj290BnvMfxl+dyDcfz9srwrxIpIedT3liUQCmjVbtTAuVXmN6jRgOefQjw8a7cQeS98N82VffVVJQkQyohZFHSorgwEDKq/YmqldG89gdKtubDxzLBx8GAwcCC1b1t4HiEi9oxZFHSlfGFdbSWItfuWBba5jMrux8YL/hi3mnntOSUJEaiyricLMDjWzj8xsppldUsn755nZDDObZmZjzKx1NuPJJwMH1t61OjafwLet9qTrrKvg6KNhxoywYs6s9j5EROqtrCUKM2sI9AcOA9oBJ5hZuwqHTQY6uPvOwAjglmzFk0/KymDFippdo0EDOLv7EvzCi3h54d5suGwePP00PPwwbLJJ7QQqIkJ2WxR7ATPdfZa7/wo8AhyVfIC7v+ruP0VPxwJbZjGeOpW8/iGdWkzJmjULPUrusPyV17nr1Z3h1luhWzd4/33o3Ll2gxYRIbuJYgtgdtLzOdFrVekGPF/ZG2bWw8wmmNmEuXPn1mKI2Vc+m6lLl8zWQZSUrEoOCxfC3//8I5SWwkEHhWbJmDEwaBBssEGtxy4iAnkymG1mXYAOwK2Vve/ug9y9g7t32HjjjXMbXIbKyjKb5pqcGNxh3rykAn3PPQft24fEcN55MG0a/PGPWYlfRKRcNqfHfgkkT7nZMnptNWbWCbgc+L27/5LFeHIm/RpMIakMG1ZF1dZ58+Ccc0LzpH17GDEC9t67VmIVEalONlsU44E2Zra1mTUGjgdGJh9gZrsR9ubu7O7fZTGWrMu0BlO5nj0rSRLu8Mgj0LYtPPYY9OkDkyYpSYhITmWtReHuy8ysF/Ai0BB4wN3fN7NrgQnuPpLQ1dQMeNzCVM4v3L3gRmTT3Syookp3kfvyy3DhkSNhzz1hyBDYaacaxSkikomsrsx291HAqAqvXZX0uFM2Pz9baqMOE4RB7gEVNwlyh8GD4YILYOlSuO220O3UsGHNPkxEJEMq4ZGG2koQJSVw112VdDV98gl07x7qMh10UCjit912NfswEZEaUqKIKZMB6mRNmoSGQqWD1cuXh8xxxRWw1lph2fbpp4dVdSIidUyJIoayspoliUrHIMpNnx4WzI0bB0ceGQY7tizadYciUoD0J2slKpb8rulK6kqTxK+/wjXXwO67w6xZofTGyJFKEiKSd9SiSJLVQepk48aFVsT06XDiiaHbqUWLmn2oiEiWqEURSSSga9eaJYnyVdULF1aRJH76Cc4/H/bdFxYsgGeeWbUAQ0QkT6lFQebrIFIOUFf06qthgHrWrNBsuflmaN48/Q8VEcmxep8oMp3N1LQpLFoU48AffoALLwxTXbfddtXUVxGRAlEvu55qWm6jceOYGw898wy0axdWVV94YSjipyQhIgWmXiWKmpb8hjAO8cAD1XQ3zZ0LJ5wQ9ocoKYF334VbboF1183sQ0VE6lC96HrKdDZTgwbw0EMxxyAglN94+GE4+2z48Ue49lq4+OLQBBERKVBFnyjKZzMtXZreeY0awdChaSSJ2bPDyrrnngvVXYcMCSXBRUQKXNF3PV1+efpJokmTNJLEihVh0UT79mGg+s474a23lCREpGgUfYvi88/TOz5luY2K/vvfUMTv9dehY8ew89w226Qdo4hIPivqFkUiEf/YlOU2Klq2DG69FXbeGaZMCYspXnpJSUJEilJRtyguv7z6Y6os+V2VadNC+Y0JE+Coo0Jm+c1vahSniEg+K9pEkUik7nYqKQlbUcf2yy9www3Qty9stFHYmvTYY8NiDBGRIlaUiaK61dZmoRUR2zvvhFbEBx/ASSeFAeuSkhrHKSJSCIpujCJOSY6ePWN2NS1eHLYh3X//UK9j1KiwsEJJQkTqkaJqUcTdYCjWgPXLL4cZTZ99Fi7cty+sv35NQxQRKThF06JIJMJyhuq0bl3NAd9/H7qZDj44bEv6+uvQv7+ShIjUW0WRKBIJOPnkUEEjlcaNw3h0lZ56KhTxe/BBuOQSmDoVDjywVmMVESk0Bd/1VF6iY8WK1Mel3Dvi22/hrLPg8cdhl11C1dc99shKvCIihabgE0WcEh0dO4YhhzW4w7BhYcB68eLQ3LjwwtDlJCIiQBF0PX3xRer3S0urSBJffAGHHw6nnAI77BBWWF92mZKEiEgFBZ8oNtqo6vdKSiqZ4bRiRRicbt8e3nwT7r47fG/bNqtxiogUqoLueiorq3oDokaNKllU99FHYd/q//wnzGoaNAi22irbYYqIFLSCbVFUNx22efOkgetly+Cmm8JA9fTp8M9/wosvKkmIiMRQsC2K3r1TT4edPz96MGVKWBcxaRIcfXTodtpss5zEKCJSDAqyRZFIVL/ndZuWP4cpUR06wJdfwogR8MQTShIiImkqyERRXfnw3zd6i3FLd4Ubb4QuXWDGDDjmmNwEJyJSZAouUXzxRdXlw5uyiPvWOptXl/+O5mstgRdeCHuappoaJSIiKRVcopg7t/LXD2Y0MxrsSOmye7FevcKg9SGH5DY4EZEiVHCJoqINmc8DdGU0h9B80yar1kast15dhyYiUhQKOlEczRPMoB0nMYwbuIzms6aEvSNERKTWZDVRmNmhZvaRmc00s0sqeX9tM3s0ev9dM9sqznU342tGcAxPcCxf8Rs6MIH7W98QKv+JiEityto6CjNrCPQHDgbmAOPNbKS7z0g6rBuwwN23M7PjgZuB41Jdt4T/MYN2rMMSLqEvt3M+NFqLoanKh4uISMay2aLYC5jp7rPc/VfgEeCoCsccBTwYPR4BdDQzS3XRrfiM6ezILkzlZi5hua3F0KExtzYVEZG0ZXNl9hbA7KTnc4C9qzrG3ZeZ2Q9ACTAv+SAz6wH0CM9KOJAlQMgM7tCly8SJXbrU/g9QAFpQ4V7VY7oXq+herKJ7scpvMz2xIEp4uPsgYBCAmU1wn9ehjkPKC+FeuO4FuhfJdC9W0b1YxcwmZHpuNruevgRaJj3fMnqt0mPMrBHQHKimOIeIiORSNhPFeKCNmW1tZo2B44GRFY4ZCZwSPT4WeMW9up2vRUQkl7LW9RSNOfQCXgQaAg+4+/tmdi0wwd1HAkOAYWY2E5hPSCbVGZStmAuQ7sUquher6F6sonuxSsb3wvQHvIiIpFLQK7NFRCT7lChERCSlvE0U2Sr/UYhi3IvzzGyGmU0zszFm1rou4syF6u5F0nHHmJmbWdFOjYxzL8zsb9G/jffN7F+5jjFXYvw/0srMXjWzydH/J4fXRZzZZmYPmNl3Zja9ivfNzO6O7tM0M9s91oXdPe++CIPfnwDbAI2BqUC7CseUAQOix8cDj9Z13HV4L/4ArBs9Lq3P9yI6bj3gDWAs0KGu467DfxdtgMnAhtHzTeo67jq8F4OA0uhxO+Czuo47S/fiQGB3YHoV7x8OPA8YsA/wbpzr5muLIivlPwpUtffC3V9195+ip2MJa1aKUZx/FwDXEeqG/ZzL4HIszr3oDvR39wUA7v5djmPMlTj3woH1o8fNga9yGF/OuPsbhBmkVTkKeMiDscAGZrZ5ddfN10RRWfmPLao6xt2XAeXlP4pNnHuRrBvhL4ZiVO29iJrSLd39uVwGVgfi/LvYHtjezN4ys7FmdmjOosutOPfiaqCLmc0BRgFn5Sa0vJPu7xOgQEp4SDxm1gXoAPy+rmOpC2bWALgDOLWOQ8kXjQjdTwcRWplvmNlO7v59nUZVN04Ahrr77Wa2L2H91o7uvqKuAysE+dqiUPmPVeLcC8ysE3A50Nndf8lRbLlW3b1YD9gReM3MPiP0wY4s0gHtOP8u5gAj3X2pu38KfExIHMUmzr3oBjwG4O7vAE0IBQPrm1i/TyrK10Sh8h+rVHsvzGw3YCAhSRRrPzRUcy/c/Qd3b+HuW7n7VoTxms7unnExtDwW5/+RpwitCcysBaEralYug8yROPfiC6AjgJm1JSSKuTmNMj+MBE6OZj/tA/zg7l9Xd1Jedj159sp/FJyY9+JWoBnweDSe/4W7d66zoLMk5r2oF2LeixeBP5nZDGA5cKG7F12rO+a9OB+438zOJQxsn1qMf1ia2cOEPw5aROMxfYC1ANx9AGF85nBgJvAT0DXWdYvwXomISC3K164nERHJE0oUIiKSkhKFiIikpEQhIiIpKVGIiEhKShSSNWZWYmZToq9vzOzL6PH30ZTNXMbyFzNrl/T82miRYrrX2aqyypzR60uin2+GmQ2IVorHve7VZnZBTeOI3htc/rOa2WfRGgrM7O2kc09MOr6Dmd0d97Ol/lGikKxx9/+5+67uviswALgzerwrUOulE6IV+lX5C6FqaHlsV7n7y7UcwifRz7dz9Fl/SSO+WuPup7v7GonY3feLHm4FnJj0+gR3PzsXsUlhUqKQutLQzO6P9kkYbWbrAJjZtmb2gplNNLM3zWyH6PWtzOwVW7XnRqvo9aHRX+/vArdUdr6Z7Qd0Bm6N/uLfNjrv2Ogae5rZ22Y21czGmdl60ee9aWaToq/9qvg51hAVqXwb2M7MTjWzkWb2CjDGzDYys6ein2Osme2cdOouZvaOmf3XzLpHsTWLft5JZvaemSVXRW1kZgkz+8DMRpjZutE5r1klZUvMbFH08Cbgd9G9ONfMDjKzZ6NjmlrY02Cchb0bjopebx+9NiWKvRhLgUhV6rp+ur7qxxeheucF0eOtgGXArtHzx4Au0eMxQJvo8d6E0iwAzwCnRI9PA56KHg8FngUaVnP+UODYpHiGEkq/NCaUtdgzen19QsWCdYEm0WttCCt8y2Nfo9Z/8uvRueOBwwgFCucAG0Xv3QP0iR7/EZiSdH+mAusQahDNBn4TxbJ+dEwLwopaiz7Pgf2j9x5Iur+vEe3DAXwGtIgeL4q+HwQ8mxT7yufAjUn/LTYg1IdqGsX99+j1xsA6df1vSl+5+8rLEh5SL3zq7lOixxOBrcysGbAfq0qRAKwdfd8XODp6PAy4Jelaj7v78mrOr8pvga/dfTyAu/8I4S9r4F4z25VQ/mL7GD/TtmY2hfAL/Gl3f97MTgVecvfyPQIOAI6JPuuVaBynfJ+Ep919CbDEzF4l7LPwHHCjmR1I6K7bAtg0On62u78VPR4OnA3cFiPOVP4EdE4aL2kCtALeAS43sy2BJ939vzX8HCkgShRSV5Ir3C4n/CXdAPjeQz9/OhZH3zM9vzLnAt8Cu0TXjbMJ0idVfPbiSl6rTMV6Og78HdgY2MPdl1qoitskxfE1ZcAx7v5Rhdc/iLr3jgBGmdkZ7v5KLXyeFACNUUjeiP6a/9TM/g9W7u+7S/T226wq/Ph34M00z19IKENe0UfA5ma2Z3TOeraqbP3XHvYrOIlQbK42vBnFj5kdBMwrb8UAR5lZEzMrIXQHjY/i+C5KEn8AkvdDb2VhbwUIg9P/iRlDVfcCQmG9syxqklmoTIyZbQPMcve7gacJA/ZSTyhRSL75O9DNzKYC77NqS8uzgK5mNo3wi7t3muc/AlwYDdBuW36wh60zjwPuic55ifAX+33AKdFrOxC/VVCdq4E9op/jJlaVygeYBrxKKI9+nbt/BSSADmb2HnAy8GHS8R8BZ5rZB8CGwD9ixjANWB4N3p9b4b3rCNVGp5nZ+9FzgL8B06OutR2Bh2J+lhQBVY8VEZGU1KIQEZGUlChERCQlJQoREUlJiUJERFJSohARkZSUKEREJCUlChERSen/Aev5GncusP2UAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 76 + }, + { + "output_type": "display_data", + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEKCAYAAAAMzhLIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3dd5xU5fXH8c8BRBQUdbFFARtGwC52Y0zAWIOJ+oslWBBBdkWxd8WOXSwYQDAoTGxoFBUVxRoV6SBiCWIBKwRUQFTK+f3x3IVh2Z29M7szOzP7fb9e+9op9945e8U9+7TzmLsjIiJSlQZ1HYCIiOQ3JQoREUlJiUJERFJSohARkZSUKEREJCUlChERSSlricLMHjCz78xsehXvm5ndbWYzzWyame2erVhERCRz2WxRDAUOTfH+YUCb6KsH8I8sxiIiIhnKWqJw9zeA+SkOOQp4yIOxwAZmtnm24hERkcw0qsPP3gKYnfR8TvTa1xUPNLMehFYHTZs23WOHHXbISYAiIvls/nz49NOq32/EMloym42Yz0RKcJ9nmXxOXSaK2Nx9EDAIoEOHDj5hwoQ6jkhEpO6UlcE/UnbWO8fzCHdzNs1ZyA1czUSeyfjz6nLW05dAy6TnW0aviYhIFTp1Sp0ktmAOI+nMw5zILLZhdyZxLX1q9Jl1mShGAidHs5/2AX5w9zW6nUREJCgrgzFjKn/PWEF3BvE+7enIGM7lDvbjbd5nx+iIFSsy/dxsTo99GHgH+K2ZzTGzbmbW08x6RoeMAmYBM4H7gbJsxSIiUsgSCWjWrOqWxLbMZAwdGcQZTGQPduI9+nEuK2hISQkMHw4weXKmn5+1MQp3P6Ga9x04M1ufLyJSDBIJ6NoVli5d872GLKM3d3EdV7KUtTid+xlCN0pLjfvuW/3YLl0yj6EgBrNFROqryy+vPEnsyHsMoRt7MZ6n6UwZ9/Fjsy0YPgD+/vfajUElPERE8tgXX6z+vDG/cDV9mMTubMVnHMcjHL/2U9wyfAsWLqz9JAFKFCIiea1Vq1WP9+JdJrIHfbiWRziedsygpPQ4lvxsWUkQ5ZQoRETy2A03QPNGi7md83iHfWnODxzBs5zMMP5W2mKNsYhs0BiFiEge+/vmr9B5o+6s990s7qOUS7iJxiXrM/yu7HQzVUYtChGRfPT999C9O3TsiDdoyHGbvkYvu4+NWq/PXTlMEqAWhYhI/nn6aSgtZcU339JvrYu4/Jur+Zl1APj8c+jRIxymFoWISH3z3Xdw/PHwl78w59eN2cvf5fylN69MEuV++ilMm80VJQoRkbrmHpZPt20L//43I/e6jq3/N4GJdKjylIrTZrNJiUJEpC7Nng1HHgknnQTbb88z103mL+OvYBlrpTwtedpstmmMQkSkLqxYAQMHwsUXw/Ll0K8f9OpF100b4p76VLMwbTZXlChERHLt44/DjKY33gh1wwcNgq23pqwM/ve/6k/v2TO3s57U9SQikivLlsEtt8Auu8C0afDAAzB69MokkXozolBBdvhwcrLILplaFCIiuTB1Kpx2GkyaBH/9K/TvD5tvHitBlFu4MLshVkUtChGRbPrlF7jySujQAebMgccfhyeeSDtJtG6d3TBTUYtCRCRb3nkHunWDDz6Ak0+GO+6AkpKVbw8cGO8yjRvndvC6IrUoRERq26JFcM45sP/+sHgxPP88PPjgyiRRvmNdnM1JmzULQxm5HLyuSC0KEZHa9NJLocbGZ59Br15w442w3nor3061Y11FpaW5H7iujFoUIiK1YcGCMFj9pz/B2mvDm2/CPfesliQgTG0tpCQBShQiIjX3739Du3bw0ENw6aUwZQoccMDKtxMJaNEiLJRbtCj1pczqZgpsKup6EhHJ1DffwFlnwYgRsOuu8NxzsPvuqx2SzswmMxg2rG7HIyqjFoWISLrcQ+uhXTt45pkwDjFu3GpJonzAOm6SgNyvuI5LLQoRkXR8/jmccQa8+CLstx8MGQI77LDy7XRaEMmGD8/PJAFKFCIi8axYEQYOLrkkPL/nnpAVGqzqmOnUCcaMSf/SpaX5myRAXU8iItX76CM48MAwHnHAAfD++2Hqa5QkyruZMkkSHTvm18B1ZZQoRESqsnQp9O0bivjNmAFDh4bFc0n1NMrKoEuXsK4uHSUlobvp5ZdrN+RsUNeTiEhlJk8O5TcmT4Zjjw1dTZttttohiQQMGJDeZRs2DIu087mrqSK1KEREkv38M1x2Gey5J3z1VSjg9/jjqyWJ8nURXbpQ7SZDyUpKCi9JgFoUIiKr/Oc/cPrpYUyia1e4/XbYcMPVDkmnBAdAkyYweHDhJYdkalGIiCxcGAanf/e7UBZ89OhQia9CkgDo3Tt+kigthSVLCjtJgBKFiNR3L74IO+4Yph6dfTa89x4cfPDKt5PLb5jF26oU8q8MR02o60lE6qf58+Hcc8MK6x12CN1O++232iHpdjOVa9268FsRydSiEJH6Z8QIaNsW/vUvuPzyMLOpQpKA8Fa6SaKuNxnKBrUoRKT++PrrMBbx5JOhLtOLL4ZiflX4/PP0Ll9SAnfdVVytCVCiEJH6wD0sljvvvDD99eabw+NGVf8KTCTCmESc6a8lJTBvXu2Fm2+y2vVkZoea2UdmNtPMLqnk/VZm9qqZTTazaWZ2eDbjEZF66NNPw2ZCp50GO+0EU6fCRRelTBIQup3iJImGDUMrophlLVGYWUOgP3AY0A44wczaVTjsCuAxd98NOB4okjkCIlLnli+Hu+8OM5rGjg1TkF57DbbfPtbpcbqdCnUBXbqy2fW0FzDT3WcBmNkjwFHAjKRjHFg/etwc+CqL8YhIffHBB6H8xjvvwGGHhTobrVrFPr2srOr3ir2bqTLZ7HraApid9HxO9Fqyq4EuZjYHGAWcVdmFzKyHmU0wswlz587NRqwiUgyWLg1TjnbdNayuHjYs7DoXM0lkstlQfVDX02NPAIa6+5bA4cAwM1sjJncf5O4d3L3DxhtvnPMgRaQATJwIHTrAFVfAX/4SWhVduoQR6RgSCTjllOqrwM6fXwuxFphsJoovgZZJz7eMXkvWDXgMwN3fAZoALbIYk4gUmyVL4OKLYe+9Ye5c+Pe/4dFHYZNN0rpM795hWKM6afRgFY1sJorxQBsz29rMGhMGq0dWOOYLoCOAmbUlJAr1LYlIPG+8EfaKuOWWsIR6xozQmkhDeYmOOKU5inExXRxZSxTuvgzoBbwIfECY3fS+mV1rZp2jw84HupvZVOBh4FT3dIr2iki99OOPYcT597+HZcvC7j/33w8bbJDWZTp1Cr1TcZJEs2ahTmCxz3CqTFYX3Ln7KMIgdfJrVyU9ngHsn80YRKTIjBoFPXvCnDmhVtN110HTpmldIpEIyyp+/TXe8aWlxVPgLxN1PZgtIhLPvHlw0klwxBGw3nrw9ttwxx1pJYnyWU1duihJpEOJQkTymzs89hi0awePPAJXXQWTJsE++6R1mbizmpKVlChJgGo9iUg+++qrMBbx9NNh6uvLL8POO6d9mUQiNEbSGQFt3Lj4S3PEpRaFiOQf97B/aLt2ocLrbbeFVdYZJomuXdPf27q+DlxXRi0KEckvs2ZB9+7wyithVtPgwbDddhlfLp09JTp2DI0WWZ1aFCKSH5YvhzvvDEX8xo+HgQNDsqhBkigri1fcr1mzsHWpkkTl1KIQkbr3/vuhiN+774ZZTQMGwJZbpnWJRCKsro67p3U5zWqqnloUIlJ3fv0Vrr0WdtsNPvkkbE36zDMZJYlTTlGSyBa1KESkbowfH1a9TZ8OJ54I/fpBhkU/e/aMV6cp2fDhGqyOSy0KEcmtn36CCy4I6yAWLICRI0OTIIMkUb6AbtGi9M5r3VpJIh1qUYhI7rz2Gpx+euhmOuOMsHd18+YZXapTJxgzJv3zGjWqn4X9aqLaFoWZ3WJm65vZWmY2xszmmlmXXAQnIkXihx9CYvjDH8LzV14JA9YZJIlEAtZeO7Mk0aQJDB2q1kS64nQ9/cndfwSOBD4DtgMuzGZQIlJEnn0W2rcP6yEuuACmTVuVMNJUVpZenaZyJSVhTGLJEiWJTMRJFOXdU0cAj7v7D1mMR0SKxdy5YZD6z3+GDTcMK6tvvRXWXTejy5WVpbdFaWlpWI3tHuoJKkFkLk6ieNbMPgT2AMaY2cbAz9kNS0QKlnuY5tq2LYwYAddcE7Yp3WuvjC6XyT7WmvZau6odzHb3S8zsFuAHd19uZj8BR2U/NBEpOHPmhN/Szz4btiYdMiR0O2Ug3RYEhDGIwYPVeqhtcQaz1wXKgPL/ZL8BOmQzKBEpMCtWhJIb7dqFUeY77oC33sooSZQPVqebJDp21BhEtsTpevon8CuwX/T8S+D6rEUkIoVl5szwW7pnT9hzz7CA7txzoWHDtC5TVgZm6Q9Wq05T9sVZR7Gtux9nZicAuPtPZmZZjktE8t2yZWE19ZVXhs0b7r8/1GtK89dDutuSJmvaFBYuTP88SU+cRPGrma0DOICZbQv8ktWoRCS/vfdeSArjx0PnzmHkeIst0r5M+V4RccuAJ2vYMPR2SfbF6XrqA7wAtDSzBDAGuCirUYlIfvrlF+jTB3bfHT77DB59FJ56KqMkAaHaayZJoqQEHnxQ4xG5EmfW00tmNgnYBzCgt7vPy3pkIpJfxo4NrYgZM8JAQr9+4Td2hsrK0qv2qhlNdafKFoWZ7RB93x1oDXwNfAW0il4Tkfpg8WI47zzYbz/48Ud47jkYNqxGSSKRiD+rqXywWjOa6k6qFsV5QA/g9krec+CPWYlIRPLHmDFhW9JPPw3rI266CdZfv0aXTCTgpJOqP04tiPxRZaJw9x7Rw8PcfbWV2GbWJKtRiUjd+v57uPDC8Ju6TRt4/XU48MCML5fO7nMNGsBDDylB5JM4g9lvx3xNRIrB00+HhXMPPAAXXQRTp9YoSZQX8os7HqEkkX+qbFGY2WbAFsA6ZrYbYSAbYH0gs6peIpK/vv0Wzj4bHnsMdt45bCjUIbMiDGVloYq4e3rnNW2qJJGPUo1RHAKcCmwJ3JH0+kLgsizGJCK55L6qb2jRIrj++tCSWGutjC6XSY2mcloXkZ9SjVE8CDxoZse4+xM5jElEcuWLL0Lpjeefh333DUX82rZN6xKJRNiTaPHimoVSWqrWRL5K1fXUxd2HA1uZ2XkV33f3Oyo5TUQKQXkRv4suCo/vugvOPDOj+kyZth7KNWsWuqmUJPJXqq6nptH3ZrkIRERy5OOPw77Vb74ZNp4eNAi23jqtSyhB1C+pup4GRt+vyV04IpI1y5aF8t99+oRFCg88AKeeGruIXzpTXFNRgig8qbqe7k51orufXfvhiEhWTJ0aSrROmgR//Sv07w+bbx7r1JpUd01WUhJ6uJQgCk+qrqeJOYtCRLLj55/DLKabbw6/qUeMgGOOSXlKbQ1Ol2vcODRelCAKV3WznmrEzA4F7gIaAoPd/aZKjvkbcDWhLMhUdz+xpp8rIsDbb4cifh9+CKecErqdNtoo5SmJBJx8chjfrg1qRRSHVF1P/dz9HDN7hmgvimTu3jnVhc2sIdAfOBiYA4w3s5HuPiPpmDbApcD+7r7AzDbJ8OcQkXKLFsHll8M990DLlvDCC3DIIbFO7d27ZkmitDRsTSHFJVXX07Do+20ZXnsvYKa7zwIws0eAo4AZScd0B/q7+wIAd/8uw88SEYDRo6FHD/j8c+jVC268EdZbL9apiUTmA9Uq4Ffcqqz15O4To++vA+8AC4D5wDvRa9XZApid9HxO9Fqy7YHtzewtMxsbdVWtwcx6mNkEM5swd+7cGB8tUs8sWBC2ijvkkPBb+803Q4siZpKA0JpIV0mJSoDXB9UWBTSzI4BPgLuBe4GZZnZYLX1+I6ANcBBwAnC/mW1Q8SB3H+TuHdy9w8Ybb1xLHy1SJJ58MhTxGzYMLr0UpkyBAw6IfXoiEaasxm1NmIUuJneYN08Joj6Is2f27cAf3H0mrNwz+zng+WrO+xJomfR8y+i1ZHOAd919KfCpmX1MSBzjY8QlUr99803oXnriCdh1Vxg1CnbbLa1LJBJhnHv58uqP1cB0/RWnzPjC8iQRmUUoDFid8UAbM9vazBoDxwMjKxzzFKE1gZm1IHRFzYpxbZH6yz1sGN2uHTz7bBiHGDcu7SQBobupuiQxfLhaD/VdqllPR0cPJ5jZKOAxwuyn/yPGX/zuvszMegEvEqbHPuDu75vZtcAEdx8ZvfcnM5sBLAcudPcarvsUKWKffx4WObz4Iuy/fxhB3mGHjC4VZ/C6pETJQVJ3Pf056fG3wO+jx3OBdeJc3N1HAaMqvHZV0mMnbLm6RtFBEUmyYkWYd3rJJeH5PfeEgksN4nQKrC7ugjqz0NUkkmrBXddcBiIiVfjww1DE7623wqymgQOhdeuMLpVIhMlRS5dWf2zPnmpNSFDtYHa0P3Y3oD2wcq9sdz8ti3GJyNKlcOutcM01Yeu3Bx+Ek06KXcQvWboF/bRwTpLFabcOAzYj7Hj3OmH2UpzBbBHJ1OTJsNdeYYV1587wwQehtkYGSSLdPatLSpQkZHVxEsV27n4lsDiq/3QEsHd2wxKpp37+OayF2HPPMP31iSfg8cdh000zuly6+0Y0bqxxCVlTnHUU5b2Z35vZjsA3gGoyidS2//wnFPH7+OMwkHD77bDhhhlfLpFIL0lonwipSpxEMcjMNgSuJKyDaBY9FpHasHBhaEX07w9bbRXqNR18cI0vm05JjpKSsE5CpDLVdj25+2B3X+Dur7v7Nu6+SfnudyJSQy+8ADvuGAYFeveG996rcZJItySHupukOnFqPZWY2T1mNsnMJppZPzMryUVwIkXrf/8LtTMOOyzMaHrrLejXL/yGr4Hygeu4mw6VlGhTIalenMHsR4DvgGOAY4F5wKPZDEqkaLmHXebatYN//QuuuCLMcNp33xpfOu7AdcOGKssh6YkzRrG5u1+X9Px6MzsuWwGJFK2vv4Yzz4R//xv22COMReyyS40vm87MpgYNwnIMJQdJR5wWxWgzO97MGkRffyPUaBKRONzhn/8MrYjnnw/7V48dW+MkkUjA2mvHTxJm8NBDShKSvlRFARcSigAacA4wPHqrAbAIuCDr0YkUuk8/DTvOvfwy/O53oYjf9tvX+LLpro8AleSQzKWq9RR/aywRWd3y5XDvvXDZZWFQ4L77QiW+DIr4VdSpE4wZk945KskhNRFnjAIz6wwcGD19zd2fzV5IIgVuxoxQxO+dd8KspoEDoWXL6s+rQrp1mpJpEZ3UhjjTY28CegMzoq/eZtY324GJFJylS+H668MGQh9/HKYWPfdcjZNE166ZJYnS0rCWT0lCaipOi+JwYFd3XwFgZg8Ck4FLsxmYSEGZOBFOOw2mTYPjjoO774ZNal7ppmfPeCXBkzVpEoZClCCktsTtMN0g6XHzbAQiUpCWLIGLLw6VXufOhaeegkceqZUk0akTLFoU//hmzUIjZskSJQmpXXFaFDcCk83sVcIMqAOBS7IalUgheP116N4d/vvfMCZx662wwQbVn5dCpuMRHTuGiVUi2ZAyUZhZA2AFsA+wZ/Tyxe7+TbYDE8lbP/4YWhEDBsA224Tf0B071viymUx5BSUJyb6UXU/RuMRF7v61u4+MvpQkpP4aNQrat4dBg+C888KYRIZJorx4n1n4SjdJlJSEriYlCcm2OGMUL5vZBWbW0sw2Kv/KemQi+WTevFBt74gjYP314e23w34RTZvGOj2RgBYtViUFs/SK9yUrLVWdJsmtOGMU5XWdzkx6zYFtaj8ckTzjDo89BmedBQsWQJ8+Ye+ItdeOfYlMu5Qq07GjFs5J7lWbKNx961wEIpJ3vvwy/JYfORI6dAjLoXfaKa1LZLKKuipaXS11pcquJzPb28ymmtkiM3vHzNrmMjCROuMO998fiviNHg233RZWWadIEhXHG8q/aiNJlE97VZKQupJqjKI/ofBfCXAH0C8nEYnUpU8+Cf07PXrA7ruHHefOPx8arWp81+Z4QyrlCUKrq6WupUoUDdz9JXf/xd0fBzbOVVAiObd8OdxxR2g1TJwY6jONGQPbbbfykLKyVUkhk5IacZXPZlKCkHyRaoxiAzM7uqrn7v5k9sISyaHp06FbNxg3Do48Mow8b7nlaofU5lhDRSUlYc9qJQXJV6kSxevAn6t47oAShRS2X3+Fvn3hhhugefOwNenxx4dmQ5KystpNElogJ4Um1X4UXXMZiEhOjRsXWhHTp8OJJ0K/frDx6r2rtTmttZyShBSimu+iIlJIfvoJLrgA9t03rIsYOTKMTicliXS3GI1Dq6ilkMXauEikKLz6aijeN2tW2G3u5ptDl1OkNloQGm+QYqREIcXvhx/gootCfaZttw0J46CDVjskk8Fq7fsg9UWcHe7WNbMrzez+6HkbMzsy+6GJ1IJnngkL5wYPDl1O06atkSQyGawuLdW+D1J/xBmj+CfwC7Bv9PxL4PqsRSRSG+bODYPUnTuH/qCxY8N+Eeuuu/KQ8tXU6XY3qZSG1DdxEsW27n4LsBTA3X8ibGAkkn/cwzTXtm1hxAi45hqYMAH23HO1wxIJOOWU9FZTN2miUhpSP8VJFL+a2TqEtROY2baEFka1zOxQM/vIzGaaWZW74pnZMWbmZtYhVtQilZk9G/7859AftN12MHkyXHUVNG68RtmNLl3CYuw4tMWo1HdxEkUf4AWgpZklgDHARdWdZGYNCfWiDgPaASeYWbtKjlsP6A28m0bcIqusWBFKbrRvHwaq77wT3norPCeMQWRSdqO8BaFSGlLfxSkz/pKZTSJsh2pAb3efF+PaewEz3X0WgJk9AhwFzKhw3HXAzcCF6QQuAoT9qrt3D/tXd+wYZjZts03Ge0+X0ziEyCpVJgoz273CS19H31uZWSt3n1TNtbcAZic9nwPsXclntHT358ysykRhZj2AHgCtWrWq5mOlXli2LKymvvLKsDpu8GASa59G772sxgX7lCREVpeqRXF7ivcc+GNNPtjMGhDKl59a3bHuPggYBNChQwevyedKEZg2LZTfmDABjjoK7ruPsut/UysrqZUkRNaUqtbTH2p47S+BlknPt4xeK7cesCPwmoUibJsBI82ss7tPqOFnSzH65ZdQwK9vX9hwQ3j0Ufi//6PsTFOSEMmiascozKwJUAYcQGhJvAkMcPefqzl1PNDGzLYmJIjjgRPL33T3H4AWSZ/zGnCBkoRUauxYvj+2Gxt8OYNhdOGcuf2Yf1zJqh3da0BlN0RSizPr6SGgPXAPcG/0eFh1J7n7MqAX8CLwAfCYu79vZteaWefMQ5Z6ZfFiOPdcVuy7Hz9+uZDDeY6TGcZ8SjK+ZPl0V/fwNW+ekoRIKnFqPe3o7snTWl81s4ozlyrl7qOAURVeu6qKYw+Kc00pfolEqNm39+Ix3E93tuFT/kEZl9KXhayf8XWbNYMBA5QURNIVp0Uxycz2KX9iZnsD6h6SWlNxMdyZXb6n3+LTGUMnltGIA3mdXvTPOEloa1GRmonTotgDeNvMvoietwI+MrP3AHf3nbMWnRS98lIa5aukj+Ip7qOMTfiOm7iYa+jDz6yT9nUbNoQHH1RiEKkNcRLFoVmPQuqdigviNuFb7uEs/sbjTGEX/swzTGKPjK6tLiaR2hVnZfbnZrYhYapro6TXq1twJ7KGRAJOOy1sVx04XRhOP86hGYu4nOu5hYtYxlppX1uzl0SyI8702OsIi+I+ISoMSC0suJP6o6pyGi35ggH05HCe5232pRtD+JC2sa6priWR3InT9fQ3QqnxX6s9UiSSqtaSsYKeDOBmLsZwzuJu7qOMFTSMdW21HERyK06imA5sAHyX5VikCKzZtbS6NnzMYE7nQN5kNAfTg0F8zlaVHquEIJIf4iSKvsBkM5tO0j4U7q5Fc7KaRAJOPjlU/a6oIcs4n9u5hj4sYR1O5Z88yClUtgeWSmmI5Jc4ieJBQhnw94BKfgWIBL17V54kdmEKQ+jGHkziSf7KmfTnGzZf4zi1IETyU5xE8ZO73531SKQgla+irmxL0bX5mSu5jou5mXm04BhG8CTHrHaMGfTsqRaESD6LkyjeNLO+wEhW73rS9Nh6rqyMKqu27svbDKEbbfmQoZzCedzBAjZa+b7WOogUjjiJYrfo+z5Jr2l6bD2WqhXRlEXcyGX04l5m05JDeIHRHLLyfXUviRSeOAvuarovhRSBOFuLHsxoBtGDVnxBf87kMm5kEetpcFqkwMVpUWBmRxDKizcpf83dr81WUJJfUnUxAWzIfG7nfLoylA/5LQfyBm9xAGYwfJhaDyKFLs7K7AHAusAfgMHAscC4LMcleaJTJxgzpur3j+YJ+nMmLZjHjVzKtVzFL9HfEz17KkmIFIM4Zcb3c/eTgQXufg2wL7B9dsOSfJAqSWzKNzzOsTzBsXzN5uzJeC7nRn6hycqNgdTdJFIc4iSKJdH3n8zsN8BSqGQSvBSF5L0hKk8SzikMZQbtOJJnuYS+7MU4prAbpaVhxzjt+yBSXOKMUTxrZhsAtwKTCDOe7s9qVFInKu4NUVFrPmMgZ3AIo3mTAzidwXzMb2nSBIYOVnIQKVZxZj1dFz18wsyeBZq4+w/ZDUtyLZGAk04KLYKKjBWcSX/6cimOcSb38g9KsQYNKD1DXUwixa7KRGFmewKz3f2b6PnJwDHA52Z2tbvPz1GMkiWp1kOU+y0fMpjTOYC3eIFDOIOBtOnYmhUv5y5OEalbqcYoBgK/ApjZgcBNwEPAD8Cg7Icm2VRWBl26VJ0kGrGUS7mRqexCO2ZwMg9yGM/TpmNrXlaSEKlXUnU9NUxqNRwHDHL3JwhdUFOyH5pkQ3VrIgB2YxJD6MZuTOEx/o+zuIfv2FQL50TqqVQtioZmVp5IOgKvJL0Xa6Ge1L2yMmjQIMxiMkudJJqwhBu5lHHsxWZ8w195kuN4jJ+abarpriL1WKpf+A8Dr5vZPMIU2TcBzGw7QveT5Kk45TYq2p//MIRu/JaPGcJpXMBtfM+GakWISNWJwt1vMLMxhDUTo91XzodpAJyVi+AkfXG6lpI1YyF9uZRe9OdTtqITLzGGTshoPtEAABFoSURBVGHRnKq7igjVdCG5+9hKXvs4e+FIpqrbgrQyh/I8AzmDLZlDP3pzBddjzZopQYjIajTWUODiTHGtaCP+x52cy8kMYwZt2Z+32K10Xxapi0lEKqFEUcCqK9i3JudYRnAvvdiI+VzLldy+9uXcN2RttSBEpEpxaj1JHqm+FlPlNuNrnuRoHudvzKYlnZpPYNvh1/LDz0oSIpKaWhQFIJNZTKs4Xfknd9p5NG30C9xwCx3OPZfXG+k/vYjEo98WeSqTsYeKtmYWj290BnvMfxl+dyDcfz9srwrxIpIedT3liUQCmjVbtTAuVXmN6jRgOefQjw8a7cQeS98N82VffVVJQkQyohZFHSorgwEDKq/YmqldG89gdKtubDxzLBx8GAwcCC1b1t4HiEi9oxZFHSlfGFdbSWItfuWBba5jMrux8YL/hi3mnntOSUJEaiyricLMDjWzj8xsppldUsn755nZDDObZmZjzKx1NuPJJwMH1t61OjafwLet9qTrrKvg6KNhxoywYs6s9j5EROqtrCUKM2sI9AcOA9oBJ5hZuwqHTQY6uPvOwAjglmzFk0/KymDFippdo0EDOLv7EvzCi3h54d5suGwePP00PPwwbLJJ7QQqIkJ2WxR7ATPdfZa7/wo8AhyVfIC7v+ruP0VPxwJbZjGeOpW8/iGdWkzJmjULPUrusPyV17nr1Z3h1luhWzd4/33o3Ll2gxYRIbuJYgtgdtLzOdFrVekGPF/ZG2bWw8wmmNmEuXPn1mKI2Vc+m6lLl8zWQZSUrEoOCxfC3//8I5SWwkEHhWbJmDEwaBBssEGtxy4iAnkymG1mXYAOwK2Vve/ug9y9g7t32HjjjXMbXIbKyjKb5pqcGNxh3rykAn3PPQft24fEcN55MG0a/PGPWYlfRKRcNqfHfgkkT7nZMnptNWbWCbgc+L27/5LFeHIm/RpMIakMG1ZF1dZ58+Ccc0LzpH17GDEC9t67VmIVEalONlsU44E2Zra1mTUGjgdGJh9gZrsR9ubu7O7fZTGWrMu0BlO5nj0rSRLu8Mgj0LYtPPYY9OkDkyYpSYhITmWtReHuy8ysF/Ai0BB4wN3fN7NrgQnuPpLQ1dQMeNzCVM4v3L3gRmTT3Syookp3kfvyy3DhkSNhzz1hyBDYaacaxSkikomsrsx291HAqAqvXZX0uFM2Pz9baqMOE4RB7gEVNwlyh8GD4YILYOlSuO220O3UsGHNPkxEJEMq4ZGG2koQJSVw112VdDV98gl07x7qMh10UCjit912NfswEZEaUqKIKZMB6mRNmoSGQqWD1cuXh8xxxRWw1lph2fbpp4dVdSIidUyJIoayspoliUrHIMpNnx4WzI0bB0ceGQY7tizadYciUoD0J2slKpb8rulK6kqTxK+/wjXXwO67w6xZofTGyJFKEiKSd9SiSJLVQepk48aFVsT06XDiiaHbqUWLmn2oiEiWqEURSSSga9eaJYnyVdULF1aRJH76Cc4/H/bdFxYsgGeeWbUAQ0QkT6lFQebrIFIOUFf06qthgHrWrNBsuflmaN48/Q8VEcmxep8oMp3N1LQpLFoU48AffoALLwxTXbfddtXUVxGRAlEvu55qWm6jceOYGw898wy0axdWVV94YSjipyQhIgWmXiWKmpb8hjAO8cAD1XQ3zZ0LJ5wQ9ocoKYF334VbboF1183sQ0VE6lC96HrKdDZTgwbw0EMxxyAglN94+GE4+2z48Ue49lq4+OLQBBERKVBFnyjKZzMtXZreeY0awdChaSSJ2bPDyrrnngvVXYcMCSXBRUQKXNF3PV1+efpJokmTNJLEihVh0UT79mGg+s474a23lCREpGgUfYvi88/TOz5luY2K/vvfUMTv9dehY8ew89w226Qdo4hIPivqFkUiEf/YlOU2Klq2DG69FXbeGaZMCYspXnpJSUJEilJRtyguv7z6Y6os+V2VadNC+Y0JE+Coo0Jm+c1vahSniEg+K9pEkUik7nYqKQlbUcf2yy9www3Qty9stFHYmvTYY8NiDBGRIlaUiaK61dZmoRUR2zvvhFbEBx/ASSeFAeuSkhrHKSJSCIpujCJOSY6ePWN2NS1eHLYh3X//UK9j1KiwsEJJQkTqkaJqUcTdYCjWgPXLL4cZTZ99Fi7cty+sv35NQxQRKThF06JIJMJyhuq0bl3NAd9/H7qZDj44bEv6+uvQv7+ShIjUW0WRKBIJOPnkUEEjlcaNw3h0lZ56KhTxe/BBuOQSmDoVDjywVmMVESk0Bd/1VF6iY8WK1Mel3Dvi22/hrLPg8cdhl11C1dc99shKvCIihabgE0WcEh0dO4YhhzW4w7BhYcB68eLQ3LjwwtDlJCIiQBF0PX3xRer3S0urSBJffAGHHw6nnAI77BBWWF92mZKEiEgFBZ8oNtqo6vdKSiqZ4bRiRRicbt8e3nwT7r47fG/bNqtxiogUqoLueiorq3oDokaNKllU99FHYd/q//wnzGoaNAi22irbYYqIFLSCbVFUNx22efOkgetly+Cmm8JA9fTp8M9/wosvKkmIiMRQsC2K3r1TT4edPz96MGVKWBcxaRIcfXTodtpss5zEKCJSDAqyRZFIVL/ndZuWP4cpUR06wJdfwogR8MQTShIiImkqyERRXfnw3zd6i3FLd4Ubb4QuXWDGDDjmmNwEJyJSZAouUXzxRdXlw5uyiPvWOptXl/+O5mstgRdeCHuappoaJSIiKRVcopg7t/LXD2Y0MxrsSOmye7FevcKg9SGH5DY4EZEiVHCJoqINmc8DdGU0h9B80yar1kast15dhyYiUhQKOlEczRPMoB0nMYwbuIzms6aEvSNERKTWZDVRmNmhZvaRmc00s0sqeX9tM3s0ev9dM9sqznU342tGcAxPcCxf8Rs6MIH7W98QKv+JiEityto6CjNrCPQHDgbmAOPNbKS7z0g6rBuwwN23M7PjgZuB41Jdt4T/MYN2rMMSLqEvt3M+NFqLoanKh4uISMay2aLYC5jp7rPc/VfgEeCoCsccBTwYPR4BdDQzS3XRrfiM6ezILkzlZi5hua3F0KExtzYVEZG0ZXNl9hbA7KTnc4C9qzrG3ZeZ2Q9ACTAv+SAz6wH0CM9KOJAlQMgM7tCly8SJXbrU/g9QAFpQ4V7VY7oXq+herKJ7scpvMz2xIEp4uPsgYBCAmU1wn9ehjkPKC+FeuO4FuhfJdC9W0b1YxcwmZHpuNruevgRaJj3fMnqt0mPMrBHQHKimOIeIiORSNhPFeKCNmW1tZo2B44GRFY4ZCZwSPT4WeMW9up2vRUQkl7LW9RSNOfQCXgQaAg+4+/tmdi0wwd1HAkOAYWY2E5hPSCbVGZStmAuQ7sUquher6F6sonuxSsb3wvQHvIiIpFLQK7NFRCT7lChERCSlvE0U2Sr/UYhi3IvzzGyGmU0zszFm1rou4syF6u5F0nHHmJmbWdFOjYxzL8zsb9G/jffN7F+5jjFXYvw/0srMXjWzydH/J4fXRZzZZmYPmNl3Zja9ivfNzO6O7tM0M9s91oXdPe++CIPfnwDbAI2BqUC7CseUAQOix8cDj9Z13HV4L/4ArBs9Lq3P9yI6bj3gDWAs0KGu467DfxdtgMnAhtHzTeo67jq8F4OA0uhxO+Czuo47S/fiQGB3YHoV7x8OPA8YsA/wbpzr5muLIivlPwpUtffC3V9195+ip2MJa1aKUZx/FwDXEeqG/ZzL4HIszr3oDvR39wUA7v5djmPMlTj3woH1o8fNga9yGF/OuPsbhBmkVTkKeMiDscAGZrZ5ddfN10RRWfmPLao6xt2XAeXlP4pNnHuRrBvhL4ZiVO29iJrSLd39uVwGVgfi/LvYHtjezN4ys7FmdmjOosutOPfiaqCLmc0BRgFn5Sa0vJPu7xOgQEp4SDxm1gXoAPy+rmOpC2bWALgDOLWOQ8kXjQjdTwcRWplvmNlO7v59nUZVN04Ahrr77Wa2L2H91o7uvqKuAysE+dqiUPmPVeLcC8ysE3A50Nndf8lRbLlW3b1YD9gReM3MPiP0wY4s0gHtOP8u5gAj3X2pu38KfExIHMUmzr3oBjwG4O7vAE0IBQPrm1i/TyrK10Sh8h+rVHsvzGw3YCAhSRRrPzRUcy/c/Qd3b+HuW7n7VoTxms7unnExtDwW5/+RpwitCcysBaEralYug8yROPfiC6AjgJm1JSSKuTmNMj+MBE6OZj/tA/zg7l9Xd1Jedj159sp/FJyY9+JWoBnweDSe/4W7d66zoLMk5r2oF2LeixeBP5nZDGA5cKG7F12rO+a9OB+438zOJQxsn1qMf1ia2cOEPw5aROMxfYC1ANx9AGF85nBgJvAT0DXWdYvwXomISC3K164nERHJE0oUIiKSkhKFiIikpEQhIiIpKVGIiEhKShSSNWZWYmZToq9vzOzL6PH30ZTNXMbyFzNrl/T82miRYrrX2aqyypzR60uin2+GmQ2IVorHve7VZnZBTeOI3htc/rOa2WfRGgrM7O2kc09MOr6Dmd0d97Ol/lGikKxx9/+5+67uviswALgzerwrUOulE6IV+lX5C6FqaHlsV7n7y7UcwifRz7dz9Fl/SSO+WuPup7v7GonY3feLHm4FnJj0+gR3PzsXsUlhUqKQutLQzO6P9kkYbWbrAJjZtmb2gplNNLM3zWyH6PWtzOwVW7XnRqvo9aHRX+/vArdUdr6Z7Qd0Bm6N/uLfNjrv2Ogae5rZ22Y21czGmdl60ee9aWaToq/9qvg51hAVqXwb2M7MTjWzkWb2CjDGzDYys6ein2Osme2cdOouZvaOmf3XzLpHsTWLft5JZvaemSVXRW1kZgkz+8DMRpjZutE5r1klZUvMbFH08Cbgd9G9ONfMDjKzZ6NjmlrY02Cchb0bjopebx+9NiWKvRhLgUhV6rp+ur7qxxeheucF0eOtgGXArtHzx4Au0eMxQJvo8d6E0iwAzwCnRI9PA56KHg8FngUaVnP+UODYpHiGEkq/NCaUtdgzen19QsWCdYEm0WttCCt8y2Nfo9Z/8uvRueOBwwgFCucAG0Xv3QP0iR7/EZiSdH+mAusQahDNBn4TxbJ+dEwLwopaiz7Pgf2j9x5Iur+vEe3DAXwGtIgeL4q+HwQ8mxT7yufAjUn/LTYg1IdqGsX99+j1xsA6df1vSl+5+8rLEh5SL3zq7lOixxOBrcysGbAfq0qRAKwdfd8XODp6PAy4Jelaj7v78mrOr8pvga/dfTyAu/8I4S9r4F4z25VQ/mL7GD/TtmY2hfAL/Gl3f97MTgVecvfyPQIOAI6JPuuVaBynfJ+Ep919CbDEzF4l7LPwHHCjmR1I6K7bAtg0On62u78VPR4OnA3cFiPOVP4EdE4aL2kCtALeAS43sy2BJ939vzX8HCkgShRSV5Ir3C4n/CXdAPjeQz9/OhZH3zM9vzLnAt8Cu0TXjbMJ0idVfPbiSl6rTMV6Og78HdgY2MPdl1qoitskxfE1ZcAx7v5Rhdc/iLr3jgBGmdkZ7v5KLXyeFACNUUjeiP6a/9TM/g9W7u+7S/T226wq/Ph34M00z19IKENe0UfA5ma2Z3TOeraqbP3XHvYrOIlQbK42vBnFj5kdBMwrb8UAR5lZEzMrIXQHjY/i+C5KEn8AkvdDb2VhbwUIg9P/iRlDVfcCQmG9syxqklmoTIyZbQPMcve7gacJA/ZSTyhRSL75O9DNzKYC77NqS8uzgK5mNo3wi7t3muc/AlwYDdBuW36wh60zjwPuic55ifAX+33AKdFrOxC/VVCdq4E9op/jJlaVygeYBrxKKI9+nbt/BSSADmb2HnAy8GHS8R8BZ5rZB8CGwD9ixjANWB4N3p9b4b3rCNVGp5nZ+9FzgL8B06OutR2Bh2J+lhQBVY8VEZGU1KIQEZGUlChERCQlJQoREUlJiUJERFJSohARkZSUKEREJCUlChERSen/Aev5GncusP2UAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "tags": [], + "needs_background": "light" + } + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "rzyVsxoCYlhw", + "colab_type": "code", + "outputId": "65ee71c9-f172-4bfb-8fd9-c68fae115584", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 119 + } + }, + "source": [ + "random = []\n", + "# generate random inputs\n", + "for i in range(X.shape[1]):\n", + " random.append(uniform(X.iloc[:, i].min(), X.iloc[:, i].max()))\n", + "# print the random inputs generated\n", + "print('random input:', random)\n", + "random.insert(0, 1) # insert 1 at the front\n", + "X_aug = sm.add_constant(X) # insert 1 at the front\n", + "est = sm.OLS(y, X_aug).fit()\n", + "prediction = est.get_prediction(np.reshape(random, (1, 6)))\n", + "pd.set_option('display.max_columns', None)\n", + "print(prediction.summary_frame(alpha = 0.05))" + ], + "execution_count": 82, + "outputs": [ + { + "output_type": "stream", + "text": [ + "random input: [2013.3002540225928, 19.80124894778612, 2.2810358454108703, 24.979226201571294, 121.53197772366114]\n", + " mean mean_se mean_ci_lower mean_ci_upper obs_ci_lower \\\n", + "0 38.346671 0.809233 36.755885 39.937457 20.070918 \n", + "\n", + " obs_ci_upper \n", + "0 56.622423 \n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "LxUcy5kfNFdl", + "colab_type": "code", + "outputId": "a9950dbc-b29b-446b-e2c1-3c3a687c3d38", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 544 + } + }, + "source": [ + "print(est.summary())" + ], + "execution_count": 83, + "outputs": [ + { + "output_type": "stream", + "text": [ + " OLS Regression Results \n", + "==============================================================================\n", + "Dep. Variable: y R-squared: 0.542\n", + "Model: OLS Adj. R-squared: 0.537\n", + "Method: Least Squares F-statistic: 96.68\n", + "Date: Mon, 08 Jun 2020 Prob (F-statistic): 5.12e-67\n", + "Time: 23:11:27 Log-Likelihood: -1505.9\n", + "No. Observations: 414 AIC: 3024.\n", + "Df Residuals: 408 BIC: 3048.\n", + "Df Model: 5 \n", + "Covariance Type: nonrobust \n", + "===================================================================================================\n", + " coef std err t P>|t| [0.025 0.975]\n", + "---------------------------------------------------------------------------------------------------\n", + "const -4.231e+04 5333.483 -7.933 0.000 -5.28e+04 -3.18e+04\n", + "X1 transaction date 4.2098 1.621 2.598 0.010 1.024 7.395\n", + "X2 house age -0.2797 0.040 -6.949 0.000 -0.359 -0.201\n", + "X4 number of convenience stores 1.5658 0.183 8.558 0.000 1.206 1.925\n", + "X5 latitude 337.6294 42.654 7.916 0.000 253.780 421.479\n", + "X6 longitude 209.3382 34.696 6.033 0.000 141.132 277.544\n", + "==============================================================================\n", + "Omnibus: 240.224 Durbin-Watson: 2.171\n", + "Prob(Omnibus): 0.000 Jarque-Bera (JB): 3935.913\n", + "Skew: 2.113 Prob(JB): 0.00\n", + "Kurtosis: 17.502 Cond. No. 2.36e+07\n", + "==============================================================================\n", + "\n", + "Warnings:\n", + "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n", + "[2] The condition number is large, 2.36e+07. This might indicate that there are\n", + "strong multicollinearity or other numerical problems.\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "dBp3VR4j5nhs", + "colab_type": "code", + "outputId": "d35dfc91-2118-404b-8807-ee285e3f5e4d", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 102 + } + }, + "source": [ + "X_ = dataset[['X1 transaction date', 'X2 house age', 'X3 distance to the nearest MRT station', 'X4 number of convenience stores', 'X5 latitude', 'X6 longitude']]\n", + "X_aug_ = sm.add_constant(X_) # augmented regressor matrix with a column of ones inserted at the front\n", + "min_r_cv = 0\n", + "# calculate the initial risk estimator with all predictors included\n", + "for i in range(X_.shape[0]):\n", + " X_cv = X_aug_.drop(X_aug_.index[i])\n", + " X_cv.reset_index(inplace=True)\n", + " X_cv = X_cv.drop('index', axis=1)\n", + " y_cv = np.delete(y, i)\n", + " est_cv = sm.OLS(y_cv, X_cv).fit()\n", + " prediction_cv = est_cv.get_prediction(X_aug_.iloc[[i]]).predicted_mean\n", + " min_r_cv += (y[i] - prediction_cv)**2\n", + "print('original risk estimator:', min_r_cv)\n", + "\n", + "variables = X_.shape[1]\n", + "min_index = -1\n", + "# keep deleting a predictor until we can't improve the model anymore\n", + "while (min_index != 0):\n", + " min_index = 0\n", + " # calculate the risk estimator when the j-th predictor is removed\n", + " for j in range(1, variables + 1):\n", + " r_cv = 0\n", + " for i in range(X_.shape[0]):\n", + " X_cv = X_aug_.drop(X_aug_.index[i]) # remove the i-th data point\n", + " X_cv.reset_index(inplace = True)\n", + " X_cv = X_cv.drop('index', axis = 1)\n", + " X_cv.drop(X_cv.columns[[j]], axis=1, inplace=True) # remove the j-th column\n", + " y_cv = np.delete(y, i) # remove the i-th data point\n", + " # refit the model and make the prediction\n", + " est_cv = sm.OLS(y_cv, X_cv).fit()\n", + " X_aug_copy = X_aug_.iloc[[i]].copy()\n", + " X_aug_copy.drop(X_aug_copy.columns[[j]], axis=1, inplace=True)\n", + " prediction_cv = est_cv.get_prediction(X_aug_copy).predicted_mean\n", + " # the risk estimator is calculated as the sum of the squared differences between each removed point and the prediction for this point\n", + " r_cv += (y[i] - prediction_cv)**2\n", + " # update the optimal predictor to drop if the risk estimator becomes lower\n", + " if (r_cv < min_r_cv):\n", + " min_index = j\n", + " min_r_cv = r_cv\n", + " # update the regressor matrix and the number of variables if deleting a predictor leads to a better model\n", + " if (min_index != 0):\n", + " X_aug_.drop(X_aug_.columns[[min_index]], axis=1, inplace=True)\n", + " variables -= 1\n", + "print('improved risk estimator:', min_r_cv)\n", + "print('remaining predictors:', X_aug_.columns.values)" + ], + "execution_count": 84, + "outputs": [ + { + "output_type": "stream", + "text": [ + "original risk estimator: [33075.17492263]\n", + "improved risk estimator: [32951.59783497]\n", + "remaining predictors: ['const' 'X1 transaction date' 'X2 house age'\n", + " 'X3 distance to the nearest MRT station'\n", + " 'X4 number of convenience stores' 'X5 latitude']\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "4n6OUTHbGLcI", + "colab_type": "code", + "outputId": "ddd641c6-a17a-402d-e62d-3b814a4d925a", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 544 + } + }, + "source": [ + "est_ = sm.OLS(y, X_aug_).fit()\n", + "print(est_.summary())" + ], + "execution_count": 85, + "outputs": [ + { + "output_type": "stream", + "text": [ + " OLS Regression Results \n", + "==============================================================================\n", + "Dep. Variable: y R-squared: 0.582\n", + "Model: OLS Adj. R-squared: 0.577\n", + "Method: Least Squares F-statistic: 113.8\n", + "Date: Mon, 08 Jun 2020 Prob (F-statistic): 4.47e-75\n", + "Time: 23:12:05 Log-Likelihood: -1487.0\n", + "No. Observations: 414 AIC: 2986.\n", + "Df Residuals: 408 BIC: 3010.\n", + "Df Model: 5 \n", + "Covariance Type: nonrobust \n", + "==========================================================================================================\n", + " coef std err t P>|t| [0.025 0.975]\n", + "----------------------------------------------------------------------------------------------------------\n", + "const -1.596e+04 3233.450 -4.936 0.000 -2.23e+04 -9602.961\n", + "X1 transaction date 5.1348 1.555 3.303 0.001 2.079 8.191\n", + "X2 house age -0.2694 0.038 -7.003 0.000 -0.345 -0.194\n", + "X3 distance to the nearest MRT station -0.0044 0.000 -8.887 0.000 -0.005 -0.003\n", + "X4 number of convenience stores 1.1361 0.188 6.056 0.000 0.767 1.505\n", + "X5 latitude 226.8816 44.174 5.136 0.000 140.044 313.719\n", + "==============================================================================\n", + "Omnibus: 232.810 Durbin-Watson: 2.154\n", + "Prob(Omnibus): 0.000 Jarque-Bera (JB): 3644.713\n", + "Skew: 2.038 Prob(JB): 0.00\n", + "Kurtosis: 16.953 Cond. No. 1.77e+07\n", + "==============================================================================\n", + "\n", + "Warnings:\n", + "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n", + "[2] The condition number is large, 1.77e+07. This might indicate that there are\n", + "strong multicollinearity or other numerical problems.\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "FD2sFRnOGNR4", + "colab_type": "code", + "colab": {} + }, + "source": [ + "X_std = StandardScaler().fit_transform(X_)" + ], + "execution_count": 0, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "pr14L41BGEPE", + "colab_type": "code", + "outputId": "d2e13cbf-ce73-4229-d205-2288416c7cae", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 367 + } + }, + "source": [ + "coefs = []\n", + "alphas = np.logspace(-2, 1, 30) # the range of the parameter\n", + "for a in alphas:\n", + " l = linear_model.Lasso(alpha=a, fit_intercept=False)\n", + " l.fit(X_std, y)\n", + " coefs.append(l.coef_)\n", + "# plot log lambda on x-axis and the corresponding values of coefficients on y-axis\n", + "ax = plt.gca()\n", + "ax.plot(alphas, coefs)\n", + "ax.set_xscale('log')\n", + "plt.xlabel('Log lambda')\n", + "plt.ylabel('Coefficients')\n", + "plt.title('Coefficients as a function of lambda')\n", + "plt.axis('tight')" + ], + "execution_count": 87, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(0.00707945784384138,\n", + " 14.12537544622754,\n", + " -6.055023647908607,\n", + " 3.7774646433491075)" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 87 + }, + { + "output_type": "display_data", + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEaCAYAAAAYOoCaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdd3gc1bn48e/Zrt5ly7LljrtNMcVAQidAqAklhOYQQkLKDSlwE5JcSm5+gRTCDQRCrwkQIAmY0AymuYAxxb3gomZLttXLauuc3x8zWq1llVXdlfb9PM88M3OmvbuzO+/Mmaa01gghhEg+tngHIIQQIj4kAQghRJKSBCCEEElKEoAQQiQpSQBCCJGkJAEIIUSSkgSQpJRSM5RSnymlmpVS/6WUSlFKLVFKNSqlnlNKXaaUeiOG+dyklHpoOGJONEqp45RSnyulWpRS5w/jcr+glNo6XMuLWu4Bv5kuhr+jlLpmCJY7SSmllVKOGMc/USlVOdhxjEYxfaEifpRSXwd+DMwEmoHPgN9orZcPcNY3Am9rrQ+1lnMFMAbI01qHrHH+1ttMtNb/b4BxYC1/ErALcEYtP9HdBtyjtf6/oVyIUkoD07XW2wG01u8DM4Zymd044DcjRj45AkhgSqkfA3cB/w9z41wC3AucNwiznwhs7NS/bQRtfBNB5+9wtEu2zzv6aa2lScAGyAJagIt6GMeNmSD2WM1dgDtq+NmYRwwNwEpgvlW+DAgDPmsZTwMBIGj1fxNYDCyPmtccYClQB+wFbrLKbwGeihrvGGtZDcBa4MSoYe8AvwZWYB7NvAHkW8PKAW0tvwVYBEwD3gUagRrg2R6+i+eAamvc94A5UcPOAjZZy9wN/LSbeUy1vptaa3l/A7K7GXcHYABtVrxuoBQ4NWqcyHcDTLI+31XWZ60BfhE1rh24yZpvM/AxMMH6LBpotZZzCXAiUBk17Szru23A3ECfGzXsMeAvwH+s+X4ITO3hezzXmkeDNc9Z3fxmDuli2neAa2L5Lq3v6gZgnfXZHsbcyXnVivNNIKfTd3ct5u+8KnodAinW56y31vMNnb6fn0V9r5uAC+L9/06UJu4BSNPNioEzgBDg6GGc24APgEKgAHPD+2tr2GHAPuBoa+NylfWnc1vDI39Wqz+ysbL6F2MlACDD+tP9BPBY/Ud3ng4otv7wZ2EeXZ5m9RdELXMHcIj1p30HuN0a1v4nd0TF8DTwC2teHuD4Hr6Lq6242pPiZ1HDqoAvWN05wOHdzGOaFbPb+j7fA+7qYZmlHLjB79wf/d20f74Hrc++APDTsYG9AViPWbWjrOF51jANTIua74lYGzjACWzHTB4u4GTMDd0Ma/hj1jo4CrPK92/AM918nkMwN8anWfO90Zq3q6vfTBfTR4b39l1a39UHmBv9Yszf6ieYv1sPZvK4udN39zSQBswD9rd/18DtwPtALmbS3MCBCeAiYJz1O7rE+oxF8f6PJ0IjVUCJKw+o0T1XyVwG3Ka13qe13g/cClxhDbsWuF9r/aHWOqy1fhxzg3NMP2I5G6jWWv9Ra+3TWjdrrT/sYrzLgVe01q9orQ2t9VJgDWZCaPeo1nqb1roN+AfQU31yELPaYZy13G7Pe2itH7Hi8mNueBcopbKi5jNbKZWpta7XWn/SzTy2a62Xaq391vd5J3BCD/H1x61a6zat9VrMI6QFVvk1wC+11lu1aa3WujaG+R0DpGMm0oDWehnwMnBp1Dj/0lqvtn5Lf6P77/wS4D/WdxAE/oCZrI7t64eM8bu8W2u9V2u9G3MD/qHW+lOttQ/4F2YyiHar1rpVa70eeDTqM16MeV6sTmtdAfy5UyzPaa33WL/JZ4HPMRNi0pMEkLhqgfxernwYB5RF9ZdZZWBuOH+ilGpobzD3jsbRdxMw99x7MxG4qNMyjweKosapjur2Ym68unMj5t7waqXURqXU1V2NpJSyK6VuV0rtUEo1Ye5dAuRb7a9iJqEypdS7SqlF3cxnjFLqGaXUbms+T0XNY7B09/lj/Y47GwdUaK2NqLIyzL3q3pbZ1bwivydrnhWd5hWTGL/LvVHdbV30d46zIqo7+rc+roth0bFcaV291P6bnNtFLElJEkDiWoW5x97T5YV7MDe67UqsMjD/EL/RWmdHNala66f7EUsFMCXG8Z7stMw0rfXtMUx70GNptdbVWutvaa3HAd8G7lVKTeti2q9jnhg/FfPcySSrXFnz+UhrfR5mVdm/MY88uvL/rDjmaa0zMY9oVAyxt2sFUqP6x/Zh2grMevO+2gNMUEpF/5dLMM919Gdekd+TUkphJqb+zGug32VXJkR1R//Wq7oYBoBSaiJmtdv3MavUsjGriAYay6ggCSBBaa0bgf8B/qKUOl8plaqUciqlzlRK/c4a7Wngl0qpAqVUvjX+U9awB4HvKKWOVqY0pdSXlVIZ/QjnZaBIKXW9UsqtlMpQSh3dxXhPAecopb5k7ZV7rGuyx8ewjP2YJ1UjiUYpdVHUtPWYGxSji2kzMJNlLeYGOHJpqlLKZd3TkGVVazR1M4/2+bQAjUqpYsx6+b74DPiatZ4WAhf2YdqHgF8rpaZb62u+UirPGraX7hPwh5h79Tdayz0ROAd4po+xg5kYv6yUOkUp5cQ85+PHPLfUVwP9LrvyK+t/MAf4BvBsVNw/V0rlWL+XH0RNk4b5u9kPoJT6BuYRgEASQELTWv8R8x6AX2L+gCsw92T+bY3yv5h17OswTyB+YpWhtV4DfAu4B3PjuR3zxG5/4mjGPKF3DmZ1wufASV2MV4G5J35TVLw3EMPvTGvtBX4DrLAO1Y8BjgQ+VEq1AC8BP9Ra7+xi8icwD/t3Y17l8UGn4VcApVZVxHcwz5105VbgcMwrif4D/LO3uDv5FeZefL01r7/3Ydo7MTdkb2AmqYcx69/BPKfxuPW9XBw9kdY6gLlezsS82uZe4Eqt9ZY+xo7Weivmnvrd1rzOAc6xltFXA/0uu/Iu5u/4LeAPWuv2GxVvxVz/uzC/vyfbJ9BabwL+iHlEvRfzBPKKQYhlVFBaywthhBAiGckRgBBCJClJAEIIkaQkAQghRJKSBCCEEElKEoAQQiSpEfU46Pz8fD1p0qR4hyGEECPKxx9/XKO1LuhcPqISwKRJk1izZk28wxBCiBFFKVXWVblUAQkhRJKSBCCEEElKEoAQQiQpSQBCCJGkJAEIIUSSkgQghBBJakRdBtpfvq1bCVZWmj2q/T0QqqNllan2YQe0o7ojnergcVTH/A4aHhknatgB5VHDbLZIubJFj2s7eBxlQ9ltkX5ls4Hd3jGOzW7Ow27vGGaTnC+EMCVFAmh49lnq/96fF2GNQkqBw4Gy2VB2u5kUotsOO8rhRNntKIfDHDfSfeAw5XKinE5zHKfTHOa0mvYyaxzlcqNcLqtxYnNH97tQThfK7cLm8aA8nkhbOZ0diVkIMaiSIgHkfetbZF94IZF3H7S/AkHrjh59cPvA8aOGdx6n8/y6Gh49LFKuO+bdXm4YB45vGFHzMKLGM4ehDXMaQ6ONMFjlkW5toENhMMLosIEOh+CAdhjCYXQ4bJaFrO5Q0OwOhazyEDoURgeDGG1eCIbMYcGg2XTRTTA48JWnlJkQ3O4DEoMtJQVbaiq2VLOtUlOxpaRaZR3lttRUbOkZ2DLSsaenY8vIwJ6ejnK5Bh6bECNcUiQAZ1ERzqKi3kcUg0prDVZSMAIBdHTj96MDAas8iA740X4/hs+P9vswfD60z4/h96HbfGa7fVibD8PXRrilmdC+fRheL0ZbG4bXi/b5YopNuVyRZGBrTwxZWdizszvakSaqPzPTPBoSYhSQX7IYMkopsKp4bGlpw7JMHQ6bCcLbim5rw2htJdzcgtHagtHcbHa3NGO0tJjdzc2EW1swmprxf/454cZGwg0NEA53uwxbVhaO/PwDGnt+Ho78ArO/IB9HXh723FyzWk2IBCUJQIwqym7Hnp6GPb3/CUdrbSaIhgaraYzqbiBcX0eoppZQTQ1t69cTqqlBe70Hz8hmw1FYaB6BjhuHc1wRDuto1DluHM6iIuyZmQP4tEIMjCQAITpRSmHPyMCekQETJsQ0jdHaSqjWTAqh/TWEamsI7d9PqKqaYFUVbevW0fTGGwedF7Glp5sJobgY16RJuCZPNtuTJuEoLJAT4GJISQIQYhDY0tJwpaXhKinpdhxtGGaCqKoiWFVFcE8VwT17zKayktZVq9B+f2R8lZqKa9JE3JOspDDZTAzuadOwpaQMw6cSo50kACGGibLZcBYW4iwsJGXBgoOGa8MgVF2Nf9cuAqWlBErLCJSW0rZ+PU2vvWZe9QVgs+GaPBnPzJl4Zs/CM2sW7lmzcOTkDPMnEiOdJAAhEoSy2axzBePguOMOGGYEAgTLy/Hv3Il/y1Z8W7bg/eQTmv7zn8g4jrFjI0nBPXMmKfPmydVvokeSAIQYAWwuF+5p03BPmwannx4pD9XX49+yBd/mLfg2b8a/ZTMt770XOVpwjhtHysIjSF24kNSFC3FNniznFUSEJAAhRjBHTg6ORYtIW7QoUmb4fPi3baNt7Tq8a9bQunIVTS8tAcCem0vqEUeQeuRCUo44As/MmXKpahJTkbtdR4CFCxfq/rwSsnJLHbW7W4cgIiLPBzqgqMsdrIMLY9kRO2icqAIVebZQ5251UL/NpszHCdnau1WnbrDZFDa72W+328zhdnVAuc3eUaZsSvYmRwCtNYHSUrxr1tC25mO8a9YQ3L0bMK9CSjn8MNK/8EXSTzoJ1/jiOEcrhoJS6mOt9cKDypMhAbz79FY2vLt7CCISNofCZrdhb08Mdhs2u8LusEWShd1hsxqrvFN/pHGajcNq7E47Dpc5zOGy4XDaO4a77DjdZuNwmslKxC5YVYV3zcd4P16D94MPCZSWAuCeMYOMU04m/aST8cyZLQ8PHCWSOgEEA2GMkBHTuAP+OrqYXndd2OflH7SutFmmrXZ0f6TbaC/TGOGOfrNbY2iNDmsMw+zXBoTDhjnMMMeLjGt1G4bR0R02CLd3hzrKw2EjUhYOa8JBg3CovbHKo/rDIYNwMLZ11BWHy2YmAysxmG0bTrcDl8eOy+PAlWLH6XFEul0ec5jTanvSnLhSHdjtybfR8+/aRcvb79CybBneTz4Bw8BRWEj6SSeRcfJJpB5zDDa3O95hin5K6gQgRgatNUZIEwqGCQXNhBAKmEkiFOgoCwbChAJmWdAfNvv9YYIBw2qHI+2gP0ygLUTAZ7aNcO+/d6fHjifViTvNgTvViSfNgTvNGSnzpDlJzXCRkukiJcNJaqYLh3P01KOH6utpefddWpa9Tcvy5WivF5WaSvpxx5Jx+ulknHYaNo8n3mGKPpAEIAQQDhoEfCGzaQtb3WZy8HtD+L1BfK1Bs7s1iK/VKrP6u0sgTo/dTAoZLlKtxJCS6SI9201atpuMXA/pOW5cKY4Rdd7E8Pvxrl5N87JltCx7m9Devdiyssg65xyyL7oIz4xD4h2iiIEkACEGSGtNKGDQ1hygrTlIW3MAb3PAbDeZZWbbalqCB1X1Odx2MnLMpJCe6yE92016jpuMPA9ZBalk5LqxJWgVlDYMvKtX0/CP52heuhQdDOJZMJ+ciy4i88wzh+2Bf6LvJAEIMcyMsEFrY4DWBj/NdT5aG/y01PlpafDRUu+npd6Pt9F/wHkfm12RmZ9CVoHVFKaSVWh2Z+Z5EiY5hOrraXzxRRqee57Ajh3Y0tLIPPtssi+6iJS5c+IdnuhEEoAQCSgcNvA2BmiubaNhXxuN+9to3Oe12m0E/R2PpbbZFBl5HnLHpZE3Pp384nTyitPJLEjBFqeroLTWtH36KQ3/eI6m115D+3y4Z88i56KLyLrgAjlXkCAkAQgxwmit8TYFIsmgcb+Xhr1e6va00rDXGzlycLhs5I5LJ7/YTAx5VmLwpDmHNd5wUxONL79Mw3PP49+8GceYMRT81w/IOv98udksziQBCDGKhAJh6qpaqd3dQm1lKzW7W6itbMHX2vG46YxcD2OnZlE0NYuxU7LIK04bliokrTXejz5i3x//iG/tOlzTplL445+QftKJI+oE+GgiCUCIUU5rjbcxEEkG+8qaqNrRiLcxAIDTbWfM5EzGTjGTwpgpWbhThu5pMFprmt9Yyv4//YlAaSkpRxxB4U9/Quphhw3ZMkXXJAEIkYS01jTX+qje2Uj1jkaqdjZSW9liVh8pyBuXxtgpWUyYncuEmbm4hiAh6GCQhhdeYP89fyFcU0PGaadS8KMf454yedCXJbomCUAIAUDAF2JvaRPVO8ykUL2zkYAvjM2mKJqexcQ5+Uycl0fO2NRBrbIxWlupffxx6h56GMPvJ/urXyX/+9/DWVg4aMsQXZMEIIToUjhssHdnI2UbainbUBt5cGJGnoeJc/OYODeP4hk5OF2DcyI3VFtLzX1/pf7ZZ1EOB3nXfJP8b30L5XINyvzFwSQBCCFi0lzno3xjLaXra6ncUkcoYGB32Ciekc3kBQVMX1iIO3XgVxgFysvZ96c/0fzqa7hnz6L4jjtwT58+CJ9AdCYJQAjRZ+GgwZ7PGyjbUEvphhoa97Vhd9qYcmgBs44tYvyMnAE/ibVp6VKq/+dmjNZWCq6/ntyrrpTLRgdZwiUApdQE4AlgDOYN8w9orf+vp2kkAQgRP1pr9pc3s2VlFds+2ovfGyIj18OMRWOZtaiIzPz+v6g+VFND1c230PLWW6QuXEjR7b/FNX78IEaf3BIxARQBRVrrT5RSGcDHwPla603dTSMJQIjEEAqG2bW2hs0rq6jYXAcaimdkM+vYcUw5rKBf5wu01jT+69/s/c1vQGvG3PRzsr76Vbl3YBAkXALoTCn1InCP1nppd+NIAhAi8TTX+dj6QRWbV1bRVOPD5bEz7cgxHHZaCdmFqX2eX3D3bvbc9Au8H35I+gknUPS/v8ZRUDAEkSePhE4ASqlJwHvAXK11U6dh1wLXApSUlBxRVlY27PEJIXqnDc2e7Q1sWVnF9o/3YRiauScUc+RZk/Gk9+2ksTYM6p98kn13/glbSgpjb7mFzDO+NESRj34JmwCUUunAu8BvtNb/7GlcOQIQYmRobfSzeskuNq/Yg9Pj4IgzJzL/pPF9fnGOf8cO9vz3z/Bt2EDm2Wcz9pabsaenD1HUo1dCJgCllBN4GXhda31nb+NLAhBiZKnd08Kqf+6gbEMtGbkejjl/CtMXjunTlUM6GKTm/geoue8+3FOmMP6+++Tl9X2UcAlAmWd2HgfqtNbXxzKNJAAhRqaKLXWsfGE7NRUtFJRkcNxXp1E8I6dP82hdtYrKH16PcjgYf8/dpB5++BBFO/p0lwDi+XaJ44ArgJOVUp9ZzVlxjEcIMUQmzMzl4p8fyamLZ9HWHODff/qU/9y7jrqq1pjnkbZoEZOeeQZ7RgblVy2m8cUXhzDi5BD3cwB9IUcAQox8oUCYtcsq+OS1MoIBg7knFLPogqkxXzoabmig8ofX4/3wQ/KuvZaC63+IsiXGm9ISVcJVAfWHJAAhRo+25gCrX97Fhnd3kz0mldOunk3hxMyYptXBINW3/ZqG554j47RTGXfHHdhS+37JabJIxCogIUQSS8lwccKlMzjv+kMJBcK8cMfHrHmlFCNs9DqtcjoZe9utjPn5z2h+axmll19OsLp6GKIeXSQBCCHiavzMXC755VFMPbyAD1/ayb/++CmN+9t6nU4pRe5VVzH+3r8QLCun9KKLaVu/fhgiHj0kAQgh4s6T5uT0a+Zy2tWzqatq5dn/Xc3mlXuIpYo648QTmfj031FOJ2WXX0HTa68NQ8SjgyQAIUTCOOSosXztV0dRUJLBsie28Nr9G2hrCfQ6neeQQ5j03D/wzJ7N7ut/RM1f748peSS7pDgJ/N62/WyqasKmQKFQyjx8tCmwqY5+hdkfXW5TCputvb9jmM2axh413KYUdpuKDLfbzGnsto7xHDYbdmt8h82GzWaO1z6Ow2bDblc4rDKHNQ8hkolhaD57s5wPX9yJJ83JyVfNYuKcvN6n8/up+uWvaFqyhNzFiyn87xvl/0P3J4GH7o3QCeSNTdU89UF5vMPoN5syE4fD3pEU7DYbDpvCYVc47e3dNlx2s+2wWeXWcJfdhtPqdjo69dttuKwyl8OG22HD7bThstsj3W6H2d0+3OO0k+K0k+Iyy+VPJgaTzaY4/PSJTJiVy5uPbuLlu9cy74Rijr1wWo+Pk7C53Yy743bsmZnUPfYYRmsLY2+5Rd4v0I2kOAIIhAzChsbQZqMBbYBGY2jMMm0+jtbQUeWGWd4+nRE1jqE1YWt4uH24oa3ldEzTvtywQVR3VGNNF7KGhcJmOxjWhA2DkDVeezsYNqy2NTysCRqaYMggZBgEw7qjHTanD4TMdjBsEAwZBMJWd9iIDBsoj9NmJgSnHY/LHulOcdlJddlJczlIcztIddtJt7rT3HazbfWnux1kpjjITHGS7nJgG+CLRsToEAqG+eBfO1m7rIKiqVmc+Z15pGT0/PpIrTX7/+//qP3r/WSedSbj7rgD5Rz4W8xGqqQ+AnA55FRHTwxDEzTMZBAIGQTCBv6ggT9k4A+F8Vvl/lA4Uu4LhvEFw7QFDdrauwNh2oJm47O6m30h9jX5afGH8AZCtPrDBGK5zE9BhttMBpkep5kYPM5If06qk9x0F7mpLnLSXORaTXaKE4dd1vdo4nDaOf7i6YydmsWbj23i+TvWcPb3F5AzNq3baZRSFF5/Pfb0dPb94Y8YrV6K/+8ubB7PMEae+JLiCEAklkDIwBsIWUkhTIs/RKs/RIsvRLMvRJMvSFNbkCZfyGoHaWrrKG9sC9IaCHc7/6wUZyQh5KW5KMx0MybDw5hMDwWRbjc5qS45yhhhqnc18sq96zDCmjOuncv4mbm9TlP/zDNU33obqQsXMv6++7Cnd584Riu5E1iMKv5QmAZvkLrWAPWtAWpbA9R7A5H+Om+QulY/Nc0B9jX7qPcGD5qHw6YozHBTmGkmhOLsVMbnpDAht6Od7k6Kg+QRpammjZf/so7GvV5OvHwGs44d1+s0jUteZs/PfoZn9mwmPHA/jpy+PYhupJMEIJKaLxhmf7Offc1+9jX52NvkY2+zn71NPvY3+6lu9FFZ30Zb8MAji5xUJ+NzUpmQm2K2c1IoyUtjemE6RVkeOfkdJ35vkNcf3EDF5noOP2Mix5w7pddHTDcvW8bu63+Ea+JEJjz8EM7CwmGKNv4kAQjRC601da0BKurbqKz3UlFntevbqKzzUtnQRiDUcf4izWVnamE609qbgnSmj8lgQk6KnIcYBuGwwXvPbGPT+3uYdkQhp1w1C0cvD5RrXbWKiu99H0d+PiWPPJI07xWQBCDEABmGZn+Ln101rWzf13JAU93ki4znstuYnJ/GtDHpzBmXyYLx2cwtziIrJXmvQhkqWms+W1rByn9tZ8ykTM66bj6pmT1fIeT99FMqvv0dbKmplDzyCO4pk4cp2viRBCDEEGryBdnRKSls29dMRV3HM20m56cxf3wW84qzWDAhmznjMkl1yTmGwbDj0328+cgmUjJdnP29BeSO6/lEr2/LFsq/eQ1ozcSnnsQ9ZcowRRofkgCEiIMGb4B1lY2s393I2ooG1u9upKrRPFqwKZhemMG88VkcXpLDsVPzmJiXKucV+mlvaROv3LuOUCDMGd+ex4RZPV8h5N+1i7LLr8CWnsbkZ5/Fnp09TJEOP0kAQiSIfc0+1lc2sraykfWVDayrbKS21XzeTXF2Coum5nHctDyOnZrPmEy5br0vmut8vHzPWhr2evnSNXOZclhBj+N7P/mU8quuImXhEZQ88MCovVlMEoAQCUprza6aVlbsqGXl9hpW7aylwbpsdWpBGsdOzee4aXkcMyWP7NSe67cF+FqDvHzPWvaVNXPq4lkcctTYHsdv+Oe/qLrpJnIuu4yxv/rlMEU5vCQBCDFCGIZmU1UTq3bUsmJHDat31eENhFEK5ozL5LRZY/ny/LFMK8yId6gJK+AL8cq969j9eQMnXTaT2cf3fK/A3jt+R92jjzL2llvI+dolwxTl8JEEIMQIFQwbrK1oYMX2Wt7/fD8fl9ejNUwvTOeseUWcNa+IQ8aky7mDTkKBMK/ev4HyjbUcf9F0FpwyodtxdThMxXXX0bpyFSUPP0za0UcNY6RDTxKAEKPE3iYfr2+s5j/rqlhdWofWZlVRezKYOTZDkoElHDR445GN7Px0P0efN4WFZ07qftzmZkq/dinhmhomPf8crgndJ4yRRhKAEKPQvmYfr2/cyyvrqvhwVy2GNi83PWveWM5dUMyMsVJNZIQN3npiM9s+3GveNXzelG4TZKCsjF0XX4KzsICJTz+NPT19mKMdGpIAhBjlalr8vL6xmlfXV7NqZy1hQ3PUpFyuWDSRM+aOxZnEdydrQ/PO01vZ9P4e5p80nuMvnt5tEmj94APKv3kN6V/4AuP/cs+oeJeAJAAhkkhti58XPqnkqQ/KKa/zUpDh5utHlfD1o0uS9tJSrTUrntvO2mUVzD6uiBMum9nt02Dr/v539t72a/Ku+SaFP/3pMEc6+CQBCJGEDEPz7rb9PL6qlHe37ceuFF+aM5YrFk3k6Mm5SXeuQGvN6iW7WPNKKdOPHMMpi2dh7+bIqOrWW2l4+hnG3XE7WeedN8yRDi5JAEIkubLaVp76oIx/rKmksS3IjDEZXLFoIhccVkxakj32+pPXy1j1rx1MXpDPl66Zi915cBLQwSDl13yLtk8+YeKTT5By6KFxiHRwSAIQQgDQFgizZO0eHl9VysY9TWS4HVx9/GSu+cJkMjyj807Yrqx7u5L3n93GtIWFnH71nC4fJx2qr6f04ksw2tqY/Nw/cBYVxSHSgZMEIIQ4gNaaT8obeOj9nby6oZrcNBffO2kalx9Tgtsx8k98xqL9SGDByRM47qJpXVaJ+bdvp/SSr+EsLqbksUdx5Pb+FrJE010CSN7LAoRIckopjpiYw32XH8GL3zuO2UWZ/PrlTZz8h3d5bk0FYWPk7Bz212GnlzD/5PGsXVbBZ0sruhzHPW0a4++5m0BZGeVXLSZUWzvMUQ4dSb2z/C4AACAASURBVABCCBZMyOapa47mqW8eTV66ixueX8cZd73H6xurGUm1BH2llOL4C6cz7YhCVv5zO1s/rO5yvLRFi5hw/18JVFRQvngxoZqaYY50aEgCEEJEHD89nxe/dxz3XXY4Ya359pMfc8G9K1m1Y/Ts9XambIpTF8+m+JBslj2+mYpNdV2Ol3bMMUy4/34Clbspu2oxof37hznSwScJQAhxAKUUZ84r4o3rv8jtX5lHdaOPSx/8gCsfWc2G3Y3xDm9I2J02zrxuPjlFabx6/3r2lzd3OV7a0Ucx4f6/EqyqouyqxQT37RvmSAeXJAAhRJccdhtfO6qEd244kZvOmsnaigbOvns5P3zmU8prvfEOb9C5Uxyc84MFuNMcLLn7Mxr3t3U5XtpRR1HywP2Eqqspv/IqgntHbhKQBCCE6JHHaefaL07lvRtP4rsnTuX1jdWccuc73PLSRmpb/PEOb1ClZbs55weHYhiaJX/+jLbmQJfjpS5cyISHHiS0fz/lV15JcO/eYY50cEgCEELEJCvFyY1nzOSdn57EhUeM54lVpXzxd2/z57c+p9Ufind4gya3KI0vf3cBLQ1+Xr5nLQFf158t9fDDzSRQU0PZFVcSrKoa5kgHLq4JQCl1hlJqq1Jqu1LqZ/GMRQgRm7FZHn77lfm88aMTOH56Pncu3cYJv3+HJz8oIxg24h3eoCiamsWXrpnD/vJmXn9wI+FuPlfqYYdR8sjDhOvqKLvyKoJ79gxzpAMTUwJQSh2nlEqzui9XSt2plJo4kAUrpezAX4AzgdnApUqp2QOZpxBi+EwrTOf+KxbywnXHMiU/jV/9ewOn3fku/1lXNSouHZ28oIATvj6D8o21vPPUlm4/U8qCBZQ8+gjhxkbKrriSQOXuYY60/2K6E1gptQ5YAMwHHgMeAi7WWp/Q7wUrtQi4RWv9Jav/5wBa6992N02y3wlsGGH8ra20NTfR1txM0NdGMOAn5PcT9PsJBfwEfT6z3d7v9xMOjZ7Dc5G49jb52FzVRLMvRFaKM27PF8oN7iU7NHiXaHoDeXj9eTgdXmwq3P2IYY1qC4EC7IP3kD0NhNBMOHoWF1x/Y7/m0d2dwLGuoZDWWiulzgPu0Vo/rJT6Zr8i6VAMRN96Vwkc3XkkpdS1wLUAJSUlA1xkYggFgwS8rfhaW822txV/ayt+byu+lmZ8Lc3WRt7c0Ld3+1qaIZY9K6Vwutw43G6cbg92hx3zVynE0FHAbBc0GyGaWoMYLcN/FJCq20jTdfiwowftN78Hh24iGHD3PqoDc4s9BB892NAy6POMNQE0W3volwNfVErZgGF5apTW+gHgATCPAIZjmZFlGwahYMDamw4QCpjtoN/a6w74CfraCLS1mW2fj6DfFykL+KxuXxt+r5eAtaEPBbu+sqCd3eEgJTOLlPQMUjIzKSiZREpGJimZmWZZRiaejExcnhRrI+/G4XLj9HhwutzYnc6ke8yvEGx7A565FCYcA5e/AM7R8d6DS16+BEMbPHfObYM+71gTwCXA14Fvaq2rlVIlwO8HuOzdQPRLN8dbZYNuy8r3qNy0nlAgSDgUJBQIEA4GrO4goWCAcDBIOGh2hwIBQn5/rxvqrrRviF0eDy5PCk5PCu7UNDLzC3GnpeFOtZq0NDypabg6dXvS03G6PbIBF6IvylbBP66AMXPg0qdHzcZ/e/12NtVu4sYj+1f105tYE8CPtNb/3d6jtS5XSs0Z4LI/AqYrpSZjbvi/hplkBt3endvZ9uFKHE4XDpcTu8OJ3ep2uFx40tOxO9vLnZE9aofLHdXtsrpdkeoVh8uNKyUlsqF3etzYbMnxFEUhEkb1evj7JZA1Hi57ATyZ8Y5o0Ly04yUcysFZk88akvnHmgBOA/67U9mZXZTFTGsdUkp9H3gdsAOPaK039nd+PTnh8qs54fKrh2LWQoh4qt0BT34F3Olwxb8hvSDeEQ2akBFiyc4lHD/+ePJS8oZkGT0mAKXUdcB3gSnWlUDtMoCVA1241voV4JWBzkcIkYSaquDJ88EIweKXIXtC79OMIKv2rKKmrYbzpg7d6yh7OwL4O/Aq8Fsg+katZq1114/ME0KIoeatg6e+YravegkKZsQ7okH30o6XyHZnc8L4fl9t36sebwTTWjdqrUu11pdiXqYZxLzAKd06ESyEEMMr0GrW+dduh6/9DYqPiHdEg67R38iy8mWcOflMnPahu+AypnMAVl39LcBeoP2eaI15Y5gQQgyPUACevQJ2r4GLHocpJ8Y7oiHxeunrBIwA500buuofiP0k8PXADK316H0rhBAisRlh+Ne3YcdbcO7dMPvceEc0ZF7a8RLTsqcxO3don44T68PgKoDR+SYIIUTi0xpeuQE2/hNOuw0OvzLeEQ2ZXY27WLt/LedOPXfI7weK9QhgJ/COUuo/QOQB4FrrO4ckKiGEiPbe72HNw3DcD81mFFuyYwk2ZePsKWcP+bJiTQDlVuOyGiGEGB6f/R3e/g0suBROvTXe0QypsBHmpR0vcey4YylIHfp7GmJKAFrrWwGUUqla69H3LjghRGLasQxe+oF5svecP8Mof0TK6urV7PXu5acLfzosy4v1fQCLlFKbgC1W/wKl1L1DGpkQIrlVr4dnr4SCmXDxk+AY/ZUPL+14iQxnBieVnDQsy4v1JPBdwJeAWgCt9Vrgi0MVlBAiyTVWwt8uMp/rc9lzo+r5Pt1pCbTwZtmbnDH5DNz2GB49PQhifmOD1rqi0xnpHt6MIIQQ/dTWAE9daN7wdfVrkDku3hENi6VlS/GFfZw7dfgub401AVQopY4FtFLKCfwQ2Dx0YQkhklLID89ebt7le/kL5uOdk8SLO15kUuYkFhQsGLZlxloF9B3ge5hv8doNHGr1CyHE4DAMePF7UPo+nH8vTBm6Z+AkmormCj7e+/GwXPsfLdargGqAy4Y4FiFEMlt2G6x/Dk75H5h/cbyjGVZLdixBoThn6jnDutzeHgd9o9b6d0qpu+niLZda6/8assiEEMnjo4dg+Z9g4dVw/I/jHc2wMrTBSzte4uiioxmbNnZYl93bEUB7Pf+aoQ5ECJGktrxiPubhkDPgzN+P+mv9O/t478fsbtnN9w4d/lr1HhOA1nqJ1X58eMIRQiSVyjXw/NVQdChc+AjYY74wcdR4acdLpDnTOKXklGFfdqw3gi1VSmVH9ecopV4furCEEKPeznfgifMhYwx8/VlwpcU7omHnDXp5o/QNTp94OqnO1GFffqxXARVorRvae7TW9UDh0IQkhBj11j9vXuufPQEWvwLpybk5eav8Lbwh77Be+x8t1gQQjn4DmFJqIl2cFBZCiF6tvBte+CZMOAq+8SpkFcc7orh5cceLjE8fz+FjDo/L8mOtcPsFsFwp9S6ggC8A1w5ZVEKI0ccwYOmvYNU9MOtc+MqD4PTEO6q4qWqpYnXVaq5bcB02Feu++OCK9T6A15RShwPHWEXXW/cGCCFE70J++Pd3YcPzcNS1cMbtYLPHO6q4WrJzCRo97Nf+R+vtPoCZWust1sYfYI/VLlFKlWitPxna8IQQI56vCZ69DHa9B6fcDMf/KOku9ezKG6VvcHjh4YzPGB+3GHo7AvgxZlXPH7sYpoGTBz0iIcTo0VwNf7sQ9m2G8/8Kh14a74gSwj7vPrbWb+X6w6+Paxy9JYClVvubWuudQx2MEGIUqfkcnvoKtNbCpc/C9FPjHVHCWLF7BQDHFx8f1zh6O/Pwc6v9/FAHIoQYRSo+godPh4AXFr8sG/9Olu9eTmFKIYfkHBLXOHo7AqhTSr0BTFFKvdR5oNY6PhevCiESk9bwyePw6s8gY6z5SOe8qfGOKqGEjBCrqlZxasmpw/rkz670lgDOAg4HnqTr8wBCCGFqrYUl/wVbXobJJ8BXH0raG7x6sr5mPc2B5rhX/0DvCeBhrfUVSqkHtdbvDktEQoiRZ/tb8O/roK0eTv9fOOZ7YIvPte2J7v3K97ErO8eMO6b3kYdYbwngCKXUOOAypdSDmDeBRWit64YsMiFE4gv64K1b4YN7zZe3X/4CjJ0X76gS2vLdy1lQsIBMV/zfc9xbAvgr8BYwBfiYAxOAtsqFEMlo70Z44Vuwb6N5c9dpt4EzJd5RJbSatho2123mvw5LjFep9PY46D8Df1ZK3ae1vm6YYhJCJDLDgNX3w9KbwZMFlz0P00+Ld1Qjwso9K4H4X/7ZLtZHQVynlDoemK61flQplQ9kaK13DW14QoiE0lxt1vXvWAaHnAnn3g3pBfGOasRYXrmcPE8eM3JnxDsUIMYEoJS6GVgIzAAeBVzAU8BxQxeaECJhaA0b/wn/+SkE2+DLd5qvb5RHOsQsbIRZWbWSE8afELeHv3UW69NALwAOAz4B0FrvUUplDFlUQojEUbEaXv8FVK6GogXwlYegIL43MI1E62vW0+hv5AvFX4h3KBGxJoCA1lorpTSAUmpAr+5RSv0eOAcIADuAb0S/cEYIkQDqS+HNW2DjvyB9jFndc+hlSf8Uz/5asWcFNmVj0bhF8Q4lItbjkH8ope4HspVS3wLeBB4cwHKXAnO11vOBbXQ8ckIIEW9tDfDGr+CeI2Hra3DCf8MPPoHDr5SN/wAsr1zOvPx5ZLmz4h1KRKwngf+glDoNaMI8D/A/WuulvUzW0/zeiOr9ALiwv/MSQgyScBDWPArv/Na8oevQr8PJv4TMcfGObMSr89WxsXYj3z30u/EO5QCxVgEBrAPcVvfaQYzhauDZ7gYqpa7FevtYSUlJd6MJIfpLa9j6Kiz9H6j9HCZ/0bybt2hBvCMbNVbsXoFGJ1T9P8R+FdDFwO+BdzBvBrtbKXWD1rrbp4Qqpd4ExnYx6Bda6xetcX4BhIC/dTcfrfUDwAMACxculPcQCzFYtDYv51z+Jyh9H/IPMR/bfMiX5OqeQbZizwpyPbnMypsV71AO0Jd3Ah+ptd4HoJQqwDwP0G0C0Fr3+PxXpdRi4GzgFK21bNiFGC7BNlj3LHxwH+zfAmmFcNYf4IjFYHfGO7pRx9AGK3ev5Lji4xLm8s92sSYAW/vG31JL7CeQD6KUOgO4EThBa+3t73yEEH3QXA0fPQRrHgFvrfnMnvP/CnO/Ag5379OLftlYs5F6f33C3P0bLdYE8JpS6nXgaav/EuCVASz3HszzCUut52F/oLX+zgDmJ4ToTtVaWHUvbHgBjBDMOBOO+S5MOl6qeobB8t3LUSiOHXdsvEM5SG8vhZ8GjNFa36CU+grQnsJW0UO9fW+01tP6O60QIgZG2Dyx+8G9ULYCnGnmnbtHf1te0DLMlu9Zztz8ueR4cuIdykF6OwK4C+safa31P4F/Aiil5lnDzhnS6IQQsdMa9nwC618wH9vQXAVZE8wreg67AlKy4x1h0mnwNbB+/3q+syAxKzh6SwBjtNbrOxdqrdcrpSYNSURCiL7ZtwU2PG9W8dTtBJvTfDrn/Nth5tlg78vV3mIwrdyzEo1OyPp/6D0B9LTLIA/+FiJe6kvNDf6Gf8LeDaBsMOkLcPyPYNY5kJJ41Q3JaPnu5WS7s5mTNyfeoXSptwSwRin1La31AY99UEpdg/mCGCHEcNDa3Lv//A1zw1/5kVk+/ig483cw+3zIGBPfGMUBDG2wYs8KFo1bhD1BH6HRWwK4HviXUuoyOjb4CzEfB33BUAYmRNJrq4dd75k3a+1YBg3lZvmYeXDKzTD3q5AzMb4xim5trttMna8u4e7+jdbbG8H2AscqpU4C5lrF/9FaLxvyyIRINuEQ7F7TscHf/TFoA1wZMOUEOO6HMOUkuYpnhFheuRwgIS//bBfrw+DeBt4e4liESC4hP1StMzf6pcvNvX1/k1mfX3wEfPEGmHqy2S136I44y3cvZ07eHPJS8uIdSrfk8gAhhkN7Hf7uj6FyjVmHX70ejKA5PLvEvCN36snmw9jkJO6I1uhvZF3NOq6Zd028Q+mRJAAhBpvW5jX4ezdZG/yPzHZbnTncmQbjDoNF34XxR0LxQsgsim/MYlCtqlqFoY2Erv8HSQBCDEzLfti3yXyo2r5N5jX5+zaDv9EaQUHBTJj5ZRi/0NzYF8yUa/NHueWVy8l0ZTIvf168Q+mR/AqF6E3QZ16BU7/LvP6+5nNzI79/s/lQtXYpOVAwC+ZdCIWzzGbsfPBkxi10MfzaL/88dtyxCXv5ZztJAEIYBnhroL6sYyNfXwp1VnfzngPHd2VAobVXXzCrY2OfPkYeribYVr+NmrYajis+Lt6h9EoSgBjdgm1mfXxTldXe06ltlbefjG2XMQ5yJsGUE8127mSznTMJ0gpkQy+6tXy3eflnoj7+IZokADFyaA1Br/nScm8NtNaYVTCt+63uGmi1+tu7I3XxUVzpkFFknnideKzZzhhn3lSVM8m8IscpTzoR/fN+5fvMyp1Ffkp+vEPplSQAMTy0hpAP/C3mte6BFrM70AL+ZrPtawJfg7mB9zVGdVv9bQ0H76m3szkgNc/cO0/NM6+ySSswm8xx1gbfakudvBgizYFm1u5fy9Vzr453KDGRBJCstDZfDhLyQzjQ0YT85oa6vR30Wf1RZSGfWbUSbDP3yINeszvQ2kWZFwLN5sZeh3uPS9nBk2U+utiTbbazJ3R0e7LN4Wn5kJpvttPyzXKplhFxtmTHEsI6zIkTTox3KDFJjgTQvpHThtVoq7H60Z2GGbE1hmFu1LRhvoBDh6220UWZ1T6gO3RgeXt/OGR2GyFzj9cIQzgYVWY14aD5uQ7qDnSM394d2cBHbewZ4KuYlc28pt2ZYjau9u5USB/b0e1ON6td3Blm40o3y9wZ5gnV9uGeTLMtG3IxAoWMEE9seoJDCw5lfsH8eIcTk+RIAK/9HNY8HO8o+k7ZzaoNuxNsVrfNabXtYHdZwxwHdjtTDy63u8zG4bb63Va/q2OY3QUODzg9Ztvh7tTuolw21kIA8GbZm+xu2c0NR94Q71BilhwJYNY55lUcymY2KKtbWU10uTI3vO1lygY224H9BzT2qOF2c8PcPr2tU1n7hlvZOrptjo5x2sdr33jLxlWIEUFrzSMbHmFS5iROmnBSvMOJWXIkgKknmY0QQgyB1dWr2Vy3mZsX3YxN2eIdTsxGTqRCCJGgHt3wKHmePM6ZOrJeky4JQAghBmBr3VZW7FnBZbMuw213xzucPpEEIIQQA/DYxsdIcaRw8YyL4x1Kn0kCEEKIfqpqqeLVXa9y4SEXkuXOinc4fSYJQAgh+unJzU8CcMWsK+IcSf9IAhBCiH5o9Dfy/LbnOXPymRSlj8wX+kgCEEKIfnhu23O0hdpYPGdxvEPpN0kAQgjRR/6wn6c2PcVx445jRu6MeIfTb5IAhBCij17e8TK1vloWz10c71AGRBKAEEL0gaENHtv4GLNyZ3H02KPjHc6ASAIQQog+eLvibUqbSvnG3G+gRvjzuiQBCCFEHzy24TGK04s5beJp8Q5lwCQBCCFEjD7d9ymf7f+MK2dficM28p+lKQlACCFi9OiGR8lyZ3H+tPPjHcqgkAQghBAx2Nm4k7cr3ubSmZeS6kyNdziDIq4JQCn1E6WUVkrlxzMOIYTozRMbn8Btd3PpzEvjHcqgiVsCUEpNAE4HyuMVgxBCxGK/dz8v7XiJ86edT64nN97hDJp4HgH8CbiRAb+ZXAghho7WmvvW3kfICHHl7CvjHc6gistpbKXUecBurfXakX4drRBi9PIGvfxyxS9ZWraUr834GiWZJfEOaVANWQJQSr0JjO1i0C+AmzCrf2KZz7XAtQAlJaPryxdCJK6ypjKuf/t6djbu5CdH/ISr5lwV75AGndJ6eGtglFLzgLcAr1U0HtgDHKW1ru5p2oULF+o1a9YMcYRCiGT3XuV7/Oy9n2G32fndF3/HonGL4h3SgCilPtZaL+xcPuxVQFrr9UBhe79SqhRYqLWuGe5YhBAimqENHlr/EPd8eg8zcmdw10l3UZxeHO+whszIv5VNCCEGQUughV8s/wXLKpbx5Slf5uZFN5PiSIl3WEMq7glAaz0p3jEIIZLbrsZd/PDtH1LeVM6NR97I5bMuH/EPeotF3BOAEELE09vlb3PT8ptw2pw8cNoDHFV0VLxDGjaSAIQQSSlkhHhg3QPct/Y+ZufN5q4T7xqx7/btL0kAQoik81H1R9y++na21W/j3Knn8qtjfoXH4Yl3WMNOEoAQImlUt1Zz55o7ebX0VYrSirjzxDs5teTUpKjv74okACHEqBcIB3hi0xM8sO4BwkaY7yz4DlfPvXrUX+XTG0kAQohR7d2Kd7njozuoaK7g5Aknc8ORNzA+Y3y8w0oIkgCEEKNSWVMZd6y+g/d3v8+kzEncf+r9HFt8bLzDSiiSAIQQo0prsJUH1z3IE5uewGlz8pMjfsJlsy7DaXfGO7SEIwlACDHiNfobebfyXd4qe4uVe1biC/s4d+q5XH/49RSkFsQ7vIQlCUAIMSLt9+5nWfky3ix/kzXVawjpEIWphVww/QLOnXouc/PnxjvEhCcJQAgxYpQ3lfNW+Vu8Vf4Wa/evBWBi5kSunHMlp5acypz8OdiUvOo8VpIAhBAJKWyE2dW4i/U161lfs55P933K9obtAMzKncX3D/0+p5ScwtTsqUl7Hf9ASQIQQiSEva172VCzgXU169hQs4GNtRtpDbYCkO5MZ27+XC6YdgGnTDxlVD+ieThJAhBCDKuQEaKiuYIdDTvY3rCdLXVbWF+znn3efQA4bA5m5MzgnCnnMK9gHnPz5zIpc5JU7QwBSQBCiCERMkJUNldGNvQ7GnawvXE7pY2lBI1gZLySjBKOHHsk8/LNjf3M3Jm47e44Rp48JAEIIQYsbITZ2biT9TXrWbffrMLZ1biLgBGIjFOcXsyUrCkcP+54pmZPZVr2NCZnTSbVmRrHyJObJAAhRJ/VtNWwfv/6jg1+7YZIfX2GK4N5+fNYNG5RZEM/JWuKbOgTkCQAIUSP6n31bKnbwrb6bWyo2cD6mvXsbtkNgEM5mJ4znbOnnM38gvnMy5/HxMyJUl8/QkgCEEIA5gvRK5or2FK3ha11W9lav5UtdVsiJ2cBxqaNZX7+fC6deSnzC+YzK3dWUj5Hf7SQBCBEkgkaQfa07KGsqYyypjJKG0vZVr+NrfVbaQu1AWBXdqZkT+GosUcxM3cmM3JnMCNnBjmenDhHLwaTJAAhRiFDG+zz7qO0qZTypnJKm0opayqjvKmcyuZKQjoUGTfDlcEhOYdwwbQLIhv7qdlT5UqcJCAJQIgRqCXQQnVrNVWtVVS1VlHdWn1A/17vXkJGx0beY/dQklnC9JzpnDrxVCZmTmRS5iRKMkvIcefInbRJShKAEAkibISp99dT21ZLbVstNb6ag7pr2mrY27qX5mDzAdPalZ3C1EKK0oqYXzCforQiitOLmZg5kYmZEylMLZQTs+IgkgCEGGSGNvAGvbQEW2j0N5pNwGw3+Bto8jfR4G/o6A80Ueero8HfgKGNg+bnsXvIS8kjLyWPCRkTOHLskRSlFTE2bWykXZBSgN1mj8OnFSOZJACR9Axt4Av5aAu14Q15zXbQe0B3W6gtMtwb9NIcaKY12EpzsJnWQCstwRazCbTQGmxFo7tdntvuJsudRZY7i2x3NpMyJ3Fo4aHkp+ST5zE39NHdqY5UqaIRQ0ISgBgWWmtCOkTICBE2woSMECFtdeuOsqARjLTbmwP6wx39/rCfQDhAwAgQDAcj3YFwINLtD/vxh/z4w358YR/+kNW2yn1h3wGPJYiF2+4m3ZlOhiuDNGca6a508lLySHemk+5KP2BY+0Y+05UZ6ZbLJkWikAQwwgTDwUh1QmuwNbJn2lMTCAcwtIGhDbTWGBiR/uiysA6jtSaswxjaIGyE0Vj9hjW8U39Yhwkb4Ui3oQ1CRsic3hoW0qEuqzYGk13ZcdldOG1OXHYXLpvLbNtdeBwesxrFmYfH4cFtd+O2uyPlbocbj91DiiOFVGcqKY4Us9uRelBZiiMFh03+NmJ0kF9yHASNIC0Bs7qgOdh8YDvQTIO/IVJH3F5P3N72hrwxLUOhIhssl92FTdmwKzs2ZUMphQ2zHV2m6Oi3KRt2m9WNDZutY3ob5jC7sne0rWEOmyOyLLuyY7PZcCgHDpvZ2JX9oO7oMqfNidPmjHRHyuxOHMphtm2OAzbwLptL6r+F6AdJAN3QWpvVBiFfpMog0m1VHfhCvh7rjL0hL21Bs91eP9wSbIncbNMdhSLTnUm2O5ssdxYFqQVMy54WqUJoL2/fM23fU/U4PJGNvtvulnpjIUSPkiIBvLj9RVbsWWHWE0fXEUfVGUfXKfvDZp1xXynUARvl9u5MVyZFaUVkuDIi9cQHdDszImWZrkwyXBlyyZ4QYsglRQKobKlkY83GA+qI3XY3Kc6UjqqE6Dpjq17YbTfrhtvrjdvb7XvYbrubNGdapJ7YY/fIXrcQYsRQWnd/uVqiWbhwoV6zZk28wxBCiBFFKfWx1nph53KpZxBCiCQlCUAIIZKUJAAhhEhScUsASqkfKKW2KKU2KqV+F684hBAiWcXlKiCl1EnAecACrbVfKVUYjziEECKZxesI4Drgdq21H0Brva+X8YUQQgyyeCWAQ4AvKKU+VEq9q5Q6Mk5xCCFE0hqyKiCl1JvA2C4G/cJabi5wDHAk8A+l1BTdxU0JSqlrgWsBSkpKhipcIYRIOnG5EUwp9Rpwh9b6bat/B3CM1np/L9PtB8o6FWcBjV2M3rk8H6jpd9AD012MQz2fWMfvbbyehsf6/XdXFq/1Eq910pdp+rteRuo6gcFZL4m4TnoaNhzrZaLWuuCgUq31sDfAd4DbrO5DgAqsZNSPeT0QSzmwJh6ftacYh3o+sY7f23g9DY/1+++hLC7rJV7rZDjWy0hdJ4O1XhJxnSTqeonXs4AeAR5RSm0AAsBV2vqE/bCkj+XxI80BZAAABSNJREFUMFix9HU+sY7f23g9De/L9y/rpG/T9He9jNR1AoMTTyKuk56GxW29jKhnAQ2EUmqN7uJZGCK+ZL0kHlkniWko1ksy3Qn8QLwDEF2S9ZJ4ZJ0kpkFfL0lzBCCEEOJAyXQEIIQQIookACGESFKSAIQQIklJAgCUUucrpR5USj2rlDo93vEIUEpNUUo9rJR6Pt6xJDulVJpS6nHrP3JZvOMRg/f/GPEJQCn1iFJqn3VPQXT5GUqprUqp7Uqpn/U0D631v7XW38K8Qe2SoYw3GQzSOtmptf7m0EaavPq4jr4CPG/9R84d9mCTRF/WyWD9P0Z8AgAeA86ILlBK2YG/AGcCs4FLlVKzlVLzlFIvd2qiH0X9S2s6MTCPMXjrRAyNx4hxHQHjMe/WBwgPY4zJ5jFiXyeDIl53Ag8arfV7SqlJnYqPArZrrXcCKKWeAc7TWv8WOLvzPJRSCrgdeFVr/cnQRjz6DcY6EUOrL+sIqMRMAp8xOnYaE1If18mmwVjmaF2ZxXTssYD5Ay7uYfwfAKcCFyqlvjOUgSWxPq0TpVSeUuqv8P/bu7sQq6owjOP/xz5QqbSrCAq06EOKyUoLgqyLurK7PpDCPvDCgkCCoKKL6C4IuvAigwItosIoqavqqinC0NKaQaYPEgpDCkKHNBTiPF3sNbibHDyOZzpnznp+cODsNfusvc687P2etTe8i+slPTPXgwtg5hi9D9wtaQuDVzpi2J00Jr06P+b9DKAXbG8GNvd7HHGC7T9onslEn9k+CjzS73HECb06P4Z1BvArcGlr+5LSFv2TmAy+xGjwzGlMhjUB7AaukLRc0rnAOuDDPo+pdonJ4EuMBs+cxmTeJwBJbwM7gaskHZC0wfbfwOPAx8AEsN32vn6OsyaJyeBLjAZPP2KSYnAREZWa9zOAiIiYnSSAiIhKJQFERFQqCSAiolJJABERlUoCiIioVBJADCVJR3rc37LpZXrPoK9tku7pYr9PJa3qxTEjTiYJICKiUkkAUQ1JKyV9KWlM0g5JF5b21aXtG0kvnuqXfpkNfC5pT3ndUtpvlzQq6QNJ+yW9IOkBSbskjUu6vNXNHZK+kvSDpLvK5xdJekfShKQdwKLWMbeU/fdJer73/52oURJA1OQN4CnbI8A48Fxp3wpstL2S7hY8+R240/YNNCvItSvJXkdTpXEFsB640vZNwGs0ZcenLKOp9b4WeEXSQuAx4C/bK8rYbmzt/6ztVcAIcJukka6/dcQMkgCiCpKWAEttj5am14E1kpYC59veWdrf6qK7c4BXJY0D79Ks1DRlt+2Dto8DPwGflPZxmov+lO22O7Z/BPYDVwNrgDcBbI8BY63975O0B9gLXDPtmBGzkvUAIk7fE8BvNL/2FwDHWn873nrfaW13+Pf5Nr0I14xFuSQtB54EVts+JGkbsHBWI49oyQwgqmB7Ejgk6dbStB4YtX0Y+FPSzaV9XRfdLQEO2u6Ufs6axZDulbSgPBe4DPge+Ay4H0DStTS3ewAuAI4Ck5IuolkfNuKMZQYQw2qxpAOt7ZeAh2juty+mue0ytcrVBppbOh1gFJg8Rd8vA+9JehD4iObifLp+AXbRXNwftX2sLLm4VdIETenfrwFsfytpL/AdzfKAX8zieBH/kXLQUT1J59k+Ut4/DVxse1OfhxUx5zIDiIC1ZWHts4GfgYf7O5yI/0dmABERlcpD4IiISiUBRERUKgkgIqJSSQAREZVKAoiIqFQSQEREpf4BYsFSfQ/YH3kAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "tags": [], + "needs_background": "light" + } + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "V9UTN6veGYtg", + "colab_type": "code", + "outputId": "d3b67ca9-6f42-48be-a28d-50f7b1f4c009", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 51 + } + }, + "source": [ + "parameters_lasso = [{'alpha': alphas}]\n", + "cv_lasso = GridSearchCV(linear_model.Lasso(), parameters_lasso, scoring = 'neg_mean_squared_error')\n", + "cv_lasso.fit(X_std, y)\n", + "scores_lasso = cv_lasso.cv_results_['mean_test_score'] # the scores are negative\n", + "scores_lasso = np.negative(scores_lasso) # reverse the sign of the score\n", + "scores_std_lasso = cv_lasso.cv_results_['std_test_score'] # standard deviation\n", + "# print the optimal values\n", + "print('best score:', np.negative(cv_lasso.best_score_))\n", + "print('lambda:', cv_lasso.best_params_.get('alpha'))" + ], + "execution_count": 88, + "outputs": [ + { + "output_type": "stream", + "text": [ + "best score: 78.31608978858921\n", + "lambda: 0.05298316906283707\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "vynCAYy4Gtjv", + "colab_type": "code", + "outputId": "e62f05f4-d06c-4ee0-c7f8-e7c69a1b2dd9", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 316 + } + }, + "source": [ + "# plot the cross validation score against log lambda\n", + "fig_lasso = plt.figure()\n", + "plt.semilogx(alphas, scores_lasso)\n", + "std_error_lasso = scores_std_lasso / np.sqrt(X_.shape[0])\n", + "# shade the region within 1 std err\n", + "plt.semilogx(alphas, scores_lasso + std_error_lasso, 'b--')\n", + "plt.semilogx(alphas, scores_lasso - std_error_lasso, 'b--')\n", + "plt.fill_between(alphas, scores_lasso + std_error_lasso, scores_lasso - std_error_lasso, alpha = 0.25)\n", + "plt.ylabel('CV score +/- std error')\n", + "plt.xlabel('Log lambda')\n", + "plt.axhline(np.min(scores_lasso), linestyle='--', color='.5')\n", + "plt.xlim([alphas[0], alphas[-1]])\n", + "plt.title('CV score as a function of lambda')" + ], + "execution_count": 89, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "Text(0.5, 1.0, 'CV score as a function of lambda')" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 89 + }, + { + "output_type": "display_data", + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEaCAYAAAAG87ApAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdd5xU5dXA8d+Z7QVY2IWlLOzSURDpzd6xksTeNUZek5hoNJY0jUZNYotJNBo1hpAo9gYqig0soBRBem8LLOwuLNvLzJz3j3sXZyszsLOz5Xw/jjvzPLecuZe5Z57nufeOqCrGGGNMsDyRDsAYY0zrYonDGGNMSCxxGGOMCYklDmOMMSGxxGGMMSYkljiMMcaExBKHMUEQke+LyHYRKRaRkc243stF5IPmWl/Aeo8RkfXu+/1ePfVbROTUMKz3RBHJDmH6a0Tk86aOwzTOEkcbJCKXicgi90O/S0TeE5FjReQS9wMvtaaPFpE9InJOpGJuBR4GblTVZFX9JhwrEJEsEVERia4uU9XnVfX0cKzvIO4FHnff75sRWL9pwSxxtDEicgvwGPAAkA70Af4BTAHeBFKAE2rNNhlQYHbzReoIPEi2cJnAykgH0Yza2/s1IbDE0YaISCecb4o/VdXXVbVEVatUdaaq3qaq5cDLwFW1Zr0KeEFVvfUsc4CIzBWR/SKSJyIvBdQNFZE5IrJXRHaLyK/d8jgReUxEdrqPx0Qkzq07UUSyReQOEckB/i0iHhG5U0Q2iki+iLwsIl0aeI+dRWSWiOSKyD73eUZA/TUisklEikRks4hc3sByxonIfBEpcFtlj4tIbD3TxYlIMRAFLBORjW65isiAgOmmich9td7jrW5LbpeIXBswbYKIPCIiW93t+rmIJADz3EkK3NbixNpdMSIySUQWuvMtFJFJAXWfisgfROQL9/1/ICJp9b1/d/rrRWSDu//eFpGebvlGoB8w040jrqFlBLMt3W31E7frq8iNsb+IfCkihe7+jq21zF+7/962BO5DEUl1Yy0Uka+B/rXm+6s4XYqFIrJYRI5rLHZziFTVHm3kgdNy8ALRjUxzDFAIJLivOwFlwIgGpp8B/AbnS0Y8cKxb3gHYBdzqlncAxrt19wILgG5AV+BL4A9u3YlujH8G4oAE4CZ3+gy37J/AjAbiSQXOBxLddb4CvOnWJbnvbbD7ugcwtIHljAYmANFAFrAauLmR7abAgEZeTwPuq/Ue7wVigLOAUqCzW/8E8CnQCychTXLfd5a73OiA5V4DfO4+7wLsA650477UfZ3q1n8KbAQGudv1U+BPDbyfk4E8YJS77r8D8wLqtwCnNrI9DtQfbFu67+ktoCMwFKgAPsJJTp2AVcDVtbbdo25cJwAlAfv0RZwvP0nAMGBH9fZx669w/41E4/zbzAHiI/3ZbGuPiAdgjybcmXA5kBPEdOuBy9zn1wPLGpl2OvA0kFGr/FLgmwbm2QicFfD6DGCL+/xEoDLww+weaE4JeN0DqKKRBBgw7Qhgn/s8CSjASSwJIW67m4E3GqkPNXGUUTMB7HEPrh637uh61pFF44njSuDrWvPMB65xn38K/Dag7ifA7Abez7+ABwNeJ7vbPMt9vYUgE8fBtqX7no4JeL0YuCPg9SPAYwHbzgskBdS/DPwOJ8lWAUMC6h4gIHHUE8u++ra1PQ7vYV1VbUs+kBbEuMF0vuuuutJ93ZDbAQG+FpGVIvJDt7w3ToKoT09ga8DrrW5ZtVx1us2qZQJvuF0dBTiJxIczRlODiCSKyD/dbp5CnO6dFBGJUtUS4GLgBmCXiLwjIkPqC1BEBrndXDnuch4AGuzWOQT5WrPrrxTn4JyG00JraNs1pvZ2xX3dK+B1Tj3rPOiyVLUY599Prwamb1CQ23J3wPOyel4HxrnP3ZfVqv/9dMVpSWyvVRcYyy9FZLXblVeA06Jpyv1qsDGOtmY+TjdAndMna/kvcIqITMT5Fvx8QxOqao6qXq+qPYH/A/7h9u1vx+lqqM9OnGRQrY9bdmCxtabfDpypqikBj3hV3VHPsm8FBuN0i3UEjnfLxY33fVU9DafVsgZ4poEYn3TrB7rL+XX1MoJUitNdVq17kPPlAeXU6pt3HexW1bW3Kzjbtr7tdDA1liUiSThdPIeyrMPdlrV1duOpVv3vJxenNdK7Vh0A7njG7cBFON2CKcD+w4zF1MMSRxuiqvuBu4AnROR77rfzGBE5U0QeDJhuC/A5zvjFHFXNqX+JICIXBgw+78M5uPmBWUAPEbnZHUDuICLj3elmAL8Vka7u4OxdwP8aCf0p4H4RyXTX2VVEpjQwbQecb6gF4gyg3x0Qa7qITHEPOhVAsRtrQ8spBIrdVsmPG4mvPkuBy0QkSkQmU/dMtXqpqh94DnhURHq68090B6Bz3XgbSsjvAoPEOd06WkQuBo7E2RehmgFcKyIj3HU/AHzl/tsI1eFuy/rcIyKxbjI4B3hFVX3A68Dv3X/bRwJX14rDi7Mdo0XkLpxxFdPELHG0Mar6CHAL8FucD9B24EacU3ED/QfnG2dj3VQAY4GvxDmz6G3gJlXdpKpFwGnAuTjdI+uBk9x57gMWAd8Cy4ElbllD/uou+wMRKcIZKB/fwLSP4Qz85rnTBZ5C7MF57zuBvTgH84YOYr8ELgOKcFolLzUwXUNuwnnvBThjS6Fc6/BLnO2y0I3zz4BHVUuB+4Ev3G67CYEzqWo+zkH0VpxupduBc1Q1L8TYUdUPccYNXsM5yaE/cEmoywl4P4ezLWvLwfmSshOnNXyDqq5x627E6dbKwRlX+nfAfO/j/HtYh9OFVU7Nbi3TRMQdQDLGGGOCYi0OY4wxIbHEYYwxJiSWOIwxxoTEEocxxpiQWOIwxhgTktZyZ9J6paWlaVZWVqTDMMaYVmXx4sV5qtr1UOdv1YkjKyuLRYsWRToMY4xpVUSk9q1rQmJdVcYYY0JiicMYY0xILHEYY4wJiSUOY4wxIbHEYYwxJiSWOIwxxoTEEocxxpiQWOIwxph25O+PH/5PaVjiMMaYdmLHDvj5zw7/l3QtcRhjTDvx7rtNsxxLHMYY007MnKnEppQd9nIscRhjTDtQVgZz5kBcv92HvSxLHMYY0w6sXAk+VeItcRhjjAnGmDFwwu/nUzJ32GEvyxKHMca0AzmFZaxYIZTmJh32sixxGGNMG7dsGYwf66FoWW+iY1rwdRwi8pyI7BGRFQFlI0RkgYgsFZFFIjLOLRcR+ZuIbBCRb0VkVLjiMsaY9mbWLNi2Lo6qXV044fjDX144WxzTgMm1yh4E7lHVEcBd7muAM4GB7mMq8GQY4zLGmHZl5kwltlshlflJnHtuC74AUFXnAXtrFwMd3eedgJ3u8ynAdHUsAFJEpEe4YjPGmPZizx74+muI65PL5IuKOPfcw19mc//m+M3A+yLyME7SmuSW9wK2B0yX7Zbtat7wjDGmbXnvPVAVOh21ixmP9yYl8fCX2dyD4z8GfqGqvYFfAP8KdQEiMtUdH1mUm5vb5AEaY0xbkpEBaUfvol/njnSIi22SZTZ34rgaeN19/gowzn2+A+gdMF2GW1aHqj6tqmNUdUzXrl3DFqgxxrQF3YfsR/ruZN6jw/nii6ZZZnMnjp3ACe7zk4H17vO3gavcs6smAPtV1bqpjDHmMGzfDi9+mkfphm507KRMmnTweYIRtjEOEZkBnAikiUg2cDdwPfBXEYkGynHOoAJ4FzgL2ACUAteGKy5jjGkvHnkE/vZEFtFxPi44T4huoiN+2BKHql7aQNXoeqZV4KfhisUYY9obVXjrbSU2fT8VO7pwzjlNt2y7ctwYY9qgtWthy2ZB4qrwRCmTa19VdxgscRhjTBs0a5bzN+PMtcybq3Tp0nTLbu7rOIwxxjSDmTOV+PQijh+TwDHHNG0bwVocxhjTBt10317ij8gm99NBVFY27bKtxWGMMW3Q4rydVG5PZ/mWjsTENO2yLXEYY0wb88gj8MqCaMq3pXLejwU5/Psa1mCJwxhj2hCvF/5wn1LeMQ1/VVSTnoZbzcY4jDGmDZk/H/YXCOKBxCTlhBMOPk+orMVhjDFtyKxZIB4/HbpWcvaJQmzT3NewBmtxGGNMG/L2TD/xffZy9W15/Pvf4VmHJQ5jjGkjiouhSqqIy9rDaUemh209ljiMMaaNSE6G0361gvK1PZn2QLewrcfGOIwxpo0oK/Mzb1kR5btSGHpk+NZjicMYY9qAoiLo0RPI6g8QltNwq1lXlTHGtAFz5kBJsQdvQSI9M/wMHx6+dVniMMaYNmDWLIiKr8K7J4Up53qa/GrxQNZVZYwxrZzfDzNnKbGZezj7LOGHl/UM6/qsxWGMMa3cwoWQlyskDdrDr26KZ8yY8K7PEocxxrRyPXvC0HO2k+BLIE1Swr4+SxzGGNPKdenmpbjnFra/O4TZs8N/WLcxDmOMacV274a/zdhPwSrngr+zzw7/Oi1xGGNMKzZzJjzwi1Ti0qMZMVLp1SuMp1O5rKvKGGNasQ8+UKKTy6nc05Hzzg1/0gBLHMYY02r5fPDBHCUqtQgI79XigSxxGGNMK7VkCewv8NBheDYLV5UyenTzrNfGOIwxppWaO9f5O2BECaOHJDXbeq3FYYwxrdSPb/TS7bxvyHn7aDZvbr71hi1xiMhzIrJHRFbUKv+ZiKwRkZUi8mBA+a9EZIOIrBWRM8IVlzHGtBXzNuRSvqcD21cmk5bWfOsNZ1fVNOBxYHp1gYicBEwBjlbVChHp5pYfCVwCDAV6Ah+KyCBV9YUxPmOMabU++QTuvD+O8k1JjB8PHTo037obbXGISJSIrDmUBavqPGBvreIfA39S1Qp3mj1u+RTgRVWtUNXNwAZg3KGs1xhj2oM334Tln3aiMrcDZ05untNwqzWaONxv/GtFpE8TrW8QcJyIfCUic0VkrFveC9geMF22W2aMMaYe7872E92lGFQ47bTmXXcwXVWdgZUi8jVQUl2oqucd4vq6ABOAscDLItIvlAWIyFRgKkCfPk2Vz4wxpvXYvh02rPOQdNR+xhyZwJgxsc26/mASx++acH3ZwOuqqsDXIuIH0oAdQO+A6TLcsjpU9WngaYAxY8ZoE8ZmjDGtwpw5zt+ex2zn8yd642nm82MPujpVnQusATq4j9Vu2aF4EzgJQEQGAbFAHvA2cImIxIlIX2Ag8PUhrsMYY9o0r1dJ6FnA2KPi8Hiad3wDgkgcInIRzkH8QuAi4CsRuSCI+WYA84HBIpItItcBzwH93FN0XwSuVsdK4GVgFTAb+KmdUWWMMfUbf+Z+4o/cwUs/H82uXc2//mC6qn4DjK0+A0pEugIfAq82NpOqXtpA1RUNTH8/cH8Q8RhjTLtVXg5zVu2hfEsaGX2UHj1aYIsD8AScNguQH+R8xhhjmthjj8FvLsyiYlsqZ02OzKE4mBbHbBF5H5jhvr4YeDd8IRljjGnI7Pf9+KOq8FcmNftpuNUaTRwiIsDfcE6dPdYtflpV3wh3YMYYY2oqKYEvvxA8XSvwFCVy0knN300FB0kcqqoi8q6qHgW83kwxGWOMqce8eVBVJaSNyOb+H3QkJSUyNzgPpoNsScAV3sYYYyJkzhzwRPsZe3IxU6+L3K9iBJM4xgPzRWSjiHwrIstF5NtwB2aMMaam40+rIHn0Jjrl9aGyMnJxBDPGMRXY2jzhGGOMaUhJ5xz8FTHMfLwncm/k4ghmjOMJd4zDGGNMhCxbBq++V0bFlj6cebIQExO5WGyMwxhjWoGHHlJmPtSXqoJEzjgjMmdTVQtmdGU8cIWIbMG5O67gNEaGhzMwY4wxDr8f3ntfiepUhq8sLmLXb1QLJnHYz7gaY0wEffst7M3zEJfho1dvPwMGRPbmHcHcHXcrzi3PT3aflwYznzHGmKZRfRv1oVes4pvFHiSyPVVB3R33buAO4FduUQzwv3AGZYwx5jsffazEdi3iuKOT6do10tEE13L4PnAe7q//qepOnN/lMMYY0wxu/tMe4gfu4tsXBuH3Rzqa4MY4Kt3TchVARJLCHJMxxpgAX2zOpWJbT/ZXJDT7r/3VJ5gQXhaRfwIpInI9zm9xPBPesIwxxgA8/jj8728dqNzZmTMjdBv12g7a4lDVh0XkNKAQGAzcpapzwh6ZMcYYnvynnx27O6N+ifhpuNWCukuWmygsWRhjTDPKyYFVKzzE9vSSkKhMnBjh06lckbu9ojHGmEZ9+KHzN7FnEZecl0JsbMtIHC2jw8wYY0wdH3ygRCdWctb1OTz5ZMs5XLecSIwxxtRQrlXE9d3NpP6pkQ6lhga7qkRkOaAN1du9qowxJryOv3Yr7yxM5vGf9+SnKyMdzXcaG+M4x/37U/fvf92/l4cvHGOMMQAVFfDZunwqtmUx8ZKoSIdTQ4OJw70vFSJymqqODKi6U0SWAHeGOzhjjGmvzj1P+Xz1QHxlMZx+eqSjqSmYMQ4RkWMCXkwKcj5jjDGHoLwc5s4Fn885i+qUUyIcUC3BnI57HfCciHRyXxcAPwxfSMYY07598QVUVggxldEcPcJP164t67t6MIljr6oeXZ04VHW/iPQNc1zGGNNuffABSJSfYd/fyn3fa3m/3B1MGnsNnIShqvvdslcPNpOIPCcie0RkRT11t4qIikia+1pE5G8iskFEvhWRUaG8CWOMaStUYdY7fmJ77mPKOR7OOivSEdXV2Om4Q4ChQCcR+UFAVUcgPohlTwMeB6bXWm5v4HRgW0DxmcBA9zEeeNL9a4wx7YoqnHThXrLf30daSc9Ih1Ovxlocg3FOyU0Bzg14jAKuP9iCVXUesLeeqr8At1PzGpEpwHR1LMC5E2+PoN6BMca0IR4PePttpXx9D97+T0qkw6lXY6fjvgW8JSITVXV+U6xMRKYAO1R1mdT87cNewPaA19lu2a56ljEVmArQp0+fpgjLGGNahOJieOzvPuZur6QyL5kzWthpuNWC+gVAEekoIjEi8pGI5IrIFaGuSEQSgV8Dd4UcZQBVfVpVx6jqmK4t4TcUjTGmibz0Evzu11Hs/MY5trW06zeqBZM4TlfVQpxuqy3AAOC2Q1hXf6AvsExEtgAZwBIR6Q7sAHoHTJvhlhljTLvx9DNKxx6lSFES3bsrQ4dGOqL6BZM4Yty/ZwOvBJxZFRJVXa6q3VQ1S1WzcLqjRqlqDvA2cJV7dtUEYL+q1ummMsaYtmr5cvj6KyHqiM1IfhcmTxakZdxFvY5gruOYKSJrgDLgxyLSFSg/2EwiMgM4EUgTkWzgblX9VwOTvwucBWwASoFrg4jLGGPajGeegahoP0nDdvDC490ZkhYX6ZAaFMxPx94pIg/itAJ8IlKKcxbUwea79CD1WQHPle9upmiMMe3Ozl1+Og3dzbC+8UwY1JmYlnVfwxqCuo5dVfeqqs99XuJ2LxljjGkil965k7hR61j44EQWL2xZtxipzX461hhjImz3buW1JdlUrMqkKC+afv0iHVHjLHEYY0wErVkDw4ZB6llxlK7MYMoUoVu3SEfVuJDaQyLy+zDFYYwx7dKzzzq30fBWRFFeHM31B70vR+SF2pF2XliiMMaYdqiiAqZNU1KOyEU39SYzSzn11EhHdXChdlW10LOKjTGm9XnzTcjPF7qdvJkpw3tz2uDOeFr2uDjQ+N1xLwU+UNX8gOLR4Q/JGGPah3/9S0lKraTzoH3cc+NwerTMexrW0Vhu6wO8IiKficjvRWQ8Ne9oa4wx5jD84dFSUiYvpcOao/AWBfNrFS1Dg4lDVf+sqifjXNG9DOfnYpeIyAsicpWIpDdXkMYY0xbNzd5OZWkUi17rxTfftJ6RgGCuHC8C3nAfiMiROD+8NB04I6zRGWNMG+T1wiWX+tmQVopvdV969lTOOqv1JI6gh2FEpDOAqq5S1UdU1ZKGMcYcgnfegdde9bB9SxT71nXh2muF6FZ0VV0o4/cfhS0KY4xpR55+WklMqYSSRNQvXHddpCMKTSiJo/W0o4wxpoXavh1mz4b4odtIj+vA2WcrfftGOqrQNNo4EpGrqp8CnQNeo6rTwxmYMca0Rc89B36/ED9sGw/95AjOHNb6vpMfrFctMA/GAVk4ScROyzXGmEPQK8NPr4m76NEllpOHdG0VF/zV1mjiUNV7qp+LyBRVvTf8IRljTNs14Ng85JvVLHnwFJ7rJfy0Ff4SUSjj+K2vPWWMMS3IRx8pM7ZmU7oyA79PWsV9qeoTSuK4MmxRGGNMG7drF5xxBnQa34nK9b04/nhl8ODW+X086N41VV0RzkCMMaYtmzYNfD5BUooozo3n+utbZ9KA0G+rbowxJkR+Pzz7rNK5fwHsTCclRTn//EhHdeha0bWKxhjTOn30EWzaJKSes5nrLo3nrF49SEiIdFSHLujEISKJqloazmCMMaYtmjVLSUrx0mHIbq45aSJH9Ih0RIfnoF1VIjJJRFYBa9zXR4vIP8IemTHGtBG3/K6MXtd8gX/eaNZ+3THS4Ry2YMY4/oJzF9x8AFVdBhwfzqCMMaYtUIXdu5U3vtnO/txYtn3dlfy81jsoXi2owXFV3V6ryBeGWIwxpk157z3I6gv/eqMQXdOX5GTl4osjHdXhCyZxbBeRSYCKSIyI/BJYHea4jDGmVVOFu+5SEjtVURC9j30r0rn0UiE5OdKRHb5gEscNwE+BXsAOYIT72hhjTAPeegsWLxY6HbOe2M19qarwcP31kY6qaRzs7rhRwF9V9fJQFywizwHnAHtUdZhb9hBwLlAJbASuVdUCt+5XwHU43WA/V9X3Q12nMca0BH4//O4uJa1nJb5+Wzm780iquitjx7b+8Q04SItDVX1ApojEHsKypwGTa5XNAYap6nBgHfArOPBztJcAQ915/uEmLWOMaXW+/hpWroDkSesYkJ7I3T9J4Zln2kbSgOCu49gEfCEibwMl1YWq+mhjM6nqPBHJqlX2QcDLBcAF7vMpwIuqWgFsFpENwDhgfhDxGWNMizJmrJ8b/7GJN9bsIHXjWBI98ZEOqUkFM8axEZjlTtsh4HG4fgi85z7vBQSeuZXtltUhIlNFZJGILMrNzW2CMIwxpumUlMCW/BI+z91M/LrBvPJkKmvWtJ3WBgTR4qj+TQ4RSXZfFx/uSkXkN4AXeD7UeVX1aeBpgDFjxtgPShljWgyvF0aOUrqPKCevh5/8z/tw7rltZ2yjWjBXjg8TkW+AlcBKEVksIkMPdYUicg3OoPnlqlp94N8B9A6YLMMtM8aYVmP6dFi/TtjmySZuzRBKi6K49962lTQguK6qp4FbVDVTVTOBW4FnDmVlIjIZuB04r9Z9r94GLhGROBHpCwwEvj6UdRhjTCRUVsK99yo9BpTi7ZrLjs96c/75yogRkY6s6QUzOJ6kqp9Uv1DVT0Uk6WAzicgM4EQgTUSygbtxzqKKA+aICMACVb1BVVeKyMvAKpwurJ+6Z3QZY0yr8NxzsHWr0Oey1Qzo3AXvOLjnnrbX2gCQ73qLGphA5A1gCfBft+gKYLSqfj/MsR3UmDFjdNGiRZEOwxjTzqnCkCOUEkqJmvIpj1w0nAtG9z74jBEiIotVdcyhzh9MV9UPga7A68BrQJpbZowxBhCBp14oJP6kJXTf258hHdIjHVJYBXNW1T7g580QizHGtDpeL1T5fczasJlyTwVL/zuYv/iFf/0r0pGFTzBnVc0RkZSA151FxG4HYowxwGOPwYSJysxFuSSsGorfB7/5TaSjCq9guqrSqu8nBQdaIN3CF5IxxrQORUXw5z8rBd5Syoo9bPmiO9deK/TrF+nIwiuYxOEXkT7VL0QkE7AL74wx7d7f/w55eYKOWkHcimF4BH7720hHFX7BnI77G+BzEZkLCHAcMDWsURljTAtXUAAPPaRkjiyE9H0MLOrAOT8W+vQ5+LytXTCD47NFZBQwwS26WVXzwhuWMca0bE89BQUFQoeRyzlxYBqP/z6GDnGRjqp5BDM4fgxQpqqzgBTg1253lTHGtFvX3eDl9JvWQ2wVk1IG0DE+Bmmb1/vVEcwYx5NAqYgcDdyCc7fc6WGNyhhjWqjiYti128+SHXlsTFxPzNLh/PLqLuzfH+nImk8wicPr3oxwCvCEqj5B09xW3RhjWhVVuP56Zdw4eOHzHVTkJrP5qy7ceKPQqVOko2s+wSSOIvdnXa8A3hERDxAT3rCMMableeIJePFFYcwZBczbnEPMsmEkJ8Ntt0U6suYVTOK4GKgArlPVHJxbnj8U1qiMMaaFWbAAbrlFGXt8Oet7LKRTWVc2L+zCzTcLqamRjq55BXNWVQ7waMDrbdgYhzGmHcnNhQsuVLp295N42mIKK/yclt6fvX2UW25pJyPiAYJpcRhjTLvm9yv9B3uZcP1athQX8H/H9+emqfFs3CCkpBx8/rbGEocxxjRCFcqiSply+yYWl2xmTFI/ipdnkJmaSEw7He1tMHGIyG0iktGcwRhjTEvy7rtw7PF+PvpmP09/tom+ySl8+fQgnn4sgeLi9tdFVa2xMY6ewHwR2QLMAF5R1dxmicoYYyJsyxa44golNd3PMwvWEePxUDJnNHm7PXz2mdChHV+U0GCLQ1V/AfQBfgscBXwrIrNF5GoRacebzBjT1pWXwwUXKFU+ZcgVy9ldWkL/nWNZOC+eRx8Vxo+PdISR1egYhzrmquqPcU7D/QtwM7C7OYIzxphIuPlmWLxYOOfGbJYX7eS0nv2Z+VxnLrlE+elPIx1d5AVzd1xE5CjgEpxrOvKAX4UzKGOMiZR9+2D2+37OvqKIr3UFo/qkMPXsbpyTqZx8sqfd3I+qMQ0mDhEZCFyKkyx8wIvA6aq6qZliM8aYZhedUMWjM/Zy3+zlpEgcZ3QfwpDuHRmTaSehVmtsS8wGYoGLVXW4qj5gScMY01YVFsJvf6cs21LEtIXrKaqqpNuasfzski7szg6qc6bdaGxrTAbSVXVFYKF7m/UcVd0Y1siMMaaZ+Hxw+eXKe+/BpoQ8vi3czwQZzksvduTnP4f+/SMdYcvSWIvjL0B9NwouBB4LTzjGGNP87rwTZs0SLvxZHvML1zOiQ29m/j2DCROVh+zOfHU01uJIV9XltQtVdbmIZIUtImOMaUbPPQcPPwxnX5apvwYAAB8RSURBVFzM8o5L6BmTzLf/OZLEBHjlZSE2NtIRtjyNJY7G7sCS0NSBGGNMcysqgttuV8ZMqqLw6EX4SpWbz+jPwhLh9FOEDLt3Rr0aSxyLROR6VX0msFBEfgQsDm9YxhgTftFxPh5/vogXVqxlRV4Jt548hHH9OnPhH6MiHVqL1tgYx83AtSLyqYg84j7mAtcBNx1swSLynIjsEZEVAWVdRGSOiKx3/3Z2y0VE/iYiG0TkWxEZdbhvzBhjGrJ/P/zrOT9rcoqYv3czy3PzOKXLEdx/TV9yNydGOrwWr7FbjuxW1UnAPcAW93GPqk50f6PjYKbhnJkV6E7gI1UdCHzkvgY4ExjoPqbi/M65McY0Oa8XLrlEueH/hP/OzuftZTuZmJ7BWw9nERcr9OtnV/gdTDA/5PQJ8EmoC1bVefUMok8BTnSf/wf4FLjDLZ/u/rb5AhFJEZEeqror1PUaY0xjbrsNZs8Wrr49j7e2rGVIagpLnh1KRZnw6cdCly6RjrDla+5LIdMDkkEOkO4+7wVsD5gu2y2rQ0SmisgiEVmUm2s36zXGBO/pp+Gxx+Dcy4pZnLCYrklxFL0/mg1ronjpJWHo0EhH2DpE7Bp6t3WhhzDf06o6RlXHdO3aNQyRGWPaoh074Gc/U8YdV0HekQvxq3LzyYPpGB/Do48KZ54Z6Qhbj+a+jn53dReUiPQA9rjlO4DeAdNluGXGGNMkunT18ed/ljAnbxVr8su4Y/IQxvRP4aK3o+zGhSFq7hbH28DV7vOrgbcCyq9yz66aAOy38Q1jTFMoKICPPvazJqeQtdHrWZmXz0kpR/D4L/oQW56Ex4MljhCFLXGIyAxgPjBYRLJF5DrgT8BpIrIeONV9DfAusAnYADwD/CRccRlj2g+vFy68UDnnbOGleXt4b0UOk7r14bUHMykrjiI52TLGoQhbV5WqXtpA1Sn1TKuA/TyKMabJqMJNN8GHHwpX35HLKys3MjQtla+eOgJR4Z1ZdgbVobJ7BRtj2hyvF378Y3j2WZhyVREL4xaTHptA3tsj2bY5ivffFwYNinSUrZf9Mokxps156SXl2Wfh0qnF5Az+Go8IUycMoSg/hieeEE4+OdIRtm7W4jDGtBmq4Fdl9MnFPDrNy4f7V5CbU8FvzzqCE4Z05NIlHuLiIh1l62ctDmNMm7BlCxx7rPLu58Vs21vCu/nfsmpnIad0GMYrD2fQLSHRkkYTsRaHMabV++YbOOsspbRMWb25nDe/WElucQWXDj6Sx2/OoE9vweezM6iairU4jDGt2ocfwgknKOrx89uncnhh6zfsL6ti6sij+O89fYiLFWbOFDp2jHSkbYe1OIwxrdannzotjcx+Pq6/fwfPLV1NQmwU1wwZwUM3dqOyXJg9W+jbN9KRti2WOIwxrVb/oRWcf6WfI8/K5p+L19OzUzx3TB5C+Z5EuqUJL74oDBsW6SjbHkscxphWxe+HRx6Bcy4qo8BbSr9zdvKvr7ZxRI8OnNdrKFlp0QwZlswPvhU81hkfFpY4jDGtRkUFXH218tJLQk6pj7L+W3h3RQ7j+3YhbfNR/OTmZKZPV4ZdaQPh4WT52BjT4qnCrFkwcqSTNG64rZgd3Vfw7ooczjiiO/LVCP7x52QuuEC58EJLGuFmicMY0+I99JBy7rlQUu7nvif3sbXXNyzYnM9FIzPZ+MowXvp3Aj/7Gbz4ohAfH+lo2z7rqjLGtEjbt0NpKaT1qmDESWXcclcMx567n798vJYd+8r4yYn9ScrvwaOzY3nwQfjlL+326M3FEocxpkUpLIQ//xkefVQZOc7LI88VE53sQ4/cwh1v7MLjgVtOGczY/p0ZnJ7I6eOEgQMjHXX7YonDGNMieL3O3WzvvlvZs0c4/bxKrv9FCR+t3s0ri7PZX1bFxP6pHNOlL3+8Lo2//x3G9Y2ikyWNZmeJwxjTIvz9cT+3/MLDiHFeHniyhIrOefxlwTZ2FJQxOL0Dvzx9EHtWdeE31yWTEC9kZli/VKRY4jDGRER5OcycCXFxytgTyhl9ehkP/jOaXkcX8MJXW1nxdSHdO8Zzy6mD6FyRxhN3JPHVZ7EMHqK8+47Qr1+k30H7ZYnDGNNsVGHxYvj3v2HGDGXfPuHYk6t4cGgphVUVrIjO5h9v5JIUF83VEzM59Yh0oqM8fDornnUrYnj0UfjJT8TuchthljiMMc3m2h8q/5kmxMUpx59eydnnV3Dk6HJeXrSTd77dhV+Vs4f34IyBGbz532TmrBZuvSmacTfGcMNV0LlzpN+BAUscxpgwqayEd96B6dOVhx/z4o8v5+hjldv7ezjl7EoKvCXMXZfLs6/mUlTuZWL/VC4c2ZuFH3Ti2psTyc/1MHWq0inBGcuwpNFyWOIwxjQZvx8WLoQZM+D555W8PCGtm/LhV6UMH+1l2EQvhRvz+NNHuWzKKyFKhFGZKZx3dE9Kt3fm9suT2LgumkmTlIffgokTbQC8JbLEYYw5LIWFsHcvZGXBlu0+JkyIIiZWOe6USs6+oILREytZtXs/f/sol0Vb91LlUzK7JHLVxEyO6Z9GUmwM0dGweU8s+KJ49VX4wQ/ELuZrwURVIx3DIRszZowuWrQo0mEY0+5s2OB0Q82apcydCyec7OOhZ4opq/TxxccxHDXKSwmlzF2fy2fr89hbUklyXDTHDEjjhEFd6RafxCfvxfHu63EMHaY881QUCbFReL0QbV9nw05EFqvqmEOd33aRMeag/H4O3KL8B+crb7zuNAf6DvRx0TVVHHdqJWWVPnKLyilJz+HRefms3V2ECIzISOGqiZmM6tOZNctief7BOD6ZHUdZqTBwkDJxtJAQ6yzbkkbrYLvJGFNDZSWsWgVLl1Y/lDVr4fMl5ZT7Kxl+jIesYR4mnVhJ9wwf6/cU8c22Av736j627ysDoFdKApeO68NxA9PwFsaTlu4nNlqY+04in82J5vLLhGuvdcYwrEuq9bGuKmPasX37YNkyJ0Fcdhl0SfXz4MPKb+6MAiA+QRkwxMvAI3xc9/NSuqQpxeVelmUX8M22fSzL3k9xhZcoEYb06MCoPp0Z2TuFLvEJzP0glndei2PRlzG895GX006IJi9PSEqCpKQIv/F2zrqqjDENqqqCHTtg2zbo3x969YLly+H222HVamXb1u++7kelFDHmuEoGjPFw71+jGXiEl4xMPyLKjoIyPt9ewDfz97FudxF+hQ7x0Yzqk8LIPp0ZntGJxNho9uULT/wxkXlz4iguFDIzlbvuEo4aHIPHA926RXBjmCYTkcQhIr8AfgQosBy4FugBvAikAouBK1W1MhLxGdOSqUJREeTnO4+9e52/w4bBUUfB1q1wySXK1m2QswtUneTwh4fKOf/yStbtgS3ZiQwc5uPsi5zWxMAjvKR2VSq8PioSi6BfKR/uLGHr8lK27S2lwusHIDM1kfOO7sWoPin075pM9tYovvwklryOymVX+uiRFcvyhXF8f4pwzTVw4on2861tUbN3VYlIL+Bz4EhVLRORl4F3gbOA11X1RRF5Climqk82tizrqjKtiep3vxexebNz8C8udh4lJdCjB0yY4Ex3zz1O+b591QlCOW+KcuPPlfwCP73TY+os/4abK/jJreXk7fVz2w3JdO/lI72Hn+49/aT39NN/iJcuqd993gtKK9maX8rW/BK27HX+7tpfTvUhITE2iszURDK7JJGVlsiwnp1ITY5j6cJo5n4Qy/xPYtm2xenS+t73lDfekDrv07RMrbWrKhpIEJEqIBHYBZwMXObW/wf4PdBo4qiocJrdgZKSOHDzs3XrnIG+QB06QGam83zNGudWzoE6dYLevZ3nq1Y5Z5ME6tzZae4DrFhRN6bUVOcA4PPB6tV167t1cx5VVU58taWnQ1qa8942bqxb36OHE0NZmXPwqa1XL+c9lJQ43RO1ZWQ426CoyOnCqK1PH0hMhP37YefOuvVZWZCQ4HzL3bXLKQv87jFwIMTFQV6eM7/qdw9wvhXHxDh19dWPGQNRUc5727HDKff7v5vmpJOc6Vatcn7ox+//7uHxwNlnO/ULFjjvv7rO54P4eLjwQqf+nXecdfh83z1SUuD66536Z591Tjn1ep19VVXlbLtf/9qpv+OO7+orKpzHUUcpj/5F8SuccrKwcSNUVjg386uogNMnK/970YtPlbFjY8nPr3l0Pe/8Kh5+ohyfKg8+2AEFOnRUOqX46Zii7C6pYOn2ClThxjvj6ZiidOzkp1NnpVOKktrNT3GFEp8Ej/1nP/tKK8kvriSvuIJdxRUsX1NJfnEFecXO35JK34F1pyXHkpWaxMR+qWSmJpGVmkhachw+r7B1UxRb1keROqiSpLgoZs1I4qP3ozjpJLjtVmeb9+373XuxpNH2NXviUNUdIvIwsA0oAz7A6ZoqUNXqw3g20Ku++UVkKjAVICbmaIYPr1k/8Rg/r7/jZIszz4pl08aa7eTTzvAx/aUqAE48KY7dOTX/lX//Ah//eNapHzc+jpLimvWXX+3l4b86YR51VN3fqPy/n3r5/f1eiovqr7/1Di+//JWXnF0wcljd+rv/UMUNP/OxcYNw7Ji6d3J78LEqrrzGx7KlwuQT69Y/8Uwl51/k54vPPJx/Tmyd+ukvVnL6mX4+eM/DVZfUrX9tViXHHOfn9Vc9/ORHdetnf1rB0SOU6dOjuOMXdb/1fr6ogv4DlKeeiuKe39WtX7K6jPTu8PBfo/nLg3Xr12wrIylZeeChGJ59sm79tvxSAO77cywzptf855uUpKzYUooC9z8Yx6w3atZ3S/cz8oQSVOHhxxL49MOa9Zn9fBxzdgmK8sy/k/hmURTR0bgPZfBQH5OvKEYVFn6bxNZNHqKiICZWiY2F3BIvC7c48WUOSaRzdyEmFmLjlNhYpe9AH2tynH+bt9zjbNuERCUxSUlIVDqn+skrdjLonGV7a3TxqCplVT52F3oprvAy9IwCiiu8FJV72VPhpXiPl8JtVex1E8Xe0kpqdyYkxUWRlhRHWnIcQ7p3IL1jPFmpifRJTSI57rttsXRhNE8/FceG1dFsWh9FVaXzGbjwTB+DM6L45xPOlxcb4G6/ItFV1Rl4DbgYKABeAV4Ffq+qA9xpegPvqeqwxpaV1W+UTr11bo2ylM5+Ro53DuwL5sZQWlrzwJ/Wzc/w0U79Fx/HUFFRsz69h5+hI5z6uXNi8Hlr1vfs7WPIMOeb2sfv1T2w9s7yMfAIH94qmPdh3fqs/j76DfJRUQ5ffFy3vv8QL5n9/JQUw4K5desHD3MGLAsLhIVf1D2wHjnCS49efvbmC0sW1K0/erSXrt395OZ4WLqo7veG0ROr6JKq7NrhYeXSuvXjjqmiY4qSvdXDupUB9e5mGn9cJUnJsG2zh41roxFxv4GKIsD446uIi4Otmzxs3xzl1nFgurHHVBEd7dTv3hmFx6NOPU79yPFeRCB7i4d9ez14PCAexeNxWhyDjnT2ze6dHkpLBPEoIhDlgehY6N7TaUIW7hd8Xqd144kCj0eJiuaw7rrqV8XrU6p8fveheH1+Kn1+vH6lyus8r/D6Ka/yUV7lp8Ib+Nd5Xl7lOzBNSaWX4gofJeVOS6UhibFRdIiPpktSLGlJcaQmx5GWHPvd36Q4EmKdbqWqStiZ7WHLxig2rI5m/epoNqyO4m/PljFurIdZr8dw96+jGTkSRowQRoyAESNgyBBrTbQVrbGr6lRgs6rmAojI68AxQIqIRLutjgygno6Umqqq8qkqn16jLLesNx+s7A9+L+z7vM48OaVZ5KzKAm8FFM6vU7+juD87VvWGylIo+bpO/bbCQWxb1RPKi6BicZ36zfuPYPOqdCgtgIqldeo3FAxjw6o0KM6Dyrp9XWv3jmBdeQpauBuqqvu6vvu0rsofzeqSDmjBTqiq29e1Im8cqwoT8e/djlbV7etaljsR2ReHP28L6t1Sp35JzrFIbjT+PRvQquw69Qt3nQi7wJ+zFq3cVbPSE8WX244FQHetgvI9zvPq/0XH8cn6CSggO1cgJfnf1QMak8B7K8c6k+5chpTtD1i44otNZtOSkYASv2spnopiAg+lVXEd+apwGKjScfc3eLzONQXVx9vyuBT2dhqM36+k5y8hyl91oE6BopjO7Eroi19hQOE3iPpRFPc/9kgXNnl64fMrE3zLUZyWgLrr2OztzBpfN6LwcVrs+jrbboMvjQ2+NOKo4qTYmvvGI8JOSScvJp2OUVUM965FRIgSISpOiEoQSB9EUtfeJPhLKd+4kCiPHHgI0O+osaT26E3+7lyWf/E5lZXCjkrYXCFUVgp9hkzk5DO7s/zLPcz/+MsD6x46AMaPgLT4yfTu0p3TTtxEwoPzDtRXVMBXX0HXrueQlpbG2rVrmT+/7mfn+9//Pp06dWLFihXUN/Z40UUXkZiYyNKlS1m6tO5n4/LLLycmJoaFCxeycuXKOvXXXHMNAF9++SXravXzxsTEcPnllwMwd+5cNtfqx01MTOSiiy4C4MMPPyQ7u+a/7Y4dO/KDH/wAgNmzZ5OTk1OjPjU1lXPPPReAmTNnkp+fX6O+e/fuTJ48GYDXX3+dwsLCGvUZGRmceuqpALz88suUlpbWqO/bty8nnHACAM8//zxVVVU16gcNGsSkSZMAmDZtGrUNHTqUsWPHUlVVxfPPP1+nfsSIEYwYMYLS0lJefvnlOvWhikTi2AZMEJFEnK6qU4BFwCfABThnVl0NvHWwBVX6/OzaX16jbMvefNasj3I/vOV15tmwN48Na3A/vHXr1+7NZbPPR5JUclxM3fqV+XvY7q+ko5QzqZ76Zfm72eUvo4uUMq6e+iX5OezxF9PNU8yo6Lr1X+ftZK8W0MNTyNH11H+5JJtCjae3p4Ch0WV16j9buJ0SjaVv1F4GR9Wt/2TBViqIYUBUHgPqqZ/z5RZ8RDEkKp+seupnf+58IIdF7yXDU7Peh4c5X2wB4OjoAnp4asZfgZdPcrYCMDp6P11r1Zeqn3m7nIGZcTFFdJGa9YUqfLnL+cBPiimhk6eC6qQqwL7iUpbk7sYjwgRPOYlSdaAOEfZVlLOmeD8eEZL9PmLxAeLWQzl+ij1ePCIo4ME5oIvb3EmJj6FvhySiRUjKiz7QCnL+Cp06pjCia29ixE/05u2IyIE6ETiiR086ZgwgRqvIWerEWb18AU4b1J/uWQMpLyli+RfZKODzOS0jrxfSE1I5YnA39uXu47MFsQfKvV5nmrx9sVx0dTR7SmNZt+q7j3ZUlNOSSo1LYkTvFGJGFbFvpzNWlZTk1AMMHuz8tbOgzMFE5AJAEbkHp6vKC3yDc2puL5yk0cUtu0JVKxpbzpCjRuhfZ8wOap0t6TLHptjmDS2hOXdnYLdFQz0Y1QfP6u4m56nUfC3flVd3WUl1Qgg4MAcuR5qpz8Tn48ABWhWSkp3yvD1CeZng80FVpVBZCXHx0H+Q01W2YG4Mhfudb/qVFc403TN8nHCak8z+en8iBXs9lJdBaYlQVipMON7LDb8oxyPC8cM6UVxU8z1eO9XLHx/y4vcKvdJi6dQJuqRCahdITRUuuwyuvNIZyP/gA+dEhz59nJMljAl0uF1VrfrK8bRuo3XE2C9rlGUN8HHdz51vwo/dl0j+nppfnwYP83LFVOeb7EN3JVG4v+aHc/hoLxde5dTfd3tSnTGQ0ROr+N4lTj773U3JdWKadGIlZ36/kopyuP/OuvUnnlHJyWdWUrhfePjuuqOLZ0yp4JiTqsjbI/z9j0594DHynAvKGTPJy87tHp7+S2Kd+c+/opyjRnnZsiGK/zyZUKf+kmvLGDzMx9qVUbz077r1V91QRtYAH8uXRPPGC3UH7390Uyk9e/tZND+aWa/Urb/xzhLSuilffBLD+2/FOYks4Kyp2+8roWMn5cN3Yvn43Vj3bCk5UH/vY0XExcPbL8Uxd04sBJxVBfDYtCIAXng2ni8+jj1wtpXfD7Fx8MT/nC6CJx9OZMG8mAN1fj90SlGeebkIEO65LZGvP4/B53PrfZCR6ef5mcUIwo8uTmLh/KgD10AADDvax2vvlyLAlFMTWb0iqsZ7n3Ssj5fersAjwrGj49i8qea/vTPO9PPSaz6iPMKEMVEUF0NyMnRIhuRk4Ywz4Je/dKb99a+dQfmUFOdMvdRUGDTIeQB2M0BzWFrjGEeT8VUJ2ZtqDiB37wyjM50D2r6dQnatU1L7ZcQyOtM5YO7ZKuTm1awfNvC7+p2bhNJavTXjR8QyOtM5YG/fUPdbb9ykGEZnJlFcDFvX1a1PPMWp3727/voOGsOoTGWzHzav8dRpQaR4nPq4Itiwsm6fQlpMNKMyoWoXrFtetz49wakv2AhrltWt75kUzcg+kL20/vqsTtEMy4T184UNK+uegjkwLZq+mfCtClvWBrQg3MfQ7jGkp8NCgT3ZdetH9u5MUhJ8GS9UFoF4wBNQPy6rCx4PfN4Fvo2XA4PiHo9zGvH4fqkAzOsL+dudchHnIJuaCqMzuwBw8iRIS3a6aaof3bt7GJ6RAsD118IZpzjzxcQ4f3v2jGJQegcA/nQ/FBQ45XFxzqm+3bpFkdHZ+bfx3rvO9qiui4uDhAQP8fHONl1R6zTy2h54oPF6Sxomklp1i8MuADTGmNAdbovDhsGMMcaExBKHMcaYkFjiMMYYExJLHMYYY0JiicMYY0xILHEYY4wJiSUOY4wxIbHEYYwxJiSt+gJAESkC1kY6DqATsP+gU4V3WaHMF8y0B5umofpQytOAvHqmbU5Nue8OZ3nNuf9CrWup+w5a3/5rKZ+9wara4SBxNExVW+0DWBTpGNw4no70skKZL5hpDzZNQ/WhlLeE/deU+6617L9Q61rqvmuN+6+tfPasq6ppzGwBywplvmCmPdg0DdWHWh5pTR1Xa9h/oda11H0HrW//tYnPXmvvqlqkh3G/FRNZtv9aL9t3rdvh7r/W3uJ4OtIBmMNi+6/1sn3Xuh3W/mvVLQ5jjDHNr7W3OIwxxjQzSxzGGGNCYonDGGNMSNps4hCR74nIMyLykoicHul4TGhEpJ+I/EtEXo10LObgRCRJRP7jfuYuj3Q8JjShft5aZOIQkedEZI+IrKhVPllE1orIBhG5s7FlqOqbqno9cANwcTjjNTU10f7bpKrXhTdS05gQ9+MPgFfdz9x5zR6sqSOU/Rfq561FJg5gGjA5sEBEooAngDOBI4FLReRIETlKRGbVenQLmPW37nym+Uyj6fafiZxpBLkfgQxguzuZrxljNA2bRvD7LyTRTRFdU1PVeSKSVat4HLBBVTcBiMiLwBRV/SNwTu1liIgAfwLeU9Ul4Y3YBGqK/WciL5T9CGTjJI+ltNwvpO1KiPtvVSjLbk07uBfffaMB5x9qr0am/xlwKnCBiNwQzsBMUELafyKSKiJPASNF5FfhDs4EraH9+Dpwvog8Scu+RUl7V+/+C/Xz1iJbHE1BVf8G/C3ScZhDo6r5OONTphVQ1RLg2kjHYQ5NqJ+31tTi2AH0Dnid4ZaZ1sH2X9tg+7F1a5L915oSx0JgoIj0FZFY4BLg7QjHZIJn+69tsP3YujXJ/muRiUNEZgDzgcEiki0i16mqF7gReB9YDbysqisjGaepn+2/tsH2Y+sWzv1nNzk0xhgTkhbZ4jDGGNNyWeIwxhgTEkscxhhjQmKJwxhjTEgscRhjjAmJJQ5jjDEhscRh2gURKW7i5WXVvl31YSxrmohcEMR0n4rImKZYpzGHwxKHMcaYkFjiMO2WiIwQkQUi8q2IvCEind3ysW7ZUhF56GAtC7f18ZmILHEfk9zyE0Vkroi8JSKbRORPInK5iHwtIstFpH/AYk4VkUUisk5EznHnTxCRF0VktYi8ASQErPNJd/qVInJP028dYxpmicO0Z9OBO1R1OLAcuNst/zfwf6o6guB+lGgPcJqqjsL5tcnAuzIfjXPX0SOAK4FBqjoOeBbn1v/VsnB+K+Fs4CkRief/27t71iiiMIrj/xMRJKhJJ5amUpQ0KlbGD2CtiOAL2GglgoXid7C0sVBBLBR7sVtBBEVDkkJFFBQhWMXFF5Jmj8WdhXFF3BkxCe75VTN3hjvPFDMP9164D5wDvtveVcW2t3b/Fdv7gGngkKTpod864i8lccRIkjQBTNruVE23gBlJk8AW20+q9jtDdLcRuC5pAbhHqazW98z2ou0V4C3wsGpfoCSLvru2e7bfAO+AncAMcBvA9jwwX7v/qKQXwCywe+CZEf/Uf1uPI2IVXQA+UUYXY8By7dpK7bhXO+/x8/c3uGncbzeRk7QDuAjst70k6SawqVXkES1kxBEjyXYXWJJ0sGo6AXRsfwa+SDpQtR8borsJYNF2r+pnQ4uQjkgaq9Y9poDXwCPgOICkPZRpKYCtwDegK2kbpX50xKrJiCNGxbikj7Xzq8ApynrCOGV6qF/B7gxl6qkHdIDuH/q+BtyXdBJ4QPmpN/UBeEpJCmdtL1dlWG9IeknZAvs5gO05SbPAK0oZ0MctnhfRWrZVjxggabPtr9XxJWC77fNrHFbEupERR8SvDku6TPk+3gOn1zaciPUlI46IiGgki+MREdFIEkdERDSSxBEREY0kcURERCNJHBER0UgSR0RENPIDVLStSLOMJj8AAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "tags": [], + "needs_background": "light" + } + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "tMFZgiDEG059", + "colab_type": "code", + "outputId": "e4a6fc31-d0ff-47a0-d2ee-0e8e2a11b304", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 187 + } + }, + "source": [ + "# print the standardized coefficients\n", + "print('Lasso Results:')\n", + "print('(Standardized)')\n", + "lasso = linear_model.Lasso(alpha = cv_lasso.best_params_.get('alpha'))\n", + "lasso.fit(X_std, y)\n", + "lasso_df = pd.DataFrame(lasso.coef_, X_.columns, columns=['Coefficient'])\n", + "print('Intercept %.6f' %(lasso.intercept_))\n", + "print(lasso_df)" + ], + "execution_count": 91, + "outputs": [ + { + "output_type": "stream", + "text": [ + "Lasso Results:\n", + "(Standardized)\n", + "Intercept 37.980193\n", + " Coefficient\n", + "X1 transaction date 1.392322\n", + "X2 house age -3.008603\n", + "X3 distance to the nearest MRT station -5.468975\n", + "X4 number of convenience stores 3.311749\n", + "X5 latitude 2.782846\n", + "X6 longitude -0.000000\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "cYEx491nG7pd", + "colab_type": "code", + "outputId": "ca0d1dfa-63ae-49ff-f6d8-4492bc8093d5", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 170 + } + }, + "source": [ + "# back-convert for ease of interpretation\n", + "# beta_real = beta_std / std err\n", + "# intercept_real = intercept_std - sum of (mean * beta_std / stderr)\n", + "print('(Real)')\n", + "lasso_real = lasso_df.copy()\n", + "intercept_real = lasso.intercept_\n", + "count_lasso = lasso_real.shape[0]\n", + "for i in range(lasso_real.shape[0]):\n", + " # decrement the total number of predictors if the coefficient is reduced to zero\n", + " if lasso_real.iloc[i, 0] == 0:\n", + " count_lasso -= 1\n", + " lasso_real.iloc[i, 0] /= np.std(X_.iloc[:, i])\n", + " intercept_real -= (lasso_real.iloc[i, 0] * np.mean(X_.iloc[:, i]))\n", + "print('Intercept %.6f' %(intercept_real))\n", + "print(lasso_real)" + ], + "execution_count": 92, + "outputs": [ + { + "output_type": "stream", + "text": [ + "(Real)\n", + "Intercept -15515.768488\n", + " Coefficient\n", + "X1 transaction date 4.943862\n", + "X2 house age -0.264406\n", + "X3 distance to the nearest MRT station -0.004338\n", + "X4 number of convenience stores 1.125679\n", + "X5 latitude 224.509957\n", + "X6 longitude -0.000000\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "rAdP1sYCHCd_", + "colab_type": "code", + "outputId": "f0d02f31-affd-4f5b-d347-3bf185feab79", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 673 + } + }, + "source": [ + "print('Ridge Results:')\n", + "alphas_ridge = np.logspace(-2, 5, 30) # the range of the parameter\n", + "parameters_ridge = [{'alpha': alphas_ridge}]\n", + "cv_ridge = GridSearchCV(linear_model.Ridge(), parameters_ridge, scoring = 'neg_mean_squared_error')\n", + "cv_ridge.fit(X_std, y)\n", + "scores_ridge = cv_ridge.cv_results_['mean_test_score'] # the scores are negative\n", + "scores_ridge = np.negative(scores_ridge) # reverse the sign of the score\n", + "scores_std_ridge = cv_ridge.cv_results_['std_test_score'] # standard deviation\n", + "\n", + "# plot the cross validation score against log lambda\n", + "fig_ridge = plt.figure()\n", + "plt.semilogx(alphas_ridge, scores_ridge)\n", + "std_error_ridge = scores_std_ridge / np.sqrt(X_.shape[0])\n", + "# shade the region within 1 std err\n", + "plt.semilogx(alphas_ridge, scores_ridge + std_error_ridge, 'b--')\n", + "plt.semilogx(alphas_ridge, scores_ridge - std_error_ridge, 'b--')\n", + "plt.fill_between(alphas_ridge, scores_ridge + std_error_ridge, scores_ridge - std_error_ridge, alpha = 0.25)\n", + "plt.ylabel('CV score +/- std error')\n", + "plt.xlabel('Log lambda')\n", + "plt.axhline(np.min(scores_ridge), linestyle='--', color='.5')\n", + "plt.xlim([alphas_ridge[0], alphas_ridge[-1]])\n", + "plt.title('CV score as a function of lambda')\n", + "plt.show()\n", + "# print the optimal values\n", + "print('best score:', np.negative(cv_ridge.best_score_))\n", + "print('lambda:', cv_ridge.best_params_.get('alpha'))\n", + "\n", + "# print the standardized coefficients\n", + "print('(Standardized)')\n", + "ridge = linear_model.Ridge(alpha = cv_ridge.best_params_.get('alpha'))\n", + "ridge.fit(X_std, y)\n", + "ridge_df = pd.DataFrame(ridge.coef_, X_.columns, columns=['Coefficient'])\n", + "print('Intercept %.6f' %(ridge.intercept_))\n", + "print(ridge_df)\n", + "\n", + "# back-convert for ease of interpretation\n", + "# beta_real = beta_std / std err\n", + "# intercept_real = intercept_std - sum of (mean * beta_std / stderr)\n", + "print('\\n(Real)')\n", + "ridge_real = ridge_df.copy()\n", + "ridge_intercept_real = ridge.intercept_\n", + "count_ridge = ridge_real.shape[0]\n", + "for i in range(count_ridge):\n", + " ridge_real.iloc[i, 0] /= np.std(X_.iloc[:, i])\n", + " ridge_intercept_real -= (ridge_real.iloc[i, 0] * np.mean(X_.iloc[:, i]))\n", + "print('Intercept %.6f' %(ridge_intercept_real))\n", + "print(ridge_real)" + ], + "execution_count": 93, + "outputs": [ + { + "output_type": "stream", + "text": [ + "Ridge Results:\n" + ], + "name": "stdout" + }, + { + "output_type": "display_data", + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEaCAYAAAAG87ApAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdd3wUdfrA8c+z2XRIg1BCgNC7AtIEFSwI9vI7PRVPUU/Pdqeed3aP89S707N7Nu5U9MSCHRv2LooIKL2ElgChppG62X1+f8yg6exCkk3C8369Fna+z8zss7ObfXa+39kZUVWMMcaYYHnCnYAxxpiWxQqHMcaYkFjhMMYYExIrHMYYY0JihcMYY0xIrHAYY4wJiRUOY4IgIqeJSJaI7BaRYU34uFNE5IOmerxKjztORFa7z/fUWuLrReSYRnjcCSKSHcL8U0Xkq4bOw9TPCkcrJCLniMh8949+i4i8JyKHichZ7h+8VJvfKyLbROTEcOXcAtwDXKmqbVR1YWM8gIhkiIiKiHdPm6rOVNVjG+Px9uJvwL/d5/tGGB7fNGNWOFoZEfkj8ADwd6Aj0A14FDgFeANIAsZXW2wyoMCcpsvUUflDspnrDiwNdxJN6EB7viYEVjhaERFJxPmmeIWqvqaqRarqU9W3VPXPqloKzALOq7boecDzqlpRyzp7i8jnIpIvIjtE5KVKsUEi8qGI7BKRrSJyk9seLSIPiMhm9/aAiES7sQkiki0i14tIDvC0iHhE5AYRyRSRnSIyS0RS6niOySLytohsF5Fc9356pfhUEVkrIoUisk5EptSxnlEiMldE8ty9sn+LSFQt80WLyG4gAvhRRDLddhWR3pXmmyEid1R7jte6e3JbROSCSvPGisi9IrLB3a5fiUgs8IU7S567t3ho9a4YERkrIt+7y30vImMrxT4TkdtF5Gv3+X8gIu1re/7u/BeLyBr39ZstImlueybQE3jLzSO6rnUEsy3dbXW52/VV6ObYS0S+EZEC9/WOqrbOm9z32/rKr6GItHNzLRCReUCvass9KE6XYoGI/CAih9eXu9lHqmq3VnLD2XOoALz1zDMOKABi3elEoAQYWsf8LwA343zJiAEOc9vbAluAa932tsBoN/Y34FugA5AKfAPc7sYmuDneBUQDscBV7vzpbtsTwAt15NMO+D8gzn3Ml4E33Fi8+9z6udOdgUF1rOcQYAzgBTKA5cDV9Ww3BXrXMz0DuKPac/wbEAkcDxQDyW78EeAzoAtOQRrrPu8Md73eSuudCnzl3k8BcoHfuHmf7U63c+OfAZlAX3e7fgb8s47ncxSwAxjuPvbDwBeV4uuBY+rZHj/H97Yt3ef0JpAADALKgI9xilMisAw4v9q2u8/NazxQVOk1fRHny088MBjYtGf7uPFz3feIF+e9mQPEhPtvs7Xdwp6A3RrwxYQpQE4Q860GznHvXwz8WM+8zwLTgfRq7WcDC+tYJhM4vtL0JGC9e38CUF75j9n9oDm60nRnwEc9BbDSvEOBXPd+PJCHU1hiQ9x2VwOv1xMPtXCUULUAbHM/XD1u7OBaHiOD+gvHb4B51ZaZC0x1738G3FIpdjkwp47n8yRwd6XpNu42z3Cn1xNk4djbtnSf07hK0z8A11eavhd4oNK2qwDiK8VnAbfiFFkf0L9S7O9UKhy15JJb27a22/7drKuqddkJtA9i3OBZfumu+o07XZfrAAHmichSEbnQbe+KUyBqkwZsqDS9wW3bY7s63WZ7dAded7s68nAKiR9njKYKEYkTkSfcbp4CnO6dJBGJUNUi4NfApcAWEXlHRPrXlqCI9HW7uXLc9fwdqLNbZx/s1Kpdf8U4H87tcfbQ6tp29am+XXGnu1SazqnlMfe6LlXdjfP+6VLH/HUKclturXS/pJbpynnmuq/lHnveP6k4exJZ1WKVc/mTiCx3u/LycPZoGvJ1NdgYR2szF6cboMbhk9X8DzhaRA7F+RY8s64ZVTVHVS9W1TTgd8Cjbt9+Fk5XQ2024xSDPbq5bT+vttr8WcBxqppU6RajqptqWfe1QD+cbrEE4Ai3Xdx831fViTh7LSuA/9SR42NuvI+7npv2rCNIxTjdZXt0CnK5HUAp1frmXXs7VXX17QrOtq1tO+1NlXWJSDxOF8++rGt/t2V1yW4+e+x5/2zH2RvpWi0GgDuecR1wJk63YBKQv5+5mFpY4WhFVDUf+AvwiIic6n47jxSR40Tk7krzrQe+whm/+FBVc2pfI4jIGZUGn3NxPtwCwNtAZxG52h1Abisio935XgBuEZFUd3D2L8Bz9aT+OHCniHR3HzNVRE6pY962ON9Q88QZQJ9WKdeOInKK+6FTBux2c61rPQXAbnev5LJ68qvNIuAcEYkQkcnUPFKtVqoaAJ4C7hORNHf5Q90B6O1uvnUV5HeBvuIcbu0VkV8DA3Fei1C9AFwgIkPdx/478J373gjV/m7L2twmIlFuMTgReFlV/cBrwF/d9/ZA4PxqeVTgbEeviPwFZ1zFNDArHK2Mqt4L/BG4BecPKAu4EudQ3MqewfnGWV83FcBI4DtxjiyaDVylqmtVtRCYCJyE0z2yGjjSXeYOYD7wE7AYWOC21eVBd90fiEghzkD56DrmfQBn4HeHO1/lQ4g9OM99M7AL58O8rg+xPwHnAIU4eyUv1TFfXa7Cee55OGNLofzW4U842+V7N8+7AI+qFgN3Al+73XZjKi+kqjtxPkSvxelWug44UVV3hJg7qvoRzrjBqzgHOfQCzgp1PZWez/5sy+pycL6kbMbZG75UVVe4sStxurVycMaVnq603Ps474dVOF1YpVTt1jINRNwBJGOMMSYotsdhjDEmJFY4jDHGhMQKhzHGmJBY4TDGGBMSKxzGGGNC0lLOTFqr9u3ba0ZGRrjTMMaYFuWHH37Yoaqp+7p8iy4cGRkZzJ8/P9xpGGNMiyIi1U9dExLrqjLGGBMSKxzGGGNCYoXDGGNMSKxwGGOMCYkVDmOMMSGxwmGMMSYkLfpwXGOMMcErKoJVq/Z/PbbHYYwxrVBuLsyb98v0lHOVtm2VUaP2/1IatsdhjDGtwIIF8O67sHAh/LBA2bBeiIpS5q/ejY8Kug+K5MLfe+g7sILrL92/x7LCYYwxLdCaNfDKK/C7SxWN9DHrDbjr9ii6ZfjpM7CC48+ooN8gP3nFPryRMPG0YjbuKmb9jqL9fmwrHMYY00KsXAkvvwwvv6L89KMAEJ1ayJgJPo46TTjmdIhvq+wuq2DDziLW7Sji8a+cYrE5v4SGuuCrFQ5jjGnGSkogNhaWLQ8waKAzLD1keAVX3VzO+EnlpHSoYFFWAet2FLF+ZxHrdxSxrbDs5+VT4qPIaBfP6J4p9GgXT0b7eE6+a/9yssJhjDHNzKpVMHMmzHpZGXxQgNsfKCI/ysdN/4xm9GE+2rbz8WNWHi8t38nC9/MoqwgA0DEhmh7t4zmqfwcy3CKRGBsJwMolESz92ss3GyL2O79GKxwi8hRwIrBNVQe7bUOBx4EYoAK4XFXniYgADwLHA8XAVFVd0Fi5GWNMc7R5M0ybBk895fQpDR1ZQb/hZeQW+Sj1+Wk3bBPPLdnFoo15lPsDJMR4Oax3e0ZmpNCnYxviorwUFgirl0Ww7AMvX+/08IcbS4iP8jL9nnjmfhVBSkrzPqpqBvBv4NlKbXcDt6nqeyJyvDs9ATgO6OPeRgOPuf8bY8wB4657Asx4RjjjvFLO/V0JsYk+FmzM494PdvJjdh4+v5IUG8n4fqmM6ZFC/04JeDzOWMesZ2J4+ZkYNm38ZY+ie4by9COxeL3wn8ehbVvo2lXw7OcPMRqtcKjqFyKSUb0ZSHDvJwKb3funAM+qqgLfikiSiHRW1S2NlZ8xxoSbzwdPPgm9+/npPrCY46f4GHcCbNYdPLVgBz9l51ERUFLiozi6f0dG90ihT4e2rFnh5dMXorjjg2hmvl5K/56RDOzm5dBRHoZfCsOHw7Bh0KGD/PxYgwY1XN5NPcZxNfC+iNyD8+PDsW57FyCr0nzZbpsVDmNMq6MKs2fDddcrq1YKp0/x8afbyskuKeD5RRvI3F5Eu/gojh3YkdE929G7Qxt2bffw0oxYbp0TxaaNEUREKEceCQkST0o8XPxb59YUmrpwXAZco6qvisiZwJPAMaGsQEQuAS4B6NatW8NnaIwxjej77+Haa5UvvxQyegW46/EiMobn86/3s1iwMZeU+CguG9+Lcb3as/ynSCpyIKaLn3bx0bw8I4Yjj4Tb/gKnnCK0bx+e59DUheN84Cr3/svAf937m4CuleZLd9tqUNXpwHSAESNGNNBRycYY0/hUlQ8/9bNshYfrbi/isBMKef3HbB57bRsx3gjOHtmVcd3SePnpOO5/M5qtmyOYfHyAC08TRIQdO5xxinBr6sKxGRgPfAYcBax222cDV4rIiziD4vk2vmGMaS0eeQQioisYO3k3h53kZ+jRAT7O3MSfXt1CRUCZNKgTpw3rwhdvteHc38ZRkC+ccAKc+U846SQP4g5VNIeiAY17OO4LOEdMtReRbGAacDHwoIh4gVLcLifgXZxDcdfgHI57QWPlZYwxTSUQgBtvhLvvhpPOqGDw+Ao+W7ONVxdkk1/iY0zPFH49ohupbWKIiRISoqIZNVK4+25h6NBwZ1830Yb6DXoYjBgxQufPnx/uNIwxpobycrjwQmXmTOG0c0oYf342L/2wkc35pfTr2JZzx3SjcF0yj9wVz7nnB/jzVV484tnvQ2WDISI/qOqIfV3efjlujDENrKwMTjhB+fhj4eJriigbtJT7P9lOWmIM107sS0p5ex6/JZ6vPomiS7rSp5sX7/7/oLvJWOEwxpgG5okI0LW3j+uPKWNF28UsWp3HqUO78KtD0pk5PZbp98XRpg384x9w1VVCbGy4Mw6NFQ5jjGkgy5ZBSZmfyNRCzryihLvfX8n67CLOH9mTI/p0ICE2gomHRRNZJtx6K6SmhjvjfWOFwxhjGsCXX8LJJytp3ZS/Pbmbu95fQV6xj0tHDWDm7V3ZNV546L4IDkoXTj8l3NnuH7t0rDHG7KdXXoGJE5XElAAX/iWHaW8tpcTn59LhQ3j02u6sWuZl/DgvIrL3lbUAtsdhjDH74cEH4ZprlMHDKjjnlo08MX8VyfGRnNF9CH+/LJVAhfDRR8Jhh4U704ZjhcMYY/aR3w+vvO7n8GP8jP9dJk/MW0uPdvH84YgBXHRCKvFxwvtzhAEDwp1pw7LCYYwxIVIFn0/ZUljMtAdLmL00m2e/38TQrklcdXQfOibE8PxzMGSIkJYW7mwbno1xGGNMiB5+GEaNCbB0XQnPzs9k9k+bmNC3A6lrhvLtewn079SWSZM8rbJogBUOY4wJycKF8Oc/K3FJFTz2zQq+WL2D0w5OJ++Dwfz3gXhWL475+eJKrZUVDmOMCVJhIZz5ayUxOYB3wvcs3ZLP1JG9mP/kAN58KZYbb3QuzNTa2RiHMcYE6corYW0mHH/9ahYX53Pl+D7MuKkHy3/y8sgjcPnl4c6waVjhMMaYIOTlwTffBjj23FwWBzI5fnAnJgxoT9HpHg6eJpx6argzbDpWOIwxJgje2Ar++cx2bn3nR7pEJHF4ai8GpbVlxK0t6OyEDcTGOIwxph5lZTDtr8rCzAKmz12NX5XSjw7hinMSqSg78IoG2B6HMcbU67rr4KGHhMXluawMFDJ410je+TqGRx9tPlfka2q2x2GMMXWYPRseegiO/r98FgZWMCgyg/efTuW005RLLw13duFjexzGGFOL7Gy44AKl94AKtg+cR6rEs+jp/nROgyefFFrJ+Qr3iRUOY4ypxWWXKSWl0P1XP7G+wse1x/fjmx3wm3OE5ORwZxdeVjiMMaYWf55WQvzB2/i2PIfzx2QwfkA7znvkwBwMr87GOIwxppJNm2Dn7nKyKrbzfWAZfaPSmHl9f3ZtjAt3as2G7XEYY4xr504YPVo59OgKNvZbTYI3msxZgynM9dCx4wE8qFGNFQ5jjME5VfqFFypbt8HubplsKyyjx8rDWbQyknffpdWe6XZfWFeVMcbgnJxw9mxh8gXbWO7byEG+QXz6egJ/+hMcd1y4s2terHAYYw54JSUwbZrS/+AyVrZfwMDOCRQuSWPESOXOO8OdXfNjXVXGmAPemjUQUCVqzDICkR5+f1Rvxl3soaxIiIoKd3bNj+1xGGMOeH36+znznsXkJ21mWNlQukQnkxDnJTU13Jk1T7bHYYw5oM2fD+Vt83h/+WYyyjJ47sEOtCsV7r8/3Jk1X422xyEiT4nINhFZUq399yKyQkSWisjdldpvFJE1IrJSRCY1Vl7GGLNHXh5MnKhc9DulON/LT8/1p28/uOOOcGfWvDXmHscM4N/As3saRORI4BTgYFUtE5EObvtA4CxgEJAGfCQifVXV34j5GWMOcPfeC3l5Qtvey4laeDBFBR5mfSzEx4c7s+at0fY4VPULYFe15suAf6pqmTvPNrf9FOBFVS1T1XXAGmBUY+VmjDHbt8MDDyg9Ru5CIwKsm5vK5ZcLBx0U7syav6YeHO8LHC4i34nI5yIy0m3vAmRVmi/bbatBRC4RkfkiMn/79u2NnK4xprW6+24oLgbf0MUcNjCBS36n3HBDuLNqGeotHCISISIrGvDxvEAKMAb4MzBLJLSTE6vqdFUdoaojUu2QB2PMPggE4NvvAvQYs5OY1CKuOL4rjz3ioWPHcGfWMtRbONwxhpUi0q2BHi8beE0d84AA0B7YBHStNF+622aMMQ3O44FbHt1GxZj5JK0YSvnmpHCn1KIE01WVDCwVkY9FZPae2z4+3hvAkQAi0heIAnYAs4GzRCRaRHoAfYB5+/gYxhhTp61bYW2Wj2fnrkfz2jL/tTS++tx+mRCKYLbWrfuyYhF5AZgAtBeRbGAa8BTwlHuIbjlwvqoqTmGaBSwDKoAr7IgqY0xjuPlmePk1D0kX5CI/jKNdO+Xqq+3Mt6HYa+FQ1c9FpCOwZyB7XqWjoepb7uw6QufWMf+dgJ0VxhjTaFavhhkzlB5HbKV8RzIbfkzg7rshISHcmbUse+2qEpEzcbqNzgDOBL4TkV81dmLGGNPQbrsNvJFKyYBlBOYPplMn5Yorwp1VyxNMV9XNwMg9exkikgp8BLzSmIkZY0xDWrIEnn9e6XXMZqJS/Ew82cvBfYU4u7BfyIIpHJ5qXVM7sZMjGmNamDlzlNh4pXTAMs4Ylsa0kyOJtjHxfRLMZpsjIu8DL7jTvwbebbyUjDGm4Z13SRnvlS1k/epk2uX0ItITEe6UWqy9/QBQgIeAJ4CD3Nt0Vb2+CXIzxpgGsWGD8t6SLawp2EXRF4N5/P5Y/Hbc5j6rd49DVVVE3lXVIcBrTZSTMcY0mK+/hiOOgCEX7CaiMJ2t62K59zmIjAx3Zi1XMGMVCyqdU8oYY1qUW29V2iT52dlmE7u/6c/AgcpZZ4U7q5YtmDGO0cAUEdkAFAGCszNi55A0xjRrn3wCn34q9Dp5LbqpK9nZ0Tz2CkTY8MZ+qbdwuGMclwAbmiYdY4xpGKpw8y1KUmoFvj6ZTEwYwnCU00+3X4nvr2DGOB5xxziMMabFyMyEBT9Al8lr6dw+mmmXpNA1xYpGQ7AxDmNMq9S7N9zy3Fp8GevplDWEpMjYcKfUagRTOEYD34pIpoj8JCKLReSnxk7MGGP2VSAAucXlvL16HdHre/Lao6nMn297Gw0lmMHxSY2ehTHGNKB774X/PBegeHyAgi97cNRRypFHWuFoKHvd41DVDTgXWTrKvV8czHLGGBMOgQA88YSyq6iMwPIMCnO93H67FY2GFMzZcacB1wM3uk2RwHONmZQxxuyrjz6CzEyBvuvJ/bYnxx+vjB0b7qxal2D2HE4DTsb5DQequhlo25hJGWPMvnrsMSU2wUdMei5Dhyk33GB7Gw0tmDGOcvewXAUQkfhGzskYY/ZJdjbMng0ph2Yxdng0zz8Wgdd+7NfggtnjmCUiTwBJInIxzrU4/tO4aRljTOiSkuCMK3cQkZHDhPRueCNsOLYxBDM4fg/ORZteBfoBf1HVhxs7MWOMCZU32k9uj+UElmdw07lplJaGO6PWKajLmKjqh8CHjZyLMcbss08+gbe/LGLZjlLylnTigqlCTEy4s2qd7PpXxphW4R//UL5dEE30sHR85R4uuyzcGbVe1gFojGnxVq+Gjz4SogdvwLe0B6PHKEOHhjur1ssKhzGmxXv8cfBEKN60nezeFsPll9khuI2pzq4qEVkMaF1xux6HMaY5KCmBp59WUgZup9+wUr68w0/XLtYL35jq27onuv9f4f7/P/f/KY2XjjHGhGbLFujQ1UfugLWcdHAafXpZ0WhsdXZVqeoG99xUE1X1OlVd7N5uAI5tuhSNMaZu3TOUUVctJLA1hbfu6kV5ebgzav2CGeMQERlXaWJskMsZY0yj2rwZ5q/czTerd1L6U3eiIiKIigp3Vq1fMAXgIuBREVkvIuuBR4EL97aQiDwlIttEZEktsWtFREWkvTstIvKQiKxxr/kxPMTnYYw5AP31r3Dk6Fjyl3dk965oLrNB8SYRTOHYpaoHAwcDB6vqUCA3iOVmAJOrN4pIV5yuro2Vmo8D+ri3S4DHgli/MeYAlp8Pzz+vtOm7DVb1IC1NOemkcGd1YAimcLwKoKr5qprvtr2yt4VU9QtgVy2h+4HrqHrE1inAs+r4Fue8WJ2DyM0Yc4B67jkoKhK02yZ2rkjmt78VIiPDndWBob7DcfsDg4BEETm9UigB2Kcf8ovIKcAmVf1RpMouZRcgq9J0ttu2ZV8exxjTuqk6p09P7lZEat8CrvhbBb+dalWjqdR33Fo/nENyk4DKO4CFwMWhPpCIxAE3sZ9HZInIJTjdWXTr1m1/VmWMaaEWL4alS4X2x61l0rAUpp3pRWx4o8nUWThU9U3gTRE5VFXnNsBj9QJ6AHv2NtKBBSIyCtiEc3naPdLdttrymg5MBxgxYkSdP1A0xrReBx0Ev31oBW9+XkripoH4/YLXfr7RZIK6AqCIJIhIpIh8LCLbReTcUB/I/Q1IB1XNUNUMnO6o4aqaA8wGznOPrhoD5KuqdVMZY2q1Y3cZ83M34l/Uj+cejcdjPxBoUsFs7mNVtQCn22o90Bv4894WEpEXgLlAPxHJFpGL6pn9XWAtsAbnIlGXB5GXMeYAdM89MOnECnLWxrIrM5FLLxUrHE0smJ27PSNOJwAvq2q+BNGZqKpn7yWeUem+8supTYwxplZ+P/z7EaXQW4GnJIPoaGXqVBvcaGrB1Om3RGQFcAjwsYikAnZdLWNMk/vgA9iwXpC+6yhYnMYZZwjt24c7qwNPMJeOvQEYC4xQVR9QjPO7C2OMaVLTpyuxCT5iOhWQkYFdrClMgr107K5K94uAokbLyBhjarFtG7z9NiSOzOaw0VH87wnBGxHurA5MNqRkjGkRIiLg2Cm5eDO2cEK/dCK9HvvtRphY4TDGtAiJSQH8Q5fgX9GD687qQkVFuDM6cIVUOETkr42UhzHG1Gn5crjvv0UsWVNG7pKOnHaa/eAvnELd4zi5UbIwxph6PPww3Pz7eIoWd8Xv89igeJiFWjisR9EY06RKSuCFF5Q2/bfiX9WNMWOUIUPCndWBrc7CISJni0i7as2HNHI+xhhTxRtvQF6e4EnbTmFOHBdeaN9fw62+XsJuwMsiEgl8DLwHzGuSrIwxxvX000p8u3I6jdrCG7f1Z+hAuzZsuNW5x6Gqd6nqUcDxwI84l4tdICLPi8h5ItKxqZI0xhyYSkpgdaYSNWAjR/ZP5fDRUbRtG+6sTDC/HC9U1ddV9XeqOgy4A0gFnm307IwxB7TYWPjdw6uISN7NihcGkp+/92VM4wv6gDYRSVbVXFVdBiwD7m28tIwxBzpV2F0c4P1lOVQsP4g1FdG2t9FMhHJU1ceNloUxxlTzxReQ3gWWzo0nPzOZC6ba6dObi1B+QmOHMhhjmsxTT0FJmeLJSQJg6tTw5mN+UW/hEJHz9twFkitNo6o2xmGMaRSFhfDKK0r8gM34VnXlqKOge/dwZ2X22NseR49K96OBDJwiYtf6NsY0mlmzoLhYaNsvm8MHJXLRr2LCnZKppN7Coaq37bkvIqeo6t8aPyVjzIHuqaeUNh1L6NS/kJnXx9LW6kazEspQk41xGGOaxJXXFxMzdhm9y3sTFxm59wVMkwqlcPym0bIwxphKVsl6AsVRvPr3nixYEO5sTHVBH1WlqksaMxFjjKmogOuuD/BJoAD/yoEMHKiMGGGdHc2NHRVtjGk2PvgA7r/Pw8Ylbclfn8gFF4hd5a8ZssJhjGk2nnpKiWnrg8I4IiKUc88Nd0amNkEXDhGJa8xEjDEHth07YPZsiB24Cc+2DpxwAnTqFO6sTG32WjhEZKyILANWuNMHi8ijjZ6ZMeaA8vzz4PMJMYM28t9XCvjvf62PqrkKZo/jfmASsBNAVX8EjmjMpIwxB56iogDt+uaS3rOcSYM7kJoa7oxMXYLqqlLVrGpN/kbIxRhzAJt8Tj4xx85n1YNH8MmHoZxGzzS1YF6dLBEZC6h7NcCrgOWNm5Yx5kCycSO89H0WxcvSyN8eZeelauaC2eO4FLgC6AJsAoa60/USkadEZJuILKnU9i8RWSEiP4nI6yKSVCl2o4isEZGVIjIp9KdijGmJyspg2DDlqXsTqViRwYiRyuDB4c7K1KfewiEiEcCDqjpFVTuqagdVPVdVdwax7hnA5GptHwKDVfUgYBVwo/s4A4GzgEHuMo+6j22MaeVmz4Zdu4SKNgUUbo7nwgtsULy5q7dwqKof6C4iIV8dXlW/AHZVa/tAVSvcyW+BdPf+KcCLqlqmquuANcCoUB/TGNPyPP20EpdcBrkJREcrZ50V7ozM3gQzxrEW+FpEZgNFexpV9b79fOwLgZfc+11wCske2W5bDSJyCXAJQLdu3fYzBWNMOGVlwfvvQ+KYLA6f4GfiVCE5OdxZmb0JpnBkujcP0CBX/BWRm4EKYGaoy6rqdGA6wIgRI+y6IMa0YE895VzcJ+6gjfz+3EEcMyDcGZlg7LVw7Lkmh4i0cad3788DishU4ETgaMMYspsAACAASURBVFXd88G/CehaabZ0t80Y04pdfa2fN3MWU7gtjfa+duFOxwQpmF+ODxaRhcBSYKmI/CAig/blwURkMnAdcLKqFlcKzQbOEpFoEekB9AHm7ctjGGNajvlZO9gmO8l8vR+vvWK/3WgpgnmlpgN/VNVPAURkAvAfYGx9C4nIC8AEoL2IZAPTcI6iigY+FOeUl9+q6qWqulREZgHLcLqwrnAH5o0xrZAqnH66siu1iNJV6agKU6eGOysTrGAKR/yeogGgqp+JSPzeFlLVs2tpfrKe+e8E7gwiH2NMCzd3LrzxhtB+UjG+Fb0ZP17p2dMOw20pgvkB4FoRuVVEMtzbLThHWhljzD55+GGIjvMjbYop2BrDhRda0WhJgikcFwKpwGvAq0B7t80YY0K2eTO88oqSNDSbVEmmazflzDPDnZUJRTBHVeUCf2iCXIwxB4DHHwe/HyIGr+Wac7tz/hghMjLcWZlQBHNU1YfVzimVLCLvN25axpjWasxYP70nbSQlIYJTh6VZ0WiBgumqaq+qeXsm3D2QDo2XkjGmNYvvsZPSgcvIfHws/7o9OtzpmH0QzFFVARHppqobAUSkO86PPY0xJiQPPax8W7GFshVdKNnt5fjjw52R2RfBFI6bga9E5HNAgMNxzxVljDHBmjcPrvqD0H5iJP4V3Rk8RDniCDuaqiUKZnB8jogMB8a4TVer6o7GTcsY09o8/DBExfrxJBWyIyueu24FsbrRIgUzOD4OKFHVt4Ek4Ca3u8oYY4KydSvMmqUkH7yZiHXdSU5WpkwJd1ZmXwUzOP4YUCwiBwN/xDlT7rONmpUxplX5z3+gvFyIGJzJdX8r4r33hLi4cGdl9lUwhaPCPYvtKcAjqvoIDXR6dWPMgSFna4COg3aR2rWcs8d1YfTocGdk9kcwhaNQRG4EzgXeEREPYEdeG2OCdvZVu4ia9B07XxzL91/YIbgtXTCF49dAGXCRqubgXCvjX42alTGm1cjMVJ7/bgOlq9LIXt6G6GgbEW/pgjmqKge4r9L0RmyMwxgThIULYfhwofNpEfgX96Jff+WYY6xwtHTB7HEYY8w+efhhiIwO4I8sZdeGNvz+SrFDcFsBKxzGmEaxYwc8/7ySMnQzsroHCQnKeeeFOyvTEOrsqhKRPwMvqGp2E+ZjjGkl/vtfKCsTPIMyOa5bHw5JFdra8ZitQn1jHGnAXBFZD7wAvKyq25skK2NMi6YKM55RUvvl075rKXdclUznpL0vZ1qGOruqVPUaoBtwCzAE+ElE5ojI+SJi3xuMMXUSgYefzyVy3CKSVg/BUxYT7pRMA6p3jEMdn6vqZTiH4d4PXA1sbYrkjDEtUyCgvLViPb4diXz5fBo//GAj4q1JMGfHRUSGAGfh/KZjB3BjYyZljGm5Fi+Gc88LsHt0MYGlB9GrlzJ5shWO1qS+wfE+wNk4xcIPvAgcq6prmyg3Y0wLdP/9sGyZkNI/gp2ZCdx8H3js+M1Wpb49jjk4g+K/VtUlTZSPMaYFW7gQZsxQOo/bhH91BnFxygUX2N5Ga1Pf94DJwJzqRUNExolIr8ZNyxjT0qjCVVcpbRIDeIYvo1NcPFOnCkl2NFWrU1/huB/Ir6W9AHigcdIxxrRUb78NX34p9DxuHYlJ8PabXh5+ONxZmcZQX+HoqKqLqze6bRmNlpExpkU6aqKfC27eyq60VRyVlkFaYqyNbbRS9b2s9e1gxjZ0IsaYlquiArJyi1gRv4SYLen8+7K+fPedjW20VvUVjvkicnH1RhH5LfDD3lYsIk+JyDYRWVKpLUVEPhSR1e7/yW67iMhDIrJGRH5yr3FujGkBNm6EXr2U26fvZFNeKaXf96Nbdxg1KtyZmcZSX+G4GrhARD4TkXvd2+fARcBVQax7Bs4Ae2U3AB+rah/gY3ca4Digj3u7BOdytcaYFuD665WcrTBv5waSs/uStTqaf/xdiIgId2amsdR5OK6qbgXGisiRwGC3+R1V/SSYFavqFyKSUa35FGCCe/8Z4DPgerf9WfcStd+KSJKIdFbVLUE+D2NMGHz1Fbz4ojD05E3kestYP6cn48YpZ51l3VStWTAXcvoU+LSBHq9jpWKQA3R073cBsirNl+22WeEwppkKBJzDb1M6+NnVezGDvT35rNTDgw/aNTdau7Ad8+DuXWioy4nIJSIyX0Tmb99uJ+s1Jlw++AAWLBC6TlpD2zbCrb9rR1aWcMgh4c7MNLamLhxbRaQzgPv/Nrd9E9C10nzpblsNqjpdVUeo6ojU1NRGTdYYU7cJR/v5w31Z5HbOZGxCXwanJZKSEu6sTFNo6sIxGzjfvX8+8Gal9vPco6vGAPk2vmFM85WfD6u3FfJN0UoSdnbhP3/qwdtvBHXOVNMKNNorLSIv4AyEtxeRbGAa8E9glohcBGwAznRnfxc4HlgDFAMXNFZexpj9s3o1HHKIctTv8thGORUfD6RXL+W002xg40DRaIVDVc+uI3R0LfMqcEVj5WKMaTjXXqv4A8piXyYpOQNYtD6KN96A6OhwZ2aaip0QwBgTtPffh7feEvpNzkYlwJr3unP00crJJ4c7M9OUrHAYY4Li88E11ygd033szFjKiOTuJLQRHnjADr890FjhMMYE5euvYdUq6HDMCpITIrjh/FTWrxcGD977sqZ1scJhjAnKqLEVXDt9LXmpGxlcNoje7ROIjAx3ViYcrHAYY/Zq7VpYvqWA97MzSd7enZl/78JrL9vJqA5UduC1MaZen3wCxx6rHHlxIbnxfko/7MegQcq559rAxoHKCocxpk7ffw+nnKKk96hgTexKUtYNYtGmSGbOAK99ehywrKvKGFOr5cvhuOOUhOQAgy9ahMfvZeWcdE45RTnmmHBnZ8LJCocxpoaiIjh2koJHOWvaOpbkbWN8t3T694d77rEuqgOd7WwaY2qIiglw6bUlFMTuYNaalRyUnsi1p3dk4B899psNY4XDGPOLvDxYvERJyiikw7CdPD1nBb3at6HbpoNpFxlnRcMA1lVljHEVF8NJJynHHQdfLingng9W0jkplvbLR/DIP9vy/rt2+K1xWOEwxuDzwZlnKl9/DRfftJNH5y4jMTaSPlsO4aUn47n0UjjvvHBnaZoLKxzGHOACAZg6VXnnHeGym/P4zLcAb4QwsuwQnn4ggbPOUv79b6ybyvzMCocxB7iZM+H554WpVxXwQ+wCyisCXDNhIK/+N5HJk5VnnhEirJfKVGKD48Yc4MYfX8xtD5fzUdFCcgvLuen4AYzIaMvXXwqdOwtRUeHO0DQ3tsdhzAFqxgz4/qcy1u7Yzde+n9iUX8KpaUP46oVO9O3Qll69hLi4cGdpmiPb4zDmABMIwKOPwu9/D6eeE0DHrmbV1kLO6DmIx69Lo317ofA2ISUl3Jma5soKhzEHkAUL4MorlblzhTFHlBM17ie+25jH//Xpy9M3dSMuTvjoQysapn5WOIw5QHzwAUyerCS3U2765262dlzJRyt2cGLvHsy6vSd+n/Dpx0KPHuHO1DR3NsZhTCvm98OaNVBS7qdjvwJ+e3UJM97dyfbOK/loxVZOPKgzvTzpFBV4ePddu5qfCY7tcRjTSn31Ffz+98qWHOWFD/OIjlH6Td7CtDkb2VVUzqRBnTh3dDcGpMVy4f8JiYnhzti0FLbHYUwrs2UL/OY3cPjhsGVbgN/fVMTW4mJuf2cZD3+yhoQYL7ceN4gVzw5h8RdJJMZGWtEwIbE9DmNakdWrYfhwpawczr+8hP+7oJB3lmfz5Os5xEV5ueiwHvSP68S9t7bhuy+jOPfMcGdsWiIrHMa0YKrOBZeWLoVTTw8QkVTCGVOFSaeWklm2lZvf3khhWQVH9+/IKYPTeeOZBO6cHovX6xySe/HF4X4GpiWywmFMC1NeDl9+CW+9BbPfUtatFWJilA+G5OGNVI48ZzfTv1lH5vYi+nVsyw1jM+jRPp4f50Xy1MNxnHWW8q9/Cenp4X4mpqWywmFMC7BzJ7RpA9HRcPudfu74WwRR0cohh/r41dRyxh3po7iinBfnbuTTldtJio3k8gm96CodWfp5JGMvFEaeGcthQ2DwYDtbodk/oqrhzmGfjRgxQufPnx/uNIxpcAUFsGoVfPIJzJ6tzJ0LTzxbxvDDS8hcA2tXRTBynA+J9PNTdj4LNuby/fpdlPkCHDekE8f2Seel6W2ZNSOGpGRYt1Zo0ybcz8o0FyLyg6qO2Nflw7LHISLXAL8FFFgMXAB0Bl4E2gE/AL9R1fJw5GdMUyguhsxMp0A4g9owcaKyak2A/n1/OR1t34F+zr+8nMTOZZT5AkSnlFGSlssDn+eybHMBFQElPiqCYd2SOXVoF5Z8nsiFV8eza4dw4YXC3/+OFQ3ToJq8cIhIF+APwEBVLRGRWcBZwPHA/ar6oog8DlwEPNbU+RnTEFauhO3bITcXdu1y/s/IgFNPBZ9P6d0HNm6o2mX0m4tLSehdRIXCZX+OoWtGgAFDKkjt5GfN9t18vTGXH+bnkbWrGIDOiTFMGtSJ4d2S6NupLV6Phy1ZHv5+QxuGDoN33xZGjQrDkzetXrjGOLxArIj4gDhgC3AUcI4bfwb4K3spHMXF8PnnVdsSE2HoUOf+vHlQUlI1npICQ4Y49+fOdQYaK+vQAQYMcO5/9ZXzy9vKOneGvn2d+9UfGyA9HXr1gooK+PrrmvHu3Z0PkLIy+PbbmvGePaFrV+e5ff99zXifPpCWBoWFznmHquvfHzp2dK4d/eOPNeODB0O7dk6f+dKlNeMHHQRJSbBtG6xYUTM+bBi0bev8VmD16prxESMgLg6ys2Ht2prx0aOdfvoNG5xbdWPHgtfrfBPPyqoZnzDB+X/FCti0yTmqaM8tIgKOPtqJL1oEmzc77YGAc4uKguOOc+Kffuqs3+//5ZaQAGef7cT/9z8n/9JS57UqLYUuXeDmm534FVc4OZSWKvkFTmEYPlz534sVVASUo4+JYlN21cJw1CQfGYcUUebzM+bIOCYlKV17+Ona3U969wDxbRVVKPFVcOhp29mSX8rrqwtY+HEuBaUVeAT6d0pgyuhuHNItmU6JsWSujODrN6J4M8vDPQ/7GDYujq+/FkaMAI/9Sss0krCMcYjIVcCdQAnwAXAV8K2q9nbjXYH3VLXGCRBE5BLgEoDIyIMP8fkWVYmPGedn1uwyAMaPimFdZtW/nqMn+Xn6eSc+YmAM27ZWjZ/yfxU8PN2pJgO6xVJUVPWP/+zzKrjrfiferV3Nc05ffLmPW2/3sbsQBmbUjF99nY8/Xu8jZ4swanBsjfgtfyvnkisqWLtGmDC6Zvwf95Ux5Xw/Py0STjy6ZvyhJ8o49Vd+5n7l4denxNSIPzmzjImT/Xw4J4KLpkTXiL/0ZimHHhbgjVci+MPvasbf/riEg4YqM5+J4MY/1ox/9l0JPXsr0x/xcsdfal7IYd6SEjp1Vu67K5IH7o6sEV+2vpg2beH2WyP5z6M14xt3Ot+2r78miheerfq9Jz5eWbbB+aZw5cVRvPV61XhqhwDfLS1BVfntlBg+/bBqPKNngPe/KUIVfnN6LN/P9eKNVKKjITJKGTDEz8PPFKIKf/ljPJs2eoiMgvg2SkJigN4D/Jx5fikA334eiXggISlA2wQlIUlp01Z//jAvKfeTU1DKlvwScvJLnVtBKVvyS9ldVvHLc4qK4OCuSQzvlszBXZNoE+1lxZII3nklhq8/jSRnk9OlNXKU8sXnQkzNl9yYGvZ3jKPJC4eIJAOvAr8G8oCXgVeAvwZTOCrr2We4/nHap1Xa2iQo/QY5uwlLFnopK626TGKy0ru/E//pBy++anscye2Unn2d+MJ5XgLV9jjadQiQ0SsAwA9za+6wdewcID0jQEUF/Ph9zXjn9ABpXQOUl8HiBTXjXboH6JQWoLQEli6qGe/W009qR6VoN6xYXDPeo4+flPZKQb6welnNy7b16ucnKUXJ2yWsWVkz3negn4REZed2Yd2amvEBg/3Et1W253jYsK7mV9rBQyuIiYWczR6y13ug2gE8Bw2vICoaNmd52Jxdc/mhIyvweiF7vYdtOTXjw8c4H6ob13nYud2DCD/fPB4YPMyJZ633UJDnQTzqxAQio/j5td2e46GszNlL8XjAE6FERkJSivP34CuHCG9w39oDqhSX+ykqq2B3WQVF7m13mZ+i8l+mi8r85Jf4yCkoJb/EV2UdKfFRdE6MoVNCDJ0SnVvnxFg6JcSQv8vDN59GceiEcrqle3j35VhuvyWKiRPhpJOE44939oSNCVZLHBw/BlinqtsBROQ1YByQJCJeVa0A0oFNe1tRaelOtuc8V6VtW0k6a/y9IFBBxPqafUVby7qzsjwDKsqI2Fizr2irryfLS7tCeTERm2r2FW3192FpURqUFhKRU7OvaGugPz8WdISSPCJyavYVbWUwC3PbIUU78eQsqRHP8RwMO5KQwq14ttXsK8qJGA5b2yIFm/Fsr9lXlBM5EjbHIXlZeHbW7CvKiR4D2dHIrvV4cp2+osqf7V/FjQOPF9mRiScvu8byn7cZ7yyzbRWegi1VYuqJ4JM1hznxrcvxFG6rGvdG8dGqQwHwbFmCFO2sGo+M5YPlTqe8Z9OPSEkeVb7WRLfhnZ8OceLZPyBluystDIHYRDIXHYQCUVnzEV+JE3D/rYhJ4fv8AQQU2myah/h96J64QmlMO/ISe+MPKB23z0cCfgLqzKGq7Ihox1pPFyr8ykjfT6gbCwSc/9f7U1jh70AEfiZG/fLaeESI9wi7vZ3wtOnCIWlxpOf9SJTX49wiPHgEuvYdQqeMnuzaVsi8jz5mU4lQWCgU73ZeobXxYzhxTH9SzthBh8S3fy5q77/v/H/EEUfQs2dPcnJymDNnTo3X7uijj6Zr165kZWXx8ccf14hPnjyZTp06sXbtWr744osa8RNPPJH27duzcuVK5s6dWyN+2mmnkZiYyJIlS6jtaMczzzyTuLg4Fi1axKJFi2rEp0yZQmRkJN9//z1La+lHnTp1KgDffPMNq1atqhKLjIxkypQpAHz++eesW7euSjwuLo4zz3R+Jv/RRx+RnV31vZ2QkMDpp58OwJw5c8jJyakSb9euHSeddBIAb731Fjt3Vn3vdurUicmTJwPw2muvUVBQUCWenp7OMcccA8CsWbMoLi6uEu/Rowfjxzt/WzNnzsTnq/rFom/fvowdOxaAGTNmUN2gQYMYOXIkPp+PmTNn1ogPHTqUoUOHUlxczKxZs2rEQxWOwrERGCMicThdVUcD84FPgV/hHFl1PvDm3lbk8wfYWlB1l2J97i5WZHrdP97SGsusyd3JmtVCND6OrCW+MncH6/wB4qWcwyNrxpfu2k5WwEeClDK2lviPu7axJVBKihQzqpb4gl05bAvspoNnN8O9NePzdm5hl+bT2VPAwbXEv9m5mQKNoasnj0G1xL9ctIkijaJHxC76RdSMf/pDFmVE0jtiJ71riX84byN+IugfsYuMWuJzvnWKzWBvLumeqnE/Hj504wd78+hcLV5GBZ/mOPFDvPmkVosXa4AvtjjxUZEFpEjVeIHCN5uc+NjI3SRUi+8q8DAv2xkYOSKyiDipuju5Pb+QhZs34xFhfGQZ0VLxc9EUYHtZKasK8ojwCAkBP14JIICIICLEeCNIaxuLN0KI3+l1YxAhQoRH6Nq+PSd060ucF3b9uAWPR4gQweM+yIReXenSawClxSV8O2cFZbuFojKhrBTKSoXszVH89sooKIhh49oIRJyjoTIynHGpY45xVhQTY+MXJrzCNcZxG05XVQWwEOfQ3C44RSPFbTtXVcvqW8+AIUP137Peb+Rszb4K9a0lIf4uTZCflxFxpiuvS5yZfp7PE+oD1KOoUCgrg7IyZ4+gIF+IjlEGHuR0hf3viRi2bYmgsEAoyBMKC4Thoyq49pYyoiIiGNarDWWlgsejdOsOffvAOecI55/vbLfMTOdAisiawzzG7LcWN8bRkFI7HKLDxnxTpS2jl58LrnQGSB+8M45dO6p+Nes7sIIpFzvfVP/1l3h2F1b9MBkyrIJfnefE77whnvKyqvFDxvg4+ddOPZt2Tc2D4w8dX87kU8spK4V/3FQzPv7Yco6cXE5BvnDfbfE14hNPKmPckT52bBP+/c+a8RPPKGXEoRVszvIw/f6ag++nTynloEMqWLc6gmceqzl4fvaFJfQb7GflkgheeKpmfOrlJWT09rN4gZdXn6s50nrJNcWkdQ0wf66Xt1+uGb/yhiLad1C++SySD9+qOXh+zV+KSEhUPv8gis8/qDl4fv0du4mOgQ9mRzH385rxafc63VNvzYpm/lznU3XPUVXR0cotdxcB8NLTMfw43/tzPBAQ2iYGuOUuJ/7EfbEsWRhJwP/LUVcdOwe446HdiAjTroln8UIvvnLBV+4UiH4D/fzvjSI8Ipw+MZ4VS6uOAY093M/r7/jwejyMGR7Jju2QlAwpyZCcLBx1FNx4ozPvp5864xI9ejhHmRnTlFriGEeD8ZUL61ZU/XBp10YZ3t35S9yR5alxyGe3jpEM7+584OWs97B9e9X4gJ6/xLNXeyiq2hXJ8MGRDO/ufOCuW1Gzv+Dw0ZEM7x7H7t2QuaxmfNIEJ751a+3x006IZHh3ZV2g9niSOPHowtrjqVFehneHipza453inHj+Wide/XtDehsnvvknWLO05vI9krwM6g5rvhXWLK35Db5Pey89usOiCmHV4prxAR29dOwI35QIK36sGR/SJYk2beDDotqXH949CYA3C4W1y6vuccTHwSHdnffDKyXC1qyqA+fRwKge0QjwaizEeCAiUn4eIO/aAUb3dN47hw6Hdu4pPqKjne6hjAwvg9Kc84//5WbnENzoaOcw3uRkSEuLoENbp5isXFH/HtSRR9YdM6a5a9F7HHbKEWOMCd3+7nHYEJsxxpiQWOEwxhgTEiscxhhjQmKFwxhjTEiscBhjjAmJFQ5jjDEhscJhjDEmJFY4jDHGhKRF/wBQRAqBlfu4eCKQvw/x2tort1WP75mubZ72wI6Qsq4/t2DmqSu/2qb3dr8l5l+5ranzD6atOeS/r+/96tP15b+vudeX397irT3/UD57+qlq29DSrkRVW+wNmL8fy07fl3ht7ZXbqsf3TNc2z77mv7fcQ8m/vum93W+J+Vdra9L8g2lrDvnv63s/lPyb499uS8+/KT579twO5K6qt/YxXlv7W/XE3wpinlAFs3yw+dc3Hcz9fRHO/Pc392DXsbf3SV1tzSH/fX3vV5+2/IPPI5h4c/jsAVp+V9V83Y/zrYSb5R9eln/4tOTcwfJv6Xsc08OdwH6y/MPL8g+flpw7HOD5t+g9DmOMMU2vpe9xGGOMaWJWOIwxxoTECocxxpiQtNrCISKnish/ROQlETk23PmESkR6isiTIvJKuHMJhojEi8gz7jafEu58QtXStnd1reD9PkBEHheRV0TksnDnsy/cv4H5InJiuHMJlYhMEJEv3ddgwt7mb5aFQ0SeEpFtIrKkWvtkEVkpImtE5Ib61qGqb6jqxcClwK8bM9/qGij/tap6UeNmWr8Qn8fpwCvuNj+5yZOtRSj5N4ftXV2I+Yft/V6XEPNfrqqXAmcC48KRb3X78Hd8PTCrabOsW4j5K7AbiAGy97ry/fn1YGPdgCOA4cCSSm0RQCbQE4gCfgQGAkOAt6vdOlRa7l5geAvO/5UW8jrcCAx153k+3O+hUPNvDtu7gfJv8vd7Q+WP84XjPeCccOceav7AROAsYCpwYrhz34f8PW68IzBzb+v20gyp6hciklGteRSwRlXXAojIi8ApqvoPoMauoYgI8E/gPVVd0LgZV9UQ+TcHoTwPnG8p6cAimsmebIj5L2va7PYulPxFZDlher/XJdTtr6qzgdki8g7wfFPmWpsQ828DxON8CJeIyLuqGmjCdGsI8XNoz/s/F4je27qbxR94kLoAWZWms922uvweOAb4lfx/e3cTGlcVhnH8/xQUW9RWXIig0CqKRYlVWwVB7UIX2p1fiBI/6MK6UkHQkoW4EwQXQWylQuMHKIotuhBxlSiitNhqglQrFBQlKEgNttIi5nVxzsB17Jg5k0nv3Mzzg8DMyXDvcy8zeXPuHc4rbVvKYF0qyi/pfEk7gWskbV/qcAU6Hcce4C5JO+jTsgZL5JT5B/h8t+t0/gft/d5Jp/O/WdK4pFeAD+uJ1pVT5o+IsYh4glTwdtVdNP5Hp/N/Zz73bwAvLbSRgZxx9ENEjAPjdefoVUT8Rrpe3QgRcRx4pO4cvWra+W63DN7vk8BkzTEWLSIm6s7Qi4jYQ/rnrytNmnH8DFxceX5RHmuKpudvafpxOH+9nL9efcnfpMKxH7hM0jpJZ5JuRH1Qc6YSTc/f0vTjcP56OX+9+pO/7jv/Hb4N8BYwC/xFuga3NY/fARwmfStgrO6cyzX/cjkO53d+51+a/F7k0MzMijTpUpWZmQ0AFw4zMyviwmFmZkVcOMzMrIgLh5mZFXHhMDOzIi4cNhQkHevz9ta2L1e9iG1NSLq7i9dNStrYj32aLYYLh5mZFXHhsKElaYOkLyRNS9or6bw8vimPfSXphYVmFnn28amkA/nnxjy+WdKUpPclHZH0vKQHJO2TNCPp0spmblXqHndYuYOcpJWS3pZ0SNJeYGVlnzvy67+R9Fz/z45ZZy4cNsxeB56OiBFgBng2j+8GHo2IDcDfXWznV+C2iLiW1H2vukrt1aRVd9cDo8DlEXE98CppKfSWtaReCVuAnZLOAh4D/oyI9TnbdZXXj0XERmAEuEXSSNdHbbZILhw2lCStBtZExFQeeg24WdIa4JyI+DyPd9NQ6Axgl6QZ4F1SM5+W/RExGxEnSWsDfZzHZ0jFouWdiJiPiO+BI8AVpA5ubwJExDQwXXn9vZIOAAeBK9v2abaklm0/DrPT6Eng5lqFJgAAAR9JREFUF9LsYgVwovK7k5XH85Xn8/z789e+aFzHReQkrQOeAjZFxFFJE6Re0WanhWccNpQiYg44KummPDQKTEXE78Afkm7I4/d1sbnVwGykrm+jpL7Ope6RtCLf97gE+A74BLgfQNJVpMtSAOcCx4E5SRcAt/ewP7OeecZhw2KVpJ8qz18EHiLdT1hFujzU6mC4lXTpaR6YAuYW2PbLwHuSHgQ+Iv1RL/UjsI9UFLZFxInchnd37id+CPgSICK+lnQQ+JbUBvSzHvZn1jMvq27WRtLZEXEsP34GuDAiHq85ltnA8IzD7L+2SNpO+nz8ADxcbxyzweIZh5mZFfHNcTMzK+LCYWZmRVw4zMysiAuHmZkVceEwM7MiLhxmZlbkH82tcc/4OBv5AAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "tags": [], + "needs_background": "light" + } + }, + { + "output_type": "stream", + "text": [ + "best score: 78.32898324034278\n", + "lambda: 2.592943797404667\n", + "(Standardized)\n", + "Intercept 37.980193\n", + " Coefficient\n", + "X1 transaction date 1.436194\n", + "X2 house age -3.048938\n", + "X3 distance to the nearest MRT station -5.543872\n", + "X4 number of convenience stores 3.336385\n", + "X5 latitude 2.806742\n", + "X6 longitude -0.104249\n", + "\n", + "(Real)\n", + "Intercept -15050.879394\n", + " Coefficient\n", + "X1 transaction date 5.099639\n", + "X2 house age -0.267951\n", + "X3 distance to the nearest MRT station -0.004398\n", + "X4 number of convenience stores 1.134053\n", + "X5 latitude 226.437833\n", + "X6 longitude -6.800912\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "qGSTHaMbLIJ8", + "colab_type": "code", + "outputId": "20032b37-bdb9-4735-cf58-1e2db11e5a8c", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 673 + } + }, + "source": [ + "print('Elastic Net Results:')\n", + "alphas_elastic_net = np.logspace(-2, 1, 30) # the range of the parameter\n", + "parameters_elastic_net = [{'alpha': alphas_elastic_net}]\n", + "cv_elastic_net = GridSearchCV(linear_model.ElasticNet(), parameters_elastic_net, scoring = 'neg_mean_squared_error')\n", + "cv_elastic_net.fit(X_std, y)\n", + "scores_elastic_net = cv_elastic_net.cv_results_['mean_test_score'] # the scores are negative\n", + "scores_elastic_net = np.negative(scores_elastic_net) # reverse the sign of the score\n", + "scores_std_elastic_net = cv_elastic_net.cv_results_['std_test_score'] # standard deviation\n", + "\n", + "# plot the cross validation score against log lambda\n", + "fig_elastic_net = plt.figure()\n", + "plt.semilogx(alphas_elastic_net, scores_elastic_net)\n", + "std_error_elastic_net = scores_std_elastic_net / np.sqrt(X_.shape[0])\n", + "# shade the region within 1 std err\n", + "plt.semilogx(alphas_elastic_net, scores_elastic_net + std_error_elastic_net, 'b--')\n", + "plt.semilogx(alphas_elastic_net, scores_elastic_net - std_error_elastic_net, 'b--')\n", + "plt.fill_between(alphas_elastic_net, scores_elastic_net + std_error_elastic_net, scores_elastic_net - std_error_elastic_net, alpha = 0.25)\n", + "plt.ylabel('CV score +/- std error')\n", + "plt.xlabel('Log lambda')\n", + "plt.axhline(np.min(scores_elastic_net), linestyle='--', color='.5')\n", + "plt.xlim([alphas_elastic_net[0], alphas_elastic_net[-1]])\n", + "plt.title('CV score as a function of lambda')\n", + "plt.show()\n", + "# print the optimal values\n", + "print('best score:', np.negative(cv_elastic_net.best_score_))\n", + "print('lambda:', cv_elastic_net.best_params_.get('alpha'))\n", + "\n", + "# print the standardized coefficients\n", + "print('(Standardized)')\n", + "elastic = linear_model.ElasticNet(alpha = cv_elastic_net.best_params_.get('alpha'))\n", + "elastic.fit(X_std, y)\n", + "elastic_df = pd.DataFrame(elastic.coef_, X_.columns, columns=['Coefficient'])\n", + "print('Intercept %.6f' %(elastic.intercept_))\n", + "print(elastic_df)\n", + "\n", + "# back-convert for ease of interpretation\n", + "# beta_real = beta_std / std err\n", + "# intercept_real = intercept_std - sum of (mean * beta_std / stderr)\n", + "print('\\n(Real)')\n", + "elastic_real = elastic_df.copy()\n", + "elastic_intercept_real = elastic.intercept_\n", + "count_elastic = elastic_real.shape[0]\n", + "for i in range(count_elastic):\n", + " elastic_real.iloc[i, 0] /= np.std(X_.iloc[:, i])\n", + " elastic_intercept_real -= (elastic_real.iloc[i, 0] * np.mean(X_.iloc[:, i]))\n", + "print('Intercept %.6f' %(elastic_intercept_real))\n", + "print(elastic_real)" + ], + "execution_count": 94, + "outputs": [ + { + "output_type": "stream", + "text": [ + "Elastic Net Results:\n" + ], + "name": "stdout" + }, + { + "output_type": "display_data", + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEaCAYAAAAG87ApAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdd3hUZfbA8e+Zkp6QhBBqQuihiDQFQUXsDfuiiL3t2nvf1bX+7Ou6urrq2lgbuvaCbREbiohIk94htEB6mXZ+f9yLpJCQCGFIOJ/nmWfmvu8tZyaTe+a+73vvFVXFGGOMaShPtAMwxhjTvFjiMMYY0yiWOIwxxjSKJQ5jjDGNYonDGGNMo1jiMMYY0yiWOIxpABE5UURWikiJiAzchdsdJyKf7qrtVdnuCBFZ6L7fE7ZRv0xEDm2C7R4kIqsaMf85IvLNzo7D1M8SRwskIqeLyDT3nz5PRD4Wkf1F5DT3H15qzO8TkfUicmy0Ym4GHgIuU9UkVf25KTYgIjkioiLi21Kmqi+r6uFNsb3tuBN43H2/70Rh+2Y3ZomjhRGRa4BHgXuBtkA28E/geOAdIBUYWWOxIwEFJu66SB1Vd5K7uc7AnGgHsQvtae/XNIIljhZERFrh/FK8VFXfUtVSVQ2q6vuqer2qVgATgLNqLHoW8Iqqhraxzu4iMllECkVko4i8XqWur4h8JiKbRGSdiNzilseKyKMissZ9PCoisW7dQSKySkRuFJG1wPMi4hGRm0RksYjki8gEEUmv4z2micgHIrJBRDa7rztVqT9HRJaISLGILBWRcXWsZ18RmSIiBe5R2eMiErON+WJFpATwAr+IyGK3XEWke5X5XhCRu2u8x2vdI7k8ETm3yrzxIvKwiCx3P9dvRCQe+MqdpcA9WtyvZlOMiAwXkR/d5X4UkeFV6r4UkbtE5Fv3/X8qIhnbev/u/BeKyCL37/eeiHRwyxcDXYH33Thi61pHQz5L97O6xG36KnZj7CYi34lIkfv3jqmxzlvc79uyqn9DEWntxlokIlOBbjWW+7s4TYpFIvKTiBxQX+zmd1JVe7SQB86RQwjw1TPPCKAIiHenWwHlwIA65n8VuBXnR0YcsL9bngzkAde65cnAULfuTuB7IBNoA3wH3OXWHeTGeD8QC8QDV7rzd3LL/gW8Wkc8rYGTgQR3m28A77h1ie576+VOtwf61rGewcAwwAfkAL8CV9XzuSnQvZ7pF4C7a7zHOwE/cDRQBqS59U8AXwIdcRLScPd957jr9VVZ7znAN+7rdGAzcKYb91h3urVb/yWwGOjpfq5fAvfV8X4OBjYCg9xt/wP4qkr9MuDQej6P3+q391m67+ldIAXoC1QCX+Akp1bAXODsGp/dI25cI4HSKn/T13B+/CQC/YDVWz4ft/4M9zviw/lurgXiov2/2dIeUQ/AHjvxjwnjgLUNmG8hcLr7+kLgl3rmfQl4GuhUo3ws8HMdyywGjq4yfQSwzH19EBCo+s/s7mgOqTLdHghSTwKsMu8AYLP7OhEowEks8Y387K4C3q6nvrGJo5zqCWC9u3P1uHV7b2MbOdSfOM4EptZYZgpwjvv6S+DPVeouASbW8X7+DTxQZTrJ/cxz3OllNDBxbO+zdN/TiCrTPwE3Vpl+GHi0ymcXAhKr1E8A/oKTZINAbpW6e6mSOLYRy+Ztfdb22LGHNVW1LPlARgP6DV5ia3PVme50XW4ABJgqInNE5Dy3PAsnQWxLB2B5lenlbtkWG9RpNtuiM/C229RRgJNIwjh9NNWISIKI/Mtt5inCad5JFRGvqpYCpwJ/AvJE5EMRyd1WgCLS023mWuuu516gzmad3yFfqzf9leHsnDNwjtDq+uzqU/NzxZ3uWGV67Ta2ud11qWoJzvenYx3z16mBn+W6Kq/LtzFdNc7N7t9yiy3fnzY4RxIra9RVjeU6EfnVbcorwDmi2Zl/V4P1cbQ0U3CaAWoNn6xhPHCIiOyH8yv45bpmVNW1qnqhqnYA/gj8023bX4nT1LAta3CSwRbZbtlvq60x/0rgKFVNrfKIU9XV21j3tUAvnGaxFOBAt1zceD9R1cNwjlrmAc/UEeOTbn0Pdz23bFlHA5XhNJdt0a6By20EKqjRNu/a3qWqa36u4Hy22/qctqfaukQkEaeJ5/esa0c/y5rS3Hi22PL92YBzNJJVow4Atz/jBmAMTrNgKlC4g7GYbbDE0YKoaiFwG/CEiJzg/jr3i8hRIvJAlfmWAd/g9F98pqprt71GEJE/VOl83oyzc4sAHwDtReQqtwM5WUSGuvO9CvxZRNq4nbO3Af+pJ/SngHtEpLO7zTYicnwd8ybj/EItEKcD/fYqsbYVkePdnU4lUOLGWtd6ioAS96jk4nri25YZwOki4hWRI6k9Um2bVDUCPAc8IiId3OX3czugN7jx1pWQPwJ6ijPc2icipwJ9cP4WjfUqcK6IDHC3fS/wg/vdaKwd/Sy35Q4RiXGTwbHAG6oaBt4C/up+t/sAZ9eII4TzOfpE5DacfhWzk1niaGFU9WHgGuDPOP9AK4HLcIbiVvUizi/O+pqpAPYBfhBnZNF7wJWqukRVi4HDgNE4zSMLgVHuMncD04CZwCxgultWl7+76/5URIpxOsqH1jHvozgdvxvd+aoOIfbgvPc1wCacnXldO7HrgNOBYpyjktfrmK8uV+K89wKcvqXGnOtwHc7n8qMb5/2AR1XLgHuAb91mu2FVF1LVfJyd6LU4zUo3AMeq6sZGxo6qfo7Tb/BfnEEO3YDTGrueKu9nRz7Lmtbi/EhZg3M0/CdVnefWXYbTrLUWp1/p+SrLfYLzfViA04RVQfVmLbOTiNuBZIwxxjSIHXEYY4xpFEscxhhjGsUShzHGmEaxxGGMMaZRLHEYY4xplOZyZdJtysjI0JycnGiHYYwxzcpPP/20UVXb/N7lm3XiyMnJYdq0adEOwxhjmhURqXnpmkaxpipjjDGNYonDGGNMo1jiMMYY0yiWOIwxxjSKJQ5jjDGNYonDGGNMo1jiMMYY0yiWOIwxZg8RicDY03f8VhqWOIwxZg/xySfw2qs7fiddSxzGGLOHePRRJSOzrrspN5wlDmOM2QPMmQOffip0Gr56h9dlicMYY/YAL74IMbHK5ux52595O5r1RQ6NMcY0zM23BVmVtoQphYEdXpcdcRhjTAunCqsLS5lZsYzBndN2eH1NljhE5DkRWS8is2uUXy4i80Rkjog8UKX8ZhFZJCLzReSIporLGGP2JMEgDB6s3PX3UorLQywcP2CH19mUTVUvAI8DL20pEJFRwPHA3qpaKSKZbnkf4DSgL9AB+FxEeqpquAnjM8aYFu/NN+Hnn4WKvTeSFMlm5pSEHV5nkx1xqOpXwKYaxRcD96lqpTvPerf8eOA1Va1U1aXAImDfporNGGP2BKrwyN+UtlkBSjNXUf5zN7Kzm98JgD2BA0TkBxGZLCL7uOUdgZVV5lvlltUiIheJyDQRmbZhw4YmDtcYY5qv77+HaT8KbfdbSUxhOktnJXD55c3vBEAfkA4MA64HJohIo96Fqj6tqkNUdUibNr/7lrnGGNPi/e1vSmJyhM0dF5K0vDuJicr55+/4enf1cNxVwFuqqsBUEYkAGcBqIKvKfJ3cMmOMMb/T2eeHWJ+2klVxER55GJLLhLQdH1S1y4843gFGAYhITyAG2Ai8B5wmIrEi0gXoAUzdxbEZY0yL0qZnEXlt5jOiewb9s1MYMmTnrLfJjjhE5FXgICBDRFYBtwPPAc+5Q3QDwNnu0cccEZkAzAVCwKU2osoYY36fkhK49S8RQrkbqKiAbx4awKQUP6ecsnPW32SJQ1XH1lF1Rh3z3wPc01TxGGPMnuKll+CxRz30vKCIVqVdmfVrDKmpO2/9dua4Mca0IJEIPPp3JTu3nPK0DWyamkPfvsohh+y8bVjiMMaYFmTiRFi4QEgZspSE/HasXhzLVVcJjRu/Wj9LHMYY04I8+qiS3iZEUbtl6OwetG6tjBu3c7dhV8c1xpgWIhSCzPYRclJXURbn4bLLIUWF+Pidux1LHMYY00L4fHDBLfmc/8JcjujVljNOiqfVjl+aqhZrqjLGmBYgPx8mfxvinZ9XE6r0kv9lL0oL/E2yLUscxhjTAjz1FBy0v4+PvismbWVPJvw7iby8ptmWNVUZY0wzFwjA408o3QeVUhlXTMF3ndh/f2Xw4J04lKoKO+Iwxphm7o03YG2e4O+/iOR12WxY4+eqq5omaYAlDmOMadZUnSG47bIDlGaupvznbnTurBx/fNNt05qqjDGmGVu1ChYuhM5HriQSE0NODx+HjRR8Tbh3tyMOY4xpxjp1Uv71UR6bOy7giP6ZvPqKhyuuaNpt2hGHMcY0UyUlUB4J8NnCNXjDfvrFZZEU2/S7dTviMMaYZurqq+HAER6+mr+BlKW9Oe/4NFau3P5yO8oShzHGNEN5efDSS0pcx0IqA7D8m3YccwxkZW1/2R1licMYY5qhxx6DYBDKe/5KWl43Cjd5m3QIblWWOIwxppkpLoYnn1T6DC+mPL6I/B8607efcvDBu2b7ljiMMaaZGT8eCgsF6T+fdpLO5rUxXL2T77lRHxtVZYwxzcxZ54aZXbSWjwrWc87B3Rl3bYTUZO8u274lDmOMaWY2lVWyOHYxaTFxHN2vPe3a7LqkAZY4jDGm2VCFo45SOu9TxvxgMa2mDuf8L5L59ht2WTMVWB+HMcY0G59+Cp98Ivy0fDMxlQnM+TqVfYbsur6NLSxxGGNMM/HAA0p6mzAb2y4kZVFfUOckwF3NEocxxjQD06fD//4ndBu1Bk/Ex6+TMhgzRsjJ2fWxWOIwxphm4MEHISExwsZOv5KZl0tZqYfrr49OLNY5bowxzcDYM4NsTMljkT/IDVf78I9VBg7cxZ0bLkscxhjTDHToU8yaH+YypH0aw3qk0WGf6CQNsKYqY4zZreXnwxVXR3h50npKKsIseGkgkz6KjWpMljiMMWY39s9/wj8e9fDp9E203tSZmVPi0Uh0d92WOIwxZjdVXg7/+IeSu28JpYmbKZ7alU5ZyqmnRjcuSxzGGLObevFF2LBBiBm4kOTiTBbNTOCaqwW/P7px1Zs4RMQrIvN2VTDGGGMc4TA89JCSk1tBQas18EsvWrVSLrgg2pFtZ1SVqoZFZL6IZKvqil0VlDHG7OmKi2HvwWGWJy8llODnwos8xIeE5ORoR9aw4bhpwBwRmQqUbilU1eOaLCpjjNnDJacop12fx/VvLuG0flmccXQsreKjHZWjIYnjL00ehTHGmN/Mng1rNwd4Z/4afIE48r7oTuUBfthNEsd2O8dVdTIwD0h2H7+6ZcYYY5rAzTcrp5zg47sF+aQu7c1LTyawaVO0o9pqu4lDRMYAU4E/AGOAH0TklAYs95yIrBeR2duou1ZEVEQy3GkRkcdEZJGIzBSRQY1/K8YY0/zNnAkffCD0OGgdRDws/LIdxx2n5OZGO7KtGtJUdSuwj6quBxCRNsDnwJvbWe4F4HHgpaqFIpIFHA5U7Ww/CujhPoYCT7rPxhizR7nvPohPjLA5Zw4ZebksK4jexQzr0pDzODxbkoYrvyHLqepXwLYOrv4G3ABolbLjgZfU8T2QKiLtGxCbMca0GIsWweuvK3sdspGgN8iKLzux337KiBHRjqy6hhxxTBSRT4BX3elTgY9+z8ZE5Hhgtar+ItVvWdURWFllepVblreNdVwEXASQnZ39e8Iwxpjd0syZ0CpNKe4xl97p6fj2gQvP3fV3+NueehOHOHv3x4B9gP3d4qdV9e3GbkhEEoBbcJqpfjdVfRp4GmDIkCG6ndmNMabZOOSoIJc+vZDxU0s5df/OnHG9B7832lHVtr0TAFVEPlLVvYC3dnBb3YAuwJajjU7AdBHZF1gNZFWZt5NbZowxe4T586EyoZSP566mfTiTDmTi9+6eV4VqSFTTRWSfHd2Qqs5S1UxVzVHVHJzmqEGquhZ4DzjLHV01DChU1VrNVMYY0xLl58OgQcpV14fYWBKg6Ks+nH1KAsFgtCPbtoYkjqHAFBFZ7A6VnSUiM7e3kIi8CkwBeonIKhE5v57ZPwKWAIuAZ4BLGhCXMca0CI89BmVlwqb2i0ivyGTu1EQuuyz6FzOsS0P6OC4Cljd2xao6djv1OVVeK3BpY7dhjDHNXXExPPaY0n9EKQWx+aROOYDkZOWyy3azHvEqGtLH8YTbx2GMMWYne+opKCgQug6eT6tAGjO/TubGG4XU1GhHVrdd1sdhjDGmOlV47XUld3AZ+fFryfV1JjUNrroq2pHVr6F9HN83to/DGGNM/UTg5XfLaDd6JumJMdx8WQKrVwlt20Y7svo15ATAI5o8CmOM2cOEQlARiPDD8g0sLc/nmM49yGmdRPxucgXc+jTk0iHLcc6xONh9XdaQ5YwxxtTt1VehZw945YuNxIXi+fc1Pfj7Q7vpMKoaGnJ13NuBG4Gb3SI/8J+mDMoYY1qySATuu0/xJ4aYV7KOtKV9CVTCmDHRjqxhGnLkcCJwHO7d/1R1Dc59OYwxxvwO770Hc+cKHUcuJyYSw+zP2nDSSdC7d7Qja5iGJI6Ae56FAohIYtOGZIwxLZcq3HOv0j4rxJr0BbRZ2YeSYg8337z7nrdRU0MSxwQR+RfOpc4vxLkXxzNNG5YxxrRM33wD034Uuhy8Ep9XWDcjkyOOUAYPjnZkDbfdUVWq+pCIHAYUAb2A21T1syaPzBhjWqDhw5W7n9zMv5fM55DcTG75Mow/1Dw6xbdoyHBc3ERhycIYY3ZQfmklq5KWohLh+L060DkzFk8zG6faoMRhjDFmx51xhpLQNsTk+PW039CLy05qxzdfC506RTuyxmlmec4YY5qnmTPh5ZeF2SuLqQxEWDEpm5QUoWPHaEfWeHbEYYwxu8B990FCYoT8znPoUNCdKcv8PPwau91tYRuizsQhIrNwh+Bui6r2b5KIjDGmhVm0CF5/Xdln9EbyPJXkfdWF7t2VU05phlmD+o84jnWft9wnY7z7PK7pwjHGmJbnnnvA74fC7nPpWJbNlPkxPPsseHfD+4k3RJ2Jw70uFSJymKoOrFJ1k4hMB25q6uCMMaYlOO+iIJtbrWeGr5RrjsvmmlFhjju2mWYNGtbHISIyQlW/dSeGY53qxhjTYEkdSliVMZceCUmMym1D98zmmzSgYQngfOCfIrJMRJYB/wTOa9KojDGmBfj1Vzh1bJg3vt7IxpIAhR8P5K2Xmv9VmxpyxLFJVfcWkVYAqlooIl2aOC5jjGn27rwT3n/PQ68ua8iozOTHz1M4/qBoR7XjGnLE8V9wEoaqFrplbzZdSMYY0/zNnu2MpNr3mHw2R0rQGbkkJyuXXrr9ZXd39Q3HzQX6Aq1E5KQqVSlAXFMHZowxzdkddygJiVDQfTbtIxn8MDmJG24QUlOjHdmOq6+pqhfOkNxUYHSV8mLgwqYMyhhjmrNffoE33xRGnrqBZZFS4n4eTGwsXHVVtCPbOeobjvsu8K6I7KeqU3ZhTMYY06xlZipjz69kduZsemQkccGVQumpQrt20Y5s52jQHQBFJEVE/CLyhYhsEJEzmjwyY4xppnxJAXodt5iiSBl/GJLFMYfEcV4LGovakMRxuKoW4TRbLQO6A9c3ZVDGGNNc3Xqr8tbHFbwzYzWdaMunT3WitKBlXRawIYljyx1GjgHeqDKyyhhjTBU//AD33iu8/n45m8uCVE7N5ZP3/c320iJ1aUjieF9E5gGDgS9EpA1Q0bRhGWNM83P77UpqWoT1WXPI0nZMm5TElVcKGRnRjmzn2m7iUNWbgOHAEFUNAmXA8U0dmDHGNCfffguffCIMGr2OkkgF5T/kkpKiXHNNtCPb+Rp0zSlV3aSqYfd1qaqubdqwjDGmebn9diWtdYR1HeeQHenAT5MTufpqIT092pHtfC2rx8YYY6IgEoHhI0PE9FjH3Egl5w/PoM+fIlx1Vcu8HmzLfFfGGLNLKYedVsCKzDkMyEpl9PBUnnrS0yLOEt+WRiUOEflrE8VhjDHN0nffwT+fDfLu9DxKKkN4pvdlzcKEaIfVpBp7xHFck0RhjDHNkCrccINy521ePvxlLV0iWbz9XCpTvmlh429raGziaJ43yDXGmCbw2Wfw7bdC/2PzKI8Eyf+qB23atIwr4NanzsQhImNFpHWN4sFNHI8xxjQLqvCX25S2HcKsbDOHrqHOzPohnhtvFJKSoh1d06rviCMbeENEvhaRv4rIUEAbumIReU5E1ovI7CplD4rIPBGZKSJvi0hqlbqbRWSRiMwXkSN+17sxxphd5OOPYeoPQr9jVlOpITZM7k7btsrFF0c7sqZXZ+JQ1ftV9WDgaOAXnNvFTheRV0TkLBFpu511vwAcWaPsM6CfqvYHFgA3A4hIH+A0nPt/HIlzq9qW3UhojGnWxKPse0AlKzPmsm9OOocdKtx9t5DQsvvFgQacx6GqxcDb7mPLTv4o4CWgziMDVf1KRHJqlH1aZfJ74BT39fHAa6paCSwVkUXAvoBdzt0Ys1vqt285Qy+ez/u/hDl1n04cc76POP/2l2sJGtw5LiJpAKo6V1UfVtUdbU46D/jYfd0RWFmlbpVbZowxu5WCArjjzgg/Lynikzlr6UFn1s5oQ0xLu5JhPRozquqLnbVREbkVCAEv/45lLxKRaSIybcOGDTsrJGOMaZDbb4c7/iq8/OlmKkMRVnzcg//7ayyBQLQj23Uakzh2ylBcETkH594e41R1S2f7aiCrymyd3LJaVPVpVR2iqkPatGmzM0IyxpgGmT0bnnhCOeLkUqaXLqVbZTfmz4zlL38W4uKiHd2uU28fh4icteUlkFZlGlV9qbEbE5EjgRuAkapaVqXqPeAVEXkE6AD0AKY2dv3GGNNUVOHyy5WkZMU/dC6yWVj1WVc6d1bOPXfPOsVte53jXaq8jgVycJLIdoflisirwEFAhoisAm7HGUUVC3wmIgDfq+qfVHWOiEwA5uI0YV265Wq8xhizO5gwAb78Uhh79Ua+27iBwdKHt+bE8OyzEBMT7eh2LdnaWrSdGUWmq+qgJo6nUYYMGaLTpk2LdhjGmD3AtOlh7nogyPr+36LAyW2GMPH1FCZ+LPib2WgqEflJVYf83uV3eR+HMcY0R6mdyuh36kLyiio4c7/OjD3FzxefN7+ksTM0JnGc2WRRGGPMbmrRIhh3ZoSpc8v47/RV9G3dmkWfd6B1XHy0Q4uaBicOVZ29/bmMMaZlufpq5d13hP9OW00orPBTPx67L54FC/bcRhi7kZMxxtThww/hgw+EY87azE8b8xjaqgsT30jkT38S9t472tFFj9061hhjtqGyEq66SsnuGmFNp5mkBWOY/WY30tLgrruiHV10NeaSI3vApbuMMcbx6KOwaJEw8uxVrCwspV9FX2ZOi+Gee4T09GhHF13bPeIQkeHAs0ASkC0iewN/VNVLmjo4Y4yJlpNPDbG8oIyvwvPo0z6Fw7sm4j9XueCCPbdvY4uGHHH8DecquPkAqvoLcGBTBmWMMdEUiSgV/lICufMpC4Q4e3gORx4Ux3PPCXvQtQzr1KCmKlVdWaPIzuo2xrRIkyfDiAOUSdNL+OLX9QxN78x/H8vEE4iNdmi7jYYkjpVuc5WKiF9ErgN+beK4jDFmlwuF4LLLleXLlXfnLiU5zseSD3ow8d0YgsFoR7f7aEji+BNwKc79MVYDA9xpY4xpUZ58EmbPEg49N49FmwsZoH2Y+lUst98utGsX7eh2H9u7Oq4X+LuqjttF8RhjTFSsXw+33aYMHh7gF+9cchKTmfRUB3rlKpdfbh3iVdV7xOFeobaziOxh1340xuxpHnhAKSmBnicsprAiSObyvVi13Ms/HpM97uq329OQEwCXAN+KyHtA6ZZCVX2kyaIyxphd7OJrK4jvUsIrq5ZxUM82nN7Xy9DuymGH2dFGTQ1JHIvdhwdIbtpwjDFm18rLA09MiA1lpcyIzCfW7+G0fbPZr3s8hw+1pLEt200cqnoHgIgkudMlTR2UMcbsCuEwjBmjbCqE8x/axKzVhRyY1Ie7Ls3gjVd9JHeIdoS7p+2OqhKRfiLyMzAHmCMiP4lI36YPzRhjmtb998M33whHjy3h+e+W0jk1kW/HZ7F6hZe0tGhHt/tqSFPV08A1qjoJQEQOAp4BhjdhXMYY06SmTYPbb1cOPSbA/KTZVJSE6VYygK8W+HjrLYjfc2+3sV0NOY8jcUvSAFDVL4HEJovIGGOaWGkpnD5OaZ0ZYdDpS5i5upATenXljWdacdhhygknRDvC3VuDRlWJyF+A8e70GTgjrYwxplkqKoJWaWFOv2YTL89ZyoCsVFZM6kxFGTz2mCDWJ16vhhxxnAe0Ad4C/gtkuGXGGNMseRMreOjFzUwqmkOc38sfD+zKbbdHePddITc32tHt/hoyqmozcMUuiMUYY5rU2rVw3fURzriinA8XrmR5fhkXDulNelw8vbMT6JsT7Qibh4aMqvpMRFKrTKeJyCdNG5YxxuxcqnDuucqbbwg/zivhg5l5jOqZyYT7srjizFaItU81WEP6ODJUtWDLhKpuFpHMJozJGGN2un/+EyZOFC67tYi3l82jbUoc3lm9mfmTn/HjsX6NRmhIH0dERLK3TIhIZ0CbLiRjjNm55s6F665T9hsZYEP2XArLghzdvg8vPp7Iqacq4+wyro3SkCOOW4FvRGQyIMABwEVNGpUxxuxE116rxCUoh/5xBS/9ks8J/bL5922ZtG0HTz5po6gaqyGd4xNFZBAwzC26SlU3Nm1Yxhiz8/z14VIm/1jO83MX0attMiM7d+Tr1sJDD4idIf47NKRzfARQrqofAKnALW5zlTHG7NYWL4Z1BZWE/BX8b7Nz49JLDurGXr1imDbVw6hRUQ6wmWpIH8eTQJmI7A1cg3Ol3JeaNCpjjNlB69bByIOUc8+F92auYd7aYk7p040XH8wgRRKseWoHNCRxhFRVgeOBJ1T1Cezy6saY3VhFBZxwopK/EYafsIk3p61iaJd0vni6Mx+/HcOG9Q3Z9Zm6NOTTKxaRm3EuNfKhiHgAf9OGZYwxv48qnH++8v0U4ab7i/rk3XAAACAASURBVPhwzVxaJfhpu7ovU76M4YEHhL52fe8d0pDEcSpQCZyvqmuBTsCDTRqVMcb8TvffD6+8IvzxmjJWtlrA2sIKTsjJ5ZkHkzniCOWyy6IdYfPXkFFVa4FHqkyvwPo4jDG7qWEjA4y7MEz2wct5+uv1HNu/PR891Y7ERHj+eRt6uzM05DwOY4zZ7a1fD/EpQeLbFjPqrCLu/Wgp/Tq24tR9svjjkDAF6/y0bx/tKFsGSxzGmGZv5UrYd1/l+LFBjjyzgr99toC2ybGM7d2L1glx9OkYBz2jHWXLUWcfh4hcLyKddmUwxhjTWCUlcOxopbgEhhxUxgOfzAPgon37cN056dx/a1KUI2x56usc7wBMEZGvReQSEWnTmBWLyHMisl5EZlcpS3evtrvQfU5zy0VEHhORRSIy0z1T3Rhj6hUOw7hxyuxZcMejRby7fB7riiq5bGRPHr0pg4J8D5dfZp0aO1udiUNVrwaygT8DewEzRWSiiJwtIg05j+MF4MgaZTcBX6hqD+ALdxrgKKCH+7gI56RDY4yp1y23wHvvCVf+uYx5/oXMWl3IeSO68N7j7fllmp8XXxSGDIl2lC1PvcNx1TFZVS/GGYb7N+AqYN32VqyqXwGbahQfD7zovn4ROKFK+Uvu9r4HUkXEurGMMfXK7R9g7PnlJA9axmdz13Fs//as+Sqbj9+O4447YMyYaEfYMjWoc1xE9gJOwzmnYyNw8+/cXltVzXNfrwXauq87AiurzLfKLcvDGGNqKCmBsCdI7/2KqehYwAOfLGNw5zTG7pPN2tbgq1D+8hdromoqdSYOEekBjMVJFmHgNeBwVV2yMzasqioijb6vh4hchHtZ9+zs7O3MbYxpaRYtghH7K1f+uZIew8p47IuFZKclcNbAHqTE+xl2WAInH2FJoynV11Q1EYgBTlXV/qp6705IGuu2NEG5z+vd8tVAVpX5Orlltajq06o6RFWHtGnTqP56Y0wzt3kzHHOsEggo7buV8+An84nxeTh3YG8uOrE177yYjNdjSaOp1Zc4jgQmqursqoUiMkJEuv3O7b0HnO2+Pht4t0r5We7oqmFAYZUmLWOMoaAAjjpaWbIE7vxHEa/N+5XNZQEuP7AX91+TQWmxcPxou3jhrlDfp/w3oHAb5UXAo9tbsYi8CkwBeonIKhE5H7gPOExEFgKHutMAHwFLgEXAM8AlDX4HxpgWr7wcDj5YmT4d7vp7MVMr5jN/XTF/PKAbrzzQjvlzvLz6qtC/f7Qj3TPU1zneVlVn1SxU1VkikrO9Favq2DqqDtnGvApcur11GmP2TP6YCMMPCXDmZQHWtVrG1z9u5ORBnZjzQRaTJsby0EMwenS0o9xz1Jc4Uuupi9/ZgRhjTE3Ll8P6jRFi2xYx7k9hpizO57X/rWS/bq05eVBHpm72cOmlcM010Y50z1Jf4pgmIheq6jNVC0XkAuCnpg3LGLOnW7QIRh2siFd5eWKYKUs38NTkxfRqm8y5Q7uREu/nigtj8Vi3xi5XX+K4CnhbRMaxNVEMwRlpdWJTB2aM2XPNnQuHHKpUVCh/e76YyQvX8e9vltKnQwpn9O3NWUem8+ij0G+MjaCKhjoTh6quA4aLyCign1v8oar+b5dEZozZI/38Mxx+uKKiPP5yEfMqVzH+m+UMyErlomG9uGJcKhVlwoD+ljSipSE3cpoETNoFsRhjDPc9EMEbozz2UhHTNq9kwrSV7NslnfP36cHNl7Ri2SIvEycKubnRjnTPZffjMMbsFlShNBDiir8WMXY9fLl6Oe/MWMP+3TM4d2g3rjq7Fb/O9DF+vHDoodGOds9m3UrGmKibOBH2Gx7hh3nFiDfCxOVLeWfGGg7OzeTig7qRlOjhiEM9vPmmMLaugf5ml7EjDmNMVL3zDowZo3TpEaG8IsLLPy/lf/PWc2S/dhzdNYeVS3wcc2ACDz3gjXaoxmWJwxgTFeEw3Hcf3H67krtXiAefKeI/Mxbx7aKNnDCgIwe2z+bS01sR4/VwynzrCN+dWOIwxkTFjTcqDz8sHHpsgOvuKObfUxcyddkmTh2SxZC0LC45PYWKUg8TJwo+21PtVuzPYYzZpcJhCEbCHHZKKbEZHg47oZxHv1jAjJUFnDmsM7mxnbh4bAqiwqRJwsCB0Y7Y1GSd48aYXSIYhJtvhmNGh5m5qpCUNkFGji7hgU/m8cvKAi44oAtH79We8U8l4PcKkydb0thd2RGHMabJrVgBY8cq330njB4TpLxcWbKpiH9MWkRheZCLD+rG/t3bEB/j5ZWXvBRuErp0iXbUpi6WOIwxTeqdd+C885RAEO54tJhDjq7knRmreXP6Ktomx3HncX0pXJLKNecm8OF7XtJbeUhvFe2oTX0scRhjmkxZGVx8idK2Y5g7Hy0mKbOC//t4EbPXFDG8W2vO378LM6fEc/MlyXTtCoEKGz3VHFjiMMbsdEuXQoeOypqSEh55IUSHThHmbSjgrrcWUxEIc9EBXRnZsw1vvBjPEw8k0K8vfPqpYHeDbh6sc9wYs9OEw/D009C/v3LVzRVsKA7QqUuI//6ygv/7eB4pcT7uPqEfo3Iz+dcjifz9nkSOORomTbKk0ZzYEYcxZqf49FO49jpl9ixh4L4hjjihgg3FlTw+aSEL1pVwcG4mZ+3XmRivlxifcO3lPob2g0suEcRaqJoVSxzGmB12883KffcJHbIi3P1YGaOOCjBt+Sb+9dZiIgqXH9ydoTkZjP9XPMsW+Hn/bS8xPg9794l25Ob3sMRhjPld1q0DEfAnBug3opLLbvJwypkViC/Ci1NW8MmctXTJSOSKg3vgD8Rz9blJTPsuhrFjFQ2L7X2aMfvTGWMapawMHnkE7r9fOWJ0kGvvKqZrb+iSq/y0YjOv/rCCNYUVHNWvHWP3zWb6d7HcdX0S5aXCs8/CeedZ01RzZ4nDGNMgkQj85z9wy63K6lXCyMMD/OG8MgCWbCjh5R9WMDeviA6pcdx4ZC4DslKpKId7b0yifVvh9deFvn2j/CbMTmGJwxjTILfdrtxzt9B7rzC3PFDKwH1D5JdU8sSklXyzaCMpcT7OG5HDqNxMNq/3oZEIfbIT+fwzD927Q0JCtN+B2VkscRhjtqmsDF57DXJ7K137VLLf0RXcnublsNEBKkNhXv9xNR/OygPguL07cPyADsT7fXz6XgyP3JHI1dcod9zupW3/KL8Rs9NZ4jDGVDN/Pjz1FLzwglJQIJx6TiVX/rmU9Ew4dHSIL+at583pqygqDzKiewanDsmiTXIsM3/y8di9Ccz9xc+QfZQzx9lpYi2VJQ5jzG/OOksZP17w+ZWDjghw4ukVDNgnhKoyY2UBL/+wgtUF5eS2S+aGI3rRrU0SAP9+LJ5/P5ZAhw7Kc8/BWWcJXrthX4tlicOYPVheHowfD5deEWZTWSXtuisXXSOM/kMFrdsoEVVmrirkg5lrmL2miHYpcVxzaE+G5KRRXOihcDNkdfAy7pQYslvDtdcKiYnRflemqVniMGYPU1kJkybBc8/B228roZCQ0rmEvYeEOHGcM09xRZAPZm7g81/Xsa6okpQ4H2ft15nDerdFwx5efz6OF56I58STlBee89K3A4w6ILrvy+w6ljiM2QNEIuDxwKJFMHCgUlIipLSK8IezKzlxbAWdciKoKos3lPLZ3LVMWZJPMKz0apvMHwZnsW+XdHweD19+EsOTDyawarmXww5TrrnK+jH2RJY4jGmBVGHuXHj/fXjvPaVf/wg33VXOZm+QY8fEMWR4kMH7BYmNhcpQmEnz8/ls7jqWbiwlzu9hZM9MDuvTluz0rWNon388nmceTaBvX+WZj+GII+xEvj2VJQ5jWpi77oLnnleWLXX26rn9wiS3rWRdUSUAV9zinLSXV1DO59PXMXnBBkoDYTqlxXPuiBz2755BQoyPYAA++yCGrM5hDhju5dpLYxnS2znz22d7jj2a/fmNaaby8+GHH+D772H2bOWlV8IUlAf4dYmXDjnCmPMCjBgVpE27CACqyprCCqYv38xPyzczf10xXo+wb5d0Du/dll7tkhER1q7xMP61WN6fEMemjR4uvEg59wSBTOjVPcpv2uwWLHEY0wwEg04fhdcLr74Kf7lNWbzIOaLweJRuvcJ8M7uIVmnK5X/eulwoEmHOmmKmL9/M9BUFrC2qAKBzegJjhmQxqlcbUhNifpv/vlsT+eCNWFTh2GPhkkvg8MOtPcpUZ4nDmN1MRYXTPzFvHvz0E/zwg/LTTzDhvQC5/QNsDEDHLrEcfmKQvnuHyN0rREKVIbAllSF+WVnA9BWb+WVlAaWBMD6P0LdDCkft1Y5B2WlkJMUCULhZ+O9/YzjljAAd0uIYMSSGvl2Fiy6CnJzovH+z+7PEYUwUqMKaNU5ymD/feT7xRGW//SNM/i7CkYf4AYiJVXr1DXHC6SHKpIKNJRGG7A9D9g/8tq5QOMLiDWXMyytm+orNzFtbREQhJc7HkJx0BmensVenVsT5nTPyQiH4eaqPD9+M5YsPY6msFI4+UMnuLtxwXVQ+DtPMWOIwpgmowsaNsGrV1kduLowaBWvylF49oaRkaxNQYpISn1lKXHYl3tZwz+MxZHcNk50Txh9Tdb3KxpIAi9aXsGh9MQvXl7Asv5RgWAHISotn9N4dGJSdRvc2SXg88ls8AKtXeLjgpFYUFnhISlLOO0+4+GLYay9rjjINZ4nDmN9h3jznRkYbNzpnX69aBT16wHnnKZWhCJmtPRQXV98ZnzKukpSuZVQEIhxzSgJZXcJkdw3TuWuYjEz9bWhrYhKMOtI5oqgIhlm4poSF60vcZFFCQXkQAL9X6JqRxOF92tE9M4kemUm0dpugANas9PDN/2L45gs/vfso9z0YoX/HGMae5uHgg+HII4Xk5F3zeZmWJSqJQ0SuBi4AFJgFnAu0B14DWgM/AWeqaqDOlRizA/LzYdMmKCqC4mLnOSYGjjzSqX/wQZgzx0kM+fnKho3Qby/lxf+ECUYiHH6En5Urtp785vMrRx4XoO+oElTh9AvjSUhSMttFaNMuTGa7CGmtlUDI6eS+8s9l1eIprQyRV1jO6oIK8grLWVNQzpqCCtYUlv92tNAuJY69Oraie2YS3TOTyG6dgM9T+wS8l5+J4+O3Y1mywPn3zu2t9O8ldEh16p98cud/nmbPsssTh4h0BK4A+qhquYhMAE4Djgb+pqqvichTwPmAfcVbqEAAysud9vZg0HkOBJwOWY8Hli2DlSudsspK5zkYhD/8wVn+889hxgzn0t/l5c6z1+vcmQ6ccxm++ALKy5XSMic5pKXBV1NCRCLKyX/wMXlS9Z1uz95hPhhYRiiivP1+PEuXeGiVprRKjdC1t9KpZ4i5ec6opGvu8OP1KimpSkZmhNR0xePZ2iR09iXltd5zKBJhXVHATQwVTnJwXxe6RxEAXhHatoqlQ6t4hnZJp0fbJLq1SSI5zl9tfaUlMGOWj7kzfCxZ6OP/HisjPTGG0g1x5HTycPnFMHo0dOtmzVBm54pWU5UPiBeRIJAA5AEHA6e79S8Cf2U7iWPtWrj88upl3bvDlVc6r++8EzZsqF7frx/88Y/O61tvdX5pVjV4MJxzjvP6uuucnVZVw4fD2LHO65rbBjj4YDjxRGdkzHXb6Gg8+mjnUVAAt9zilG3Z2QCcfDIceqjz3m6/fWvdluczz4QDD3R2rHfeWbv+T3+CYcOcUTl33+2UqzqXnFCFm25y3uPUqXDPPU551ceDD0L//s6O+c47tVpdOALPP6/k5sKEN+CuO8UpD299fPpZhM458NSTcPednt+SwpbHvEUhMtsp99zp5aH7a18+dfGaCuITlHse8PPsk7W/ngvXlQDKMy/GMuE/TuO/P0aJi4NWaRHGXVmEqrJ0XRyFZX7i4pWMVCU7UcnIVOasdn7pH3+Gn5HHCAmJSmKSkpispLRSNpY45zw8/Hyw1rarGnpA9fpAKMLmkgD5pQE2lQbYVFK59bX7KCwPUuVPTVKsjw6pcQzMSqV9ajwdUuPo2CqeNimxtY4kwuGtlw3538cxPP94PEsWeFF1kkKPnkp2YiwZGfDcs9gZ3aZJ7fLEoaqrReQhYAVQDnyK0zRVoKohd7ZVQMdtLS8iFwEXAcTE7sX4/0Sq1Q8YEuagk0oBmPBWEquWV/8H3G9kiGFHOzuP195IJn9D9f+wNflBBh7i/Fp85bUUykqrb39zeYA++zu/Osf/J6VWfJWeAF2HVFBaAi+/Urvel1JJx70q2bheeG3C1gbmLf/oaVkVZPQKsHKZh7feTnIrt87TuW85yTkBFsz38tEnSW65/lbff/9y/O2DzF7g5ZspiYhQ7fHjglIirUP8vMTHrwsT8HicnZF4nF/MPy8rpSwpzK95PkoDCXi8injA4wO/B+auLaU4LsKGSj/tO8fi8fLbOrxeZWF+GZu9ii/dx6ijY/H5FK+P355XFFWQr0qPwT6uuMWHzwden7PtmFhlVUEAfxkcfIKHvYZ58fmVmBh+e15fFEYELro+wEXXQ2wc1S7fHXC/QRdeXY7z9dq2YSPrTgyqTj9FcUWI4oogxRUhSiq3vi6u+roiREFZgKKKUK31JMR4SU+MIT0xhs6tE0hPjKF1YiztU+PokBpPSo0jiC2CAViyzMuKpV7mzfYy9xcfv8708+KEcg4Y4SGvk58eXTyccZowbBjsuy+kp2/9HlvSME1NtOrP3V2xQZE04L/AqUAB8AbwJvBXVe3uzpMFfKyq/epbV9/+A/XZtz9v4ojN7kxVCYaVQChCZShMIBwhEHIf4QiVoQjBUITyYJiKYJiygPNcHgxTHghTHoxQHgxREYxQHghTFgxTHgj9NkqpJgESY32kxPlIjvOTFOcjNd7vJIWkGNITY51kkRBDfEzdN6RQhc2bhBVLvCxf7CSJ/Q8JMHR4mF9nxHDmCc6PAp9P6d8fhg0TLr0U+vRpik/R7GlE5CdVHfJ7l49GU9WhwFJV3QAgIm8BI4BUEfG5Rx2dgNXbW1Eoomwq3X37zxualGvOVX0x/a1MtzGPsrVCt8ytTvmWZVTVfXbmi2iVdbp1EdXfpiNaZRpnWt2yLeWRLfNF1Jknsu36UESJRJSwOs+hiDNPOLL1sWU6tOURjhAKO6+D4cg2yiO/JYtAuPoRZ0P4vUK830t8jJc4v5d4v5e0BD/tW8UR7/eSEOMlKc5PcpyP5DgfKXF+kmKd14kxvt+GuNZHFYqLhPVrPWzI87B+rYesnDADh4bYnC+MPTyVosKtR8NxccqwvWMZ3NlDz3T4z3+c4bu9e4vdq9vsdqKROFYAw0QkAact4RBgGjAJOAVnZNXZwLvbW9GmTfm8/+Yr1cqWhdOZF87ES5jDYhbWWmZROINF4QxiCTIqZnGt+vnhTJaG00mUAAf4l9SqnxNqx8pIKilSwXD/slr1v4Q6kBdJIV3K2Ne/olb99FAn1keSyPSUMMi3qlb91GA2mzSB9p4i9vatqVX/XTCHIo0jy1NAX9/aWvVfB7tSqjF08W6il3d9rfpJgW5U4qe7dyPdvRtr1X8W6EEYL7ne9eR4N9WqnxjIBaCfby2dPAXV6sJ4+CzQE4C9fWto76negVSJj69CPfCKMNC3kgwpIUZkS0sclRLL3NhcfB6hZ3AxSZFSRMADiAhBXyIbM/vh83hos3kW/lAZHgFB8Ah4EtOI6zKIGJ+HioVT0EC5Uy9OfauMduQOGo7P62HG5I8IVlZUiy89I4tu/fcB4Kf/vUckFEKBQvfRplMOyX0GAfDNe28TDEEoKISCEAgI/oSeHH1abzwa4pn7PiYcrv7ZFeftzXl/6E8ws4LLLn2J+HhISHAesbGwzz5DgH5EIoUEg28zaxbMmrV1+f32249evXqxceNGPvjgg1p/mwMPPJCuXbuydu1aJk6cWKv+kEMOISsri5UrV/LFF1/Uqj/yyCNp164dS5Ys4auvvqpVf+yxx5KRkcH8+fOZMmVKrfoTTzyRVq1aMXv2bKZNm1arfsyYMSQkJDBjxgxmzJhRq37cuHH4/X5+/PFH5syZU6v+HLfz8bvvvmPBggXV6vx+P+PGOTcTmTx5MkuXLq1Wn5CQwJgxYwD4/PPPWbWq+v9eSkoKJ510EgATJ05k7drq/1utW7dm9OjRALz//vvk5+dXq2/Xrh1HukPy3nrrLYpqdJ526tSJQw89FIAJEyZQVlZ9VF2XLl0YOXIkAC+//DLBYPWm1J49ezJ8+HAAXnjhBWrq27cv++yzD8FgkJdffrlW/YABAxgwYABlZWVMmDChVn1jRaOP4wcReROYDoSAn4GngQ+B10Tkbrfs39tbV4zXQ/tW8dXKOqS3Zv/MLmg4RGRh7R1zh4w2HJSRgwYriSyuvWPukJnJIelZaKCMyJK8WvUd27VFUjugFcVElq2rXd+hHZLSFi0rILJiQ6364zp1QJIy0JKNRFZV+fK5e8+TszsiCWlo4TrCeZtrLT82JwtPfAqRgtWE8gpr1Z/dvTPeuERCG1cQXFdSc/Vc0qcbXn88let8VKwr+61c3Ibxm/bOxevzU7LKQ9n6Sqr+thaE+4bthUeETUs8lKwPgWxZt+Dx+ThpxCA8Iqz+FQo3KMLWS2/HxMZx0cihACz8OUTBhur/nHEJSZy7/94AzJtWSPGm6sdiCSkp9B3mXGVvzvfLKCuqvmdOTk8gt0cbAGatjqGC6kejsT4PPq/zKz8YFCor5LdO/UhEKKv00K2/M2/eSg8V5V7CIfmtY79iWhzXD/bh93pYPN9Xa+BEfEIMQ7ukEw6HyMpy+l5iYpykEBsLQ4YIcX4vkaDQo0etP50xzcYu7+PYmayPo/lQhXAIvD6n87aiHMrKhHAIQiH3OShkdw3j8cCq5R7WrfEQDAjBoLOjD4Xg8NFOMvj+Kz8L5nqprJDfHh6vcvVfnF9yz/0jnh+/8xOoECoqoKzUQ2pahDc/KcXrES44LYHvvqr+u6lX7wjfTA3iFeHk43wsWCBkZEBGa+d54EC48UZn3okTnQEBrVtD+/bQti12j23TbOxoH0ezThxt2g7WQcO+q1aW0z3MOe4Y+n/8XwL5G2qM1e8T4vQLnCaKh/+aSHFR9fbqvQYGOflM56fk/92SSKCyev2goUFGj3Hq77wuqVZMww4McPhxASor4L5ba9ePPDzAQUcEKCoUHrmj9s2ZDxtdyYhRQTasEx6/L7HWcNvjT61gyPAQq5Z7ePLBBGr++cacU8GAfUIsnu/lXw8nbO3ncIfjnn9FOX0HhJg13cczj8YTiQgawenHiMA1t5eS2zfMlMl+/vVIwtahvO489z1RQtceESa+G8M/H4wnHBEiYac+HIHx75SQlRPh5X/H8Nj98c4v+pDzix5g8oxiMtsq/3ggln/+LZaa5iwrJTEJ7v5LDM8+WXvU0drCCrwe4ZrLfYx/wdlTx8UpCQnOTnzW3AheEe68Q/j6ayE+HuLjITkZOnSAe+911jNxonNyX0qKU5eS4pzn0bVrrU0a0+I0x87xnSZYKSyaE1OtLC1BGdTZ2SGtXeph+fLqy3Rq42dQ5zgAVi/ysL5Ga1KvHD+DOjvNX0vneiit3hTJ/7d39zFyVWUcx7+/bnezbd22KqEhRbavYnlpy7KAIaEQUwykRBSrRRHUNMaaaAiJUYgkRv6RhIQ/SIDSoizG8CZaQBLEvyzG1EAtbdfSF2uJtQRtFKxdEKLs4x/3Djvd7Xbn7MzOnbv7+yST7pxz7pmn+/T22XNn9twVy4b69+0a+Vu7H+9pp6d7JgMDcKB/ZP+Vq7L+o0fh4J6h/srlnM+saaenO3h1EF7dN+2EPgk+1NbOhd0wYwD+dlgj+ufPaqd3AegfcPyN7DJR27R8jGDpae1cshD++xp0kH3Mtq1N73+kduVZc1mxCP5zGBZ3D33UtvLoWTiXxd3w7/PhwCeGtvpuyz+W27t0NvPmwRuXw3vHob0dpk8f+vOCRV10dcFN6+DcJVlb5dHRAUvOmEVHB3znFvjSuqytcrmnowNO7+pEgo33wgP3QWfn0GW2TFZM7rhjxLf+BJXfEDezdKVecfT29sbJ3oQzM7PR1bvi8J3mzcwsiQuHmZklceEwM7MkLhxmZpbEhcPMzJK4cJiZWRIXDjMzS+LCYWZmSUr9C4CSjgP7i44DmEO2gWqRc6UcV8vYscaM1p/Sfhowcove5mpk7uqZr5n5S+1r1dxB+fLXKufe2RHRdZKxtYmI0j6A7UXHkMexqei5Uo6rZexYY0brT2lvhfw1MndlyV9qX6vmroz5myznni9VNcYvW2CulONqGTvWmNH6U9uL1ui4ypC/1L5WzR2UL3+T4twr+6Wq7VHHfitWLOevvJy7cqs3f2VfcWwqOgCri/NXXs5dudWVv1KvOMzMrPnKvuIwM7Mmc+EwM7MkLhxmZpZk0hYOSZ+WtFnS45I+WXQ8lkbSIkk/kvRk0bHY2CTNkvRwfs7dUHQ8lib1fGvJwiHpx5KOSvrjsParJO2XdFDSraeaIyKeioivARuAdRMZr52oQfk7FBHrJzZSO5XEPF4HPJmfc59qerA2Qkr+Us+3liwcQB9wVXWDpDbgXuBq4BzgC5LOkXS+pGeHPU6vOvT2/Dhrnj4alz8rTh815hE4E/hrPuy9JsZoo+uj9vwlmd6I6BotIl6QtGBY88XAwYg4BCDpMeDaiPghcM3wOSQJuBN4LiJ2TGzEVq0R+bPipeQROEJWPHbSuj+QTimJ+XslZe4yJXg+Qz/RQPYPdf4pxn8LWA2slbRhIgOzmiTlT9KHJW0ELpB020QHZzUbLY+/AD4r6X5ae4uSqe6k+Us931pyxdEIEXEPcE/Rcdj4RMQ/yd6fshKIiLeArxYdh41P6vlWphXHa8BHqp6fmbdZOTh/k4PzWG4NyV+ZCsdLwFJJCyV1ANcDzxQck9XOcT8k+AAAAslJREFU+ZscnMdya0j+WrJwSHoU2AacLemIpPUR8T/gm8DzwF7giYjYU2ScdnLO3+TgPJbbRObPmxyamVmSllxxmJlZ63LhMDOzJC4cZmaWxIXDzMySuHCYmVkSFw4zM0viwmFTgqSBBs+3YPh21XXM1SdpbQ3jfiOptxGvaVYPFw4zM0viwmFTlqSVkn4vabekLZI+mLdflLftlHTXWCuLfPXxW0k78selefsVkrZKelrSIUl3SrpB0ouS+iUtrppmtaTtkg5IuiY/foakxyTtlbQFmFH1mvfn4/dI+kHjvztmo3PhsKnsJ8B3I2I50A98P29/CPh6RKyktpsSHQWujIgesrtNVu/KvIJs19FlwI3ARyPiYuBBsq3/KxaQ3SthDbBRUifwDeDtiFiWx3Zh1fjvRUQvsBy4XNLymv/WZnVy4bApSdIcYG5EbM2bHgZWSZoLdEXEtrz9kRqmawc2S+oHfkZ2Z7WKlyLi9Yh4F/gz8Ou8vZ+sWFQ8ERGDEfEn4BDwMWAV8FOAiNgN7K4a/3lJO4CXgXOHvabZhJq09+Mwa6JbgL+TrS6mAe9U9b1b9fVg1fNBTjz/hm8aN+omcpIWAt8GLoqINyX1AZ3jitxsHLzisCkpIo4Bb0q6LG+6EdgaEf8Cjku6JG+/vobp5gCvR8RgPk/bOEL6nKRp+fsei4D9wAvAFwEknUd2WQpgNvAWcEzSPLL7R5s1jVccNlXMlHSk6vndwJfJ3k+YSXZ5qHIHu/Vkl54Gga3AsTHmvg/4uaSbgF+R/aee6jDwIllR2BAR7+S3YX1I0l6yLbD/ABARuyS9DOwjuw3o78bxembj5m3VzYaR9IGIGMi/vhU4IyJuLjgss5bhFYfZSGsk3UZ2fvwF+Eqx4Zi1Fq84zMwsid8cNzOzJC4cZmaWxIXDzMySuHCYmVkSFw4zM0viwmFmZkn+D+5EXPvuF/nrAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "tags": [], + "needs_background": "light" + } + }, + { + "output_type": "stream", + "text": [ + "best score: 78.32578208939992\n", + "lambda: 0.020433597178569417\n", + "(Standardized)\n", + "Intercept 37.980193\n", + " Coefficient\n", + "X1 transaction date 1.417179\n", + "X2 house age -3.024698\n", + "X3 distance to the nearest MRT station -5.430362\n", + "X4 number of convenience stores 3.333001\n", + "X5 latitude 2.811782\n", + "X6 longitude -0.003293\n", + "\n", + "(Real)\n", + "Intercept -15725.668991\n", + " Coefficient\n", + "X1 transaction date 5.032121\n", + "X2 house age -0.265821\n", + "X3 distance to the nearest MRT station -0.004308\n", + "X4 number of convenience stores 1.132902\n", + "X5 latitude 226.844458\n", + "X6 longitude -0.214813\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "Kd0WjA_kLSQW", + "colab_type": "code", + "outputId": "4fc30baf-8999-4168-b623-bee31aa34ec7", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 68 + } + }, + "source": [ + "y_pred_lasso = lasso.predict(X_std) # predictions\n", + "r2_lasso = metrics.r2_score(y, y_pred_lasso) # R-squared\n", + "adj_r2_lasso = 1 - (1 - r2_lasso) * (X_.shape[0] - 1) / (X_.shape[0] - count_lasso - 1) # adjusted R-squared\n", + "print('Lasso adj. R2:', adj_r2_lasso)\n", + "\n", + "y_pred_ridge = ridge.predict(X_std) # predictions\n", + "r2_ridge = metrics.r2_score(y, y_pred_ridge) # R-squared\n", + "adj_r2_ridge = 1 - (1 - r2_ridge) * (X_.shape[0] - 1) / (X_.shape[0] - count_ridge - 1) # adjusted R-squared\n", + "print('Ridge adj. R2:', adj_r2_ridge)\n", + "\n", + "y_pred_elastic = elastic.predict(X_std) # predictions\n", + "r2_elastic = metrics.r2_score(y, y_pred_elastic) # R-squared\n", + "adj_r2_elastic = 1 - (1 - r2_elastic) * (X_.shape[0] - 1) / (X_.shape[0] - count_elastic - 1) # adjusted R-squared\n", + "print('Elastic Net adj. R2:', adj_r2_elastic)" + ], + "execution_count": 95, + "outputs": [ + { + "output_type": "stream", + "text": [ + "Lasso adj. R2: 0.5771295619995425\n", + "Ridge adj. R2: 0.576190574926105\n", + "Elastic Net adj. R2: 0.5761109296860583\n" + ], + "name": "stdout" + } + ] + } + ] +} \ No newline at end of file diff --git a/LinearRegression/linear_regression_blog.md b/LinearRegression/linear_regression_blog.md new file mode 100644 index 0000000..310f20f --- /dev/null +++ b/LinearRegression/linear_regression_blog.md @@ -0,0 +1,692 @@ +# Intro to Multiple Linear Regression Analysis in Python + +**Linear Regression models** allow us to summarize and study relationships between variables +by fitting a straight line to the observed data. You may have heard of **simple linear regression**, +which establishes the relationship between two quantitative variables–one predictor variable and +one target variable. + +However, in reality, it is rare that a target variable is explained by only one predictor. So +what if we want to estimate a continuous variable using more than one predictors? If two or +more predictor variables have a linear relationship with the target variable, a **multiple linear +regression analysis** allows us to quantify the strength of the relationship and estimate the target +variable at a certain value of the predictor variables. + +This tutorial will show you step-by-step how to perform a multiple linear regression analysis +using Python, from checking assumptions to evaluating performance. + + + +## Getting Started + +### Import Libraries + +First, let’s import all necessary libraries in the beginning to keep our code organized: + +```python +import pandas as pd +import numpy as np +from scipy import stats +import statsmodels.api as sm +from statsmodels.stats.outliers_influence import variance_inflation_factor +from sklearn import datasets +from sklearn import linear_model +from sklearn import metrics +from sklearn.preprocessing import StandardScaler +from sklearn.model_selection import GridSearchCV +import matplotlib.pyplot as plt +import seaborn as sn +from random import uniform +from google.colab import files +import io +``` + +### Read Dataset + +For this analysis, I will use a real estate dataset that can be downloaded from [here](https://www.kaggle.com/quantbruce/real-estate-price-prediction). I choose to +upload it to Google Colab from local drive. Of course, you can also replace the first argument in +read_csv() with the filepath or URL to the dataset that you’d like to use instead. The delimiter +parameter is set to comma by default, but you can change it depending on the delimiter of your +csv file. + +```python +uploaded = files.upload() +dataset = pd.read_csv(io.BytesIO(uploaded['Real estate.csv']), delimiter=',') +``` + +We can take a look at what the data looks like: + +```python +dataset.head() +``` + +![](Blog/4.png) + +As you can see, the first column in the real estate data set is redundant, so we should probably +remove it. Otherwise, this data set seems directly usable. However, it’s likely that your own data +set will require more preprocessing–dealing with missing/invalid values, restructuring the data, and +encoding dummy variables, etc. + +```python +dataset.drop(['No'], axis=1) +``` + +After data preprocessing, we can use the describe() function to obtain some summary statistics +of our target variable, and the displot() function from the seaborn library to plot its distribution. + +```python +y = dataset['Y house price of unit area'].values # target vector +print(dataset['Y house price of unit area'].describe()) +sn.distplot(y, bins = 13) +``` + + + +We see that the values of house price of unit area are normally distributed with perhaps a few +outliers on the right. + + + +## Checking Multicollinearity + +Since a multiple regression involves more than one predictor variables, multicollinearity occurs when some of them are highly correlated with each other, which means each of these predictors will account for similar variance in the target variable. Though the presence of multicollinearity will not affect the predictive power of our model, it will make it more difficult for us to assess the individual influence of a predictor on our target variable. Therefore, it is important to check the ”no multicollinearity” condition before our analysis. We can detect multicollinearity using either a correlation matrix or VIF factors. + +### Correlation Matrix + +The correlation coefficient is a statistic that measures linear correlation between two variables. Its +value lies between -1 and +1. The absolute value gives the strength of the relationship between the +two variables. The larger its magnitude, the stronger their relationship. In general, a correlation +coefficient above +0.8 or below -0.8 indicates a strong correlation, and we should take some measures +to reduce multicollinearity before we continue. + +The pandas corr() function calculates the correlations among all variables in the dataframe. +We can visualize the resulting correlation matrix using a heatmap from the seaborn library. The +diagonal values should always equal 1 since they are the correlations of a variable with itself. + +```python +df = pd.DataFrame(dataset,columns=['X1 transaction date', 'X2 house age', 'X3 distance to the nearest MRT station', 'X4 number of convenience stores', 'X5 latitude', 'X6 longitude']) # regressor matrix +corrMatrix = df.corr() +sn.heatmap(corrMatrix, annot=True) # visualize the correlation matrix using a heatmap +``` + + + +### VIF Factors + +VIF factor is an alternative measure for multicollinearity, which ranges from 1 upwards. A rule of +thumb used in practice is if a VIF value is greater than 4, then multicollinearity is a problem. We +can check VIF factors using the variance_inflation_factor() function from the statsmodels package. + +```python +X = dataset[['X1 transaction date', 'X2 house age', 'X3 distance to the nearest MRT station', 'X4 number of convenience stores', 'X5 latitude', 'X6 longitude']] +X2 = sm.add_constant(X) # augmented regressor matrix +vif = pd.DataFrame() +vif['VIF Factor'] = [variance_inflation_factor(X2.values, i) for i in range(X2.shape[1])] +vif['Predictor'] = X2.columns +vif = vif.drop(0) +vif +``` + +![](Blog/11.png) + +You should see in the correlation matrix that the correlation coefficient between X3 and X6 is +−0.81 < −0.8, and the VIF factor for X3 is also slightly higher than 4, which is unfavorable. + +A simple solution is to remove one of the correlated variables from our model. After removing X3 +from the list of column names and rerunning the previous code, the new correlation matrix should +show no significant multicollinearity, and the VIF factors of all the remaining variables are around +1, so we are in good shape, and can proceed with our regression. + + + +![](Blog/13.png) + +But wait, you may ask, why did I choose to delete X3 rather than X6? Indeed, my choice was +kind of arbitrary here. To remove the redundant variables more carefully, we can consider using +stepwise regression to only keep the predictors that lead to the best performance; I will describe +backwards stepwise regression later. + +Another way to deal with multicollinearity without the need to drop predictors before the analysis is to perform regression with regularization techniques, such as Lasso and Ridge. Regularization can help you handle multicollinearity, so if you don’t want to delete any variables, you may choose to skip the OLS model below and directly jump to the Lasso/Ridge/Elastic Net regression presented in the end. + + + +## Performing the Regression + +### A Bit of Math + +Now it’s finally time to introduce our regression model. Under the linear model, the actual value +of our target variable should be a linear combination of all predictor variables plus a constant +(intercept) as well as some errors. Thus it can be written in the form of an equation $y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \dots + \epsilon_i$, and our goal is to estimate $\beta_0$, $\beta_1$, $\beta_2$, ... + +Assuming you’re not scared of matrices, let's first understand some mathematical background for linear regression in linear algebra. More generally speaking, suppose our data consisits of $n$ observations, each of which contains the values of $k$ predictor variables (not including the intercept) as well as the corresponding value of the target variable. In matrix notation, we can write the regressor matrix $\text{X}$ as $\begin{bmatrix} +1 & X_{11} & \dots & X_{1k} \\ +1 & X_{21} & \dots & X_{2k} \\ +\vdots & \vdots & \ddots & \vdots \\ +1 & X_{n1} & \dots & X_{nk} \\ +\end{bmatrix}$, the target vector $\mathbf{y}$ as $\begin{bmatrix} +y_1 \\ +y_2 \\ +\vdots \\ +y_n \\ +\end{bmatrix}$, the coefficient vector $\mathbf{\beta}$ as $\begin{bmatrix} +\beta_0 \\ +\beta_1 \\ +\vdots \\ +\beta_k \\ +\end{bmatrix}$, and the residual vector $\mathbf{\epsilon}$ as $\begin{bmatrix} +\epsilon_1 \\ +\epsilon_2 \\ +\vdots \\ +\epsilon_n \\ +\end{bmatrix}$. Then, our linear model can be rewritten as the following: $\begin{bmatrix} +y_1 \\ +y_2 \\ +\vdots \\ +y_n \\ +\end{bmatrix} = +\begin{bmatrix} +1 & X_{11} & \dots & X_{1k} \\ +1 & X_{21} & \dots & X_{2k} \\ +\vdots & \vdots & \ddots & \vdots \\ +1 & X_{n1} & \dots & X_{nk} \\ +\end{bmatrix} +\begin{bmatrix} +\beta_0 \\ +\beta_1 \\ +\vdots \\ +\beta_k \\ +\end{bmatrix} + +\begin{bmatrix} +\epsilon_1 \\ +\epsilon_2 \\ +\vdots \\ +\epsilon_n \\ +\end{bmatrix}$. Or, more simply, as $\mathbf{y} = \text{X} \mathbf{\beta} + \mathbf{\epsilon}$, where for each $i \in [1, n]$, $y_i = \beta_0 + \beta_1 X_{i1} + \beta_2 X_{i2} + \dots + \beta_k X_{ik} + \epsilon_i$. + +The simplest model that we're going to start with is the ordinary least squares (OLS) regression, which aims to minimize the sum of the squared residuals $||\epsilon||^2$, which is equivalent to $|\mathbf{y} - \text{X} \mathbf{\beta}||^2$. It can be shown via matrix calculus that the least squares estimator for our coefficient vector $\mathbf{\beta}$ is given by $\mathbf{\hat{\beta}} = (\text{X}^\intercal \text{X})^{-1} \text{X}^\intercal \mathbf{y}$. + +### Getting the Coefficients + +The above calculation has already been implemented in the scikit-learn library so we don't need to calculate $\mathbf{\hat{\beta}}$ manually. After instantiating the LinearRegression class, we can easily obtain the coefficients along with the intercept by calling the fit() method along with our data. + +```python +regressor = linear_model.LinearRegression() +regressor.fit(X, y) +coeff_df = pd.DataFrame(regressor.coef_, X.columns, columns=['Coefficient']) +# print the calculated coefficients as well as the intercept +print('Intercept %.6f' %(regressor.intercept_)) +print(coeff_df) +``` + + + +This means, for example, that for every one unit of change in X1 transaction date, the change +in the house price of unit area is approximately 4.21. + +We can compare and visualize some predicted values of the house price with the corresponding +actual values in a bar graph: + +```python +y_hat = regressor.predict(X) # predicted values +df_comparison = pd.DataFrame({'Actual': y, 'Predicted': y_hat}) # put the actual and predicted values together +df_comparison = df_comparison.head(50) # show top 50 rows of this data frame +print(df_comparison) +df_comparison.plot(kind = 'bar') # visualize the comparison +plt.grid(which = 'major', linestyle = '-', linewidth = '0.5', color = 'green') +plt.grid(which = 'minor', linestyle = ':', linewidth = '0.5', color = 'black') +``` + + + +![](Blog/18.png) + + + +## Checking the Residuals + +Congrats! You have successfully obtained the regression line. Now we can get the residuals of the regression by calculating the difference between each data point and the corresponding predicted value. In order to ensure the validity of our analysis, we need to use the residuals to verify a few more conditions apart from multicollinearity. + +First, we can use the scatter() function from the matplotlib library to plot the residuals (between the actual and predicted values) against the predicted values. We expect to see that the points are randomly distributed in the scatter plot. More specifically, they should show no curvature (which ensures linearity), and constant variation along the predicted value (which ensures homoscedasticity). + +```python +epsilon_hat = y - y_hat # residuals +plt.grid(color = 'b', linestyle= '--' , linewidth = 0.5) +plt.scatter(y_hat, epsilon_hat) # scatter plot +``` + +![](Blog/20.png) + +Finally, let’s generate a normal probability plot using probplot() to test the normality of the +residuals. If the residuals are normally distributed, the points in the normal probability plot should +stick to the red diagonal line as closely as possible. As shown in the plot below, the residuals of +our real estate model show a slight deviation from the ideal line, but the deviation or bend is not +very serious. + +```python +pp = sm.ProbPlot(epsilon_hat, fit=True) +pp.ppplot(line='45') # normal probability plot +``` + +![](Blog/22.png) + +The above procedure indicates that our residuals satisfy the regression conditions. If, however, +your residuals do show some strong patterns, you should consider transforming your data using +negative reciprocal, logarithm, square root, square, or exponentiation and then redo the regression. +If none of the above transformation options can solve the problem, it’s possible that your data set +does not fit the linear model, and you may resort to a nonlinear model. + + + +## Making Predictions + +After all the previous conditions have been verified, you now have a trustworthy regression equation +which enables you to make some predictions if given an extra piece of data. While we can use +predict() in scikit-learn to get the mean of our prediction, confidence intervals are available via the get_prediction() in the statsmodels package. However, unlike the scikit-learn library, statsmodels does not add a column of ones to the regressor matrix by default, so before calling get_prediction() we need to add the ones using the add_constant() function. + +```python +random = [] +# generate random inputs +for i in range(X.shape[1]): + random.append(uniform(X.iloc[:, i].min(), X.iloc[:, i].max())) +# print the random inputs generated +print('random input:', random) +random.insert(0, 1) # insert 1 at the front +X_aug = sm.add_constant(X) # insert 1 at the front +est = sm.OLS(y, X_aug).fit() +prediction = est.get_prediction(np.reshape(random, (1, 6))) +pd.set_option('display.max_columns', None) +print(prediction.summary_frame(alpha = 0.05)) +``` + +![](Blog/24.png) + +Note that apart from the mean and standard error, summary_frame() spits out two intervals at +95% confidence level (we have set alpha = 0.05). The first pair of lower bound and upper bound +forms a confidence interval, which is an estimate of the mean price of all the houses that match our +randomly generated data. The second pair, on the other hand, forms a prediction interval, which +estimate the price of a particular house with this set of data. Because individual values vary more +than means, we can expect the prediction interval to be wider than the confidence interval at the +same confidence level. + + + +## Evaluations + +While it is relatively straightforward to obtain regression lines and predictions, there are numerous +evaluation metrics to measure the accuracy of our model. In fact, the summary() function from +statsmodels provides you with an overwhelming amount of data: + +```python +print(est.summary()) +``` + +![](Blog/25+.png) + +Don’t panic, let’s interpret some important values one by one. + +### R-squared and Adjusted R-squared + + + +R-squared, a.k.a. coefficient of determination, measures the proportion of the variance of the target variable that can be explained by the predictors in our model. $R^2$ ranges from 0 to 100%. The higher the value, the better the model. Our model has $R^2 = 0.542$, which means approximately 54% of the variation in the house price can be accounted for by our OLS model. + +Nevertheless, $R^2$ only works as intended in a simple linear regression model with only one predictor. In a multiple regression, when a new predictor is added to the model, $R^2$ can only increase but never decrease. This implies that a model may have a higher $R^2$ simply because it has more predictors. On the other hand, the adjusted $R^2$ takes into account the number of predictors in the model. As a result, it only increases if the new predictor improves the model more than expected by chance and decreases if it fails to do so. + +### F-statistic + + + +Below $R^2$ and the adjusted $R^2$ is the F-statistic which is the statistic of an F-test checking whether our model fits the data well. The null hypothesis of the F-test is that the model with no predictors but only an intercept can fit the data as well as the proposed one. To put it simply, it's testing whether $R^2$ is significantly greater than zero. The F-statistic is the ratio of the explained variance to the unexplained variance, so a higher value of F-statistic is evidence against the null hypothesis. + +The fourth row on the right gives the p-value corresponding to the F-statistic. Our model yields a very small p-value of 5.12e-67, so there is sufficient evidence to reject the null hypothesis and conclude that our model does have some predictive power. + +### Confidence Intervals and t-tests + + + +Perhaps the most important information is located in the central part of the summary--the estimated values of each coefficient in the first column, followed by an estimate of the standard +deviation of the coefficient. The last two columns provide 95% confidence intervals constructed +from the coefficients and their standard errors in the first two columns. + +The two columns in the middle are the results of t-tests checking for the relevancy of each +coefficient. The null hypothesis of each t-test is that the corresponding coefficient equals zero, +meaning no linear relationship between this predictor and the target variable. The t-statistic is +the value in the first column (mean) divided by the second column (standard error); the p-values +corresponding to each t-statistic are presented in the next column. + +In our case, you can see that the p-values of all the coefficients except X1 are smaller than .01, so we can safely reject the null hypotheses and say that they are indeed relevant to predicting our target variable. However, X1 may be a redundant variable at .01 significance level. + + + +## Stepwise Regression + +The results of the OLS regression seem significant but the real estate data set involves only 6 predictors. When we have a substantial number of predictors in the model, it's unlikely that all of them will have significant p-values. Oftentimes, there exist predictors that give redundant information. As the model increases in complexity, the variance of our estimator can get very large. Thus we may consider getting rid of some of the predictors in order to simplify the model and prevent overfitting. + +In theory, we could try all possible combinations of our predictors and choose the best combination (for example, the one that yields the highest adjusted $R^2$). However, imagine your data set contains have 20 predictors, then you would need to compare $2^{20} > 1,000,000$ potential models! + +Backwards stepwise regression provides an easier procedure to select reliable predictors. Starting with all predictors in the model, in each iteration we find out which predictor to delete, if any, will lead to the best model. We repeat this step until we can't improve our model by removing any variables. + +The question then arises: how do we determine quantitatively which model is the best? One method to measure the performance of models is the leave-one-out cross-validation--in each iteration, we pick a data point (a row of $X_{i\cdot}$ and $y_i$) from our data set, and refit an OLS model on the remaining data points. Based on this model, we make a prediction $\hat{y}_{i}$ for the data point $y_i$ that we picked in the current iteration. We sum up the squared differences between each data point and its predicted value, i.e. $\sum_{i=1}^{n} (y_i - \hat{y}_i)^2$ as our evaluation metric, called the risk estimator. A good model should yield small difference between each pair of predicted and actual values, so smaller risk estimators are preferred. + +Using the leave-one-out cross-validation, we are able to quantitatively compare the models during each iteration of backwards stepwise regression. This procedure is not supported in Python libraries, but the process is not hard to implement: + +```python +X_ = dataset[['X1 transaction date', 'X2 house age', 'X3 distance to the nearest MRT station', 'X4 number of convenience stores', 'X5 latitude', 'X6 longitude']] +X_aug_ = sm.add_constant(X_) # augmented regressor matrix with a column of ones inserted at the front +min_r_cv = 0 +# calculate the initial risk estimator with all predictors included +for i in range(X_.shape[0]): + X_cv = X_aug_.drop(X_aug_.index[i]) + X_cv.reset_index(inplace=True) + X_cv = X_cv.drop('index', axis=1) + y_cv = np.delete(y, i) + est_cv = sm.OLS(y_cv, X_cv).fit() + prediction_cv = est_cv.get_prediction(X_aug_.iloc[[i]]).predicted_mean + min_r_cv += (y[i] - prediction_cv)**2 +print('original risk estimator:', min_r_cv) + +variables = X_.shape[1] +min_index = -1 +# keep deleting a predictor until we can't improve the model anymore +while (min_index != 0): + min_index = 0 + # calculate the risk estimator when the j-th predictor is removed + for j in range(1, variables + 1): + r_cv = 0 + for i in range(X_.shape[0]): + X_cv = X_aug_.drop(X_aug_.index[i]) # remove the i-th data point + X_cv.reset_index(inplace = True) + X_cv = X_cv.drop('index', axis = 1) + X_cv.drop(X_cv.columns[[j]], axis=1, inplace=True) # remove the j-th column + y_cv = np.delete(y, i) # remove the i-th data point + # refit the model and make the prediction + est_cv = sm.OLS(y_cv, X_cv).fit() + X_aug_copy = X_aug_.iloc[[i]].copy() + X_aug_copy.drop(X_aug_copy.columns[[j]], axis=1, inplace=True) + prediction_cv = est_cv.get_prediction(X_aug_copy).predicted_mean + # the risk estimator is calculated as the sum of the squared differences between each removed point and the prediction for this point + r_cv += (y[i] - prediction_cv)**2 + # update the optimal predictor to drop if the risk estimator becomes lower + if (r_cv < min_r_cv): + min_index = j + min_r_cv = r_cv + # update the regressor matrix and the number of variables if deleting a predictor leads to a better model + if (min_index != 0): + X_aug_.drop(X_aug_.columns[[min_index]], axis=1, inplace=True) + variables -= 1 +print('improved risk estimator:', min_r_cv) +print('remaining predictors:', X_aug_.columns.values) +``` + +We can apply the above function to our real estate data, though it is too small in size to let the +above algorithm run for many rounds and make significant improvement. Still, backwards stepwise +regression reveals that X6 is the only predictor among X1-X6 that should be dropped so that the +risk estimator can be improved from 33075 to 32952 (obviously, very little improvement here). + + + +We can try to rerun the OLS regression with X6 removed from the model. The adjusted $R^2$ now increases to 0.577. Thus to reduce multicollinearity, deleting X6 is perhaps a better option than X3. + +```python +est_ = sm.OLS(y, X_aug_).fit() +print(est_.summary()) +``` + +![](Blog/32.png) + + + +## Regularization + +However, stepwise regression has many potential drawbacks. For example, as a greedy search algorithm, it is not guaranteed to find the best model, especially when predictors are correlated. + +Another approach that performs predictor selection and also works in the presence of multicollinearity is regularization, a technique for reducing the complexity of the model by adding an extra penalty term to the errors. More specifically, our objective here is still to minimize $||\mathbf{y} - \text{X} \mathbf{\beta}||^2$ (same as the OLS model), but we now impose an additional constraint that the coefficients should not exceed some threshold. The specific constraint depends on the type of the regularization. Three popular regularization techniques are mentioned below--Lasso, Ridge, and Elastic Net. + +Regardless of the type of regularization, the new model will always penalize large coefficients, so we need to standardize our predictor variables to ensure that the penalty is applied fairly across all predictors. Standardization means converting the values in each column of the regressor matrix into z-scores, so the standard score of $x$ is calculated as $z = \frac{x - \mu}{\sigma}$, where $\mu$ is the mean of all values in the same column and $\sigma$ is their standard deviation. + +```python +X_std = StandardScaler().fit_transform(X_) +``` + +### Lasso + +After the standardization, we are ready to choose a specific model to perform. Let's start with Lasso regression. Lasso regression requires that the sum of the absolute values of the coefficients $||\beta||_1$ is no larger than some prespecified value. Using Lagrange Multiplier, we can write our objective function to minimize as $||\mathbf{y} - \text{X} \mathbf{\beta}||^2_2 + \lambda ||\beta||_1$. $\lambda ||\beta||_1$ is the extra penalty term here and can be controlled by changing the value of $\lambda$. When $\lambda$ is zero, the model reduces to OLS; when $\lambda$ tends to infinity, all coefficients approach zero. As you gradually increase $\lambda$, the magnitude of the coefficients has to decrease towards zero. This can be illustrated by the code below: + +```python +coefs = [] +alphas = np.logspace(-2, 1, 30) # the range of the parameter +for a in alphas: + l = linear_model.Lasso(alpha=a, fit_intercept=False) + l.fit(X_std, y) + coefs.append(l.coef_) +# plot log lambda on x-axis and the corresponding values of coefficients on y-axis +ax = plt.gca() +ax.plot(alphas, coefs) +ax.set_xscale('log') +plt.xlabel('Log lambda') +plt.ylabel('Coefficients') +plt.title('Coefficients as a function of lambda') +plt.axis('tight') +``` + +![](Blog/35+.png) + +To improve the predictive power of our model, we can again resort to cross-validation and choose the $\lambda$ that optimizes some evaluation metrics, such as the cross-validated sum of squared residuals. This can be done by the GridSearchCV() function in scikit-learn. It searches over some range of $\lambda$ to find the optimal evaluation score. Note that the scoring metric is negative mean squared error since the scoring API is designed to maximize rather than minimize the score. Of course, we can easily convert the score back to positive using the negative() function in numpy. + +```python +parameters_lasso = [{'alpha': alphas}] +cv_lasso = GridSearchCV(linear_model.Lasso(), parameters_lasso, scoring = 'neg_mean_squared_error') +cv_lasso.fit(X_std, y) +scores_lasso = cv_lasso.cv_results_['mean_test_score'] # the scores are negative +scores_lasso = np.negative(scores_lasso) # reverse the sign of the score +scores_std_lasso = cv_lasso.cv_results_['std_test_score'] # standard deviation +# print the optimal values +print('best score:', np.negative(cv_lasso.best_score_)) +print('lambda:', cv_lasso.best_params_.get('alpha')) +``` + + + +We can visualize where the optimal value of $\lambda$ is located in the interval: + +```python +# plot the cross validation score against log lambda +fig_lasso = plt.figure() +plt.semilogx(alphas, scores_lasso) +std_error_lasso = scores_std_lasso / np.sqrt(X_.shape[0]) +# shade the region within 1 std err +plt.semilogx(alphas, scores_lasso + std_error_lasso, 'b--') +plt.semilogx(alphas, scores_lasso - std_error_lasso, 'b--') +plt.fill_between(alphas, scores_lasso + std_error_lasso, scores_lasso - std_error_lasso, alpha = 0.25) +plt.ylabel('CV score +/- std error') +plt.xlabel('Log lambda') +plt.axhline(np.min(scores_lasso), linestyle='--', color='.5') +plt.xlim([alphas[0], alphas[-1]]) +plt.title('CV score as a function of lambda') +``` + +![](Blog/39.png) + +Putting the optimal value of $\lambda$ in the parameter of Lasso(), we can obtain the coefficients under the Lasso model: + +```python +# print the standardized coefficients +print('Lasso Results:') +print('(Standardized)') +lasso = linear_model.Lasso(alpha = cv_lasso.best_params_.get('alpha')) +lasso.fit(X_std, y) +lasso_df = pd.DataFrame(lasso.coef_, X_.columns, columns=['Coefficient']) +print('Intercept %.6f' %(lasso.intercept_)) +print(lasso_df) +``` + + + +Note that X6 is again reduced to zero. The significant changes in the magnitude of other +coefficients is due to the fact that we have standardized our predictors before the regression. To +recover the interpretability of our results, we may choose to back-convert the coefficients (by dividing +by the standard deviation) and the intercept (by subtracting all the coefficients times the means of +the corresponding predictors and divided by the corresponding standard deviations): + +```python +# back-convert for ease of interpretation +# beta_real = beta_std / std err +# intercept_real = intercept_std - sum of (mean * beta_std / stderr) +print('(Real)') +lasso_real = lasso_df.copy() +intercept_real = lasso.intercept_ +count_lasso = lasso_real.shape[0] +for i in range(lasso_real.shape[0]): + # decrement the total number of predictors if the coefficient is reduced to zero + if lasso_real.iloc[i, 0] == 0: + count_lasso -= 1 + lasso_real.iloc[i, 0] /= np.std(X_.iloc[:, i]) + intercept_real -= (lasso_real.iloc[i, 0] * np.mean(X_.iloc[:, i])) +print('Intercept %.6f' %(intercept_real)) +print(lasso_real) +``` + + + +We can also try Ridge and Elastic Net regression. Their procedures are exactly the same except that we need to call different functions for different types of regressions. + +### Ridge + +For Ridge regression, the only difference is that the objective now becomes $\min\limits_{\beta} ||Y - X\beta||^2_2 + \lambda ||\beta||^2_2$. (The constraint is now on the sum of squared coefficients.) While Lasso promotes predictor selection, Ridge will decrease the values of coefficients but is unable to force a coefficient to exactly 0. + +```python +print('Ridge Results:') +alphas_ridge = np.logspace(-2, 5, 30) # the range of the parameter +parameters_ridge = [{'alpha': alphas_ridge}] +cv_ridge = GridSearchCV(linear_model.Ridge(), parameters_ridge, scoring = 'neg_mean_squared_error') +cv_ridge.fit(X_std, y) +scores_ridge = cv_ridge.cv_results_['mean_test_score'] # the scores are negative +scores_ridge = np.negative(scores_ridge) # reverse the sign of the score +scores_std_ridge = cv_ridge.cv_results_['std_test_score'] # standard deviation + +# plot the cross validation score against log lambda +fig_ridge = plt.figure() +plt.semilogx(alphas_ridge, scores_ridge) +std_error_ridge = scores_std_ridge / np.sqrt(X_.shape[0]) +# shade the region within 1 std err +plt.semilogx(alphas_ridge, scores_ridge + std_error_ridge, 'b--') +plt.semilogx(alphas_ridge, scores_ridge - std_error_ridge, 'b--') +plt.fill_between(alphas_ridge, scores_ridge + std_error_ridge, scores_ridge - std_error_ridge, alpha = 0.25) +plt.ylabel('CV score +/- std error') +plt.xlabel('Log lambda') +plt.axhline(np.min(scores_ridge), linestyle='--', color='.5') +plt.xlim([alphas_ridge[0], alphas_ridge[-1]]) +plt.title('CV score as a function of lambda') +plt.show() +# print the optimal values +print('best score:', np.negative(cv_ridge.best_score_)) +print('lambda:', cv_ridge.best_params_.get('alpha')) + +# print the standardized coefficients +print('(Standardized)') +ridge = linear_model.Ridge(alpha = cv_ridge.best_params_.get('alpha')) +ridge.fit(X_std, y) +ridge_df = pd.DataFrame(ridge.coef_, X_.columns, columns=['Coefficient']) +print('Intercept %.6f' %(ridge.intercept_)) +print(ridge_df) + +# back-convert for ease of interpretation +# beta_real = beta_std / std err +# intercept_real = intercept_std - sum of (mean * beta_std / stderr) +print('\n(Real)') +ridge_real = ridge_df.copy() +ridge_intercept_real = ridge.intercept_ +count_ridge = ridge_real.shape[0] +for i in range(count_ridge): + ridge_real.iloc[i, 0] /= np.std(X_.iloc[:, i]) + ridge_intercept_real -= (ridge_real.iloc[i, 0] * np.mean(X_.iloc[:, i])) +print('Intercept %.6f' %(ridge_intercept_real)) +print(ridge_real) +``` + + + +### Elastic Net + +Lastly, Elastic Net regression is a combination of Lasso and Ridge--the objective is $\min\limits_{\beta} ||Y - X\beta||^2_2 + \lambda \left(\alpha ||\beta||_1 + \frac{1 - \alpha}{2} ||\beta||^2_2\right)$, and $\alpha = 0.5$ by default. + +```python +print('Elastic Net Results:') +alphas_elastic_net = np.logspace(-2, 1, 30) # the range of the parameter +parameters_elastic_net = [{'alpha': alphas_elastic_net}] +cv_elastic_net = GridSearchCV(linear_model.ElasticNet(), parameters_elastic_net, scoring = 'neg_mean_squared_error') +cv_elastic_net.fit(X_std, y) +scores_elastic_net = cv_elastic_net.cv_results_['mean_test_score'] # the scores are negative +scores_elastic_net = np.negative(scores_elastic_net) # reverse the sign of the score +scores_std_elastic_net = cv_elastic_net.cv_results_['std_test_score'] # standard deviation + +# plot the cross validation score against log lambda +fig_elastic_net = plt.figure() +plt.semilogx(alphas_elastic_net, scores_elastic_net) +std_error_elastic_net = scores_std_elastic_net / np.sqrt(X_.shape[0]) +# shade the region within 1 std err +plt.semilogx(alphas_elastic_net, scores_elastic_net + std_error_elastic_net, 'b--') +plt.semilogx(alphas_elastic_net, scores_elastic_net - std_error_elastic_net, 'b--') +plt.fill_between(alphas_elastic_net, scores_elastic_net + std_error_elastic_net, scores_elastic_net - std_error_elastic_net, alpha = 0.25) +plt.ylabel('CV score +/- std error') +plt.xlabel('Log lambda') +plt.axhline(np.min(scores_elastic_net), linestyle='--', color='.5') +plt.xlim([alphas_elastic_net[0], alphas_elastic_net[-1]]) +plt.title('CV score as a function of lambda') +plt.show() +# print the optimal values +print('best score:', np.negative(cv_elastic_net.best_score_)) +print('lambda:', cv_elastic_net.best_params_.get('alpha')) + +# print the standardized coefficients +print('(Standardized)') +elastic = linear_model.ElasticNet(alpha = cv_elastic_net.best_params_.get('alpha')) +elastic.fit(X_std, y) +elastic_df = pd.DataFrame(elastic.coef_, X_.columns, columns=['Coefficient']) +print('Intercept %.6f' %(elastic.intercept_)) +print(elastic_df) + +# back-convert for ease of interpretation +# beta_real = beta_std / std err +# intercept_real = intercept_std - sum of (mean * beta_std / stderr) +print('\n(Real)') +elastic_real = elastic_df.copy() +elastic_intercept_real = elastic.intercept_ +count_elastic = elastic_real.shape[0] +for i in range(count_elastic): + elastic_real.iloc[i, 0] /= np.std(X_.iloc[:, i]) + elastic_intercept_real -= (elastic_real.iloc[i, 0] * np.mean(X_.iloc[:, i])) +print('Intercept %.6f' %(elastic_intercept_real)) +print(elastic_real) +``` + + + +### Comparing Performances + +Again, we can calculate the adjusted $R^2$ of each of these regressions. You can see that their values are all around 0.576, which is better than the adjusted $R^2$ of our initial OLS model. + +```python +y_pred_lasso = lasso.predict(X_std) # predictions +r2_lasso = metrics.r2_score(y, y_pred_lasso) # R-squared +adj_r2_lasso = 1 - (1 - r2_lasso) * (X_.shape[0] - 1) / (X_.shape[0] - count_lasso - 1) # adjusted R-squared +print('Lasso adj. R2:', adj_r2_lasso) + +y_pred_ridge = ridge.predict(X_std) # predictions +r2_ridge = metrics.r2_score(y, y_pred_ridge) # R-squared +adj_r2_ridge = 1 - (1 - r2_ridge) * (X_.shape[0] - 1) / (X_.shape[0] - count_ridge - 1) # adjusted R-squared +print('Ridge adj. R2:', adj_r2_ridge) + +y_pred_elastic = elastic.predict(X_std) # predictions +r2_elastic = metrics.r2_score(y, y_pred_elastic) # R-squared +adj_r2_elastic = 1 - (1 - r2_elastic) * (X_.shape[0] - 1) / (X_.shape[0] - count_elastic - 1) # adjusted R-squared +print('Elastic Net adj. R2:', adj_r2_elastic) +``` + + + + + +That’s it for a complete procedure to perform linear regression analysis in Python! We have +checked the necessary conditions, calculated the coefficients under different models, and compared +their performance. If you want to find an even better regression model, perhaps you could go on +to try some nonlinear models in the future! diff --git a/LinearRegression/linear_regression_proposal.md b/LinearRegression/linear_regression_proposal.md new file mode 100644 index 0000000..84cfbf8 --- /dev/null +++ b/LinearRegression/linear_regression_proposal.md @@ -0,0 +1,28 @@ +### :pushpin: Step 1 + +**TITLE:** +Intro to Multiple Linear Regression Analysis in Python + +**TOPIC:** +Statistics + +**DESCRIPTION (5-7+ sentences):** +A step-by-step tutorial on multiple linear regression analysis; the process involves checking assumptions, calculating coefficients using different techniques, making predictions, and evaluating performances under different models. These models include OLS, Lasso/Ridge, and Elastic Net. + +### :pushpin: Step 2 + +:family: **TARGET AUDIENCE (3-5+ sentences):** +People who have a basic understanding of elementary statistics topics such as confidence intervals and hypothesis testing, and are interested in obtaining estimates from multiple predictors with basic regression techniques. + +### :pushpin: Step 3 + +> Outline your learning/teaching structure: + +**Beginning (2-3+ sentences):** +Begin by explaining the usage of multiple linear regression analysis and preparing an example dataset. + +**Middle (2-3+ sentences):** +Show how the analysis can be applied to the dataset using a simplest model. Apart from performing the regression itself, its conditions, interpretations, and applications are also explained. + +**End (2-3+ sentences):** +Introduce some more complicated techniques such as stepwise regression and regularizations, and compare their results and performances. \ No newline at end of file