diff --git a/source/best-practice.rst b/source/best-practice.rst index d66a724..531c7fe 100644 --- a/source/best-practice.rst +++ b/source/best-practice.rst @@ -63,8 +63,8 @@ solutions: * Contain too many known “bad” words, e.g. “low interest rate loans” * Has “.exe” attachments -* An AI approach that learns from a corpus of previous emails to read an emails - content, and classifies it as phishing, or not phishing. +* An AI approach that learns from a corpus of previous emails to read an email's + content, and classify it as phishing, or not phishing. The respective advantages of each approach are: @@ -197,7 +197,7 @@ natural part of using the guide. Our previous effort is not wasted, but informs our current and future decisions. We can break AI approaches into 3 categories. We provide a more detailed -explanation of these terms in the [appendix]. However, briefly: +explanation of these terms in the [appendix - WHERE?]. However, briefly: * In supervised learning approaches we are learning by example. We have a set of input data points with corresponding known output data points (labels) for @@ -247,13 +247,13 @@ We will not cover again the work of general goal setting here, which we assume should be familiar to any competent engineer. However, over other engineering projects AI and Machine learning projects have several uncommon considerations. -The first is that AI and Machine Learning algorithms are often stochastic in +One such consideration is that AI and Machine Learning algorithms are often stochastic in nature, behaving in a non-deterministic and statistical way. All goals and KPIs must be aware of this. For example, for a face detection software it is appropriate to set goals such as “95% of this set of faces can be recognised” rather than “the software must recognise all these specific people”. -The second is that human interpretability is a difficult topic in AI and Machine +A second consideration is that human interpretability is a difficult topic in AI and Machine Learning, and it may be difficult to define why the software is doing the things or making the decisions that it makes. We will cover this topic (and its mitigations in more detail) in later sections, but it is important to be aware @@ -492,6 +492,8 @@ a high AI Risk category, and be subject to extra requirements. :ref:`Version Control, CI/CD for Data` +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +IS "CI/CD" REALLY THE APPROPRIATE TERMINOLOGY HERE? + Any electronic systems engineer should be familiar with version control. These ideas are just as important in developing AI applications as any other software product. In this step we explore: @@ -526,7 +528,7 @@ Examples :ref:`Documentation` +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -Another task that any electronic systems engineer should be familiar with is +Another task that any software engineer should be familiar with is documentation. As with version control, good documentation is just as important, or perhaps even more so, as any other software development project. In this step we will explore: @@ -824,7 +826,7 @@ wealth of resources elsewhere (imputation), but we note that the extent to which any of these strategies is plausible depends on how the missingness in our data has come about. There are three ways the data might be missing: -* Missing totally at random (MCAR) - the data is missing completely randomly +* Missing completely at random (MCAR) - the data is missing completely randomly with no pattern at all * Missing random (MAR) - the data is missing randomly, but in a way that is explained by the data you are using to predict things diff --git a/source/definitions.rst b/source/definitions.rst index 6c34d0e..c50c73b 100644 --- a/source/definitions.rst +++ b/source/definitions.rst @@ -5,6 +5,8 @@ Application Specific Requirements ================================== +IS "APPLICATION SPECIFIC" THE PHRASE? IF SO, PERHAPS CLARIFY WHY + .. _AI Risk: AI Risk diff --git a/source/using-the-guide.rst b/source/using-the-guide.rst index ee7a3a5..fa92241 100644 --- a/source/using-the-guide.rst +++ b/source/using-the-guide.rst @@ -2,31 +2,31 @@ .. _Using this Guide: -Using this guide: The 5 questions +Using this guide: The five questions ================================= -This guide will help you answer 5 questions step by step: +This guide will help you answer five questions step by step: -* Should I use AI/ML? -* How do I define my AI/ML project -* How do I collect data? -* How do I train my AI/ML application? -* How do I deploy my AI/ML application? +1. Should I use AI/ML? +2. How do I define my AI/ML project +3. How do I collect data? +4. How do I train my AI/ML application? +5. How do I deploy my AI/ML application? Answering these questions will take you from conceptualizing your AI application through to deploying it in the real world. -These 5 questions are broken into smaller questions, requirements, and tasks to +These five questions are broken into subsidiary questions, requirements, and tasks to be completed. We set out the question in the order they should be considered. We strongly recommend you don't skip ahead, although from time to time you may want to go back and review your answers to earlier questions. -These smaller questions, requirements and tasks for each of the 5 main questions +These smaller questions, requirements and tasks for each of the five main questions are recorded in a table, showing what the activity is, and where you should record the documentary evidence of that activity being completed. Some of the entries may be optional, depending on earlier answers. Once all the required -smaller questions, requirements, and tasks are addressed, you will have your -answer to the main question, and can move on to the next of these 5 questions. +subsidiary questions, requirements, and tasks are addressed, you will have your +answer to the main question, and can move on to the next of these five questions. +----------------------------+--------------------------------------+----------+ | Question/Task/Requirement | Evidence | Done | @@ -41,12 +41,12 @@ answer to the main question, and can move on to the next of these 5 questions. .. _Should I Use AI/ML?: -Should I use AI/ML? -------------------- +Question 1: Should I use AI/ML? +------------------------------- Developing an AI application can present significant challenges. Collection of -data, testing and validation are challenges. As seen in the previous section, -many applications of AI will come with special requirements that can be a +data, testing and validation are challenges. +Many applications of AI will come with special requirements that can be a challenge in themselves. To address this, the very first step in this guide is to be able answer the following: should I use AI to solve my problem? We break this problem down into two parts: @@ -55,10 +55,9 @@ this problem down into two parts: approaches? * Does the problem touch on any application or data areas that may effectively prohibit development? - - * What is the risk that the project falls into the unacceptable AI Risk - category? - * What is the risk that the project makes use of problematic data? + * What is the risk that the project falls into the :ref:`unacceptable AI Risk + category`? + * What is the risk that the project makes use of :ref:`problematic data`? +--------------------------------------------------------+----------+----------+ | Requirement | Evidence | Complete | @@ -72,15 +71,15 @@ this problem down into two parts: .. _How do I define my AI project?: -How do I define my AI project? ------------------------------- +Question 2: How do I define my AI project? +------------------------------------------ In the previous step we worked on an engineering case for AI in our project. In this step, we will take the first step in realizing our project by setting the goals and bounds of the project. There are three steps to complete in this section: -* Defining goals and metrics for success +* Defining goals and metrics for success (KPIs) * Defining limitations and boundaries on the project * Completing a risk assessment of the project as a whole @@ -89,23 +88,25 @@ section: +========================================================+==========+==========+ | :ref:`Establish Goals and KPIs` | - | | +--------------------------------------------------------+----------+----------+ +| LIMITATIONS AND BOUNDARIES ON THE PROJECT? | - | | ++--------------------------------------------------------+----------+----------+ | :ref:`Risk Assessment` | - | | +--------------------------------------------------------+----------+----------+ .. _How do I collect data for my AI/ML project?: -How do I collect data for my AI/ML project? -------------------------------------------- +Question 3: How do I collect data for my AI/ML project? +------------------------------------------------------- In the previous step, we defined the scope of our project. In this step, we move on to the first part of the practical engineering of our AI project: collecting the data. AI/ML applications are, at their core, data driven. At some level, data will need to be collected, or at the very least, processed. It’s very important to get this right, as the strength of the data will have a strong -effect on the efficacy of training and deploying our AI application. We set out +impact on the efficacy of training and deploying our AI application. We set out a number of steps for this section, but our primary challenges are: -* Making sure the data we’re is collecting useful, truthful, and effective data +* Making sure the data we’re is collecting useful, truthful, and effective * Making sure we transform our raw data into a form that can effectively utilized by AI/ML algorithms * Making sure our infrastructure for collection, storage, and access is @@ -134,8 +135,8 @@ a number of steps for this section, but our primary challenges are: .. _How do I train my AI/ML application?: -How do I train my AI/ML application? ------------------------------------- +Question 4: How do I train my AI/ML application? +------------------------------------------------ In the previous step, we collected the data for our AI project. In this step, we will make use of it by using it to train an AI algorithm of our choice to @@ -169,8 +170,8 @@ steps for this section, but our primary challenges are: .. _How do I deploy my AI application?: -How do I deploy my AI application? ----------------------------------- +Question 5: How do I deploy my AI application? +---------------------------------------------- After training our AI application, we can finally deploy it and (hopefully) achieve the goals set out in our previous steps. This step will likely represent @@ -178,7 +179,7 @@ a welcome return to familiarity for the professional engineer, as the process for deploying an AI application is fairly similar to that of deploying any other software application. Our process will proceed broadly in three steps: -* Preparing our trained the model for a live environment +* Preparing our trained model for a live environment * Engineering a process for deployment and model updating * Setting up continuous monitoring for our model