[alpha2] move from twisted to asyncio + new network

.travis.yml

@@ -1,6 +1,10 @@
 language: python
-python:
-  - "3.6"
+
+matrix:
+  include:
+    - python: 3.7
+      dist: xenial
+      sudo: true
 
 sudo: required
 services:

CHANGELOG.rst

@@ -20,6 +20,17 @@ All notable changes to this project are documented in this file.
 - Improve Connection Failure Handling in NodeLeader `#915 <https://github.com/CityOfZion/neo-python/issues/915>`_
 - Improve transaction coverage and fix `PublishTransaction.Size()` `#929 <https://github.com/CityOfZion/neo-python/issues/929>`_
 - Align ``Fixed8`` ``ToString()`` output with C# `#941 <https://github.com/CityOfZion/neo-python/pull/941>`_
+- Move from using ``Twisted`` to ``asyncio`` as event driven framework `#934 <https://github.com/CityOfZion/neo-python/pull/934>`_
+- Add new networking code
+- Add IP Filtering
+- Tighten smart contract storage context validation
+- Update VM to 2.4.3 (Also updates ``ApplicationEngine``, ``StateReader``  and ``StateMachine``)
+- Add support for new Ping/Pong network payload
+- Add neo-vm JSON test support + add new Debugger wrapping class
+- Various VM performance updates
+- Various code cleaning updates
+- Ensure LevelDB iterators are close, ensure all ``MemoryStream`` usages go through ``StreamManger`` and are closed.
+
 
 
 [0.8.4] 2019-02-14

README.rst

@@ -27,7 +27,7 @@ Overview
 What does it currently do
 ~~~~~~~~~~~~~~~~~~~~~~~~~
 
--  This project aims to be a full port of the original C# `NEO
+-  This project aims to be an alternative implementation for the original C# `NEO
    project <https://github.com/neo-project>`_
 -  Run a Python based P2P node
 -  Interactive CLI for configuring node and inspecting blockchain
@@ -64,7 +64,7 @@ Get help or give help
 -  Open a new
    `issue <https://github.com/CityOfZion/neo-python/issues/new>`_ if
    you encounter a problem.
--  Or ping **@localhuman**, **@metachris** or **@ixje** on the `NEO
+-  Or ping **@ixje** on the `NEO
    Discord <https://discord.gg/R8v48YA>`_.
 -  Pull requests welcome. Have a look at the issue list for ideas.
    You can help with wallet functionality, writing tests or documentation,
@@ -77,8 +77,7 @@ neo-python has two System dependencies (everything else is covered with
 ``pip``):
 
 -  `LevelDB <https://github.com/google/leveldb>`_
--  `Python
-   3.6 <https://www.python.org/downloads/release/python-366/>`_ or `Python 3.7 <https://www.python.org/downloads/release/python-370/>`_ (3.5 and below is not supported)
+-  `Python 3.7 <https://www.python.org/downloads/release/python-373/>`_ (3.6 and below is not supported)
 
 We have published a Youtube
 `video <https://www.youtube.com/watch?v=ZZXz261AXrM>`_ to help get you
@@ -110,63 +109,19 @@ OSX
 
     brew install leveldb
 
-Ubuntu/Debian 16.10+
+Ubuntu/Debian 18.04+
 ^^^^^^^^^^^^^^^^^^^^
 
-Ubuntu starting at 16.10 supports Python 3.6+ in the official repositories.
-
-First, ensure Ubuntu is fully up-to-date with this:
-
-::
-
-   sudo apt-get update && sudo apt-get upgrade
-
-You can install Python 3.7 and all the system dependencies like this:
-
-::
-
-   sudo apt-get install python3.7 python3.7-dev python3.7-venv python3-pip libleveldb-dev libssl-dev g++
-
-
-Or, you can install Python 3.6 and all the system dependencies like this:
-
-::
-
-    sudo apt-get install python3.6 python3.6-dev python3.6-venv python3-pip libleveldb-dev libssl-dev g++
-
-Older Ubuntu versions (eg. 16.04)
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-For older Ubuntu versions you'll need to use an external repository like
-Felix Krull's deadsnakes PPA at
-https://launchpad.net/~deadsnakes/+archive/ubuntu/ppa (read more
-`here <https://askubuntu.com/questions/865554/how-do-i-install-python-3-6-using-apt-get>`__).
-
-(The use of the third-party software links in this documentation is done
-at your own discretion and risk and with agreement that you will be
-solely responsible for any damage to your computer system or loss of
-data that results from such activities.)
-
-::
-
-    apt-get install software-properties-common python-software-properties
-    add-apt-repository ppa:deadsnakes/ppa
-    apt-get update
-    apt-get install python3.6 python3.6-dev python3.6-venv python3-pip libleveldb-dev libssl-dev g++
+At the time of writing this message Ubuntu's package manager only lists Python `3.7.0`. Variously memory leaks have been addressed
+since and it is recommended to run Python `3.7.3` or newer. Read `this Ubuntu guide <https://linuxize.com/post/how-to-install-python-3-7-on-ubuntu-18-04/>`_ on how to add an alternative PPA to your sources list to install from
+or how to compile the latest version manually.
 
 Centos/Redhat/Fedora
 ^^^^^^^^^^^^^^^^^^^^
 
-::
-
-    # Install Python 3.6:
-    yum install -y centos-release-scl
-    yum install -y rh-python36
-    scl enable rh-python36 bash
+Note: - Not tested -
 
-    # Install dependencies:
-    yum install -y epel-release
-    yum install -y readline-devel leveldb-devel libffi-devel gcc-c++ redhat-rpm-config gcc python-devel openssl-devel
+Please correct the REAMDE if there are issues. Install Python `3.7.3` following `this CentOS guide <https://tecadmin.net/install-python-3-7-on-centos/>`_.
 
 Windows
 ^^^^^^^
@@ -182,14 +137,13 @@ Help needed for running natively. Installing the Python package plyvel seems to
 compiler support tied to Visual Studio and libraries. Refer to
 `documentation <https://neo-python.readthedocs.io/en/latest/installwindows.html>`__.
 
-Python 3.6+
+Python 3.7+
 ~~~~~~~~~~~
 
-neo-python is compatible with **Python 3.6 and later**.
+neo-python is compatible with **Python 3.7 and later**.
 
-On \*nix systems, install Python 3.6 or Python 3.7 via your package manager, or
-download an installation package from the `official
-homepage <https://www.python.org/downloads/>`__.
+On \*nix systems, install Python Python 3.7 via your package manager, or
+download an installation package from the `official homepage <https://www.python.org/downloads/>`__.
 
 
 Install
@@ -214,10 +168,6 @@ could lead to version conflicts.
     python3.7 -m venv venv
     source venv/bin/activate
 
-    # create virtual environment using Python 3.6 and activate
-    python3.6 -m venv venv
-    source venv/bin/activate
-
     # install the package in an editable form
     (venv) pip install wheel -e .
 
@@ -233,10 +183,6 @@ could lead to version conflicts.
     python3.7 -m venv venv
     source venv/bin/activate
 
-    # create virtual environment using Python 3.6 and activate
-    python3.6 -m venv venv
-    source venv/bin/activate
-
     (venv) pip install wheel neo-python
 
 
@@ -473,7 +419,7 @@ Troubleshooting
 If you run into problems, check these things before ripping out your
 hair:
 
--  Double-check that you are using Python 3.6.x or Python 3.7.x
+-  Double-check that you are using Python 3.7.x
 -  Update the project dependencies (``pip install -e .``)
 -  If you encounter any problems, please take a look at the
    `installation

docs/source/install.rst

@@ -139,6 +139,24 @@ The solution probably is
 
     brew reinstall openssl
 
+-----
+
+If you encounter an issue installing the ``scrypt`` module (possibly after updating OSX) with an error like this:
+
+.. code-block:: sh
+
+    ld: library not found for -lcrypto
+    clang: error: linker command failed with exit code 1 (use -v to see invocation)
+    error: command 'gcc' failed with exit status 1
+
+The solution probably is
+
+.. code-block:: sh
+
+    $ brew install openssl
+    $ export CFLAGS="-I$(brew --prefix openssl)/include $CFLAGS"
+    $ export LDFLAGS="-L$(brew --prefix openssl)/lib $LDFLAGS"
+
 
 Install from PyPi
 =================

examples/node.py

@@ -1,18 +1,16 @@
 """
-Minimal NEO node with custom code in a background thread.
+Minimal NEO node with custom code in a background task.
 
 It will log events from all smart contracts on the blockchain
 as they are seen in the received blocks.
 """
-import threading
-from time import sleep
+import asyncio
 
 from logzero import logger
-from twisted.internet import reactor, task
 
-from neo.Network.NodeLeader import NodeLeader
 from neo.Core.Blockchain import Blockchain
 from neo.Implementations.Blockchains.LevelDB.LevelDBBlockchain import LevelDBBlockchain
+from neo.Network.p2pservice import NetworkService
 from neo.Settings import settings
 
 
@@ -21,16 +19,11 @@
 # settings.set_logfile("/tmp/logfile.log", max_bytes=1e7, backup_count=3)
 
 
-def custom_background_code():
-    """ Custom code run in a background thread.
-
-    This function is run in a daemonized thread, which means it can be instantly killed at any
-    moment, whenever the main thread quits. If you need more safety, don't use a  daemonized
-    thread and handle exiting this thread in another way (eg. with signals and events).
-    """
+async def custom_background_code():
+    """ Custom code run in the background."""
     while True:
         logger.info("Block %s / %s", str(Blockchain.Default().Height), str(Blockchain.Default().HeaderHeight))
-        sleep(15)
+        await asyncio.sleep(15)
 
 
 def main():
@@ -40,18 +33,17 @@ def main():
     # Setup the blockchain
     blockchain = LevelDBBlockchain(settings.chain_leveldb_path)
     Blockchain.RegisterBlockchain(blockchain)
-    dbloop = task.LoopingCall(Blockchain.Default().PersistBlocks)
-    dbloop.start(.1)
-    NodeLeader.Instance().Start()
-
-    # Start a thread with custom code
-    d = threading.Thread(target=custom_background_code)
-    d.setDaemon(True)  # daemonizing the thread will kill it when the main thread is quit
-    d.start()
-
-    # Run all the things (blocking call)
-    reactor.run()
-    logger.info("Shutting down.")
+
+    loop = asyncio.get_event_loop()
+    # Start a reoccurring task with custom code
+    loop.create_task(custom_background_code())
+    p2p = NetworkService()
+    loop.create_task(p2p.start())
+
+    # block from here on
+    loop.run_forever()
+
+    # have a look at the other examples for handling graceful shutdown.
 
 
 if __name__ == "__main__":