ModuleNotFoundError: No module named 'tornet.data'

[cgeronasso]

I have tried to install the module and also uncomment the Sys line, but still ModuleNotFoundError: No module named 'tornet.data', could you help?

[tonyreisll]

I assume you are trying to run one of the notebooks? It seems like a PATH related problem, python can't seem to find tornet.data. This should be taken care of when you run from the tornet root directory:

 pip install .

A quick solution would be to manually add tornet to sys.path at the beginning of your script.

sys.path.append("path/to/tornet")

[AAM-code614]

I am a high school student interested in doing a deep learning project with the TorNet dataset using Tensorflow/Keras. I have been receiving the same error message when trying to import the dataset into Google Colab. I have tried using the DataLoaders notebook provided, and tried using some of my own code.

What I have currently is as follows:

`import sys

 Uncomment if tornet isn't installed in your environment or in your path already
sys.path.append('../')
import os
import glob
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

try:
    from tornet.data.loader import read_file, TornadoDataLoader
except ImportError as e:
    print("WARNING: cannot import tornet library. Many cells below won't work. Install tornet or run sys.path.append('../') and rerun this cell")
    raise e

Set path to tornet dataset
Assumes .tar.gz files are extracted.
this directory should contain catalog.csv, train/ , test/
data_root=os.environ['TORNET_ROOT']

Load catalog of all TORNET samples
catalog = pd.read_csv(os.path.join(data_root,'catalog.csv'),parse_dates=['start_time','end_time'])

from tornet.data.tf.loader import create_tf_dataset
tf_ds = create_tf_dataset(file_list,variables=variables,n_frames=n_frames)`

Since I am quite new to this, I think I am doing something wrong but I can't imagine what. Any and all guidance is greatly appreciated!

[Higgs32584]

Hey bro, take a look at my scripts they might help.

It won't translate directly but it should be able to help!

[Higgs32584]

You can also do this this could help:

pip install git+https://github.com/username/repo.git

And just replace the username and the repo with the specific implementation.

[AAM-code614]

Thank you for your help, I will check it out and let you know if it works.

[AAM-code614]

@Higgs32584, thank you for your help. I was able to get the dataset installed (I think) but got another error message.

Cell 1 - 
 import os
os.environ['KERAS_BACKEND']='tensorflow' # set to 'tensorflow', 'torch' or 'jax' (installs require
Cell 2 - 
import sys
!pip install git+https://github.com/mit-ll/tornet
#Uncomment if tornet isn't installed in your environment or in your path already
sys.path.append("path/to/tornet")

import os
import glob
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import keras

from tornet.data.tf.loader import create_tf_dataset
from tornet.data.constants import ALL_VARIABLES

outputted: Collecting git+https://github.com/mit-ll/tornet
  Cloning https://github.com/mit-ll/tornet to /tmp/pip-req-build-t9yzk3kc
  Running command git clone --filter=blob:none --quiet https://github.com/mit-ll/tornet /tmp/pip-req-build-t9yzk3kc
  Resolved https://github.com/mit-ll/tornet to commit 92d9d97bbd174be2e5cea4896c62208c5be7900d
  Preparing metadata (setup.py) ... done

Cell 3 - 
`#keras accepts most data loaders (tensorflow, torch).
#A pure keras data loader, with necessary preprocessing steps for the cnn baseline, is provided
from tornet.data.keras.loader import KerasDataLoader
data_root = os.environ['TORNET_ROOT']
ds = KerasDataLoader(data_root=data_root,
                     data_type='train',
                     years=[2018,2019],
                     batch_size = 8,
                     workers = 4,
                     use_multiprocessing = True)

outputted: KeyError                                  Traceback (most recent call last)
[/tmp/ipython-input-2697221153.py](https://localhost:8080/#) in <cell line: 0>()
      2 #A pure keras data loader, with necessary preprocessing steps for the cnn baseline, is provided
      3 from tornet.data.keras.loader import KerasDataLoader
----> 4 data_root = os.environ['TORNET_ROOT']
      5 ds = KerasDataLoader(data_root=data_root,
      6                      data_type='train',

/usr/lib/python3.11/os.py in __getitem__(self, key)

KeyError: 'TORNET_ROOT'`

Not sure why I'm getting this and would appreciate some input. Thanks in advance!

[Higgs32584]

oh, so you have to set:

os.environ["TORNET_ROOT"] = "WHERE YOU DOWNLOADED YOUR DATA TO"

# optional, if you are using a TFDS data format
os.environ["TFDS_DATA_DIR"] = "LIST TFDS DATA DIR HERE"
you dont have to unless you want to build a tfds data directory, I think it speeds it up, but it took me like 2 days for the system to make,

You are basically just telling the system where your data is