Repeated training on 5000 step chunks

[lee-erhu]

When I run the code molecule_generation train MoLeR TRACE_DIR to train with my own dataset, it keeps running and has now repeated the 5000 steps three times. May I ask if there is a problem with this?
ain: 0.0008 loss |
Avg weighted sum. of graph losses: 0.2413
Avg weighted sum. of prop losses: 0.0322
Avg node class. loss: 0.0094
Avg first node class. loss: 0.4492
Avg edge selection loss: 0.0361
Avg edge type loss: 0.0001
Avg attachment point selection loss: 0.0361
Avg KL divergence: 4.9250
Property results: sa_score: MAE 0.25, MSE 0.10 (norm MAE: 0.46) | clogp: MAE 0.50, MSE 0.39 (norm MAE: 0.38) | mol_weight: MAE 30.13, MSE 1433.69 (norm MAE: 0.76) | 31.31 graphs/s
WARNING:tensorflow:From C:\Users\86134\anaconda3\envs\molecule-generation-main\lib\site-packages\tensorflow\python\util\deprecation.py:561: calling function (from tensorflow.python.eager.def_function) with experimental_relax_shapes is deprecated and will be removed in a future version.
Instructions for updating:
experimental_relax_shapes is deprecated, use reduce_retracing instead
Valid: 0.1066 loss |
Avg weighted sum. of graph losses: 15.8818
Avg weighted sum. of prop losses: 0.0478
Avg node class. loss: 8.5787
Avg first node class. loss: 4.0964
Avg edge selection loss: 3.2877
Avg edge type loss: 2.7245
Avg attachment point selection loss: 0.8569
Avg KL divergence: 4.1490
Property results: sa_score: MAE 0.36, MSE 0.24 (norm MAE: 0.68) | clogp: MAE 0.94, MSE 1.47 (norm MAE: 0.72) | mol_weight: MAE 39.45, MSE 2397.00 (norm MAE: 1.00) | 30.46 graphs/s
Step: 100 | Epoch graph avg. loss = 0.00078 | Batch graph avg. loss = 0.00074 | Moving avg. loss = 0.00078 , avg #graphs = 290.08 , avg #trace steps = 1322.70 | Steps p Step: 101 | Epoch graph avg. loss = 0.00078 | Batch graph avg. loss = 0.00074 | Moving avg. loss = 0.00078 , avg #graphs = 290.19 , avg #trace steps = 1322.56 | Steps p
0078 | Batch graph avg. loss = 0.00082 | Moving avg. loss = 0.00078 , avg #graphs = 290.46 , avg

[kmaziarz]

Sorry, I missed your question. The 5000 step chunks are "mini-epochs": we don't wait until the entire dataset is exhausted as that can take quite a while in our case, and just report validation metrics every 5000 steps. If these metrics are improving over time then likely everything is working correctly.